Famous Deep Learning Research Papers

[ashutosh1919]

Opening this thread so that people can post the arxiv links of some of the famous papers of deep learning. This thread may help beginner in deep learning to cope up.

You can send the link and also mention the high level research topic for which the paper has been famous. (See the below example).

Topic: Generative Adversarial Networks (GANs)
Paper: GAN paper

[8bitmp3]

Nice idea @ashutosh1919

Topic: Core Deep Learning

• Deep Learning (LeCun, Bengio & Hinton, 2015) (Nature)
• A Fast Learning Algorithm for Deep Belief Nets (Hinton, Osindero & Teh, 2006)
• Reducing the dimensionality of data with neural networks (Hinton & Salakhutdinov, 2006) (Science)
• Learning representations by back-propagating errors (Rumelhart, Hinton & Williams, 1986) (Nature)

[Jean]

Topic: Convolutional Neural Networks (CNNs)

• Fast Algorithms for Convolutional Neural Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Searching for MobileNetV3
• Fully Convolutional Networks for Semantic Segmentation
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Context R-CNN: Long Term Temporal Context for Per-Camera Object Detection
• Mask R-CNN

[8bitmp3]

Pre-“deep learning”:

• [Artificial neuron] A logical calculus of the ideas immanent in nervous activity (McCulloch & Pitts, 1943)
• Perceptron: A Perceiving and Recognizing Automaton (Rosenblatt, 1957)
• An adaptive “ADALINE” neuron using chemical “memistors” (Widrow, 1960)
• Perceptrons (Minsky & Papert, 1969)
• Neural networks and physical systems with emergent collective computational abilities (Hopfield, 1982)
• A learning algorithm for Boltzmann machines (Ackley, Hinton & Sejnowski, 1985)

Professor’s perceptron paved the way for AI – 60 years too soon | Cornell...

[Sid]

Topic : NLP (Summarization centred).

While I was researching about summarization using Machine Learning these are papers that I found interesting maybe it might help others.

Research papers

• Original BERT Paper : [1810.04805] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (arxiv.org)

• [1907.11692] RoBERTa: A Robustly Optimized BERT Pretraining Approach (arxiv.org)

• [1909.11942v1] ALBERT: A Lite BERT for Self-supervised Learning of Language Representations (arxiv.org)

• [1706.03762v5] Attention Is All You Need (arxiv.org)

• [1909.03186] On Extractive and Abstractive Neural Document Summarization with Transformer Language Models (arxiv.org)

• Root Cause Analysis of Network Failures Using Machine Learning and Summarization Techniques | IEEE Journals & Magazine | IEEE Xplore

Educative Blogs ( Credit : Jay Alammar ) which I used while learning about all these things. They are very helpful for people starting to understand these things.

there are many other blogs i found were also interesting but that doesn’t include in this topic

• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time. (jalammar.github.io)

• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time. (jalammar.github.io)

I am not an expert by any means but I do hope someone finds this helpful as I did.

[Jean]

More ConvNets Papers:

• LeNet-5: Gradient Based Learning Applied to Document Recognition (Yann LeCun, 1998)
• AlexNet: ImageNet Classification with Deep Convolutional Neural Networks(Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton)
• VGGNet(Visual Geometry Group): Very Deep CNN for Large Scale Image Recognition(Karen Simonyan, Andrew Zisserman, 2015)
• GoogLeNet: Going Deeper with Convolutions
• ResNet: Deep Residual Learning for Image Recognition
• Xception: Deep Learning with Depthwise Separable Convolutions(Francois Chollet, 2016)

[Jean]

Convnets and RNNs for Texts Classification

• Text Classification from Scratch(Yann LeCun, Xiang Zhang, 2016)

• Recurrent Convolutional Neural Networks for Text Classification(Siwei Lai, Liheng Xu, Kang Liu, Jun Zhao, 2015)