Reference

Artificial Intelligence

From Turing's first ideas to today's large language models and AGI/ASI.

Media Research

Media

Podcasts and Talks

Podcasts

Research

Foundational Papers

1940s-1950s - Foundations and Early Concepts

"A Logical Calculus of Ideas Immanent in Nervous Activity" - Neural network foundations - McCulloch & Pitts (1943)
"Computing Machinery and Intelligence" - Turing Test - Alan Turing (1950)
"The Organization of Behavior" - Hebbian learning - Donald Hebb (1949)
"A Chess-Playing Program for the IBM 704" - Early game AI - Bernstein & Roberts (1958)
Dartmouth Conference - Term "Artificial Intelligence" coined - McCarthy, Minsky, Rochester, Shannon (1956)
"A Proposal for the Dartmouth Summer Research Project" - AI as a field - McCarthy et al. (1955)
Logic Theorist - First AI program, proved mathematical theorems - Newell, Shaw & Simon (1956)
General Problem Solver (GPS) - Means-ends analysis - Newell & Simon (1957)
"Perceptrons: An Introduction to Computational Geometry" - Early neural networks - Rosenblatt (1958)
LISP Programming Language - AI programming language - John McCarthy (1958)

1960s - Early AI Systems and Search

ELIZA - First chatbot, pattern matching - Joseph Weizenbaum, MIT (1966)
"Steps Toward Artificial Intelligence" - AI research program - Marvin Minsky (1961)
"Programs with Common Sense" - Knowledge representation - John McCarthy (1959)
DENDRAL - Expert system for chemistry - Feigenbaum, Buchanan & Lederberg, Stanford (1965)
SHRDLU - Natural language understanding in blocks world - Terry Winograd, MIT (1970)
Microworlds - Simplified AI environments - Minsky & Papert (1969)
"Perceptrons" (Book) - Limitations of single-layer perceptrons - Minsky & Papert (1969)
"A Basis for a Mathematical Theory of Computation" - Theoretical foundations - McCarthy (1963)
"Some Philosophical Problems from AI Standpoint" - Philosophy of AI - McCarthy (1969)
Resolution Theorem Proving - Automated reasoning - J.A. Robinson (1965)

1970s - Expert Systems and Knowledge Representation

MYCIN - Medical diagnosis expert system - Shortliffe, Stanford (1975)
PROSPECTOR - Mineral exploration expert system - Duda, Hart & Nilsson, SRI (1978)
"A Framework for Representing Knowledge" - Frames - Marvin Minsky (1974)
"The Use of the LISP Programming Language" - AI programming - McCarthy (1978)
"Production Systems as a Programming Language" - Rule-based systems - Newell (1973)
Prolog Programming Language - Logic programming - Colmerauer & Roussel (1972)
"Conceptual Dependency Theory" - Natural language understanding - Roger Schank (1972)
"Scripts, Plans, Goals and Understanding" - Story understanding - Schank & Abelson (1977)
Backpropagation Discovery - Neural network training (unpublished) - Paul Werbos PhD thesis (1974)
"The Lighthill Report" - AI criticism, led to AI winter - James Lighthill, UK (1973)
"What Computers Can't Do" - AI limitations - Hubert Dreyfus (1972)
"Computation: Finite and Infinite Machines" - Automata theory - Marvin Minsky (1967)
STRIPS - Automated planning - Fikes & Nilsson, SRI (1971)
Blocks World Problem - Classic AI planning domain - Various (1970s)
"An Overview of Production Systems" - Knowledge representation - Davis & King (1977)

1980s - Neural Networks Revival and ML Foundations

"Learning Representations by Back-Propagating Errors" - Backpropagation popularized - Rumelhart, Hinton & Williams (1986)
"Parallel Distributed Processing" - Connectionism revival - Rumelhart, McClelland & PDP Group (1986)
Hopfield Networks - Recurrent neural networks - John Hopfield (1982)
"Neural Networks and Physical Systems" - Associative memory - Hopfield (1982)
Boltzmann Machines - Stochastic neural networks - Hinton & Sejnowski (1983)
"Learning Internal Representations" - Feature learning - Hinton & Sejnowski (1986)
Neocognitron - Early convolutional neural network - Kunihiko Fukushima (1980)
"The Society of Mind" - Modular intelligence - Marvin Minsky (1986)
ID3 Decision Tree Algorithm - Machine learning - Ross Quinlan (1986)
C4.5 Algorithm - Improved decision trees - Ross Quinlan (1993)
"Induction of Decision Trees" - ML methodology - Quinlan (1986)
"Learning Quickly When Irrelevant Attributes Abound" - PAC learning - Valiant (1984)
"A Theory of the Learnable" - Computational learning theory - Leslie Valiant (1984)
Explanation-Based Learning - Knowledge-intensive learning - Mitchell, Keller & Kedar-Cabelli (1986)
R1/XCON Expert System - Configuration system at DEC - McDermott (1980)
CYC Project - Common sense knowledge base - Doug Lenat (1984)
"The Fifth Generation Computer Systems" - Japanese AI initiative - MITI (1982)
Lisp Machines - Dedicated AI hardware - MIT & Symbolics (1980s)
Self-Organizing Maps (SOM) - Unsupervised learning - Teuvo Kohonen (1982)
"Connectionist Learning Procedures" - Neural network training - Hinton (1989)

1990s - Statistical ML and Practical Applications

LeNet - Convolutional neural network for digits - LeCun et al., AT&T (1989)
"Handwritten Digit Recognition with a Back-Propagation Network" - Practical CNNs - LeCun et al. (1989)
"Backpropagation Applied to Handwritten Zip Code Recognition" - MNIST - LeCun et al. (1989)
Support Vector Machines - Margin-based learning - Cortes & Vapnik (1995)
"A Training Algorithm for Optimal Margin Classifiers" - SVM theory - Boser, Guyon & Vapnik (1992)
"The Nature of Statistical Learning Theory" - SVM foundations - Vladimir Vapnik (1995)
LSTM Networks - Long Short-Term Memory - Hochreiter & Schmidhuber (1997)
"Long Short-Term Memory" - Recurrent neural networks - Hochreiter & Schmidhuber (1997)
"Gradient Flow in Recurrent Nets" - Vanishing gradients - Hochreiter et al. (2001)
Random Forests - Ensemble learning - Leo Breiman (2001)
AdaBoost - Boosting algorithm - Freund & Schapire (1995)
"A Decision-Theoretic Generalization of On-Line Learning" - AdaBoost theory - Freund & Schapire (1997)
Gradient Boosting - Sequential ensemble method - Jerome Friedman (1999)
TD-Gammon - Reinforcement learning for backgammon - Gerald Tesauro, IBM (1992)
"Temporal Difference Learning and TD-Gammon" - RL success - Tesauro (1995)
Q-Learning - Model-free RL - Watkins & Dayan (1992)
"Learning from Delayed Rewards" - Q-learning thesis - Chris Watkins PhD (1989)
"Reinforcement Learning: An Introduction" - RL textbook - Sutton & Barto (1998)
"Simple Statistical Gradient-Following Algorithms" - Policy gradient - Williams (1992)
PageRank Algorithm - Graph-based ranking (applied to web) - Page & Brin (1998)
Bayes Networks - Probabilistic graphical models - Pearl (1988)
"Probabilistic Reasoning in Intelligent Systems" - Bayes nets - Judea Pearl (1988)
Expectation-Maximization (EM) Algorithm - Statistical inference - Dempster, Laird & Rubin (1977)
Hidden Markov Models - Sequence modeling - Rabiner (1989)
"A Tutorial on Hidden Markov Models" - HMM introduction - Rabiner (1989)
Latent Dirichlet Allocation (LDA) - Topic modeling - Blei, Ng & Jordan (2003)
"Indexing by Latent Semantic Analysis" - LSA - Deerwester et al. (1990)
"Machine Learning" - Textbook - Tom Mitchell (1997)
IBM Deep Blue - Chess computer defeats Kasparov - IBM (1997)
"Deep Blue" - Computer chess achievement - Campbell, Hoane & Hsu, IBM (2002)

2000-2005 - Kernel Methods and Graphical Models

"A Few Useful Things to Know About Machine Learning" - ML best practices - Pedro Domingos (2012)
Kernel Methods - Non-linear feature mapping - Schölkopf & Smola (2002)
"Learning with Kernels" - Kernel theory - Schölkopf & Smola (2002)
Manifold Learning - Non-linear dimensionality reduction - Various (2000s)
"A Global Geometric Framework for Nonlinear Dimensionality Reduction" - Isomap - Tenenbaum, Silva & Langford (2000)
"Nonlinear Dimensionality Reduction by Locally Linear Embedding" - LLE - Roweis & Saul (2000)
"Laplacian Eigenmaps for Dimensionality Reduction" - Spectral methods - Belkin & Niyogi (2003)
"Stochastic Neighbor Embedding" - Visualization - Hinton & Roweis (2002)
Conditional Random Fields (CRF) - Sequence labeling - Lafferty, McCallum & Pereira (2001)
"Conditional Random Fields: Probabilistic Models" - CRF theory - Lafferty et al. (2001)
"An Introduction to Conditional Random Fields" - CRF tutorial - Sutton & McCallum (2010)
Semi-Supervised Learning - Learning with limited labels - Zhu (2005)
"Semi-Supervised Learning Using Gaussian Fields" - Graph-based SSL - Zhu, Ghahramani & Lafferty (2003)
Active Learning - Selective labeling - Settles (2009)
Transfer Learning - Knowledge transfer across tasks - Pan & Yang (2010)
Multi-Task Learning - Joint learning - Caruana (1997)
"Learning Many Related Tasks at the Same Time" - MTL - Caruana (1993)
Collaborative Filtering - Recommender systems - Various (2000s)
"Amazon.com Recommendations: Item-to-Item Collaborative Filtering" - Recommendation - Linden, Smith & York (2003)
Netflix Prize - Recommendation system competition - Netflix (2006-2009)

2006-2012 - Deep Learning Renaissance

"A Fast Learning Algorithm for Deep Belief Nets" - Pre-training breakthrough - Hinton, Osindero & Teh (2006)
"Reducing the Dimensionality of Data with Neural Networks" - Autoencoders - Hinton & Salakhutdinov (2006)
Deep Belief Networks - Greedy layer-wise training - Hinton et al. (2006)
Restricted Boltzmann Machines (RBM) - Energy-based models - Hinton (2002)
"Greedy Layer-Wise Training of Deep Networks" - Pre-training - Bengio et al. (2007)
"Why Does Unsupervised Pre-training Help?" - Theory - Erhan et al. (2010)
"Understanding the Difficulty of Training Deep Feedforward Neural Networks" - Initialization - Glorot & Bengio (2010)
ReLU Activation - Modern activation function - Nair & Hinton (2010)
Dropout - Regularization technique - Hinton et al. (2012)
"Improving Neural Networks by Preventing Co-Adaptation" - Dropout - Hinton et al. (2012)
"ImageNet Classification with Deep Convolutional Neural Networks" - AlexNet - Krizhevsky, Sutskever & Hinton (2012)
AlexNet Victory - ImageNet 2012 winner, 15.3% error - Krizhevsky et al. (2012)
ImageNet Database - Large-scale visual recognition - Deng et al. (2009)
"ImageNet: A Large-Scale Hierarchical Image Database" - Dataset paper - Deng et al. (2009)
Word2Vec - Word embeddings - Mikolov et al., Google (2013)
"Efficient Estimation of Word Representations in Vector Space" - Word2Vec - Mikolov et al. (2013)
"Distributed Representations of Words and Phrases" - Skip-gram model - Mikolov et al. (2013)
GloVe - Global vectors for word representation - Pennington, Socher & Manning, Stanford (2014)
"GloVe: Global Vectors for Word Representation" - Alternative embeddings - Pennington et al. (2014)
"A Neural Probabilistic Language Model" - Neural language modeling - Bengio et al. (2003)
"Recurrent Neural Network Based Language Model" - RNN LM - Mikolov et al. (2010)
"Extensions of Recurrent Neural Network Language Model" - Improved RNN LM - Mikolov et al. (2011)
"Context-Dependent Pre-Trained Deep Neural Networks" - Speech recognition - Dahl et al., Microsoft (2012)
"Deep Neural Networks for Acoustic Modeling in Speech Recognition" - DNN speech - Hinton et al. (2012)
"Acoustic Modeling Using Deep Belief Networks" - DBN for speech - Mohamed, Dahl & Hinton (2009)

2013-2016 - Computer Vision and NLP Breakthroughs

"Visualizing and Understanding Convolutional Networks" - CNN interpretation - Zeiler & Fergus (2013)
ZFNet - ImageNet 2013 winner - Zeiler & Fergus (2013)
"Network In Network" - 1x1 convolutions - Lin, Chen & Yan (2013)
VGGNet - Very deep networks - Simonyan & Zisserman, Oxford (2014)
"Very Deep Convolutional Networks for Large-Scale Image Recognition" - VGG - Simonyan & Zisserman (2014)
GoogLeNet/Inception - Inception modules - Szegedy et al., Google (2014)
"Going Deeper with Convolutions" - Inception architecture - Szegedy et al. (2014)
ResNet - Residual learning, 152 layers - He et al., Microsoft (2015)
"Deep Residual Learning for Image Recognition" - ResNet - He, Zhang, Ren & Sun (2015)
"Identity Mappings in Deep Residual Networks" - ResNet improvements - He et al. (2016)
Batch Normalization - Training acceleration - Ioffe & Szegedy, Google (2015)
"Batch Normalization: Accelerating Deep Network Training" - BatchNorm - Ioffe & Szegedy (2015)
"Layer Normalization" - Alternative normalization - Ba, Kiros & Hinton (2016)
"Group Normalization" - Small batch normalization - Wu & He, Facebook (2018)
"Instance Normalization: The Missing Ingredient for Fast Stylization" - Style transfer - Ulyanov et al. (2016)
R-CNN - Region-based object detection - Girshick et al., Berkeley (2014)
"Rich Feature Hierarchies for Accurate Object Detection" - R-CNN - Girshick et al. (2014)
Fast R-CNN - Faster object detection - Girshick, Microsoft (2015)
Faster R-CNN - Region proposal networks - Ren et al., Microsoft (2015)
"Faster R-CNN: Towards Real-Time Object Detection" - Unified detection network - Ren, He, Girshick & Sun (2015)
YOLO - "You Only Look Once", real-time detection - Redmon et al. (2016)
"You Only Look Once: Unified, Real-Time Object Detection" - YOLO - Redmon et al. (2016)
SSD - Single Shot MultiBox Detector - Liu et al. (2016)
Semantic Segmentation FCN - Fully convolutional networks - Long, Shelhamer & Darrell, Berkeley (2015)
"Fully Convolutional Networks for Semantic Segmentation" - FCN - Long et al. (2015)
U-Net - Medical image segmentation - Ronneberger, Fischer & Brox (2015)
DeepLab - Atrous convolution for segmentation - Chen et al., Google (2014)
Mask R-CNN - Instance segmentation - He et al., Facebook (2017)
"Mask R-CNN" - Object detection and segmentation - He, Gkioxari, Dollár & Girshick (2017)
Sequence-to-Sequence Learning - Encoder-decoder architecture - Sutskever, Vinyals & Le, Google (2014)
"Sequence to Sequence Learning with Neural Networks" - Seq2seq - Sutskever et al. (2014)
Neural Machine Translation - Attention based translation - Bahdanau, Cho & Bengio (2014)
Attention Mechanism - Alignment in NMT - Bahdanau et al. (2014)
"Neural Machine Translation by Jointly Learning to Align and Translate" - Attention - Bahdanau et al. (2014)
"Show, Attend and Tell" - Image captioning with attention - Xu et al. (2015)
"A Neural Image Caption Generator" - Image captioning - Vinyals et al., Google (2015)
"Deep Visual-Semantic Alignments" - Image-text alignment - Karpathy & Fei-Fei, Stanford (2015)
GRU (Gated Recurrent Unit) - Simplified LSTM - Cho et al. (2014)
"Learning Phrase Representations Using RNN Encoder-Decoder" - GRU - Cho et al. (2014)
"Empirical Evaluation of Gated Recurrent Neural Networks" - GRU vs LSTM - Chung et al. (2014)

2014-2017 - Generative Models and RL

Generative Adversarial Networks (GANs) - Adversarial training - Goodfellow et al. (2014)
"Generative Adversarial Networks" - GAN introduction - Goodfellow et al. (2014)
DCGAN - Deep convolutional GANs - Radford, Metz & Chintala (2015)
"Unsupervised Representation Learning with DCGANs" - Stable GAN training - Radford et al. (2015)
Conditional GAN - Class-conditional generation - Mirza & Osindero (2014)
Pix2Pix - Image-to-image translation - Isola et al., Berkeley (2016)
"Image-to-Image Translation with Conditional Adversarial Networks" - Pix2Pix - Isola et al. (2016)
CycleGAN - Unpaired image translation - Zhu et al., Berkeley (2017)
"Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks" - CycleGAN - Zhu et al. (2017)
StyleGAN - High-quality image generation - Karras et al., NVIDIA (2018)
"A Style-Based Generator Architecture for GANs" - StyleGAN - Karras, Laine & Aila (2018)
Progressive GAN - Progressive training - Karras et al., NVIDIA (2017)
"Progressive Growing of GANs" - High-resolution generation - Karras et al. (2017)
Variational Autoencoders (VAE) - Probabilistic latent variables - Kingma & Welling (2013)
"Auto-Encoding Variational Bayes" - VAE - Kingma & Welling (2013)
"Tutorial on Variational Autoencoders" - VAE explanation - Doersch (2016)
β-VAE - Disentangled representations - Higgins et al., DeepMind (2017)
Normalizing Flows - Invertible generative models - Rezende & Mohamed, DeepMind (2015)
"Variational Inference with Normalizing Flows" - Flow-based models - Rezende & Mohamed (2015)
"Density Estimation Using Real NVP" - Real-valued non-volume preserving - Dinh, Sohl-Dickstein & Bengio (2016)
"Glow: Generative Flow Using Invertible 1x1 Convolutions" - Flow model - Kingma & Dhariwal, OpenAI (2018)
Deep Q-Network (DQN) - Deep RL for Atari - Mnih et al., DeepMind (2015)
"Playing Atari with Deep Reinforcement Learning" - DQN - Mnih et al., DeepMind (2013)
"Human-Level Control Through Deep RL" - DQN Nature paper - Mnih et al. (2015)
Double DQN - Overestimation reduction - van Hasselt, Guez & Silver, DeepMind (2015)
Dueling DQN - Value and advantage separation - Wang et al., DeepMind (2016)
Prioritized Experience Replay - Sample efficiency - Schaul et al., DeepMind (2015)
A3C - Asynchronous advantage actor-critic - Mnih et al., DeepMind (2016)
"Asynchronous Methods for Deep Reinforcement Learning" - A3C - Mnih et al. (2016)
DDPG - Deep deterministic policy gradient - Lillicrap et al., DeepMind (2015)
"Continuous Control with Deep Reinforcement Learning" - DDPG - Lillicrap et al. (2015)
Trust Region Policy Optimization (TRPO) - Stable policy updates - Schulman et al., Berkeley (2015)
Proximal Policy Optimization (PPO) - Simplified TRPO - Schulman et al., OpenAI (2017)
"Proximal Policy Optimization Algorithms" - PPO - Schulman et al. (2017)
AlphaGo - Go playing AI defeats Lee Sedol - Silver et al., DeepMind (2016)
"Mastering the Game of Go with Deep Neural Networks" - AlphaGo - Silver et al. (2016)
AlphaGo Zero - Self-play from scratch - Silver et al., DeepMind (2017)
"Mastering the Game of Go Without Human Knowledge" - AlphaGo Zero - Silver et al. (2017)
AlphaZero - General game playing - Silver et al., DeepMind (2017)
"A General Reinforcement Learning Algorithm that Masters Chess, Shogi and Go" - AlphaZero - Silver et al. (2018)

2017-2019 - Transformers and Transfer Learning

Transformer Architecture - Scaled attention over recurrence - Vaswani et al., Google (2017)
"Attention Is All You Need" - Transformer - Vaswani et al. (2017)
Self-Attention Mechanism - Scaled dot-product attention - Vaswani et al. (2017)
Multi-Head Attention - Parallel attention mechanisms - Vaswani et al. (2017)
Positional Encoding - Position information for transformers - Vaswani et al. (2017)
BERT - Bidirectional encoder representations - Devlin et al., Google (2018)
"BERT: Pre-training of Deep Bidirectional Transformers" - BERT - Devlin et al. (2018)
Masked Language Modeling - BERT pre-training objective - Devlin et al. (2018)
"ELMo: Deep Contextualized Word Representations" - Context-dependent embeddings - Peters et al., AI2 (2018)
GPT (Generative Pre-trained Transformer) - Autoregressive language model - Radford et al., OpenAI (2018)
"Improving Language Understanding by Generative Pre-Training" - GPT - Radford et al. (2018)
GPT-2 - Large-scale language model - Radford et al., OpenAI (2019)
"Language Models are Unsupervised Multitask Learners" - GPT-2 - Radford et al. (2019)
XLNet - Permutation language modeling - Yang et al., CMU/Google (2019)
RoBERTa - Robustly optimized BERT - Liu et al., Facebook (2019)
ALBERT - Lite BERT with parameter sharing - Lan et al., Google (2019)
T5 - Text-to-text transfer transformer - Raffel et al., Google (2019)
"Exploring the Limits of Transfer Learning with T5" - Unified framework - Raffel et al. (2019)
Vision Transformer (ViT) - Transformers for images - Dosovitskiy et al., Google (2020)
"An Image is Worth 16x16 Words" - ViT - Dosovitskiy et al. (2020)
DETR - Detection transformer - Carion et al., Facebook (2020)
"End-to-End Object Detection with Transformers" - DETR - Carion et al. (2020)
Swin Transformer - Hierarchical vision transformer - Liu et al., Microsoft (2021)
"Swin Transformer: Hierarchical Vision Transformer" - Shifted windows - Liu et al. (2021)
CLIP - Contrastive language-image pre-training - Radford et al., OpenAI (2021)
"Learning Transferable Visual Models From Natural Language Supervision" - CLIP - Radford et al. (2021)
DALL-E - Text-to-image generation - Ramesh et al., OpenAI (2021)
"Zero-Shot Text-to-Image Generation" - DALL-E - Ramesh et al. (2021)
DALL-E 2 - Improved text-to-image - Ramesh et al., OpenAI (2022)
"Hierarchical Text-Conditional Image Generation with CLIP Latents" - DALL-E 2 - Ramesh et al. (2022)

2020-2023 - Large Language Models Era

GPT-3 - 175B parameter language model - Brown et al., OpenAI (2020)
"Language Models are Few-Shot Learners" - GPT-3 - Brown et al. (2020)
In-Context Learning - Learning from examples without gradient updates - Brown et al. (2020)
Few-Shot Prompting - Task specification via examples - Brown et al. (2020)
Zero-Shot Learning - Task performance without examples - GPT-3 (2020)
"Scaling Laws for Neural Language Models" - Model scaling behavior - Kaplan et al., OpenAI (2020)
"Training Compute-Optimal Large Language Models" - Chinchilla scaling laws - Hoffmann et al., DeepMind (2022)
Chinchilla - Compute-optimal 70B model - Hoffmann et al., DeepMind (2022)
PaLM - 540B parameter model - Chowdhery et al., Google (2022)
"PaLM: Scaling Language Modeling with Pathways" - Large-scale training - Chowdhery et al. (2022)
LaMDA - Language model for dialogue - Thoppilan et al., Google (2022)
"LaMDA: Language Models for Dialog Applications" - Conversational AI - Thoppilan et al. (2022)
InstructGPT - RLHF for alignment - Ouyang et al., OpenAI (2022)
"Training Language Models to Follow Instructions with Human Feedback" - InstructGPT - Ouyang et al. (2022)
Reinforcement Learning from Human Feedback (RLHF) - Alignment technique - Christiano et al., OpenAI (2017)
"Deep Reinforcement Learning from Human Preferences" - RLHF foundations - Christiano et al. (2017)
Constitutional AI - Self-alignment technique - Bai et al., Anthropic (2022)
"Constitutional AI: Harmlessness from AI Feedback" - Self-supervised alignment - Bai et al. (2022)
ChatGPT Launch - Conversational AI breakthrough - OpenAI (Nov 2022)
GPT-4 - Multimodal large language model - OpenAI (2023)
"GPT-4 Technical Report" - State-of-the-art LLM - OpenAI (2023)
Claude - Large language model - Anthropic (2023)
Claude 2 - Enhanced capabilities - Anthropic (2023)
LLaMA - Open-weight language models - Touvron et al., Meta (2023)
"LLaMA: Open and Efficient Foundation Language Models" - 7B-65B models - Touvron et al. (2023)
LLaMA 2 - Improved open models - Touvron et al., Meta (2023)
Alpaca - Instruction-following LLaMA - Taori et al., Stanford (2023)
Vicuna - Chatbot trained from ShareGPT - Chiang et al., LMSYS (2023)
Mistral 7B - High-performance 7B model - Jiang et al., Mistral AI (2023)
Mixtral - Mixture of experts model - Jiang et al., Mistral AI (2024)
Gemini - Multimodal AI model - Google DeepMind (2023)
"Gemini: A Family of Highly Capable Multimodal Models" - Gemini - Google (2023)
"Chain-of-Thought Prompting Elicits Reasoning" - Reasoning prompts - Wei et al., Google (2022)
Chain-of-Thought (CoT) - Step-by-step reasoning - Wei et al. (2022)
"Self-Consistency Improves Chain of Thought Reasoning" - Ensemble reasoning - Wang et al., Google (2022)
"Tree of Thoughts: Deliberate Problem Solving with LLMs" - Advanced reasoning - Yao et al., Princeton (2023)
"ReAct: Synergizing Reasoning and Acting in Language Models" - Reasoning + actions - Yao et al., Princeton (2022)
"Toolformer: Language Models Can Teach Themselves to Use Tools" - Tool use - Schick et al., Meta (2023)
"Sparks of Artificial General Intelligence: Early Experiments with GPT-4" - GPT-4 capabilities - Bubeck et al., Microsoft (2023)
Stable Diffusion - Open-source text-to-image - Rombach et al., Stability AI (2022)
"High-Resolution Image Synthesis with Latent Diffusion Models" - Stable Diffusion - Rombach et al. (2022)
Diffusion Models - Denoising diffusion probabilistic models - Ho, Jain & Abbeel, Berkeley (2020)
"Denoising Diffusion Probabilistic Models" - DDPM - Ho et al. (2020)
"Diffusion Models Beat GANs on Image Synthesis" - Diffusion superiority - Dhariwal & Nichol, OpenAI (2021)
"Score-Based Generative Modeling Through SDEs" - Score matching - Song et al., Stanford (2021)
ControlNet - Conditional image generation - Zhang et al., Stanford (2023)
"Adding Conditional Control to Text-to-Image Diffusion Models" - ControlNet - Zhang et al. (2023)
Midjourney - AI art generation platform - Midjourney Inc. (2022)
DALL-E 3 - Latest text-to-image - OpenAI (2023)
Imagen - Google's text-to-image - Saharia et al., Google (2022)
"Photorealistic Text-to-Image Diffusion Models" - Imagen - Saharia et al. (2022)

2023-2025 - Multimodal AI and AGI Research (Active)

GPT-4V (Vision) - Multimodal GPT-4 - OpenAI (2023)
Claude 3 (Opus, Sonnet, Haiku) - Model family - Anthropic (2024)
Claude 3.5 Sonnet - Enhanced reasoning - Anthropic (2024)
Gemini 1.5 - Long context multimodal - Google DeepMind (2024)
"Gemini 1.5: Unlocking Multimodal Understanding Across Millions of Tokens" - Long context - Google (2024)
GPT-4 Turbo - Improved efficiency - OpenAI (2023)
o1 (Strawberry) - Reasoning-focused model - OpenAI (2024)
LLaMA 3 - Latest open-weight models - Meta (2024)
"The LLaMA 3 Herd of Models" - Model family - Meta (2024)
"Multimodal Learning with Transformers" - Multimodal survey - Xu et al. (2023)
Video Generation Models - Sora and others - Various (2024)
Sora - Text-to-video generation - OpenAI (2024)
"World Models: Simulating Reality with Video Diffusion" - Video generation - Various (2024)
Retrieval-Augmented Generation (RAG) - Knowledge grounding - Lewis et al., Meta (2020)
"Retrieval-Augmented Generation for Knowledge-Intensive NLP" - RAG - Lewis et al. (2020)
"In-Context Retrieval-Augmented Language Models" - RAG improvements - Ram et al. (2023)
Long Context Models - 100K+ token context - Various (2023-2024)
"Extending Context Window of Large Language Models" - Long context techniques - Various (2023)
Mixture of Experts (MoE) - Sparse models - Shazeer et al. (2017)
"Outrageously Large Neural Networks: The Sparsely-Gated MoE" - MoE - Shazeer et al. (2017)
"Switch Transformers: Scaling to Trillion Parameter Models" - Large MoE - Fedus et al., Google (2021)
Model Quantization - Efficient inference - Various (2020s)
"LLM.int8(): 8-bit Matrix Multiplication" - Quantization - Dettmers et al. (2022)
LoRA - Low-rank adaptation for fine-tuning - Hu et al., Microsoft (2021)
"LoRA: Low-Rank Adaptation of Large Language Models" - Efficient fine-tuning - Hu et al. (2021)
QLoRA - Quantized LoRA - Dettmers et al. (2023)
"QLoRA: Efficient Finetuning of Quantized LLMs" - Efficient training - Dettmers et al. (2023)
Parameter-Efficient Fine-Tuning (PEFT) - Adapter methods - Various (2020s)
Prompt Tuning - Soft prompt optimization - Lester et al., Google (2021)
"The Power of Scale for Parameter-Efficient Prompt Tuning" - Prompt tuning - Lester et al. (2021)
"P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning" - P-tuning - Liu et al. (2021)
Direct Preference Optimization (DPO) - RLHF alternative - Rafailov et al., Stanford (2023)
"Direct Preference Optimization" - Simplified alignment - Rafailov et al. (2023)
"Constitutional AI: Harmlessness from AI Feedback" - AI safety - Bai et al., Anthropic (2022)
Red Teaming Language Models - Adversarial testing - Perez et al., Anthropic (2022)
"Red Teaming Language Models to Reduce Harms" - Safety testing - Perez et al. (2022)
Mechanistic Interpretability - Understanding model internals - Anthropic/OpenAI (2020s)
"Towards Monosemanticity: Decomposing Language Models" - Interpretability - Anthropic (2023)
"Superposition, Memorization, and Double Descent" - Model behavior - Various (2020s)
Scaling Laws - Model performance prediction - Kaplan et al., OpenAI (2020)
"Emergent Abilities of Large Language Models" - Capability phase transitions - Wei et al., Google (2022)
"Language Models (Mostly) Know What They Know" - Uncertainty - Kadavath et al., Anthropic (2022)
Anthropic's Sleeper Agents - Deceptive alignment research - Hubinger et al., Anthropic (2024)
"Sleeper Agents: Training Deceptive LLMs That Persist Through Safety Training" - AI safety - Hubinger et al. (2024)
AI Agents - Autonomous LLM-based agents - Various (2023-2024)
"AutoGPT: An Experimental Open-Source Attempt at Making GPT-4 Fully Autonomous" - Agent framework - AutoGPT Project (2023)
"ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs" - Tool use - Qin et al. (2023)
"Voyager: An Open-Ended Embodied Agent with LLMs" - Minecraft agent - Wang et al., NVIDIA (2023)
"Generative Agents: Interactive Simulacra of Human Behavior" - Social simulation - Park et al., Stanford (2023)
World Models - Model-based RL - Ha & Schmidhuber (2018)
"World Models" - Learning environment models - Ha & Schmidhuber (2018)
DreamerV3 - General world model - Hafner et al., DeepMind (2023)
"Mastering Diverse Domains Through World Models" - DreamerV3 - Hafner et al. (2023)
MuZero - Model-based planning without explicit model - Schrittwieser et al., DeepMind (2019)
"Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model" - MuZero - Schrittwieser et al. (2019)
Decision Transformer - RL as sequence modeling - Chen et al., Berkeley (2021)
"Decision Transformer: Reinforcement Learning via Sequence Modeling" - Transformer RL - Chen et al. (2021)
Robotics with Foundation Models - LLMs for robotics - Various (2023-2024)
"RT-2: Vision-Language-Action Models" - Robotic transformer - Brohan et al., Google (2023)
"PaLM-E: An Embodied Multimodal Language Model" - Embodied AI - Driess et al., Google (2023)
"Code as Policies: Language Model Programs for Embodied Control" - Robot control - Liang et al., Google (2022)
AlphaFold 2 - Protein structure prediction - Jumper et al., DeepMind (2021)
"Highly Accurate Protein Structure Prediction with AlphaFold" - AlphaFold 2 - Jumper et al. (2021)
AlphaFold 3 - Protein interactions and complexes - Abramson et al., DeepMind (2024)
ESMFold - Fast protein folding - Lin et al., Meta (2022)
"Language Models of Protein Sequences at Scale" - ESM-2 - Lin et al. (2022)
"A Survey of Large Language Models" - LLM overview - Zhao et al., RUC (2023)
"Foundation Models: Opportunities and Risks" - Stanford HAI report - Bommasani et al., Stanford (2021)
"On the Opportunities and Risks of Foundation Models" - Comprehensive analysis - Stanford HAI (2021)

Key AI Resource Repositories

arXiv.org (cs.AI, cs.LG, cs.CL, cs.CV)
Papers with Code
Google Scholar
ACL Anthology (NLP)
NeurIPS/ICML/ICLR Proceedings
OpenReview.net
Distill.pub (interpretability)
AI Alignment Forum

Artificial Intelligence

Podcasts and Talks

Podcasts

History and Foundations

Deep Learning Explained

Major Breakthroughs and Demos

AI Safety and Ethics

Industry Leaders and Visionaries

Future of AI and AGI

Foundational Papers

1940s-1950s - Foundations and Early Concepts

1960s - Early AI Systems and Search

1970s - Expert Systems and Knowledge Representation

1980s - Neural Networks Revival and ML Foundations

1990s - Statistical ML and Practical Applications

2000-2005 - Kernel Methods and Graphical Models

2006-2012 - Deep Learning Renaissance

2013-2016 - Computer Vision and NLP Breakthroughs

2014-2017 - Generative Models and RL

2017-2019 - Transformers and Transfer Learning

2020-2023 - Large Language Models Era

2023-2025 - Multimodal AI and AGI Research (Active)

Key AI Resource Repositories

Request an Introduction