Some places to start learning AI & ML
Here’s a (irregularly updated) collection of links to my favourite MOOCs and other ressources on learning or deepen Artificial Intelligence and in particular Machine Learning / Deep Learning skills:
Image: Copyright (c) Walter Ritter
- Machine Learning by Andrew Ng, Stanford (very intuitive and practical, includes the basics)
- Deep Learning Specialization by Andrew Ng’s deeplearning.ai
- CS231n: Convolutional Neural Networks for Visual Recognition by Andrej Karpathy, Stanford (video lectures)
- Theories of Deep Learning by Hatef Monajemi, David Donoho and Vardan Papyan, Stanford
- Advanced Topics: Reinforcement Learning by David Silver, University College London (video lectures)
- Deep Reinforcement Learning Bootcamp by Pieter Abbeel et al., UC Berkeley
- Foundations of Machine Learning by David Rosenberg, Bloomberg (more geared towards math and statistical learning than Andrew’s course above)
- KI1: Artificial Intelligence by Thilo Stadelmann at ZHAW, based on the AIMA book
An alternative collection of courses (partially overlapping, though) is curated by the KDnuggest team.
If you are completely new to the topic of deep learning or even consider yourself a layperson in computer science, but want to get an introduction to some current frontiers of the state of the art, you can have a look at the excellent popular scientific compilation of Spektrum der Wissenschaft kompakt 06.18: Künstliche Intelligenz - wie Maschinen lernen lernen.
These reading resources might also be of interest (use cases, deep dives into specific methods):
- on CNNs
- Deep learning and neural networks by Michael Nielsen: a gentle introduction to neural networks in e-book form
- Beyond ImageNet - Deep Learning Industrial Practice by a few colleagues and myself
- on generative models
- Image Completion with Deep Learning in TensorFlow by Brandon Amos (on DCGANs)
- Generative Models by the OpenAI team (on DCGANs, VAEs and PixelRNNs)
- WaveNet: A Generative Model for Raw Audio by the DeepMind team (on PixelCNNs)
- …this area of course exploded in 2022 with new image and text generating models - see my AI2 module
- on RNNs
- The Unreasonable Effectiveness of Recurrent Neural Networks by Andrej Karpathy
- Understanding LSTM Networks by Chris Olah
- Attention and Augmented Recurrent Neural Networks by Chris Olah and Chan Carter
- on Reinforcement Learning
- OpenAI Spinning Up by Joshua Achiam of OpenAI (on how to start research in deep RL)
- Simple AlphaZero by Surag Nair (on MCTS with neural nets)
- Deep Reinforcement Learning Doesn’t Work Yet by Alex Irpan
- Lessons Learned Reproducing a Deep Reinforcement Learning Paper by Matthew Rahtz
- Deep Reinforcement Learning: Pong from Pixels by Andrej Karpathy (on Policy Gradients)
- Answer to “What is the way to understand Proximal Policy Optimization Algorithm in RL?” by Matthew Wilson (on PPO)
- Demystifying Deep Reinforcement Learning by Tambet Matiisen (on DQN)
- on meta learning
- Deep Learning-to-Learn Robotic Control by Pieter Abbeel
- Learning to learn by Chelsea Finn
- From zero to research — An introduction to Meta-learning by Thomas Wolf
- other
- How to avoid machine learning pitfalls: a guide for academic researchers by Michael A. Lones
- Deep Learning Tuning Playbook by staff @Google
- Making Deep Learning Go Brrrr From First Principles by Horace He of PyTorch on how to optimize DL code on GPUs
- A Recipe for Training Neural Networks by Andrej Karpathy
- ML best practices
- Interesting 4-post series on Learning in Brains and Machines
Generally: If you want to make AI/ML (or more broadly, data science) a career, check out the following guides:
- ML Job Interviews: The Ultimate Guide by Silvia Sapora
- Just know stuff (Or, how to achieve success in a machine learning PhD) by Patrick Kidger
- Spinning Up as a Deep RL Researcher by Joshua Achiam
- How to Become a Data Scientist by Alex Smith (also part 2)
- but also: Data science is different now by Vicki Boykis
- Data Scientists from our own upcoming book
- A survival Guide to a PhD by Andrej Karpathy
Additionally, try attending events of the Swiss Alliance for Data-Intensive Services, think about joining SGAICO, and subscribe to the The Batch.