Diving into the Deep End
Hi everyone! When I want to dive into a topic, I need to fully immerse myself. I crave hours of solitude, diving into the weeds, solving problems, experimenting, and simply learning. As part of this journey, I’m also trying to figure out who I am and what I want to achieve in this emergent (for me) field of AI and machine learning.
The Starting Point: A Course in Deep Learning
I began my journey by taking an incredible course — [Introduction to Deep Learning](http://introtodeeplearning.com/) offered by the brilliant Alexander and Ava Amini. This course provided me with the language and understanding of the advances across AI and ML. We reviewed and built various neural networks, explored the fundamentals of image processing, and delved into the history and heritage of these techniques. This course helped me ‘catch up’ to be able to follow and understand some of the AI papers I come across.
For me, it struck the right balance between technical depth and practical functionality. Sure, there were portions that were way over my head, but I was able to follow along conceptually, which was my main goal. After completing this course, I realized that I do not want to be an AI/ML engineer focused on researching and building new components of large language models (LLMs). That’s not my path.
That being said, the engineer in me had to build networks and see nodes populated with content and activated. I had to do this to feel like I understand the inner workings. This course helped me achieve that even without fully grasping the deep math involved.
Building Mental Models
The class slides and visual depictions have become the foundation of my AI/ML mental models. I can now vividly picture forward-feed neural activity at various time steps, just as it was depicted in class. The gradual progression of topics, building on previous concepts — both historically and functionally — helped ground my understanding. I can now speak the language and grasp the core ideas.
Key Insights from the Course
1. The Avalanche Analogy:
Douglass Eck from Google DeepMind provided a compelling analogy, comparing the job market to an avalanche with the emergence of AI. The idea is that if you are mid-career (like me), you need to keep your head above the snowline — keep learning and adapting to this new technology. Future generations will use AI as seamlessly as we use the internet today, but since we’re here at its inception, we must figure out how to integrate it into our roles.
**2. The Evolution of AI:
**It’s astounding to see how various learnings, technologies, and discoveries have been cobbled together over time to create what we now know as AI. The mathematics underlying AI is genuine and profound.
**3. Mathematical Marvels:
** — Loss Functions and Backpropagation: These tools guide AI to reach ‘convergence,’ enabling it to arrive at acceptable answers based on its understanding of relationships.
— The Miracle of the Transformer: The math behind self-attention and the Q (query), K (key), and V (value) comparison is mind-blowing. Geometric mathematics is used to determine the similarity of input elements, which is fundamental to self-attention mechanisms.
— Convolutional Neural Networks (CNNs): The simplicity and elegance of CNNs, along with their impressive performance at scale, are fascinating. The masking algorithm combined with the convolution operation truly blew my mind.
Let me know if there’s anything specific you’re interested in, and I’ll happily dive deep to provide my perspectives and understanding.
Stay tuned for more insights from my learning journey!