How to Create a Neural Network (and Train it to Identify Doodles)

1,816,692
0
Published 2022-08-12
Exploring how neural networks learn by programming one from scratch in C#, and then attempting to teach it to recognize various doodles and images.

Source code: github.com/SebLague/Neural-Network-Experiments
Demo: sebastian.itch.io/neural-network-experiment

If you'd like to support me in creating more videos (and get early access to new stuff), you can join my patreon here: www.patreon.com/SebastianLague

The data is from:
Digits yann.lecun.com/exdb/mnist/
Fashion github.com/zalandoresearch/fashion-mnist
Doodles github.com/googlecreativelab/quickdraw-dataset
Cifar10 www.cs.toronto.edu/~kriz/cifar.html

Chapters
0:00 Introduction
2:39 The decision boundary
3:49 Weights
5:42 Biases
6:45 Hidden layers
7:45 Programming the network
9:57 Activation functions
12:42 Cost
15:07 Gradient descent example
18:22 The cost landscape
19:55 Programming gradient descent
21:10 It's learning! (slowly)
23:21 Calculus example
27:34 The chain rule
29:50 Some partial derivatives
33:14 Backpropagation
39:25 Digit recognition
43:56 Drawing our own digits
47:37 Fashion
48:25 Doodles
52:00 The final challenge

Music:
Cosmic Waves - Michael FK
Amber - The Stolen Orchestra
Beyond the Horizon - Sounds Like Sander
Air - Assaf Ayalon
Purest Form - Sounds Like Sander
Hear Wide Open - Sounds Like Sander
Universal Wonder - Moments
Roman P - Moments
All In Good Time - Shimmer
It Will Come Back - The Stolen Orchestra
Frontier - Shimmer
New Moon - Cloud Wave
Sunflower - Cody Martin
Inner Peace - Moments
Enchanted - Cody Martin
Just Around The Corner - Shimmer

All Comments (21)
  • Hey everyone, at 20:17 it should be -= costGradientW (the minus sign is missing). I somehow managed to delete it while formatting the code for the recording! Thanks for letting me know in the comments.
  • As someone who has some experience with machine learning i can say this has to be the most intuitive explanation i have ever seen
  • I’ve never had someone explain calculus so intuitively. The quality of this content is absolutely incredible.
  • @wlockuz4467
    I am a software engineer and I've always wanted to learn machine learning, being able to code is not a problem for me but being terrible at maths and statistics makes it hard to get accustomed to all the terms and concepts. I've tried multiple courses from different platforms and instructors and all of them try to teach you "what" to do instead of "why" to do it. I personally find learning more intuitive when I know why am I doing something instead of blindly following steps. This video is exactly the type of introduction to ML that I've always wished for, The explanations are highly intuitive and most importantly visual, there are no assumptions and no brushing over concepts, Nothings being done just for the sake of it, Everything is explained in simple language. I admit I'll still have to watch this video a couple of more times to make full sense of it because its jam-packed with so much information. You'll probably miss this comment in the sea of other comments (although I hope note) but I genuinely want to thank you, This video has relit my once dead interest in ML, I would love to see more videos from you on this topic or least get some recommendation on where I can learn more.
  • @CYON4D
    The production quality is off the charts. As a software developer I can't even imagine the amount of hard work, research, technical knowledge, expertise, patience and determination this must have required. Hats off to you :)
  • @TAP7a
    "I'm bad at naming things" There are only 2 hard problems in computer science: cache invalidation, naming variables, and off-by-one errors
  • @saar5947
    You're probably not gonna see this message, but I want you to know you give me inspiration and motivation to learn, do and achieve more as i believe you do for many others
  • @Suburp212
    I wish somebody had explained calculus like this in school. Intuitive, descriptive, visual, simple, elegant. This content is marvellous.
  • @mangoalias608
    everything else aside, can we all appreciate the incredible visuals all of Sebastian's videos have? the animated graphs, the visualizations, the explanations, its so pleasing :)
  • @The14Some1
    25:18 OMG, and at this point I've completely realized the true nature of the derivative — why it becomes a slope function, why x^2 turns into 2x and so on. It was one of those mind-blowing moments of insight, which most of us have experienced at least once in our life. Thank you, Sebastian!
  • @timothysnave
    Holy crap that car in the beginning even found like a line of best fit. Crazy.
  • @blitzarsun
    This is the most intuitive explanation of machine learning I've watched. I hope you return back to it soon!
  • @oskar7063
    This video is looking awesome, do you have you're own animation library like manim or the one from aarthificial or Freya Holmer?
  • @cvntdav
    I love every single video that you make, you make such amazing stuff! I wish you could bring back the "How computers work" series which made me discover my passion ★
  • @GumRamm
    Any reasonable person: “I will use one of the many existing Python libraries that implement backpropagation for me in an efficient and easy-to-use way” Sebastian: “I like writing C#” Jokes aside, very cool and impressive project as always. And of course presented in a stunning and intuitive fashion, keep ‘em coming!
  • @realryleu
    That graph renderer you made looks almost exactly like desmos with a dark theme enforces by the dark reader extension.
  • @vb0t429
    It's impressive what Sebastian can achieve with enough time and dedication
  • Sebastian Lague, your videos might go down in history as the most well-produced educational videos on YouTube of all time. And it’s hard to even say that something beats 3B1B’s videos.
  • @Kibito80
    It’s such a bittersweet moment watching your videos as they’ve recently come out because I know such great content with such level of detail takes so long to produce and it’s going to be a long and sad time until your next video comes out. I just love your videos man, everything you touch becomes gold, you make so many topics that are so boringly taught at uni seem sooooo exciting!