But what is a neural network? | Chapter 1, Deep learning

16,308,531
0
Published 2017-10-05
What are the neurons, why are there layers, and what is the math underlying it?
Help fund future projects: www.patreon.com/3blue1brown
Written/interactive form of this series: www.3blue1brown.com/topics/neural-networks

Additional funding for this project provided by Amplify Partners

Typo correction: At 14 minutes 45 seconds, the last index on the bias vector is n, when it's supposed to in fact be a k. Thanks for the sharp eyes that caught that!

For those who want to learn more, I highly recommend the book by Michael Nielsen introducing neural networks and deep learning: goo.gl/Zmczdy

There are two neat things about this book. First, it's available for free, so consider joining me in making a donation Nielsen's way if you get something out of it. And second, it's centered around walking through some code and data which you can download yourself, and which covers the same example that I introduce in this video. Yay for active learning!
github.com/mnielsen/neural-networks-and-deep-learn…

I also highly recommend Chris Olah's blog: colah.github.io/

For more videos, Welch Labs also has some great series on machine learning:
   • Learning To See [Part 1: Introduction]  
   • Neural Networks Demystified [Part 1: ...  

For those of you looking to go even deeper, check out the text "Deep Learning" by Goodfellow, Bengio, and Courville.

Also, the publication Distill is just utterly beautiful: distill.pub/

Lion photo by Kevin Pluck

Thanks to these viewers for their contributions to translations
German: @fpgro
Hebrew: Omer Tuchfeld
Hungarian: Máté Kaszap
Italian: @teobucci, Teo Bucci

-----------------
Timeline:
0:00 - Introduction example
1:07 - Series preview
2:42 - What are neurons?
3:35 - Introducing layers
5:31 - Why layers?
8:38 - Edge detection example
11:34 - Counting weights and biases
12:30 - How learning relates
13:26 - Notation and linear algebra
15:17 - Recap
16:27 - Some final words
17:03 - ReLU vs Sigmoid

Correction 14:45 - The final index on the bias vector should be "k"

------------------
Animations largely made using manim, a scrappy open source python library. github.com/3b1b/manim

If you want to check it out, I feel compelled to warn you that it's not the most well-documented tool, and has many other quirks you might expect in a library someone wrote with only their own use in mind.

Music by Vincent Rubinetti.
Download the music on Bandcamp:
vincerubinetti.bandcamp.com/album/the-music-of-3bl…

Stream the music on Spotify:
open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u

If you want to contribute translated subtitles or to help review those that have already been made by others and need approval, you can click the gear icon in the video and go to subtitles/cc, then "add subtitles/cc". I really appreciate those who do this, as it helps make the lessons accessible to more people.
------------------

3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).

If you are new to this channel and want to see more, a good place to start is this playlist: 3b1b.co/recommended

Various social media stuffs:
Website: www.3blue1brown.com/
Twitter: twitter.com/3Blue1Brown
Patreon: patreon.com/3blue1brown
Facebook: www.facebook.com/3blue1brown
Reddit: www.reddit.com/r/3Blue1Brown

All Comments (21)
  • @EebstertheGreat
    Most educational videos give viewers the impression that they are learning something, while in reality, they cannot reliably explain any of the important points of the video later, so they haven't really learned anything. But your videos give me the impression that I haven't learned anything, because all the points you make are sort of obvious in isolation, while in reality, after watching them I find myself much better able to explain some of the concepts in simple, accurate terms. I hope more channels follow this pattern of excellent conceptual learning.
  • @kummer45
    I study mathematics, physics and architecture. By definition this man is an ORACLE in the strict meaning of the word. With all honesty I never imagined someone explaining complex topics with the dexterity this man has. He is literally an institution and an outstanding teacher. The computer graphics and the illustrations are simply perplexing. This guy never evades complexity. He never evades complex arguments. He illustrate the complexity and dive into the exhaustive explanation of the details. It's extremely rare to see a professor and a dedicated user to put a lot of effort explaining, animating and describing mathematics the way he does.
  • @ss_avsmt
    No man, we don't get notifications for your videos. We search for 3b1b. That's how powerful your content is.
  • I'm currently taking a computer science math course where the professor strongly advised everyone to watch this exact video series to get an intuition about what all the math is actually used for.
  • Quote: “Any fool can make something complicated. It takes a genius to make it simple.”…..nailed.
  • @AwesumBear
    I can't wait for neural networking to be able to recognize my doctor's prescription.
  • @tvo18868
    Your videos are singlehandedly keeping my PhD research on track. Thank you for your time and effort!
  • @shivshankarpe
    I am blown away by the visual clarity of this description of otherwise a complex technology! More please, I am willing to pay!
  • @BhuvanGabbita
    It takes 3000-4000 lines of code to make those graphics possible, he's a freakin legend
  • @buihung3704
    This is how you taught Deep Learning, people. I've seen lectures that either be categorized into 2 groups: too hard or too shallow/general. You have balanced between them. Thanks you so much!
  • @luukburger
    I just love the way the concepts of neural networks are explained in this video. After watching it, you feel like you have an idea about the "building blocks" of a neural network. Since I'm new to the topic, it's hard to judge whether crucial things are left out or over-simplified, but I feel it's a great introduction to the topic. Thanks a lot for sharing this!
  • @mheidari988
    I am Programming for more than ten years and I never saw anyone explain a complex idea by such a clean and clear terms. Well done.
  • 3Blue1Brown is the go-to channel that explains complex math concepts with the highest clarity without any loss of complexity of the topic. Simply brilliant!
  • @bambambhole8282
    In schools everyone taught us to practice maths but this man teaches us to imagine maths
  • @MrTyty527
    Around 2 years ago I was a sophomore statistics student and had no idea what deep learning is, until I met this video and 3b1b channel. His clear explanation of neural network and animations blew my mind. Since then I started my journey in machine learning. For a random reason I clicked onto this video again, and realized how long my journey in this field have been. This video really changed my life and I am really grateful about it.
  • This channel and the visualizations it produces to teach subjects like this one is the best advance in the history of communicating mathematical ideas. It's extraordinarily inspiring that one person can have such a large impact on the world today (and for generations to come). Thank you, Grant Sanderson.
  • @hawkite3185
    The fact that I was sent here by my university lecturer is a testament to how good 3Blue1Brown is.
  • @mgonetwo
    Thanks for all the effort you put into the work! Cannot stress enough how glad and inspired I am for having people like you on the planet.