Deep learning is a machine learning method in which a neural network with many layers learns from examples instead of people writing rules by hand. The network adjusts millions to trillions of numbers (the weights) until its predictions match the examples.
Picture an intern whom nobody tells how to recognize a cat. Instead they get thousands of photos with the right answer and correct themselves a little after every mistake. Eventually they even recognize cats they have never seen. An image recognition network works exactly like that.
1. Where deep learning sits within AI
The terms nest like Russian dolls. Artificial intelligence is the big frame. Inside it sits machine learning, meaning all methods in which a system learns from data. Inside that sits deep learning: learning with deep neural networks.
- AI without machine learning: a hand-written rule set, such as a chess program with fixed evaluation rules.
- Machine learning without deep learning: a decision tree that learns from data whether an email is spam.
- Deep learning: a network with dozens of layers that understands language or generates images.
That most well-known AI applications now rest on deep learning is a development of the last decade or so. The method itself is much older.
2. Neural networks: neuron, layer, depth
A neural network is built from very simple parts, the artificial neurons. A neuron does two things:
- Weighted sum: It takes several numbers, multiplies each by its own weight and adds everything up. A fixed extra value, the bias, is added on top.
- Activation function: It applies a simple rule to the result, for example “negative values become zero, positive ones stay” (ReLU). Only this rule keeps the whole network from collapsing into one simple calculation.
Many neurons side by side form a layer. The outputs of one layer are the inputs of the next. The raw data arrives at the front, such as the brightness values of a photo’s pixels. The result sits at the back, such as “cat: 94%”. In between lie the hidden layers.
The “deep” in deep learning simply means many hidden layers. Each layer builds on the one before. With images, early layers often detect edges, middle layers shapes such as ears or eyes, late layers whole faces. The network finds these intermediate steps itself, you do not specify them. LeCun, Bengio and Hinton describe exactly this as the core of deep learning: data is represented at several levels of abstraction.
3. How a network learns: one neuron worked by hand
During training only the weights and bias values change, together the parameters. The process repeats millions of times:
- Prediction: The network runs an example through and outputs an answer.
- Error (loss): A formula measures how far the answer is from the correct one.
- Backpropagation: The method works backwards through all layers to calculate how much each weight contributed to the error. Rumelhart, Hinton and Williams described it in Nature in 1986.
- Gradient descent: Each weight is moved a small step in the direction that lowers the error. The step size is called the learning rate.
The principle can be checked on a single neuron. An invented example with ReLU as the activation and “squared deviation” as the error measure:
Inputs: x1 = 2, x2 = 3
Weights: w1 = 0.5, w2 = -0.2 Bias: b = 0.1
Desired output: 1.0
1. Prediction
Sum = 2 * 0.5 + 3 * (-0.2) + 0.1 = 0.5
ReLU(0.5) = 0.5 -> output 0.5
2. Error (loss)
(0.5 - 1.0)^2 = 0.25
3. Backpropagation (derivatives)
Error with respect to output: 2 * (0.5 - 1.0) = -1
ReLU slope at 0.5: 1
Error with respect to w1: -1 * x1 = -2
Error with respect to w2: -1 * x2 = -3
Error with respect to b: -1
4. Gradient descent with learning rate 0.02
w1 = 0.5 - 0.02 * (-2) = 0.54
w2 = -0.2 - 0.02 * (-3) = -0.14
b = 0.1 - 0.02 * (-1) = 0.12
New pass
Sum = 2 * 0.54 + 3 * (-0.14) + 0.12 = 0.78
Output 0.78, error (0.78 - 1.0)^2 = 0.0484The error fell from 0.25 to about 0.05 after a single step. The learning rate needs care: with 0.1 instead of 0.02 the output would jump to 1.9 and overshoot the target. A real network does the same with billions of weights at once. Backpropagation is just the method that delivers these many derivatives efficiently.
4. Milestones
- 1986: Rumelhart, Hinton and Williams show in Nature how to adjust the weights of a multilayer network with backpropagation, so that hidden layers pick up useful features on their own.
- 2012: Krizhevsky, Sutskever and Hinton train a network with five convolutional layers and 60 million parameters on graphics cards. It sorts 1.3 million images into 1,000 classes and beats the previous methods by a wide margin. The network is known as AlexNet.
- 2017: Vaswani and colleagues introduce the transformer, an architecture without any recurrent layers that trains well in parallel. Today’s language models build on it.
- 2018: Yoshua Bengio, Geoffrey Hinton and Yann LeCun receive the Turing Award for breakthroughs that made deep neural networks a critical component of computing.
- 2024: John J. Hopfield and Geoffrey Hinton receive the Nobel Prize in Physics for discoveries that enable machine learning with artificial neural networks.
5. How big do the networks get?
A network’s size is counted in parameters, the number of its weights and bias values. AlexNet had 60 million. The table shows the largest models whose developers have published the number:
- Developer
- Moonshot AI
- Year
- 2026
- Total parameters
- 2.8 trillion
- Active per token
- 104 billion
- Developer
- Alibaba
- Year
- 2026
- Total parameters
- 2.4 trillion
- Active per token
- 95 billion
- Developer
- Alibaba
- Year
- 2026
- Total parameters
- 2.4 trillion
- Active per token
- n/a
- Developer
- Meta
- Year
- 2025
- Total parameters
- 2 trillion
- Active per token
- 288 billion
- Developer
- DeepSeek
- Year
- 2026
- Total parameters
- 1.6 trillion
- Active per token
- 49 billion
gradually.aiIn our directory, 177 models have a confirmed parameter count. The largest of them, Kimi K3, has 2.8 trillion parameters, which is about 16 times the size of GPT-3 at 175 billion. Many developers do not disclose the size of their current models at all. The table therefore shows the largest network with a published number, not necessarily the largest that exists.
The “Active per token” column works like this: in an ordinary (“dense”) model, all parameters compute. In a mixture-of-experts model the network is split into expert blocks, and only part of them works on each token. Total size and actual compute are far apart there. More in the entry on LLMs.
6. What deep learning is used for
- Images: face recognition, reading X-rays, quality control in manufacturing.
- Listening and speaking: dictation, subtitles, voice assistants, read-aloud text.
- Translation: machine translators that process whole sentences instead of single words.
- Text: language models that write, summarize and code. How words become numbers is explained in the entries on tokens and embeddings.
- Generating: image, video and music generators, see generative AI.
- Recommending and checking: suggestions in streaming services, fraud detection on payments.
A pretrained network can be trained further for your own task. When that makes sense is covered in the entry on fine-tuning.
7. Limits and misconceptions
- Hunger for data. A network learns from examples. If a task has little or poor data, the result is poor. If the data contains a bias, the network learns it too.
- Compute. Training needs many graphics cards and a lot of electricity. That is why only a few companies train the largest models themselves.
- Black box. With millions of weights, nobody can read off why the network gives exactly this answer. The entry on explainable AI shows how people approach this.
- Plausible instead of true. A language model optimizes what sounds likely, not what is correct. That is where hallucinations come from.
- Misconception: “The network thinks like the brain.” The name comes from a loose analogy. An artificial neuron is a formula, not a replica of a nerve cell.
- Misconception: “More layers are always better.” Depth helps with complex tasks but costs data and compute.
8. What does this mean for you?
- You do not need to do the math to use it. The picture is what matters: a network has learned patterns from examples and applies them. On its own it does not look things up and does not work through logic.
- Good results mean good data. If you want to use or adapt a network yourself, the quality of your examples matters more than the choice of architecture.
- Do not trust the output blindly. A network always produces an answer, even when it does not know. Check figures and sources, especially for important decisions.
- Size is not a quality mark. Which model suits your task is shown by benchmarks and prices in the model directory.