A neural network is a chain of functions. Each layer takes a tensor in, does one simple mathematical operation, and passes a tensor out. There’s no magic in it. It’s linear algebra, a nonlinearity, and calculus, repeated. Here’s the chain in the order data actually flows through it: forward pass first, then how it learns. Every equation gets paired with its exact PyTorch equivalent, so you can see where the math actually lives in code. ...
Most neural-network explanations start with math. That’s honest. But the ideas stick when you’ve actually broken something. Each section below is a live demo: click Run, watch it train, change a control, run it again. Activations exist for a reason Depth without nonlinearity is a lie Embeddings learn similarity from next-token alone Memorization vs generalization 1. Activations exist for a reason A linear model and a ReLU model, side by side, trying to separate a red ring from a blue one. The linear model can’t draw a curved boundary no matter how long it trains. Without a nonlinear activation, a stack of layers collapses to one matrix multiply. The curved shape you need is impossible to express. ...