A stepped, visual walk from the black box down to the multiply-accumulate that silicon is built around.
Scroll to beginMatrix multiplication takes two matrices and produces a third. A is 2×3, B is 3×2, and the result C is 2×2. The inner dimensions have to match — that shared 3 is the axis we sum over, and it disappears. The outer dimensions survive into the shape of the output.
Open the box and each matrix is just a grid of numbers. Note the arrangement: A sits to the left of C and shares its rows, B sits above C and shares its columns. Every output cell now lies at the intersection of exactly one row and one column.
Take C[0,0]. It depends on one row of A and one column of B — nothing else. That independence is why matmul parallelizes so well: every output cell can be computed at the same time, by a different worker, with no coordination between them.
Select any cell in C to compute it.
Zoom in on that single cell. Pair the row against the column element by element, multiply each pair, and add the products. Three multiplies, two adds. This multiply-accumulate is the primitive that GPUs and NPUs build entire hardware pipelines around.
Now repeat for all four outputs. This toy example costs 2×2×3 = 12 multiply-accumulates. Scale the same loop to a transformer and it runs billions of times per token — which is why the arithmetic is rarely the bottleneck. Feeding it is.