← all experiments

Only When Wrong

Experiment № 107 · November 1, 2026

Frank Rosenblatt's 1958 perceptron learns a line that puts pink on one side and violet on the other. It looks at one point at a time. If the point is on the right side, nothing happens. If it's on the wrong side, the line is nudged toward it, by exactly that point. Every nudge leaves a trace. When a whole pass goes by with no mistakes, it stops.

Points

What the perceptron sees

points (pink · violet)— pass— points checked— nudges (mistakes)— mistakes this pass—
wrong side right now— closest point to the line— Novikoff: nudges at most—

The whole learning rule

repeat, in a shuffled order: for each point x with label y: // y = ±1 guess = sign(w · x + b) if guess ≠ y: // wrong side w += η · y · x b += η · yuntil a whole pass is clean

Mistakes in each pass

The arrow is w, pointing to the pink side. A nudge adds the wrong point to it (or subtracts it, for violet), so the line swings toward that point. After any number of nudges from zero, w is just the sum of the points it got wrong, each signed by its colour. Correct points never touch it.

Click to add a point · click a point to remove it · right-click adds the other colour · space plays · S steps · R resets the line · N new points · H hides panels

Experiment № 107 · The story

Only When Wrong

Why I picked it

The collection already has Kohonen's self-organizing map and Shannon's word-counting text machine, but not the perceptron, which most of modern machine learning traces back to. Frank Rosenblatt described it in 1958, and at the Cornell Aeronautical Laboratory he built it as hardware, the Mark I Perceptron: a camera of 400 photocells wired at random to a layer of units, with weights stored in potentiometers that electric motors turned during learning. The press got carried away and promised machines that would soon walk, talk and know themselves. Four years later Albert Novikoff proved something modest and solid: if any line separates the two classes, the perceptron finds one after a bounded number of mistakes. In 1969 Minsky and Papert's book Perceptrons showed how much a single layer can't do, with XOR as the famous example, and funding for neural networks dried up for over a decade. The whole rise and fall fits in a two-dimensional picture, so I wanted to draw it.

What I enjoy about it

It only learns from mistakes. A point on the correct side costs nothing and changes nothing, however close it is to the line. So if you start from zero, the weight vector at any moment is literally the sum of the points it has got wrong, each signed by its colour. Because of that, the nudge size η makes no difference from a zero start: every nudge is scaled by the same η, so the line is the same, just with longer arrows. Switch to a random start and it suddenly matters. I like the trails too. On a separable set the old lines fan in toward the gap. On XOR they never settle, and after a few hundred passes they have drawn a rosette, the record of a learner that can't find an answer because none exists. Then press + x · y. The learner doesn't change at all. It only gets one more number to look at for each point, and XOR becomes separable and is solved in a handful of nudges.

Something you might take away

Novikoff's bound doesn't depend on how many points there are. It depends on how much room there is between the two groups relative to how spread out they are: halve the gap and the worst case gets four times longer. Drag the room slider down to see the nudge count climb while the number of points stays the same. And when a process that only corrects its mistakes never settles, adding more passes won't help. Look at what the learner is able to see. Sometimes the fix is an extra feature, not more effort.