Experiment № 077 · The story
Whoever Points at You
Why I picked it
I have spent seventy-six days on algorithms that mostly nobody argues about, and then realised I had skipped the one that quietly decided what a generation of people read. Brin and Page published PageRank in 1998 with a definition that shouldn't work: a page's importance is the sum of the importance of every page pointing at it, each divided by how many links that page hands out. Read it again — the answer appears on both sides. You need every score to compute any score. The fix is almost insultingly simple. Give every page the same score, apply the rule once, apply it again, and keep going until the numbers stop moving. They always stop moving, and they always stop in the same place no matter what you guessed. I wanted to watch that happen rather than take it on faith.
What I enjoy about it
Two completely different-looking things converge on the same numbers. Iterate the whole web is the linear algebra: every page pushes its score down its out-links, all at once, and the whole vector settles in about twenty passes. Follow one random surfer throws that away and just wanders — click a link, click a link, occasionally get bored and jump somewhere random — and the fraction of time it spends on each page creeps toward exactly the same answer, one hop at a time. Watching the bars agree is the moment the abstraction lands. My favourite thing to do, though, is the link farm: a client page and a stack of junk pages that exist only to point at it. It collects more inbound links than anything else on the board and still finishes fourth or fifth, because the pages voting for it have nothing to give. The pink tick on every bar shows where crude link-counting would have put that page, and above the client it towers over the bar by a factor of three. Then try rank sink and drag damping up towards 1.00: a small clique with no way out slowly swallows every point on the board, and the moment you pull damping back, the score floods out again.
Something you might take away
The damping factor is the interesting part, and it is not a fudge. Fifteen percent of the time the surfer abandons the link graph entirely and teleports to a random page, and that single concession does three jobs at once: it rescues pages with no outgoing links, it stops closed cliques hoarding the whole score, and it is what guarantees there is one answer to converge to at all. It also sets the speed — each pass shrinks the remaining error by roughly the damping factor, which is why 0.85 needs about twenty passes and 0.99 needs hundreds. More generally: this is the difference between counting votes and weighing them. Anyone can manufacture a thousand pages that link to you; nobody can manufacture a thousand pages that matter, because mattering is defined recursively in terms of everyone else. Any time you need a reputation number that is expensive to fake, that recursion — importance is borrowed from whoever already has some — is the trick worth stealing.