Outsmarting AI, Part 3: River Crossings and the Wall of Complexity

A boat holds two. Everyone must get across. Certain combinations, left alone on a bank, end badly. That is the whole genre, and it has been entertaining people since at least the eighth century, when the puzzle of the wolf, the goat and the cabbage appeared in a collection attributed to Alcuin of York.

Its charm is that the rules fit in one breath and the solution does not. You must be willing to row something back — to undo progress on purpose — and that single idea is what makes it hard for people and, it turns out, for machines.

The version everyone knows

A farmer has a wolf, a goat and a cabbage. The boat carries the farmer and one item. Leave the wolf with the goat and there's no goat. Leave the goat with the cabbage and there's no cabbage. Our Wolf, Goat, Cabbage page has the full walkthrough, but the key move is taking the goat across first and then bringing it back later. People hate that step. It looks like failure.

Where scale breaks the machine

Researchers at Apple tested language models on river crossings and on the Tower of Hanoi, where disks move one at a time and a larger disk may never sit on a smaller one. With small numbers, the models were fine. Push past roughly six travelers or six disks and performance fell away sharply.

The paper went viral, and the argument about it was as interesting as the finding. Critics pointed out that humans also make mistakes as steps multiply — so is this a special limitation of machine reasoning, or just ordinary error accumulation in a system with no scratch paper and no patience?

Both readings are defensible. What isn't in dispute is the shape of the curve: a confident performance that degrades not gently but suddenly, and with no warning in the model's tone. It sounds exactly as sure at step forty as it did at step four.

The human advantage: the backward step

Ask a person who solves these well how they do it and they rarely describe searching every branch. They describe noticing a constraint — the goat is the problem, the goat must move twice — and then building around it.

That is compression. You replace a search through hundreds of states with one idea about which object is unstable. It's the same instinct that lets you pack a car boot without trying every arrangement.

Try a harder crossing

Six travelers, three pairs, a boat for two, and a rule that no one may be left on a bank with the wrong company. It's the same puzzle with the volume turned up — and it is exactly where the models started to slide. Give yourself paper and a quiet ten minutes.

If you want the classic on a shirt, the Wolf, Goat, Cabbage tee carries the whole scene in burnt orange, with the answer a QR scan away instead of printed on your back.

The studies described here are discussed in "Can you outsmart AI?" by Grace Huckins, MIT Technology Review.