Can You Outsmart an AI? A Six-Part Puzzle Series

The cover of this month's MIT Technology Review carries a line that could have been written for this shop: "Test your wits on puzzles that stump AI." Inside, Grace Huckins lays out seven puzzles that have, at one time or another, defeated the most advanced language models on earth.

We found that irresistible. So this is the first of six articles about the strange gap between machine cleverness and human common sense — one theme at a time, each with a puzzle of our own that you can try right now.

Why puzzles are the real test

Puzzles have measured machine progress since the beginning. The phrase "machine learning" was popularised in 1959 by IBM's Arthur Samuel, writing about a program that taught itself checkers. Chess fell. Go fell. In late 2024, researchers found the best models could crack only about 18% of the New York Times Connections puzzles; months later, some were solving them almost perfectly.

But the failures are more interesting than the wins. A model that can summarise a legal filing will confidently tell you how many ice cubes survive in a hot frying pan. Change three words in a classic riddle and it answers the version it memorised instead of the one in front of it. Show it a picture instead of a paragraph and its reasoning quietly falls apart.

Those failures aren't random. They cluster into categories, and each category is a small window into how the machine thinks — and how you think.

What's coming in the series

1. Knights, knaves and the trap of familiarity — why a puzzle a model has seen before can be harder for it than one it hasn't.
2. River crossings and the wall of complexity — models handle three travellers and collapse at six.
3. Simple questions that machines get wrong — the arithmetic decoy, and why reading carefully beats computing quickly.
4. What AI can't see — mental rotation, coloured grids, and the weakness nobody has fixed.
5. Logic grids, step by step — the deduction puzzle as a fair fight between person and machine.
6. What the gap actually means — where this leaves us, and why we keep printing puzzles on shirts.

Warm up: three quick ones

Answer these as fast as you can, then check yourself.

1. A colony of bats doubles in number every day. The cave is completely full on day 60. On what day was it half full?

2. In which famous novel does Alice say, "I'm late, I'm late, for a very important date"?

3. A cook drops three ice cubes into a hot frying pan in the first minute, four in the second and five in the third. How many whole ice cubes are in the pan at the end of the fourth minute?

Answers

1. Day 59. If it doubles daily, the day before full is half full. Most people reach for 30.

2. She doesn't. That's the White Rabbit — and the line isn't in Lewis Carroll's book at all. It comes from the Disney film.

3. None. The pan is hot. The numbers are there to make you add instead of picture the scene. You can try our version at Hot Pan Count.

If you got all three, you did better than several models that can otherwise write a passable sonnet about thermodynamics.

Next up

Part two takes on knights and knaves — the island where everyone either always lies or always tells the truth. Our own version, Three Islanders, is already live if you'd like a head start.

The seven puzzles that inspired this series appear in "Can you outsmart AI?" by Grace Huckins, MIT Technology Review. The puzzles on this site are our own.