Problems by difficulty  / How this is scored

How problem difficulty is scored

Find problems How this is scored

Is this AI?

Yes, plainly: a language model rates each of the 2,023 Savchenko problems on 22 axes — from how much of an "aha" a problem takes, to how beautiful the solution turns out to be. The scores help a reader choose what to attempt next. They do not replace or check the solutions themselves.

Why "difficulty" isn't the raw model score

The model first gives every problem a 0-100 number on the overall_difficulty axis. That still isn't the number shown on the problem page — it first becomes a percentile rank across all 2,023 problems in the book: "harder than N% of the rest." That way the scale uses its full 0-100 width instead of clustering wherever the model happened to be generous.

The number actually shown is a rating from 800 to 3500 — nothing ever reads as "0" (which sounds like "no difficulty at all" rather than "easiest in the book"). That rating's own distribution isn't stretched linearly, which would produce an artificially flat histogram (a percentile is uniform by construction) — it's deliberately reshaped to look like a real difficulty scale: a large cluster of easy problems at the bottom, a broad plateau in the middle, a long declining tail at the top.

The 22 axes, grouped

The problem page shows 9 of these — a panel about 280px wide can't fit more and stay legible. Here are all 22, with what each one actually measures.

Headline

DifficultyThe headline number: overall difficulty for the reference solver, described below.

What it costs

Insighton the pageThe size of the "aha". Low: grind it out with standard machinery. High: nothing happens until you see the one idea.
Mathematicson the pageHow much mathematical machinery is needed — integration, differential equations, series, vector calculus.
Physics depthHow much physics beyond the school syllabus you need to already know, walking in.
Computationon the pageThe algebraic grind after the idea is found. Independent of how hard the idea itself was.
Trapson the pageHow easy it is to take a plausible-looking step that's actually wrong.
Specialist knowledgeon the pageMachinery from outside any school curriculum — a felt cost, distinct from general physics depth.
Modelling judgementChoosing the right physical model and deciding what can be neglected.
EstimationHow much the problem calls for order-of-magnitude reasoning rather than an exact answer.
BranchingHow many separate cases or regimes have to be enumerated and handled one by one.

The shape of the problem

Simplicity gapHow far the real difficulty exceeds what the statement looks like — an innocent-looking sentence hiding a hard problem.
Self-containednessCan it be solved from the statement plus general reasoning alone, with nothing looked up? High means yes.
Hidden dataDoes solving it secretly require a number or fact the statement never gives you? High is a real defect.
Answer opacityWould seeing the final answer help you find the route to it? High means no — the answer alone tells you nothing.
Well-posednessIs the problem sound — a consistent idealisation, enough data, a determinate answer? Low flags a defect in the problem itself, not in the solver.

What it repays

Eleganceon the pageThe beauty of the route the solution actually takes — the short way past the integral, not the answer itself.
CreativityHow inventive the solver has to be, as opposed to simply well-drilled.
Curiosityon the pageDoes the setup itself make you want to know the answer?
Pleasureon the pageThe satisfaction of finally getting it out.
Noveltyon the pageThe opposite of formulaic. High means you haven't seen this kind of problem before.
GeneralityDoes the result reach further than what was actually asked?
InstructivenessDoes solving it teach a transferable idea you'll use again?

The five "shape of the problem" axes, and a few more from "costs"/"repays", aren't shown on the solution page — they're here and in the finder's filter panel for anyone who wants to go deeper.

Is there ever a maximum rating?

Yes. Right now exactly 50 problems reach the maximum — 3500 — because two effects compound at the very top of the scale: the model's raw scores genuinely tie across several problems, and rounding into rating buckets merges neighbouring percentiles into the same step. Here they are:

Records

How reliable this is

Savchenko marked 565 of 2,023 problems with his own ∗ in the printed book — "harder than its neighbours in that section." It's the only independent ground truth this project has, and the model is never shown it — otherwise the score would just echo the star back. The check is whether the model ranks starred problems above unstarred ones within the same section more often than chance.

0.714
model AUC
0.584
baseline (solution length alone)
11,974
within-section pairs

0.714 isn't perfect, but it's clearly above 0.584 — what solution length alone predicts. The model isn't just guessing that starred problems have longer solutions.

What this does not claim

This is a model's judgement, not a person's — see the note on every problem page. It hasn't yet been checked against how long solving actually takes real readers; that's what readers' ratings right below the AI panel start to collect. Savchenko's ∗ is a within-section relative marker — "harder than its neighbours" — never an absolute scale: chapter 14 is intrinsically harder than chapter 1, and the two are never compared directly.