FineBooks published a leaderboard comparing 14 open OCR models on 2,165 historical pages checked by experts. The post was published on 10 August 2026 at 11:57:17.658 UTC. It also provides a public ground-truth dataset, finebooks/bhl-impact-gt, and an evaluation harness, bhl-ocr-eval.
A benchmark built for old pages
The release gives tool choosers a common input set instead of a general reputation contest. The pages come from historical books, so the result is most useful for that document class. A model that reads engraved plates or old typography well may still need a separate test on modern forms, receipts, tables or technical PDFs.
Reported accuracy and cost
The FineBooks material reports these figures for three examples:
| Model | Reported reading accuracy | Reported cost per 1,000 pages |
|---|---|---|
| dots.mocr 3B | 97.6% | $1.94 |
| OvisOCR2 0.9B | 96.9% | $0.46 |
| PaddleOCR-VL-1.6 1B | 96.1% | $0.34 |
These are reported leaderboard values and Hugging Face Jobs estimates. They are not universal prices or production guarantees. The public inputs and code are more important than the ranking alone because they make a task-specific rerun possible.
Where the leaderboard stops
Reading accuracy does not establish that a model produces ALTO XML, word-level coordinates or another format required by a downstream workflow. Cost changes with hardware, page mix, volume and operating conditions. The supplied material does not establish an independent full rerun, so the responsible label is FineBooks-reported performance on a historical-page evaluation.
Turn the table into your own test
- Pin the evaluation harness revision and model revisions.
- Run the same models on the public ground-truth dataset.
- Record character error rate, recall, output format, runtime and actual cost.
- Run a separate fixed sample of modern documents.
That split distinguishes reproduction of the published benchmark from the buying question: which OCR system fits your pages, output contract and operating budget? Watch for independent reruns, local-hardware measurements and evidence that the reported performance transfers beyond historical scans.
Sources: FineBooks leaderboard, ground-truth dataset and evaluation harness.