Why a 350M model beats a frontier model for document extraction.

Accuracy on a narrow task is a function of training data, not parameter count. That single sentence decides most of the architecture for private AI — and it runs against the instinct to reach for the largest model available.

We tested both options on the same corpus: 40,000 supplier invoices across four languages, pulled from three years of a client archive. One pipeline called a frontier model through its API. The other ran a 350M-parameter REX42 model fine-tuned on 6,000 labelled documents from that same archive, on a single CPU-only server.

What we measured

Field-level accuracy on twelve extracted fields — supplier, VAT number, invoice number, currency, line totals, dates. The frontier model scored 94.1% out of the box with prompt engineering alone. The fine-tuned small model scored 97.8% after one training run, and 98.6% after we corrected the 400 documents it got wrong and trained again.

The gap comes from format, not intelligence. Frontier models are extraordinary generalists that have never seen your supplier layouts. A small model trained on your archive has seen almost nothing else.

What it costs to run

The API pipeline cost roughly USD 1,900 a month at 11,000 invoices, and every document left the building. The on-prem model runs on hardware the client already owned, with a perpetual licence and no per-document cost. At that volume it paid for itself inside the first quarter.

Where the small model loses

It has no general knowledge. Ask it to summarise a contract clause it was never trained on and it will fail in ways a frontier model would not. That is the trade we make deliberately: specialists for the repeatable, sensitive work; public models for open-ended tasks on non-sensitive data.

How to decide

If the task is narrow, repeats daily, and touches data you cannot send outside your network, a small private model is almost always the better engineering choice. If the task is open-ended and the data is public, use the biggest model you can afford. Most businesses need both, routed by sensitivity.