InferenceToken economics

Batch OCR on serverless GPUs: cost and throughput

Price a 10,000-page batch OCR run two ways, per token and per GPU-hour, from published European rate cards, with the crossover shown.

, AI Researcher7 min read
In this article7
  1. Why batch is the opposite shape to a live endpoint
  2. Sizing a run: pages, tokens and throughput
  3. The same batch priced per token and per hour
  4. Where the time actually goes
  5. Choosing a billing route for a batch job
  6. Frequently asked questions
  7. Sources
A queue of scanned document pages flowing into a single GPU server for batch OCR, with an hourly cost meter on one side and a per-token price tag on the other
A queue of scanned document pages flowing into a single GPU server for batch OCR, with an hourly cost meter on one side and a per-token price tag on the other

Why batch is the opposite shape to a live endpoint

A one-off batch job has no latency budget. Nobody is waiting on a single page, so the job can saturate a GPU for as long as it needs and pay only for the seconds it runs. That inverts the economics of a live endpoint, which is priced per request and sized to stay responsive between them. For batch, the useful unit is pages per minute of measured throughput, and the bill is GPU time.

Sizing a run: pages, tokens and throughput

A measured anchor shows the shape. GigaGPU's benchmark of PaddleOCR v4 on an RTX 5090-class GPU processed a full pipeline of mixed A4 pages end-to-end at 285 pages per minute, with a stated cost of $0.11 per 1,000 pages on a $1.80/hr server. A 10,000-page run at that rate takes about 35 minutes of GPU time. Throughput varies sharply by GPU and model: the same benchmark family lists the RTX 3090 at 145 pages per minute and DocTR on an RTX 5090 at 210, so the sizing has to come from the reader's own model and card.

The per-token route needs a figure providers do not publish: tokens per page. A token is the unit of text a model processes, roughly a word or character fragment. Vision-language OCR models emit output in tokens, but no rate card states pages-to-tokens for a given document set, so the per-token route cannot be priced from published rates alone. The per-hour route can: it is measured time multiplied by an hourly rate.

The same batch priced per token and per hour

A team with 10,000 documents brings its own OCR model and runs it on Lyceum's serverless GPU product, which takes a Python script or a Docker image and bills per second[1], at L40S $1.69/hr and H100 $3.59/hr. The table below shows what each provider's own page states for this question.

ProviderGPU and hourly rate (billing mode)Per-token or per-page OCR priceRetrieved
LyceumL40S $1.69/hr; H100 $3.59/hr (per-second serverless GPU)– (no OCR or vision model in the per-token catalogue)2026-09-22
ExoscaleA40 €1.0453/hr; A30 €1.2265/hr; RTX Pro 6000 €2.1528/hr (per-hour, excl. tax)–2026-10-01
HetznerGEX45, RTX PRO 4000: €0.3429/hr or €214/month; GEX131, RTX PRO 6000: from €1.9215/hr (dedicated server, plus one-time setup fee)–2026-10-01
IONOSH200 €3.00/hr (Cloud GPU VM, net)LightOnOCR 2: $0.17 per 1M input, $0.33 per 1M output tokens (per-token)2026-10-01
NebiusHGX H100 $4.50/hr (on demand from 1 Oct 2026); L40S from $1.55/hr (on demand); preemptible H100 from $0.79/hrMiniCPM-V 4.5 (vision, eu-north1): $0.658 per 1M input, $1.11 per 1M output tokens2026-10-01
OVHcloudL4-1-GPU $0.91/hr; L40S-1-GPU $1.69/hr (per-hour, AI Training/Deploy rate, excl. VAT)Qwen2.5-VL-72B (vision): €0.91 per 1M input and per 1M output tokens2026-10-01
ScalewayL4-1-24G €0.79/hr; L40S from €1.47/hr; H100-1-80G €2.8665/hr (per-hour, PAR-2, before tax)pixtral-12b (vision): €0.20 per 1M input and per 1M output tokens2026-10-01
STACKIT1× L40S €1.7429/hr; 1× H100 €8.0413/hr (per-hour, net list price)Qwen3-VL 235B (vision): €1.50 per 1M input, €1.75 per 1M output tokens2026-10-01

A dash means the provider's own page lists no value for that cell. Pricing a 10,000-page run both ways: at the benchmarked 285 pages per minute[9], the job takes about 35 minutes, or about $0.99 of L40S GPU time at the serverless rate of $1.69/hr[10]. The per-page route through Mistral OCR 4.1 lists $4 per 1,000 pages standard, with batch inference at half price, so the same 10,000 pages cost $40 standard and $20 batched[11]. Among the per-hour options, Scaleway lists an L4 instance at €0.79/hr[7], a smaller GPU priced in euros, not the same card as the L40S's $1.69/hr in dollars, so the two figures are not directly comparable; Hetzner[3] and Exoscale[2] also publish hourly GPU rates: Hetzner's RTX PRO 4000 dedicated server works out at €0.34/hr on a monthly contract with a setup fee, and an Exoscale A40 instance costs €1.05/hr; a wider comparison of AI compute rental prices covers the hourly-rate market in more depth.

The crossover is arithmetic, not assertion. At the benchmarked 285 pages per minute[9], a $1.69/hr L40S[10] costs about $0.0001 per page, well under the batch API's per-page rate of half the $4 per 1,000 pages Mistral lists for OCR 4.1[11]. The break-even sits near 14 pages per minute at $1.69/hr: below that, per-hour compute costs more per page than the batch API; above it, less. A slower model on a pricier GPU moves toward the line; a fast pipeline model at volume moves far under it. Fast pipeline models at volume favour per-hour compute; slow or accuracy-heavy models at small volume favour the per-page API.

Where the time actually goes

Not all of the run is the model. The measured gap between a pipeline's peak pages per minute and its effective rate goes to page loading, result writing, queueing and retries. GigaGPU's pipeline benchmark shows the pattern: adding layout analysis and an LLM stage to PaddleOCR on an RTX 5090 drops throughput from 285 pages per minute of OCR[9] to a much lower effective rate for the full pipeline. Extraction to structured output adds CPU-side work, which is billed separately or runs on cheaper instances than the GPU. The same factors that limit GPU throughput in a live deployment apply to a batch run, and memory bandwidth sets the ceiling on token rate for vision-language models.

Choosing a billing route for a batch job

  • High volume, fast pipeline model: per-hour compute. The measured time is short and the per-page cost falls below any per-page API rate.
  • Low volume, or a slow accuracy-heavy model: per-token or per-page. The GPU sits idle between pages and the API absorbs the overhead.
  • Job that checkpoints: preemptible rates. Nebius lists preemptible GPU instances below its on-demand rates[5], a discount for the risk of interruption.

The decision reduces to two measured numbers: the reader's own pages-per-minute on their model and card, and the published hourly rate that divides into it. Everything else in the bill is pipeline overhead, and it scales with the pipeline, not the model.

Frequently asked questions

What does OCR on 10,000 documents cost?

It depends on the billing route. At GigaGPU's benchmarked 285 pages per minute, 10,000 pages take about 35 minutes of GPU time; at Lyceum's published serverless L40S rate of $1.69 per hour that is roughly $0.99 of GPU time. Through Mistral's OCR 4.1 API, which lists $4 per 1,000 pages standard and half price with batch inference, the same pages cost $40 standard and $20 batched.

Is batch cheaper per token?

On the per-page route, yes: Mistral's price list shows OCR 4.1 at $4 per 1,000 pages standard and half price with batch inference. On GPU compute, batch is cheaper per page because the GPU stays saturated instead of waiting for requests.

Which models read documents?

Vision-language OCR models such as LightOnOCR 2, offered per token on IONOS's AI Model Hub at $0.17 per 1M input tokens, and open-source engines such as PaddleOCR v4, which run on any CUDA GPU under per-hour billing. This article prices the routes; it does not compare OCR quality.

Per-hour or per-token for a batch job?

Per-hour wins when a fast pipeline model keeps cost per page below the API price: at the benchmarked 285 pages per minute, a $1.69 per hour GPU costs about $0.0001 per page, far under the batch API's per-page rate. Per-token or per-page wins at low volume, or with slow accuracy-heavy models, where the GPU sits idle between pages.

Sources

  1. lyceum.technology
  2. exoscale.com
  3. hetzner.com
  4. cloud.ionos.com
  5. nebius.com
  6. ovhcloud.com
  7. scaleway.com
  8. stackit.com
  9. gigagpu.com
  10. lyceum.technology
  11. mistral.ai

Rankings and scores on Optinest cannot be bought — no commercial relationship changes how a provider is assessed. How we score

Related reading

All Inference →