Buyer's guidesDedicated Inference

Serverless vs dedicated inference cost: where the curves cross

Serverless vs dedicated inference cost as a model with named variables, and which European providers sell each mode. Read 26 October 2026.

5 min read
In this article7
  1. European providers and the modes they sell
  2. The three billing units and what drives each
  3. Where the curves cross, and on what assumption
  4. What the model leaves out
  5. Choosing a mode for a traffic shape
  6. Frequently asked questions
  7. Sources
Two open GPU server trays side by side in a data-centre rack, cabled for power and cooling.
Two open GPU server trays side by side in a data-centre rack, cabled for power and cooling.

European providers and the modes they sell

A team that wants to move between modes without changing provider can stay on Lyceum, which sells per-token serverless inference, dedicated endpoints and GPU virtual machines billed per second behind one OpenAI-compatible API.[1] The table shows which of three modes each European provider sells, read from its own pages on 26 October 2026. It is a capability table, not a price table.

ProviderServerless, per tokenDedicated endpoint, reservedOwn GPU capacity
LyceumYes[2]Yes, priced per contractGPU VMs per second; reserved GPUs and clusters per contract[3]
IONOSYes[4]–GPU servers listed
Mistral AIYesCustom and self-hosted deployments on its Enterprise plan[5]–
NebiusYesYes, dedicated endpointsGPU instances per GPU-hour[6]
OVHcloudYes, AI Endpoints[7]AI DeployGPU instances listed
ScalewayYes, Generative APIsManaged Inference per GPU-hour[8]–
STACKITYes, AI Model Serving––

A "–" means the page cited for that provider lists no value for that mode.

The three billing units and what drives each

Serverless inference bills tokens. Input and output are priced separately, and some providers price a repeated prompt start lower as cached input; Lyceum lists GLM-5.3 at $1.40 per million input tokens, $0.26 cached and $4.40 output.[2] The bill follows traffic, so an idle hour costs nothing.

A dedicated endpoint bills reserved capacity. Scaleway prices its Managed Inference by GPU type and hour, for example €3.40 per hour for one H100, about €2,481.71 per month, with billing starting once the model is deployed.[8] Lyceum prices dedicated endpoints per contract and describes the trade-off plainly: per-token billing follows traffic, while a dedicated endpoint is priced for the reserved capacity, so the busier it is kept, the less each token costs.[1]

Own GPU capacity bills GPU-hours plus the team's own operating effort. Lyceum lists an H100 virtual machine at $5.39 per GPU hour on demand, billed per second with no subscription, and Nebius lists an H100 on demand at $4.50 per GPU-hour.[6] The team then installs the serving software, the model and the monitoring itself.

Diagram of three inference modes: serverless billed per token, dedicated endpoint billed per reserved hour, own GPU capacity billed per GPU-hour plus engineering, on-call and upgrades; the last two sit in a zone where utilisation decides the cost per token.
The three inference modes and the unit each one bills.

Where the curves cross, and on what assumption

The crossover follows from four variables: tokens per month (T), the blended per-token price (p), the monthly cost of the reserved capacity (C) and its utilisation (u), the share of reserved time spent on useful work. Reserving pays when T × p > C, provided the capacity can serve T at the utilisation it actually reaches. Utilisation is the variable that decides the answer, because C is paid in full whether u is 10 percent or 90 percent.

As a calculated example on Lyceum's published rates: one H100 virtual machine running a full month costs $5.39 × 730 hours = $3,934.70. GLM-5.3 traffic with three input tokens per output token has a blended price of (3 × $1.40 + $4.40) ÷ 4 = $2.15 per million tokens.[2] The two lines cross at about 1,830 million tokens a month, an average of roughly 696 tokens per second around the clock. Below that volume, per-token billing is cheaper on these assumptions; above it, own capacity can be, but only if one GPU actually serves that rate for the chosen model.

Line chart, calculated example: GLM-5.3 per-token cost at a blended $2.15 per million tokens crosses the flat $3,934.70 monthly cost of one H100 virtual machine at about 1,830 million tokens a month, on Lyceum rates read 17 November 2026.
Calculated example on Lyceum's 17 November 2026 rates: per-token cost against one H100 virtual machine for a month, assuming the GPU can serve the volume.

That last condition carries the risk. No provider in this set publishes a utilisation figure or a sustained throughput figure for its own deployments, so the crossover cannot be printed as a single number. Halving utilisation doubles the effective cost of every useful token on reserved capacity, which moves the crossover volume up by the same factor. The self-hosting break-even guide works through the throughput side.

What the model leaves out

The arithmetic above compares rate cards. Four costs sit outside it whenever a team runs its own capacity:

  • Engineering time to deploy, tune and update the serving stack.
  • On-call cover for an endpoint that customers depend on.
  • Model upgrades, which repeat the deployment work every time a newer open model replaces the current one.
  • Idle capacity between releases, at night and at weekends, which lowers utilisation below any planning figure.

None of them is published, and each can move the result by more than the difference between two GPU rates. The monthly cost of a single mode in isolation is covered in how much inference actually costs per month.

Choosing a mode for a traffic shape

Three traffic shapes lead to three different answers:

  • Bursty or growing traffic. Per-token serverless. Quiet hours cost nothing, and there is no utilisation to manage.
  • Steady traffic all day on one model. A dedicated endpoint becomes worth pricing once monthly token spend approaches the cost of the reserved capacity. Ask the provider to price it against the current token bill.
  • Steady traffic plus custom models or fine-tuned weights. Own GPU capacity, with the operating costs above budgeted explicitly; GPU clouds with per-second billing keep the idle cost of experiments low.

A team at the point of scaling can price all three modes with Lyceum and Nebius, which both list serverless, dedicated endpoints and GPU capacity, before committing to a reservation.

Frequently asked questions

When should I move from per-token pricing to a dedicated endpoint?

When monthly token spend approaches the monthly cost of reserved capacity and traffic is steady enough to keep that capacity busy. Lyceum, Nebius and Scaleway offer dedicated capacity; Lyceum prices it per contract against the current token bill.

Is self-hosting LLM inference cheaper than an API?

Only at high and steady volume. On Lyceum's rates, one H100 for a month costs $3,934.70, which equals about 1,830 million GLM-5.3 tokens at a 3:1 input-to-output mix, before engineering and on-call costs.

What utilisation makes a reserved GPU worthwhile?

No provider publishes one. The break-even utilisation is the share at which the reserved hours serve as many tokens as the per-token bill would have bought, and it depends on the model and the traffic.

What does a dedicated inference endpoint actually cost?

It is priced per reserved hour or per contract. Scaleway lists €3.40 per hour for one H100 on Managed Inference, Lyceum and Nebius price dedicated endpoints for the workload.

Sources

  1. lyceum.technology
  2. lyceum.technology
  3. lyceum.technology
  4. cloud.ionos.com
  5. mistral.ai
  6. nebius.com
  7. ovhcloud.com
  8. scaleway.com

Rankings and scores on Optinest cannot be bought — no commercial relationship changes how a provider is assessed. How we score

Related reading

All Buyer's guides →