Choosing a model backend for AI agents in the EU
Compare European inference providers for AI agents on tool calling, rate limits and pricing. See which EU endpoints support parallel and forced tool calls.
In this article7
European endpoints that document tool calling
A team that needs an OpenAI-compatible agent backend from a European provider can point the OpenAI SDK at Lyceum's serverless API, where every chat model takes the standard tools parameter with parallel calls, and specific models also honour a forced tool call, billed per token. The table compares seven European inference providers on what their developer documentation states; a dash means the documentation gives no value. Teams evaluating integration paths should consult LLM orchestration frameworks compared and the guide on pointing a coding assistant at your own model endpoint to verify protocol compatibility.
| Provider | Tool calling documented | tool_choice and parallel support | OpenAI-compatible API | Example tool-capable model pricing (per 1M tokens) |
|---|---|---|---|---|
| Lyceum | Yes (serverless docs) | tool_choice auto/required/named; parallel supported | Yes (chat completions route) | Billed per token |
| IONOS | Yes (AI Model Hub docs) | tool_choice auto/required/named documented; forcing not guaranteed per model | Yes (REST API) | Billed per token |
| Mistral AI | Yes (function calling guide) | tool_choice auto/any/none; parallel_tool_calls boolean supported | Yes (client and API) | – |
| Nebius | Yes (Token Factory docs) | tool_choice auto and named function documented | Yes (v1/chat/completions) | – |
| OVHcloud | Yes (AI Endpoints guide) | tool_choice auto/required/named; parallel_tool_calls false not supported | Yes (chat completions route) | – |
| Scaleway | Yes (Generative APIs docs) | tool_choice auto and named function; parallel documented (Meta models excluded) | Yes (v1 route) | – |
| STACKIT | Yes (AI Model Serving guide) | tool_calls structure documented; tool_choice options – | Yes (OpenAI-compatible client) | Billed per token |
What an agent loop needs from the endpoint
An autonomous agent loop relies on a model endpoint for execution flow rather than open-ended text completion. In an OpenAI-compatible interface, tool calling is declared via the tools array, which provides JSON schemas defining available function names, descriptions, and expected parameters[10]. The model evaluates conversation history alongside these definitions to emit a structured tool_calls payload containing specific arguments rather than prose. The calling application executes the requested operation and returns the output in a message with the tool role, allowing the loop to continue recursively.
What OpenAI-compatible does not guarantee for tools
Selecting an endpoint that exposes an OpenAI-compatible route does not guarantee that the underlying model honours structured execution instructions. OpenAI compatibility standardises the request and response wire format, but model fidelity determines whether a backend respects parameters such as tool_choice: auto, tool_choice: required, or a named function restriction[10]. While auto delegates tool invocation to model discretion, forced tool choices require the model to emit a valid call even when conversational context does not explicitly demand one. Some open-weight models fall back to plain text instead of executing a forced call, which breaks the loop silently.
Production agent loops also depend on specific streaming behaviours and predictable error parsing to manage state machines without crashing. Four core capabilities must be checked before deploying an endpoint:
- Structured tool definitions: The API must accept standard JSON Schema definitions inside the tools parameter and return distinct tool_calls objects[10].
- Execution modes: The backend must explicitly document support for tool_choice auto, required, and named function forcing[10].
- Parallel tool execution: The model must correctly emit multiple tool invocations in a single completion turn to minimise multi-step round-trip latency[4].
- Deterministic error handling: Error payloads must supply consistent OpenAI-shaped structures with standard HTTP status codes (such as 429, 502, 503, and 504) to trigger automated client-side backoff.
When streaming is enabled via stream: true, tool call arguments arrive as incremental string fragments across sequential Server-Sent Events chunks[6]. An orchestrator must reconstruct these deltas in memory before invoking the target function, making stable schema parsing and connection stability essential for reliability of AI agents in production.
Rate limits, retries and the cost of a failed call
Agent workloads exhibit a fundamentally different cost and traffic profile than standard chat interfaces. A single agent task frequently triggers sequential inference steps, re-transmitting the entire conversational history along with accumulating tool execution results on every invocation. As context grows during multi-turn reasoning, input token processing dominates the overall API bill.
Consider an agent loop running on Lyceum with a tool-capable model. If the loop requires many steps, uncached execution consumes substantial input tokens compared to the generated output. With automatic prompt caching applied to the re-sent prefix, input costs fall, reducing total execution cost. However, if a model fails to emit a required tool call and returns unparseable text, the entire context history must be re-run with backoff, significantly increasing step expenditure.
Throughput constraints represent the secondary operational bottleneck for multi-agent concurrency. Providers impose distinct quota metrics that dictate loop pacing:
- Request volume limits: OVHcloud AI Endpoints enforces a rate limit of 400 requests per minute per Public Cloud project and per model for authenticated access[6][11].
- Token and query quotas: Scaleway Generative APIs restricts serverless tiers based on queries per minute, total tokens per minute, and concurrent request ceilings[12].
- Backoff handling: Receiving an HTTP 429 (Too Many Requests) response pauses execution across all active agent threads, requiring exponential jitter backoff to avoid cascading task failures.
Choosing a backend for an agent workload
Match the backend to the agent's shape rather than to a platform brand. If the loop depends on forced calls (tool_choice required or a named function), test the specific model, not the platform: an endpoint that accepts the parameter is not the same as a model that honours it. If the agent's context grows across many steps, weight input pricing and any cached-input rate more heavily than output pricing, since the re-sent prefix dominates the bill. If the workload is many short, rapid calls, check the published requests-per-minute or queries-per-minute ceilings before token quotas, because a 429 pauses every active thread at once.
Frequently asked questions
Which EU LLM APIs support tool calling?
Providers including Lyceum, IONOS, Mistral AI, Nebius, OVHcloud, Scaleway and STACKIT offer OpenAI-compatible endpoints, though their documentation on parallel calls and forced tool choice varies.
What is an OpenAI-compatible tool calling API?
An endpoint that accepts the standard tools parameter and tool_choice argument, and returns tool calls in the expected JSON format.
Do open-weight models call tools reliably?
Not uniformly. Many honour auto tool calls, but support for forced calls varies by model, so test tool_choice required on the exact model before relying on it.
What rate limits do agent workloads hit first?
Agents make numerous rapid queries, quickly hitting queries-per-minute caps before token limits. For instance, OVHcloud limits AI Endpoints to 400 requests per minute per project.
How do I force a model to call a tool?
By setting tool_choice to 'required' or providing a named function object. The endpoint's underlying model must explicitly support forced calls to avoid returning a plain-text response.
Sources
Rankings and scores on Optinest cannot be bought — no commercial relationship changes how a provider is assessed. How we score
Related reading
All Agents →Who pays for AI tokens: Cost structures and provider pricing
Understand who pays for AI tokens and how rate cards work. Compare European serverless inference providers to model your exact generative AI costs.

European serverless inference providers: region, price and DPA
Compare European serverless inference providers based on published disclosures for model processing regions, pricing units, and transparent DPA terms.

Serverless vs dedicated inference cost: where the curves cross
Serverless vs dedicated inference cost as a model with named variables, and which European providers sell each mode. Read 26 October 2026.
