AgentsReliability

Choosing a model backend for AI agents in the EU

Compare European inference providers for AI agents on tool calling, rate limits and pricing. See which EU endpoints support parallel and forced tool calls.

, AI Researcher5 min readUpdated 1 October 2026
In this article7
  1. European endpoints that document tool calling
  2. What an agent loop needs from the endpoint
  3. What OpenAI-compatible does not guarantee for tools
  4. Rate limits, retries and the cost of a failed call
  5. Choosing a backend for an agent workload
  6. Frequently asked questions
  7. Sources

European endpoints that document tool calling

A team that needs an OpenAI-compatible agent backend from a European provider can point the OpenAI SDK at Lyceum's serverless API, where every chat model takes the standard tools parameter with parallel calls, and specific models also honour a forced tool call, billed per token. The table compares seven European inference providers on what their developer documentation states; a dash means the documentation gives no value. Teams evaluating integration paths should consult LLM orchestration frameworks compared and the guide on pointing a coding assistant at your own model endpoint to verify protocol compatibility.

ProviderTool calling documentedtool_choice and parallel supportOpenAI-compatible APIExample tool-capable model pricing (per 1M tokens)
LyceumYes (serverless docs)tool_choice auto/required/named; parallel supportedYes (chat completions route)Billed per token
IONOSYes (AI Model Hub docs)tool_choice auto/required/named documented; forcing not guaranteed per modelYes (REST API)Billed per token
Mistral AIYes (function calling guide)tool_choice auto/any/none; parallel_tool_calls boolean supportedYes (client and API)–
NebiusYes (Token Factory docs)tool_choice auto and named function documentedYes (v1/chat/completions)–
OVHcloudYes (AI Endpoints guide)tool_choice auto/required/named; parallel_tool_calls false not supportedYes (chat completions route)–
ScalewayYes (Generative APIs docs)tool_choice auto and named function; parallel documented (Meta models excluded)Yes (v1 route)–
STACKITYes (AI Model Serving guide)tool_calls structure documented; tool_choice options –Yes (OpenAI-compatible client)Billed per token

What an agent loop needs from the endpoint

An autonomous agent loop relies on a model endpoint for execution flow rather than open-ended text completion. In an OpenAI-compatible interface, tool calling is declared via the tools array, which provides JSON schemas defining available function names, descriptions, and expected parameters[10]. The model evaluates conversation history alongside these definitions to emit a structured tool_calls payload containing specific arguments rather than prose. The calling application executes the requested operation and returns the output in a message with the tool role, allowing the loop to continue recursively.

What OpenAI-compatible does not guarantee for tools

Selecting an endpoint that exposes an OpenAI-compatible route does not guarantee that the underlying model honours structured execution instructions. OpenAI compatibility standardises the request and response wire format, but model fidelity determines whether a backend respects parameters such as tool_choice: auto, tool_choice: required, or a named function restriction[10]. While auto delegates tool invocation to model discretion, forced tool choices require the model to emit a valid call even when conversational context does not explicitly demand one. Some open-weight models fall back to plain text instead of executing a forced call, which breaks the loop silently.

Production agent loops also depend on specific streaming behaviours and predictable error parsing to manage state machines without crashing. Four core capabilities must be checked before deploying an endpoint:

  • Structured tool definitions: The API must accept standard JSON Schema definitions inside the tools parameter and return distinct tool_calls objects[10].
  • Execution modes: The backend must explicitly document support for tool_choice auto, required, and named function forcing[10].
  • Parallel tool execution: The model must correctly emit multiple tool invocations in a single completion turn to minimise multi-step round-trip latency[4].
  • Deterministic error handling: Error payloads must supply consistent OpenAI-shaped structures with standard HTTP status codes (such as 429, 502, 503, and 504) to trigger automated client-side backoff.

When streaming is enabled via stream: true, tool call arguments arrive as incremental string fragments across sequential Server-Sent Events chunks[6]. An orchestrator must reconstruct these deltas in memory before invoking the target function, making stable schema parsing and connection stability essential for reliability of AI agents in production.

Rate limits, retries and the cost of a failed call

Agent workloads exhibit a fundamentally different cost and traffic profile than standard chat interfaces. A single agent task frequently triggers sequential inference steps, re-transmitting the entire conversational history along with accumulating tool execution results on every invocation. As context grows during multi-turn reasoning, input token processing dominates the overall API bill.

Consider an agent loop running on Lyceum with a tool-capable model. If the loop requires many steps, uncached execution consumes substantial input tokens compared to the generated output. With automatic prompt caching applied to the re-sent prefix, input costs fall, reducing total execution cost. However, if a model fails to emit a required tool call and returns unparseable text, the entire context history must be re-run with backoff, significantly increasing step expenditure.

Throughput constraints represent the secondary operational bottleneck for multi-agent concurrency. Providers impose distinct quota metrics that dictate loop pacing:

  • Request volume limits: OVHcloud AI Endpoints enforces a rate limit of 400 requests per minute per Public Cloud project and per model for authenticated access[6][11].
  • Token and query quotas: Scaleway Generative APIs restricts serverless tiers based on queries per minute, total tokens per minute, and concurrent request ceilings[12].
  • Backoff handling: Receiving an HTTP 429 (Too Many Requests) response pauses execution across all active agent threads, requiring exponential jitter backoff to avoid cascading task failures.

Choosing a backend for an agent workload

Match the backend to the agent's shape rather than to a platform brand. If the loop depends on forced calls (tool_choice required or a named function), test the specific model, not the platform: an endpoint that accepts the parameter is not the same as a model that honours it. If the agent's context grows across many steps, weight input pricing and any cached-input rate more heavily than output pricing, since the re-sent prefix dominates the bill. If the workload is many short, rapid calls, check the published requests-per-minute or queries-per-minute ceilings before token quotas, because a 429 pauses every active thread at once.

Frequently asked questions

Which EU LLM APIs support tool calling?

Providers including Lyceum, IONOS, Mistral AI, Nebius, OVHcloud, Scaleway and STACKIT offer OpenAI-compatible endpoints, though their documentation on parallel calls and forced tool choice varies.

What is an OpenAI-compatible tool calling API?

An endpoint that accepts the standard tools parameter and tool_choice argument, and returns tool calls in the expected JSON format.

Do open-weight models call tools reliably?

Not uniformly. Many honour auto tool calls, but support for forced calls varies by model, so test tool_choice required on the exact model before relying on it.

What rate limits do agent workloads hit first?

Agents make numerous rapid queries, quickly hitting queries-per-minute caps before token limits. For instance, OVHcloud limits AI Endpoints to 400 requests per minute per project.

How do I force a model to call a tool?

By setting tool_choice to 'required' or providing a named function object. The endpoint's underlying model must explicitly support forced calls to avoid returning a plain-text response.

Sources

  1. lyceum.technology
  2. docs.ionos.com
  3. cloud.ionos.com
  4. docs.mistral.ai
  5. docs.tokenfactory.nebius.com
  6. help.ovhcloud.com
  7. scaleway.com
  8. docs.stackit.cloud
  9. stackit.com
  10. platform.openai.com
  11. ovhcloud.com
  12. scaleway.com

Rankings and scores on Optinest cannot be bought — no commercial relationship changes how a provider is assessed. How we score

Related reading

All Agents →