Inference Providers documentation

Routing and Providers

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Routing and Providers

Every request goes through the Hugging Face router, which handles authentication and billing. For language models, OpenRouter picks a provider serving the model and retries on another one if it fails.

Default routing

Without a suffix, chat requests use OpenRouter’s default routing. For each request, it compares the cost of a good answer from each provider serving the model:

  • Effective price: what the provider actually charges, with its real cache hit rate and the model’s usual mix of input and output tokens, not just the list price.
  • Speed: throughput and time to first token, so a faster provider can win the request even at a higher price.
  • Reliability: how often the provider fails, returns malformed tool calls, or turns requests away because it’s out of capacity.

The measures are updated in real time: when a provider slows down or fails, requests move away from it until it recovers.

Pick a policy with a suffix

Append a suffix to the model id. It works in every client, including tools that only let you set a model name.

Model idRoutes to
deepseek-ai/DeepSeek-V4.1-FlashDefault routing: the best mix of price, speed and reliability
deepseek-ai/DeepSeek-V4.1-Flash:fastestHighest throughput, in tokens per second
deepseek-ai/DeepSeek-V4.1-Flash:cheapestLowest price
deepseek-ai/DeepSeek-V4.1-Flash:novitaOnly Novita, or an error if it doesn’t serve the model. Each provider’s suffix is listed in Providers

Turn providers on and off

In your Inference Providers settings, you can turn off the providers you don’t want. The router never sends your requests to them, and pinning one with a suffix returns an error. Requests billed to an organization follow the organization’s settings instead of yours.

Failover

If a provider fails, OpenRouter retries the request on another provider serving the model. A pinned provider (:novita) is never replaced.

Update on GitHub