Provider Selection

Route requests to preferred providers with explicit policies

TheRouter.ai load balances across available providers by default. Add aprovider object in your request body to override routing behavior.

provider-fields.json

{
  "provider": {
    "order": ["openai", "anthropic"],
    "allow_fallbacks": true,
    "only": ["openai", "anthropic"],
    "ignore": ["together"],
    "quantizations": ["int8", "fp8"],
    "sort": "latency"
  }
}

Ordering providers

Use provider.order to force a strict provider priority list. TheRouter.ai tries each provider in order, then continues to other providers whenallow_fallbacks is enabled.

TypeScript

const response = await fetch('https://api.therouter.ai/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <THEROUTER_API_KEY>',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'mistralai/mixtral-8x7b-instruct',
    messages: [{ role: 'user', content: 'Hello' }],
    provider: {
      order: ['openai', 'together'],
      allow_fallbacks: true,
    },
  }),
});

Sorting strategy

Set provider.sort to price, throughput, or latency to disable weighted load balancing and pick providers in deterministic order.

Simple

{
  "model": "meta-llama/llama-3.3-70b-instruct",
  "provider": {
    "sort": "throughput"
  },
  "messages": [{ "role": "user", "content": "Generate 5 taglines." }]
}

Allow and ignore lists

Restrict the provider pool with only and explicitly remove providers with ignore.

json

{
  "model": "deepseek/deepseek-r1",
  "provider": {
    "only": ["deepinfra", "novita"],
    "ignore": ["deepinfra/turbo"],
    "allow_fallbacks": false
  },
  "messages": [{ "role": "user", "content": "Solve this optimization problem." }]
}

Quantization filters

Use provider.quantizations to route only to endpoints with specific quantization levels.

json

{
  "model": "meta-llama/llama-3.3-70b-instruct",
  "provider": {
    "quantizations": ["int4", "int8"],
    "sort": "price"
  },
  "messages": [{ "role": "user", "content": "Summarize this code review." }]
}

Related shortcut

Appending :nitro to a model slug is equivalent to settingprovider.sort to throughput. See theNitro variant guide.