> ## Documentation Index > Fetch the complete documentation index at: https://docs.nika.sh/llms.txt > Use this file to discover all available pages before exploring further. # Providers > 14 canonical providers, local-first (plus a wider engine catalog) behind one unified InferRequest. Switch provider = change one field. export const CANON = { schemaVersion: 1, verbs: 4, verbNames: ["infer", "exec", "invoke", "agent"], namespaces: 5, namespaceNames: ["vars", "with", "tasks", "env", "secrets"], builtins: 23, builtinNames: ["assert", "compose", "convert", "date", "done", "edit", "emit", "fetch", "glob", "grep", "hash", "inspect", "jq", "json_diff", "json_merge_patch", "log", "notify", "prompt", "read", "uuid", "validate", "wait", "write"], providers: 14, providersCloud: 8, providersLocal: 5, providersTest: 1, providerIdsCloud: ["mistral", "anthropic", "openai", "gemini", "deepseek", "xai", "groq", "openrouter"], providerIdsLocal: ["ollama", "lmstudio", "llamacpp", "localai", "vllm"], providerIdsTest: ["mock"], extractModes: 9, extractModeNames: ["article", "feed", "jq", "links", "markdown", "metadata", "selector", "sitemap", "text"], errorNamespaces: 14, errorNamespaceNames: ["NIKA-AGENT", "NIKA-BUILTIN", "NIKA-CANCEL", "NIKA-DAG", "NIKA-EXEC", "NIKA-IMPL", "NIKA-INFER", "NIKA-INVOKE", "NIKA-MCP", "NIKA-PARSE", "NIKA-PROVIDER", "NIKA-SEC", "NIKA-TIMEOUT", "NIKA-VAR"], errorCategories: 12, errorCodes: 50, pillars: 5 }; export const STATUS = { head: "95962d5cd", branch: "main", version: "0.91.0", cratesWorkspace: 39, cratesAdmitted: 39, cratesTarget: "42", wipCrates: [], libTests: 2989, clippyWarnings: 0, adrs: 62, adrsAccepted: 42, adrsProposed: 18, providers: 32, capabilityRules: 49, hygieneVectors: 38, hygieneGreen: 28, hygieneYellow: 3, hygieneRed: 0, lastUpdated: "2026-06-25" }; Two provider numbers exist, on purpose: the **language spec** locks **{CANON.providers} canonical providers** (what a conformant engine MUST ship · see the [spec catalog](https://github.com/supernovae-st/nika-spec/blob/main/stdlib/providers-v0.1.md)); the **reference engine's catalog** goes wider. Nika ships with **{STATUS.providers} LLM providers** in its catalog, orchestrated behind one unified `InferRequest`. The `infer:` verb takes a `model: /` string, looks up capability rules ([{STATUS.capabilityRules} rules](/reference/capabilities)), and dispatches through the right API dialect. **Full inventory with env vars and default models:** [Providers catalog](/reference/providers-catalog). This page explains the design: why a single `InferRequest` works across {STATUS.providers} providers. ## One `InferRequest` for all providers ```rust theme={"system"} #[non_exhaustive] pub struct InferRequest { pub provider: ProviderId, // e.g., "anthropic", "openai", "moonshot" pub model: ModelId, // e.g., "claude-sonnet-4-6" pub messages: Vec, pub response_format: Option, pub tools: Vec, pub temperature: Option, pub top_p: Option, pub max_tokens: Option, pub stream: bool, // reserved for future minors: memory, budget, baggage, trust_level, trace_id } ``` Every L1 provider crate (`nika-provider-anthropic`, `nika-provider-openai`, `nika-provider-gemini`, …) implements the single `Provider` trait from `nika-kernel`: ```rust theme={"system"} #[trait_variant::make(ProviderDyn: Send)] pub trait Provider: sealed::Sealed + Send + Sync + 'static { async fn infer(&self, req: &InferRequest) -> Result; async fn stream(&self, req: &InferRequest) -> Result; fn capabilities(&self, model: &str) -> ModelCapabilities; fn provider_id(&self) -> &'static ProviderId; } ``` Switching provider is **one line** in YAML: ```yaml theme={"system"} tasks: - id: first infer: model: ollama/llama3.2:3b # / · swap to mistral/… · anthropic/… · gemini/… prompt: "Hello." ``` ## API dialects (7) Providers speak 7 distinct dialects. One provider crate per dialect + an OpenAI-compatible family that covers \~22 providers. | Dialect | Providers | Notes | | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | | `openai-chat` | openai, mistral, groq, deepseek, xai, openrouter, together, fireworks, cerebras, sambanova, perplexity, moonshot, qwen, minimax, zhipu, nvidia-nim, deepinfra, replicate, hyperbolic, writer, databricks, azure, cloudflare | 23 providers share OpenAI's `/v1/chat/completions` shape with per-provider base URL + auth header | | `anthropic` | anthropic | Native Messages API, tool use as schema, extended thinking | | `gemini` | gemini, vertex | Native `generateContent` API, safety settings, structured output via `response_schema` | | `cohere` | cohere | Native Command API | | `ai21` | ai21 | Native Jamba API | | `bedrock` | bedrock | AWS SigV4, model-prefix routing (`anthropic.claude-...`) | | `voyage` | voyage | Embeddings-only | **Local runner:** the `native` provider uses `mistral.rs` to load GGUF models on CPU / Metal / CUDA without any HTTP round-trip. Feature-gated in `nika-provider-native`, off by default. ## Capability-aware routing Every `InferRequest` is routed through **capability rules** before dispatch. The rules decide: * What **modalities** the model accepts (text / image / audio / pdf / …). * Whether **tool calling** + **parallel tool calls** are supported. * Whether **JSON mode** is `unavailable` / `object` / `schema`. * Whether **streaming** is native or emulated. * Whether **prompt caching** / **context caching** reduces cost. * Which **supported parameters** are legal (reasoning-effort, thinking- budget, computer-use, citations, …). If you send `tools: [...]` to a model whose capability rule says `tool_calling = false`, Nika fails fast with `NIKA-13x` before hitting the network: no silent provider-side error. See [Capability rules](/reference/capabilities) for the full resolution algorithm (defaults → scan rules in file order → apply matches). ## Fallback chains (target design) When a provider fails, Nika can route to a fallback: ```yaml target design · a future minor theme={"system"} tasks: - id: primary infer: model: anthropic/claude-sonnet-4-6 # / prompt: "..." fallback: - { model: openai/gpt-4o } - { model: groq/llama-3.3-70b-versatile } ``` `fallback:` is reserved in the kernel DTO but not yet parsed at v{STATUS.version}. Model fallback chains need a separate runtime policy gate before they become user-facing syntax. ## Why {STATUS.providers} and not 9 Earlier drafts of this page listed 9 providers. That was a snapshot from mid-2025. Today's catalog is {STATUS.providers}-deep, covering frontier cloud labs, fast-inference shops (Groq, Cerebras, SambaNova), Chinese labs (Moonshot, Qwen, MiniMax, Zhipu), enterprise platforms (Bedrock, Azure, Vertex), and niche (Voyage embeddings, Writer enterprise). Source of truth lives in the TOML and is build-time validated. ## Adding a new provider Declare `id`, `name`, `aliases`, `env_var`, `key_prefixes`, `default_model`, `cheap_model`, `api_dialect`, `tags`, and at least one `[[providers.models]]` sub-block. Put rules in `model-capabilities.toml` scoped via `scope.providers`. Inherit defaults if the provider is standard (most openai-chat providers need no rules). If the `api_dialect` already exists, wire a new provider instance with the correct base URL + auth. If new dialect, new L1 crate `nika-provider-`. Property test against the provider's API contract. Criterion bench for token throughput. Capability parity test. [How admission works →](/architecture/admission). ## See also All {STATUS.providers} providers with env vars, default / cheap models, dialects. Resolution algorithm and {STATUS.capabilityRules}-rule contract. The `infer:` verb and its three siblings (exec / invoke / agent) · fetch is the `nika:fetch` builtin under `invoke:`. Why capability rules use prefix matching, not regex (Q1).