Skip to main content
Nika is local-first: point model: at a local server and your data never leaves the machine. This page is the ladder, from the one-command path to production servers.

​
1. The one-command path: Ollama

Install Ollama, then:
That’s it. In any workflow:
nika doctor confirms the wiring (it detects the running Ollama server and prints the exact fix if something is off).
llama3.2:3b is the showcase default: small enough for most laptops, good enough for structured extraction. Swap the tag for anything in the Ollama library: qwen2.5, mistral, phi4, larger llama variants.

​
2. Any Hugging Face GGUF, still one command

Ollama pulls directly from the Hugging Face Hub. Any public GGUF repo works, no account needed:
Pick a quantization explicitly with a tag:
Then reference it in the workflow exactly as pulled:
Rule of thumb for quantizations: Q4_K_M is the sane default (quality per GB), Q8_0 when you have RAM to spare, Q2/Q3 only when memory is tight. The Hub’s GGUF filter lists every compatible repo.

​
3. LM Studio: the visual browser

Prefer a GUI? LM Studio browses Hugging Face, downloads models with a click, and serves an OpenAI-compatible endpoint. Start its server, then:

​
4. Servers: llama.cpp and vLLM

For shared machines and production:
  • llama.cpp llama-server serves any GGUF: model: llamacpp/<model>
  • vLLM serves full-precision Hub models at datacenter throughput: model: vllm/<hub-repo-id>
Both speak the same OpenAI-compatible dialect; the prefix picks the provider, nothing else in the file changes.

​
Checking what’s wired

Doctor reports every provider (local servers detected, cloud keys present) with a fix-form per missing piece. The audit before a run prices local models at $0.000:

​
Thinking time is budgeted for

A local thinking model legitimately needs minutes for one completion on consumer hardware — well past the 30s a cloud round-trip expects. Nika’s provider deadlines are per class: local providers (ollama, lmstudio, llamacpp, localai, vllm) get ≥ 300s by default, cloud providers 30s. No 30s-everywhere default silently kills a local run mid-think. When a task needs more than the default, the task’s own timeout: governs the provider deadline directly:
Full semantics (per-class defaults, the 600s silent-connection ceiling, streaming): YAML reference · Timeouts.

​
Swapping between local and cloud

The file does not change shape. One line moves the workflow between a laptop and an API:
The permits boundary applies either way: a permits: block with no net.http entry means the workflow cannot reach the network even if the model could. Local model + closed permits = fully air-gapped AI work.

Providers

The full catalog and how one InferRequest speaks every dialect.

First workflow

Five minutes from install to a checked, runnable file.

Model bench

Pick your default local model on a measured table, not vibes.

Cost honesty

Why a local run is unpriced — never « free » — and how floors work.