Ollama Beta

6 min read

Ollama lets the model run on a machine you control instead of through a cloud provider. You keep processing in your infrastructure and are responsible for its resources, availability, and security.

Experimental

The Ollama integration is in testing and can change. Use it first in a controlled environment.

Before connecting it

Install Ollama from ollama.com/download, download the model, and ensure its address is reachable from the agent container.

ollama list

Ollama listens locally by default. When an agent accesses it from another container or machine, listen on a network interface and protect access through a private network, VPN, or firewall.

OLLAMA_HOST=0.0.0.0 ollama serve

Do not expose Ollama without protection

Ollama does not add authentication. Never expose its port directly to the internet; use a VPN or a proxy with authentication and HTTPS.

Create a connection

In Settings > Models > Ollama, select Add connection. A connection represents one specific machine-and-model combination, not a provider account: three machines running Ollama are three connections and none is redundant.

1. Give it a name—it is required, and it is the only thing that tells one machine from another in the list and in each agent's configuration. Something like “Development Ollama”.
2. Enter the URL reachable by the agent. For a model on the same agent server, use http://host.docker.internal:11434.
3. Under Server location, choose Private network for the same machine, your LAN, or a VPN, or Public access for a tunnel, your own domain, or dynamic DNS. Public access requires HTTPS.
4. Under Model, pick one from the list: Kujira asks your Ollama —from your server— which models it has downloaded, and shows each one's context window and size. Picking one fills in the window. Models that support 64k tokens or less are shown dimmed and cannot be picked: an agent does not fit in them. If the list does not appear (a server not yet updated, or an address that does not answer), type the exact name returned by ollama list and its window, which must be larger than 64k.
5. Select the connection under Connection in the agent's Configuration > Model. The connection carries its own model, so that is the one the agent runs. Neither saving the assignment nor changing the connection later (its model, window, key or address) restarts anyone: its agents use it from their next reply.

Choose the right address

Situation	Typical address	Recommendation
Same server	http://host.docker.internal:11434	Confirm that model is installed on that server.
Local network	http://192.168.x.x:11434	Limit access to the Kujira server.
Private network or VPN	Private URL or IP	Recommended for another location.
Internet	Protected HTTPS URL	Use only behind authentication and a secure proxy.

Several machines, several connections

Each machine running Ollama is a connection with its own name, address, and model. One of them is marked as the default—the one inherited by agents that pick no connection—but with different machines the usual choice is to pin each agent to its own by name rather than inherit.

To remove a connection agents already use, Kujira first asks which one they move to: pick it under Move the agents to and confirm with Move and remove. If you are only shutting the machine down for a while, turn the connection off instead of removing it.

Context and performance

A model with 64k or less does not work

On every message, an agent sends the model its instructions and the description of all its tools, before anything you write: about 30,000 tokens for a freshly created agent, and more the more memories, skills, and external tools it has. That does not fit in a 32k model, and Ollama does not warn you: it cuts off the beginning and the agent answers nonsense—often the example of a tool written out as text. A 64k model fits it, but leaves no room for the conversation.

That is why Kujira only accepts local models with a window larger than 64k tokens. When you connect one, it only lets you pick those windows, and Test turns red if the model you chose falls short or your server serves 64k or less—the norm on a fresh Ollama install, which serves 4,096. Models such as qwen2.5-coder (32k) are left out; look for one with a 128k window or more, and raise it in Ollama with OLLAMA_CONTEXT_LENGTH so it is served in full.

Match the real window

The context window must match the model's real capacity. A value that is too high can cause errors in long conversations; one that is too low compacts sooner than necessary.

To find out what it is, press Test on the connection: the check shows you both figures together — what the model supports and what your server is actually serving — red if your server serves 64k or less, and amber if it serves more but less than the model supports. Ollama serves 4,096 tokens by default even when the model supports far more; you raise it with OLLAMA_CONTEXT_LENGTH on the machine where it runs. Details in Checking a local engine.

There is no cloud per-token cost, but consumption still depends on your GPU, CPU, memory, and electricity. Larger models require more resources and can respond more slowly.

Common errors

• The agent cannot connect: check the address, port, firewall, and that Ollama listens outside localhost.
• The first message fails: ensure the configured name exactly matches ollama list, including the tag and version.
• Only one agent works: agents can run on different servers; give each access to the same instance or model name.
• The agent boots against another machine: check which connection it has pinned under Configuration > Model. With Inherit from the organization it uses the default, which may belong to a different machine.
• It answers nonsense from the first message: usually a model with too small a window—what the agent sends does not fit and Ollama cuts it silently. Press Test on the connection; if it says the model supports 64k or less, switch to one with a larger window. See A model with 64k or less does not work.
• Long conversations fail: adjust the context window on the connection; its agents use it from their next reply, without restarting.

See Models to manage models and credentials through external providers.