Best Local LLM for AI Agents: Ollama Models by GPU (2026)

For a 24 GB GPU or a 32 GB Mac, Qwen3.8 27B (qwen3.8:27b, 18 GB) is the best all-round local model for agents in October 2026. With 16 GB, use gpt-oss 20B. On a laptop, use Qwen3.5 9B. For coding agents, Qwen3.6 27B reports the highest SWE-bench score in the table below. All of them carry the "tools" tag in the Ollama library, and all scores below are the vendor's own, from the model card.

These are the best local agentic models per memory size on October 20, 2026. Download sizes are for the default tag on ollama.com: 4-bit Q4_K_M, or MXFP4 for gpt-oss.

Key takeaways:

• Memory decides more than the model. The weights must fit with the context on top. Ollama gives a 4K context on GPUs under 24 GiB of VRAM by default, which is too small for an agent.

• Tool calls must come back as tool calls. A wrong chat template or tool parser turns them into plain text, and nothing raises an error.

• Vendor numbers are a shortlist. Most local models aren't on the independent boards, so test your own tools before you choose.

• Plan for the local server being down. A fallback to a hosted model keeps the agent answering, at the cost of sending that request off your network.

A hosted API hides most of the tool-calling machinery. A local server makes you own it. Your agent sends the tool definitions as JSON schemas with the conversation. The server's chat template renders them, with the messages, into the prompt format the model was trained on. The model writes its tool call as text in its own syntax, such as Qwen's <tool_call> tags or gpt-oss's harmony format. A parser on the server recognizes that text and returns it as OpenAI-style tool_calls. Your agent runs the tool and sends the result back.

agent (Promptise, MCP tools)                local server (Ollama, LM Studio, vLLM, llama.cpp)
----------------------------                -------------------------------------------------
messages + tool schemas ---- POST /v1/chat/completions ---> 1. chat template renders the tools
                                                            2. model writes a call in its own syntax
                                                               <tool_call>{"name": "get_order", ...}</tool_call>
                                                            3. parser turns that text into
tool runs, result goes back <------- tool_calls JSON ------    OpenAI "tool_calls"

Four things must hold, and each fails in its own way:

• The model was trained for tool calling. Without it, the model answers in prose and never calls anything. Ollama marks these models with a "tools" tag.

• The chat template includes the tools. With a wrong template, the model never sees the tools, or sees them in a format it wasn't trained on.

• The server parses the model's syntax. If not, the call arrives as plain text in the reply, the tool never runs, and no error is raised.

• Everything fits in the context window. When the conversation is longer than the context, the server drops part of it. Ollama keeps the system messages and the latest message and drops the older messages between them, which in an agent loop means the user's question and earlier tool results (source).

Two things share the GPU: the weights, roughly the download size, and the KV cache, which holds the context and grows with every token. Agents need long contexts: the system prompt, every tool definition and every tool result stay in it for the whole run. Ollama's docs recommend at least 64,000 tokens for agents and coding tools.

The KV cache size follows from the model's architecture: two vectors (key and value) per KV head, per full-attention layer, per token. This script computes it from each model's config.json on Hugging Face and adds the weights:

"""Estimate the memory a local model needs: its weights plus the KV cache for the context.

Weights: the download size of the default Ollama tag (ollama.com/library/<model>/tags).
KV cache: from each model's config.json on Hugging Face. Every layer with full
attention stores one key and one value vector per KV head for every token in
the context. Layers with linear or sliding-window attention keep a fixed-size
state instead, so they are left out. Add 1-2 GB for the runtime itself.
"""

MODELS = {
    # name: (Ollama tag, download GB, full-attention layers, KV heads, head dim)
    "Qwen3.5 9B": ("qwen3.5:9b", 6.6, 8, 4, 256),
    "gpt-oss 20B": ("gpt-oss:20b", 14, 12, 8, 64),
    "Qwen3.8 27B": ("qwen3.8:27b", 18, 16, 4, 256),
    "Qwen3.6 35B-A3B": ("qwen3.6:35b", 23, 10, 2, 256),
    "Devstral Small 2 24B": ("devstral-small-2", 15, 40, 8, 128),
    "gpt-oss 120B": ("gpt-oss:120b", 65, 18, 8, 64),
}
BYTES = {"f16": 2, "q8_0": 1}  # bytes per cached value (OLLAMA_KV_CACHE_TYPE)


def kv_gb(layers: int, heads: int, head_dim: int, tokens: int, cache: str = "f16") -> float:
    return 2 * layers * heads * head_dim * BYTES[cache] * tokens / 1e9  # 2 = key + value


print(f"{'model':<22}{'tag':<18}{'weights':>8}{'KiB/token':>10}{'+32k':>7}{'+64k':>7}{'+128k':>7}  (GB)")
for name, (tag, weights, layers, heads, dim) in MODELS.items():
    per_token_kib = 2 * layers * heads * dim * BYTES["f16"] / 1024
    totals = [weights + kv_gb(layers, heads, dim, n) for n in (32_768, 65_536, 131_072)]
    print(f"{name:<22}{tag:<18}{weights:>8.1f}{per_token_kib:>10.0f}" + "".join(f"{t:>7.1f}" for t in totals))

layers, heads, dim = MODELS["Devstral Small 2 24B"][2:]
print(f"\nDevstral Small 2 at 64k: KV cache {kv_gb(layers, heads, dim, 65_536):.1f} GB in f16, "
      f"{kv_gb(layers, heads, dim, 65_536, 'q8_0'):.1f} GB with OLLAMA_KV_CACHE_TYPE=q8_0")

model                 tag                weights KiB/token   +32k   +64k  +128k  (GB)
Qwen3.5 9B            qwen3.5:9b             6.6        32    7.7    8.7   10.9
gpt-oss 20B           gpt-oss:20b           14.0        24   14.8   15.6   17.2
Qwen3.8 27B           qwen3.8:27b           18.0        64   20.1   22.3   26.6
Qwen3.6 35B-A3B       qwen3.6:35b           23.0        20   23.7   24.3   25.7
Devstral Small 2 24B  devstral-small-2      15.0       160   20.4   25.7   36.5
gpt-oss 120B          gpt-oss:120b          65.0        36   66.2   67.4   69.8

Devstral Small 2 at 64k: KV cache 10.7 GB in f16, 5.4 GB with OLLAMA_KV_CACHE_TYPE=q8_0

What the numbers show:

• Newer models need far less context memory. Qwen3.5 and later run full attention in only one layer in four, and gpt-oss uses a 128-token sliding window in half its layers. Devstral Small 2 runs full attention in all 40 layers, so it needs 2.5 times as much memory per token as Qwen3.8 27B.

• Qwen3.8 27B fits a 24 GB GPU at 32K context (about 20 GB plus runtime) but not comfortably at 64K. At 64K, quantize the cache or use a smaller model.

• A q8_0 KV cache halves the context memory with "a very small loss in precision", according to the Ollama FAQ. Avoid q4_0 for agents: llama.cpp's function-calling docs warn that extreme KV quantization can substantially degrade tool calling.

The weights have the same trade-off. On Llama 3.1 8B, the llama.cpp quantize docs list 4.58 GiB at Q4_K_M and 7.95 GiB at Q8_0, with Q4_K_M generating faster. Q4_K_M is the usual default; move up to Q8_0 when you have memory to spare and the model makes argument mistakes. These are estimates: ollama ps shows what a loaded model really uses and whether part of it spilled to the CPU.

The gap is real. Qwen3.6 35B-A3B reports 62.8 on MCP-Atlas, while Claude Fable 5.1 scores 87.2 on Scale's own run of it; the best LLM for AI agents has the hosted picks. A common setup does both: local for routine steps, hosted for the hard ones, as LLM routing describes.

• A model server: Ollama, LM Studio or vLLM, and a GPU or Apple silicon Mac with enough memory for your pick.

• Python 3.10 or later and Promptise Foundry 1.4.3 or later: pip install "promptise>=1.4.3".

• Optional: a hosted model key, such as OPENAI_API_KEY, for the fallback in step 5.

A note on what ran for this guide: Ollama isn't installed on the machine it was written on, and no models were downloaded. The server commands in steps 1 and 2 come from each project's docs. Everything in steps 3 to 5 ran, against a stand-in server that speaks the same API, with real output.

Pull your pick and start Ollama with a longer context and a quantized KV cache. These settings are global for the server:

ollama pull qwen3.8:27b

# 64K context, 8-bit KV cache (Ollama turns on flash attention when the GPU supports it)
OLLAMA_CONTEXT_LENGTH=65536 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve

# After the first request: the context it got, and how much runs on the GPU
ollama ps

The OpenAI-compatible endpoint that agents use has no way to set the context per request, as Ollama's compatibility docs say, so set it on the server or in a Modelfile with PARAMETER num_ctx. The same page lists tools as supported and tool_choice as not supported, so you can't force a tool call through it.

Watch for :cloud tags. The large open models, such as glm-5.3-flash, kimi-k3 and deepseek-v4-pro, have only :cloud tags in the library. Those run in Ollama's cloud with an Ollama account, not on your machine.

LM Studio, vLLM and llama.cpp's llama-server all speak the OpenAI chat API, and each needs tool calling switched on in its own way:

# LM Studio: load a model with native tool support (hammer icon), then start the server on port 1234
lms server start

# vLLM: tool calling needs both flags. The parser depends on the model.
vllm serve openai/gpt-oss-20b --enable-auto-tool-choice --tool-call-parser openai
vllm serve Qwen/Qwen3.6-27B --max-model-len 65536 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3

# llama.cpp: --jinja applies the model's own chat template (on by default in current builds)
llama-server -m qwen3.8-27b-Q4_K_M.gguf --jinja -c 65536 --port 8080

• LM Studio distinguishes native and default tool support. With native support, the model's template and LM Studio's parser both know its format. With default support, LM Studio describes the tools in a system prompt, and small models may write malformed calls.

• vLLM needs --enable-auto-tool-choice plus the right --tool-call-parser, per its tool calling docs: openai for gpt-oss, mistral for Mistral models, and for Qwen3.6 the flags from its model card. vLLM loads the checkpoint as published, so a 27B model in BF16 needs about 55 GB for its weights.

• llama.cpp uses the template stored in the GGUF file. If it doesn't recognize the template, it falls back to a generic format; pass --chat-template-file to fix it.

Promptise reaches Ollama through its OpenAI-compatible /v1 route, and any other local server through the openai provider with an endpoint. Building the model makes no request, so this runs with no server at all:

"""Point Promptise at Ollama, LM Studio, vLLM and llama.cpp, and show where each request goes.

resolve() builds the chat client without calling it, so this runs with no
model server running at all.
"""
from promptise import Model, check_model

LOCAL = {
    # Ollama: the model part is the tag you pulled, colon included.
    "Ollama": Model("qwen3.8:27b", provider="ollama"),
    "Ollama on a GPU box": Model("gpt-oss:120b", provider="ollama", endpoint="gpu-box:11434"),
    # LM Studio, vLLM and llama.cpp's llama-server speak the OpenAI chat API,
    # so they use the openai provider with the server's /v1 URL.
    # The model name is the identifier the server lists at GET /v1/models.
    "LM Studio": Model("qwen3.8-27b", provider="openai", endpoint="http://localhost:1234/v1", api_key="lm-studio"),
    "vLLM": Model("Qwen/Qwen3.8-27B", provider="openai", endpoint="http://localhost:8000/v1", api_key="local"),
    "llama.cpp": Model("qwen3.8-27b", provider="openai", endpoint="http://localhost:8080/v1", api_key="local"),
}

for label, model in LOCAL.items():
    llm = model.resolve()
    print(f"{label:<20} {model.spec:<28} {type(llm).__name__:<11} {llm.openai_api_base}")

# The string form works too. The first colon separates provider and model.
check = check_model("ollama:qwen3.8:27b")
print(f"\nollama:qwen3.8:27b -> provider={check.provider.key}, model={check.model}")

Ollama               ollama:qwen3.8:27b           ChatOpenAI  http://localhost:11434/v1
Ollama on a GPU box  ollama:gpt-oss:120b          ChatOpenAI  http://gpu-box:11434/v1
LM Studio            openai:qwen3.8-27b           ChatOpenAI  http://localhost:1234/v1
vLLM                 openai:Qwen/Qwen3.8-27B      ChatOpenAI  http://localhost:8000/v1
llama.cpp            openai:qwen3.8-27b           ChatOpenAI  http://localhost:8080/v1

ollama:qwen3.8:27b -> provider=ollama, model=qwen3.8:27b

Every route is the same OpenAI chat client with a different URL. The Ollama address accepts host:port without a scheme, and the model part of ollama:qwen3.8:27b keeps its own colon, because only the first colon separates provider and model. For LM Studio and llama.cpp, use the model name the server lists at GET /v1/models.

The most common local failure is silent: the model "calls" a tool by writing the call as text, and the server returns it as an ordinary reply. Test for it before you build anything on top. This script asks a question that needs a tool and fails if no tool ran:

"""Fail fast if a local server returns tool calls as text instead of real tool calls.

Usage: python tool_smoke_test.py ollama:qwen3.8:27b
Exit code 0 when a tool ran, 1 when it did not.
"""
import asyncio
import sys

from promptise import build_agent
from promptise.config import StdioServerSpec


async def main(model: str) -> int:
    agent = await build_agent(
        model=model,
        servers={"shop": StdioServerSpec(command=sys.executable, args=["shop_server.py"])},
        instructions="Look everything up with your tools.",
    )
    try:
        result = await agent.ainvoke({"messages": [{"role": "user", "content": "What's the status of order A-1005?"}]})
    finally:
        await agent.shutdown()
    tools_run = [m.name for m in result["messages"] if m.type == "tool"]
    reply = result["messages"][-1].text
    if tools_run:
        print(f"PASS {model}: ran {tools_run}")
        return 0
    print(f"FAIL {model}: no tool ran. Reply was: {reply[:120]!r}")
    return 1


sys.exit(asyncio.run(main(sys.argv[1])))

To show both outcomes without a GPU, this guide ran it against a stand-in server, stub_server.py. It answers /v1/chat/completions like a local server and is not a model. In tools mode it returns a proper tool call, as a server with a working parser does. In text mode it writes the same call into the reply text, as a server with a wrong template or parser does. It also logs how many tokens the tool definitions take:

"""A stand-in for a local model server, for testing the wiring without a GPU.

It speaks just enough of the OpenAI chat API (POST /v1/chat/completions) that
Ollama, LM Studio, vLLM and llama.cpp all serve. It is not a model:

  --mode tools  asks for one get_order tool call, then echoes the tool result
                (what a model with working tool calling does)
  --mode text   writes the tool call as plain text in the reply instead
                (what you see when the chat template or tool parser is wrong)

Each request is logged with the model name, the number of tools and an
estimate of how many tokens the tool definitions take.
"""
import argparse
import json
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

import tiktoken

ENCODING = tiktoken.get_encoding("o200k_base")  # an estimate; local models use their own tokenizers
MODE = "tools"


class ChatCompletions(BaseHTTPRequestHandler):
    def do_POST(self):
        body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
        if self.path != "/v1/chat/completions":
            self.send_error(404)
            return
        tools = body.get("tools", [])
        tool_tokens = len(ENCODING.encode(json.dumps(tools)))
        last = body["messages"][-1]
        print(f"POST {self.path} model={body['model']} tools={len(tools)} "
              f"(~{tool_tokens} tokens of definitions) last={last['role']}", flush=True)

        call = {"name": "get_order", "arguments": {"order_id": "A-1001"}}
        if last["role"] == "tool":
            message = {"role": "assistant", "content": f"Stub reply. The tool returned: {last['content']}"}
            finish = "stop"
        elif MODE == "text":
            # The model "called" the tool, but the server did not parse it into tool_calls.
            message = {"role": "assistant", "content": f"<tool_call>{json.dumps(call)}</tool_call>"}
            finish = "stop"
        else:
            message = {"role": "assistant", "content": None, "tool_calls": [{
                "id": "call_1", "type": "function",
                "function": {"name": call["name"], "arguments": json.dumps(call["arguments"])},
            }]}
            finish = "tool_calls"

        reply = json.dumps({
            "id": "chatcmpl-stub", "object": "chat.completion", "created": int(time.time()),
            "model": body["model"],
            "choices": [{"index": 0, "message": message, "finish_reason": finish}],
            "usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
        }).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(reply)))
        self.end_headers()
        self.wfile.write(reply)

    def log_message(self, *args):
        pass  # one line per request is printed above


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--port", type=int, default=11555)
    parser.add_argument("--mode", choices=["tools", "text"], default="tools")
    args = parser.parse_args()
    MODE = args.mode
    ThreadingHTTPServer(("127.0.0.1", args.port), ChatCompletions).serve_forever()

python stub_server.py --mode tools &
OLLAMA_HOST=127.0.0.1:11555 python tool_smoke_test.py ollama:qwen3.8:27b

PASS ollama:qwen3.8:27b: ran ['get_order']
exit 0

And with the stand-in in text mode:

FAIL ollama:qwen3.8:27b: no tool ran. Reply was: '<tool_call>{"name": "get_order", "arguments": {"order_id": "A-1001"}}</tool_call>'
exit 1

The second run is what a broken parser looks like from the agent's side: an HTTP 200, a reply that contains the call, and no tool run. Run this check against the real server whenever you change the model, the server version or its flags. The stub's log also shows the cost of the shop server's seven tools: about 500 tokens of definitions on every request.

A local server goes down for ordinary reasons: a reboot, a model still loading, a GPU busy with something else. FallbackChain tries the local model first and moves to the next model when a call fails or times out:

"""A local-first agent: a local model answers, a hosted model takes over when it is down.

Usage: python local_first.py          (LOCAL_URL defaults to http://localhost:11434)
"""
import asyncio
import os
import sys

from promptise import FallbackChain, Model, build_agent
from promptise.config import StdioServerSpec

# A model you pulled with `ollama pull qwen3.8:27b`. The first call loads it, so allow time.
LOCAL = Model("qwen3.8:27b", provider="ollama", endpoint=os.environ.get("LOCAL_URL"), timeout=120)
HOSTED = os.environ.get("HOSTED_MODEL", "openai:gpt-6-luna")


def report(failed: str, next_model: str, error: Exception) -> None:
    print(f"!!! {failed} failed ({type(error).__name__}); trying {next_model}")


async def main() -> None:
    agent = await build_agent(
        model=FallbackChain([LOCAL, HOSTED], on_fallback=report),
        servers={"shop": StdioServerSpec(command=sys.executable, args=["shop_server.py"])},
        instructions="You are a shop assistant. Look everything up with your tools; never guess.",
        trace_tools=True,
    )
    try:
        result = await agent.ainvoke({"messages": [{"role": "user", "content": "Where is order A-1001?"}]})
        answer = result["messages"][-1]
        print(f">>> [{answer.response_metadata.get('fallback_model')}] {answer.text}")
    finally:
        await agent.shutdown()


asyncio.run(main())

The runs below point the local model at the stand-in with LOCAL_URL=http://127.0.0.1:11555 python local_first.py. With the server up in tools mode, the local model answers:

→ Invoking tool: get_order with {'order_id': 'A-1001'}
✔ Tool result from get_order: {"order_id": "A-1001", "customer_id": "C-8", "items": [{"sku": "MUG-01", "qty": 2}], "status": "shipped", "total_eur": 25.0}
>>> [ollama:qwen3.8:27b] Stub reply. The tool returned: {"order_id": "A-1001", "customer_id": "C-8", "items": [{"sku": "MUG-01", "qty": 2}], "status": "shipped", "total_eur": 25.0}

With nothing listening on the local port, each model call fails over to GPT-6 Luna:

!!! ollama:qwen3.8:27b failed (OpenAIConnectionError); trying openai:gpt-6-luna
→ Invoking tool: get_order with {'order_id': 'A-1001'}
✔ Tool result from get_order: {"order_id": "A-1001", "customer_id": "C-8", "items": [{"sku": "MUG-01", "qty": 2}], "status": "shipped", "total_eur": 25.0}
!!! ollama:qwen3.8:27b failed (OpenAIConnectionError); trying openai:gpt-6-luna
>>> [openai:gpt-6-luna] Order A-1001 has shipped. I don’t have tracking or a more precise location for it.

The fallback happened twice because the agent made two model calls, one to pick the tool and one to answer, and each tried the local model first. After three failures in a row, the circuit breaker skips the local model for 60 seconds, then tries it again. response_metadata["fallback_model"] records which model wrote each answer, so you can count fallbacks in production.

Now the case a fallback can't catch. With the stand-in in text mode, the local server answers with HTTP 200, so there is nothing to fall back from:

>>> [ollama:qwen3.8:27b] <tool_call>{"name": "get_order", "arguments": {"order_id": "A-1001"}}</tool_call>

The user gets the raw tool call. That is why step 4 exists: a fallback handles a server that is down, and only a test catches a server that is up but wrong.

local-agent/
├── shop_server.py        # MCP server with seven tools (from the best LLM for AI agents guide)
├── local_models.py       # where each local server's requests go (step 3)
├── stub_server.py        # stand-in server for tests without a GPU (step 4)
├── tool_smoke_test.py    # fails when tool calls come back as text (step 4)
├── local_first.py        # local model first, hosted fallback (step 5)
└── kv_cache.py           # memory estimate: weights plus context (memory section)

pip install "promptise>=1.4.3"
ollama pull qwen3.8:27b
OLLAMA_CONTEXT_LENGTH=65536 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve &
python tool_smoke_test.py ollama:qwen3.8:27b     # must print PASS
export OPENAI_API_KEY="sk-..."                    # only for the fallback
python local_first.py

shop_server.py is the seven-tool MCP server from the best LLM for AI agents. For a step-by-step Ollama agent with your own MCP server, see Ollama AI agent with MCP.

Promptise Foundry doesn't run models. Ollama, LM Studio, vLLM or llama.cpp serve the model, and Promptise is the agent around it. It reads the tools from your MCP servers, sends them to the local server as tool definitions, runs the calls the model makes, and returns the results. Because the model is one argument to build_agent, the same agent runs on a laptop model in development, on a GPU server in production, and on a hosted model when the local one is down.

Where other tools are the better choice: for a coding assistant in your editor or terminal, Ollama documents integrations with coding agents such as Cline and OpenCode, with no agent code to write. To serve one model to many users, vLLM's server with a gateway in front is the serving layer; Promptise is a client of it, not a replacement.

Keep the server private

Ollama binds to 127.0.0.1 by default, and its FAQ shows proxies and tunnels for exposing it but no authentication of its own. Keep model servers on a private network or behind a proxy that checks a token. For a protected proxy, send the header with Model(..., extra={"default_headers": {"Authorization": "Bearer ..."}}).

Size for concurrency

Ollama handles one request per model at a time by default. Raising OLLAMA_NUM_PARALLEL multiplies the context memory, because "Required RAM will scale by OLLAMA_NUM_PARALLEL * OLLAMA_CONTEXT_LENGTH". If several agents share one GPU, measure first; for many users, a server built for batching such as vLLM fits better.

Pin model versions and keep them warm

A tag such as qwen3.8 can point to a new build after an ollama pull. Pin the full tag, such as qwen3.8:27b-q4_K_M, note the ID that ollama list shows, and rerun step 4 and your eval after every pull. Ollama unloads a model after five minutes idle, so the next call waits for it to load; set OLLAMA_KEEP_ALIVE or give the local model a generous timeout, as local_first.py does.

Decide what the fallback may see

A hosted fallback sends the request, tool results included, to that provider. Where data must stay on your network, fall back to a second local server instead, or let the request fail. LLM semantic cache and model fallback covers the chain in more depth.

Keep the tool list short

Every tool definition costs context on every request, about 70 tokens per tool in the shop server. Connect only the servers an agent needs, or let Promptise select tools per step as too many MCP tools explains. Context engineering for AI agents covers trimming long histories.

• No local model ran for this guide. Ollama isn't installed here and no models were downloaded. The server commands come from each project's docs, and the agent code ran against a stand-in server and a hosted model.

• Most local scores are the vendor's own. The independent boards rarely include models small enough to run on one GPU, and BFCL's last update (April 2026) has none of the models above.

• Memory numbers are estimates from the architecture. Runtimes add their own buffers, so check ollama ps.

• Promptise doesn't serve models or fix templates. It works with whatever the server returns, which is why the smoke test matters.

Which Ollama model is best for agents?

With a 24 GB GPU or a 32 GB Mac, Qwen3.8 27B (qwen3.8:27b, 18 GB) is the best all-round choice in October 2026. With 16 GB, gpt-oss 20B fits with room for context. On a laptop, Qwen3.5 9B reports BFCL-V4 66.1 at 6.6 GB. Check the "tools" tag, give it at least a 32K context, and run your own tool-calling tasks before you commit.

What is the best open-source LLM for agentic workflows?

Among open-weight models, GLM-5.3-Flash leads Toolathlon-Verified at 78.4 under an MIT license, and Kimi K3 leads MCPMark Verified. Both need multi-GPU servers. On one GPU, the Qwen3.6 and Qwen3.8 27B models report the strongest agent scores, and gpt-oss 20B and 120B are the Apache-2.0 options from OpenAI.

What are the best Ollama models for agentic coding?

Qwen3.6 27B (qwen3.6:27b) reports SWE-bench Verified 77.2 and fits a 24 GB GPU. Devstral Small 2 (devstral-small-2, 15 GB) reports 68.0 and is built for coding agents, but its context costs more memory. With 80 GB, gpt-oss 120B reports 62.4. Pair the model with a coding agent such as Cline or OpenCode, or call it from your own agent.

Can a local model call MCP tools?

Yes, if the model supports tool calling and the server parses its calls. The MCP client, such as Promptise, reads the tools from your MCP servers and sends them to the local model as tool definitions. Then it runs the calls the model returns. Run a smoke test like the one in step 4, because a broken parser fails silently.

How much VRAM do I need for a local AI agent?

Add the weights (about the download size) to the context memory. Qwen3.8 27B needs about 20 GB at a 32K context and 22 GB at 64K, plus 1 to 2 GB for the runtime. gpt-oss 20B needs about 15 GB at 32K, and Qwen3.5 9B about 8 GB. An 8-bit KV cache halves the context part.

Does Promptise work with Ollama, LM Studio and vLLM?

Yes. Use "ollama:<tag>" or Model("<tag>", provider="ollama") for Ollama. For LM Studio, vLLM and llama.cpp, use Model("<name>", provider="openai", endpoint="http://host:port/v1"). Everything works with pip install promptise, and FallbackChain adds a hosted model for when the local server is down.

• Compare local and hosted models on your tasks: the best LLM for AI agents.

• Build the agent step by step with Ollama: Ollama AI agent with MCP.

• Use open-weight models without hosting them: OpenRouter AI agent with MCP.

• Send easy steps to the local model and hard ones to a hosted one: LLM routing.

• Make the fallback robust: LLM semantic cache and model fallback.

• Test models properly: how to evaluate AI agents.

• Reference: custom and self-hosted endpoints in the Promptise docs.