Ollama Decision Models: Ask a Local Model a Question, Get a Probability
A model class that answers typed questions with probabilities instead of prose. The --ask flag, the three question types, which model to pull, and the limits.
Ollama decision models are the least chatty thing you can run locally. Here is one in full:
ollama run nimble --ask "Is a refund being requested?" < ticket.txt
It prints yes 97%. Not a paragraph explaining that the customer does appear to be requesting a refund. Not JSON you have to parse out of a wrapper of politeness. A verdict and a number.
Ollama decision models have been quietly arriving all month — the terminal route landed on 12 October, cloud access two days later, a 27B model from Perplexity on 18 October, and a whole new family on 24 October — and as far as I can find, nobody has written about them. The documentation exists and is good. Everything else is silence. So this is what the model class actually is, how to drive it, and where its edges are.
What a decision model is
A decision model takes two things: a state, which is text or JSON you supply, and a set of named questions with declared types. It answers every question about that state in one pass, and each answer comes back as a probability rather than prose.
That is the whole idea, and the reason it matters is the failure mode it removes. The usual way to make a local model classify something is to ask a chat model nicely and hope: reply with only one word, either billing or technical. Sometimes it replies billing. Sometimes Billing. Sometimes “Based on the ticket, this should go to billing.” You end up writing a parser for a model’s manners. A decision model has no manners to parse — Ollama’s decision documentation is explicit that it classifies text, estimates yes/no probabilities, and scores against ordered criteria, and that is all it does.
The three question types
Everything you can ask is one of three shapes, and the API reference defines each precisely.
choice picks one option from a labelled set. You supply between 2 and 26 options at the command line, or up to 255 through the API, each with a description. You get back the winning label, a probability for every option normalised to sum to 1, and a confidence value.
noul is the yes/no type, and the detail that trips people up is in the return: it is a number from 0 to 1 giving the probability of “true” relative to “false”. It is not a boolean. You choose the threshold, which is the right way round — a refund request at 0.97 and one at 0.51 are not the same event, and a boolean would have thrown that away.
score rates the state on an ordered scale of 2 to 10 descriptions, lowest first. What comes back is not a rounded level but a probability-weighted average of the indices — for three levels, the docs give it as 0 × P(lowest) + 1 × P(next) + 2 × P(highest). That is why the terminal prints something like Soon 64% 1.2 / 2: the most likely level, its probability, and the weighted position on the scale.
Asking from the terminal
The capability requires Ollama v0.35.0 or later, and the quickest model to start with is Nimble:
ollama pull nimble
The state is piped in on stdin or given after the model name. Each question is a --ask, answered yes or no unless you follow it with labels or levels:
# A yes/no question
ollama run nimble --ask "Is a refund being requested?" < ticket.txt
# Choose a label
ollama run nimble --ask "Which team should handle this?" \
--choice billing="Payments and refunds" \
--choice technical="Software problems" < ticket.txt
# Score on a scale, lowest level first
ollama run nimble --ask "How urgent is this?" \
--level "Routine" --level "Soon" --level "Immediate" < ticket.txt
All three can go in one command, and that is the point — one pass over the state, three answers, which is what the image above shows. Add --format json when you want the full response rather than the printed summary.
Two conveniences worth knowing. If your state is JSON, you can point a question at one of its fields by naming it in backticks:
ollama run nimble --ask 'Is a refund being requested in `body`?' < ticket.json
And running the model with no state at all opens an interactive prompt where your first message becomes the state and every line after it is a yes/no question about it, with /clear to start again.
Which model to pull
Four options, and the gap between them is mostly size.
Nimble and Tev1 are the original Jev-style set announced at the end of September — Nimble at 9B, Tev1 at 4B, and a 0.8B version of Tev1. These are what the documentation’s examples use, and they are small enough to be unremarkable on any machine that already runs a local model.
pplx-decider is a 27B model from Perplexity, fine-tuned from Qwen3.8-27B, listed at 17GB with a 256K context and released under Apache 2.0. Its page records the publisher’s own comparison — 85.7% against 84.5% for Jev — which is a publisher figure on a publisher benchmark, and the margin is narrow enough to treat as a tie until somebody independent measures it.
d3 is the newest, from vLLM Semantic Router, shipped with v0.40.11 on 24 October and requiring Ollama 0.40.9 or later. It comes in four sizes with honest numbers attached: d3:2b is a 3.1GB download that loads into 3.62GB on Apple Silicon, d3:4b is 5.6GB and 5.63GB loaded, d3:9b is 10GB and 10.09GB, and d3:27b is a 29GB download that wants 28.61GB of memory and a 64GB machine. It is Apache 2.0, and unlike the others it judges still images as well as text — though the page is clear that video and Box/Track decisions did not come across to Ollama.
If you are picking blind, start at d3:4b or Nimble. The 27B models are for when you have measured the 4B one failing on your own examples, which is a different situation from assuming it will.
Calling it from code
The endpoint is POST /v1/systemone and the shape is flat: model, state, and a questions object holding between 1 and 64 named questions. Each question carries a type, instructions, and usually criteria. There is an optional keep_alive to control how long the model stays resident, and an images field for the vision-capable models, which takes base64 — URLs and data URLs are rejected.
curl http://localhost:11434/v1/systemone \
-H 'Content-Type: application/json' \
-d '{
"model": "nimble",
"state": "Our checkout has returned 500 errors since 9am.",
"questions": {
"label": {
"type": "choice",
"instructions": "Which label fits this ticket?",
"criteria": {
"billing": "Payments and refunds",
"bug": "Software errors",
"account": "Login and account access"
}
}
}
}'
The response carries model, an answers object keyed by your question names, and usage. One line in the usage definition is worth reading twice: shared context is counted again for each separately scored question. Asking eight questions about one long document is not the same cost as asking one.
Two limits bite locally. Text and schema, excluding images, must be 64 KiB or less; the whole request must be under 32 MiB. And input is never truncated — if it does not fit the loaded model’s context window, the request is rejected rather than silently trimmed, which is the behaviour you want but not the behaviour people expect. There is no streaming, no tool calling and no generation controls on this endpoint; it answers and stops. If you are used to wiring Ollama into a tool-calling agent loop, this is the opposite shape — one request, one structured answer, nothing to orchestrate.
What the numbers do and do not mean
The confidence value is the one most likely to be misread. The API reference defines it as a measure of how concentrated the distribution is: 0 when every candidate is equally likely, approaching 1 when one dominates. It explicitly “measures distribution shape only and is not a calibrated estimate of correctness”. A confident model can be confidently wrong, and the number will not tell you.
The same caution applies to the probabilities themselves. d3’s page puts it plainly: scores and probabilities are model outputs, not guaranteed real-world frequencies, and you should test on your own examples before setting thresholds. That is the honest instruction, and it is the one piece of work this feature cannot do for you — if noul > 0.9 is going to trigger a refund, you need to know what fraction of your own tickets land above 0.9 and how many of those were actually refund requests.
The sense in which these models are fast is also worth pinning down. Ollama’s own announcement cites 91ms per decision for Nimble 9B on an M5 Max — a vendor figure, on vendor hardware, from the launch post. Treat it as an order of magnitude rather than a benchmark: fast enough to sit inside a request, not fast enough to assume anything about your 27B model on your laptop.
Picking a threshold instead of guessing one
The docs tell you to test on your own examples before setting thresholds, and it is worth spelling out what that actually involves, because it is an hour’s work and it is the difference between a useful classifier and a confident one.
Take 50 items you already know the answer for — tickets you have already routed, emails you have already filed. Run each through with --format json and keep the probability, not the verdict:
for f in samples/*.txt; do
printf '%s\t' "$f"
ollama run nimble --ask "Is a refund being requested?" --format json < "$f"
done > results.tsv
Now sort by probability and look at where your known answers stop agreeing. You are looking for two numbers: the point above which the model has never been wrong on your data, and the point below which it has never been right. Those are your automation thresholds, and the band between them is the pile that still needs a person.
That band is the real output of the exercise. If it turns out to be four items in fifty, you can automate the rest and route the rest to a human queue with a clear conscience. If it is thirty, the model is not reading your data the way you assumed, and the fix is almost always the wording of the criteria rather than a bigger model — which is cheaper to discover now than after you have downloaded 29GB.
Rough edges
The documentation has not caught up with the library. The capability page lists the decision models as Nimble, Tev1, Clef and Clef Flash. It does not mention pplx-decider or d3, both of which have library pages, release notes and CLI examples of their own. Nothing is broken; the list is just stale, and if you go by that page alone you will not know the two largest options exist.
d3’s page contradicts itself on context. Each tag is listed with a 256K context window, while the same page’s limits section states that these packages use 4,096 tokens. The smaller figure is the one stated for the Ollama packages, so plan for 4,096 and verify before feeding it anything long.
The cloud route has a price you cannot see from here. Adding :cloud to a model name works from v0.40.5 onwards, but it needs a paid plan or purchased credits, and usage is billed per input token. Neither the capability page nor the API reference states the rate — they say “the listed price” and link away. If cloud is part of your plan, get the number before you build on it.
It will not do the judgement for you. d3’s page puts it directly: the model only scores the options and levels you provide, so clear and distinct criteria matter — which is another way of saying vague criteria produce confident nonsense. Writing good --choice descriptions is the actual work, and it is the same discipline that decides whether automated ticket triage sorts anything correctly — the model is not the part that knows what your categories mean.
What makes this worth an evening is not that it is clever. It is that a whole class of small, boring, high-volume jobs — route this, flag this, rank this — has been waiting for something that answers a question instead of making conversation, and you can now pull one with ollama pull. If your machine already runs local models for the reasons in the complete Ollama guide, the marginal cost of trying it is a 5.6GB download and twenty minutes.
Sources
Every figure in this article traces to one of these. Dates are when the page was read.
- Decision capabilityOllamaAccessed Oct 27, 2026Official docs
- POST /v1/systemoneOllamaAccessed Oct 27, 2026Official docs
- d3OllamaAccessed Oct 27, 2026Official docs
- pplx-deciderOllamaAccessed Oct 27, 2026Official docs
- ReleasesOllamaAccessed Oct 27, 2026Official docs
- Ollama now supports Jev-style decision modelsOllamaAccessed Oct 27, 2026Vendor figure
Written by
Full-stack developer who builds and runs AI automation systems in production. Runs local LLMs on personal hardware, builds N8N pipelines that actually ship, and deploys on Cloudflare Pages. Every guide on Pipeline Monk is tested on real consumer hardware — a Ryzen 7 5800HS with 16GB RAM and a GTX 1650. If it works on that, it works.