Jev: when your agent doesn't need an LLM for every decision

I was at my usual cafe, terminal open, watching a tiny agent read a few emails and guess which ones were urgent. Two possible answers. The log still showed 11 model calls.

At the eleventh one I stopped. Not because the model was wrong — because that question did not need a language model. It needed a structured answer, fast enough that code could decide what to do next.

Most decisions inside an agent loop do not need text generation. They need a probability, a choice, or a score.

That is the gap Jev from TypeSafe AI is aiming at. On their numbers, Jev is up to 200x faster and 400x cheaper than a typical LLM on classification. I read LangChain's post on wiring Jev into a harness, then rewrote it from the angle I care about: not "new hot model", but "where am I paying for thinking I do not need".

The agent loop is powerful — and every arrow costs

The familiar loop: LLM decides → tool runs → model evaluates → repeat. Tool calling and structured outputs made it easier for models to talk to software. They only fixed the interface. They did not fix the fact that every small decision — even "urgent or not?" — still costs a full model round trip.

For an agent that runs a few steps a day, fine. For a system with thousands of runs, money and latency leak quietly. My mail agent was a miniature of that: one state, asked many times, the expensive way.

Jev is not a replacement LLM

The important part: Jev does not generate text. TypeSafe calls it a System One model — a class of models built for fast, typed decisions that software can consume directly.

You send a state (context) and a set of questions. No "please answer warmly". Just questions, then probabilities:

{
  "model": "jev-latest",
  "state": "Hi, I've been trying to connect my Stripe account for 3 days and it keeps failing. I'm losing sales. Please help ASAP.",
  "questions": {
    "is_urgent": {
      "type": "noul",
      "instructions": "The message conveys urgency or time-sensitivity"
    }
  }
}

You get something like 0.999 — 99.9% chance it is urgent. The app compares a threshold and routes the ticket.

Three question types are enough to start:

• Choice — pick from a fixed set, with a probability per option.
• Score — rate against ordered levels (low / medium / high).
• Noul — yes/no; returns the probability that the statement is true.

The detail I like: ask many questions on the same state in one request, evaluated in parallel. Extra questions barely move latency — you mostly pay the tokens for the questions. Those eleven calls of mine could, in theory, collapse into one.

Three places I would wire first

1. Model routing

A simple lookup should not share a model with a nasty bug. Let Jev read the request, pick the cheapest model that can finish, and use it for the run. Hard work still escalates. Easy work stops sharing the bill.

2. A guardrail before tools run

Agents are still hard to trust by default. One bad instruction — accidental or injected — and they can touch a shell. Big coding harnesses already classify dangerous actions, but that step often lives in closed code.

A cheap, fast classifier lets the same pattern live in your agent: inspect the tool call, block before it runs. This is the one I care about most, because it changes how it feels to leave an agent near a terminal.

3. Ticket and email triage

Urgent or not, topic, how annoyed the customer is, which team should take it — all classification. One state, five questions, five probabilities. For a support inbox, that is almost the job description.

LangChain exposes this through TypeSafeClassifier:

from langchain_typesafe import Noul, TypeSafeClassifier

classifier = TypeSafeClassifier()
response = classifier.invoke({
    "state": "The deploy failed twice and customers are seeing 500s. Can someone look now?",
    "questions": {
        "urgent": Noul(instructions="Does this need attention right now?"),
    },
})
urgency = response.nouls["urgent"].noul

Install langchain-typesafe, set TYPESAFE_API_KEY, call it from a node or middleware with the context your agent already has. The guardrail path is shorter with AutoModeMiddleware — Jev checks risky tool calls and blocks them before the tool runs.

When not to use Jev

I would not push Jev into places that need writing, explanation, or open reasoning. It will not draft an email, plan a bug fix, or "understand" in the storytelling sense.

I also would not treat 200x / 400x as a promise. Those are vendor numbers on their tasks. Long state, slow network, or sloppy questions — your numbers will differ. The right move: count how many model calls in a week are really yes/no questions, then replace those with one classifier request.

Short checklist I use:

1. Is the answer in a fixed set? (yes/no, pick one, scored level)
2. If it is wrong once, is the damage acceptable? (wrong model route ≠ delete production)
3. Can several questions share the same state?
4. Is there a clear threshold for the next code path?

Three yeses → try Jev. Need prose, a long why, or invention → keep the LLM.

End at the real pain

Jev is not a ChatGPT replacement toy. It is a reminder: we have been using LLMs as a Swiss army knife for jobs that only needed a switch.

If you also have an agent calling a model too many times for questions that never needed text — start there. Count the calls. Collapse the yes/no questions. Measure bill and latency before and after. Marketing numbers will have to argue with your own.

The post I read: Building a harness with Jev on LangChain.