JIMDRURY.
Jim Drury
A hedging chat answer and the regex needed to parse it are crossed out; instead a state document goes into evaluate() and comes back as typed answers with probabilities: Boolean refunded 0.97 and Choice next_tool 0.81 are automated, Score risk 0.46 is routed to a human
ai

Stop asking an LLM to say yes or no

3 MIN READ

Yes. No. Maybe parse that.

I keep catching myself doing the same silly dance: ask a chat model a binary question, then spend ten minutes making the answer behave like a boolean.

Strip the markdown. Guess which synonym meant false. Retry when it hedges. For agent loops and guardrails, that tax adds up fast.

Then TypeSafe AI posted that Jev is on the Vercel AI Gateway, and the changelog (16 Sep 2026) made the pitch concrete.

What Jev actually returns

Jev is a probabilistic decision model. State goes in. Typed answers come out: Choice, Score, and Boolean, with probabilities attached.

Instead of generating a paragraph you later parse, it evaluates the questions you declare, in parallel, in one request, and hands back structured results. Automate the clear cases. Route the uncertain ones. That is the whole trick.

Where I'd reach for it: pick the next tool or subagent, decide continue / retry / ask / stop, score urgency or risk before an action, verify outputs before they leave the building.

The catch: evaluate, not chat

Model ID on the gateway: typesafe-ai/jev.

You call it through AI SDK 7's experimental evaluate API (ai@7.0.105 onwards). It is not available via the OpenAI / Anthropic / Cohere-compatible gateway endpoints. If your stack only speaks chat completions, this will look invisible until you wire evaluate.

Install the current AI SDK:

Install AI SDK 7
pnpm add ai@latest

Minimal shape from the changelog: one support case, one boolean, ZDR on:

evaluate with typesafe-ai/jev
import { experimental_evaluate as evaluate } from 'ai';

const result = await evaluate({
  model: 'typesafe-ai/jev',
  state: 'The support agent issued a full refund to the customer.',
  questions: {
    refunded: {
      type: 'boolean',
      instructions: 'Was a refund issued?',
    },
  },
  providerOptions: {
    gateway: { zeroDataRetention: true },
  },
});

console.log(result.answers.refunded);

You pass model, shared state, and a questions map. Boolean estimates P(true). Choice picks an option. Score grades an ordered rubric. Stack several questions in one call and get them back in parallel.

Zero Data Retention / No Training is per request via providerOptions.gateway, the changelog example sets zeroDataRetention: true. Same object takes the rest of the usual Gateway provider options.

Price and claims

Gateway model page pricing: $0.042 per million input tokens.

TypeSafe reports (their numbers, not mine) up to 193.6x faster and 444.6x cheaper than LLMs on their workflow evaluations. Treat that as vendor eval until you measure your own path.

Honest caveat: probabilities still need calibration against labelled examples from your workflow. A confident wrong boolean is still wrong.

What I'll try first

I'm not ripping chat models out of generation. I am looking hard at every place I currently ask an LLM to say yes or no, pick a bucket, or score risk, then parse the poetry afterwards.

If the job is a decision, stop paying for a novel.

Similar Articles