Jev confidence routing

A few terms come up again and again. None of them needs a technical background.

• Question types. Jev answers three kinds of question. A Choice picks one option from a list you write, such as the topic of an email. A Score rates something on a scale you define, such as how urgent a message is. A Noul is a yes/no question.
• Probability. For a Choice or Score, Jev gives every option a share of 100%. For example, 80% for billing, 15% for sales and 5% for technical.
• Confidence. One number from 0 to 1 that sums up how sure Jev is. It is 1 when all the probability sits on one option, and 0 when it is spread evenly, which means Jev is effectively guessing.
• Cutoff. A number you choose, such as 0.85, that your software compares the confidence against. TypeSafe's documentation calls it a threshold.
• Routing. Deciding where a case goes next: straight through, to a check, or to a person.
• Your code. The software you write around Jev. Jev only answers; your code decides what to do with the answer.

Picture a new employee answering a customer's request. Sometimes they say: definitely a billing question. Sometimes they say: it could be billing or sales, I'm not sure. You would treat those two answers differently. You would let the first go through, and you would check the second with a manager.

Confidence routing is the same idea in software. Your code reads how sure Jev is and sends the case down a different path: act on it, check it first, or hand it to a person. TypeSafe, which makes Jev, describes confidence as a second thing to read next to the answer itself. The answer says what Jev picked, and the confidence says whether to act on it.

TypeSafe's example is a banking app where customers say what they want. Two of the things they might ask for are to see their balance, which is how much money is in the account, and to approve a transfer, which moves money out. Jev reads each request and picks what the customer wants, together with its confidence. Then the app's code decides what happens.

• Jev is unsure (under 0.6), or the request is something else: hand the customer to a person.
• Balance request, 0.6 or more: show the balance. If Jev is wrong, the worst outcome is that the customer hears a balance they did not need.
• Transfer request, 0.6 to 0.85: ask the customer to confirm first. Moving money is riskier, so a fairly sure answer is not enough to act alone.
• Transfer request, above 0.85: go ahead automatically.

Because a wrong answer does not cost the same everywhere. Showing the wrong screen is easy to undo. Approving the wrong transfer is not. TypeSafe's guidance is that the cutoff is not one number for the whole system: the riskier the action, the surer Jev should be before your software acts without asking.

TypeSafe's two examples use different numbers. The banking example uses 0.6 as the floor and 0.85 for transfers. Its confidence guide uses 0.5 and 0.9, and still asks for confirmation before acting at the top level. Our read: the numbers are illustrations, and the structure is the point. There is a floor below which a person takes over, and a higher bar for riskier actions.

You never have to calculate it, because Jev returns it with every Choice and Score answer. For the curious, here is how it behaves:

• Choice: it measures how far the top option's share sits above an even split. If three options get a third each, confidence is 0. If one option gets everything, it is 1. Only the top share counts, so splits of 60/30/10 and 60/20/20 both give 0.4.
• Score: the levels are in order, so being torn between two neighbouring levels lowers confidence less than being torn between the lowest and the highest.
• Noul (yes/no): there is no separate confidence number. The probability does the job: a value near 50% means Jev is unsure.

No. TypeSafe calls its confidence one reasonable way to summarise how sure Jev is, not the only way. Every answer also includes the full list of probabilities, so you can use another measure.

TypeSafe names two simple ones worth trying: the share of the top option on its own, and how far ahead the top option is of the runner-up. The top share reads directly as how likely the pick is, but what counts as high depends on how many options there are, so set its cutoff for each question.

US-listed companies file an annual report, and each company can be sorted into an industry. TypeSafe tested Jev on 60 of these reports, asking it to pick one of 75 industry groups, such as chemicals or life insurance. Those groups roll up into ten broad divisions, such as manufacturing or finance.

The rule was simple. If Jev's confidence was 0.9 or higher, report the specific industry group. If it was lower, report only the broad division that group belongs to. That needs no second call to Jev.

The cutoff split the 60 reports in half. The 30 confident answers were right 27 times, which is 90%. The other 30 were right 12 times (40%) when forced to name an industry group, and 70% when reported as the broad division instead. Across all 60, that moves the useful answers from 39 to 48, or from 65% to 80% by our arithmetic.

Read it with its limits. It is TypeSafe's own test, on 60 reports, with labels the companies chose themselves, run on jev-1.12 on 12 August 2026, before the current versions. The 0.9 cutoff happened to split this sample in half. It shows the shape of the result, not a number to copy.

• Document classification use case

TypeSafe says the right values depend on your field and on how the model performs for your use case, and recommends starting cautious and testing on your own data. The steps below are our read of how to do that, not a TypeSafe procedure.

• Collect a sample. Take a few hundred real cases and write down the right answer for each.
• Run them through Jev and sort by confidence. Group the answers into bands, such as 0.9 and above, 0.7 to 0.9 and so on, and measure how often each band is right.
• Pick the lowest cutoff that is safe enough for the action. A cheap action you can undo can use a lower cutoff than one you cannot.
• Decide what the in-between path does before you pick numbers. A confirmation, a second model and a person each cost something different.
• Run the sample again when the model version changes. Confidence values can shift between versions.

Yes. In TypeSafe's test, 3 of the 30 confident answers were wrong. Confidence measures how concentrated Jev's probability is, not whether the answer is correct, and TypeSafe's own list of weak spots includes text written to steer the model.

So keep the hard safety rules in ordinary code, and let confidence decide only the judgment calls around them. Our guide on coding agents shows that split: a fixed list of dangerous commands is checked first, and only then does Jev score the rest, with approval only when both scores are at least 0.9.

• Where Jev struggles

Our read is that the value of this pattern is making not sure a normal path in your software instead of an error. The hard part is not choosing 0.85 or 0.9. It is deciding what happens to the answers in between, because that is where the cost sits: a person to review, a second call, a question to the customer.

If you can describe that path in one sentence, the cutoffs are just tuning. If you cannot, no number will fix it.