Where Jev struggles, and what to do instead

Jev is a decision model, not a general-purpose assistant. It reads some input and your questions, and returns a typed answer with a probability. That design is what makes it fast and cheap, and it is also why some jobs suit it badly.

TypeSafe publishes its own list of known problems, which it calls the model's jaggedness. The page says Jev is "fast, calibrated, and good at common-sense judgment" but "not perfect", that it may struggle with tasks that need extra levels of indirection, and that many of the issues "will be fixed in later versions". This guide explains each one in plain words.

No, and TypeSafe says so directly: Jev "is not a calculator". The documentation names three separate problems.

• Counting. It does not count reliably, whether that is letters in a word, mentions of a term in a passage or items in a long list, and the error grows with the size of the thing counted. If a regular expression or a parser can find the unit, the count belongs in code.
• Numbers in disguise. Questions about colours do better with English names than with hex values, and Jev cannot reliably tell whether two RGB or hex values are close. Likewise, high-level code is judged better than assembly or binary. The fix is to convert in code and pass in a number or a named bucket.
• Score maths. A score gives you levels with probabilities. TypeSafe says to use them to check whether something passes a threshold, not to reconstruct an exact number between two levels.

Not reliably. Jev reads a date as text, not as a number on a timeline, so questions such as which of two dates comes first, how far apart they are or whether one falls inside a window are unreliable. Mixed formats, relative phrases such as "next Friday" and special periods such as quarters or settlement windows make it worse.

TypeSafe's suggested split is to give Jev the part that is a judgment and keep the rest in code. Each part of a date is a small closed set (twelve months, up to thirty-one days, a bounded range of years), so extracting it becomes a Choice question with an explicit "not stated" option. Code then builds the real date and does the ordering and the arithmetic. TypeSafe has a date extraction cookbook with a worked example.

Four of the nine weak spots come down to how the question is written:

• It reads literally. Jev answers the question you wrote, not the one you meant. TypeSafe's rule of thumb: when you find yourself explaining what you really meant after a wrong answer, that explanation is the missing half of the instruction.
• Indirection costs accuracy. Double negatives, a property of a property and multi-step reasoning are answered less reliably. Write the instruction directly and name the relevant part of the input.
• Instructions and criteria must agree. A yes/no question where "true" maps to no is the documented example. Aim for wording an average person would understand at once.
• Option order can matter. Jev leans toward the first option in a Choice, so check that the answer holds when you reorder them.

• The question design guide

Yes. Accuracy falls as the input grows with material unrelated to the decision, because the extra detail acts as a distractor and makes it harder to see which part produced a wrong answer. TypeSafe's page puts it as "context rot".

The recommendation is to retrieve and filter in code first and send only the fields the question needs. When filtering in code is not possible, a yes/no question can be used to pick out the relevant parts. The default limits are 64,000 tokens per request and 32,000 per question, and the input counts against both. TypeSafe also offers a longer-context option, marked experimental and recommended only when needed.

It can. TypeSafe says the input is data and Jev does not treat it as hostile by default. An injected instruction, a deliberately misleading framing, or text that argues for its own classification can move the answer, and TypeSafe says it expects to improve this.

Its advice is to be explicit in the criteria and to test your integration thoroughly before it reaches many users. Treat any decision that triggers a real action as one that needs a check outside Jev.

Jev is not trained to generate text. You can force it to by chaining choices, but TypeSafe says that works poorly and is very slow. For data extraction, the better pattern is to find the possible values with a regular expression or a language model, then let Jev pick the correct one as a Choice. If you need written text, you need a different model.

TypeSafe's page ends with four things to avoid:

• Asking the model something that code can compute exactly.
• Hiding several judgments inside one question.
• Tasks that need more layers of indirection.
• Giving it more input than the question needs.

Our read: every item on the list points the same way. Jev works best when each question is one narrow, literal judgment on a small, clean piece of input, and ordinary code does everything else: the counting, the dates, the filtering and the final decision about what to do.

The list applies to Jev 1.13. Jev 1.14 is described as small improvements across the board with the biggest gain in languages other than English, and TypeSafe's documentation has no separate list for it at the time of writing, so check the current page before relying on a fix.

• What is Jev?
• How to use Jev
• Jev 1.14: what changed