Jev intent routing

A few terms come up again and again. None of them needs a technical background.

• Intent. What the person sending a request wants. For example, to check an order, ask about a product, return an item or make a complaint.
• Handler. Whatever deals with the request once it has been sorted: plain code, a specialist language model or a person.
• Routing. Deciding which handler gets which request.
• Language model. A text-writing AI such as ChatGPT or Claude. It can draft a reply, which Jev cannot.
• Plain code. Ordinary software, as opposed to AI. It does exactly the same thing every time for the same input. TypeSafe's documentation calls this deterministic code.
• Question types. Jev answers three kinds of question. A Choice picks one option from a list you write. A Score rates something on a scale you define. A Noul is a yes/no question.
• Confidence. A number from 0 to 1 that says how sure Jev is about an answer. Our guide on confidence routing explains it in full.

• Jev confidence routing

Think of the receptionist at a busy office. They read each message and send it to billing, to returns or to a manager. They do not solve the problem themselves. They only make sure it reaches the right desk, quickly.

Intent routing is the same idea in software, and Jev plays the receptionist. It reads a request, works out what it is about, and tells your software which handler should take it. Jev is fast and cheap, and it never writes the reply, which is why it suits this job.

• Jev for support ticket routing

TypeSafe's example is a customer service system. A message comes in, and one request sends it to Jev with two questions: what does the customer want (a Choice), and how complex is the request (a Score). Both answers come back together, each with its confidence. Then your code decides where the message goes.

• Jev is not sure what the request is (confidence under 0.5): a person takes it.
• Order status: plain code looks up the order. No language model is involved.
• Product question: a specialist language model that has been given your product information.
• Return or exchange: a specialist language model that has been given your returns rules.
• Complaint: Jev's complexity score decides. A simple complaint goes to a language model that handles complaints. In TypeSafe's example, a complaint goes to a person if the complexity score is above 1, or if Jev is not sure about the complexity.

TypeSafe's argument is that not every request needs the same kind of handler. Some can be answered by a database lookup, some need a language model with knowledge of one area, and some need a person. Instead of sending every message to an expensive language model just to find out what kind of request it is, you sort first and route accordingly.

TypeSafe says this means the costly handlers are used only for the requests that need them. That is the company's own claim. For one independent measurement of what routing costs with Jev, see our support ticket routing page.

Because the answers are the same either way and the cost is lower. TypeSafe's parallel questions cookbook ran 13 questions about one long document, once as 13 separate requests and once as a single request. The answers did not change. Each question is judged on its own against the text, so what else is in the request does not affect it.

The single request cost $0.000497, against $0.006090 for the 13 separate ones, which is 12.2 times cheaper, because the long document was read once instead of 13 times. It took 0.27 seconds, against 2.71 seconds for the 13 requests added up, which is 10.0 times faster. That time figure assumes the 13 requests run one after another: sent at the same time, the gap shrinks, but the cost saving stays. This was TypeSafe's own test, on jev-1.12 and a document of about 54,000 characters.

Our read: a customer message is far shorter than that document, so expect a smaller saving. The direction is the same, though. Two questions about one message are best sent together.

The example puts a confidence check in front of every route. If Jev's confidence in what the request is falls below 0.5, the message goes to a person, whatever the best guess was. Complaints get a second check on the complexity score.

TypeSafe stresses that what a low confidence means depends on the system and on the stakes of the decision. Misrouting a product question costs little. Misrouting an angry complaint can cost a customer.

• Jev confidence routing

TypeSafe does not publish a step list for this. The steps below are our read of how to apply the pattern, using the question design guide for the wording.

• List your handlers first. Decide what can be done by plain code, what needs a specialist language model, and who the people are.
• Write the kinds of request. One line for each, so two kinds do not overlap.
• Add a catch-all kind that goes to a person. Anything that fits nothing else should not be forced into the nearest box.
• Test on real messages. Run a few hundred past requests through, and check where Jev's answers and your own sorting differ.
• Set the confidence floor from that test. TypeSafe's example uses 0.5. Yours may differ.

• How to write Jev questions

Our read is that the person route decides whether this works. Every unsure and complex case ends up there, so it needs a queue, an owner and a target time to answer. A router that sends hard cases into a pile nobody reads is worse than having no router.

Once that route works, adding another specialist handler is a small change: one more option in the list, and one more branch in your code.