Jev speculative fan-out

A few terms come up again and again. None of them needs a technical background.

• Fan-out. Sending many questions to Jev in a single request, instead of one request per question.
• Speculative question. A question you ask in case it turns out to matter. If it does not, your code ignores the answer.
• Request. One message from your software to Jev, which comes back with the answers. TypeSafe also calls it a call.
• Round trip. One request and its reply. A follow-up question means a second round trip.
• Question types. A Choice picks one option from a list you write. A Score rates something on a scale you define. A Noul is a yes/no question, answered as a probability: 0.7 means a 70% chance of yes.
• Your code. The software you write around Jev. Jev only answers; your code decides which answers to use.

Think of a clinic's intake form. The nurse asks every question in one visit: age, symptoms, allergies, medication. Some answers will turn out not to matter for this patient. That is fine, because the alternative is to make the patient come back for each question one at a time.

Speculative fan-out is the same idea for Jev. Instead of asking for the category first and the details in a follow-up, you send all of the questions together, and your code picks out what is relevant once the answers arrive. TypeSafe recommends putting all the questions your system needs into a single request.

A support system has to sort each ticket into a category. If it is a bug report, it also needs to know how severe the bug is. Rather than asking the category first and the severity second, TypeSafe's example sends one request with the ticket and five questions, which are in the table above. The five answers come back together, each with its probability.

Then your code decides what matters, using cutoffs that are TypeSafe's illustration:

• Bug report: if the severity score is above 1.5 and the chance of reproducible steps is above 0.6 (60%), escalate to engineering. Otherwise add it to the bug backlog.
• Billing: if the chance of a refund request is above 0.7 (70%), send it to billing with a flag. Otherwise send it to billing as normal.
• Feature request: log it. The bug and refund answers are simply ignored.
• Any ticket: if the frustration score is above 1.5, flag it for a priority reply.

TypeSafe gives two reasons. The first is that it saves a round trip: when the extra answer turns out to matter, you already have it, and when it does not, you ignore it.

The second is speed. TypeSafe says all the questions in a request are evaluated in parallel, so adding more of them usually has little effect on response time. That is the company's own claim, and the figures below are from its own cookbook.

TypeSafe says no. Each question is scored on its own against the input, so an answer does not depend on what else is in the request. Its cookbook checked this with 13 questions about one long document, asking each one several times in one request and again as 13 separate requests.

The answers matched. The choices, the scores and six of the eight yes/no questions came back identical every time. The other two moved slightly from one repeat to the next, by the same small amount either way. So adding speculative questions does not disturb the ones you care about.

In TypeSafe's cookbook, the one request with all 13 questions cost $0.000497, against $0.006090 for 13 separate requests. That is 12.2 times cheaper, mainly because the long document was sent once instead of 13 times. The time figure, 0.27 seconds against 2.71 seconds, adds up the 13 separate requests as if they ran one after another. Sent at the same time, that gap shrinks, but the cost saving stays. This was a TypeSafe test on jev-1.12.

Our read: speculative questions are cheap but not free. Every question, and every option in a Choice, adds some input to the request, so keep them short. For how the number of options affects cost, see our support ticket routing page.

• Jev cost calculator

TypeSafe does not list exceptions, so this is our read. Fan-out works when the questions are independent: each can be answered from the input alone. If a later question needs an earlier answer written into it, for example, you must first learn the product the customer named and then ask something about that product, you need a second request.

Also be selective. A question that your code will never use under any branch is not speculative, it is just a cost.

TypeSafe does not publish a step list for this. The steps below are our read of how to apply the pattern.

• Write down what happens after the first sort. For each kind of ticket or request, what else do you need to know?
• Turn each of those into a question. A Choice for the sort, then a Score or a yes/no for each detail.
• Send them all in one request. Let your code read the Choice first, then use only the answers for that branch.
• Set the cutoffs from real cases. TypeSafe's numbers, such as 1.5 and 0.6, are examples. Our guide on confidence routing covers how to set them.

• Jev confidence routing

Our read is that fan-out changes how you design, more than how you code. Before writing any code, draw every decision the system makes, and turn each one into a question. Once the tree is on paper, putting all of it in one request is easy, and gaps show up early.

It pairs naturally with sorting requests to the right handler, which our guide on intent routing covers.

• Jev intent routing