[ Use case · Search ]
Can Jev re-rank search results?
Published By Jev Hub
[ Sponsors ]Launch price
These products help keep Jev Hub updated. Want yours up here?
- Yes. A fast search such as keyword matching builds a shortlist, and Jev then scores every shortlisted result against the query and puts the best first.
- In TypeSafe's cookbook on 3,565 court-opinion passages, re-ranking a 30-result keyword shortlist raised the share of queries with the right passage in first place from 5% to 18%, and in the top ten from 38% to 62%.
Is Jev a fit for results?
Use Jev when:
- Your search finds roughly the right documents but often not the best one first.
- Relevance depends on meaning, not shared words: legal, support or product search.
- You can keep the shortlist small (tens of results, not thousands).
Skip it when:
- Your keyword search already puts the right result first most of the time.
- You need to score millions of documents per query. Re-rank a shortlist, not the whole index.
Which questions does Jev answer?
One request per result, one question. Jev answers them in parallel.
- How well does this passage answer the query? (score. Scale: Unrelated → Same topic, does not answer → Partly answers → Answers the query)
What does it cost?
About $3.78 a month for 300,000 results of about 190 words each: 1.3¢ per 1,000 at list price, billed by TypeSafe AI. Adjust the numbers in the calculator.
What does TypeSafe's worked example show?
TypeSafe's cookbook builds a 30-passage keyword shortlist for each of 40 legal queries from the CLERC dataset (3,565 court-opinion passages), then scores every query and candidate pair with one Jev question: 1,200 calls in all.
- Right passage in first place: 5% with keyword search alone, 18% after re-ranking.
- In the top five: 15% to 35%. In the top ten: 38% to 62%.
Source: TypeSafe's cookbook Re-ranking (opens in a new tab). These are the maker's own results on its chosen data; test on yours.
How do you get started?
- Test it in the playground. No code needed. Paste a real result into TypeSafe's playground and ask: “How well does this passage answer the query?” with the options Unrelated, Same topic, does not answer, Partly answers, Answers the query. You'll see the answer and its probability. Open the playground (opens in a new tab).
- Use an existing tool. We haven't found a ready-made tool for this workload yet. Browse what people have built with Jev.
- Integrate it. To run it on every result automatically, call Jev from the system where your results live. The developer details below include the full request.
Common questions
TypeSafe's cookbook notes that a re-ranker needs a comparable score for every pair, and a general model asked to score can give different numbers for the same pair on repeat calls, while adding time and cost. Jev returns one score per pair.
Tens, not thousands. The cookbook re-ranks 30 per query, so each search costs 30 short requests.
How do developers set it up?
The request
One request per result with all 1 questions batched over the same state, so Jev reads the state once. Keys below are the names you'll see in the response.
{
"model": "jev-latest",
"state": "Query: When can a court refuse to enforce an arbitration clause in an employment contract?\nCandidate: ...the clause was unconscionable because the employee had no meaningful opportunity to negotiate its terms...",
"questions": {
"relevance": {
"type": "score",
"instructions": "How well does this passage answer the query?",
"criteria": [
"Unrelated",
"Same topic, does not answer",
"Partly answers",
"Answers the query"
]
}
}
}How to wire it up
- Run your fast search (the cookbook uses BM25 keyword search) and keep a shortlist; the cookbook keeps 30.
- Send Jev the query and one candidate at a time with one Score question.
- Sort the shortlist by the score and show the top results.
- Cache scores for popular queries so you do not pay twice.
Watch out for
- Re-ranking cannot rescue a result the fast search missed. If the right document is not on the shortlist, widen the shortlist.
- Describe every level of the score in words; the scale is the whole definition of relevance.
New to writing Jev questions? Read the question design guide.
Related use cases
Can Jev tell whether two records are the same thing?
Decide which records to merge, which to keep apart and which a person should check.
Published
Can Jev sort documents into a long list of categories?
Pick one of dozens of categories, and fall back to a broader one when unsure.
Published
Can Jev extract dates, emails and amounts from text?
Find the right value in messy text without letting a model make one up.
Published