[ Use case · AI safety ] # Can Jev check an AI's citations? Catch made-up quotes and claims the source does not support. Published 17 October 2026By Jev Hub [Back to Use cases](https://jevhub.io/use-cases) [ TLDR ] Strong fit - Yes, with a little code around it. - A string match first catches quotes that do not exist in the source. - Then one Jev Choice question per citation reads the quoted section and decides whether it supports the claim, contradicts it or does not address it. - In TypeSafe's own cookbook, all four planted failures among eight citations were caught. ## Is Jev a fit for citations? Use Jev when: - An LLM writes answers or reports with citations to documents you hold. - A wrong citation is costly: legal, medical, financial or technical writing. - You want people to check only the doubtful citations, not all of them. Skip it when: - You do not have the source documents. Jev can only judge a quote against the text you send it. - Quotes are loosely paraphrased. An exact string match marks those as made up, so you would need fuzzy matching first. ## Which questions does Jev answer? One request per citation, one question. Jev answers them in parallel. - **Does the quoted section support the claim?** (choice. Options: Supports, Contradicts, Unsupported) One citation, one question, then your code. Jev answers the questions about each citation in one request. Your own code reads the answers and their probabilities and decides what happens next. ## What does it cost? About 64¢ a month for 20,000 citations of about 530 words each: 3.2¢ per 1,000 at list price, billed by TypeSafe AI. [Adjust the numbers in the calculator](https://jevhub.io/calculator?items=20000&state=700&q=1&qt=60). ## What does TypeSafe's worked example show? TypeSafe's cookbook checks eight citations an LLM wrote about RFC 7519 (JSON Web Token). Four are accurate; four were edited to fail. - The four accurate citations came back verified at confidence 0.93 or higher. - One quote was not in the RFC at all, so the string match marked it fabricated before any model call. - One quote was word for word, but its section says the claim is optional: contradicted, at confidence 0.99. - Two came back unsupported at 0.27 and 0.56, under the 0.8 threshold, so both went to a person. Source: TypeSafe's cookbook [Double-checking citations](https://docs.typesafe.ai/cookbooks/citation_check), numbers from jev-1.12 on 16 August 2026. These are the maker's own results on its chosen data; test on yours. ## How do you get started? 1. **Test it in the playground.** No code needed. Paste a real citation into TypeSafe's playground and ask: *“Does the quoted section support the claim?”* with the options Supports, Contradicts, Unsupported. You'll see the answer and its probability. [Open the playground](https://console.typesafe.ai/playground). 2. **Use an existing tool.** We haven't found a ready-made tool for this workload yet. [Browse what people have built with Jev](https://jevhub.io/builds). 3. **Integrate it.** To run it on every citation automatically, call Jev from the system where your citations live. The developer details below include the full request. ## Common questions In TypeSafe's approach, no model is needed for that: a plain text search finds the quote and tells you which section it came from. Jev then judges whether that section supports the claim. One short request per citation. At Jev's list price of $0.042 per million input tokens, a section of a few hundred words costs a small fraction of a cent. **How do developers set it up?** ### The request One request per citation with all 1 questions batched over the same state, so Jev reads the state once. Keys below are the names you'll see in the response. POST api.typesafe.ai/v1/systemone ``` { "model": "jev-latest", "state": "Claim: The \"exp\" claim is required in every JSON Web Token.\nQuoted section (RFC 7519, 4.1.4): The \"exp\" (expiration time) claim identifies the expiration time on or after which the JWT MUST NOT be accepted for processing. ... Use of this claim is OPTIONAL.", "questions": { "verdict": { "type": "choice", "instructions": "Does the quoted section support the claim?", "criteria": { "supports": "The section states or directly implies the claim", "contradicts": "The section says the opposite of the claim", "unsupported": "The section does not address the claim" } } } } ``` ### How to wire it up 1. Normalise spaces and quote marks, then look for each quote in the source as plain text. A quote that is not there is fabricated, with no model call needed. 2. Send the claim plus the section the quote came from to Jev as one Choice question. 3. Accept the verdict when confidence is at or above your threshold (the cookbook uses 0.8); below it, a person confirms. 4. Start with a high threshold and lower it as you see how Jev does on your own documents. ### Watch out for - A quote can be real and still not back the claim. That is why the string match alone is not enough. - The cookbook splits an RFC by its section numbers; documents of another shape need their own splitting. New to writing Jev questions? Read the [question design guide](https://jevhub.io/resources/question-design). ## Related use cases - [Can Jev filter what a RAG pipeline retrieves?](https://jevhub.io/use-cases/rag-passage-filtering) - [Can Jev re-rank search results?](https://jevhub.io/use-cases/search-reranking) - [Can Jev tell whether two records are the same thing?](https://jevhub.io/use-cases/record-matching)