Jev answers in 400ms for two hundredths of a cent. Here is what it is, how it works, and the exact steps for Claude Code, Codex, Cursor and seven more.
Interesting, my question is how can this be best used? For e.g. can I call it with OpenClaw and have it read all the documents in my knowledge vault and chat logs and assign a score to each document in terms of accuracy and consistency? The low-score documents then passed to another round of detailed checking.
Jev scores consistency, not accuracy. Worth separating those two before you build it.
It can tell you a document contradicts itself, or that a claim has nothing behind it in the text you sent. It cannot tell you whether something is true in the world, because it has no reference to check against. So ask "does this contradict the rest" instead of "is this right".
Your cascade is the right shape. Score everything, then send only the low scorers for the detailed pass. The expensive model sees the 10% that need thinking, not all of it.
Two practical notes. One document per call, since state plus questions caps at 64k tokens. And ask several narrow questions per document in one call, because they answer in parallel and the wait barely moves.
A mid-sized vault costs cents. I want to run this over mine. Tell me what your score distribution looks like if you get there first.
Interesting, my question is how can this be best used? For e.g. can I call it with OpenClaw and have it read all the documents in my knowledge vault and chat logs and assign a score to each document in terms of accuracy and consistency? The low-score documents then passed to another round of detailed checking.
Jev scores consistency, not accuracy. Worth separating those two before you build it.
It can tell you a document contradicts itself, or that a claim has nothing behind it in the text you sent. It cannot tell you whether something is true in the world, because it has no reference to check against. So ask "does this contradict the rest" instead of "is this right".
Your cascade is the right shape. Score everything, then send only the low scorers for the detailed pass. The expensive model sees the 10% that need thinking, not all of it.
Two practical notes. One document per call, since state plus questions caps at 64k tokens. And ask several narrow questions per document in one call, because they answer in parallel and the wait barely moves.
A mid-sized vault costs cents. I want to run this over mine. Tell me what your score distribution looks like if you get there first.