How accurate is it?
A confidence number only helps if it is honest. So we ran 56 made-up denial letters through the same steps the app uses, and checked each answer against the right one. No real patient data was used. The charts below use all 56.
Source: jev-latest. Scored on 2026-10-31. This is a small set written by one team, so treat these numbers as rough.
The honest test
We tuned our questions on 42 letters, so scores on those run high. Then we locked the questions and wrote 14 new letters. On those new letters, it got 11 of 14 fully right. At 75% sure, it answered 10 and got 9 right. That is 90%, and the rest stopped to ask.
95%
Right when it answers (38 of 40)
71%
Letters it answers (40 of 56)
29%
Letters where it stops and asks
88%
Right if it never stopped to ask
Does “80% sure” mean 80% right?
See this as a table
| Sure | Answers | Actually right |
|---|---|---|
| 40–50% | 1 | 0% |
| 50–60% | 11 | 73% |
| 60–70% | 4 | 100% |
| 70–80% | 9 | 67% |
| 80–90% | 12 | 100% |
| 90–100% | 187 | 100% |
Ask more, get more right
See this as a table
| Must be | Letters it answers | Right when it answers |
|---|---|---|
| 50% | 98% | 89% |
| 55% | 91% | 92% |
| 60% | 80% | 96% |
| 65% | 77% | 95% |
| 70% | 73% | 95% |
| 75% | 71% | 95% |
| 80% | 63% | 100% |
| 85% | 61% | 100% |
| 90% | 54% | 100% |
| 95% | 34% | 100% |
Each step on its own
| Step | Right |
|---|---|
| Insurance type | 54 of 56 |
| Denial reasons (all must match) | 51 of 56 |
| Appeal level | 56 of 56 |
| Appeal strength | 56 of 56 |
What one question does
At 75%, the app stopped on 18 letters. With one true answer, 4 of those became clear, and 4 of those 4 were right. The other 14 went to a person.
The answers here were simulated from the correct labels, so this is the best case.