Repository navigation
Feature/grounding abstention confidence - #140
vedantlbhatt wants to merge 5 commits into
Conversation
|
The main issue we identified is that this is not a solid basis for abstention: the answer-grounding check measures word overlap, without checking whether the passages actually support the answer's claims. The overall score also includes retrieval embedding similarity, but that measures topical relevance and does not resolve this problem. A concrete example in This can fail in both directions: an unsupported or contradictory answer can reuse the source's words and pass, while a valid paraphrase can have low overlap and be suppressed. Pooling words across all excerpts also loses which passage supports which claim. We should not use this heuristic alone to force abstention or present a “Supported” label as evidence of grounding. We need to evaluate whether the retrieved evidence supports the actual claims, with tests covering contradictions, negation, paraphrases, and unanswerable questions, and measure both missed abstentions and incorrect refusals before relying on it. |
Grow the NLI benchmark from 10 to 35 questions (210 answers), split into dev and held-out questions, and add a threshold sweep that replays the TypeScript grading over one model run. Raise the entailment bar to 0.8, accept answers with half their claims entailed, and let a contradiction at least as strong as the best entailment win. 190/210 -> 201/210 overall; held-out 60/72 -> 66/72; negations caught 22/35 -> 30/35. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The badge showed the mean entailment as a percentage, so an answer with
one contradicted sentence could still read 73%. It now shows the level the
verdict assigns: High when every sentence is supported, Medium when most
are, Low when one is unsupported or contradicted.
Sentence citations and per-source contribution are hidden behind
showCitations (off) until they are ready. A small number now matches its
spelled-out word, so "2 phases" is no longer rejected when the book says
"two". Replaces the unused answerStatusLabel ('Supported').
Note: the previous merge commit on this branch says answers were bucketed
into low/medium/high. That was not true until this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…reach High
High required every sentence at 0.85+ entailment. Real llama3.2 answers
paraphrase the textbook and add a linking sentence, so 1 of 15 correct
answers read High and some read Low.
Confidence is now the share of claim credit, as grounding checkers report
it (RAGAS faithfulness, Google check-grounding): 1 per claim at 0.8+, 0.5
at 0.7+, High at 2/3, Medium at 1/3; a contradicted claim still forces Low.
Negated claims score in the same 0.7-0.8 band as paraphrases, so half
credit also needs the claim and its evidence to agree on negation.
Claims are also stripped of attribution and connectives ("According to
passage [4],", "In other words,") before checking; the bracketed passage
number used to read as a missing term and zero the claim.
15 real llama3.2 answers: High 1 -> 9. BuzzDB benchmark: 201 -> 197/210,
negations caught 30 -> 28/35; budgets raised once with the reason recorded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"as shown in Section 4.7.3" or "(see Figure 18.8)" is a reference, not a fact, but its number was read as a number the claim asserts; when the passages did not contain it, the whole claim scored 0. Section, figure and chapter references are now dropped before the check. Page and table numbers are left alone, since "a page holds 4 KB" or "table 2" can be the fact. Rescoring 50 real llama3.2 answers changed no badge; the NLI benchmark is unchanged at 197/210. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
d5c7585 to
dcf51fd
Compare
No description provided.