PRISM

The problem

Hallucination-mitigation techniques for LLMs have split into distinct families, such as retrieval-based correction, contrastive decoding, and chain-of-verification, and each wins on a different slice of queries. Under one fixed backbone, the benchmark winner changed identity on three of six development-to-test transitions. Deployment traffic doesn't arrive pre-sorted by which family it needs, either: it's a mixture, and no single fixed technique fits all of it.

Approach

PRISM is a policy-routed system: a fine-tuned neural encoder scores each incoming query along four risk axes, then routes it to one of four mitigation strategies: direct generation, CRAG-style corrective retrieval, DoLa contrastive decoding, or factored chain-of-verification, all on one shared, quantized 7B backbone (Mistral-7B-Instruct) so every strategy is compared under the same conditions. A calibrated verification and confidence layer scores the output afterward, with an optional abstention mechanism for when an answer isn't worth giving. This is my final-year major project, built with two co-authors and a faculty mentor. It's submitted to icSoftComp 2026 (Springer CCIS) and currently in double-blind review: no decision yet.

Results

On a mixed-workload stream pooling six sources, PRISM posts the highest pooled accuracy of the seven systems tested (0.528), ahead of every fixed-strategy baseline and a trained routing baseline, with the pre-registered pairwise bootstrap intervals excluding zero (+0.030 to +0.050 over the alternatives). It gets there at roughly half the tokens per query of the two costliest baselines. On pure, single-domain benchmarks it statistically ties whichever technique wins after the fact, everywhere (regret at most 0.020), where every fixed technique risks at least 0.115 regret somewhere.

What broke

Fixing a formatting bug in the CoVe strategy's revision stage raised PubMedQA's single-word-answer compliance from 24% to 83%, and lowered the benchmark's measured accuracy from 0.628 to 0.504. The earlier, higher number wasn't a real capability difference; the scorer had been silently rewarding malformed short answers. Caught it, fixed the scorer, reported the corrected number.