HeatDoc

The problem

Public heat-wave guidance in India is published as PDFs, and the corpus is mixed at the page level, not the document level: only 2 of 34 documents are wholly scanned, yet a third of all pages still need OCR. A document-level "is this scanned?" flag gets most of the corpus wrong, and a system that answers confidently from bad OCR text is worse than one that admits it doesn't know.

Approach

Built with Atharva Indulkar as a coursework project (Generative AI, Internal Assessment I) at K. J. Somaiya. HeatDoc routes per page rather than per document: 875 pages across 34 documents, 32.9% of them routed to OCR. It keeps word-level OCR confidence and bounding boxes all the way through the pipeline, so a citation can point at an exact region of an exact page. Retrieval is a FAISS-plus-BM25 hybrid over 1,053 chunks; answers are numbered against retrieved evidence and verified before they're shown. When the best-matching passage falls below a fixed corpus-wide similarity floor (0.62), the system abstains instead of guessing.

Results

On a 60-question gold set (47 answerable, 13 designed to be unanswerable), the abstention mechanism is the headline behavior: questions outside the corpus get a refusal with the measured similarity score that triggered it, not a confident wrong answer.

What broke

A fine-tuned OCR corrector produced no measurable downstream benefit: on every metric, the raw uncorrected text fell inside the corrected version's 95% confidence interval. The reason: the two indexes genuinely differ, but the cross-encoder reranker absorbs almost all of it. BM25 scores moved on 60 of 60 questions when the underlying text changed, but reranked positions moved on 0 of 60. A separate sweep found that a plain confidence threshold doesn't predict which OCR corrections get damaged; a length floor does: filtering out corrections shorter than 15 characters cut severe-damage cases far more than any confidence cutoff, because the worst OCR output wasn't damaged English, it was chart axes and non-English text OCR'd with the wrong language model.