RAG Retrieval Debugging: Metadata & Stale Documents
A synthetic RAG-style policy assistant misses indexed policies even though the documents are present.
These results were measured only on 4 controlled retrieval questions. They are not client results, live production improvements, or evidence of a formal security audit.
Reproduced root cause
The baseline retriever expected a single document-type field, while upstream records used three schema variants. It also admitted older or draft records into search results.
Focused correction
Normalize metadata aliases, require active status and the requested version year before ranking, and retain a test for every previously missing policy.
Regression evidence
5 tests: bug reproduced; normalization; active/version constraints; source ranking; product boundary.
All listed tests executed and passed on November 12, 2026. Source fixtures, baseline code, corrected code, and test scripts are available in the downloadable project package.
Limits and responsible interpretation
This fixture uses transparent lexical token overlap. It does not run embeddings, a vector database, a production index or an LLM.