RAG Interview Questions for Senior (7+ Years)

Use RAGAS (RAG Assessment) framework which measures: (1) Faithfulness — are answers grounded in retrieved context? (2) Answer relevancy — does the answer actually address the question? (3) Context precision — are retrieved chunks relevant? (4) Context recall — were all necessary chunks retrieved? Run automated evaluation with a test set of 100+ question-answer pairs. Set up LangSmith to track these metrics over time as you change chunking or retrieval strategies.

Reranking is a second-pass filtering step after initial retrieval. Retrieve top-20 chunks with fast vector search, then use a cross-encoder model (Cohere Reranker, BGE-Reranker) to score all 20 against the query and return top-5. Use reranking when: query-document relevance is nuanced, initial retrieval precision is low, or accuracy is more important than latency. Adds ~200ms latency but significantly improves answer quality.

HyDE generates a hypothetical answer to the query, embeds that hypothetical answer, and uses it for retrieval instead of the original query. This works because the hypothetical answer is stylistically closer to real documents in the knowledge base. Implement: query → LLM generates hypothetical answer → embed hypothetical answer → search vectorstore → retrieve real chunks → answer with real chunks. Improves recall for complex queries by ~15–25%.

Use specialized loaders: PyMuPDF for PDF text extraction, UnstructuredLoader for mixed content, AWS Textract for complex layouts. For images: extract with Unstructured + use GPT-4V to generate text descriptions of images. For tables: use TableTransformer or extract HTML representation. Store image descriptions as text chunks with metadata linking to original page. Consider multi-modal embeddings (CLIP) for image-first retrieval.

The context window problem: you retrieve 20 relevant chunks but they exceed the LLM's context limit. Solutions: (1) Contextual compression — use LLM to extract only the relevant sentences from each chunk, (2) Map-reduce — answer per-chunk then synthesize, (3) Refine chain — iteratively refine answer with each chunk, (4) Sentence window retrieval — embed sentences but return surrounding paragraphs. Always monitor average retrieved context length and set hard limits.

Use metadata filtering: tag each document with user_id at ingestion time, then filter by user_id at retrieval. In Pinecone: upsert with metadata={user_id: "123"}, query with filter={user_id: "123"}. In Chroma: use where={"user_id": "123"} in query. Namespace isolation (Pinecone namespaces) provides stronger isolation. Never let user A's filter accidentally retrieve user B's data — test this explicitly in your test suite.