20 viva questions every RAG project should be ready for

A viva panel doesn't need to be an expert in your specific project to ask a question that exposes whether you actually understand it. The questions below are the ones that come up again and again for RAG-based projects — chatbots, document Q&A tools, anything that retrieves and then answers. Knowing the shape of these questions in advance is most of the battle.

Conceptual

1. What is RAG, and why not just use ChatGPT directly? Because a general model has never seen your specific content. RAG retrieves the relevant passage first, then asks the model to answer from it — grounding the answer instead of relying on the model's memory.

2. What is an embedding? A numerical vector representation of text, positioned so that text with similar meaning ends up close together in that vector space — which is what allows searching by meaning instead of exact keywords.

3. What is hallucination, and how does your project reduce it? A confident but false answer, typically produced when a model is asked about something outside its training data. Explicitly instructing the model to answer only from retrieved context — and to say when the answer isn't present — is the direct mitigation.

4. Why do you need chunking? Why not embed the whole document as one vector? A single vector for an entire document loses fine-grained detail — you couldn't tell which part matched a query. Chunking also respects the model's context-length limits at answer time.

Architecture

5. Walk me through what happens end to end, from upload to answer. Have this memorized as a clean five-step story: source content in, chunked, embedded, indexed; question in, embedded, matched against the index, top chunks retrieved, sent to the model with the question, answer generated.

6. Why did you choose your specific chunk size? There's a real tradeoff here — too large and retrieval gets imprecise, too small and you lose surrounding context. Be ready to say what you tried and why you landed where you did, even if the honest answer is "empirically, by testing a few sizes."

7. Why FAISS (or whichever vector library you used)? Fast, free, runs locally without a server, and sufficient at the scale of a single-user or small project. Contrast with a hosted vector database, which adds cost and operational complexity that isn't justified at this scale.

8. What embedding model did you use, and why that one? If you used a compact model like all-MiniLM-L6-v2: it runs on CPU without a GPU, and its size-to-quality tradeoff is appropriate for a project at this scale, versus a larger model that would be more accurate but slower and often GPU-dependent.

Technical deep-dive

9. What is hybrid retrieval, and why does it matter? Combining exact keyword/identifier matching with vector similarity search. It matters because embeddings struggle to distinguish similar-looking specifics — two different years, two similar-sounding names, an exact quoted phrase — that a literal match catches immediately.

10. How do you decide how many chunks to retrieve per question? This is usually a fixed number (a "k" value) chosen as a tradeoff: too few and you might miss the answer, too many and you dilute the context with irrelevant material, which can actually make the model's answer worse, not better.

11. What happens if the retrieved chunks don't actually contain the answer? The model should be explicitly instructed to say it doesn't have enough information, rather than guessing — this is a specific, testable behavior worth demonstrating live if asked.

12. How would you evaluate whether your retrieval is actually good? Precision@k and recall@k against a labeled set of question-answer pairs with known correct sources — worth mentioning even if you only did informal manual testing, since it shows you know what rigorous evaluation would look like.

13. What's the difference between your system and a keyword search (like Ctrl+F)? Keyword search only matches exact words; semantic search matches meaning, so a question phrased differently from the source text can still retrieve the right passage.

Testing

14. How did you test this system? Have specific test cases ready: a narrow factual question, a broad summary question, a question with no answer in the source, an edge case specific to your domain (a video with no captions, a PDF with no text layer, etc.).

15. What's a case where your system fails, and why? Naming a real, honest limitation is a stronger answer than pretending there isn't one — panels notice when a limitations section is suspiciously empty.

16. How do you know your answers are actually grounded, not hallucinated? Citations — if every answer is traceable to a specific chunk, source, or timestamp, that traceability is itself the evidence.

Tricky / comparative

17. Why not just increase the context window and skip retrieval entirely? For short documents this can work, but it doesn't scale — longer sources exceed context limits, and even within limits, sending everything is slower and more expensive than retrieving only what's relevant.

18. How would this scale to thousands of documents instead of a handful? An exact nearest-neighbor index becomes slow at scale; an approximate index (like FAISS's IVF or HNSW variants) trades a small amount of accuracy for much faster search.

19. What would you improve if you had another month? Have two or three specific, technically grounded answers ready — re-ranking retrieved results with a cross-encoder, calibrating chunk size against a real benchmark, adding a broader evaluation suite — not just "make it faster" or "make it better."

20. If I gave you a completely different kind of document right now, would your system work? Talk through what would and wouldn't transfer — the embedding and retrieval logic is generally domain-agnostic, but the source-to-text extraction step is often what needs to change.

Walking in with clear, specific answers to these — not memorized scripts, but a real understanding of why each design decision was made — is what actually reads as confidence to a panel.