Final Exam: RAG Systems Complete
Comprehensive assessment covering all 10 modules. 50 questions testing your mastery of RAG concepts and implementation.
Instructions
- Answer all 50 questions
- Each question is worth 2 points
- Passing score: 70% (70 points)
- Time suggested: 60-90 minutes
Part 1: Fundamentals (Q1-10)
Q1: What does RAG stand for and what is its main purpose?
Answer: Retrieval-Augmented Generation. It enhances LLMs by providing relevant retrieved documents as context for more accurate, grounded responses.
Q2: Name the three main phases of a RAG pipeline.
Answer: 1) Indexing (offline), 2) Retrieval (online), 3) Generation (online).
Q3: What are the four main problems RAG solves for LLMs?
Answer: 1) Hallucinations, 2) Knowledge cutoff, 3) No private data access, 4) Token/context limits.
Q4: Explain the difference between RAG and fine-tuning.
Answer: RAG adds knowledge through retrieval (dynamic, can be updated); fine-tuning changes model behavior (static, requires retraining).
Q5: What is the typical chunk size and overlap for RAG systems?
Answer: Chunk size: 500-1500 characters; Overlap: 10-20% of chunk size (e.g., 200 characters for 1000 char chunks).
Q6: What are embeddings and why are they important for RAG?
Answer: Dense vector representations of text that capture semantic meaning. They enable similarity search to find relevant documents.
Q7: What is cosine similarity and what is its range?
Answer: A metric measuring the angle between vectors; range is -1 to 1 (1 = identical direction, 0 = orthogonal).
Q8: Name three popular vector databases.
Answer: Any three of: Pinecone, Weaviate, Chroma, Qdrant, Milvus, pgvector, FAISS.
Q9: What is HNSW and why is it commonly used?
Answer: Hierarchical Navigable Small World - a graph-based algorithm for fast approximate nearest neighbor search with good quality.
Q10: What is a retriever in LangChain?
Answer: A component that takes a query and returns relevant documents from a vector store.
Part 2: Document Processing (Q11-18)
Q11: Name three types of document loaders in LangChain.
Answer: Any three of: PyPDFLoader, TextLoader, CSVLoader, WebBaseLoader, UnstructuredPDFLoader, Docx2txtLoader.
Q12: What is RecursiveCharacterTextSplitter and how does it work?
Answer: A text splitter that tries to split by paragraph, then sentence, then word, then character - following a hierarchy of separators.
Q13: Why is chunk overlap important?
Answer: It prevents information from being cut off at chunk boundaries and preserves context between chunks.
Q14: How should tables in documents be handled?
Answer: Keep tables as single chunks to preserve structure; use specialized extractors like Camelot for PDFs.
Q15: What is OCR and when is it needed?
Answer: Optical Character Recognition - needed for scanned PDFs and images where text isn't machine-readable.
Q16: What metadata should be preserved during document processing?
Answer: Source file, page number, chunk index, date, author, document type, and any domain-specific attributes.
Q17: What is semantic chunking?
Answer: Splitting text based on semantic similarity rather than fixed character counts.
Q18: How do you handle very large CSV files?
Answer: Process in batches using pandas chunksize or similar streaming approaches.
Part 3: Embeddings (Q19-25)
Q19: What determines embedding quality?
Answer: Model architecture, training data, and dimension count. Use MTEB benchmarks to compare.
Q20: Name two commercial and two open-source embedding models.
Answer: Commercial: OpenAI text-embedding-3, Cohere embed-v3. Open source: BGE, E5, Sentence Transformers.
Q21: What is MTEB?
Answer: Massive Text Embedding Benchmark - the standard benchmark for comparing embedding model quality.
Q22: When should you consider self-hosting embeddings?
Answer: High volume (>10M docs), data privacy requirements, latency requirements (<20ms), or cost optimization.
Q23: What happens if you use different embedding models for indexing and querying?
Answer: Retrieval fails or performs very poorly - vectors must be from the same model to be comparable.
Q24: What is the tradeoff with higher embedding dimensions?
Answer: Better quality but more storage and slower search.
Q25: How does dimension reduction work in text-embedding-3-large?
Answer: The dimensions parameter truncates the vector while maintaining quality through training-time optimization.
Part 4: Retrieval (Q26-32)
Q26: What is hybrid search?
Answer: Combining keyword search (BM25) with semantic vector search for better results.
Q27: What is MMR and when should you use it?
Answer: Maximum Marginal Relevance - balances relevance with diversity. Use when you want varied results.
Q28: What is reranking?
Answer: Re-scoring initial retrieval results with a more powerful model (cross-encoder) for better accuracy.
Q29: What is Reciprocal Rank Fusion?
Answer: A method to combine rankings from multiple retrievers into a single final ranking.
Q30: What is pre-retrieval vs post-retrieval filtering?
Answer: Pre-retrieval: filter before search (more efficient). Post-retrieval: filter results after search.
Q31: What is HyDE?
Answer: Hypothetical Document Embeddings - generate a hypothetical answer, embed it, and use that for retrieval.
Q32: What is self-query retrieval?
Answer: Using an LLM to extract both semantic query and metadata filters from natural language input.
Part 5: Frameworks & Evaluation (Q33-40)
Q33: What is LCEL in LangChain?
Answer: LangChain Expression Language - a composable way to build chains using the pipe (|) operator.
Q34: What is the difference between LangChain and LlamaIndex?
Answer: LangChain: general-purpose chain building. LlamaIndex: focused on data indexing and querying.
Q35: What does the "stuff" chain type do?
Answer: Puts all retrieved documents into a single prompt - best for small contexts.
Q36: What is RAGAS?
Answer: Retrieval Augmented Generation Assessment - framework for automated RAG evaluation.
Q37: Name four RAGAS metrics.
Answer: Faithfulness, answer_relevancy, context_precision, context_recall.
Q38: What does faithfulness measure?
Answer: Whether the generated answer is grounded in (supported by) the retrieved context.
Q39: What is LangSmith used for?
Answer: Tracing, debugging, and monitoring LangChain applications.
Q40: How do you debug "wrong documents retrieved"?
Answer: Print retrieved docs, check chunk size, verify embedding model, test with known queries.
Part 6: Production (Q41-50)
Q41: What is semantic caching?
Answer: Caching results for semantically similar queries (not just exact matches).
Q42: What is multi-tenancy in RAG?
Answer: Serving multiple organizations from one system with data isolation.
Q43: Name two approaches to tenant isolation.
Answer: 1) Namespace isolation, 2) Metadata filtering with tenant_id.
Q44: What should be tracked for billing in SaaS RAG?
Answer: Queries, tokens used, documents indexed, storage used.
Q45: What is prompt injection?
Answer: Malicious input designed to manipulate the LLM's behavior.
Q46: How do you prevent prompt injection?
Answer: Input validation, sanitization, and secure prompt design.
Q47: What should be audit logged in production RAG?
Answer: User ID, query, documents accessed, timestamp, response metadata.
Q48: At what scale should you consider sharding?
Answer: Around 1M+ documents typically require sharding.
Q49: What is queue-based indexing?
Answer: Adding documents to a queue for asynchronous background processing.
Q50: What's the main factor in RAG response quality?
Answer: Retrieval quality - if you retrieve the right documents, the LLM can generate good answers.
Scoring
| Score | Grade | Assessment |
|---|---|---|
| 90-100 | A | Excellent - RAG expert |
| 80-89 | B | Good - Solid understanding |
| 70-79 | C | Pass - Review weak areas |
| 60-69 | D | Below passing - More study needed |
| <60 | F | Significant review required |
🎉 Congratulations!
If you've completed this exam and the course, you now have comprehensive knowledge of RAG systems. Continue building and experimenting!