Applied LLMs & GenAI 🤖

— system design side of AI interviews —

#placements-2026 #RAG #fine-tuning #high-priority

①Prompt Engineering Techniques

Zero-shot — just ask. No examples given, model relies purely on pre-trained knowledge + instruction. Fast, cheap, but weaker on tricky/unfamiliar formats.
Few-shot — show 2–5 example (input → output) pairs inside the prompt before the real question. Model pattern-matches the desired format/style. Great for niche formatting, classification labels, tone control.
Chain-of-Thought (CoT) — ask the model to "think step by step" before giving the final answer. Forces intermediate reasoning tokens → big accuracy gains on math/logic/multi-hop questions.
ReAct (Reason + Act) — interleaves reasoning with tool calls: Thought → Action (call a tool/API) → Observation (tool result) → repeat → Final Answer. This is the backbone of most modern agents.

fig 1 — from zero-shot to agentic ReAct loops

interview line: "CoT improves reasoning; ReAct grounds reasoning in real tool outputs — that's what turns an LLM into an agent"

②Embeddings & Vector Databases

An embedding = a piece of text squeezed into a dense numeric vector where "semantic closeness" ≈ "geometric closeness." Vector DBs store millions of these and answer: "give me the k nearest vectors to this query."

Distance / similarity metrics —

fig 2 — cosine ignores scale, dot product rewards larger/confident vectors, euclidean = literal gap

Cosine — most common for text embeddings; robust when vector magnitude carries no meaning (normalized embeddings).
Dot product — equivalent to cosine if vectors are normalized; faster to compute (no division); used when magnitude encodes confidence/popularity.
Euclidean (L2) — sensitive to magnitude; common in image/clustering contexts, less common for un-normalized text embeddings.

Vector Indexing (how to search millions of vectors fast) —

fig 3 — exact search → clustered search → graph-navigated search (speed ↑, exactness trade-off ↑)

Index	Idea	Trade-off
Flat	Linear scan, exact k-NN	100% accurate, but slow / doesn't scale past ~1M vectors
IVF	k-means cluster centroids; only probe nearest few clusters	Much faster; accuracy depends on nprobe (clusters searched)
HNSW	Multi-layer proximity graph; greedy hop from sparse top layer down to dense bottom layer	Very fast + high recall; more memory, slower to build/insert

③RAG Pipeline Architecture

RAG = Retrieval-Augmented Generation: fetch relevant external knowledge at query time and feed it to the LLM, instead of relying only on frozen training-time knowledge.

fig 4 — retrieve wide (recall), re-rank narrow (precision), then synthesize

Chunking strategies — how you split docs before embedding decides retrieval quality:

Fixed-size — split every N tokens (e.g., 512) with some overlap; simple, but can cut mid-thought.
Recursive / structure-aware — split on paragraph → sentence boundaries, respecting headings; keeps semantic units intact.
Semantic chunking — split where embedding similarity between adjacent sentences drops, i.e. where "topic" actually shifts.
Sliding window overlap — small overlap between chunks (~10-20%) so context isn't lost right at a chunk boundary.

Why re-ranking? Vector search is fast but approximate — it optimizes recall over a huge corpus using cheap similarity. A re-ranker (usually a cross-encoder that jointly reads query+passage) is slower but far more accurate, applied only to the top-k shortlist to pick the best few for the final prompt.

📏 RAGAS evaluation framework

Faithfulness — is the generated answer actually grounded in the retrieved context (no hallucinated facts)?
Answer Relevance — does the answer actually address the user's question (not just faithful, but on-topic)?
(also commonly paired with) Context Precision / Context Recall — did retrieval fetch the right, and only the right, passages?

gotcha: faithfulness ≠ relevance — an answer can be 100% grounded in context yet not actually answer the question asked!

④Customization Decision Framework

Interview favorite: "Would you prompt-engineer, RAG, fine-tune, or pre-train?" — answer by asking what's actually missing.

fig 5 — pick the cheapest lever that actually fixes the gap

PEFT / LoRA / QLoRA intuition — full fine-tuning updates all model weights (huge, expensive, needs lots of GPU memory). PEFT (Parameter-Efficient Fine-Tuning) freezes the base model and trains a tiny number of extra parameters instead.

LoRA — freeze original weight matrix W; learn a small "correction" as two tiny low-rank matrices A·B added on top (W + A·B). Only A, B are trained — a fraction of the parameters, so it's cheap and the adapter is a small file you can swap in/out.
QLoRA — same idea, but the frozen base model is quantized to 4-bit first to shrink memory further, then LoRA adapters (kept in higher precision) are trained on top — lets you fine-tune huge models on a single consumer GPU.

one-liner: "LoRA = train a small side-patch instead of the whole model; QLoRA = do that on a compressed (4-bit) base model to fit on less GPU"

⑤Model Compression — Quantization

Quantization = storing weights with fewer bits → smaller model, faster inference, small accuracy loss.

fig 6 — the precision/size/speed trade-off ladder

FP16 / BF16 — the default for GPU training/inference; halves memory vs FP32 with almost no quality loss.
INT8 — weights mapped to 256 discrete levels; common "safe" compression for serving on cheaper hardware.
INT4 — 16 discrete levels; aggressive, needs careful calibration (e.g. GPTQ/AWQ) to avoid quality collapse — but lets 7B–70B models run on consumer GPUs / even CPUs.
GGUF — a file format (used by llama.cpp) packaging quantized weights (various bit-widths) + metadata + tokenizer into one portable file for easy local/CPU inference — not a quantization method itself, but how quantized models are shipped and loaded.

⑥Tooling Landscape

	LangChain	LlamaIndex
Core focus	General orchestration — chains, agents, tool-calling, memory, many integrations	Data-centric — ingesting, indexing, and retrieving your documents efficiently
Mental model	"Lego blocks" to wire an LLM to tools, APIs, and multi-step logic/agents	"Smart librarian" — specializes in turning messy docs into query-ready indexes
When to reach for it	Complex agent workflows, multiple tools, conversational chains	RAG-heavy apps where retrieval quality over your own data is the bottleneck

In practice teams often combine both — LlamaIndex for the retrieval/indexing layer, LangChain (or a custom agent loop) for orchestration on top.

⑦End-to-End RAG Pipeline (with re-ranker) — pseudo-code

Two phases: offline indexing (once) and online query (every request).

# ---- OFFLINE: build the index ---- for doc in corpus: chunks = chunk_document(doc, size=512, overlap=50) for chunk in chunks: vector = embed_model.encode(chunk.text) vector_db.upsert(id=chunk.id, vector=vector, metadata={text: chunk.text, source: doc.id}) # ---- ONLINE: answer a user query ---- def rag_answer(user_query): q_vector = embed_model.encode(user_query) # 1. Retrieve — cast a wide, cheap net candidates = vector_db.search(q_vector, top_k=20) # 2. Re-rank — slow but precise cross-encoder scoring scored = [(c, reranker.score(user_query, c.text)) for c in candidates] top_chunks = sort_desc(scored)[:5] # 3. Synthesize context + prompt context = join([c.text for c in top_chunks], sep="\n---\n") prompt = build_prompt(system=RAG_SYSTEM_PROMPT, context=context, question=user_query) # 4. Generate answer = llm.generate(prompt) # 5. (optional) Evaluate before returning / logging metrics = ragas_eval(question=user_query, answer=answer, context=top_chunks) # faithfulness, relevance return answer, metrics

🧩 why each step earns its place

Retrieve wide (k=20) — vector search alone is imprecise; over-fetch to protect recall.
Re-rank narrow (→5) — cross-encoder is too slow to run on the whole corpus, but cheap on 20 candidates; boosts precision right before it matters most.
Synthesize — dedupe/order chunks, respect the LLM's context window, cite sources if needed.
Evaluate — RAGAS-style scoring in the loop (or offline on a test set) catches hallucination/off-topic drift before it ships.