ML system design — notes
the "senior interview" phase · goal: a repeatable framework you can apply cold to any unseen prompt, not memorized answers
01 the universal framework
walk through these 7 steps out loud, in order, for any "design a system that does X" question
1· clarify requirements
→
2· data
→
3· feature engineering
→
4· model choice
5· evaluation
→
6· deployment (serving)
→
7· monitoring & feedback loop
↺ back to data
1 · clarify requirements
- what's the business goal, not just the ML goal (e.g. "reduce churn" not "predict churn")
- scale: how many users/requests?
- latency budget: real-time or can it wait?
- constraints: interpretability needed? regulatory limits?
2 · data
- what signal exists today — logs, transactions, clicks?
- labels: do we have ground truth, or is it implicit (clicks, purchases)?
- volume, freshness, class balance
3 · feature engineering
- what signals actually predict the target
- batch-computed vs real-time (streaming) features
- avoid data leakage — no future info at train time
4 · model choice
- always start with a simple baseline (logistic regression / heuristic) — say this out loud
- add complexity only if the baseline can't hit the bar
- choice is driven by requirements from step 1, not by "which model is coolest"
5 · evaluation
- offline metrics (AUC, precision/recall, NDCG) on held-out data
- online: A/B test against the real business metric
- offline metric ≠ business impact — always tie back to step 1's goal
6 · deployment
- batch scoring vs real-time API — decided by the latency requirement
- shadow deploy / canary before full rollout
- rollback plan if metrics regress
how to actually use this live
always start by asking clarifying questions — don't jump to modeling. e.g. for "design a recommendation system": ask what platform, what's the objective (clicks vs watch time), how many users, real-time or batch. state assumptions explicitly if the interviewer doesn't answer, then walk the 7 steps. this alone separates strong candidates from weak ones.
02 trade-off toolkit
every case study below hinges on these three — name them explicitly when they come up
latency vs accuracy
real-time systems (fraud, ranking, ads) have a hard latency ceiling (often <100ms). the fix isn't "pick a worse model" — it's usually a two-stage design: cheap/fast model narrows candidates, expensive/accurate model only runs on the shortlist.
batch vs real-time inference
batch: precompute predictions on a schedule (nightly). cheaper, simpler, fine when staleness is ok (churn scores, email recs). real-time: compute on request. needed when context matters this second (fraud at checkout, search query).
complexity vs interpretability
deep models often win on raw accuracy but are black boxes. regulated / high-stakes domains (credit, fraud, healthcare) favor simpler, explainable models (logistic regression, GBM + SHAP) even at a small accuracy cost — explainability is a requirement, not a nice-to-have.
03 case studies — key decisions only
not full solutions — just the choices an interviewer wants to hear you justify
recommendation system — objective: engagement (CTR / watch time)
candidate generation
→
(1000s → 100s)
→
ranking model
→
(100s → top 10)
→
re-rank for diversity
- why 2-stagecan't run a heavy model over the entire catalog per request — cheap retrieval (collaborative filtering / embeddings + ANN search) first
- cold startnew users/items → fall back to popularity or content-based features
- evaloffline: precision@k, NDCG. online: A/B on CTR + session time (offline metrics often don't match business impact here)
- servingcandidate gen usually batch/precomputed; ranking stage real-time on the shortlist
fraud detection — objective: flag fraud at transaction time
transaction event
→
real-time feature lookup
→
model score (<100ms)
→
block / allow / review
- datahighly imbalanced (fraud is rare) — needs resampling, class weights, or anomaly-detection framing
- featuresreal-time aggregates (txn velocity, location jump) → needs a streaming feature store, not just batch features
- modelgradient boosted trees favored over deep nets — fast inference + easier to justify a "why was this blocked" decision
- evalprecision/recall trade-off explicitly — false positives annoy real customers, false negatives cost money. use PR-AUC, not ROC-AUC, given the imbalance
churn prediction — objective: flag users likely to leave in next N days
usage / billing logs
→
weekly batch feature build
→
score all users
→
retention team acts on top-k
- latencyno real-time need — batch scoring weekly is standard, so favor accuracy over speed here
- featurestrend signals matter more than snapshots — e.g. "usage dropped 40% over 4 weeks" beats "current usage = X"
- interpretabilitybusiness wants to know why a user is at risk to design the right intervention — favor SHAP/feature-importance-friendly models
- evalAUC is fine offline, but frame it as precision@top-k — retention budget is limited, so ranking the highest-risk users matters more than overall AUC
search ranking — objective: rank results by relevance to a query
query
→
retrieval (BM25 / embeddings)
→
learning-to-rank model
→
ranked results (<200ms total)
- why 2-stagesame pattern as recs — cheap retrieval over the whole index, expensive ranker only on top candidates, to fit the latency budget
- labelsclick logs give implicit relevance, but suffer from position bias (top result gets clicks just for being on top) — needs correction
- personalization vs generalizationper-user ranking improves relevance but adds complexity/cold-start risk vs a single global ranker
- evaloffline: NDCG, MRR. online: CTR, dwell time, A/B test — offline ranking quality doesn't always translate 1:1 to satisfaction
closing line that ties it together
notice the pattern across all four: clarify → simple baseline → two-stage if latency-bound → pick model complexity based on the interpretability/latency constraint → offline metric ≠ online success, so always mention the A/B test → close with monitoring for drift. say this pattern explicitly if a case study feels unfamiliar — it shows you're reasoning from the framework, not guessing.