Transformers & Attention 🧠

— the notes that actually get asked in interviews —
#AI-ML-2026 #must-know #concepts-not-code

①Why did we need Attention? (RNN's problem)

Before Transformers, sequence models = RNNs / LSTMs. They read a sentence one word at a time, carrying a "memory" (hidden state) forward.

RNN — sequential, one word waits for the last "The" "cat" "sat" "on..." last word by word 50, info from word 1 has faded — "vanishing memory" + can't parallelize → slow training (must wait your turn!)
fig 1 — the sequential bottleneck
💡 the fix

Let the decoder "look back" at all encoder words directly, and learn which ones matter right now — that's Attention. Transformers (Vaswani et al., 2017) took this idea and threw away recurrence entirely: "Attention is All You Need."

interview line: "Attention removes the sequential bottleneck & lets every token see every other token directly, in parallel"

②Self-Attention — the Query/Key/Value intuition

Think of it like a library search, done by every word, for every word:

Every word compares its Query against every other word's Key (a dot-product "match score"), turns those scores into weights (softmax), then takes a weighted average of all Values — that weighted mix is the word's new, context-aware representation.

Sentence: "The animal didn't cross the street because IT was tired" — what does "IT" refer to? Q: "IT" asks — K: "animal" K: "street" K: "tired" 0.7 (high match!) 0.05 0.25 V: "animal" V: "street" V: "tired" weighted sum (mostly "animal") → new vector for "IT" ✓ resolved!
fig 2 — Q from "IT" matches K of "animal" strongly → output leans on V of "animal"
🔑 in one line

score = Q·K (similarity) → softmax → weights → weighted average of V = output. Every word does this for every other word, all at once (matrix ops), which is why it's parallel and captures long-range links in a single step — no chain of 50 hops needed.

③Multi-Head Attention

One attention "head" learns one kind of relationship (say, subject↔verb). Language has many relationship types at once — so we run several attention heads in parallel, each with its own learned Q/K/V projections, each free to specialise.

same sentence, viewed through different "lenses": Head 1 tracks pronouns Head 2 subject–verb link Head 3 nearby adjectives Head 8... long-range topic concat + linear mix → one rich output per word
fig 3 — heads run in parallel → concatenated → projected back down
why it works: 1 head = 1 view; 8 heads = 8 simultaneous views, richer representation

④Positional Encoding — why order matters

Self-attention on its own is a "bag of words" — it computes weighted sums with no idea of order. "Dog bites man" and "Man bites dog" would look identical to raw attention!

Fix: inject a unique position signal (built from sine/cosine waves of different frequencies, or learned) and add it to each word's embedding before attention — so the model gets both "what" and "where" at once.

word embedding + position signature = position-aware input "dog" embed + pos = 2 wave = "dog" @ pos 2 (unique per position, smooth so nearby = similar)
fig 4 — position encoding gives attention a sense of sequence

⑤The Full Transformer — Encoder–Decoder

Original Transformer (built for translation) = a stack of Encoders feeding a stack of Decoders.

ENCODER stack (×N) Self-Attention (sees whole sentence, both directions) Feed-Forward Layer ↕ repeated N times, + residual/norm encodes input → rich "understanding" vectors passed as K, V DECODER stack (×N) Masked Self-Attention (only looks left / past words) Cross-Attention Q = decoder, K/V = encoder Feed-Forward Layer ↕ repeated N times generates output, one token at a time Input: "Le chat est noir" Output: "The cat is black" encoder understands the source; decoder writes the target, peeking at the encoder via cross-attention at every step
fig 5 — encoder = understand, decoder = generate (with a bridge between them)
⚙️ 3 flavours of attention here
  • Encoder self-attention — bidirectional, sees full input
  • Decoder masked self-attention — causal, can't peek at future tokens (else it'd cheat!)
  • Cross-attention — decoder's Query meets encoder's Key/Value — "translation alignment"

⑥BERT vs GPT — the classic interview flip side

Both are built only from one half of the original transformer:

BERT = Encoder-only The [MASK] sat looks BOTH ways ↔ predicts masked word using full context GPT = Decoder-only The cat ? looks ONLY left → predicts next word from past words only
fig 6 — bidirectional understanding vs left-to-right generation
BERTGPT
Built from Encoder stack only Decoder stack only (self-attn part, no cross-attn since there's no separate encoder)
Attention direction Bidirectional — sees full sentence, left & right of a word Unidirectional / causal — masked, can only see past tokens
Pre-training task Masked Language Modeling — guess the hidden word (+ Next Sentence Prediction) Next-token prediction — guess the next word, repeatedly
Core strength Understanding — building a deep contextual representation of given text Generation — producing new, coherent text token by token
Typical use-cases Classification, NER, sentiment, question-answering (extractive), sentence embeddings Chatbots, text completion, summarization, code generation, open-ended writing
Analogy A reader who highlights & studies a whole page at once A writer composing a sentence word by word, never peeking ahead
gotcha to remember: BERT can't generate text (no causal masking = would cheat by seeing the answer); GPT can still "understand" via next-token prediction, just less directly than BERT
— end of notes · review fig 1→6 before the interview, you've got this ✏️ —