EDA & Feature Engineering

practical, placement-focused notes

Missing Value Strategies

The choice depends on how much is missing, why it's missing, and what the downstream model needs.

Drop

Missing < ~5% of rows, and missingness looks random (MCAR). Or drop a whole column if >50–60% missing and it can't be reasonably imputed.

Impute

Numeric: mean (symmetric data), median (skewed / has outliers). Categorical: mode, or a new "Missing" category. Advanced: KNN or model-based imputation when missingness is meaningful and data volume allows it.

Flag

Add a binary is_missing indicator column alongside the (imputed) value — useful when the fact that it's missing is itself predictive (e.g. "income not disclosed").

Quick decision rule: Is missingness random and small? → drop rows. Is the column mostly empty? → drop column. Otherwise → impute, and if you suspect the missingness itself carries signal, add a flag too.

Watch for: MCAR (missing completely at random) vs MAR (depends on other observed features) vs MNAR (depends on the missing value itself, e.g. high earners skip the income field). MNAR is the case where dropping or naive imputation can bias your model.

Outlier Detection

IQR method

Outlier if value < Q1 − 1.5×IQR or > Q3 + 1.5×IQR, where IQR = Q3 − Q1. Robust to skew and extreme values — the default for non-normal data.

Z-score method

Outlier if |z| = |(x − mean) / std| > ~3. Assumes roughly normal distribution — misleading on skewed data since mean/std are themselves pulled by outliers.

Boxplot view: whiskers extend to 1.5×IQR from the box; points beyond (pink dots) flag as outliers.

Treat vs keep

Treat / cap / remove when the outlier is a data-entry error, a sensor glitch, or would disproportionately skew a distance- or mean-based model (linear regression, k-NN, k-means).
Keep (or specially model) when it's a genuine rare event that matters — fraud detection, high-value transactions, medical anomalies. Tree-based models (Random Forest, XGBoost) are naturally robust to outliers, so aggressive removal there can throw away signal.

Interview line: "I wouldn't blindly delete outliers — first I'd check if they're errors or real signal, since fraud/churn problems are often *defined* by their outliers."

Encoding Techniques

Method	How it works	Use when
One-Hot	New binary column per category	Low cardinality (few unique values), no ordinal relationship, linear/distance-based models.
Label	Each category → an integer	Ordinal categories with real order (e.g. Low/Medium/High), or tree-based models that split on thresholds anyway.
Target	Category → mean of target for that category	High-cardinality categoricals (zip code, product ID) — but must use CV/smoothing to avoid leakage.

import pandas as pd

# One-hot encoding
df_ohe = pd.get_dummies(df, columns=['city'], drop_first=True)

# Label encoding
from sklearn.preprocessing import LabelEncoder
df['size_encoded'] = LabelEncoder().fit_transform(df['size'])

# Target encoding (with smoothing via category_encoders)
import category_encoders as ce
enc = ce.TargetEncoder(cols=['zip_code'], smoothing=10)
df['zip_encoded'] = enc.fit_transform(df['zip_code'], df['target'])

Leakage trap: fit target encoding only on the training fold (e.g. inside cross-validation), never on the full dataset — otherwise the target leaks into features used to predict it.

Label encoding trap: using it on a non-ordinal category (e.g. Red=0, Blue=1, Green=2) for a linear model implies a false order/distance — the model may treat Green as "twice" Blue. Fine for tree models, risky for linear/distance-based ones.

Scaling: Standardization vs Normalization

Standardization (Z-score)

(x − mean) / std → mean 0, std 1. Doesn't bound values to a fixed range. Preferred default for most ML algorithms, especially when data is roughly Gaussian.

from sklearn.preprocessing import StandardScaler
X_scaled = StandardScaler().fit_transform(X)

Normalization (Min-Max)

(x − min) / (max − min) → bounded to [0,1]. Sensitive to outliers (a single extreme value compresses everything else). Good when you need a fixed range, e.g. neural net inputs, image pixels.

from sklearn.preprocessing import MinMaxScaler
X_scaled = MinMaxScaler().fit_transform(X)

Which models actually need scaling: anything based on distance or gradient step size — k-NN, k-means, SVM, PCA, linear/logistic regression, neural networks. Tree-based models (Decision Tree, Random Forest, XGBoost, LightGBM) split on thresholds per feature independently, so they're scale-invariant and don't need it.

Feature Selection Methods

Correlation

Drop one of a pair of features highly correlated with each other (e.g. |r| > 0.9) — redundant info, hurts interpretability and can destabilize linear models.

VIF

Variance Inflation Factor quantifies multicollinearity across multiple features at once (not just pairs). VIF > 5–10 typically flags a feature worth dropping or combining.

Feature Importance

From tree-based models (Gini/split importance) or permutation importance — ranks features by how much they actually help predictions, model-driven rather than purely statistical.

Why it matters: multicollinearity doesn't hurt tree-model predictions much, but it destabilizes linear model coefficients (they become unreliable/swap signs) — check VIF especially before linear/logistic regression.

Feature Creation Ideas

Date/time parts: day-of-week, month, is_weekend, days_since_last_event, hour-of-day.
Ratios & interactions: price_per_sqft, income/expense ratio, feature_A × feature_B when their combined effect matters more than either alone.
Aggregations per entity: avg spend per customer, count of past transactions, rolling 7/30-day averages.
Binning: turn continuous age into age groups when the relationship with target is non-linear/step-like.
Text-derived: word count, presence of keywords, sentiment score, TF-IDF features.
Domain-specific flags: is_first_purchase, has_missing_value_in_X, is_holiday.

Interview line: "Good feature engineering usually beats a fancier model" — always tie new features back to domain reasoning, not just automated generation.

Handling Imbalanced Datasets

Technique	What it does	Trade-off
SMOTE	Synthesizes new minority-class samples by interpolating between existing ones	Can create unrealistic points in noisy/high-dimensional data; apply only on training fold.
Class weights	Penalize misclassifying the minority class more in the loss function	No new data created — often the simplest, safest first try.
Undersampling	Randomly drop majority-class rows to balance classes	Throws away data — risky when the majority class is already small in absolute terms.

from imblearn.over_sampling import SMOTE
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y)

# fit SMOTE ONLY on the training data, never on test data
sm = SMOTE(random_state=42)
X_train_res, y_train_res = sm.fit_resample(X_train, y_train)

Golden rule: resample after the train/test split, and only on the training set — resampling before splitting leaks synthetic/duplicated points across the split and inflates test performance.

Also remember: accuracy is a poor metric on imbalanced data — use precision/recall, F1, PR-AUC, or ROC-AUC instead.

★

Cheat Table: Encoding & Scaling by Model

Model	Encoding	Scaling
Linear / Logistic Regression	One-hot (low cardinality); target encoding (high cardinality)	Standardization — required
k-NN	One-hot	Standardization or Min-Max — required (distance-based)
SVM	One-hot	Standardization — required
k-Means / PCA	One-hot (rarely used directly on categoricals)	Standardization — required
Decision Tree / Random Forest	Label encoding is fine	Not needed — scale-invariant
XGBoost / LightGBM	Label or target encoding; native categorical support in LightGBM	Not needed
Neural Networks	One-hot or learned embeddings	Min-Max or Standardization — required (helps gradient descent converge)
Naive Bayes	Label / one-hot depending on variant	Not needed for categorical NB; standardization for Gaussian NB helps stability