50 Machine Learning Interview Questions (With Answers)

Machine learning interviews test your grasp of statistical foundations, algorithm internals, model evaluation, and practical engineering tradeoffs. This guide covers the 50 most common questions — with concise answers and real examples.

Quick reference

Fundamentals

1. What is the bias-variance tradeoff?

Bias is error from wrong assumptions (underfitting). Variance is error from sensitivity to training data fluctuations (overfitting). Total error = bias² + variance + irreducible noise.

• High bias → model too simple, misses patterns (linear model on non-linear data)
• High variance → model too complex, memorises noise (deep unpruned decision tree)
• Goal: find the sweet spot — complex enough to capture signal, simple enough to generalise

2. What is overfitting and how do you prevent it?

Overfitting occurs when a model performs well on training data but poorly on unseen data — it has memorised noise instead of learning the underlying pattern.

Prevention techniques:

• More data — the most effective remedy
• Regularisation — L1 (Lasso), L2 (Ridge), Elastic Net
• Dropout — randomly disable neurons during training (deep learning)
• Early stopping — halt training when validation loss increases
• Cross-validation — catch overfitting during model selection
• Simpler model — reduce parameters/depth
• Data augmentation — artificially expand training set

3. What is the difference between supervised, unsupervised, and reinforcement learning?

4. What is the curse of dimensionality?

As feature count (dimensions) grows, data becomes increasingly sparse. Consequences:

• Distance metrics lose meaning — everything becomes equidistant
• Exponentially more data needed to cover the space
• Models overfit more easily

Remedies: feature selection, PCA/dimensionality reduction, regularisation.

5. Explain the difference between a parametric and a non-parametric model.

6. What is the difference between classification and regression?

• Classification — predict a discrete label (spam/not-spam, digit 0–9)
• Regression — predict a continuous value (house price, temperature)

Some algorithms handle both (decision trees, neural networks). Others are specific (logistic regression → classification; linear regression → regression).

Algorithms

7. How does linear regression work? What are its assumptions?

Linear regression fits a hyperplane ŷ = Xβ + ε that minimises mean squared error (or equivalently, residual sum of squares).

OLS assumptions (GAUSS-MARKOV):

1. Linearity — relationship between X and y is linear
2. Independence — observations are independent
3. Homoscedasticity — constant variance of residuals
4. No multicollinearity — features not perfectly correlated
5. Normality of residuals (needed for inference, not prediction)

Violations are diagnosed with residual plots, VIF scores, and the Durbin-Watson statistic.

8. How does logistic regression differ from linear regression?

Logistic regression predicts probabilities using the sigmoid function σ(z) = 1 / (1 + e⁻ᶻ), keeping outputs in [0, 1]. It is trained by maximising log-likelihood (cross-entropy minimisation) rather than minimising MSE.

Despite the name it is a classification algorithm. Its decision boundary is linear in feature space — for non-linear boundaries you need feature engineering or a different model.

9. Explain how a decision tree splits data.

A decision tree greedily chooses the split that most reduces impurity at each node:

• Classification trees use Gini impurity or information gain (entropy)
• Regression trees use variance reduction (MSE decrease)

Gini impurity: G = 1 − Σ pᵢ² (0 = pure, 0.5 = maximum impurity for binary)

Information gain: IG = H(parent) − Σ (|child|/|parent|) × H(child)

Trees continue splitting until a stopping criterion (max depth, min samples per leaf, min impurity decrease).

10. What is a Support Vector Machine?

An SVM finds the hyperplane that maximises the margin — the distance to the nearest data points (support vectors) on each side.

• Hard-margin SVM — requires linearly separable data
• Soft-margin SVM — allows misclassifications via slack variables and penalty C
• Kernel trick — implicitly maps data to higher dimensions via kernel functions (RBF, polynomial, sigmoid) without computing the mapping explicitly

SVMs work well in high dimensions and with clear margin separation; they scale poorly to very large datasets.

11. How does KNN work? What are its tradeoffs?

K-Nearest Neighbours classifies a point by majority vote (or average) among its K closest training examples using a distance metric (usually Euclidean).

Choosing K: small K → high variance; large K → high bias. Use cross-validation.

12. What is Naive Bayes and when is it useful?

Naive Bayes applies Bayes' theorem assuming feature independence: P(y|X) ∝ P(y) × Π P(xᵢ|y).

Despite the "naive" independence assumption almost never holding in practice, it works surprisingly well for:

• Text classification (spam detection, sentiment)
• Real-time prediction (very fast)
• Small datasets

Variants: Gaussian (continuous features), Multinomial (word counts), Bernoulli (binary features).

Ensemble Methods

13. How does a Random Forest work?

Random Forest builds many decision trees, each trained on a bootstrap sample (random sample with replacement) of the data. At each split, only a random subset of features (√p for classification, p/3 for regression) is considered.

Predictions are aggregated by majority vote (classification) or average (regression). The randomness in sampling and feature selection decorrelates trees, reducing variance without increasing bias much.

14. What is the difference between bagging and boosting?

Boosting tends to achieve lower error but is more prone to overfitting and harder to parallelise.

15. How does Gradient Boosting work?

Gradient Boosting builds an additive ensemble by fitting each new tree to the residuals (negative gradients of the loss function) of the current ensemble:

1. Start with a constant prediction (e.g., mean of y)
2. Compute pseudo-residuals = −∂L/∂F for each sample
3. Fit a new decision tree to the residuals
4. Add the tree to the ensemble with learning rate η
5. Repeat for T iterations

XGBoost adds L1/L2 regularisation on leaf weights, second-order Taylor approximation for faster convergence, and built-in handling for missing values.

LightGBM uses histogram-based splitting and leaf-wise tree growth for speed on large datasets.

16. When would you choose XGBoost vs a neural network?

Neural Networks & Deep Learning

17. Explain backpropagation.

Backpropagation computes gradients of the loss with respect to all weights using the chain rule of calculus, flowing error backwards from output to input layers.

Steps:

1. Forward pass — compute activations and loss
2. Compute output gradient — ∂L/∂ŷ
3. Backward pass — apply chain rule layer by layer: ∂L/∂Wₗ = ∂L/∂aₗ × ∂aₗ/∂zₗ × ∂zₗ/∂Wₗ
4. Update weights — W ← W − η × ∂L/∂W

18. What are common activation functions and when do you use each?

19. What is the vanishing gradient problem?

In deep networks, gradients are multiplied through many layers during backprop. When gradients are small (<1), they shrink exponentially towards the input layers, making weights update negligibly — the network stops learning.

Solutions:

• ReLU/Leaky ReLU — gradients don't saturate for positive inputs
• Batch normalisation — normalises activations between layers
• Residual connections (ResNets) — gradient highways that bypass layers
• LSTM/GRU — gating mechanisms for RNNs
• Careful initialisation (He, Xavier/Glorot)

20. Explain CNN architecture and why it works for images.

A Convolutional Neural Network (CNN) exploits the spatial structure of images through three key ideas:

1. Local connectivity — neurons connect to a small region (receptive field), not all inputs
2. Weight sharing — same filter (kernel) is applied across all positions → translation invariance
3. Pooling — reduces spatial dimensions, provides invariance to small shifts

Typical architecture: Input → [Conv → BatchNorm → ReLU → Pool] × N → Flatten → FC layers → Output

21. What is an LSTM and why does it solve the vanishing gradient problem?

Long Short-Term Memory networks use gating mechanisms to control information flow:

• Forget gate — decides what to discard from cell state: f = σ(Wf·[h, x] + b)
• Input gate — decides what new info to store: i = σ(Wi·[h, x] + b)
• Cell state update — C = f ⊙ C_prev + i ⊙ tanh(Wc·[h, x] + b)
• Output gate — controls what cell state to expose: h = o ⊙ tanh(C)

The cell state acts as a gradient highway — gradients can flow unchanged through the forget gate when f ≈ 1, allowing the network to learn long-range dependencies.

22. What is the attention mechanism and how do Transformers use it?

Self-attention allows each token to attend to all other tokens in a sequence, computing a weighted sum where weights reflect relevance:

Attention(Q, K, V) = softmax(QKᵀ / √dₖ) × V

Where Q (queries), K (keys), V (values) are linear projections of the input.

Multi-head attention runs H attention functions in parallel with different projections, then concatenates and projects the results — capturing different relationship types simultaneously.

Transformers stack self-attention + feed-forward layers with residual connections and layer norm. They replaced RNNs for NLP because they parallelise training and handle long-range dependencies better.

23. What is batch normalisation and why does it help?

Batch normalisation normalises layer inputs to zero mean and unit variance (then scales/shifts with learnable γ, β):

x̂ = (x − μ_B) / √(σ_B² + ε), y = γx̂ + β

Benefits:

• Reduces internal covariate shift — each layer sees more stable input distributions
• Allows higher learning rates
• Acts as mild regulariser (noise from mini-batch statistics)
• Reduces sensitivity to weight initialisation

Model Evaluation

24. What is the difference between precision, recall, and F1 score?

For a binary classifier with Positive/Negative labels:

• Use precision when false positives are costly (spam filter)
• Use recall when false negatives are costly (cancer detection)
• Use F1 when you need a single metric balancing both

25. What is ROC AUC and when should you use it?

The ROC curve plots True Positive Rate vs False Positive Rate at all classification thresholds. AUC (Area Under Curve) summarises this in one number:

• AUC = 1.0 → perfect classifier
• AUC = 0.5 → random classifier
• AUC = 0.0 → perfectly wrong (just flip predictions)

Use AUC when:

• Class imbalance is present (AUC is threshold-independent)
• You need to compare models across thresholds
• The final decision threshold is unknown at evaluation time

26. How does k-fold cross-validation work?

1. Split data into k equal folds
2. For each fold i: train on all folds except i, evaluate on fold i
3. Report mean ± std of evaluation metric across k folds

Advantages over single train/test split:

• Uses all data for both training and validation
• Reduces variance of performance estimate

Common choices: k=5 or k=10. Stratified k-fold maintains class proportions in each fold (important for imbalanced datasets).

27. What metrics would you use for an imbalanced classification problem?

Avoid accuracy — a model predicting "always majority class" gets high accuracy trivially.

Preferred metrics:

• Precision-Recall AUC (more informative than ROC AUC when positives are rare)
• F1 score (or Fβ if you want to weight precision/recall differently)
• Cohen's Kappa — agreement beyond chance
• MCC (Matthews Correlation Coefficient) — balanced metric for all four confusion matrix quadrants

Techniques: SMOTE oversampling, class-weighted loss, threshold tuning, undersampling.

28. What is the difference between RMSE and MAE?

Use RMSE when large errors are especially bad. Use MAE when outliers should not dominate.

Feature Engineering

29. Why is feature scaling important and when is it required?

Feature scaling ensures features with larger numerical ranges don't dominate distance-based or gradient-based algorithms.

Required for: KNN, SVM, PCA, neural networks, logistic regression, linear regression (for regularisation)

Not required for: tree-based models (Decision Tree, Random Forest, XGBoost)

30. How do you handle missing values?

Strategy depends on the mechanism (MCAR/MAR/MNAR) and percentage missing:

31. How do you encode categorical variables?

32. What is the difference between feature selection and dimensionality reduction?

Feature selection preserves interpretability; dimensionality reduction can capture more variance with fewer components.

33. What is target leakage and how do you prevent it?

Target leakage occurs when features used in training contain information about the target that would not be available at prediction time — leading to optimistic evaluation and poor real-world performance.

Examples:

• Including "days in hospital" to predict "was hospitalised"
• Using post-event features in fraud detection
• Imputing missing values using statistics from the full dataset before splitting

Prevention:

• Understand data generation process thoroughly
• Create train/val/test split before any preprocessing
• Use pipelines (sklearn) to ensure transforms fit only on training data
• Validate with time-based splits for temporal data

Regularisation & Optimisation

34. What is L1 vs L2 regularisation?

Both add a penalty to the loss function to discourage large weights:

L1 produces sparse solutions because the L1 ball has corners aligned with axes.

35. What is dropout and how does it work?

During training, dropout randomly sets a fraction p of neuron activations to 0 at each forward pass. During inference, all neurons are active but weights are scaled by (1−p).

Effects:

• Prevents co-adaptation — neurons can't rely on specific other neurons
• Acts as an ensemble of 2ᴴ thinned networks
• Effective regulariser for large networks

Typical values: p = 0.2–0.5 for hidden layers, p = 0.1 for inputs. Not commonly used with batch norm (they can interfere).

36. What are common gradient descent optimisers?

Learning rate scheduling (cosine decay, warm-up) further improves convergence.

Practical ML

37. How do you split your data and why?

Standard split: 70–80% train / 10–15% validation / 10–15% test.

• Training set — fit model parameters
• Validation set — tune hyperparameters and select model; influences model indirectly
• Test set — unbiased final estimate of generalisation; touch only once

For time-series data: always use a chronological split — never random shuffle, as this leaks future information into training.

38. How do you tune hyperparameters?

39. What is an ML pipeline and why use one?

A pipeline chains preprocessing and modelling steps into a single object:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier

pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', RandomForestClassifier(n_estimators=100))
])
pipe.fit(X_train, y_train)

Benefits:

• Prevents data leakage — transforms fit only on training fold during CV
• Reproducible experiments
• Easy deployment — serialise one object
• Consistent preprocessing between train and inference

40. What is A/B testing and how does it relate to ML?

A/B testing is a randomised controlled experiment comparing a control (A) and treatment (B) to determine which performs better on a metric.

In ML context:

• Compare two models by routing live traffic to each
• Requires statistical significance testing (t-test, chi-squared, Mann-Whitney)
• Must watch for novelty effect, network effects, and seasonal confounding

Online A/B test → real users; offline evaluation → historical data. Both are needed: offline selects candidates, online validates real impact.

Advanced & MLOps

41. What is transfer learning?

Transfer learning uses a model pre-trained on a large dataset as the starting point for a new task. The pre-trained weights capture general features (edges, textures, grammar) that are useful across tasks.

Strategies:

• Feature extraction — freeze all layers, add a new head, train only the head
• Fine-tuning — unfreeze some/all layers and train with a low learning rate
• Domain adaptation — adapt to a different but related domain

Common base models: ResNet/EfficientNet (vision), BERT/GPT (NLP).

42. What is the difference between clustering algorithms?

43. Explain PCA.

Principal Component Analysis finds orthogonal directions (principal components) of maximum variance in the data:

1. Centre data (subtract mean)
2. Compute covariance matrix C = XᵀX / (n−1)
3. Compute eigenvectors/eigenvalues of C
4. Sort by eigenvalue (variance explained)
5. Project data onto top k eigenvectors

Use cases: visualisation (2D/3D), dimensionality reduction before ML, noise removal.

PCA is unsupervised — it finds directions of maximum variance, not maximum class separation (use LDA for the latter).

44. What is model drift and how do you monitor for it?

Data drift (covariate shift) — input feature distribution P(X) changes over time.
Concept drift — the relationship P(y|X) changes (same features, different outcomes).
Label drift — distribution of y changes.

Monitoring techniques:

• Statistical tests — KS test, Population Stability Index (PSI), chi-squared for categorical
• Model performance tracking — log predictions vs actuals; alert on metric degradation
• Shadow mode — run new model alongside old, compare predictions
• Data quality checks — schema validation, nulls, range checks on incoming features

45. What is SHAP and how does it explain model predictions?

SHAP (SHapley Additive exPlanations) assigns each feature a contribution value for each prediction based on cooperative game theory:

φᵢ = Σ (|S|!(|F|−|S|−1)!/|F|!) × [f(S∪{i}) − f(S)]

In practice:

• TreeExplainer — exact, fast for tree models (XGBoost, LightGBM, RF)
• LinearExplainer — exact for linear models
• KernelExplainer — model-agnostic, slow

SHAP values are locally accurate (sum to prediction) and consistent — making them preferable to LIME and permutation importance for feature attribution.

46. What is the difference between online learning and batch learning?

Online learning enables models to adapt to new patterns without full retraining.

47. What is the difference between generative and discriminative models?

Generative models can generate new samples and handle missing features more naturally; discriminative models tend to be more accurate for classification.

48. What is a Variational Autoencoder (VAE)?

A VAE is a generative model that learns a latent representation:

• Encoder — maps input X to a distribution q(z|X) (mean μ, log variance log σ²)
• Reparameterisation trick — z = μ + σ·ε, ε ~ N(0,1) — enables backprop through sampling
• Decoder — maps z back to reconstructed input p(X|z)
• Loss — reconstruction loss + KL divergence (forces latent space to be smooth Gaussian)

VAEs enable controlled generation by sampling from the prior p(z) ~ N(0,1) and decoding.

49. How do you deploy a machine learning model?

Common deployment patterns:

Production requirements:

• Versioned models (MLflow, DVC)
• Containerised (Docker)
• Monitoring (Prometheus + Grafana, Evidently AI)
• Rollback strategy

50. What is the difference between online evaluation and offline evaluation for ML models?

Best practice: use offline evaluation to filter candidates, then validate top models with online A/B tests before full rollout.

Common mistakes

ML vs related fields

Frequently asked questions

Q: Do I need to know maths for ML interviews?
Yes — linear algebra (matrix multiplication, eigenvectors), calculus (chain rule, gradients), probability (Bayes theorem, distributions), and statistics (hypothesis testing, p-values) all appear. You don't need to derive proofs, but conceptual understanding is expected.

Q: What Python libraries should I know?
Core: NumPy, pandas, scikit-learn, matplotlib/seaborn. Deep learning: PyTorch (most research/industry) or TensorFlow/Keras. Additional: XGBoost/LightGBM, SHAP, Optuna, MLflow. For large data: PySpark, Dask.

Q: How do I approach a case study / ML design question?
Follow a structured flow: (1) Clarify business objective → (2) Define ML task and success metrics → (3) Describe data collection and labelling → (4) Feature engineering → (5) Model selection with justification → (6) Evaluation strategy → (7) Deployment and monitoring plan.

Q: What is the difference between deep learning and machine learning?
Deep learning is a subset of ML that uses multi-layer neural networks to automatically learn hierarchical feature representations from raw data. Traditional ML requires manual feature engineering; deep learning learns features end-to-end at the cost of requiring much more data and compute.

Q: Should I memorise algorithm equations?
For senior roles: understand the intuition and be able to derive key equations (sigmoid, softmax, gradient of cross-entropy). For most interviews: understand tradeoffs and when to use each algorithm. Coding questions test implementation, not derivation.

Q: What's the most important concept for ML interviews?
The bias-variance tradeoff underpins almost every practical ML decision: model selection, regularisation, ensemble methods, hyperparameter tuning, and evaluation. Master this and many other topics become intuitive.