50 Pandas Interview Questions (With Answers)

Pandas interviews test your practical ability to manipulate, clean, and analyze tabular data. This guide covers 50 of the most common pandas interview questions with clear, accurate answers and code examples.

Quick reference

Core Data Structures

1. What is the difference between a DataFrame and a Series?

A Series is a one-dimensional labeled array that can hold any data type. A DataFrame is a two-dimensional labeled data structure — essentially a collection of Series sharing the same index.

import pandas as pd

# Series
s = pd.Series([10, 20, 30], index=["a", "b", "c"])
print(type(s))  # <class 'pandas.core.series.Series'>

# DataFrame
df = pd.DataFrame({"x": [1, 2], "y": [3, 4]})
print(type(df))  # <class 'pandas.core.frame.DataFrame'>

# A DataFrame column is a Series
print(type(df["x"]))  # <class 'pandas.core.series.Series'>

Key differences:

2. What is an Index in pandas, and why does it matter?

An Index is the row label array attached to every Series and DataFrame. It enables fast label-based lookups (O(1) for unique indexes via hash table), alignment during arithmetic, and meaningful row identification.

df = pd.DataFrame({"val": [10, 20, 30]}, index=["a", "b", "c"])
print(df.loc["b"])   # val    20
print(df.index)      # Index(['a', 'b', 'c'], dtype='object')

# RangeIndex is the default (0, 1, 2, …)
df2 = pd.DataFrame({"val": [1, 2, 3]})
print(df2.index)     # RangeIndex(start=0, stop=3, step=1)

Setting a meaningful index speeds up row lookups and joins:

df = df.set_index("user_id")   # use a column as index
df = df.reset_index()           # move index back to column

3. What is a MultiIndex and when would you use it?

A MultiIndex (hierarchical index) supports multiple levels of row or column labels. Use it for panel data, grouped results, or when your data naturally has two or more key dimensions.

arrays = [["CA", "CA", "NY", "NY"], ["LA", "SF", "NYC", "Buffalo"]]
mi = pd.MultiIndex.from_arrays(arrays, names=["state", "city"])
df = pd.DataFrame({"pop": [4M, 0.9M, 8.3M, 0.3M]}, index=mi)

# Select all rows for California
print(df.loc["CA"])
# Select specific (state, city)
print(df.loc[("NY", "NYC")])

Use df.xs() for cross-sections and df.stack() / df.unstack() to reshape between MultiIndex and wide format.

Data Loading & Inspection

4. How do you read a CSV file with pandas?

df = pd.read_csv("data.csv")

# Common options
df = pd.read_csv(
    "data.csv",
    sep=";",              # delimiter
    header=0,             # row to use as column names
    index_col="id",       # use column as index
    usecols=["a", "b"],   # load only certain columns
    dtype={"price": float},
    parse_dates=["date"],
    na_values=["N/A", "?"],
    chunksize=10_000,     # iterator for large files
    encoding="utf-8",
)

For large files use chunksize to read in batches:

chunks = []
for chunk in pd.read_csv("big.csv", chunksize=50_000):
    chunks.append(chunk[chunk["revenue"] > 0])
df = pd.concat(chunks, ignore_index=True)

5. What do df.info(), df.describe(), and df.head() tell you?

6. How do you check for and handle missing values?

# Detect
df.isnull().sum()           # count NaN per column
df.isnull().any(axis=1)     # True for rows with any NaN

# Drop
df.dropna()                 # drop rows with any NaN
df.dropna(subset=["age"])   # drop only where 'age' is NaN
df.dropna(thresh=3)         # keep rows with at least 3 non-NaN

# Fill
df.fillna(0)
df.fillna(df.mean())        # fill numeric cols with mean
df["col"].fillna(method="ffill")  # forward fill
df["col"].fillna(method="bfill")  # backward fill

# Interpolate
df["price"].interpolate(method="linear")

Selection & Indexing

7. What is the difference between loc and iloc?

df = pd.DataFrame({"val": [10, 20, 30]}, index=["a", "b", "c"])

df.loc["a":"b"]    # rows a and b (inclusive)
df.iloc[0:2]       # rows at position 0,1 (exclusive end)

df.loc["a", "val"]    # single value by label
df.iloc[0, 0]         # single value by position

# Boolean indexing with loc
df.loc[df["val"] > 15]

8. What are at and iat used for?

at and iat are scalar accessors — faster than loc/iloc when you need a single value because they skip multi-value overhead.

df.at["a", "val"]    # label-based scalar
df.iat[0, 0]         # position-based scalar

# Faster for repeated single-cell access in loops
df.at["a", "val"] = 99   # also works for setting

9. How do you filter rows with multiple conditions?

# AND — use & (not 'and')
df[(df["age"] > 25) & (df["salary"] > 50_000)]

# OR — use | (not 'or')
df[(df["dept"] == "Engineering") | (df["dept"] == "Data")]

# NOT — use ~
df[~df["name"].str.startswith("A")]

# isin
df[df["city"].isin(["London", "Berlin", "NYC"])]

# between
df[df["score"].between(80, 100)]

# query string (readable alternative)
df.query("age > 25 and salary > 50_000")

10. How do you select columns by data type?

df.select_dtypes(include="number")          # int + float
df.select_dtypes(include=["float64"])
df.select_dtypes(exclude=["object", "bool"])
df.select_dtypes(include="datetime")

Data Cleaning

11. How do you remove duplicate rows?

df.duplicated()                        # boolean Series
df.duplicated(subset=["email"])        # duplicates by column(s)
df.duplicated(keep="last")             # mark all except last occurrence

df.drop_duplicates()
df.drop_duplicates(subset=["email"], keep="first")
df.drop_duplicates(keep=False)         # drop all duplicates, keep none

12. How do you rename columns?

# dict mapping (rename specific columns)
df.rename(columns={"old_name": "new_name", "a": "b"})

# function applied to all column names
df.rename(columns=str.lower)
df.rename(columns=lambda x: x.strip().replace(" ", "_"))

# direct assignment (replaces all column names)
df.columns = ["col1", "col2", "col3"]

# using str accessor for bulk transformations
df.columns = df.columns.str.lower().str.replace(" ", "_")

13. How do you change data types?

df["age"] = df["age"].astype(int)
df["price"] = df["price"].astype(float)
df["category"] = df["category"].astype("category")   # memory efficient
df["date"] = pd.to_datetime(df["date"])
df["amount"] = pd.to_numeric(df["amount"], errors="coerce")  # NaN for bad values

Memory savings by dtype:

14. How do you apply a function to a column?

# map — element-wise on Series (best for simple transformations)
df["upper"] = df["name"].map(str.upper)
df["label"] = df["score"].map({1: "low", 2: "mid", 3: "high"})

# apply — flexible, works on Series or DataFrame
df["length"] = df["text"].apply(len)
df["result"] = df.apply(lambda row: row["a"] + row["b"], axis=1)

# applymap / map (pandas 2.1+) — element-wise on entire DataFrame
df = df.map(lambda x: round(x, 2) if isinstance(x, float) else x)

Performance preference: vectorized operations > map > apply > Python loops.

15. How do you replace values?

df["status"].replace("active", "ACTIVE")
df["status"].replace({"active": 1, "inactive": 0})
df.replace({"col1": {"old": "new"}})

# regex
df["text"].str.replace(r"\d+", "NUM", regex=True)

# numpy where (vectorized conditional replace)
import numpy as np
df["label"] = np.where(df["score"] >= 60, "pass", "fail")

Aggregation & Grouping

16. How does groupby work?

g = df.groupby("dept")

g["salary"].mean()         # mean salary per dept
g["salary"].agg(["mean", "min", "max", "count"])

# Multiple groupby keys
df.groupby(["dept", "level"])["salary"].mean()

# Custom aggregation per column
df.groupby("dept").agg(
    avg_salary=("salary", "mean"),
    headcount=("name", "count"),
    max_bonus=("bonus", "max"),
)

# Apply arbitrary function
df.groupby("dept")["score"].apply(lambda x: x.quantile(0.9))

groupby creates a GroupBy object; it's lazy until you call an aggregation method.

17. What is the difference between transform and agg in groupby?

# agg reduces
df.groupby("dept")["salary"].agg("mean")
# dept
# Eng    90000
# HR     55000

# transform keeps original index
df["dept_avg"] = df.groupby("dept")["salary"].transform("mean")
# Every row now has its department average

18. How do you create a pivot table?

pt = df.pivot_table(
    values="sales",
    index="region",
    columns="quarter",
    aggfunc="sum",
    fill_value=0,
    margins=True,     # adds row/column totals
)

# crosstab — for frequency tables
pd.crosstab(df["gender"], df["dept"])
pd.crosstab(df["gender"], df["dept"], normalize="index")  # row percentages

pivot_table is the flexible aggregation version; pivot is the strict reshape (no aggregation, fails on duplicates).

19. How do you compute running totals or moving averages?

df["cumsum"] = df["revenue"].cumsum()
df["cumpct"] = df["revenue"].cumsum() / df["revenue"].sum()

# Rolling window
df["ma7"] = df["price"].rolling(window=7).mean()
df["std30"] = df["price"].rolling(window=30).std()

# Expanding (all data up to current row)
df["expanding_mean"] = df["price"].expanding().mean()

# Exponential weighted
df["ewm"] = df["price"].ewm(span=10).mean()

20. What does value_counts() return and how do you use it?

df["city"].value_counts()              # descending frequency
df["city"].value_counts(normalize=True)  # proportions
df["city"].value_counts(ascending=True)
df["city"].value_counts(dropna=False)  # include NaN count

# Bin continuous data first, then count
pd.cut(df["age"], bins=[0, 18, 35, 60, 100]).value_counts()

Merging & Reshaping

21. What is the difference between merge, join, and concat?

# merge (most flexible)
pd.merge(df1, df2, on="id", how="inner")
pd.merge(df1, df2, left_on="user_id", right_on="id", how="left")

# join (index-based)
df1.join(df2, how="left", lsuffix="_left", rsuffix="_right")

# concat (stacking)
pd.concat([df1, df2], axis=0, ignore_index=True)   # vertical
pd.concat([df1, df2], axis=1)                       # horizontal

22. What are the join types in pd.merge()?

# Detect merge result size
result = pd.merge(df1, df2, on="id", how="outer", indicator=True)
result["_merge"].value_counts()
# left_only     → rows only in df1
# right_only    → rows only in df2
# both          → matched rows

23. How do you reshape data with melt and pivot?

# Wide → Long with melt
df_long = df.melt(
    id_vars=["name"],
    value_vars=["q1", "q2", "q3", "q4"],
    var_name="quarter",
    value_name="sales",
)

# Long → Wide with pivot
df_wide = df_long.pivot(
    index="name",
    columns="quarter",
    values="sales",
)

# stack / unstack (for MultiIndex)
df.stack()     # columns → innermost row index level
df.unstack()   # innermost row level → columns

24. How does combine_first work?

combine_first fills NaN values in the calling DataFrame with values from another DataFrame, aligning on index and columns.

df1 = pd.DataFrame({"a": [1, np.nan, 3]})
df2 = pd.DataFrame({"a": [10, 20, 30]})

df1.combine_first(df2)
# a
# 0    1.0   (df1 value preserved)
# 1   20.0   (df1 was NaN → df2 fills in)
# 2    3.0

Useful for patching incomplete DataFrames with fallback data.

String Operations

25. How do you use the str accessor?

df["name"].str.upper()
df["name"].str.lower()
df["name"].str.strip()
df["name"].str.replace("  ", " ", regex=False)
df["email"].str.contains("@gmail", na=False)
df["email"].str.startswith("admin")
df["url"].str.extract(r"https://([^/]+)", expand=False)   # first group
df["full_name"].str.split(" ", expand=True)               # → DataFrame of splits
df["name"].str.len()
df["code"].str[:3]     # slice first 3 chars

Always pass na=False to .str.contains() to avoid NaN propagation in boolean results.

26. How do you extract parts of strings with regex?

# extract — returns captured groups as columns
df[["area", "number"]] = df["phone"].str.extract(r"\((\d{3})\)\s*(\d{7})")

# extractall — returns all matches, MultiIndex
df["phone"].str.extractall(r"(\d+)")

# findall — list of all matches
df["text"].str.findall(r"\b[A-Z][a-z]+\b")

# count occurrences
df["text"].str.count(r"\bpython\b")

Time Series

27. How do you work with datetime data in pandas?

# Parse on load
df = pd.read_csv("data.csv", parse_dates=["timestamp"])

# Convert existing column
df["date"] = pd.to_datetime(df["date"], format="%Y-%m-%d")

# Extract components
df["year"] = df["date"].dt.year
df["month"] = df["date"].dt.month
df["day_of_week"] = df["date"].dt.day_name()
df["quarter"] = df["date"].dt.quarter
df["hour"] = df["date"].dt.hour

# Arithmetic
df["days_since"] = (pd.Timestamp("today") - df["date"]).dt.days
df["next_week"] = df["date"] + pd.Timedelta(days=7)

28. How do you resample time series data?

# Set datetime index first
df = df.set_index("date")

df.resample("D").sum()    # daily
df.resample("W").mean()   # weekly
df.resample("ME").sum()   # month-end
df.resample("QE").last()  # quarter-end last value
df.resample("h").ffill()  # hourly, forward fill

# With groupby
df.groupby(df.index.year).resample("ME").sum()

Common frequency aliases: D day, h hour, min minute, W week, ME month-end, QE quarter-end, YE year-end.

29. What is the difference between shift and diff?

df["prev_day"] = df["price"].shift(1)    # lag 1 period
df["next_day"] = df["price"].shift(-1)   # lead 1 period

df["daily_change"] = df["price"].diff(1)         # price[t] - price[t-1]
df["pct_change"] = df["price"].pct_change()      # (price[t] - price[t-1]) / price[t-1]

shift moves values without computing differences; diff computes the difference between shifted and original values.

Performance

30. When should you use apply vs vectorized operations?

Always prefer vectorized operations first.

# SLOW — apply calls Python for every row
df["total"] = df.apply(lambda row: row["price"] * row["qty"], axis=1)

# FAST — vectorized (C-level NumPy operation)
df["total"] = df["price"] * df["qty"]

# SLOW
df["upper"] = df["name"].apply(str.upper)

# FAST — str accessor is vectorized
df["upper"] = df["name"].str.upper()

Performance order: NumPy ufuncs > pandas vectorized > .map() > .apply() > Python loop.

31. How do you reduce memory usage in pandas?

# Check memory
df.memory_usage(deep=True).sum() / 1024**2  # MB

# Downcast integers
df["age"] = pd.to_numeric(df["age"], downcast="integer")

# Float32 instead of float64
df["score"] = df["score"].astype("float32")

# Category for low-cardinality strings
df["country"] = df["country"].astype("category")

# Sparse arrays for data with many zeros
df["flag"] = df["flag"].astype(pd.SparseDtype("int8", 0))

# Avoid object dtype — use specific types
df["date"] = pd.to_datetime(df["date"])

32. How do you process files too large to fit in memory?

# chunking
total = 0
for chunk in pd.read_csv("huge.csv", chunksize=100_000):
    total += chunk["amount"].sum()

# Collect filtered chunks
results = []
for chunk in pd.read_csv("huge.csv", chunksize=100_000):
    results.append(chunk[chunk["status"] == "active"])
df = pd.concat(results, ignore_index=True)

# Use dask for distributed/out-of-core processing
import dask.dataframe as dd
ddf = dd.read_csv("huge.csv")
result = ddf.groupby("dept")["salary"].mean().compute()

33. How does copy() affect DataFrame operations?

Pandas uses copy-on-write semantics (default in pandas 2.0+). To be explicit:

# Slice returns a view (may trigger SettingWithCopyWarning in older pandas)
subset = df[df["age"] > 30]

# Explicit copy — safe to modify without affecting original
subset = df[df["age"] > 30].copy()
subset["label"] = "senior"   # no warning

In pandas 2.0+ with CoW enabled, modifying a subset always creates a copy — the warning becomes unnecessary. Use .copy() to be explicit and backward-compatible.

Advanced Operations

34. How do you use cut and qcut?

# cut — fixed bin edges
df["age_group"] = pd.cut(
    df["age"],
    bins=[0, 18, 35, 60, 100],
    labels=["under18", "young_adult", "adult", "senior"],
)

# qcut — equal-frequency bins (quantile-based)
df["score_quartile"] = pd.qcut(df["score"], q=4, labels=["Q1", "Q2", "Q3", "Q4"])

df["age_group"].value_counts()

Use cut when you want meaningful boundary values; use qcut when you want equal-sized groups.

35. How does the where method work?

# where keeps values where condition is True, replaces with other where False
df["capped"] = df["score"].where(df["score"] <= 100, other=100)

# Equivalent numpy approach
import numpy as np
df["capped"] = np.where(df["score"] <= 100, df["score"], 100)

# mask is the inverse of where
df["flagged"] = df["score"].mask(df["score"] > 100, other=np.nan)

36. How do you use assign for method chaining?

result = (
    df
    .assign(
        full_name=lambda x: x["first"] + " " + x["last"],
        age_group=lambda x: pd.cut(x["age"], bins=[0, 30, 60, 100]),
        senior=lambda x: x["age"] > 60,
    )
    .query("senior == True")
    .sort_values("salary", ascending=False)
    .reset_index(drop=True)
)

assign returns a new DataFrame (immutable pattern), enabling clean pipeline-style transformations.

37. How do you explode a column containing lists?

df = pd.DataFrame({
    "user": ["alice", "bob"],
    "tags": [["python", "sql"], ["java"]],
})

df_exploded = df.explode("tags")
#    user    tags
# 0  alice  python
# 0  alice     sql
# 1    bob    java

# If column contains JSON strings, parse first
df["tags"] = df["tags"].apply(json.loads)
df.explode("tags")

38. How do you create a DataFrame from a list of dicts or JSON?

data = [{"name": "Alice", "age": 30}, {"name": "Bob", "age": 25}]

# From list of dicts (most common)
df = pd.DataFrame(data)

# From JSON string
import json
df = pd.DataFrame(json.loads(json_string))

# From nested JSON — normalize
from pandas import json_normalize
df = json_normalize(data, record_path="orders", meta=["user_id"])

# From dict of lists
df = pd.DataFrame({"a": [1, 2], "b": [3, 4]})

Error Handling & Debugging

39. What is SettingWithCopyWarning and how do you fix it?

This warning appears when you try to modify a slice that may be a copy:

# Triggers warning (chained indexing)
df[df["age"] > 30]["salary"] = 100_000   # may not modify original!

# Fix 1: loc on original
df.loc[df["age"] > 30, "salary"] = 100_000

# Fix 2: explicit copy then modify
subset = df[df["age"] > 30].copy()
subset["salary"] = 100_000

# Fix 3: pandas 2.0 CoW mode (eliminates warning entirely)
pd.options.mode.copy_on_write = True

40. What causes a ValueError: cannot reindex from a duplicate axis?

This happens when merge/join/reindex encounters duplicate index values where unique keys are expected.

# Check for duplicates
df.index.duplicated().any()
df[df.index.duplicated()]

# Fix: reset index
df = df.reset_index(drop=True)

# Fix: handle before join
df = df[~df.index.duplicated(keep="first")]

Interview Scenarios

41. How do you find the top N rows per group?

# Top 3 salary earners per department
top3 = (
    df
    .sort_values("salary", ascending=False)
    .groupby("dept")
    .head(3)
)

# Using rank
df["rank"] = df.groupby("dept")["salary"].rank(method="dense", ascending=False)
top3 = df[df["rank"] <= 3]

42. How do you calculate year-over-year growth?

df = df.set_index("date").sort_index()

# Monthly revenue, YoY growth
df["yoy_pct"] = df["revenue"].pct_change(periods=12) * 100

# Quarterly
quarterly = df.resample("QE")["revenue"].sum()
quarterly["yoy_growth"] = quarterly.pct_change(4) * 100

43. How do you detect and handle outliers?

# IQR method
Q1 = df["salary"].quantile(0.25)
Q3 = df["salary"].quantile(0.75)
IQR = Q3 - Q1

lower = Q1 - 1.5 * IQR
upper = Q3 + 1.5 * IQR

outliers = df[(df["salary"] < lower) | (df["salary"] > upper)]

# Cap instead of remove
df["salary_capped"] = df["salary"].clip(lower=lower, upper=upper)

# Z-score method
from scipy import stats
df["z"] = stats.zscore(df["salary"])
df_clean = df[df["z"].abs() < 3]

44. How do you do a self-join (compare rows in the same DataFrame)?

# Find pairs of employees in the same department
df_merged = df.merge(df, on="dept", suffixes=("_a", "_b"))
pairs = df_merged[df_merged["id_a"] < df_merged["id_b"]]

# Find employees earning more than their manager
emp = df.merge(df, left_on="manager_id", right_on="id", suffixes=("", "_mgr"))
emp[emp["salary"] > emp["salary_mgr"]]

45. How do you create lag features for machine learning?

df = df.sort_values("date")

# Lag features
for lag in [1, 7, 30]:
    df[f"sales_lag_{lag}"] = df["sales"].shift(lag)

# Rolling features
df["sales_roll7_mean"] = df["sales"].rolling(7).mean()
df["sales_roll7_std"] = df["sales"].rolling(7).std()

# Drop NaN rows introduced by lag
df = df.dropna()

46. How do you compute correlation between columns?

# Full correlation matrix
df.corr(method="pearson")   # or "spearman", "kendall"

# Correlation with target variable
df.corr()["price"].sort_values(ascending=False)

# Correlation between two specific columns
df["age"].corr(df["salary"])

# Visualize (with seaborn)
import seaborn as sns
sns.heatmap(df.corr(), annot=True, cmap="coolwarm")

47. How do you encode categorical variables?

# One-hot encoding
dummies = pd.get_dummies(df["city"], prefix="city", drop_first=True)
df = pd.concat([df, dummies], axis=1)

# Ordinal mapping
order = {"low": 1, "medium": 2, "high": 3}
df["priority_enc"] = df["priority"].map(order)

# Label encoding (scikit-learn)
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
df["dept_enc"] = le.fit_transform(df["dept"])

48. How do you handle categorical dtype efficiently?

df["status"] = df["status"].astype("category")

# Access category-specific properties
df["status"].cat.categories
df["status"].cat.codes          # underlying integer codes
df["status"].cat.ordered

# Add/remove categories
df["status"].cat.add_categories(["pending"])
df["status"].cat.remove_unused_categories()
df["status"].cat.reorder_categories(["inactive", "active"])

# groupby on category is faster than on object
df.groupby("status")["revenue"].sum()

49. How do you write a DataFrame to different output formats?

# CSV
df.to_csv("output.csv", index=False)

# Excel
df.to_excel("output.xlsx", sheet_name="data", index=False)

# JSON
df.to_json("output.json", orient="records", lines=True)

# Parquet (columnar, fast, recommended for large data)
df.to_parquet("output.parquet", engine="pyarrow", compression="snappy")

# SQL
from sqlalchemy import create_engine
engine = create_engine("postgresql://user:pass@host/db")
df.to_sql("table_name", engine, if_exists="replace", index=False)

# Feather (fast pandas-native format)
df.to_feather("output.feather")

50. What are the most common pandas anti-patterns to avoid?

Common Mistakes

Pandas vs Related Tools

FAQ

Q: Is pandas thread-safe? No. pandas DataFrames are not thread-safe for concurrent writes. For parallel read-only operations on separate copies it is generally safe, but avoid shared mutable state across threads. Use multiprocessing or Dask for parallel processing.

Q: What changed in pandas 2.0? Key changes: Copy-on-Write (CoW) enabled by default, nullable integer types (Int8, Int64) are first-class, df.append() removed (use pd.concat()), datetime64 now timezone-naive by default, Pyarrow backend support. Performance improved significantly.

Q: When should I use polars instead of pandas? Use polars when: data exceeds ~5M rows, you need lazy evaluation, you want parallel execution by default, or you need significantly faster groupby/join operations. Polars is 5-10× faster than pandas for many operations.

Q: What is the difference between pandas NA and numpy NaN? np.nan is a float — it silently converts integer columns to float64 when introduced. pd.NA is a pandas-native missing value that works with nullable integer, boolean, and string dtypes without type coercion. Prefer pd.NA in new code.

Q: How do you debug a slow pandas operation? Profile with %timeit in Jupyter, use .dtypes to check for object columns that should be numeric/category, check if you can vectorize an apply, verify you're not iterating rows with iterrows. For memory, use df.memory_usage(deep=True).

Q: Is it better to use inplace=True or reassignment? Reassignment is preferred. inplace=True does not save memory (still makes a copy internally in most cases), can cause issues with method chaining, and is being reconsidered for deprecation. Use df = df.sort_values(...) instead of df.sort_values(..., inplace=True).