Pandas
The dataframe library that made Python a serious tool for data analysis.
What it is
Pandas is a powerful Python library for data manipulation and analysis. It provides fast, flexible, and expressive data structures such as Series and DataFrame for working with structured data.
Pandas allows for easy reading, writing, and manipulation of data from multiple sources including CSV, Excel, SQL databases, and more. You can filter, aggregate, group, pivot, merge, and reshape datasets efficiently.
- Watch for
- Chained assignment and the SettingWithCopyWarning — use .loc for assignment
- Licence
- BSD 3-clause
- Rule of thumb
- Vectorise; if you are writing `.iterrows()`, there is almost always a faster way
When to use it
The question documentation cannot answer for you — because it cannot recommend something else.
Reach for it when
- Exploratory analysis on tabular data that fits comfortably in memory
- Cleaning, reshaping, joining and aggregating data before analysis or modelling
- Reading and writing CSV, Excel, Parquet, SQL and JSON with one consistent interface
Look elsewhere when
- The dataset is larger than roughly half your RAM — reach for Polars, Dask or DuckDB
- You need predictable low-latency performance in a production hot path
Installation
pip install pandasGetting started
The smallest useful thing you can do with it, and what each part means.
import pandas as pd
df = pd.read_csv('data.csv')
print(df.head())print(df['column_name'])
print(df.iloc[0])Advanced usage
Where the library earns its place over a simpler alternative.
filtered = df[df['age'] > 30]
grouped = filtered.groupby('department').mean()merged = pd.merge(df1, df2, on='id', how='inner')pivot = df.pivot_table(index='department', columns='gender', values='salary', aggfunc='mean')Errors and fixes
The failures you are most likely to hit, and what actually resolves them.
- FileNotFoundError
- Ensure the file path is correct when reading CSV/Excel files.
- KeyError
- Verify column names exist before accessing them.
- ValueError
- Check the shape and alignment of DataFrames when merging or concatenating.
Best practices
- Use vectorized operations instead of loops for performance.
- Clean data before analysis: handle missing values, duplicates, and inconsistent types.
- Use descriptive column names for readability.
- Leverage built-in aggregation functions for efficiency.
- Profile large datasets with df.info() and df.describe() before processing.
Alternatives
Comparable options, and the reason you would pick one over the other.
Background
Why it exists, and what it was reacting to.
Pandas was created by Wes McKinney in 2008 to provide a high-performance, user-friendly data analysis tool for Python. It has become the standard library for data manipulation in Python, widely used in data science, finance, research, and analytics.