Pandas groupby follows a split-apply-combine pattern: divide rows into groups, apply a calculation and combine the results. It is useful for summarizing sales by month, grades by course, expenses by category or measurements by location. Clear grouping keys and deliberate missing-value handling are essential for reliable results.
Create a small DataFrame
Suppose a dataset has department, employee, and hours columns. Grouping by department can calculate the total or average hours for each department. Each row should represent a well-defined observation so that the aggregation answers a meaningful question.
import pandas as pd
df = pd.DataFrame({
'department': ['Sales', 'Sales', 'IT', 'IT'],
'employee': ['Ava', 'Noah', 'Mia', 'Leo'],
'hours': [8, 6, 7, 9],
})Group and calculate one metric
Call groupby with the column used to form groups, then choose the column and aggregation function. The result is often a Series indexed by the group labels. Use reset_index() when you want the group labels returned as ordinary columns.
hours_by_department = (
df.groupby('department')['hours']
.sum()
)
print(hours_by_department)
# Tabular result
summary = hours_by_department.reset_index(name='total_hours')Use named aggregations for multiple metrics
You can compute several statistics in one pass, such as total hours, average hours and employee count. Named aggregations make the resulting column names explicit and are convenient for a reporting table.
Be clear about what count means. Count on a particular column excludes missing values in that column; size counts the number of rows in each group.
summary = df.groupby('department').agg(
total_hours=('hours', 'sum'),
average_hours=('hours', 'mean'),
record_count=('employee', 'count'),
).reset_index()Group by more than one column
Grouping by department and another field, such as month or project, creates one result for each combination. This is useful for trend analysis, but it can produce many small groups if you add too many keys.
If categories are missing, inspect the source and decide whether to preserve, label or exclude those records. Groupby defaults and categorical handling can vary with parameters and pandas version, so set options explicitly when they matter to the analysis.
monthly = df.groupby(
['department', 'employee'],
dropna=False,
).agg(total_hours=('hours', 'sum')).reset_index()Common groupby mistakes
The most common issue is grouping at the wrong level of detail. If you need a month-by-department report, grouping only by department collapses all months together. Another issue is aggregating duplicate records or mixing quantities with different units.
Inspect row counts before and after aggregation, check missing group keys and validate a few group totals manually. If the data contains repeated measurements, decide whether sum, mean, median or another metric answers the question.
- Choose keys that match the question.
- Understand the difference between count and size.
- Check missing values and duplicate records.
- Use clear output column names and units.
- Validate the aggregate against a small manual example.
Understanding the concept is the first step.
Explore related resources or learn about subject-focused tutoring designed to help you understand the material.