Grouping and the median
The average salary over a whole table says little if the table holds several teams. You need the mean for each team separately. In pandas this takes no loop:
import pandas as pd
df = pd.DataFrame({
"team": ["ml", "ml", "ml", "web", "web", "web", "web"],
"level": ["jun", "mid", "sen", "jun", "jun", "mid", "sen"],
"salary": [100, 120, 140, 90, 95, 105, 610],
})
means = df.groupby("team")["salary"].mean()
print(means.to_dict()) # {'ml': 120.0, 'web': 225.0}
print(df.groupby("team")["salary"].count().to_dict()) # {'ml': 3, 'web': 4}Grouping
Grouping splits rows by the value of a column. The call df.groupby("team") collects the rows with "ml" into one group and the rows with "web" into another. Then you select a column, ["salary"], and compute one number per group: mean, median, sum or count.
The result is a Series column where each group has its number. The to_dict method turns it into a dictionary. The count method counts filled values, so web has four. Missing values are skipped: if a group holds only missing values, mean and median give it NaN, and count gives 0.
Median
The web team has a mean of 225.0, although three of its four people earn 105 or less. One salary of 610 pulls the mean up. The median is the value in the middle of the sorted list, and one large number barely affects it.
To find the median, the values are sorted: for web they are 90, 95, 105, 610. For an even count, the median is the mean of the two central values: for web, (95 + 105) / 2.
medians = df.groupby("team")["salary"].median()
print(medians.to_dict()) # {'ml': 120.0, 'web': 100.0}Salaries, prices and incomes usually form skewed data: most values are small, and rare ones are very large. On such data the median describes a typical object better than the mean. In the ml team the values are symmetric, and both measures give 120.0.
If the mean is well above the median, the data has rare large values. The mean is still useful: multiplied by the headcount, it gives the payroll, and the median does not.
Common mistake
The mean is computed over the whole grouping without selecting a column:
print(df.groupby("team").mean()) # ✗TypeError: agg function failed [how->mean,dtype->object]The table has a text column level, and pandas tries to average it too. Before version 2.0, pandas dropped such columns, so old examples often contain this call. Select a numeric column before computing: df.groupby("team")["salary"].mean().
Subscribe to take the course
The course is free. Sign in with Telegram and subscribe to the channel to open every lesson.