Module 2 · Python for Data Analysis
Clean, Filter and Calculate
Fix column types, add a revenue column, filter rows that match a condition and sort the results.
About 25 minutes
Fix the date column
Dates loaded as text can't be sorted or grouped by month properly. Convert them:
orders["order_date"] = pd.to_datetime(orders["order_date"])
orders.info()order_date is now datetime64. You can pull out parts of the date:
orders["year"] = orders["order_date"].dt.year
orders["month"] = orders["order_date"].dt.to_period("M")Add a revenue column
Revenue for each line is quantity × unit price, minus the discount:
orders["revenue"] = (
orders["quantity"] * orders["unit_price"] * (1 - orders["discount_pct"] / 100)
)
orders["revenue"].sum()Total revenue is ₦830,541,245. Notice there's no loop: pandas does the maths for every row at once.
Filter rows
Put a condition inside square brackets to keep only the matching rows:
big = orders[orders["quantity"] >= 20]
len(big) # 1,032 lines
big["revenue"].sum()Combine conditions with & (and) or | (or), with brackets around each one:
discounted_2026 = orders[(orders["discount_pct"] > 0) & (orders["year"] == 2026)]Sort
orders.sort_values("revenue", ascending=False).head(5)This shows the five biggest order lines. Use ascending=True (the default) for smallest first.
Check for problems
Real data is rarely this clean. These checks are worth running on any dataset:
orders.isna().sum() # missing values per column
orders.duplicated().sum() # fully duplicated rows
(orders["quantity"] <= 0).sum() # impossible valuesThis dataset passes all three, but make it a habit.
Try it
- Convert
order_dateto a date and addyear,monthandrevenuecolumns. - Check the total revenue is ₦830,541,245.
- How many order lines had a 10% discount? How much revenue did they bring in?
- Show the 10 biggest order lines from 2026.
Earn your badge
Take the module check
Five quick questions about this module. Pass and you earn the Data Cleaning with Python badge, free.
Create a free account to earn the badge