FreeLevel 3 · Professional

Observability and Site Reliability

Across this track, Tallybook's month-end outage has been blamed on too few web servers and on a cryptominer. In this course you find the real cause. With a day of per-minute metrics you'll read the four golden signals and see that the database connection pool, not the web tier, ran out; with structured logs you'll find the bulk invoice-sending job that started at 09:40:02; with distributed traces you'll see requests spending seconds waiting for a connection while their queries barely slowed. Then you'll build the reliability practice that prevents a repeat: SLIs and SLOs with error budgets (two mornings used about two and a half months' worth), multi-window burn-rate alerts, a clean-up of pages that were nine-tenths noise, connection pool sizing with Little's law and job isolation, and an error budget policy with toil measured and cut. Every number comes from running the code on the incident's data.

About 7 hours
10 modules
1 badge
Final project
Optional verified certificate

Curriculum

01What Observability IsWhat observability means, the three kinds of telemetry (metrics, logs and traces), the four golden signals, and a first look at the minute-by-minute record of Tallybook's month-end outage.15 min
02Metrics and PercentilesWhy averages hide what users experience, how percentiles (p50, p95, p99) show it, how to read latency over a day, and which metrics to put on a service's first dashboard.25 min
03Structured LogsWhy logs should be structured (JSON) rather than free text, how to filter and count them like data, and how to use them to find the event that started an incident.25 min
04Distributed TracesRead traces made of spans, rebuild a request's path through the web tier, API, database and email provider, and compare where time goes in normal and incident traffic.15 min
05SLIs and SLOsDefine service level indicators from what users experience, set objectives and error budgets, and measure Tallybook's August against them, including how much of the month's budget two mornings used.25 min
06Burn-Rate AlertsAlert on how fast the error budget is being spent rather than on fixed thresholds, compute burn rates over short and long windows, and see when each alert would have fired on 31 August.25 min
07Alert Quality and On-CallMeasure how useful each alert is (how often it needs action, when it wakes people, how fast it's acknowledged), find the noisy ones, and design an on-call that people can sustain.25 min
08Capacity and SaturationUse Little's law to work out how many database connections a workload needs, see exactly when Tallybook's pool ran out, and size the pool and the bulk job so month-end can't do it again.25 min
09Toil and Reliability WorkMeasure toil (repetitive manual operations work), decide what to automate first, and set the error budget policy and practices that keep reliability work from being crowded out.20 min
10Final ProjectPlan your final project, a reliability review that finds the real cause of the month-end outages with metrics, logs and traces, and sets the SLOs, alerts, capacity plan and practices that prevent them.20 min
Final assessmentPass it (60% or more) to earn the Observability and Site Reliability course completion badge.

Loading…

Observability and Site Reliability

About 7 hours

10 modules

1 badge

Final project

Optional verified certificate

Curriculum

01What Observability IsWhat observability means, the three kinds of telemetry (metrics, logs and traces), the four golden signals, and a first look at the minute-by-minute record of Tallybook's month-end outage.15 min

02Metrics and PercentilesWhy averages hide what users experience, how percentiles (p50, p95, p99) show it, how to read latency over a day, and which metrics to put on a service's first dashboard.25 min

03Structured LogsWhy logs should be structured (JSON) rather than free text, how to filter and count them like data, and how to use them to find the event that started an incident.25 min

04Distributed TracesRead traces made of spans, rebuild a request's path through the web tier, API, database and email provider, and compare where time goes in normal and incident traffic.15 min

05SLIs and SLOsDefine service level indicators from what users experience, set objectives and error budgets, and measure Tallybook's August against them, including how much of the month's budget two mornings used.25 min

06Burn-Rate AlertsAlert on how fast the error budget is being spent rather than on fixed thresholds, compute burn rates over short and long windows, and see when each alert would have fired on 31 August.25 min

07Alert Quality and On-CallMeasure how useful each alert is (how often it needs action, when it wakes people, how fast it's acknowledged), find the noisy ones, and design an on-call that people can sustain.25 min

08Capacity and SaturationUse Little's law to work out how many database connections a workload needs, see exactly when Tallybook's pool ran out, and size the pool and the bulk job so month-end can't do it again.25 min

09Toil and Reliability WorkMeasure toil (repetitive manual operations work), decide what to automate first, and set the error budget policy and practices that keep reliability work from being crowded out.20 min

10Final ProjectPlan your final project, a reliability review that finds the real cause of the month-end outages with metrics, logs and traces, and sets the SLOs, alerts, capacity plan and practices that prevent them.20 min

Final assessmentPass it (60% or more) to earn the Observability and Site Reliability course completion badge.