arrow_backBACK TO OBSERVABILITY & PRODUCTION DEBUGGING — FROM THE THREE PILLARS TO DEBUGGING A LIVE INCIDENT
Lesson 09Observability & Production Debugging — From the Three Pillars to Debugging a Live Incident5 min read

SLOs, SLIs & Error Budgets

October 28, 2026

TL;DR

You can't manage reliability as a feeling — turn it into numbers. An SLI (Service Level Indicator) is a measured number: % of requests under 300ms, % of non-5xx responses. An SLO (Service Level Objective) is your target for the SLI: '99.9% of requests succeed'. The error budget is 100% − SLO: a 99.9% SLO allows 0.1% failure ≈ 43 minutes/month. Why this changes everything: 100% reliability is the wrong goal — impossible and ruinously expensive — so pick an SLO users are happy with and treat the rest as a BUDGET you're allowed to spend. The budget is permission to take risks (ship features, do maintenance) while you're within it; burn it and you slow down and fix reliability; lots left and you ship faster. Alert on BURN RATE (Lesson 8) — page when you're spending the month's budget in hours. An SLO aligns engineers and the business on one number and tells you when to ship vs when to stabilise.

“Is the service reliable enough?” is a question teams argue about endlessly and vaguely — engineers want to fix things, product wants to ship features, and nobody agrees on how reliable is reliable enough. SLOs end the argument by turning reliability into a number everyone shares. This lesson covers the three linked concepts — SLIs, SLOs, and error budgets — and why error budgets, in particular, change how a team operates.

SLI, SLO, Error Budget

Three terms, building on each other:

SLIs, SLOs and error budgets: define 'reliable enough' as a number, then spend it wisely. SLI (Service Level Indicator): a measured number, % of requests under 300ms, % non-5xx. SLO (Service Level Objective): your target for the SLI, '99.9% of requests succeed'. Error budget (100% minus SLO): the allowed failure, 99.9% SLO = 0.1% = about 43 min/month. Why error budgets change everything: 100% reliability is the wrong goal, it's impossible and ruinously expensive, pick an SLO users are happy with (99.9%? 99.99%?) and accept the rest as a BUDGET you're allowed to spend; the budget is permission to take risks, ship features do maintenance as long as you're within it; burned the budget? slow down freeze risky changes fix reliability, lots of budget left? ship faster; alert on BURN RATE (Lesson 8), page when you're spending the month's budget in hours not when a single request is slow, this turns 'is it reliable?' from a feeling into a shared numeric decision
  • SLI (Service Level Indicator) — a measured number that reflects user experience: the percentage of requests that complete under 300ms, the percentage of responses that aren’t errors (non-5xx), the percentage of successful logins. A good SLI tracks something users actually care about. Pick SLIs that mean “users are having a good time.”
  • SLO (Service Level Objective) — your target for an SLI: “99.9% of requests succeed,” “99% of requests are faster than 300ms, measured over 30 days.” The SLO is the line between “reliable enough” and “not.”
  • Error budget — the inverse of the SLO: 100% − SLO = the failure you’re allowed. A 99.9% SLO means 0.1% of requests may fail — which over a month is about 43 minutes of downtime-equivalent. A 99.99% SLO allows ~4 minutes. The error budget is how much unreliability you can “spend” before you’ve broken your promise.

The three fit together: you measure the SLI, target it with the SLO, and the gap to 100% is your budget. Simple, but the error budget idea is quietly radical.

Why 100% Reliability Is the Wrong Goal

The instinct is to aim for 100% — no errors, no downtime, ever. This is a mistake, for two reasons:

  • It’s impossible. Everything fails sometimes — hardware, networks, dependencies, deploys. Chasing 100% is chasing a mirage.
  • It’s ruinously expensive and slows you to a crawl. Each additional “nine” of reliability costs exponentially more (more redundancy, more caution, more process) — and past a point, users can’t even tell the difference. The jump from 99.9% to 99.99% is a huge amount of engineering for a difference most users never notice.

So the mature move is to pick an SLO that users are genuinely happy with — maybe 99.9%, maybe 99.99%, depending on the service — and deliberately accept the rest as a budget you’re allowed to spend. You’re not aiming for perfect; you’re aiming for “reliable enough that users are happy,” and treating the remaining unreliability as a resource.

Error Budgets Change How You Operate

Here’s where it gets powerful. The error budget isn’t just a measure of failure — it’s permission to take risks:

  • Within budget? You’re free to ship features, do risky migrations, run maintenance — because you have reliability to spare. Reliability isn’t blocking you; you’ve earned the room to move fast.
  • Burned the budget (too many failures this month)? Slow down. Freeze risky changes, stop shipping features, and spend engineering on reliability until you’re back under budget. The budget tells you when to stop pushing and fix things.
  • Lots of budget left? You might even be too cautious — ship faster, take more risk; you have the headroom.

This resolves the eternal engineers-vs-product tension with a shared number instead of opinions. Nobody argues “should we ship this risky feature or fix reliability?” in the abstract — they look at the error budget. Within budget: ship. Over budget: stabilise. It aligns engineers and the business on one objective measure, and turns “is it reliable enough?” from a feeling into a decision anyone can read off a dashboard.

Burn Rate: SLOs Power Good Alerting

Finally, SLOs give you the foundation for the good alerting from Lesson 8: burn-rate alerts. Instead of paging on an arbitrary threshold (“latency > 500ms” — noisy, ad hoc), you alert on how fast you’re consuming the error budget:

  • Burning the month’s budget slowly → maybe a ticket; you have time.
  • Burning it fast → paging now, because at this rate you’ll blow the whole month’s budget in hours. That’s a real incident.

Burn-rate alerting is better because it’s tied to actual user impact over time, not a number someone guessed. A single slow request doesn’t page (it barely touches the budget); a sustained spike that’s eating your reliability does. This is why SLOs aren’t just a planning ritual — they become the basis of alerting that’s both sensitive to real problems and quiet about noise.

Getting Started with SLOs

A few practical notes for actually adopting them:

  • Start with one or two SLIs per critical service — availability (% non-error) and latency (% under some threshold) cover most cases. Don’t over-engineer a dozen SLOs on day one.
  • Base SLOs on what users need, not what you currently achieve. The question is “how reliable does this need to be for users to be happy?”, informed by data, not “what number makes us look good.”
  • Measure from the user’s perspective where you can — at the edge/load balancer, or client-side — so the SLI reflects what users actually experience, not just internal health.
  • Review and adjust. SLOs aren’t forever; revisit them as the service and user expectations change.

Done well, an SLO is one of the highest-leverage things a team can adopt: it defines reliability, aligns everyone, guides when to ship vs stabilise, and powers your alerting — all from a single, shared number.

Pitfalls to Avoid

  1. Aiming for 100%. Impossible and ruinously expensive, and users can’t tell. Pick an SLO users are happy with and spend the rest as budget.
  2. SLIs that don’t reflect user experience. Measuring internal health instead of what users feel. Choose SLIs (availability, latency) from the user’s perspective.
  3. Treating the error budget as pure failure, not a resource. It’s permission to take risks while you’re within it. Use it to decide ship-vs-stabilise.
  4. No link between SLO and action. If burning the budget doesn’t change behaviour (slow down, fix reliability), the SLO is decorative. Make it drive decisions.
  5. Arbitrary alert thresholds instead of burn rate. Page on SLO burn rate (Lesson 8), not a guessed number — it’s tied to real user impact over time.
  6. A dozen SLOs on day one. Start with one or two per critical service (availability, latency); add more only as needed.

Quick Recap

  • SLI = a measured number reflecting user experience; SLO = your target for it (“99.9% succeed”); error budget = 100% − SLO (the allowed failure, e.g. ~43 min/month at 99.9%).
  • 100% reliability is the wrong goal — impossible, ruinously expensive, and users can’t tell. Pick an SLO users are happy with and treat the rest as a budget.
  • The error budget is permission to take risks: within budget → ship; over budget → slow down and fix reliability. It aligns engineers and the business on one number.
  • SLOs power burn-rate alerting (Lesson 8): page when you’re spending the month’s budget in hours, not on arbitrary thresholds.
  • Start with one or two user-centric SLIs per critical service; base SLOs on user needs; make them drive decisions.

Next lesson: putting observability to work under pressure — a repeatable workflow for debugging a live incident, from the alert to the fix.