SRE — Site Reliability Engineering — is the practice of treating service availability as an engineering problem with metrics and budgets, rather than as a watchkeeping task. It originated at Google and is now the reference operating model for the moment availability stops being an infrastructure matter and becomes a commitment to the business.
Its central idea is counterintuitive: the goal is not for the system to never fail. The goal is to decide, explicitly and numerically, how much failure the business can tolerate, and to use that margin as a budget for innovation.
What problem does SRE solve?
There is a conversation that repeats in almost every organisation shipping software continuously. Development wants to release; operations wants stability. Every release is a risk, and every week without releasing is a missed opportunity. The argument gets settled by hierarchy, by habit, or by whoever argues best — but rarely by evidence.
The classic symptom: after a visible outage, deployments are frozen “until further notice”. Nobody knows when that notice arrives, because no criterion defines it. Months later the cycle repeats.
SRE solves precisely this. It turns reliability into a figure both sides agree on before the conflict, and lets that figure decide. When the service is above its objective, you ship. When it drops below, priorities change. The discussion stops being about opinions and becomes about a number both sides accepted.
SRE and DevOps: how they relate
This is the most common confusion, and it is worth clearing up early.
DevOps describes the goal: removing the wall between the people who build software and the people who operate it, so changes ship frequently and safely. It is a philosophy and a set of cultural principles.
SRE describes one concrete implementation of that goal, with prescriptive practices and defined metrics. Where DevOps says “share responsibility for production”, SRE says how: define these indicators, agree on these objectives, measure this budget, and when it runs out, do this.
| Dimension | DevOps | SRE |
|---|---|---|
| What it is | Philosophy and culture | Engineering discipline with defined practices |
| Question it answers | How do we deliver software better? | How much reliability do we need and how do we sustain it? |
| Typical unit of measure | DORA metrics | SLIs, SLOs and error budgets |
| Origin | Community movement (2009) | Internal Google practice, published in 2016 |
| Relationship to risk | Reduce it through automation | Budget it explicitly |
The most quoted formulation captures it well: SRE is what happens when you ask a software engineer to design an operations team. It is not a choice between the two; in practice most teams adopting SRE were already working with DevOps practices and needed the next level of precision.
SLI, SLO and SLA: the three terms that organise the conversation
These three concepts are the minimum vocabulary of SRE, and they get mixed up constantly because in commercial language almost everything ends up being called an “SLA”.
- SLI (service level indicator): a carefully defined quantitative measure of some aspect of the service. The most common ones are request latency, error rate, and availability expressed as the proportion of successful requests.
- SLO (service level objective): the value or range that indicator must meet. For example: “99% of read requests respond in under 100 milliseconds”.
- SLA (service level agreement): the contract with the customer, which carries consequences — usually financial — if the objective is missed.
The test that separates an SLO from an SLA takes one question: what happens if it is missed? If there is no explicit consequence, it is an SLO. A genuine SLA breach is a contractual matter.
Three practical criteria when defining objectives, drawn from the documented experience of those who have been doing this longest:
- Do not pick the target by looking at current performance. Adopting today’s incidental results as a goal ties the team to sustaining a system that demands heroics and cannot be improved without a redesign.
- Have as few objectives as possible. If an objective never helps you win a prioritisation conversation, it is probably not worth keeping.
- Do not overachieve. Users build on what the service actually does, not on what it promises. A service that promises 99.9% and consistently delivers 99.99% creates dependencies that will break the day it returns to its objective.
The error budget: the number that unblocks the argument
The error budget is the gap between 100% and the agreed objective. If the objective is 99.9% successful requests for the quarter, the budget is that 0.1%: the amount of failure the service can spend without breaking its commitment.
The mechanism is simple, which is why it works. While budget remains, changes ship. If the budget runs out, releases pause and effort redirects to testing, resilience and root-cause fixes. An incident that consumes 20% of the budget is exactly that: 20%, not “a serious problem” or “a minor thing” depending on who tells the story.
The organisational effect is more interesting than the technical one. The development team no longer needs anyone to hold it back, because it manages its own margin: nobody wants to spend the budget on a rushed deployment if that blocks next month’s launch. And an outage at a cloud provider also consumes budget, which aligns everyone around the same number instead of distributing blame.
What each nine costs
The decision about the target is economic before it is technical. This table translates each level into tolerated downtime over a calendar year:
| Availability objective | Tolerated downtime per year | Tolerated downtime per month |
|---|---|---|
| 99% | 3 days 15.6 hours | 7 hours 18 minutes |
| 99.9% | 8 hours 45 minutes | 43.8 minutes |
| 99.95% | 4 hours 23 minutes | 21.9 minutes |
| 99.99% | 52.56 minutes | 4.38 minutes |
| 99.999% | 5.26 minutes | 26.3 seconds |
Each additional nine cuts tolerated downtime by an order of magnitude, but the cost of achieving it does not grow proportionally: it grows much faster. That is why the right level is not the highest the technology allows, but the one matching what the business loses per hour of downtime. A monthly billing system and a payments gateway do not need the same number, and treating them alike is expensive in one case and insufficient in the other.
Toil: the operational debt you can actually measure
SRE gave a name to something every operations team knows: toil is manual, automatable, repetitive work with no enduring value that grows in direct proportion to the size of the service. Restarting a process by hand every night, applying the same patch server by server, running the same twelve-step procedure every time the same alert fires.
It is not the same as “boring work” or operational work in general. The defining characteristic is that it scales linearly with growth: if the service doubles in size, this work doubles in hours. It is debt that accrues interest.
The discipline sets an explicit, verifiable ceiling: keep that work below 50% of each engineer’s time, and spend at least the other half on engineering work that reduces it or adds capability to the service. The value of fixing the ceiling is that it makes visible a deterioration that normally goes unnoticed: when a team crosses that line consistently, it stops being an engineering team and becomes a task team, without anyone having decided that.
Blameless post-mortems: learning from failure
After every significant incident, the practice is to write an analysis reconstructing what happened, what allowed it, and what changes so it does not happen the same way again. The condition that makes it work is that it be blameless: the focus is on system conditions, not on the person who ran the command.
The underlying assumption is engineering, not politeness. If a single individual action could take the service down, the relevant finding is not that someone made a mistake, but that the system allowed that mistake with no safety net. That is the fix worth making.
The practical effect shows up in early reporting. In a culture looking for someone to blame, whoever notices an anomaly at 2 a.m. waits to see whether it resolves itself. In a culture that examines conditions, they report it immediately. The difference between those two reactions is hours of downtime.
Where SRE ends and continuity begins
It is worth drawing a line that often stays blurry. SRE deals with reliability in normal operation: the service meeting its objective day to day, and degradations being detected and corrected before the customer feels them.
Continuity objectives — RTO and RPO — belong to a different conversation: disaster recovery, which defines how long the service may take to come back and how much data is acceptable to lose when something exceptional happens, such as losing an entire region.
They complement and need each other, but they are not interchangeable. A service can meet its availability objective every month and still have a recovery strategy that does not work. And a flawless recovery plan does not prevent death by a thousand small degradations.
SRE on AWS
SRE practices are provider-independent, but they need instrumentation to exist: without measurement there is no indicator, without an indicator there is no objective, and without an objective there is no budget. AWS provides the components to close that loop as managed services.
- Amazon CloudWatch consolidates metrics, logs and alarms. CloudWatch Application Signals lets you define service level objectives over applications and track error budget consumption from the objectives dashboard in the console, instead of calculating it by hand in a spreadsheet.
- AWS X-Ray provides the distributed tracing that pinpoints which hop degrades a request — essential when the indicator is latency and the architecture is microservices.
- AWS Fault Injection Service runs controlled failure experiments to confirm the system behaves as expected, rather than discovering it during the real incident. It is the foundation of our chaos engineering practice.
- Automation, the AWS Systems Manager capability for operational procedures, uses reusable runbooks to execute the maintenance and remediation tasks that repeat — the concrete way to push toil below the ceiling.
- The AWS Well-Architected Framework, and its reliability pillar in particular, structures periodic review of the architecture against known good practice. It is the usual starting point for a Well-Architected Review.
All of this rests on a foundation of observability: correlated metrics, logs and traces. A service level objective with no observability behind it is a statement of intent.
How to start with SRE without reorganising the company
Adoption does not require creating a new department or hiring a scarce profile. It starts by agreeing on vocabulary and measuring:
- Pick the two or three services that sustain the business. Not every system needs formal objectives.
- Define one indicator per service, from the customer’s experience. Proportion of successful requests or perceived latency, not server CPU.
- Agree the objective with the business owner, not only with the technical team. It is a decision about what an outage costs, and that figure does not live in the systems department.
- Measure the error budget for a quarter before applying consequences. The first quarter is for calibrating the number, not for halting deployments.
- Run blameless post-mortems from the very first incident. It is the cheapest practice to adopt and the one that changes team behaviour fastest.
At Caleidos we support this adoption as part of our DevOps practice and sustain it over time with Caleidos Lens©, our 24×7 service desk: objectives agreed in a workshop only become real when someone measures them every day and acts when they drift.
Want to see how we apply it? Check our success stories or let’s talk.
