True

Turn Platform Operations Into Improvement

A practical workflow for using reliability, incidents, toil and cost data to improve the platform.

How to Turn Platform Operations Into a Continuous Improvement Loop

A practical workflow for using reliability, incidents, toil and cost data to decide what the platform should improve next.

Getting a platform into production isn't really the end of the work. It's actually where the most useful information starts showing up.

Real workloads arrive, developers use the platform in ways nobody quite expected, incidents expose weak assumptions nobody noticed before, costs shift, manual work quietly piles up, and some automation works great while other bits just move the problem somewhere else. The common mistake is treating operations as a separate phase whose only job is keeping the lights on.

A better model looks more like this:

                Operate → Observe → Learn → Prioritise → Improve → Measure
              

At m-tech1, Operate & Improve is really about turning production behaviour into actual engineering decisions. Here's roughly how we go about it.


Step 1: Define what healthy actually means

Before improving anything at all, the expected operating state needs to be defined first.

For a critical service, that usually includes availability, latency, error rate, deployment success, recovery time, infrastructure cost, and operational workload, all expressed as measurable targets rather than vague goals.

Metric    Target
Availability    99.95%
P95 latency    < 300 ms
Change failure rate    < 10%
MTTR    < 60 min
Manual deployment steps    0
Platform tickets    < 5/team/month


Tools like Grafana, Prometheus, Datadog, AWS CloudWatch, or New Relic can make these indicators visible day to day. The goal here isn't more dashboards for their own sake. It's agreement on what "healthy" actually means in the first place.


Step 2: Connect technical signals to user impact

Watching CPU utilisation on its own rarely tells you whether customers are actually having a problem. Operational signals need to connect infrastructure behaviour all the way to service behaviour.

A useful way to think about this chain:

                Infrastructure
      ↓
Platform
      ↓
Application
      ↓
Service SLO
      ↓
Customer impact
              

High Kubernetes CPU utilisation by itself might not matter much. But trace it further and the picture changes:

                CPU saturation
      ↓
Pod throttling
      ↓
API latency
      ↓
Checkout SLO breach
      ↓
Customer impact
              

That version matters a lot. This is exactly where OpenTelemetry earns its keep, since it connects metrics, logs, and traces across the whole delivery stack. The question worth asking isn't "is infrastructure healthy?" It's "is the service behaving within the conditions the business actually expects?"


Step 3: Treat incidents as engineering input

A postmortem that ends with "engineer made a configuration error" doesn't really tell you anything useful. What's worth doing instead is reconstructing the system conditions that allowed the incident to happen at all.

                Production incident
       ↓
Manual configuration change
       ↓
No policy validation
       ↓
No automatic rollback
       ↓
Recovery required senior engineer
              

Look at that chain and suddenly there are four concrete platform improvements sitting right there:

  • move configuration to Git
  • add policy-as-code
  • introduce automated rollback
  • create a recovery runbook

Tools that help here include PagerDuty or Opsgenie, Jira, Confluence, GitHub, Grafana, and incident.io. The postmortem should really produce platform backlog items, not just a document that gets filed away.


Step 4: Measure operational toil

One of the strongest signs a platform is weak somewhere is repetitive manual work. The simplest way to find it is to just ask senior engineers what they end up doing over and over every week.

Common answers tend to include restarting workloads, fixing pipeline permissions, provisioning environments, rotating credentials, changing IAM roles, debugging deployment failures, resizing infrastructure, and restoring databases.

Then it's worth actually quantifying it:

Task    Frequency/month    Time each    Monthly effort
Environment provisioning    20    45 min    15 h
IAM requests    30    20 min    10 h
Pipeline troubleshooting   15    40 min    10 h
Manual scaling    8    30 min    4 h




Once that's on paper, prioritisation gets a lot easier. If an automation project takes 12 engineering hours but removes 120 hours of annual toil, the business case basically makes itself, which is a much stronger argument than just saying "we should automate more." 


Step 5: Bring cost into the same operating loop

Reliability and cost really shouldn't be treated as separate conversations.

Picture a service running at 99.99% availability, but sized at four times what it actually needs. Technically healthy. Economically wasteful. Worth combining operational metrics with cloud cost data to catch things like this:

                Usage
  +
Performance
  +
Reliability
  +
Cloud Cost
       ↓
Architecture decision
              

Useful tools include AWS Cost Explorer, the AWS Cost and Usage Report, various FinOps platforms, Grafana, and Kubernetes cost tools like OpenCost. That combination lets teams ask sharper questions: can this workload be rightsized? Are non-production resources running around the clock for no reason? Is autoscaling actually behaving the way it should? Are commitments aligned with real usage? Is reliability quietly being bought through overprovisioning nobody questioned?

Done well, FinOps stops being a monthly finance exercise and just becomes part of engineering operations.


Step 6: Build one improvement backlog

Keeping separate backlogs for reliability, cloud cost, platform requests, and incidents tends to scatter attention. Better to pull the most important operational signals into a single prioritisation process.

Finding Impact Frequency Effort Priority
Manual environment provisioning High High Medium VERY HIGH
Recurring deployment failures High Medium Low VERY HIGH
Oversized dev clusters Medium Continuous Low HIGH
Missing traces in one service Medium Low Low MEDIUM
Rare specialist request Low Low High LOW

A useful formula here is impact multiplied by frequency, toil, and risk, weighed against implementation effort. That keeps the platform team focused on where the actual leverage is.


Step 7: Improve through small controlled changes

Operational maturity tends to come from lots of small improvements rather than one big transformation project. A monthly cycle might look something like:

                Observe production
       ↓
Identify recurring friction
       ↓
Choose highest-leverage problem
       ↓
Implement improvement
       ↓
Pilot with one workload
       ↓
Measure
       ↓
Standardise
              

Month 1 might mean automating environment provisioning. Month 2, adding standard observability. Month 3, introducing deployment policy checks. Month 4, automating rollback. Each change quietly becomes another platform capability that sticks around.


Step 8: Prove that the platform is getting better

After every improvement, it's worth comparing before and after directly.

Metric Before After
MTTR 150 min 45 min
Platform tickets/month 42 16
Environment provisioning 2 days 20 min
Change failure rate 14% 6%
Non-prod cloud spend €18k €12k

These numbers are illustrative, but the method matters more than any specific figure. If a platform team can't show whether reliability, delivery friction, cost, or operational toil actually improved, it's genuinely hard to know whether the investment created any value at all.


The operating model changes

Eventually this builds a different relationship with production altogether.

Instead of "incident happens, fix it, move on," it becomes "incident happens, learn from it, improve the platform, prevent it from recurring." Instead of "ticket comes in, engineer completes the request," it becomes "tickets repeat, spot the pattern, build self-service instead." Instead of "cloud bill goes up, finance asks why," it becomes "cost signal appears, engineering investigates, architecture improves."

Operations basically becomes an input into product development for the platform itself.


The final test

Worth asking your platform and SRE teams directly: what did production teach us last month, and what did we actually change because of it?

If there's no clear answer, the organisation is probably operating the platform. It's probably not improving it though.

At m-tech1, Operate & Improve is built entirely around closing that loop: observe real behaviour, identify the highest-value friction, improve the platform, and measure whether the change actually worked.

Turn operations into measurable platform improvement.

To install this Web App in your iPhone/iPad press and then Add to Home Screen.