Posts

Showing posts from 2026

Design Degraded Modes Before the Next Incident

Most reliability plans focus on keeping every capability available. That is an admirable goal, but it is not a complete operating strategy. Complex systems will eventually lose a dependency, exhaust a resource, or encounter demand they cannot serve normally. When that happens, teams need more than alerts and runbooks. They need an agreed way for the service to become less capable without becoming completely unavailable. Degradation Is a Product Decision A degraded mode defines which capabilities can be limited, delayed, simplified, or disabled to protect the service’s most important outcomes. An online service might pause recommendations while preserving core transactions. An internal developer platform might delay nonessential analytics while continuing deployments. A data pipeline might prioritize current operational events over historical backfills. Engineering cannot make these tradeoffs alone during an incident. Product, operations, security, and business leaders need to deci...

Put Expiration Dates on Your Golden Paths

A paved road can become a legacy road Golden paths help engineering teams start quickly and operate consistently. They package sensible defaults for repositories, pipelines, observability, security controls, and runtime configuration. But a golden path is not automatically a current path. Without deliberate maintenance, the recommended approach slowly accumulates outdated dependencies, unnecessary controls, and assumptions that no longer match how teams build software. Because the path carries an official label, developers may continue using it long after better options exist. Platform leaders should treat every golden path as a versioned product with an expiration date. Expiration does not mean automatic deletion. It creates a decision point where the organization must renew, replace, or retire the path. Define renewal criteria before launch A golden path should not remain recommended simply because it still works. Its owners should periodically verify that it remains supportabl...

Treat Platform Backlogs as Investment Decisions

Platform teams often manage their backlog like a collection of internal feature requests. The loudest stakeholder, the largest team, or the most urgent escalation moves to the front. That approach feels responsive, but it produces a fragmented platform. Teams deliver one-off integrations and narrow workflow improvements while foundational problems remain unresolved. A platform backlog should be treated as an investment portfolio. Every item must compete for limited engineering capacity based on the value it creates across the organization. Prioritize Reusable Leverage The strongest platform investments remove recurring effort from multiple teams. They make a secure, reliable delivery path easier to use without requiring repeated coordination. Evaluate backlog items against a small set of practical questions: How many teams face this problem? How frequently does the friction occur? Does solving it reduce risk, delay, or operational work? Can the solution become a reusable cap...

Every Critical Service Needs a Reliability Owner

Shared ownership sounds collaborative, but it often produces unclear decisions. When a critical service has no named reliability owner, operational risks remain visible without becoming actionable. A reliability owner is not the person expected to fix every incident. The role is accountable for ensuring that reliability decisions are made, funded, and followed through. That distinction matters. Accountability should be clear even when implementation is distributed across several teams. Ownership Must Include Decision Rights Assigning a name in a service catalog is not enough. The owner needs authority to influence priorities when reliability work competes with feature delivery. For each critical service, the reliability owner should be able to: Define service-level objectives with product and business partners. Escalate unresolved operational risks. Prioritize work triggered by error budget consumption. Confirm that incident actions have owners and deadlines. Challenge d...

AI Coding Assistants Need Faster Feedback, Not Bigger Promises

Generated Code Is Only the First Step AI coding assistants can reduce the effort required to draft code, tests, documentation, and configuration. That matters, but typing has rarely been the main constraint in software delivery. Code still has to be understood, reviewed, integrated, secured, deployed, and operated. If those feedback loops are slow or unreliable, generating more code simply moves the queue downstream. Technology leaders should therefore treat AI adoption as a delivery-system change, not a tool rollout. The goal is not to maximize suggestions accepted. The goal is to improve the safe flow of valuable changes into production. Find the New Bottleneck Once teams produce changes faster, existing weaknesses become more visible. Test suites may take too long. Review queues may grow. Environments may be difficult to obtain. Security checks may generate noisy findings. Deployment pipelines may require manual coordination. These constraints will limit the value of AI regard...

Platform Exceptions Are Signals, Not Policy Failures

Platform teams often treat exceptions as a governance problem. A team requests a different deployment pattern, runtime, or security control, and the immediate response is to defend the standard. Standards matter. They reduce cognitive load, improve supportability, and make controls easier to automate. But a platform that cannot learn from exceptions will eventually become a constraint rather than an accelerator. Separate valid constraints from avoidable preferences Not every exception deserves approval. Some requests are simply attempts to preserve familiar tools or bypass reasonable controls. Platform teams should ask for the engineering constraint behind the request, not just the requested solution. A useful exception process should capture: The requirement the standard path cannot meet The operational and security risks introduced The owner and expected duration of the exception The evidence needed to remove or renew it This creates accountability without turning the conv...

Incident Reviews Should Change the System, Not Grade the People

Incident reviews often begin with the right intention and drift into the wrong question. Instead of asking how the system allowed a failure to reach customers, teams start evaluating whether individuals followed the expected process. That shift produces cautious meetings, shallow conclusions, and action items focused on reminders or training. It may create the appearance of accountability, but it rarely improves reliability. Look Beyond the Last Human Action An engineer’s mistake may be the final event before an incident, but it is seldom the complete cause. Reliable systems assume that people will mistype commands, misunderstand alerts, overlook documentation, and make decisions with incomplete information. The useful question is not, “Why did someone make that mistake?” It is, “Why was one ordinary mistake able to create this level of impact?” That question directs attention toward weak deployment controls, unclear ownership, unsafe defaults, missing observability, excessiv...

Documentation Is Part of the Platform Contract and Makes Your Agents Better

Platform teams often treat documentation as the final step before launch. That is backwards. If developers cannot understand, evaluate, and safely use a capability, the capability is not finished. Good documentation is more than a collection of setup instructions. It is part of the interface between a platform team and its users. Like any interface, it needs clear ownership, predictable structure, and ongoing maintenance. That has always mattered for developers. In an AI-enabled engineering organization, it matters even more. Documentation exposes platform complexity When a basic task requires a long guide filled with exceptions, the documentation is not necessarily the problem. It may be revealing unnecessary complexity in the product. That makes documentation work valuable design feedback. Ask an engineer unfamiliar with the service to follow the guide. Watch where assumptions appear, terminology shifts, or required decisions lack context. Those points often indicate that the platfor...

Standardize Delivery Outcomes, Not Every Team’s Workflow

Engineering leaders often pursue standardization by asking every team to use the same workflow. The intent is reasonable: reduce risk, simplify support, and make delivery easier to govern. But identical workflows are rarely the right goal. A mobile application, a shared library, and a regulated backend service have different release constraints. Forcing them through one rigid process creates workarounds, manual exceptions, and resentment toward the platform. The better approach is to standardize the outcomes that matter while allowing teams flexibility in how they achieve them. Define the delivery contract A delivery contract describes what must be true before software reaches production. It should focus on observable controls rather than a prescribed sequence of tools and pipeline stages. For example, the contract might require: Traceable source, build, artifact, and deployment records Automated security and policy checks A tested rollback or recovery mechanism Clear service...

Make Toil Reduction Part of Capacity Planning

Most engineering organizations say they want to reduce toil. Far fewer reserve capacity to do it. That gap matters. Repetitive operational work rarely disappears through good intentions. It competes with product delivery, security requirements, incidents, and customer commitments. Unless leaders account for toil during planning, teams will keep absorbing it as an unofficial tax. Toil is a capacity problem Manual access changes, recurring deployment fixes, repetitive incident recovery, and routine environment maintenance consume real engineering time. They also interrupt concentration and make delivery less predictable. Leaders often treat this work as the cost of operating a system. Some of it is. The mistake is accepting the same cost indefinitely without deciding whether automation, simplification, or removal would be a better investment. Do not ask teams to eliminate toil alongside a fully committed roadmap. Make the tradeoff visible. If reducing a recurring task requires ...

The Best Deployment Approval Is the One You Can Remove

Manual deployment approvals often begin as a reasonable response to risk. Over time, they become permanent fixtures, even when they add little protection. A person clicks a button, but rarely has enough context to independently validate the change. The approval delays delivery while creating the appearance of control. Leaders should ask a harder question: what evidence would allow us to remove this step? Replace permission with evidence Approval gates are most useful when judgment is genuinely required. They are less useful when reviewers simply confirm that automated checks passed or that a ticket contains the expected fields. Those conditions should be enforced by the delivery system. Build pipelines can verify test results, security policies, artifact provenance, change windows, and deployment readiness more consistently than a hurried reviewer. The objective is not to eliminate accountability. It is to make accountability explicit and repeatable. Define an approval remo...

Build Platform Roadmaps Around Friction, Not Features

Internal platform teams often inherit the planning habits of customer-facing product teams. They create feature backlogs, publish quarterly roadmaps, and celebrate shipped capabilities. That approach can produce a busy team without producing a better engineering system. A platform is valuable when it removes friction from software delivery. The roadmap should reflect that purpose. Start With the Cost of Friction Platform planning should begin with how engineers currently get work done. Where do they wait? Which tasks require tickets? Where do teams create their own scripts because the supported path is incomplete? These questions reveal more than a list of feature requests. They expose recurring costs across the organization: delayed environments, inconsistent deployment patterns, duplicated pipelines, unclear ownership, and security controls applied too late. A request from one team may be an isolated preference. The same workaround appearing across several teams is evidence of a...

An Error Budget Without a Decision Rule Is Just a Dashboard

Error budgets are useful because they translate reliability into a constraint the business can understand. But many organizations stop after calculating the number and displaying it on a dashboard. That is not an operating model. If consuming the entire error budget changes nothing, the budget has no authority. It is another metric teams can acknowledge and ignore. Define the consequence before the incident Leaders should agree on decision rules while services are healthy. Waiting until reliability deteriorates invites negotiation under pressure. Product leaders will defend commitments, engineering teams will debate severity, and customers will continue absorbing the impact. A practical policy should answer a few direct questions: What happens when budget consumption exceeds an agreed threshold? Which releases may continue, and which must pause? Who can approve an exception? What evidence is required before normal delivery resumes? The response should be proportional. ...

Stop Measuring Your Platform by Adoption Alone

Platform adoption is easy to celebrate. More teams using the deployment pipeline, service catalog, or infrastructure templates appears to prove that the platform is working. But adoption alone can hide an uncomfortable reality. Teams may use the platform because it is mandatory, while still losing time to confusing workflows, missing capabilities, and slow support. A platform can have near-universal usage and still deliver a poor developer experience. Measure the Friction Removed The purpose of an internal platform is not to centralize tools. It is to reduce the effort required to build, deliver, and operate software safely. Technology leaders should therefore look beyond registration counts and pipeline executions. Better questions focus on the work developers can complete without waiting for another team. Can a team create a production-ready service without filing tickets? Can developers understand why a deployment failed? Can teams make routine infrastructure changes th...

Your Internal Platform Needs an Escape Hatch

Platform engineering teams often aim to create a paved road: a supported path that makes delivery faster, safer, and more consistent. That is the right goal. The mistake is turning the paved road into a mandatory checkpoint for every workload. No internal platform can anticipate every technical requirement. When teams cannot work outside its abstractions, unusual but legitimate needs become platform backlogs, exception meetings, or hidden workarounds. The platform intended to accelerate delivery becomes another dependency teams must manage. Standardize the common path A strong platform should optimize the work most teams perform repeatedly. That usually includes repository setup, build pipelines, deployment patterns, observability, identity, secrets, and infrastructure provisioning. These capabilities should be easier to adopt than to rebuild. Adoption driven by clear value produces better results than adoption driven by policy alone. The platform team should be opinionated a...

DevOps Is the Foundation That Enables Enterprise AI

AI may be the most important technology shift of our time, but organizations will not create sustainable value simply by purchasing an AI platform or giving developers access to new tools. AI must still be integrated, tested, secured, deployed, monitored, governed, and continuously improved. These are challenges that mature DevOps organizations already know how to solve. AI does not make DevOps less relevant. It makes DevOps essential. Moving From AI Experiments to Business Outcomes It has become relatively easy to create an AI proof of concept. A small team can connect to a large language model, build a demonstration, and produce impressive results in a short period of time. The difficult part is turning that demonstration into a reliable production capability. Enterprise AI introduces questions that the initial experiment may not answer: How will the application securely access company data? How will sensitive information be protected? How will model, prompt, and configuration...