With AI cranking out code, the real job of software leaders isn’t shipping faster — it’s building trust and resilience that lasts.
For most of my career, I measured my team’s success in terms of output. Like most engineering leaders, I have used lines of code, function points and more recently, agile velocity or sprint burndown charts. While all of these were imperfect, they felt responsible at the time because I was optimizing what I thought was the primary constraint: humans creating features.
These constructs disappeared with the emergence of generative AI. When my team started to use AI as a tool to generate code, define test cases or specify deployment configurations — all in a matter of seconds — we began to create output at a rate I never thought we could maintain. And then output was not our constraint, trust was our constraint: Trust that the AI-augmented systems would behave consistently under duress, trust that the systems would operate within compliance and trust that they would scale safely.
This was awakening to me. My role had to change: I was no longer a taskmaster; I was no longer helping lead teams to create features; I had to be an orchestrator — a leader combining human experience and AI acceleration for the purpose of developing systems that fostered reliability and trust.
From outputs to outcomes
I learned that the velocity does not equal the value. A codebase with AI contributions without vetting can degrade reliability more quickly than it increases feature layers.
At Twilio, I had a view of service-level objectives (SLOs) and error budgets that anchored our messaging infrastructure processing billions of interactions. Later, at Coinbase, I used the same metrics to help my teams make decisions for trading systems handling some of the greatest loads in the market. These measurements mattered more than any burndown chart, because they were tied to customers and trusted revenue. Currently, I am working on dashboards that review:
- SLOs: Are we meeting our commitment to reliability?
- Error budgets: Where is our risk threshold before affecting customers?
- Mean time to resolution (MTTR): How quickly can we restore services after an incident?
This is the executive conversation I had shifted to — not the question around “how quickly can we ship,” but the question of “how resilient are the services that we are shipping?” Resilience is not overhead and is all about strategy, much like we see with Google’s Site Reliability Engineering.
Managing cognitive load in teams leveraging AI
Generative AI is going to help speed the output, but it is also going to have a toll on the cognitive load of people doing work. I had a junior developer on my team use AI for all their code, which had a great pedigree in a review, but created a subtle performance issue in production. I learned a very valuable lesson about generating output from AI: AI output is a hypothesis, not an end state.
To help manage this, I introduced three practices:
- Tiered review standards (for critical paths): Need to increase scrutiny with AI-generated code on critical paths and look for security and performance profiling and stress testing.
- Critical inquiry coaching: I coach engineers on how to interrogate AI output. What assumptions were made? Where did it fail under load?
- Psychological safety: I work hard to create a context to encourage psychological safety for my teammates to feel safe when questioning their colleagues and machines.
This suite of practices is consistent with recommendations in the literature around cognitive load in engineering teams, which indicates that without cultural safeguards, AI displaces risk, from observable error to unobserved systemic fragility.
People management in an AI-influenced world
Leadership is always about people, even in the presence of AI, although our incentives and skillset have shifted. At Coinbase, I have seen junior engineers ramp up rapidly via AI, but I saw the potential for skills atrophy as well. To help mitigate this, I actively coached for depth: encouraging engineers to sketch alternative solutions without AI and develop their critical review habits.
I have also adjusted our contribution evaluation. Closed tickets and merged commits were often inflated because of automation, so they lost their meaning. The invisible work mentoring, improving observability and preventing outages became the extent of differentiation. I updated the performance rubrics to specifically reward this work.
The highest-valued individual on today’s teams will be the multiplier: The person who benefits resilience across the entire system.
Managing risk in high-stakes evolution
One of the more pressing insights within orchestration revolved around some sizable infrastructure migration. We were contemplating migrating from Redis to Valkey. While AI tools are available to re-run existing code, they cannot own risk. The risks were not related to syntax; they were eviction policies, latency of operation in burst traffic and the maturity of support libraries.
The job was not to write migration scripts. Our job was to design the phased rollout, go/no-go decisions vs narratives, validate monitoring and enforce rollback strategies. While AI is good at accelerating execution, we had to underwrite the strategy. This orchestration is what made migration feel like a viable option.
Output is no longer the bottleneck
As a consequence of the fast pace of development of generative AI, we have changed not only the form of tools that we have available to us, but the very nature of software.
For me, the bottleneck is no longer output, but trust, and I am no longer providing oversight on activities; I am orchestrating resilience. This is the next evolution of software leadership. It represents an existential adaptation for technology leadership. In an AI world, our purpose is no longer to simply manage code in production. It is to underwrite the trustworthiness of the living and evolving digital systems that run our businesses. The leaders who successfully master this shift from managing production to orchestrating resilience will be the leaders who create lasting value for years to come.
This article is published as part of the Foundry Expert Contributor Network.
Want to join?
