Bloomberg's Lei Zhang, head of technology infrastructure, details how AI fundamentally reshapes software engineering. His team, managing 9,000 engineers and a massive codebase, found that AI alters the cost function of development, making some tasks cheaper and others more expensive, demanding a strategic re-evaluation of engineering practices.
AI's Double-Edged Sword for Productivity
- “We quickly form a team, people start kind of like release a set of capabilities so that people start iterating on utilizing the toolings... We look at the typical developer productivity measurements. We ran a few surveys. It was very obvious that people felt like there's much quicker proof of concept, people rolled out tests. There's a lot of one-time use scripts being generated.”
- Initial Gains, Hidden Costs: AI accelerates greenfield development, proof-of-concepts, and one-off scripts. This is like a powerful calculator for simple arithmetic – fast, but not a solution for complex equations without human guidance.
- Quality Bottlenecks: Rapid code generation increases open pull requests and merge times, as human review and verification remain critical. This is a factory producing parts faster, but if quality control doesn't scale, inventory piles up.
- Operational AI: AI agents improve incident response by quickly analyzing telemetry and code traces without human bias, leading to faster problem resolution. Imagine an unbiased, super-fast detective sifting through all evidence instantly.
The "Golden Path" Platform Strategy
- “Bloomberg is kind of in the middle. If you look at the golden ones, we kind of believe in providing a golden path with enablement teams... One of the guiding principles for us is we want to make the right thing extremely easy to do and we want to make sure the wrong thing is ridiculously hard to do.”
- Controlled Experimentation: Bloomberg built a "golden path" platform, allowing engineers to experiment with AI tools within defined boundaries. This is like a well-designed playground where kids explore freely, but safely.
- Centralized Guidance: A gateway and tool directory prevent duplication and guide teams to suitable AI models. This is a library catalog, helping you find the right book without searching every shelf.
- Standardized Deployment: A Platform-as-a-Service (PaaS) handles the SDLC and runtime for AI tools, reducing friction. This is a pre-built kitchen where chefs focus on cooking, not building the stove.
Organizational Adaptation and Cultural Shift
- “Our data shows individual contributors have a much better, stronger adoption than our leadership team... Often times the stuff that they learned before might not be exactly applicable, still very valuable, but there's some missing piece there to make sure they can continue to guide the team to do the right thing.”
- Onboarding as a Catalyst: Integrating AI coding into new hire training drives adoption and challenges existing norms. New hires, learning the "new way," become internal advocates.
- Community-Driven Learning: Leveraging existing internal communities (champs and guilds) for AI productivity fosters shared learning and boosts inner-source contributions.
- Leadership Knowledge Gap: Individual contributors adopt AI faster than leadership. Managers and tech leads require specific training to guide teams effectively in an AI-augmented development environment.
Key Takeaways:
- Strategic Implication: AI fundamentally changes the economics of software development. Organizations must re-evaluate what constitutes "high-quality" engineering and adapt their processes.
- Builder/Investor Note: Prioritize platforms that provide guardrails and guidance for AI tool usage, focusing on deterministic verification and robust testing. Uncontrolled AI deployment risks technical debt.
- The "So What?": The next 6-12 months will see a bifurcation: companies that strategically integrate AI into their engineering culture and platforms will gain significant efficiency, while those that don't will struggle with quality and adoption.
Podcast Link: https://www.youtube.com/watch?v=Q81AzlA-VE8
Bloomberg's Lei Zhang reveals the strategic shifts and organizational re-engineering required to integrate AI at scale within a 9,000-engineer organization, exposing critical bottlenecks and new opportunities in software development.
Initial AI Deployment: Productivity Plateaus and Complexity
- Bloomberg's early AI for coding initiatives, launched over two years ago, initially showed promise in generating quick proofs of concept and one-off scripts. However, these gains quickly diminished when applied to complex, existing codebases. Lei Zhang highlights the inherent challenge: managing hundreds of millions of lines of code with AI requires extreme caution due to the exponential complexity of software assets.
- Initial AI tools boosted rapid prototyping and test generation.
- Productivity measurements dropped significantly beyond "greenfield" projects.
- AI-generated code increased open pull requests and time-to-merge, creating new review burdens.
- “The benefits are it’s very fast. The challenge is also it’s very wrong.” – Lei Zhang
- Zhang emphasizes the shift from "how to achieve" to "what to achieve" with generative AI.
Strategic AI Agents: Automating Mundane and Critical Tasks
- Bloomberg pivoted its AI strategy from general coding assistance to targeted, high-impact applications. The focus shifted to automating less preferred developer tasks and critical incident response, leveraging AI's speed and unbiased analysis.
- Uplift Agents: These agents broadly scan codebases to identify and apply patches, significantly improving upon previous regex-based refactoring tools.
- Incident Response Agents: AI agents rapidly analyze telemetry, codebases, feature flags, and call traces during incidents, providing unbiased insights faster than human troubleshooting.
- Zhang notes the challenge of deterministic verification for AI-generated patches, especially without robust testing and linting.
- “In an instance, it can go through your codebase really quickly, it can go through your telemetry system very quickly, it can go through your feature flags very quickly… in an unbiased lens.” – Lei Zhang
The "Paved Path": Centralizing AI Development and Deployment
- To prevent chaos and duplication across its vast engineering team, Bloomberg established a "paved path" for AI tool development. This centralized approach provides a gateway for model experimentation, a directory for tool discovery, and a standardized deployment platform.
- A gateway enables teams to experiment with different AI models, gain visibility into usage, and receive guidance on optimal model selection.
- A tool discovery hub (MCP directory) prevents redundant development of incident response agents (Monitoring, Control, and Processing servers), encouraging collaboration.
- Standardized tool creation and deployment via a platform-as-a-service (PaaS) handles Software Development Life Cycle (SDLC) and runtime environments, including operational and security aspects.
- Bloomberg balances rapid proof-of-concept generation with strict quality control for production deployments, prioritizing system reliability.
- “We want to make the right thing extremely easy to do, and we want to make sure the wrong thing is ridiculously hard to do.” – Lei Zhang
Organizational Adoption and Leadership Upskilling
- Driving AI adoption across 9,000 engineers required a multi-pronged organizational strategy. Bloomberg integrated AI coding into its 20-year-old training programs and leveraged existing internal communities. A critical insight emerged: individual contributors adopted AI faster than leadership.
- Onboarding Training: New hires learn AI coding best practices from day one, acting as "change agents" within their teams.
- Community Programs: Existing "champ" and "guild" programs (cross-organizational tech communities) were leveraged to foster an AI productivity community, promoting shared learning and inner-source contributions.
- Leadership Workshops: Bloomberg identified a gap in leadership's ability to guide AI-driven software development and is rolling out workshops to equip managers with necessary knowledge.
- “Our data shows individual contributors have a much better, stronger adoption than our leadership team.” – Lei Zhang
Investor & Researcher Alpha
- Capital Reallocation: AI's impact on software engineering cost functions suggests a shift in where development capital yields the highest return. Investors should scrutinize companies still investing heavily in manual, repetitive tasks now automatable by AI, favoring those strategically deploying AI for high-leverage problems like incident response and automated patching.
- New Bottlenecks: The rise of AI-generated code creates new bottlenecks in code review and verification. Solutions that enhance AI-driven code quality, provide deterministic verification, or streamline human review processes will become critical infrastructure.
- Research Direction: The "paved path" approach highlights the need for AI platforms that balance rapid experimentation with robust production quality. Research into AI governance, model selection guidance, and secure, scalable deployment environments for internal AI agents offers significant value.
Strategic Conclusion
- AI fundamentally redefines the economics of software engineering, making some tasks cheaper and others more expensive. The industry's next step involves a critical re-evaluation of what constitutes high-quality software engineering, leveraging AI to optimize for reliability and strategic impact.
