Bloomberg's Lei Zhang reveals the critical lessons learned from integrating AI across a 9,000-engineer organization, exposing the hidden complexities of scaling AI beyond initial productivity boosts.
Bloomberg's AI Engineering Scale
- Over 500 employees focus exclusively on AI product research and engineering for the Bloomberg Terminal.
- The company operates one of the world's largest private networks and JavaScript codebases, supporting thousands of distinct applications ("functions").
- Bloomberg actively contributes to open-source initiatives, including the creation of K8s Envoy AI gateways.
- Zhang emphasizes the necessity of leveraging AI tools to optimize internal engineering workflows given the scale of software and data.
"We are building a lot of software and use a lot of data to empower our flagship product, which is called the Bloomberg Terminal." – Lei Zhang
Beyond Greenfield: AI's Productivity Plateau
- Initial AI tools boosted quick test generation and one-off script creation.
- Productivity measurements "dropped pretty quickly" when moving beyond "greenfield" (new, unconstrained) projects.
- Managing hundreds of millions of lines of code with AI-generated patches introduces exponential system complexity.
- Bloomberg shifted focus to AI applications for less preferred developer tasks, like maintenance and migration.
"Unless we deploy and try, we wouldn't know what's the best way to benefit from all the awesome work." – Lei Zhang
AI Agents for Code Evolution and Incident Response
- Uplift Agents: These agents broadly scan codebases, identify applicable patches, and generate pull requests with fixes and explanations. They significantly outperform previous regex-based refactoring tools.
- Incident Response Agents: These agents rapidly analyze telemetry, codebases, feature flags, and call traces without human bias, accelerating incident troubleshooting.
- Deployment Challenges: AI-generated patches often lack deterministic verification, increasing the average number of open pull requests and time-to-merge due to necessary human review.
- The shift in focus moves from "how to achieve" to "what to achieve" with AI.
"The benefits is it's very fast, the challenge is also it's very [complex]." – Lei Zhang
The "Paved Path" for AI Tooling
- AI Gateway: Provides a central point for teams to experiment with different models, gain visibility into model usage, and receive guidance on optimal model selection.
- Tool Discovery Hub: A directory for Multi-Component Platform (MCP) servers (custom integration servers connecting to metrics, logs, topology, alarms, and SLOs), fostering collaboration and preventing duplication.
- Standardized Deployment: Utilizes a Platform-as-a-Service (PaaS) for Software Development Life Cycle (SDLC) and runtime environments, abstracting operational and security complexities.
- Quality Control: Facilitates easy proof-of-concept generation for creativity but enforces strict quality control for production deployments, prioritizing stability and reliability.
"We want to make the right thing extremely easy to do, and we want to make sure the wrong thing is ridiculously hard to do." – Lei Zhang
Cultivating AI Adoption and Leadership
- Onboarding Integration: AI coding best practices are now part of Bloomberg's 20-year-old onboarding program, creating "change agents" who challenge existing methods within their teams.
- Community-Driven Adoption: Leveraging existing "champ" and "guild" programs (cross-organizational tech communities), Bloomberg bootstrapped an AI productivity community, fostering shared learning and inner-source contributions.
- Leadership Workshops: Recognizing that individual contributors adopt AI faster than leadership, Bloomberg is rolling out workshops to equip managers with the knowledge needed to guide AI-driven software development.
"Our data shows individual contributors have a much better, stronger adoption than our leadership team." – Lei Zhang
Investor & Researcher Alpha
- Capital Reallocation: AI fundamentally alters the cost function of software engineering. Tasks previously expensive become cheaper, while others (like robust verification of AI-generated code) become more critical and potentially costlier. Investors should seek companies optimizing for this new cost landscape, particularly those investing in AI-driven verification and integration platforms.
- Bottleneck Shift: The bottleneck shifts from raw code generation to the integration, verification, and secure deployment of AI-generated code. Solutions addressing code quality, security, and efficient human-AI collaboration in complex, existing systems will capture significant value.
- Research Direction: Purely generative AI research for "greenfield" projects may yield diminishing returns for enterprise adoption. Focus shifts to AI for code evolution, migration, incident response, and deterministic verification within large, legacy codebases. Research into AI-driven code quality assurance and automated integration testing gains urgency.
Strategic Conclusion
AI fundamentally redefines the economics of software engineering, making certain tasks cheaper and others more expensive. The industry's next step involves a critical re-evaluation of "high-quality software engineering" principles, leveraging AI to achieve new levels of efficiency and reliability while managing inherent complexities.
