Zapier, a company built on connecting thousands of third-party APIs, faces a constant battle against "app erosion"—the relentless stream of bugs and reliability issues caused by external API changes. Lisa Orr, a product manager at Zapier, details how they're tackling this by empowering their support team to ship code, leveraging AI-driven tools to transform a traditional cost center into a high-velocity bug-fixing engine.
The Erosion Problem & Support's Unique Edge
- “At Zapier, we have over 8,000 integrations built on third-party APIs, and they are constantly changing, which I'm now thinking of as app erosion. API changes and deprecations impact us and create reliability issues. Again, it never stops.”
- The Constant Grind: Zapier's vast integration network means continuous maintenance. API changes create a never-ending backlog of support tickets, impacting customer experience and driving churn.
- Support's Superpowers: Support teams are uniquely positioned to fix these issues. They possess real-time customer context, fresh logs, and direct insight into user pain, making them ideal first responders. This is like having the frontline medics also be the surgeons.
- Talent Pipeline: Many support members aspire to engineering roles, creating a natural alignment for upskilling and direct contribution.
Orchestrating AI for Workflow Integration
- “One big discovery we had is how much time is spent gathering the context, going to the third-party API docs, even crawling the internet looking for information about a bug that's emerging... This is something we knew we needed to solve for.”
- Context is King: A major bottleneck in bug fixing is the time spent gathering context from disparate sources—API docs, internal logs, and external forums. This is like a detective spending 80% of their time just finding the case files.
- Embedding, Not Just Building: Early AI tools, while powerful (like an LLM-powered diagnosis API), saw low adoption when offered as standalone "playgrounds." Success came when the diagnosis tool was embedded directly into the support team's Jira workflow via a Zapier integration.
- Scout Agent's Symphony: Zapier's "Scout Agent" orchestrates these individual tools into an automated pipeline. It categorizes issues, assesses fixability, generates merge requests, and even allows support to request adjustments via chat, kicking off rapid iterations. This is a conductor bringing individual instruments into a cohesive performance.
Impact: Doubled Velocity & Focused Engineering
- “40% of support team's app fixes are being generated by Scout. So we're doing more of the work on behalf of the support team. This is resulting in, for some of our support team, it's doubling their velocity from one to two tickets per week... to now shipping three to four with the help of Scout.”
- Efficiency Multiplier: Scout Agent now generates 40% of app fixes, doubling individual support member velocity. This frees up engineering to focus on complex, strategic development rather than reactive bug squashing.
- Proactive Triage: The system identifies "potentially fixable" tickets early, streamlining the triage process and reducing friction.
- Career Advancement: The program provides a clear pathway for support members to transition into engineering, fostering internal talent.
Key Takeaways:
- Workflow Automation is the New Frontier: The real value of AI in developer tools comes from orchestrating entire workflows, not just individual point solutions.
- Embed for Adoption: Tools must integrate seamlessly into existing workflows and IDEs (like Cursor) to achieve high usage.
- Support as a Code-Shipping Powerhouse: Empowering non-traditional roles with AI-driven code generation leverages their unique, real-time context, creating significant operational leverage.
Podcast Link: https://www.youtube.com/watch?v=RmJ4rTLV_x4
Zapier transforms its support function into a code-shipping powerhouse, leveraging AI to automate bug fixes and redefine developer workflows.
The App Erosion Crisis
- Zapier, with over 8,000 integrations built on third-party APIs, faces constant reliability challenges. API changes and deprecations, termed "app erosion" by Lisa Orr, create a relentless stream of bugs. This leads to a backlog crisis, where support tickets accumulate faster than teams can resolve them, impacting customer experience and increasing churn.
- Zapier's extensive integration ecosystem (8,000+ apps) suffers continuous degradation from external API updates.
- "App erosion" describes the ongoing impact of third-party API changes, causing reliability issues.
- A growing backlog of support tickets directly correlates with poor customer experience and churn.
- Lisa Orr states, "Tickets were coming in faster than we could handle them. [This] creates integration reliability issues, poor customer experience, even churn."
Empowering Support to Ship Code
- Zapier initiated two parallel experiments to combat app erosion. The first focused on shifting the support team's role from mere triage to active bug fixing. This required securing internal buy-in by demonstrating the critical need and the support team's unofficial existing contributions to app maintenance.
- Support teams possess a strong desire to transition into engineering roles, providing a natural talent pool.
- Unofficial contributions from support members already maintained Zapier applications.
- Initial guardrails included focusing on four target applications and requiring engineering review for all support-generated merge requests.
- Lisa Orr notes, "Support is eager for this experience... unofficially many support members were already helping to maintain our apps."
AI for Context and Diagnosis: The Scout Project
- The second experiment, "Scout," explored using code generation (codegen) to accelerate app erosion solutions. Initial discovery revealed significant time spent by engineers and support gathering context from external API documentation and internal logs. Zapier developed APIs, some utilizing Large Language Models (LLMs) for diagnosis, to automate this context aggregation.
- Engineers and support staff spend substantial time manually collecting bug context from diverse sources.
- Zapier built internal APIs, including an LLM-powered diagnosis tool, to curate relevant information for bug resolution.
- Other tools, like a unit test generator, also contributed to the initial suite.
- Lisa Orr highlights, "One big discovery we had is how much time is spent gathering the context going to the third-party API docs even crawling the internet looking for information about a bug."
Embedding Tools and Orchestrating Agents
- Early challenges arose from low adoption of Zapier's new APIs, as they were not integrated into existing developer workflows. The key to usage proved to be embedding these tools directly. The launch of internal platforms like MCP (a Zapier internal platform for embedding tools) and the adoption of Cursor (an AI-native code editor) allowed Zapier to integrate its AI-powered APIs directly into engineers' Integrated Development Environments (IDEs). This led to the development of "Scout Agent," an orchestrated system combining diagnosis, codegen, and merge request generation.
- Initial API tools saw limited adoption due to a lack of integration into developer workflows.
- Embedding tools directly into engineers' IDEs via platforms like MCP and Cursor proved critical for usage.
- Scout Agent orchestrates multiple AI tools, including diagnosis and codegen, to automate the bug-fixing process.
- Lisa Orr explains, "Embedding tools is the key to usage as we find out."
Scout Agent's Impact and Support Superpowers
- Scout Agent now processes support issues, categorizes them, assesses fixability, and generates merge requests (MRs). Support teams review and test these AI-generated MRs, requesting adjustments via chat within GitLab. This process significantly boosts support velocity and frees engineering to focus on complex issues. Support teams possess three "superpowers" making them ideal for this role: proximity to customer pain, real-time troubleshooting context, and superior validation capabilities.
- Scout Agent automates issue categorization, fixability assessment, and initial merge request generation.
- Support teams review and refine AI-generated MRs, with a rapid iteration loop via chat.
- Scout Agent generates 40% of support team app fixes, doubling individual support velocity from 1-2 to 3-4 tickets per week.
- Lisa Orr asserts, "There is a really powerful magic between support and empowering them with codegen... they have three superpowers."
Investor & Researcher Alpha
- AI-Augmented Internal Tooling: The success of Scout Agent demonstrates the critical value of deeply embedding AI-powered tools directly into existing developer and support workflows. Generic AI playgrounds yield low adoption; integrated solutions drive tangible productivity gains. Investors should seek companies prioritizing workflow-native AI integration over standalone AI features.
- Shifting Engineering Focus: Automating routine bug fixes via AI-augmented support teams allows core engineering resources to reallocate towards higher-complexity problems and innovation. This represents a significant efficiency gain for R&D budgets.
- The "Support Engineer" Archetype: The emergence of AI-powered support teams shipping code creates a new, highly efficient role. This model could redefine talent acquisition and training strategies for SaaS companies facing similar "app erosion" challenges.
Strategic Conclusion
- Zapier's Scout Agent proves AI can transform support into a proactive code-shipping function, directly addressing integration reliability. The next step for the industry involves widespread adoption of AI-orchestrated agents to automate routine development tasks, fundamentally altering the division of labor between support and engineering.
