Who's Coding Now? - AI and the Future of Software Development

This episode delves into how AI is rapidly transforming software development, examining the burgeoning AI coding market, the evolving role of developers, and the potential for AI to become a new abstraction layer in programming.

The Dawn of AI in Compiler Design and Coding Market Dominance

• The discussion opens with the provocative idea that AI, specifically LLMs (Large Language Models) – AI models trained on vast text datasets to understand and generate human-like language – could fundamentally alter traditional software tools.
• Guido suggests that if LLMs were available during the development of classic compiler design, the approach to building compilers might have been vastly different. He posits, "If I can basically define certain things in human language in efficient way... that could change a lot of things."
• The conversation then shifts to the AI coding market's current size. It's proposed that AI coding is likely the second biggest AI market after general consumer chatbots, with an argument made that it could even be the largest homogenous AI market, surpassing AI companions.
  • This is attributed to coding being an existing behavior (developers seeking help on platforms like Stack Overflow, a popular Q&A website for programmers) now being enhanced by AI.
  • The foundational work of GitHub Copilot, an AI pair programmer, is acknowledged for transitioning users from Stack Overflow to AI-driven assistance.

Why AI Coding is Thriving: Existing Behaviors and Verifiability

• The panel discusses unique factors contributing to AI coding's success.
  • It leverages an existing user behavior: developers were already seeking external help (e.g., Stack Overflow), and AI offers a more efficient solution.
  • Developers are natural early adopters, keen to solve their own problems and boost productivity. Yoko notes, "developers are always early adopters for new technologies... and they're lazy so anything that actually... increase the productivity they will adopt."
  • Yoko also emphasizes that coding is a "somewhat verifiable problem," where inputs and outputs are clear, making it easier for AI to generate and validate solutions compared to more subjective tasks. She even suggests that some art generation can be reframed as a coding problem.
• Strategic Implication for Investors: The verifiability of coding tasks makes AI solutions in this space more robust and easier to evaluate. Investors should prioritize AI coding tools that demonstrate clear, measurable improvements in verifiable outcomes.

The Trillion-Dollar Impact of AI on Developer Productivity

• The sheer market size and potential value creation are highlighted. With approximately 30 million developers worldwide, each creating an average of $100,000 in value annually, the total market impact is $3 trillion.
• Guido points out that even a conservative 15% productivity increase from tools like GitHub Copilot, as estimated by large financial institutions, is significant. He believes this can be pushed much higher, potentially doubling developer productivity and unlocking an additional $3 trillion in value.
  • This dwarfs previous concerns about overinvestment in AI, making the $200 billion annual AI investment seem small in comparison.
• The "bootstrapping effect" is also mentioned: better AI coding models lead to the creation of better software and new AI applications, creating a virtuous cycle.
• Actionable Insight for Researchers: Research should focus on quantifying and maximizing AI-driven developer productivity, exploring models and integrations that can achieve gains significantly beyond the current 15-20% benchmarks.

The Evolving Role of the Software Developer

• The conversation explores how AI will change the day-to-day job of a software developer.
• Guido describes his current workflow: writing specifications, discussing implementation with AI models, and asking AI to implement easy features for review.
• The question arises: Will developers become product managers writing specs, with AI handling the coding, or will they shift to QA roles? The end state remains unclear.
• The Host humorously notes the irony: "We all got into this to avoid being QA engineers."
• Strategic Consideration: The shift in developer roles suggests a need for new training paradigms and tools that support specification, review, and AI interaction, rather than just code generation.

Personal Experiences: How AI is Changing Coding Workflows

• Guido shares his evolving use of AI in coding over the past 6-9 months:
  • From using standalone chatbots like ChatGPT as a Stack Overflow replacement (copy-pasting code).
  • To integrated IDEs (Integrated Development Environments) – software applications that provide comprehensive facilities to computer programmers for software development – like GitHub Copilot and Cursor (an AI-first code editor), enabling autocomplete and in-flow assistance.
  • To IDEs using command-line tools for project setup (e.g., setting up a Python project with UV, a fast Python package installer and resolver).
  • Currently, he starts new projects by writing a high-level spec, then iteratively refines it with an AI model (like Claude 3.5/3.7 or Gemini) acting as a "sparring partner." This involves providing context like coding guidelines and development methodology.
• Yoko describes her workflow changes:
  • Giving AI coding agents more "world knowledge" beyond their foundational model's cutoff date.
  • Integrating tools like Linear (a project management tool) with Cursor, where an AI agent takes a first pass at implementing a ticket.
  • Using tools like Firecrawl (a tool to crawl websites and convert them into LLM-ready data) to fetch up-to-date documentation for the AI agent. She mentions, "it will actually fetch a page and it will you know read up that works."
• The Host shares a more spontaneous approach, using AI for high-complexity, high-annoyance tasks like front-end development, especially remembering numerous CSS (Cascading Style Sheets) classes for layout.

Challenges and Limitations: Hallucinations and Agent Behavior

• The discussion touches upon issues encountered with AI-assisted coding.
• Yoko recounts an instance where a Cursor agent, when asked to implement code based on output from another tool, decided the tool's output was good but then generated its own "new version" instead of adopting it, highlighting interesting "agent to agent communication" quirks.
• Yoko explains MCP (MetaCall Protocol), a protocol for creating polyglot runtimes, as a way to provide relevant context to LLMs, empowering experiences like fetching documentation or integrating with tools like Linear and GitHub. The core is providing the "most relevant thing" to the model.
• The panel discusses AI's effectiveness for senior developers or "neckbeards" (highly experienced, often skeptical engineers). Yoko believes it depends on what senior engineers optimize for; AI is not yet adept at highly specialized, state-dependent problems like distributed systems due to context limitations and the current cap on tool handling in IDEs (around 40-50 tools).
• Guido notes that the more esoteric or novel the problem, the more context the AI needs. For common problems with ample training data, AI is excellent. For novel problems, it struggles and can "very confidently give you a wrong answer too," including hallucinating non-existent functions. He states, "I think what models today are very bad at is telling you if they don't know something."
• Crypto AI Relevance: The challenge of AI hallucination is critical for Crypto AI, where code correctness and security are paramount. Research into verifiable AI outputs and robust context provision is essential before AI can be trusted for critical smart contract development.

"Vibe Coding": Democratizing Software Development?

• The concept of "vibe coding" emerges – people who aren't traditional developers using AI to write code. This is seen as a positive trend, democratizing access to computing power.
• The Host expresses optimism that this new pool of software creators, approaching problems differently, will lead to novel applications and uses of computing.
• Yoko likens the vibe coding wave to the early 2000s blogging boom with tools like WordPress, enabling personal software creation (e.g., personal CRMs). She questions the depth of such software but acknowledges its personal utility.
• Yoko references a tweet by Martin (likely Martin Casado) about needing to understand the abstraction layer below where one operates. This raises the question: what is the lower-level abstraction for vibe coders?
• Strategic Implication: The rise of "vibe coders" could expand the market for AI development tools but also necessitates tools that bridge the gap between generated code and the user's ability to understand and modify it.

The Future of Computer Science Education

• The panel speculates on how CS education will adapt. Guido admits, "Honestly, I have absolutely no idea how the equivalent of computer science education will look like in 5 years."
• He draws an analogy to how calculators changed bookkeeping: manual calculation became less important, while higher-level abstract concepts (accounting) grew in significance.
• This suggests that problem statement explanation, algorithmic foundations, architecture, and data flow might become more crucial than nitty-gritty coding details.
• The Host reflects on traditional CS education, which often starts with fundamentals like assembly language and processor architecture, questioning if AI is just another layer on top or something fundamentally different.

Is AI a New Programming Interface or Just a Tool?

• The Host wonders if we're waiting for the next iteration where AI truly changes how computers are programmed, perhaps through prompts directly translated to code, with current agents being a starting point. "Today I think AI is not just a higher level language abstraction... but could it become one over time?"
• Guido believes it could, suggesting LLMs could revolutionize compiler design if human language inputs could be made efficient and tight enough for direct compiler use.
• Yoko draws an analogy to operating systems: agent-based systems built with AI often mirror OS concepts like processes and resource management. She argues CS education remains vital to provide these comparative frameworks.
• The Host emphasizes the enduring need for formal languages for their high-bandwidth, expressive power in software design, doubting that languages like Python will disappear entirely.
• Actionable Insight for Researchers: Explore the potential of LLMs to create new, more intuitive programming paradigms. This includes research into formal prompting languages and AI-native development environments that could bridge natural language intent with executable code.

The Role of Abstraction and Optimization

• Yoko reiterates the importance of understanding lower abstraction levels for optimization. If you're building a simple marketing website via vibe coding, deep knowledge isn't needed. But for scalable services, understanding underlying concepts like CDNs (Content Delivery Networks) or JVM (Java Virtual Machine) internals becomes essential.
• Guido agrees that formal languages won't vanish, as they are often the simplest representation of intent. The question is whether AI, with sufficient context, can accurately translate natural language descriptions for a subset of problems.
• The Host notes the irony: AI coding is easier to use but far more complex under the hood than traditional programming languages, which are hard to learn but fundamentally simple. He mentions Cursor's focus on formal specifications as a way to manage this, an "annealing process" between human and AI to reach a tight spec.

The "Vibe Coder" Experience and the Intent Gap

• Yoko shares an anecdote about a "vibe coder" who felt empowered seeing AI generate code but didn't know how to modify it, highlighting a gap between AI generation and user operability.
• She extends this to experienced programmers, noting that even they would find it difficult to edit AI-generated code after several iterations due to its opacity. She experienced this herself when using an MCP for Blender (3D modeling software), easily prompting a model but struggling to modify the complex 3D representation.
• Opportunity for Crypto AI: Tools that can translate complex AI-generated code (e.g., for smart contracts or zk-circuits) into more understandable or auditable forms would be highly valuable for the Crypto AI space.

AI, Legacy Code, and Capturing Intent

• The discussion turns to AI's potential in porting legacy code, like old COBOL (a historically significant programming language, especially in finance) or PL/1 (another early high-level language) systems.
• The Host initially expresses skepticism, arguing AI can transpile code (e.g., COBOL to Java) but can't recover lost context and original intent from decades-old systems.
• However, he notes a fascinating implication: if AI had been used to create those systems, a record of the original intent would exist. This "metadata" of software intent is a valuable byproduct of current AI-assisted development.
• Guido confirms this, sharing that large enterprises are finding it more efficient to first use AI to create a specification from legacy code, then use AI to reimplement that spec in a modern language. This yields better, more modern code.
• Yoko adds that rewriting modern software (e.g., Angular to React – both popular JavaScript frameworks for building user interfaces) is easier for AI, especially if both frameworks are well-understood by the agent. Migrating older systems like PHP (a server-side scripting language) or those with hardware-specific dependencies and lost runtime configurations remains challenging.

AI and Non-Deterministic Behavior in Software

• The Host poses a question to Guido: Does AI as a primitive in applications push the frontier of uncertainty and non-deterministic behavior in software, similar to how networking introduced unpredictability?
• Guido agrees, drawing parallels to how networking introduced new failure modes (timeouts) and remedies (retries), and how distributed databases brought complexities like atomicity. He suggests that for AI, "an infinitesimally small change in the input can have an arbitrary large effect," describing it as a chaotic system.
• He shares an example of a bank trying to build an LLM-powered system that must never give investment advice—an almost unsolvable problem. The bank shifted its metrics to an acceptable error rate compared to a well-trained human.
• Strategic Consideration for Crypto AI: The chaotic nature of AI outputs poses significant risks for decentralized applications requiring deterministic and verifiable behavior. Investors should be cautious about AI systems where outputs cannot be rigorously validated, especially in financial or governance applications.

The "Narrow Waist" of AI: Is it the Prompt?

• Yoko asks Guido if AI will see a "narrow waist" phenomenon similar to the internet (where IP served as a unifying layer). Guido suggests the prompt might be AI's narrow waist.
• He explains that major tech cycles are often built on abstractions that encapsulate complexity via a narrow API (e.g., SQL for databases). For modern ML, the prompt allows even mediocre programmers to leverage powerful LLMs without needing deep ML expertise.
• The Host questions if prompts, currently varying by model and not being a formal language, truly fit this role. Guido acknowledges this, noting we're learning a new "language" for prompting, with dialects for different models. He expresses hope for a formal prompting language.
• Yoko suggests agent frameworks might be a step towards formal prompting languages. Guido agrees, pointing to emerging structured prompts (e.g., user/agent tags, JSON mode output where models generate structured JSON (JavaScript Object Notation) data based on a defined schema).
• Guido speculates that future models might separate the reasoning layer from the output generation layer, allowing different "personalities" or output formats (chatty, terse, JSON) from the same core reasoning engine.

Vibe Coding vs. Enterprise Coding Models

• The episode concludes with Yoko questioning if there will be different AI models for "vibe coding" (less formal, outcome-focused) versus enterprise coding (more constrained, specific SDKs).
• She defines vibe coding as caring about the outcome but not the implementation details, letting the model choose.
• Guido implies he doesn't see a fundamental difference, suggesting enterprise users could also engage in "vibe coding" if the focus is on achieving a higher-level need without dictating every technical detail.

Conclusion

AI is fundamentally reshaping software development, moving beyond a mere tool to potentially a new abstraction layer. For Crypto AI investors and researchers, the key is to track how AI enhances verifiable code generation, manages non-determinism, and captures developer intent, as these will dictate AI's true value in building secure, decentralized systems.