Svetlana Jitomirskaya, a mathematical physicist at UC Berkeley, shares her journey from AI skeptic to impressed user, exploring whether machines can master the unwritten, "folklore" knowledge that defines human expertise in mathematics.
The Folklore Knowledge Barrier
- "In order to solve my problem, a machine would really need to argue in a way that is not even written in any paper, although maybe understood by some people in the field."
- "It is not a very difficult problem for someone who has certain knowledge, but this knowledge is not widespread. This knowledge is not in the literature. This is kind of a little bit of a folklore knowledge."
Jitomirskaya submitted a math problem designed to test AI's limits. While a knowledgeable grad student could solve it, current AI models are stumped because the solution relies on implicit, unwritten expertise—the "folklore" of the field. This highlights a key bottleneck for AI, which excels at processing documented information but struggles with the intuitive understanding that experts develop over time. She predicts it will take AI 10-20 years to solve her problem.
From Skeptic to Believer
- "I always thought that they were just regurgitating, but they developed logic right now. I was kind of proven wrong."
Jitomirskaya’s perspective on AI has shifted dramatically. Initially believing large language models were mere parrots, she now acknowledges their capacity for genuine logic. A recent experience where an AI model instantaneously handled a lengthy computational argument she was "lazy to do" left her "so impressed" and committed to using it more in her research.
The Future of Math: More Creativity, Less Routine
- "The work of mathematicians will become more creative; there will be less time spent on routine because this would be delegated to the machines."
AI is poised to automate the tedious, routine parts of mathematical work. This shift will free mathematicians to focus on what humans currently do best: abstract thinking and creativity. Jitomirskaya defines this creativity as the ability to "try something that nobody has tried." While this may lead to a more exciting field, it could also mean fewer mathematicians are needed.
Lean and the Verification Revolution
- "My biggest hope is that all mathematics will be put in the verified Lean database. When you submit a paper to a journal, you need to prove that it was Lean-verified."
Jitomirskaya envisions a future where tools like Lean create a database of formally verified mathematical proofs. An AI-powered automatic translator, which she believes is only a "couple of years away," would bridge the gap between human-written papers and machine-readable code. This would revolutionize peer review, with machines handling correctness and human referees focusing on a paper's significance and novelty.
Key Takeaways:
- AI's Next Frontier is Unwritten Knowledge. AI has mastered logic, but its true test is acquiring the implicit, "folklore" knowledge that experts use intuitively but rarely write down.
- Human Mathematicians Will Become Purely Creatives. As AI automates routine calculations and arguments, the role of a mathematician will shift entirely to high-level abstraction, creative problem formulation, and intuition.
- Formal Verification Will Revolutionize Publishing. The combination of AI translators and formal proof assistants like Lean will soon make it possible to automatically verify all mathematical papers, fundamentally changing how research is validated and published.
For further insights and detailed discussions, watch the full podcast: Link
This episode reveals the critical gap between AI's emerging logical prowess and its inability to grasp the unwritten, intuitive "folklore knowledge" that defines true expertise.
An Expert’s Perspective on AI in Mathematics
Svetlana Jitomirskaya, a mathematical physicist from UC Berkeley specializing in spectral theory, frames the conversation with a blend of fascination and professional concern. She is exploring AI's capabilities not just out of curiosity, but to understand its potential to reshape her field and the very nature of mathematical work. Her perspective is that of a seasoned expert evaluating a powerful new tool, grounding the discussion in practical, high-level applications.
The "Folklore Knowledge" Gauntlet: A Test for True AI Reasoning
- The Challenge: The solution relies on what Svetlana calls "folklore knowledge"—insights and heuristics understood by experts in the field but never explicitly published.
- Human vs. Machine: A knowledgeable graduate student could solve it with relative ease, but initial tests show the AI has "no clue."
- Strategic Implication: This highlights a critical vulnerability and opportunity in AI development. Models that can successfully mine and apply this uncodified, domain-specific knowledge will unlock significant value. For investors, this points to a new frontier beyond training on public data sets.
"In order to solve my problem, machine would really need to argue in a way that is not even written in any paper although maybe understood by some people in the field."
From Regurgitation to Reasoning: AI's Unexpected Leap
- Evolving Capabilities: She was recently amazed when an AI model assisted her in creating a problem, executing lengthy computations and arguments "momentarily."
- The Creativity Barrier: Despite this, she remains unconvinced of AI's ability to achieve true creativity—the act of trying something entirely novel. She notes that humans can form abstract patterns from just a few examples, a skill she doesn't see a clear path for AI to replicate.
"I always thought that they were just regurgitating, but they developed logic right now."
The Future of Verifiable Mathematics with Lean
- The Bottleneck: Currently, translating a written argument into Lean is extremely time-consuming, taking roughly seven times longer than writing the original paper.
- The AI-Powered Solution: An automated translator would be a revolutionary tool, creating a fully verifiable database of all mathematics. This would eliminate errors in papers and shift the role of human referees from verifying correctness to judging a work's significance.
- Relevance for Crypto AI: This vision directly parallels the goals of verifiable computation and zkML (Zero-Knowledge Machine Learning). The development of automated formal verification tools is a foundational technology for building trustless, decentralized AI systems and secure smart contracts.
AI-Generated Proofs and the Nature of Knowledge
- A New Class of Proof: Svetlana categorizes this potential outcome alongside existing computer-assisted proofs, which are mathematical proofs that require extensive computer calculations to be verified. While accepted by the community, they are considered a different category from elegant, intuitive "proofs from the book."
- Value vs. Understanding: Such a proof would be mathematically valid and immensely valuable, but it would lack the explanatory power and beauty that humans seek in mathematics. This suggests a future where mathematical discovery could be bifurcated into human-comprehensible and machine-generated domains.
Final Outlook: A Tool for Augmentation, Not Replacement
- Advice for Students: Learn the fundamentals the "old-fashioned way" but embrace AI as a powerful assistant for experimentation and routine tasks.
- The Future of Mathematicians: If AI masters even the creative aspects of math, the field may evolve into a new meta-level of formulating models for the AI to solve, perpetually adding new layers of abstraction.
- Timeline Prediction: She estimates her "folklore knowledge" problem will withstand AI for "more than 10 years, maybe 20," providing a tangible timeline for researchers and investors tracking the progress of advanced AI reasoning.
Conclusion
This discussion highlights that AI's next major hurdle is not just logic, but the acquisition of unwritten, intuitive expert knowledge. For investors and researchers, the key takeaway is to focus on systems that can either bridge this "folklore" gap or create powerful tools that augment the irreplaceable creative and abstract reasoning of human experts.
