Svetlana Jitomirskaya, a mathematical physicist at UC Berkeley, sits down to discuss the frontier of AI in mathematics. She explores how AI has mastered logic but still stumbles on the unwritten, intuitive "folklore knowledge" that defines human expertise.
The Folklore Knowledge Barrier
- "In order to solve my problem, a machine would really need to argue in a way that is not even written in any paper, although maybe understood by some people in the field."
- "It is not a very difficult problem for someone who has certain knowledge, but this knowledge is not widespread. This knowledge is not in the literature. This is kind of a little bit of a folklore knowledge."
Jitomirskaya submitted a problem to an AI competition designed to probe this exact weakness. The problem is simple for a knowledgeable grad student but nearly impossible for an AI that relies solely on published texts. This "folklore"—the unwritten intuition, context, and shared understanding within a field—is currently AI's Achilles' heel. Early tests confirm this; the model had "no clue" and attempted to bluff its way to a solution.
The New Job of a Mathematician
- "The work of mathematicians will become kind of more creative; there will be less time spent on routine because this would be able to be delegated to the machines."
The rise of AI won't make mathematicians obsolete; it will upgrade their jobs. By automating routine computations and arguments, AI will free up human minds to focus on higher-level creativity and problem formulation. Jitomirskaya envisions a future where formal verification systems like Lean are standard. An AI-powered translator would convert papers into a verifiable format, shifting the role of peer reviewers from checking for correctness to judging a paper’s novelty and importance.
Logic is Learned, Creativity is Human
- "I always thought that they were just regurgitating, but they developed logic right now... But how do you train them to be creative? I don't know."
While initially skeptical, Jitomirskaya now concedes that LLMs have developed genuine logical reasoning abilities, not just pattern matching. However, she draws a sharp line between logic and true creativity—the ability to "try something that nobody has tried." Human intuition, which can build abstract models from just a few examples, remains a uniquely powerful and mysterious capability that we don't yet know how to replicate in a machine.
Key Takeaways:
- The core tension is that AI is mastering the logic of math but not the culture. Its inability to grasp "folklore knowledge" is a key barrier, pushing human mathematicians toward a future defined by creativity, not computation.
- AI's Blind Spot is "Folklore Knowledge." AI excels at digesting published literature but fails on problems requiring unwritten, community-held intuition, which remains a key human advantage for now. Jitomirskaya predicts her problem will take AI 10-20 years to solve.
- Mathematicians Won't Be Replaced, They'll Be Upgraded. The future role of a mathematician is less about routine work and more about creative problem formulation. AI tools like Lean will handle verification, shifting peer review from "Is it correct?" to "Is it interesting?"
- Math May Become a Sport. If AI eventually masters creativity, the human practice of mathematics may persist like chess—an activity pursued for its intrinsic value and intellectual challenge, even if a machine is the undisputed world champion.
For further insights, watch the video here: Link
This episode reveals a critical frontier in AI development: the gap between its powerful logic and its inability to grasp the unwritten, intuitive "folklore knowledge" that drives expert human reasoning.
Introduction to a Mathematical Physicist's Perspective
- Svetlana Jitomirskaya, a mathematical physicist from UC Berkeley specializing in spectral theory—the study of eigenvalues and operators, often used to describe quantum systems—shares her perspective on AI's role in mathematics.
- Driven by a mix of fascination and professional concern, she attended the event to understand AI's latest capabilities and its potential to reshape her field. Jitomirskaya expresses a common sentiment among experts: “I'm a little worried whether it will put us out of job, out of work.”
The "Folklore Knowledge" Challenge
- Jitomirskaya submitted a problem designed to test AI's limits, rooted in the interplay of number theory and spectral theory.
- The problem is not exceptionally difficult for a human expert; she notes it could be solved by a knowledgeable graduate student. Its true challenge lies in the fact that its solution requires "folklore knowledge"—niche, unwritten understanding shared among specialists but not documented in formal literature.
- This setup creates a perfect test case: can AI reason beyond its training data and access the implicit, contextual knowledge that defines deep expertise? Jitomirskaya would be "very surprised" if the current AI solves it, as initial tests on simpler versions showed it "had no clue."
A Shift in Perspective: Witnessing AI's Power
- Despite her skepticism about AI's creative limits, Jitomirskaya recounts a recent, "amazing" experience where an AI tool assisted her in creating a problem.
- The AI executed lengthy computations and formulated complex arguments "momentarily," tasks she had been avoiding due to their tedious nature.
- This firsthand demonstration of AI's practical power convinced her of its utility, leading her to declare, "Yes. Yes. Yes. I'll be using it a lot." This highlights a key adoption driver: AI's ability to dramatically accelerate the non-creative, laborious aspects of research.
The Future of Mathematics: More Creativity, Fewer Mathematicians
- Jitomirskaya predicts AI will fundamentally change the nature of mathematical work. It will automate routine tasks and arguments based on analogy, filtering out less innovative research.
- This automation will free up human mathematicians to focus on higher-level, creative thinking. "The work of mathematicians will become kind of more creative, less time spent on routine because this would be able to delegate it to the machines," she explains.
- Strategic Implication: This shift suggests a future where the primary value of human researchers is in creative problem formulation and abstract thinking, while AI handles execution. For investors, this points to opportunities in tools that augment, rather than replace, expert creativity.
The Unsolved Problem of AI Creativity
- Jitomirskaya expresses awe that Large Language Models (LLMs) have developed logical capabilities, which she once doubted. She theorizes this emerged from being trained to summarize and extrapolate key ideas.
- However, she sees no clear path for training true creativity—the ability to devise a novel approach that "nobody has tried." Human intuition, she observes, can build abstract patterns from just a few examples, a feat current AI cannot replicate.
- This remains the core difference: AI excels at interpolation and logical deduction within known frameworks, while human creativity involves abstract leaps into the unknown.
A New Era of Verifiable Truth in Mathematics
- Jitomirskaya’s biggest hope is for the widespread adoption of Lean, a formal proof assistant and programming language that allows mathematical proofs to be written in a machine-verifiable format.
- She envisions a future where all mathematical papers must be "Lean verified" before publication, creating a fully trustworthy and error-free database of mathematical knowledge. This would shift the role of peer reviewers from verifying correctness to judging a paper's interestingness and significance.
- Crypto AI Relevance: This concept of a decentralized, verifiable, and immutable ledger of knowledge directly mirrors the core principles of blockchain technology. The development of an automatic translator from written text to Lean, which she suggests is only a "couple of years away," would be a monumental step toward this vision.
The Value of Inaccessible Proofs
- When asked about a hypothetical AI that proves a major theorem like the Riemann Hypothesis—a famous unsolved problem in mathematics concerning the distribution of prime numbers—with a proof too complex for any human to understand, Jitomirskaya offers a pragmatic view.
- She places it in the same category as existing computer-assisted proofs, which are accepted by the mathematical community despite being unverifiable by a single human without a machine.
- While such a proof would be considered valid math, it would lack the elegance and insight of a "proof from the book"—a term for proofs that are beautiful and illuminating. This highlights a potential split between "correct" and "insightful" mathematics in an AI-driven world.
Guidance for Future Mathematicians
- Jitomirskaya advises students to learn fundamentals the "old-fashioned way" to build a strong conceptual foundation.
- However, she strongly encourages using AI as a powerful assistant for experimentation, much like computers have been used for decades. The key is to "make it your friend and helper, but not an enemy of your development."
Final Outlook: Optimism with Caveats
- Jitomirskaya believes her "folklore knowledge" problem will remain unsolved by AI for more than 10 years.
- She rates AI's future impact on math research a 5/10 but its impact on the world a 10/10, comparing it to the Industrial Revolution.
- Even if an AI could perform all creative mathematical tasks, she speculates that humans would continue doing math, much like people still play chess or run for sport. The discipline might evolve to focus on creating new models and frameworks for the AI to explore, constantly raising the bar of abstraction.
- Her overall attitude toward accelerating AI progress is an 8/10, reflecting a strong belief in progress, tempered by the need for safeguards.
Conclusion
This discussion highlights the critical challenge of imbuing AI with implicit, "folklore" knowledge. For investors and researchers, the key takeaway is the immense opportunity in developing systems for automated proof verification like Lean and in pioneering new methods to train AI for true, abstract creativity beyond its current logical capabilities.
