Subnet 50 :: SYNTH :: Bitcoin Forecasting on Mathy AI Steroids

This episode unveils how Subnet 50 (Synth) leverages BitTensor's decentralized intelligence to tackle Bitcoin price forecasting, offering a sophisticated probabilistic approach that challenges traditional prediction models and creates new opportunities for Crypto AI investors.

Episode Introduction and Team Overview

• Const, the host, introduces the episode by recounting his initial skepticism about a Bitcoin price prediction subnet, a notoriously difficult problem. He acknowledges that Synth, launched three months prior, has demonstrated a non-naive approach, backed by a team with a strong background in statistical finance.
• James from the Synth team introduces the core members:
  • James: Founder of Mode (DeFi and AI intersection), with 8 years in crypto finance and AI startup investment experience.
  • Sam: Focused on quant modeling, a physicist with experience in sports betting markets and algorithmic trading.
  • Nathan: Lead architect, software engineer with experience in crypto wallets, bridges, and dApps.
  • Stas: Expert in large distributed systems.
  • David: Recently hired for quantum finance research.
  • Amber: Leads strategy, ecosystem development, and BD.
• The team has also partnered with GTV, known for their proximity to the Roundt 21 team, for guidance on the BitTensor ecosystem. BitTensor is a decentralized network that incentivizes the creation and operation of specialized AI models, or "subnets."

The Vision for Synth: Predictive Intelligence Engine

• James outlines Synth's grand vision: to create a "predictive intelligence engine for finance and beyond."
• The initial focus is on Bitcoin forecasting, using synthetic data generation to demonstrate the model's power. This is presented as the first stage of a larger plan.
  • James: "This is very much the first kind of stage and we're super excited to tell you more about get the results and yeah, where we're going with it."

The Forecasting Data Challenge on BitTensor

• James addresses the inherent difficulty and skepticism surrounding prediction subnets. A common critique is that if one could predict the future, they wouldn't share that ability.
• Synth's response is that they aim to build a better model of the future, not a perfect one, superior to what retail traders or even some professional teams might develop.
• A key focus is providing probabilistic models for AI agents, which will increasingly need to model future scenarios for trading and risk management.
  • Point predictions, which offer a single future value, are deemed "fairly useless" for AIs needing a full understanding of probability distributions.
• Synth differentiates itself by the volume of data: approximately 165 million data points submitted daily, compared to roughly 10,000 for typical point prediction subnets. This is because Synth focuses on predicting entire price paths, not single future prices.

Synth Architecture: Generating Monte Carlo Paths

• Sam explains that Synth is a competition for quant research teams and data scientists to build generative models of time series data, starting with Monte Carlo paths for Bitcoin prices. Monte Carlo paths are multiple simulated future price trajectories generated using a probabilistic model.
• The goal is for these paths to encompass nuances of real Bitcoin price data: volatility dynamics, volatility clustering, heavy tails (representing rare, high-impact events or "black swan" events), and potential skew in distributions.
• Phase One (live since January): Miners predict 100 paths of the Bitcoin price 24 hours into the future, with 5-minute intervals (each path containing 289 prices).
• Process:
  • Validators request predictions from all miners roughly every hour.
  • Miners have two minutes to run their pre-built models and return 100 simulated paths.
  • Validators store these paths.
  • After 24 hours, validators compare the predictions against the actual Bitcoin price path (using an oracle like Pyth, a decentralized oracle network providing real-world data to blockchains).
  • A scoring mechanism determines the best-performing models. This process is repeated hourly.

The Scoring Mechanism: Continuous Ranked Probability Score (CRPS)

• Sam details the scoring for each 5-minute interval. The predicted price changes are converted into a histogram.
• The core of the scoring is a proper scoring rule, specifically the Continuous Ranked Probability Score (CRPS). A proper scoring rule is a mathematical function that incentivizes forecasters to report their true belief about the probability of an outcome.
  • Sam: "As soon as you deviate from that kind of mathematical definition of proper scoring rules, it means there's abilities to game the system."
• CRPS Explained:
  • The goal is to minimize CRPS; a perfect score (zero) means predicting a completely narrow distribution exactly on the real price.
  • Predicting a very narrow distribution and being wrong is heavily penalized (first term of the CRPS formula).
  • Predicting too wide a distribution is also penalized (second term, related to pairwise differences in the miner's own predictions).
  • The optimal strategy is to predict a distribution that matches the miner's true belief about the underlying uncertainty of the future Bitcoin price.
• Synth is not about predicting price directionality, which Sam views as "a very difficult if not kind of foolish direction to go in." Instead, it focuses on modeling volatility and distributions.
• Scores are calculated for 5-minute, 30-minute, 3-hour, and full 24-hour intervals to test long-term price dynamics and volatility clustering.

From Raw Scores to Reward Weights

• Sam explains the process of refining raw CRPS sums:
  1. Difference from Median: Scores are compared against a median score to highlight relative performance.
  2. Smoothing: An exponential weighted moving average (half-life of ~3.5 days, max 7-day window) is applied to look for consistent performance, reducing noise from individual prompts. This involves averaging over roughly a billion data points per week.
  3. Comparative Normalization: A softmax function with a controllable beta parameter is used to determine reward weights, spacing out miners based on their smoothed scores.
• Sam notes the current reward curve (Pareto distribution) can and will be made steeper to more heavily reward top miners and reduce "free money" to underperformers, in tandem with ensuring the smoothing period is adequate.

Validator Performance and Identifying Good Miners

• Sam, referencing TAOStats (a platform for BitTensor network analytics), shows that validators applying the correct scoring mechanism (e.g., RoundTable21, Synth's own validator, Rizzo) achieve similar scores for miners, indicating robustness. This leads to good vTrust (validator trust scores).
• The key test for the subnet's success is whether the same miners consistently perform well over independent time periods.
• A month-long performance graph shows that good miners with good models do emerge and maintain high rankings, even with a 7-day smoothing window ensuring weekly independence.
  • Const clarifies his understanding: miners are scored on how close their predicted distribution is to the actual price, penalized for both inaccuracy and excessive breadth of the distribution. The optimal strategy is to match the true underlying distribution.

Miner Strategies: From Naive to Sophisticated

• Const asks about the evolution of miner strategies.
• Sam reveals insights into what top miners are doing:
  • Base Miner (Geometric Brownian Motion): Assumes price changes are normally distributed with fixed sigma (volatility) and mean. This was the default and quickly superseded. Geometric Brownian Motion is a mathematical model where the logarithm of the randomly varying quantity follows a Brownian motion (random walk) with drift.
  • Key Improvement: Modeling Volatility (Sigma):
    • Realizing volatility varies (e.g., higher during US trading hours, lower at night/weekends).
    • Accounting for longer-term volatility trends (periods of low vs. high activity).
  • Beyond Normal Distributions: Fat Tails (High Kurtosis):
    • Incorporating fat tails (or long tails) into distributions to account for rare, large price movements (Black Swan events). Kurtosis is a statistical measure that describes the "tailedness" of a probability distribution. High kurtosis indicates heavier tails and a greater propensity for outliers.
    • Sam: "Even on short time scales, most of the time nothing much is happening... but then all of a sudden you'll get very large price changes."
  • The graph of kurtosis vs. leaderboard position shows top miners consistently incorporate fat tails.
• Strategic Implication for Miners: To be competitive, miners must move beyond simple models to incorporate dynamic volatility and fat-tailed distributions.

The Power of Monte Carlo Simulations: Use Cases

• Sam explains why Synth focuses on Monte Carlo simulations (paths) rather than just distributions or point predictions: versatility.
• With path data, users can derive probabilities for complex, path-dependent events.
  • Example: "What's the probability the Bitcoin price will be above $103,000 at this time tomorrow?" – simply count the paths.
• Use Cases & Live Metrics Pages (free access):
  1. Liquidation Probabilities: For leveraged traders (e.g., on perp dexes – decentralized exchanges for perpetual contracts), Synth can estimate the probability of liquidation within various timeframes (6, 12, 24 hours). This allows traders to adjust leverage based on risk.
  2. Options Pricing: Synth's path data can be used to price options. Their calculated prices are often "bang on within the bid and the ask prices on Deribit" (a major crypto options exchange).
     • Commercial Angles: Building options trading bots or educational tools for options platforms.
  3. Liquidity Provision on Concentrated AMMs: For LPs on AMMs (Automated Market Makers like Uniswap V3), Synth can predict the probability of the price staying within a chosen liquidity range and estimate expected time in range, helping optimize fee accrual and manage impermanent loss.
• Actionable Insight for Investors/Developers: The path-based data from Synth opens up numerous applications in DeFi for risk management, trading strategy development, and financial product structuring.

Performance Validation and Commercial Strategy

• David from the Synth team conducted backtesting of liquidation probabilities, showing Synth's model significantly outperforms a basic Geometric Brownian Motion benchmark in predicting 1% price movements.
• James discusses the commercialization strategy, initiated 4-5 weeks ago once data quality was confirmed:
  • B2B Data Sales: API access for data marketplaces, hedge funds, exchanges.
  • Tooling Integration: Dashboards and specific metrics (like liquidation probabilities) integrated into exchange UIs.
  • LLM Integration: Enabling LLMs (Large Language Models like ChatGPT) to answer probabilistic financial questions using Synth data.
  • Consumer Products (Exploratory): Ideas like a Synth Chrome extension to overlay price paths on exchange windows.
  • Business to AI (B2AI): Integrations into AI agents, providing training and backtesting data.
• Early Partnerships:
  • Chainrisk: A risk manager for economic security (securing ~$5B in digital assets), integrating Synth for volatility and liquidation insights.
  • Giza: An AI agent builder (tens of thousands of transactions, ~$2M AUM), using Synth API for their stablecoin and other agents.
  • Mode: Synth integrated into Mode's AI terminal (used by ~400 users).
  • NewData: A large alt-data marketplace, planning educational seminars for traditional finance.
• Const asks if a Degen trader could use these signals. James confirms that synthdata.co provides access to predicted paths and dashboards, with API access prioritized for larger organizations. An LLM terminal is also in development.

Live Demo and Future Roadmap

• The team briefly showcases the live volatility dashboard on synthdata.co, noting a predicted spike in volatility. The dashboards provide percentile predictions for future price movements.
• Roadmap:
  • Q1 Complete: Core benchmark improvement, data validation.
  • Q2 (In Progress):
    • ETH predictions added: Miners given notice to adapt models for ETH, with requests starting towards the end of the following week. This will require modeling correlations between BTC and ETH in single paths.
    • Potential TAO integration.
  • Future Assets: Commodities (gold, silver – easier due to near 24/7 markets), then equities (harder due to shorter trading windows, possibly S&P 500 index first, then individual stocks based on demand).
  • Q3/Q4: Expansion into new verticals/subnets (energy pricing, weather, traffic, health).
  • 2026 & Beyond: Powering AGI overlords with probabilistic models for all decision-making. The end goal is for Synth to be the forecasting engine for AIs and LLMs across all industries requiring probabilistic data.
• Sam elaborates that the Monte Carlo structure is adaptable to any time series forecasting, envisioning industry experts building models on Synth for diverse areas.

Monetization, BitTensor Ecosystem, and Alpha Tokens

• Sam poses a question to Const and Joseph (a guest, likely from BitTensor or a related entity) about the best way for Synth to give back its earnings to the BitTensor ecosystem.
• Const suggests that subnets should actively use and monetize their own digital commodities, potentially even "mining" other protocols or subnets. He emphasizes proving the commodity's value.
  • Const: "If you guys get to that point where you know it's true, going out and just monetizing and using the network yourself, I think, is a great way to just lead the field."
• Joseph draws parallels to Bitcoin, starting as an abstract incentive and evolving into a store of value. He posits that each subnet's Alpha token (the native token of a BitTensor subnet, representing stake and access) can be both a medium of exchange for its specific commodity and a store of value, with subnet teams needing to manifest this.
  • Joseph: "I think for alpha tokens to really succeed... is to be a gating mechanism for the commodity, to be a payment mechanism for the commodity that they are the representation of."
• Sam sees this aligning with Synth's vision: the Synth Alpha token could be the currency AIs use to pay for real-time access to its probabilistic data paths.
• Const reiterates the importance of teams using their own networks, creating FOMO and demonstrating value. He highlights the significance of Synth's use of proper scoring rules, solving a critical problem that plagued earlier prediction subnets which were susceptible to "martingale style strategies."

Reflective and Strategic Conclusion

• Synth's sophisticated probabilistic forecasting for Bitcoin, validated by its CRPS mechanism and early use-case traction, signals a mature application on BitTensor. Crypto AI investors should monitor its expansion into new assets (like ETH) and verticals, while researchers can explore its rich, path-based data for novel AI agent development and financial modeling.