Novelty Search August 14, 2025

This episode reveals how the Gradients subnet built a world-leading AI training platform that now outperforms major models like Quen 3, and is strategically pivoting to an open-source framework to capture enterprise revenue.

Introduction: The Freedom of Decentralized AI Development

The episode opens with a discussion on the decentralized and global nature of the BitTensor ecosystem, where developers and miners operate with complete freedom from anywhere in the world. The speakers highlight the lifestyle of a "BitTensor miner," who can contribute to cutting-edge AI from remote locations, such as a cargo boat on the Amazon River, equipped with just a laptop and a Starlink connection. This freedom, they argue, is a key driver of the creativity and relentless problem-solving that defines the network.

• Speaker Perspective: Const, the host, champions this decentralized ethos, stating, "The more freedom that you give people, the more creative and more driven they are to solve the problem." This sets the stage for understanding the culture behind Gradients' rapid innovation.

Gradients: The "Meta" Subnet for Automated Model Training

Const introduces Gradients as one of the most "meta" subnets on BitTensor—a decentralized marketplace designed to incentivize machine learning engineers to build models that automatically train other AI models. He admits to initially believing the problem was impossible to solve but acknowledges that the Wandering Weights team has proven its viability and achieved incredible success. The core function of Gradients is to take a base model that understands language but lacks specific knowledge and fine-tune it to become intelligent and useful for specific applications.

Platform Demonstration: Gradients 101

The Wandering Weights team provides a live demonstration of the Gradients platform, showcasing its user-friendly interface for both text and image model training.

• Text Model Training (LLMs): Users can select any model from Hugging Face, choose a dataset (e.g., a math dataset), map the input and output columns, set the training duration, and launch the job to the network of miners.
• Diffusion Model Training (Images): Users can drag and drop a set of images (e.g., photos of a person) and the platform can now auto-caption them. By training a model on these images, users can then generate new images based on text prompts, like creating different artistic versions of the person.

Team and Development Velocity

The Gradients team, composed of researchers with publications in top-tier AI conferences like NeurIPS and ICML, emphasizes their hunger and drive over academic credentials. This has translated into an extremely rapid development cycle over the last nine months.

• Key Development Milestones: The platform has seen five major releases: Instruct, Diffusion, DPO, GPO, and now Open Source.
• Development Statistics: This progress was built on nearly half a million lines of code and over 4,000 commits, demonstrating the team's relentless work ethic.

Advancing the State-of-the-Art: DPO and GRPO Training

Gradients has integrated two cutting-edge techniques for model fine-tuning, moving beyond simple instruction-following to more nuanced and controllable AI behavior.

• DPO (Direct Preference Optimization): This technique trains a model to prefer certain types of answers over others. DPO allows users to provide a prompt with both a "chosen" (preferred) and a "rejected" answer, teaching the model to align with specific human preferences, such as tone, courtesy, or censorship rules.
• GRPO (Group Reward Policy Optimization): A more novel method inspired by the DeepSeek model, GRPO allows users to define custom reward functions to guide a model's output. For example, a user can create a reward function that gives the model points for structuring its answer in a specific format (e.g., "Think:" followed by "Answer:"). This opens the door for highly customized and complex model behaviors, with plans to support programming containers for even more advanced reward logic.

Performance Benchmark: The Best Training Platform on the Planet

The team conducted an extensive experimental study, running over 180 model-dataset training pairs to compare Gradients against major platforms like Databricks, GCP, Hugging Face, and Together AI. The goal was to determine which platform produced the model with the lowest loss on an unseen test set.

• The Results: The outcome was a decisive victory for Gradients. It consistently outperformed all competitors across various tasks, including translation, math, code, and reasoning. The only exception was against Hugging Face on tiny models, where results were within the margin of noise.
• Strategic Implication: The speaker makes a bold claim based on these results: "If you're a miner on another subnet and you're doing anything that requires training... stop doing that and come and do it on gradients." This positions Gradients as the default, superior choice for any training task on BitTensor.

Gradients 5.0: The Strategic Pivot to Open Source

Despite proven technical superiority, the team encountered a major barrier to enterprise adoption: data privacy. Customers were hesitant to send their proprietary data to an anonymous, decentralized network of miners. This led to Gradients 5.0, a strategic shift to an open-source model.

• The New Mechanism: Miners no longer submit a trained model (a "black box"). Instead, they submit their open-source training scripts. Validators run these scripts on a restricted compute environment, ensuring transparency and preventing miners from simply throwing excessive compute at the problem.
• Tournament Structure: The new system operates like a "World Cup" tournament. The top 16 miners compete in group stages and knockout rounds. The winner must then beat the previous tournament's champion to claim the top spot and earn the majority of emissions.
• Actionable Insight: This open-source model directly addresses enterprise security concerns. By making the training process transparent, Gradients can offer a trustworthy solution to customers, unlocking a significant revenue stream while also accelerating innovation as miners learn from each other's public scripts.

Beating the Best: Gradients Instruct 8B Outperforms Quen 3

The episode culminates in a major announcement: the Gradients team has created a model that beats a leading open-source model from a major AI lab.

• The Model: Gradients Instruct 8B, a fine-tune of the Quen 3 Base model, was trained on a custom dataset using the Gradients platform.
• The Result: On zero-shot benchmarks (testing the model's ability to perform tasks without examples), Gradients Instruct 8B significantly outperforms the official Quen 3 Instruct model, particularly in math and instruction-following tasks.
• Strategic Significance: This is a landmark achievement, proving that a model trained entirely on a decentralized BitTensor subnet can surpass a state-of-the-art model from a world-class AI company. The team's next goal is to prove it is the best 8B parameter model on the planet.

Future Directions: The AutoML Flywheel

The Q&A session explores the future of Gradients, focusing on its self-improving "flywheel" effect. The open-source, competitive environment ensures that the platform's capabilities will continuously accelerate. Miners who slow down are automatically replaced by more innovative competitors, creating a system that never stops improving.

• Exploring an AutoML Script: The speaker provides a deep dive into a miner's script, revealing its complexity. The script makes thousands of micro-decisions—adjusting learning rates, choosing kernel optimizations, and managing batch sizes based on the specific model and dataset. This demonstrates the sophisticated, automated expertise that the Gradients network has cultivated.
• Future Integrations: The discussion touches on integrating other BitTensor subnets, such as compute providers (e.g., Shoots), to further optimize costs and tie the technology deeper into the ecosystem.

Conclusion

Gradients has validated BitTensor's core premise: decentralized incentive mechanisms can produce world-leading AI. By proving its technical superiority and strategically shifting to an open-source model to solve for enterprise trust, Gradients is positioned for significant growth. Investors should watch for revenue traction, while researchers can now access a treasure trove of state-of-the-art training techniques.