Researchers at Stanford and the University of Washington have unveiled a new AI model, s1, that rivals leading reasoning models, such as OpenAI's o1 and DeepSeek's r1, with just a fraction of the resources. Utilizing a method called supervised fine-tuning (SFT), the team developed s1 using a modest dataset and completed the training in under 30 minutes on 16 Nvidia H100 GPUs. The new approach demonstrates that high-performance reasoning models can be created efficiently and cost-effectively.
The s1 paper suggests that SFT allows for the distillation of reasoning models using a small dataset. This process involves training an AI model to replicate specific behaviors found in the dataset. The researchers achieved this by creating a curated set of 1,000 questions from Google's Gemini 2.0 Flash Thinking Experimental, including answers and detailed explanations of the "thinking" process behind each response. With this data, they trained the s1 model through SFT, achieving strong results on AI benchmarks measuring math and coding abilities.
s1's performance has been noted as comparable to cutting-edge reasoning models like OpenAI's o1 and DeepSeek's r1. The innovative use of SFT, as opposed to DeepSeek's large-scale reinforcement learning method, proved to be more cost-effective. According to the researchers, they aimed to find "the simplest approach to achieve strong reasoning performance and 'test-time scaling,' or allowing an AI model to think more before it answers a question."
The distillation method used to create s1 is similar to an approach employed by Berkeley researchers who developed an AI reasoning model for approximately $450 last month. While distillation can efficiently replicate existing AI capabilities, it does not necessarily produce models that surpass current technology. However, it offers a viable path for developing high-performance models with limited resources.
The foundation of s1 lies in a small, off-the-shelf AI model from Qwen, a Chinese AI lab owned by Alibaba, which is freely available for download. The s1 team leveraged this base model and applied distillation to extract reasoning capabilities from another AI model by training on its answers. This approach has proven successful in creating a competitive reasoning model at a fraction of the cost.
Google provides free access to Gemini 2.0 Flash Thinking Experimental through its Google AI Studio platform, albeit with daily rate limits. This accessibility was integral to the researchers' ability to create their dataset and train s1 effectively within budget constraints. The breakthrough achieved by s1 signifies a pivotal moment in AI research, proving that significant advancements can be made without extensive financial investment.
Leave a Reply