RLVR might be disproportionately bad at science The article explores the difficulty of recognizing scientific progress, challenging the idea that AI will excel due to its ability to handle verification loops. It highlights historical examples where accepted theories made worse predictions and where progress took centuries, emphasizing the role of human judgment, heuristics, and persistent dedication to hypotheses.

• Recognizing scientific progress is a complex and often slow process, taking decades or centuries.
• Historical scientific theories, like Copernicus’s model, sometimes made worse predictions than older models before later advancements provided context.
• AI’s strength in domains with tight verification loops (coding, math) may not translate to scientific discovery where verification can be lengthy and ambiguous.
• Human scientific progress has relied on judgment, heuristics, and even seemingly unreasonable obstinacy in pursuing research agendas.
• Conceptual breakthroughs are hard to verify ex ante, and their value is often recognized much later when they prove more productive than alternatives.
• The society of AI scientists may need to support individual AI instances with idiosyncratic biases to keep dormant research agendas alive. Continue reading https://foxvector.com/articles/feb4d01d-92bf-41b4-8f9a-c8b5073d627f