AIAI Snake Oilalmost 2 years ago
Can AI automate computational reproducibility?
6 min read
Sayash Kapoor and Arvind Narayanan introduced CORE-Bench, a benchmark measuring how well AI can automate computational reproducibility. The benchmark evaluates AI agents on reproducing a paper's findings when code and data are available. Sakana AI's "AI Scientist" lacked novelty checks and human review, producing flawed papers.
Level
Hype check
Tap to vote and see what everyone thinks.
