Firehose

This paper introduces SciGame, a controlled benchmark for evaluating language model agents in scientific discovery, and finds that familiar priors can hinder performance, adaptive experimentation can be beneficial but also unfair, and that agents struggle to induce exact symbolic laws in unfamiliar settings.