SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?
This paper introduces SWE-bench Science, a benchmark for evaluating coding agents' performance in repairing scientific software, and identifies common failure mechanisms that hinder their success. Practitioners may care about this research as it can help improve the reliability and reproducibility of scientific findings by developing more effective coding agents.