Firehose

Improving document extraction accuracy with ai_extract and MLflow Prompt Registry involves using schema descriptions for field-level rules, global instructions for document-level logic, precision mode for structural complexity, and citations to diagnose errors. A repeatable workflow can be built using Genie Code, which automates the core of this workflow, scoring extraction field by field against labeled data, proposing schema and instruction fixes as reviewable diffs, and re-running the eval to show before-and-after results. AI summary

Large language models (LLMs) can produce highly variable judgments of novel research ideas, with small changes in the evaluation setup resulting in dramatically different verdicts. This instability is particularly pronounced when evaluating AI-generated ideas, where the choice of prompt, reasoning budget, and model can significantly impact the outcome, and more search or reasoning does not necessarily produce more consistent judgments. AI summary

The AI SDK HarnessAgent layer now accepts image inputs across every harness adapter. Pass images alongside text in the standard AI SDK message format: a content array mixing { type: 'text' } and { type: 'file' } parts. Image input is proces…