The research introduces AuditForecast, a system for structured probabilistic forecasting using LLM agents. Current agentic forecasting frequently relies on implicit narrative aggregation, limiting accuracy and auditability. AuditForecast addresses this by anchoring forecasts with a quantitative baseline model and using model-guided data retrieval to establish a base probability. Situational factor updates are then applied through mechanical aggregation in odds space. This creates a structured process with explicit intermediate objects. Across multiple live forecasting benchmarks, AuditForecast demonstrates improved accuracy and calibration compared to strong agentic baselines. It surpasses market-implied references in several settings and outperforms substantially more expensive deep-research agents. Furthermore, the system produces an auditable forecasting report, enabling explicit forecast construction and systematic post hoc analysis.
The system’s design prioritizes both performance and transparency. The mechanical aggregation in odds space provides a clear, auditable trail of how the final forecast was derived. This allows for better understanding of the forecasting process and facilitates systematic post hoc analysis. The system’s performance gains are significant, exceeding the accuracy of existing agentic baselines and competitive deep-research agents.
Key details include the system’s ability to remain Pareto-dominant in the cost-accuracy tradeoff. This means it achieves superior performance compared to more expensive alternatives without increasing costs. The system’s architecture allows for straightforward integration into existing agentic workflows.
AuditForecast provides a framework for creating more reliable and understandable forecasts, crucial for applications requiring verifiable predictions. The system’s auditable report supports systematic analysis and improves trust in the forecasting process.
Source: https://arxiv.org/abs/2609.05905
