The paper argues that current approaches relying on large language models for consequential quantitative tasks are fundamentally flawed. The core issue is that language models are trained on representations of the world derived from human descriptions, a process inherently prone to loss of quantitative information. This loss is irreversible, meaning no matter the model size, it cannot fully reconstruct the original quantitative record. The research formalizes this as a property of the training representation, rather than model capacity.
Specifically, the paper identifies three properties crucial for consequential settings that a language substrate cannot provide: reproducibility, the ability to trace outputs back to their source records, and calibrated uncertainty. These elements are necessary for reliable decision-making in domains where accuracy and traceability are paramount. The authors propose a distinct model class, termed Large Quantitative Models (LQMs), designed to meet these requirements.
LQMs are characterized by their focus on directly processing quantitative data and maintaining a clear lineage from input records to outputs. This contrasts with LLMs, which primarily operate on textual representations. The research suggests that achieving the necessary levels of accuracy and reliability necessitates a fundamentally different approach to model design and training.
This research highlights the importance of considering the specific requirements of different domains when selecting and deploying AI models. It suggests that for tasks involving quantitative reasoning and consequential outcomes, LQMs represent a more suitable architecture than relying solely on large language models. The paper emphasizes the need for models that can accurately reproduce results and provide calibrated uncertainty estimates.
Source: https://arxiv.org/abs/2609.12105
