At Noora Health, a large language model (LLM) was used to classify WhatsApp messages regarding maternal and newborn care as emergencies. Initial implementation struggled with interpretability, requiring manual analysis of reasoning chains for each message. This approach was impractical at scale, and prompt changes necessitated full system evaluations. The system relied on a flat list of danger signs, lacking clinical decision-making guidance. The new system decomposes triage into two steps: an LLM extracts symptoms and context using a clinician-authored vocabulary, and a deterministic rule engine identifies emergencies. This decomposition increased recall from 0.565 to 0.810 and F1 from 0.606 to 0.702. Clinical experts can now inspect each stage of the system to identify errors, adding new rules independently without causing regressions. Since deployment, the system has processed 152,421 patient queries and flagged 28,535 (18.7%) as emergencies.
Source: https://arxiv.org/abs/2609.09356