A study evaluated safety monitors by measuring their ability to flag harmful prompts sent to deployed language models. The evaluation used recall against harmfulness labels, but found that a monitor’s effectiveness depends on whether the model would have complied with the prompt in the first place. The research sampled repeated responses from six model configurations, including activation probes, fine-tuned text guards, and a 120B policy-conditioned reasoning classifier. Recall on prompts the monitor successfully caught was 0.22 to 0.38 lower than recall on prompts the model would have answered regardless.
The study revealed that prompts a monitor missed were 2.8 to 5.6 times more likely to be complied with. This gap was observed across three model families, indicating the issue is not specific to a particular model architecture. The findings suggest standard recall metrics may overestimate the protection provided by safety monitors.
Furthermore, the results held true even in text-only monitors, independent of the target model. This implies the problem isn't specific to the model being protected, but rather the evaluation method itself. The research emphasizes the importance of evaluating monitors based on what their models will actually answer, rather than simply measuring recall against harmfulness labels.
Source: https://arxiv.org/abs/2609.05797
