The research introduces a framework for online adaptation of web agents, addressing the cost of using powerful teacher models. The approach focuses on optimizing the teaching process to minimize resource consumption. Conventional trajectory-level preference optimization is found to be inefficient due to unresolvable episodes and redundant execution turns. The proposed framework, Score-Guided Online Teaching with Budgeted Trajectory Trimming, integrates a solvability-aware teacher gate to determine when to query the teacher model. Simultaneously, a score-guided turn selection mechanism decides which informative turns to retain. Experiments on MiniWoB and TimeWarp demonstrate a 22.6% reduction in teacher calls and a 52.1% reduction in student training compute on average.
Source: https://arxiv.org/abs/2609.05513