The research addresses challenges in cascaded simultaneous speech translation where the machine translation system lacks control over the recognizer's read-write schedule. The approach involves fine-tuning a decoder-only LLM, specifically Qwen3-8B for EN to DE, JA, and ZH, on stable prefixes derived from shared source and target text. This prefix-aware training improves the quality of chunks early in the translation process. The system utilizes a single force-decode turn, continuously committing target text as the source stream arrives, resulting in a flicker-free translation.
The key finding is that a single training-free threshold based on token-level commit confidence proved most effective as a latency control. This threshold creates a continuous quality-latency frontier, outperforming discrete wait-$k$ and target-suffix-deletion strategies. Evaluation on FLEURS, WMT24++, and CoVoST~2 test sets, under COMET and MetricX, consistently validated this performance.
This method improves calibration of token-level commit confidence, reducing expected calibration error (ECE) on early source prefixes. The research suggests that this dynamic lagging approach offers a practical solution for managing latency in real-time simultaneous translation systems. This is relevant for engineers deploying and optimizing models for production environments.
Source: https://arxiv.org/abs/2609.05799