Research1 min read
Correctness-Gated Distillation Shows No Grounding Benefit
Experiments with a 63.9M-parameter student model using correctness-gated multi-teacher distillation showed minimal accuracy gains compared to hard filtering. The audit of the weighted arm revealed significant label loss and failed to establish a grounding effect, indicating no incremental decision benefit.
From arXiv cs.AI