Neuron-Guided Fine-Tuning (NGFT) is a framework designed to improve the efficiency of Large Language Model fine-tuning. The approach addresses issues of parameter redundancy, inconsistent data quality, and catastrophic forgetting commonly found in existing Supervised Fine-Tuning paradigms. NGFT operates through three interconnected mechanisms: Adaptive Task-Specific Neuron Selection identifies key neurons for updates, Activation-Based Data Selection prioritizes information-dense samples, and Neuron Activation Alignment anchors activations to pre-trained states. These mechanisms work together to optimize the fine-tuning process. Experimental results across three models on both domain-specific and general benchmarks demonstrate that NGFT outperforms mainstream fine-tuning methods in both efficiency and performance while mitigating catastrophic forgetting. The framework focuses on leveraging neuron activation patterns as a universal proxy to unify the entire fine-tuning lifecycle.
Source: https://arxiv.org/abs/2609.05913