Amazon Nova 2 Enhances Fine-Tuning with Self-Distilled Reasoning

Fine-tuning large language models (LLMs) often faces challenges when datasets lack detailed reasoning traces, leading to a decline in models' reasoning abilities. Amazon's Nova 2 family addresses this by proposing Self-Distilled Reasoning (SDR), a method that generates reasoning tokens from a base model to augment training data. This approach helps models retain their reasoning skills and improves task performance without requiring additional human annotation or complex post-processing. Experiments across multiple benchmarks, including MedMCQA, CoCoHD, and Invoice-OCR, show that SDR not only enhances target domain accuracy but also mitigates catastrophic forgetting, a common issue in fine-tuning. Unlike model merging, which can cause a trade-off between specialized and general skills, SDR maintains a balance, preserving broad capabilities while boosting domain-specific performance. This technique is particularly useful in scenarios where reasoning data is scarce or expensive to curate, offering a scalable and cost-effective solution for model customization. The research highlights the importance of internal model distillation processes, aligning with broader trends in continual learning and model robustness. Overall, Amazon's innovation in self-distilled reasoning provides a practical pathway for improving AI model fine-tuning, especially in resource-constrained settings.