# AWS teaches Amazon Nova models to keep their math skills during fine-tuning

**AWS built a technique called Self-Distilled Reasoning that stops Amazon Nova models from losing their reasoning ability during fine-tuning.**

## What happened

Creating high-quality chain-of-thought (CoT) reasoning traces for a Supervised Fine-Tuning (SFT) dataset is often impractical and expensive, so many teams skip reasoning traces entirely and train Amazon Nova models on plain inputs and outputs. According to [AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/exploring-self-distilled-reasoning-for-supervised-fine-tuning-with-amazon-nova), reasoning is a core capability of the Amazon Nova 2 family and drives real gains on hard problems such as coding and math, so training without it has a cost. AWS set out to keep that reasoning ability alive even when a fine-tuning dataset has no reasoning traces of its own, and the result is a new method called Self-Distilled Reasoning (SDR).

## Why fine-tuning breaks reasoning

AWS calls this the reasoning suppression problem. When a model is fine-tuned on input-output pairs with no intermediate reasoning steps, the training loss is computed across every token, including the ones that would normally carry a chain of thought. Because there is no supervision signal telling the model to produce reasoning, the loss function ends up rewarding it for skipping straight to an answer, an example of what AWS calls shortcut learning, where a model leans on shortcuts instead of a reliable reasoning path. The effect showed up clearly in AWS's own numbers: standard SFT dropped math performance from 70 percent at the base model down to 6 percent on average once fine-tuned. The usual fix is model merging, which blends the fine-tuned checkpoint back with the base model to bring reasoning back, but that trade-off also gives up some of the accuracy gained from fine-tuning in the first place.

## How Self-Distilled Reasoning works

Instead of pulling reasoning traces from a separate teacher model or interpolating checkpoints after training, SDR reuses the base Amazon Nova 2 Lite model's own chain of thought and inserts it into training examples that never had reasoning traces to begin with. That makes SDR a form of self-distillation, applied as in-training regularization rather than as a repair step done afterward. Because the reasoning traces come from the model itself, SDR needs no human annotation, no golden CoT dataset, and no separate merging step, and AWS says it can be applied to any existing SFT dataset regardless of domain.

## What the results show

Compared with model merging, SDR reaches nearly identical math performance, 70 percent versus 68 percent, while improving target-task accuracy by more than 6.5 percent on average, since it does not give up SFT gains to get there. A related test on the LLaVA CoT dataset compared reasoning-on versus reasoning-off performance across different LoRA (Low-Rank Adaptation) merge weights. At a 0.7 merge weight, target performance reached 66.67% with reasoning on versus 54.05% with reasoning off, a 12.6% gap, and a full merge at a 1.0 weight reached 65.17% versus 47.90%, a 17.3% gap, evidence that reasoning underlies a broad set of downstream skills, not just math. AWS validated SDR itself across three real-world benchmarks: ToolACE, a tool-calling benchmark scored on exact function-call matches; CoCoHD, which extracts fields from long congressional hearing transcripts; and GovReport, which scores government-report summaries against a reference.

## Key facts

- Standard Supervised Fine-Tuning (SFT) dropped Amazon Nova math performance from 70 percent (base) to 6 percent on average.
- Self-Distilled Reasoning (SDR) recovered math performance to 68 percent, close to the 70 percent base score, versus model merging.
- SDR improved target-task accuracy by more than 6.5 percent on average compared with model merging.
- At a 0.7 LoRA merge weight, reasoning-on scored 66.67% versus 54.05% reasoning-off, a 12.6% gap; a full 1.0 merge widened that to 65.17% versus 47.90%, a 17.3% gap.
- SDR needs no separate teacher model, no human annotation, and no post-hoc checkpoint interpolation.

## Check your understanding

- Why did standard Supervised Fine-Tuning drop Amazon Nova's math performance from 70 percent to 6 percent?
- How does Self-Distilled Reasoning recover math performance without a separate teacher model or a model-merging step?
- What does the LoRA merge-weight test on the LLaVA CoT dataset suggest about reasoning's role beyond math?

## Sources

- [AWS Machine Learning Blog: Exploring self-distilled reasoning for supervised fine-tuning with Amazon Nova](https://aws.amazon.com/blogs/machine-learning/exploring-self-distilled-reasoning-for-supervised-fine-tuning-with-amazon-nova)

---

- Category: learn (research)
- Affects: agent builders · training data quality
- Published: 2026-07-21T16:23:12.000Z
- Original: https://aws.amazon.com/blogs/machine-learning/exploring-self-distilled-reasoning-for-supervised-fine-tuning-with-amazon-nova
- Generated by tinker news