Commit Graph

4 Commits (ccdf6c576fecb90eeab5d0e351f3738d6e82a6e6)

Author SHA1 Message Date
Gouryella ccdf6c576f Improve training performance with torch.compile and torch.amp
- Update AMP usage from `torch.cuda.amp` to `torch.amp` for broader device compatibility.
- Add `torch.compile` to models in various training scripts (SFT, LoRA, DPO, Distillation, Pretrain, Distill Reason) to leverage potential performance gains.
- Implement a custom `load_state_dict` in `MiniMindForCausalLM` to handle state dict keys potentially modified by `torch.compile`.
2025-05-21 12:16:02 +08:00
jingyaogong caae54a89e fix bugs 2025-04-29 20:45:39 +08:00
jingyaogong 29454c31af fix bugs 2025-04-27 09:56:49 +08:00
jingyaogong a62faf34bd 250426 2025-04-26 10:05:47 +08:00