b3bd25fb19
Merge 9cb07dd1ec into f659b55761
2026-09-29 16:15:42 +08:00
a6ffddc0ea
fix: update residual gradients before saving final checkpoints
2026-09-21 23:24:55 -07:00
859c8ee2a3
perf: enable fused AdamW in full SFT training
...
Use PyTorch's fused AdamW implementation to reduce optimizer kernel
launch overhead and speed up the training step on CUDA devices.
Guard with device check so the CPU-only fallback path is unaffected.
Co-authored-by: TRAE CLI <traecli@bytedance.com>
2026-09-05 18:37:04 +08:00
6fc918beb6
[update] remove expert args
2026-09-04 22:16:38 +08:00
fc585a9bf9
[update] 把随机种子暴露到命令行
...
trainer 里的种子此前硬编码为 42（setup_seed(42 + rank) 与每轮的
setup_seed(42 + epoch)），要跑多种子实验只能改源码。新增 --seed，
默认 42，与改动前完全一致。
2026-09-03 15:36:14 +07:00
c3199c7240
[update] 把 num_experts / num_experts_per_tok 暴露到命令行
...
此前要改专家数只能去编辑 model_minimind.py。三个入口
（train_pretrain / train_full_sft / eval_llm）的默认值与 MiniMindConfig
一致（4 / 1），默认行为不变；RL 与 agent 相关的 trainer 不动。
2026-09-03 15:13:29 +07:00
3f1a7cc25b
[fix] ddp exit hang
2026-06-01 17:50:39 +08:00
4497610ec0
[fix] issue#771
2026-05-19 17:40:03 +08:00
5416a44471
[fix] bugs
2026-04-21 13:03:34 +08:00
b8b3d35257
[update] change default seq_len
2026-03-26 10:09:06 +08:00
101d7df2da
[update] minimind-3
2026-03-25 23:57:45 +08:00
cf4b49a348
[fix] align log/save last-step check and ETA with 1-indexed step
2026-03-24 02:01:40 +01:00
d25500d363
[fix] gradient accumulation step alignment
2026-03-24 01:45:04 +01:00
fea69cf338
[fix] data skip
2026-01-18 16:56:29 +08:00
f7ffdf1fdb
[update] shuffle data
2026-01-18 16:39:34 +08:00
c090b69c4d
[update] align loss
2026-01-15 00:56:32 +08:00
e119db8478
[fix] compile unpack
2026-01-14 20:13:32 +08:00
81d24a4f16
[feat] add compile
2026-01-14 14:42:30 +08:00
42a4e8c86a
[fix] dist cleanup
2026-01-02 22:25:55 +08:00
9d898576ac
[update] aux loss
2026-01-01 22:41:46 +08:00
c65335b56f
[fix] experts unused
2025-12-31 21:47:04 +08:00
bc8fd82166
[fix] layers set 8
2025-12-31 21:06:37 +08:00
5dd4df7e18
[fix] moe unused
2025-12-31 21:00:06 +08:00
6242980917
[feat] update lr
2025-12-31 10:27:09 +08:00
7eae14f3ce
[feat] remove empty_cache
2025-12-27 07:14:36 +08:00
fe24501602
[feat] adjust seq length
2025-12-14 20:41:58 +08:00
5e1447b913
[fix] cuda memory #559
2025-12-01 16:17:43 +08:00
6b86ea399a
[feat] release memory
2025-11-27 19:39:49 +08:00
509d8dacf1
[feat] clear cache
2025-11-06 13:12:28 +08:00
bf123b585d
[feat] add args
2025-10-30 10:05:12 +08:00
1713c24114
[fix] model device
2025-10-29 10:36:28 +08:00
8f7e07b8ef
[feat] update trainer
2025-10-28 23:30:10 +08:00
e8484874f5
[feat] pause-training
2025-10-26 18:49:52 +08:00
a82526da11
[feat] shuffle data
2025-10-23 20:13:28 +08:00
805744e60a
[fix] loss-issues-430
2025-10-23 19:08:42 +08:00
463044e92a
[fix] sampler-ddp
2025-10-23 15:03:19 +08:00
fa7dff8291
[fix] graph-oom & ddp-pos_cis
2025-10-23 14:22:13 +08:00
7d877db79b
[feat] minimind-2510
2025-10-21 21:19:47 +08:00
caae54a89e
fix bugs
2025-04-29 20:45:39 +08:00
29454c31af
fix bugs
2025-04-27 09:56:49 +08:00
a62faf34bd
250426
2025-04-26 10:05:47 +08:00
Neil Ge
Daoyuan Li
neil.ge
jingyaogong
Justin Guese
jingyaogong
readlnh