859c8ee2a3
perf: enable fused AdamW in full SFT training
...
Use PyTorch's fused AdamW implementation to reduce optimizer kernel
launch overhead and speed up the training step on CUDA devices.
Guard with device check so the CPU-only fallback path is unaffected.
Co-authored-by: TRAE CLI <traecli@bytedance.com>
2026-09-05 18:37:04 +08:00
6fc918beb6
[update] remove expert args
2026-09-04 22:16:38 +08:00
fc585a9bf9
[update] 把随机种子暴露到命令行
...
trainer 里的种子此前硬编码为 42（setup_seed(42 + rank) 与每轮的
setup_seed(42 + epoch)），要跑多种子实验只能改源码。新增 --seed，
默认 42，与改动前完全一致。
2026-09-03 15:36:14 +07:00
c3199c7240
[update] 把 num_experts / num_experts_per_tok 暴露到命令行
...
此前要改专家数只能去编辑 model_minimind.py。三个入口
（train_pretrain / train_full_sft / eval_llm）的默认值与 MiniMindConfig
一致（4 / 1），默认行为不变；RL 与 agent 相关的 trainer 不动。
2026-09-03 15:13:29 +07:00
89d674b8a5
[refactor] PPO cleanup
2026-07-23 11:11:05 +08:00
2d4c385b0f
[fix] PPO rollout mode
...
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-23 11:05:51 +08:00
1191fc8e9f
[fix] PPO: align mb_resp_logp with old_resp_logp under autocast
...
Move log_softmax for mb_resp_logp inside autocast_ctx so it is upcast
to fp32, matching the rollout path (compute_per_token_logps runs under
no_grad+autocast). Previously log_softmax ran outside autocast on
fp16/bf16 logits, introducing a ~1e-2 systematic bias between
mb_resp_logp and old_resp_logp, so ratio != 1 and KL != 0 even before
any weight update. Under sparse rule-based rewards (e.g. math PPO)
this bias destabilises training.
Also add:
- --rollout_eval_mode (default 1): switch actor to eval during rollout
to disable dropout and keep old/new logp consistent.
- --debug_log_ratio: optional diagnostic printing |mb-old| on the first
minibatch to verify ratio ~= 1.
2026-07-21 18:51:42 +08:00
3f1a7cc25b
[fix] ddp exit hang
2026-06-01 17:50:39 +08:00
4497610ec0
[fix] issue#771
2026-05-19 17:40:03 +08:00
802c15b2b4
[feat] reduce RL memory
2026-05-06 15:07:28 +08:00
10776417aa
fix: resolve CUDA OOM in train_grpo.py on GPUs with <=8GB VRAM
2026-05-06 14:14:15 +08:00
bdee223036
[fix] inference bug
2026-05-03 20:48:48 +08:00
773e451b11
[fix] bugs
2026-04-27 19:16:08 +08:00
6361510016
[fix] rollout bugs
2026-04-27 17:54:09 +08:00
5416a44471
[fix] bugs
2026-04-21 13:03:34 +08:00
e796b8028a
[fix] lora compile
2026-04-10 00:01:21 +08:00
5351424bf0
[fix] lora moe
2026-04-09 16:36:48 +08:00
aa3e6affa1
[update] add a comment
2026-04-09 15:09:58 +08:00
b8b3d35257
[update] change default seq_len
2026-03-26 10:09:06 +08:00
101d7df2da
[update] minimind-3
2026-03-25 23:57:45 +08:00
cf4b49a348
[fix] align log/save last-step check and ETA with 1-indexed step
2026-03-24 02:01:40 +01:00
d25500d363
[fix] gradient accumulation step alignment
2026-03-24 01:45:04 +01:00
11a44340ba
[update] save interval
2026-01-30 20:30:50 +08:00
fea69cf338
[fix] data skip
2026-01-18 16:56:29 +08:00
f7ffdf1fdb
[update] shuffle data
2026-01-18 16:39:34 +08:00
c090b69c4d
[update] align loss
2026-01-15 00:56:32 +08:00
e119db8478
[fix] compile unpack
2026-01-14 20:13:32 +08:00
81d24a4f16
[feat] add compile
2026-01-14 14:42:30 +08:00
df89069362
[update] params log
2026-01-07 23:08:45 +08:00
07364c3fbe
[update] rename train tokenizer
2026-01-06 01:17:33 +08:00
4e73f34823
[update] rename reason
2026-01-05 23:12:29 +08:00
42a4e8c86a
[fix] dist cleanup
2026-01-02 22:25:55 +08:00
9d898576ac
[update] aux loss
2026-01-01 22:41:46 +08:00
c65335b56f
[fix] experts unused
2025-12-31 21:47:04 +08:00
bc8fd82166
[fix] layers set 8
2025-12-31 21:06:37 +08:00
5dd4df7e18
[fix] moe unused
2025-12-31 21:00:06 +08:00
9236260a4a
[feat] get params
2025-12-31 20:46:59 +08:00
288a1d7212
[feat] get params
2025-12-31 20:44:34 +08:00
6242980917
[feat] update lr
2025-12-31 10:27:09 +08:00
7eae14f3ce
[feat] remove empty_cache
2025-12-27 07:14:36 +08:00
11b962da06
[feat] explicit left padding
2025-12-23 18:59:48 +08:00
fe24501602
[feat] adjust seq length
2025-12-14 20:41:58 +08:00
5129f0e2a2
[fix] dtype & lr
2025-12-09 13:01:38 +08:00
bf3878ace8
[fix] Refactor get_lr function to include min_lr calculation
...
这里的退火算法会让参数里的lr的起始值变成原来lr的1.1倍，作出如下修改
2025-12-06 17:09:51 +08:00
5e1447b913
[fix] cuda memory #559
2025-12-01 16:17:43 +08:00
6b86ea399a
[feat] release memory
2025-11-27 19:39:49 +08:00
d7f4f4eab8
[fix] ppo mask
2025-11-19 23:39:02 +08:00
9c98cabc9a
[fix] prompt length calculate
2025-11-15 18:25:37 +08:00
509d8dacf1
[feat] clear cache
2025-11-06 13:12:28 +08:00
0323815729
[feat] update import
2025-10-31 23:45:55 +08:00
neil.ge
jingyaogong
Justin Guese
FloatFrank
TKiteRunner
jingyaogong
readlnh
dyhuachi