21b8a8b70a
feat: 添加 git 交互脚本
2026-08-27 22:30:52 +08:00
2b250aca51
chore: remove .idea and add to .gitignore
2026-08-27 22:10:02 +08:00
387925b6bc
[update] readme content
2026-08-21 16:49:00 +08:00
8425924da2
[update] readme content
2026-08-21 16:30:30 +08:00
32928c3a33
[update] readme content
2026-08-21 16:25:20 +08:00
f7ea0bc4f6
[update] supporter charts
2026-08-06 22:07:29 +08:00
f07588a31b
[refactor] PPO cleanup
2026-07-23 11:11:05 +08:00
185baaa022
[fix] PPO rollout mode
...
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-23 11:05:51 +08:00
1b43526b16
Merge pull request #814 from FloatFrank/fix/ppo-logp-autocast-align-v3
...
[fix] PPO: align mb_resp_logp with old_resp_logp under autocast to stabilize training
2026-07-23 10:32:06 +08:00
27b9c0082f
[fix] PPO: align mb_resp_logp with old_resp_logp under autocast
...
Move log_softmax for mb_resp_logp inside autocast_ctx so it is upcast
to fp32, matching the rollout path (compute_per_token_logps runs under
no_grad+autocast). Previously log_softmax ran outside autocast on
fp16/bf16 logits, introducing a ~1e-2 systematic bias between
mb_resp_logp and old_resp_logp, so ratio != 1 and KL != 0 even before
any weight update. Under sparse rule-based rewards (e.g. math PPO)
this bias destabilises training.
Also add:
- --rollout_eval_mode (default 1): switch actor to eval during rollout
to disable dropout and keep old/new logp consistent.
- --debug_log_ratio: optional diagnostic printing |mb-old| on the first
minibatch to verify ratio ~= 1.
2026-07-21 18:51:42 +08:00
237f25bc70
[fix] robustness
2026-06-28 18:03:34 +08:00
13739bdbc5
[fix] ddp exit hang
2026-06-01 17:50:39 +08:00
e1df1a50d7
[fix] lora dims
2026-05-31 13:38:51 +08:00
6534447b62
[fix] issue#771
2026-05-19 17:40:03 +08:00
e90f92d701
[update] requirements
2026-05-16 18:08:37 +08:00
4b0993fcc2
[fix] repetition_penalty
2026-05-07 19:08:52 +08:00
a22f1f1486
[feat] reduce RL memory
2026-05-06 15:07:28 +08:00
ed7cbcee19
Merge pull request #759 from TKiteRunner/fix/grpo-oom-reorder-rewards
...
fix: resolve CUDA OOM in train_grpo.py on GPUs with <=8GB VRAM
2026-05-06 15:00:17 +08:00
1c583deb28
fix: resolve CUDA OOM in train_grpo.py on GPUs with <=8GB VRAM
2026-05-06 14:14:15 +08:00
43eee6add5
[update] readme
2026-05-06 13:41:46 +08:00
58959684c7
[fix] inference bug
2026-05-03 20:48:48 +08:00
d1148341bb
[update] readme
2026-05-01 12:06:47 +08:00
7b4f029ee3
[update] readme
2026-04-28 17:22:25 +08:00
4b85ba8b28
[fix] bugs
2026-04-27 19:16:08 +08:00
3b7a4d04a2
[fix] rollout bugs
2026-04-27 17:54:09 +08:00
b45c36b8b5
[update] readme
2026-04-27 10:59:41 +08:00
e4e9153a1f
[update] readme
2026-04-24 20:16:06 +08:00
29c87cfc8d
[update] readme
2026-04-24 20:14:10 +08:00
c0eb98393f
[update] readme
2026-04-21 14:34:46 +08:00
568a863520
[fix] bugs
2026-04-21 13:03:34 +08:00
761710654a
[fix] transformers-5.x
2026-04-19 23:48:54 +08:00
145a7ac6f6
[update] tie embedding
2026-04-19 21:57:28 +08:00
83b4c697a3
[update] readme
2026-04-19 14:51:47 +08:00
b9a229e1ae
[update] readme
2026-04-10 10:55:24 +08:00
e612cc26e8
[fix] lora compile
2026-04-10 00:01:21 +08:00
903eb1dd37
[update] readme
2026-04-09 19:00:41 +08:00
6b9588d619
[update] readme
2026-04-09 18:54:17 +08:00
d942f24bf1
[update] bench
2026-04-09 17:12:48 +08:00
862dd151f2
[fix] lora moe
2026-04-09 16:36:48 +08:00
b703844a60
[update] add a comment
2026-04-09 15:09:58 +08:00
428f017e21
[update] image
2026-04-04 14:54:13 +08:00
ef1c130a03
[update] readme
2026-04-04 11:25:21 +08:00
7546729893
[update] open causal
2026-04-02 15:28:58 +08:00
6512d5647a
[update] readme
2026-04-02 15:28:29 +08:00
ea57409fd2
[update] readme
2026-04-01 14:00:21 +08:00
d52691f2e3
[update] default model
2026-03-31 13:40:16 +08:00
5e9d15614e
[update] random seed
2026-03-27 21:20:02 +08:00
8f3169dba2
[update] fp16 inference
2026-03-27 16:29:46 +08:00
deed5674c4
[update] image
2026-03-26 15:35:42 +08:00
2c24568176
[update] change default seq_len
2026-03-26 10:09:06 +08:00
jinhu
jingyaogong
FloatFrank
TKiteRunner
jingyaogong