c31f66525d
Merge 100a49bad2 into 393e387e9a
2026-08-07 08:20:34 +08:00
393e387e9a
[update] supporter charts
2026-08-06 22:07:29 +08:00
100a49bad2
feat: make code agent runs reproducible
2026-08-01 16:18:41 +08:00
89d674b8a5
[refactor] PPO cleanup
2026-07-23 11:11:05 +08:00
2d4c385b0f
[fix] PPO rollout mode
...
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-23 11:05:51 +08:00
e9f46506da
Merge pull request #814 from FloatFrank/fix/ppo-logp-autocast-align-v3
...
[fix] PPO: align mb_resp_logp with old_resp_logp under autocast to stabilize training
2026-07-23 10:32:06 +08:00
1191fc8e9f
[fix] PPO: align mb_resp_logp with old_resp_logp under autocast
...
Move log_softmax for mb_resp_logp inside autocast_ctx so it is upcast
to fp32, matching the rollout path (compute_per_token_logps runs under
no_grad+autocast). Previously log_softmax ran outside autocast on
fp16/bf16 logits, introducing a ~1e-2 systematic bias between
mb_resp_logp and old_resp_logp, so ratio != 1 and KL != 0 even before
any weight update. Under sparse rule-based rewards (e.g. math PPO)
this bias destabilises training.
Also add:
- --rollout_eval_mode (default 1): switch actor to eval during rollout
to disable dropout and keep old/new logp consistent.
- --debug_log_ratio: optional diagnostic printing |mb-old| on the first
minibatch to verify ratio ~= 1.
2026-07-21 18:51:42 +08:00
1e0981d1e3
feat: make code generation runs resumable
2026-07-14 20:27:11 +08:00
cf2329fab0
feat: add reproducible MBPP code evaluation
2026-07-14 18:42:22 +08:00
8cbbda005c
fix: tighten runtime and GRPO metrics
2026-07-14 18:41:40 +08:00
6a98206aef
docs: keep repository documentation project-only
2026-07-13 19:19:38 +08:00
4ec1b84d96
docs: document code agent and neutralize identity
2026-07-12 17:56:29 +08:00
65def67cee
feat: add code agent model service runner
2026-07-12 17:53:02 +08:00
125b49dfc1
feat: integrate verifiable rewards into GRPO
2026-07-12 17:47:36 +08:00
bfb5a1261b
feat: add execution-feedback repair agent
2026-07-12 17:42:52 +08:00
07643d5f05
feat: add pass-at-k code evaluation
2026-07-12 17:38:20 +08:00
8e06fb6d52
feat: add code execution verifier and rewards
2026-07-12 17:29:07 +08:00
512eed0b65
[fix] robustness
2026-06-28 18:03:34 +08:00
3f1a7cc25b
[fix] ddp exit hang
2026-06-01 17:50:39 +08:00
4a68da72d5
[fix] lora dims
2026-05-31 13:38:51 +08:00
4497610ec0
[fix] issue#771
2026-05-19 17:40:03 +08:00
9da8e1ab18
[update] requirements
2026-05-16 18:08:37 +08:00
dddedc6881
[fix] repetition_penalty
2026-05-07 19:08:52 +08:00
802c15b2b4
[feat] reduce RL memory
2026-05-06 15:07:28 +08:00
e73a407f7a
Merge pull request #759 from TKiteRunner/fix/grpo-oom-reorder-rewards
...
fix: resolve CUDA OOM in train_grpo.py on GPUs with <=8GB VRAM
2026-05-06 15:00:17 +08:00
10776417aa
fix: resolve CUDA OOM in train_grpo.py on GPUs with <=8GB VRAM
2026-05-06 14:14:15 +08:00
5020dc9dd4
[update] readme
2026-05-06 13:41:46 +08:00
bdee223036
[fix] inference bug
2026-05-03 20:48:48 +08:00
06d882e4ef
[update] readme
2026-05-01 12:06:47 +08:00
da865af63d
[update] readme
2026-04-28 17:22:25 +08:00
773e451b11
[fix] bugs
2026-04-27 19:16:08 +08:00
6361510016
[fix] rollout bugs
2026-04-27 17:54:09 +08:00
d4c6bc5c7e
[update] readme
2026-04-27 10:59:41 +08:00
24896cd2c4
[update] readme
2026-04-24 20:16:06 +08:00
e2fc397176
[update] readme
2026-04-24 20:14:10 +08:00
693fb1ccf1
[update] readme
2026-04-21 14:34:46 +08:00
5416a44471
[fix] bugs
2026-04-21 13:03:34 +08:00
1718e9a44d
[fix] transformers-5.x
2026-04-19 23:48:54 +08:00
5704766352
[update] tie embedding
2026-04-19 21:57:28 +08:00
1ea113ea2c
[update] readme
2026-04-19 14:51:47 +08:00
487f78754d
[update] readme
2026-04-10 10:55:24 +08:00
e796b8028a
[fix] lora compile
2026-04-10 00:01:21 +08:00
b2488e6440
[update] readme
2026-04-09 19:00:41 +08:00
939dc8ff42
[update] readme
2026-04-09 18:54:17 +08:00
cadacabecb
[update] bench
2026-04-09 17:12:48 +08:00
5351424bf0
[fix] lora moe
2026-04-09 16:36:48 +08:00
aa3e6affa1
[update] add a comment
2026-04-09 15:09:58 +08:00
25a7edcd6f
[update] image
2026-04-04 14:54:13 +08:00
cacf1d4cd0
[update] readme
2026-04-04 11:25:21 +08:00
2ab6455d9d
[update] open causal
2026-04-02 15:28:58 +08:00
RileyChen23
jingyaogong
FloatFrank
TKiteRunner
jingyaogong