e84437a8ca
[add] Create train_gated_grpo.py
...
add gating term on grpo algorithm
2026-02-03 10:20:56 +08:00
c540ea2537
[add] created train_gated_ppo.py
...
modify ppo algorithm by adding gating term on importance ratio
2026-02-03 10:17:56 +08:00
ce9ed24dcd
[add] add CISPO (Clipped Importance Sampling Policy Optimization) algorithm
...
This the main addition reference GRPO algorithm: By changing the gradient of out-of-bounds tokens from "directly set to 0" to "bounded clipping", we ensure that high-value exploration tokens can continue to participate in parameter updates while maintaining training stability.
Actually, this change is not very well in experiment, because the ratio is almost nearly at 1, which means seldom out-of-bound.
2026-02-02 22:23:27 +08:00
7389f64dee
[add] add DAPO argorithm (Decoupled Clip and Dynamic sAmpling Policy Optimization)
...
I mainly add two points into original grpo algorithm according to this [paper](https://arxiv.org/pdf/2503.14476 ): Clip-Higher & Dynamic Sampling
2026-02-02 22:16:52 +08:00
35fe1399c8
Update README.md
2026-02-02 16:33:02 +08:00
020bd44f3f
[mod] fix spo algorithm in RLAIF part
2026-01-30 11:03:35 +08:00
a9c56b20e9
[fix] lora weight
2025-12-22 21:27:29 +08:00
048d84abc7
Merge pull request #594 from whiteswordLI/fix/lora-load-ddp-weights
...
Fix: support loading DDP-saved LoRA weights for inference
2025-12-22 21:19:16 +08:00
3a18fdd666
Fix: support loading DDP-saved LoRA weights for inference
2025-12-22 20:50:25 +08:00
fe24501602
[feat] adjust seq length
2025-12-14 20:41:58 +08:00
fa82707c9c
[feat] update readme
2025-12-11 15:45:50 +08:00
5129f0e2a2
[fix] dtype & lr
2025-12-09 13:01:38 +08:00
aa7dc0f61e
Merge pull request #571 from dyhuachi/dyhuachi-patch-1
...
[fix] Refactor get_lr function to include min_lr calculation
2025-12-09 12:59:11 +08:00
bf3878ace8
[fix] Refactor get_lr function to include min_lr calculation
...
这里的退火算法会让参数里的lr的起始值变成原来lr的1.1倍，作出如下修改
2025-12-06 17:09:51 +08:00
ecd1ae1563
[fix] reduce aux_loss_alpha
2025-12-05 23:08:29 +08:00
5e1447b913
[fix] cuda memory #559
2025-12-01 16:17:43 +08:00
151fdf7e76
[feat] update yarn
2025-12-01 16:15:05 +08:00
6b86ea399a
[feat] release memory
2025-11-27 19:39:49 +08:00
d7f4f4eab8
[fix] ppo mask
2025-11-19 23:39:02 +08:00
f5374dc87f
[fix] model attn_mask
2025-11-19 22:26:53 +08:00
a044578d73
[fix] update model
2025-11-18 13:07:20 +08:00
ce9394670b
Merge pull request #536 from yuyu5333/fix/attn_forward
...
fix: attn_forwad when is_causal=True assert attn_mask is None
2025-11-18 13:02:46 +08:00
7d02ce673c
fix: attn_forwad when is_causal=True assert attn_mask is None
2025-11-18 03:17:17 +00:00
9c98cabc9a
[fix] prompt length calculate
2025-11-15 18:25:37 +08:00
f3441b0078
Merge pull request #528 from wangzhaode/feat/add_mnn_support
...
[feat] add MNN support to README.
2025-11-10 22:46:15 +08:00
5959396096
[feat] add MNN support to README.
2025-11-10 21:59:22 +08:00
bf60bde8fb
[fix] model-name
2025-11-07 19:38:20 +08:00
81e869fc3e
[fix] harmonize template
2025-11-06 13:14:08 +08:00
509d8dacf1
[feat] clear cache
2025-11-06 13:12:28 +08:00
8a0b04ed82
[fix] harmonize template
2025-11-02 23:18:11 +08:00
0323815729
[feat] update import
2025-10-31 23:45:55 +08:00
8d71754e05
[feat] update readme
2025-10-30 23:39:25 +08:00
d8ac558ce2
[feat] update readme
2025-10-30 23:30:12 +08:00
e4807a5214
[feat] update readme
2025-10-30 23:27:15 +08:00
de23e1ea39
[feat] update datasets
2025-10-30 11:08:13 +08:00
08ce3da228
[feat] update args
2025-10-30 10:48:31 +08:00
bf123b585d
[feat] add args
2025-10-30 10:05:12 +08:00
800fed4639
[feat] update readme
2025-10-29 12:13:25 +08:00
1713c24114
[fix] model device
2025-10-29 10:36:28 +08:00
acd5925193
[feat] update trainer
2025-10-29 00:52:37 +08:00
8f7e07b8ef
[feat] update trainer
2025-10-28 23:30:10 +08:00
eb96113cd4
[feat] update readme
2025-10-27 16:31:40 +08:00
cc81925ecc
[feat] update readme
2025-10-26 18:52:50 +08:00
35bed8936d
[feat] update eval-llm
2025-10-26 18:52:01 +08:00
e8484874f5
[feat] pause-training
2025-10-26 18:49:52 +08:00
6efba3249a
[feat] update readme
2025-10-24 01:18:33 +08:00
ea2abb5fb3
[feat] update readme
2025-10-24 00:45:11 +08:00
c4e9789d7e
[feat] update requirements
2025-10-23 23:54:55 +08:00
4276f8a72f
[feat] update requirements
2025-10-23 23:50:21 +08:00
d762926f48
[feat] update readme
2025-10-23 23:23:52 +08:00
vanking
Your Name
jingyaogong
jingyaogong
whitesword
dyhuachi
yuyu5333
yanxing