github
/
minimind
mirror of https://github.com/jingyaogong/minimind.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
minimind/trainer
History
baihaochen bc3d137248 [feat] add MLA with decoupled RoPE (DeepSeek-V2 style) 
- Add model_minimind_mla.py with Q/KV low-rank compression and decoupled RoPE
- Add kv_norm on kv_latent, q_compress + q_up + q_rope_proj for Q path
- Cache format: (kv_latent, k_rope) tuple, 176 floats/tok/layer (4.4x compression)
- Add --use_mla flag to all training scripts and utils
- Add benchmark_gqa_vs_mla.py script
 		2026-05-12 18:38:47 +08:00
..
rollout_engine.py [fix] inference bug 2026-05-03 20:48:48 +08:00
train_agent.py [feat] add MLA with decoupled RoPE (DeepSeek-V2 style) 2026-05-12 18:38:47 +08:00
train_distillation.py [feat] add MLA with decoupled RoPE (DeepSeek-V2 style) 2026-05-12 18:38:47 +08:00
train_dpo.py [feat] add MLA with decoupled RoPE (DeepSeek-V2 style) 2026-05-12 18:38:47 +08:00
train_full_sft.py [feat] add MLA with decoupled RoPE (DeepSeek-V2 style) 2026-05-12 18:38:47 +08:00
train_grpo.py [feat] add MLA with decoupled RoPE (DeepSeek-V2 style) 2026-05-12 18:38:47 +08:00
train_lora.py [feat] add MLA with decoupled RoPE (DeepSeek-V2 style) 2026-05-12 18:38:47 +08:00
train_ppo.py [feat] add MLA with decoupled RoPE (DeepSeek-V2 style) 2026-05-12 18:38:47 +08:00
train_pretrain.py [feat] add MLA with decoupled RoPE (DeepSeek-V2 style) 2026-05-12 18:38:47 +08:00
train_tokenizer.py add Chinese annotations pretrain 2026-05-08 20:44:02 +08:00
trainer_utils.py [feat] add MLA with decoupled RoPE (DeepSeek-V2 style) 2026-05-12 18:38:47 +08:00