github
/
minimind
mirror of https://github.com/jingyaogong/minimind.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
370 Commits
2 Branches
2 Tags
417 MiB
Commit Graph

370 Commits (5012b232ff535d5c9917ef2f06eab38927cc0a3a)

Author SHA1 Message Date
baihaochen 5012b232ff fix: harden DeepSpeed SFT resume 2026-06-11 21:45:31 +08:00
baihaochen eaf3a8cbec feat: add pretrain evaluation scripts 2026-06-07 19:46:48 +08:00
baihaochen 4dcda45bfa fix: recover DeepSpeed resume step from checkpoint tag 2026-06-04 14:20:21 +08:00
baihaochen e9c93951cc fix: keep only latest DeepSpeed checkpoint 2026-06-03 21:35:47 +08:00
baihaochen 0425c4f622 fix: make DeepSpeed resume checkpoints robust 2026-06-02 20:41:38 +08:00
baihaochen 33233fed6e feat: add SearchShortQA DeepSpeed pipeline 2026-06-02 00:59:46 +08:00
baihaochen 2201060169 fix: switch DeepSpeed config from bf16 to fp16, ZeRO-2 to ZeRO-1 (RTX 3090 compat) 2026-05-23 16:45:56 +08:00
baihaochen 46dabcfc19 chore: remove unused import 2026-05-23 16:42:02 +08:00
baihaochen 907f31f357 chore: add deepspeed suffix to save_weight, update default data_path 2026-05-23 16:38:49 +08:00
baihaochen 4f5cdfe59a fix: avoid double init_process_group in DeepSpeed pretrain 2026-05-23 16:21:57 +08:00
baihaochen ed006a254f [feat] add DeepSpeed pretrain (ZeRO-2, BF16, auto LR, gradient accumulation) 2026-05-22 00:27:06 +08:00
baihaochen 579cc79cd8 fix: extract tools and tool_calls before apply_chat_template in PPL eval 2026-05-16 11:33:55 +08:00
baihaochen 15645bbf56 [feat] add GQA vs MLA evaluation script (PPL + Q&A) 2026-05-16 11:27:49 +08:00
baihaochen 09e491f940 Merge branch 'master' of github.com:daiweidaichenzi/minimind 2026-05-14 14:29:06 +08:00
baihaochen 8baa9c5f71 fix: add missing --use_mla to train_full_sft.py 2026-05-14 14:28:11 +08:00
rainpeach-m de683eafae merge local changes 2026-05-12 11:18:45 +00:00
baihaochen 49a05a1307 增加mla注意力模型 2026-05-12 18:57:53 +08:00
baihaochen bc3d137248 [feat] add MLA with decoupled RoPE (DeepSeek-V2 style) 
- Add model_minimind_mla.py with Q/KV low-rank compression and decoupled RoPE
- Add kv_norm on kv_latent, q_compress + q_up + q_rope_proj for Q path
- Cache format: (kv_latent, k_rope) tuple, 176 floats/tok/layer (4.4x compression)
- Add --use_mla flag to all training scripts and utils
- Add benchmark_gqa_vs_mla.py script
 2026-05-12 18:38:47 +08:00
baihaochen 9c23a92922 add Chinese annotations pretrain 2026-05-08 20:44:02 +08:00
jingyaogong dddedc6881 [fix] repetition_penalty 2026-05-07 19:08:52 +08:00
jingyaogong 802c15b2b4 [feat] reduce RL memory 2026-05-06 15:07:28 +08:00
jingyaogong e73a407f7a
Merge pull request #759 from TKiteRunner/fix/grpo-oom-reorder-rewards 
fix: resolve CUDA OOM in train_grpo.py on GPUs with <=8GB VRAM
 2026-05-06 15:00:17 +08:00
TKiteRunner 10776417aa fix: resolve CUDA OOM in train_grpo.py on GPUs with <=8GB VRAM 2026-05-06 14:14:15 +08:00
jingyaogong 5020dc9dd4 [update] readme 2026-05-06 13:41:46 +08:00
jingyaogong bdee223036 [fix] inference bug 2026-05-03 20:48:48 +08:00
jingyaogong 06d882e4ef [update] readme 2026-05-01 12:06:47 +08:00
jingyaogong da865af63d [update] readme 2026-04-28 17:22:25 +08:00
jingyaogong 773e451b11 [fix] bugs 2026-04-27 19:16:08 +08:00
jingyaogong 6361510016 [fix] rollout bugs 2026-04-27 17:54:09 +08:00
jingyaogong d4c6bc5c7e [update] readme 2026-04-27 10:59:41 +08:00
jingyaogong 24896cd2c4 [update] readme 2026-04-24 20:16:06 +08:00
jingyaogong e2fc397176 [update] readme 2026-04-24 20:14:10 +08:00
jingyaogong 693fb1ccf1 [update] readme 2026-04-21 14:34:46 +08:00
jingyaogong 5416a44471 [fix] bugs 2026-04-21 13:03:34 +08:00
jingyaogong 1718e9a44d [fix] transformers-5.x 2026-04-19 23:48:54 +08:00
jingyaogong 5704766352 [update] tie embedding 2026-04-19 21:57:28 +08:00
jingyaogong 1ea113ea2c [update] readme 2026-04-19 14:51:47 +08:00
jingyaogong 487f78754d [update] readme 2026-04-10 10:55:24 +08:00
jingyaogong e796b8028a [fix] lora compile 2026-04-10 00:01:21 +08:00
jingyaogong b2488e6440 [update] readme 2026-04-09 19:00:41 +08:00
jingyaogong 939dc8ff42 [update] readme 2026-04-09 18:54:17 +08:00
jingyaogong cadacabecb [update] bench 2026-04-09 17:12:48 +08:00
jingyaogong 5351424bf0 [fix] lora moe 2026-04-09 16:36:48 +08:00
jingyaogong aa3e6affa1 [update] add a comment 2026-04-09 15:09:58 +08:00
jingyaogong 25a7edcd6f [update] image 2026-04-04 14:54:13 +08:00
jingyaogong cacf1d4cd0 [update] readme 2026-04-04 11:25:21 +08:00
jingyaogong 2ab6455d9d [update] open causal 2026-04-02 15:28:58 +08:00
jingyaogong 9348fde743 [update] readme 2026-04-02 15:28:29 +08:00
jingyaogong 90cd275524 [update] readme 2026-04-01 14:00:21 +08:00
jingyaogong b7e0ae21d6 [update] default model 2026-03-31 13:40:16 +08:00