🚀 「大模型」2小时完全从0训练64M的小参数LLM！🌏 Train a 64M-parameter LLM from scratch in just 2h!

artificial-intelligencelarge-language-model

skaura 126f6b036e 123 ... 123		2025-04-17 22:04:51 +08:00
model	rmsnorm float convert	2025-04-01 16:03:44 +08:00
scripts	update web_demo	2025-02-14 19:38:55 +08:00
.gitignore	update track	2025-02-10 00:32:33 +08:00
LICENSE	Create LICENSE	2024-08-28 16:56:11 +08:00
eval_model.py	update tokenizer load	2025-02-19 23:24:29 +08:00
requirements.txt	update readme	2025-02-23 20:07:26 +08:00
train_distill_reason.py	fix bugs	2025-02-13 20:56:14 +08:00
train_distillation.py	add minimind2	2025-02-09 23:49:47 +08:00
train_dpo.py	update dpo_loss	2025-04-01 17:32:50 +08:00
train_full_sft.py	update lr	2025-02-11 23:53:48 +08:00
train_lora.py	add minimind2	2025-02-09 23:49:47 +08:00
train_pretrain.py	update lr	2025-02-11 23:52:40 +08:00