From c050af959ac77b1039ed9d8d48ac7e83463fa817 Mon Sep 17 00:00:00 2001 From: jingyaogong Date: Tue, 21 Oct 2025 22:22:49 +0800 Subject: [PATCH] [feat] update website --- index.html | 886 +++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 792 insertions(+), 94 deletions(-) diff --git a/index.html b/index.html index 3531468..b5d0f0e 100644 --- a/index.html +++ b/index.html @@ -3,134 +3,832 @@ - MiniMind Project - + MiniMind - Train LLMs from Scratch - -
-
- - - Visitor Badge - GitHub Stars - License - Last Commit - Pull Requests Welcome - MiniMind Collection +
+ +
+
-
-
-

"The Greatest Path is the Simplest"

- 🔗 MiniMind Github Repo 🔗 + +
+ Visitor Badge + GitHub Stars + License + Last Commit
-
+ +
+

+ Train a 26M ChatBot from zero.
+ 2 hours. ¥3. One 3090.
+ That's it. +

+
-
    -
  • The MiniMind series is extremely lightweight, with the smallest version being 1/7000 the - size of GPT-3, making it possible to train quickly on even the most ordinary personal GPUs. -
    • -
  • The project also open-sources the minimalist structure of the large model, including extensions for shared - mixed experts (MoE), dataset cleaning, pretraining, supervised fine-tuning (SFT), LoRA fine-tuning, direct - preference optimization (DPO) algorithms, and model distillation algorithms, along with the full code of the - process. -
    • -
  • MiniMind also expands into vision multimodal VLM: MiniMind-V. -
    • -
  • All core algorithm code is reconstructed from scratch using native PyTorch! It does not rely on abstract - interfaces provided by third-party libraries. -
    • -
  • This is not only a full-stage open-source reproduction of a large language model but also a tutorial for - beginners in LLM. -
    • -
  • We hope this project will serve as an inspiring example for everyone, helping to enjoy the fun of creation - and promoting the progress of the wider AI community! -
    • - -
  • To avoid misunderstanding, the "2 hours" test is based on NVIDIA 3090 hardware (single - GPU), and the "3 - RMB" refers to the GPU server rental cost. Details of the specifications can be found below. -
    • -
- - -
- -
- Streamlit Demo - LLM Structure - LLM Structure MOE + +
+
+

26M

+

Parameters

-
- 🔗🍓Reason Model | - 🔗🤖Standard Model | - 🔗🎞️Video Introduction -
-
+
+

2h

+

Training

+
+
+

¥3

+

Cost

+
+
+

1/7000

+

vs GPT-3

+
+
+ + +
✨ What You Get
+
+
+

💰 Ultra-Cheap

+

One 3090. Zero to ChatBot in 2 hours, costing just ¥3.

+
+
+

📦 Full Stack

+

Complete pipeline: Tokenizer → Pretrain → SFT → LoRA → PPO/GRPO/SPO

+
+
+

🔬 Latest RL (2025)

+

PPO, GRPO, SPO + YaRN length extrapolation. Native PyTorch implementation.

+
+
+

📖 Learn by Reading

+

Pure PyTorch. No black boxes. Understand every line of code.

+
+
+

🔌 Plug & Play

+

Compatible with vLLM, ollama, llama.cpp, transformers.

+
+
+

⚡ OpenAI API

+

Drop-in replacement for FastGPT, Open-WebUI, Dify.

+
+
+ + +
📦 Models
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModelParametersHidden DimLayersMemory
MiniMind2-Small26M5128~0.5 GB
MiniMind2104M76816~1.0 GB
MiniMind2-MoE145M6408~1.0 GB
+
+ + +
📰 What's New
+
+
+
+ 🔥 2025-10-24 (Latest) + − +
+
+
    +
  • 🔥 RLAIF algorithms: PPO, GRPO, SPO (native PyTorch)
    • +
  • YaRN algorithm for RoPE length extrapolation
    • +
  • Adaptive Thinking in reasoning models
    • +
  • Tool Calling & Reasoning tags support
    • +
  • SwanLab integration (WandB alternative for China)
    • +
  • Bug fixes and performance improvements
    • +
+
+
+ +
+
+ ⚙️ 2025-04-26 (Major Refactor) + + +
+
+
    +
  • Model parameter renaming (align with Transformers)
    • +
  • Generate method refactor (GenerationMixin)
    • +
  • ✅ llama.cpp, vllm, ollama support
    • +
  • Vocab update: <s></s> → <|im_start|><|im_end|>
    • +
  • Code structure standardization
    • +
+
+
+ +
+
+ 🎉 2025-02-09 (MiniMind2 Release) + + +
+
+
    +
  • Complete codebase rewrite
    • +
  • MiniMind2 series: 26M, 104M, 145M models
    • +
  • JSONL data format (no preprocessing needed)
    • +
  • LoRA from scratch (no peft dependency)
    • +
  • DPO native PyTorch implementation
    • +
  • White-box model distillation
    • +
  • DeepSeek-R1 distillation models
    • +
  • HQ pretraining data (2h on single 3090)
    • +
+
+
+ +
+
+ 🎬 2024-10-05 (Vision Multimodal) + + +
+
+
    +
  • Vision multimodal support
    • +
  • MiniMind-V project launch
    • +
+
+
+ +
+
+ 🔧 2024-09-27 (Data Preprocessing) + + +
+
+
    +
  • Pretrain dataset preprocessing update
    • +
  • Text integrity preservation
    • +
  • Code cleanup
    • +
+
+
+ +
+
+ 📝 2024-09-17 (MoE & Tokenizer) + + +
+
+
    +
  • MiniMind-V1-MoE model release
    • +
  • Standardized minimind_tokenizer
    • +
  • Removed mistral_tokenizer variants
    • +
+
+
+ +
+
+ 🚀 2024-09-01 (V1 Release) + + +
+
+
    +
  • MiniMind-V1 (108M) release
    • +
  • 3 epochs pretrain + 10 epochs SFT
    • +
  • ModelScope online demo
    • +
+
+
+ +
+
+ ⭐ 2024-08-27 (First Release) + + +
+
+
    +
  • 🚀 First open-source release
    • +
  • MiniMind project launched
    • +
+
+
+
+ + +
🎮 Inside MiniMind
+
+
+ Streamlit Demo + LLM Structure + LLM Structure MOE +
+
+ + +
+

🎯 Try It Online

+
+ + 🍓 Reasoning Model + + + 💬 Chat Model + + + 📹 Video Demo + +
+ +
+ +

📦 Get the Code

+
+ + GitHub + + + 🤗 HuggingFace + + + ModelScope + +
+
+ + +
+

💡 Why MiniMind?

+
    +
  • Train from zero, not just finetune
    • +
  • Pure PyTorch—no magic, no black boxes
    • +
  • Understand by building, not by reading docs
    • +
  • Works on your laptop—no cloud GPUs needed
    • +
  • 2025 RLAIF algorithms: PPO, GRPO, SPO
    • +
  • OpenAI API compatible—plug into any UI
    • +
  • Vision support via MiniMind-V
    • +
  • Code you can actually read and modify
    • +
+

+ 💭 "Building a Lego plane beats flying first class." +

+
+ + +
+

+ Timings based on single 3090. YMMV.
+ Apache-2.0 License · + GitHub +

- + + - \ No newline at end of file +