diff --git a/docs/index.en.md b/docs/index.en.md
deleted file mode 100644
index f4a60b3..0000000
--- a/docs/index.en.md
+++ /dev/null
@@ -1,53 +0,0 @@
-# Welcome to MiniMind!
-
-
- 
- "Simplicity is the ultimate sophistication"
-
-
-## 📌 Introduction
-
-MiniMind is a super-small language model project trained completely from scratch, requiring **only $0.5 + 2 hours** to train a **26M** language model!
-
-- **MiniMind** series is extremely lightweight, the smallest version is **1/7000** the size of GPT-3
-- The project open-sources the minimalist structure of large models, including:
- - Mixture of Experts (MoE)
- - Dataset cleaning
- - Pretraining
- - Supervised Fine-Tuning (SFT)
- - LoRA fine-tuning
- - Direct Preference Optimization (DPO)
- - Model distillation
-- All core algorithm code is reconstructed from scratch using native PyTorch, without relying on third-party abstract interfaces
-- This is not only a full-stage open-source reproduction of large language models, but also a tutorial for getting started with LLMs
-
-!!! note "Training Cost"
- "2 hours" is based on NVIDIA 3090 hardware (single card) testing, "$0.5" refers to GPU server rental cost
-
-## ✨ Key Features
-
-- **Ultra-low cost**: Single 3090, 2 hours, $0.5 to train a ChatBot from scratch
-- **Complete pipeline**: Covers Tokenizer, pretraining, SFT, LoRA, DPO, distillation full process
-- **Education-friendly**: Clean code, suitable for learning LLM principles
-- **Ecosystem compatible**: Supports `transformers`, `llama.cpp`, `vllm`, `ollama` and other mainstream frameworks
-
-## 📊 Model List
-
-| Model (Size) | Inference Memory (Approx.) | Release |
-|------------|----------|---------|
-| MiniMind2-small (26M) | 0.5 GB | 2025.04.26 |
-| MiniMind2-MoE (145M) | 1.0 GB | 2025.04.26 |
-| MiniMind2 (104M) | 1.0 GB | 2025.04.26 |
-
-## 🚀 Quick Navigation
-
-- [Quick Start](quickstart.en.md) - Environment setup, model download, quick testing
-- [Model Training](training.en.md) - Pretraining, SFT, LoRA, DPO training process
-
-## 🔗 Related Links
-
-- **GitHub**: [https://github.com/jingyaogong/minimind](https://github.com/jingyaogong/minimind)
-- **HuggingFace**: [MiniMind Collection](https://huggingface.co/collections/jingyaogong/minimind-66caf8d999f5c7fa64f399e5)
-- **ModelScope**: [MiniMind Models](https://www.modelscope.cn/profile/gongjy)
-- **Online Demo**: [ModelScope Studio](https://www.modelscope.cn/studios/gongjy/MiniMind)
-
diff --git a/docs/index.md b/docs/index.md
index 3fbeef1..9e41e7d 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -1,53 +1,53 @@
# Welcome to MiniMind!
-

+ "Simplicity is the ultimate sophistication"
-## 📌 项目简介
+## 📌 Introduction
-MiniMind 是一个完全从 0 开始训练的超小语言模型项目,**仅需 3 块钱成本 + 2 小时**即可训练出仅为 **26M** 的语言模型!
+MiniMind is a super-small language model project trained completely from scratch, requiring **only $0.5 + 2 hours** to train a **26M** language model!
-- **MiniMind** 系列极其轻量,最小版本体积是 GPT-3 的 **1/7000**
-- 项目开源了大模型的极简结构,包含:
- - 混合专家模型(MoE)
- - 数据集清洗
- - 预训练(Pretrain)
- - 监督微调(SFT)
- - LoRA 微调
- - 直接偏好优化(DPO)
- - 模型蒸馏
-- 所有核心算法代码均从 0 使用 PyTorch 原生重构,不依赖第三方抽象接口
-- 这不仅是大语言模型的全阶段开源复现,也是一个入门 LLM 的教程
+- **MiniMind** series is extremely lightweight, the smallest version is **1/7000** the size of GPT-3
+- The project open-sources the minimalist structure of large models, including:
+ - Mixture of Experts (MoE)
+ - Dataset cleaning
+ - Pretraining
+ - Supervised Fine-Tuning (SFT)
+ - LoRA fine-tuning
+ - Direct Preference Optimization (DPO)
+ - Model distillation
+- All core algorithm code is reconstructed from scratch using native PyTorch, without relying on third-party abstract interfaces
+- This is not only a full-stage open-source reproduction of large language models, but also a tutorial for getting started with LLMs
-!!! note "训练成本"
- "2小时" 基于 NVIDIA 3090 硬件设备(单卡)测试,"3块钱" 指 GPU 服务器租用成本
+!!! note "Training Cost"
+ "2 hours" is based on NVIDIA 3090 hardware (single card) testing, "$0.5" refers to GPU server rental cost
-## ✨ 主要特点
+## ✨ Key Features
-- **超低成本**:单卡 3090,2 小时,3 块钱即可从 0 训练 ChatBot
-- **完整流程**:涵盖 Tokenizer、预训练、SFT、LoRA、DPO、蒸馏全流程
-- **教育友好**:代码简洁,适合学习 LLM 原理
-- **生态兼容**:支持 `transformers`、`llama.cpp`、`vllm`、`ollama` 等主流框架
+- **Ultra-low cost**: Single 3090, 2 hours, $0.5 to train a ChatBot from scratch
+- **Complete pipeline**: Covers Tokenizer, pretraining, SFT, LoRA, DPO, distillation full process
+- **Education-friendly**: Clean code, suitable for learning LLM principles
+- **Ecosystem compatible**: Supports `transformers`, `llama.cpp`, `vllm`, `ollama` and other mainstream frameworks
-## 📊 模型列表
+## 📊 Model List
-| 模型 (大小) | 推理占用 (约) | Release |
+| Model (Size) | Inference Memory (Approx.) | Release |
|------------|----------|---------|
| MiniMind2-small (26M) | 0.5 GB | 2025.04.26 |
| MiniMind2-MoE (145M) | 1.0 GB | 2025.04.26 |
| MiniMind2 (104M) | 1.0 GB | 2025.04.26 |
-## 🚀 快速导航
+## 🚀 Quick Navigation
-- [快速开始](quickstart.md) - 环境安装、模型下载、快速测试
-- [模型训练](training.md) - 预训练、SFT、LoRA、DPO 等训练流程
+- [Quick Start](quickstart.md) - Environment setup, model download, quick testing
+- [Model Training](training.md) - Pretraining, SFT, LoRA, DPO training process
-## 🔗 相关链接
+## 🔗 Related Links
- **GitHub**: [https://github.com/jingyaogong/minimind](https://github.com/jingyaogong/minimind)
- **HuggingFace**: [MiniMind Collection](https://huggingface.co/collections/jingyaogong/minimind-66caf8d999f5c7fa64f399e5)
-- **ModelScope**: [MiniMind 模型](https://www.modelscope.cn/profile/gongjy)
-- **在线体验**: [ModelScope 创空间](https://www.modelscope.cn/studios/gongjy/MiniMind)
+- **ModelScope**: [MiniMind Models](https://www.modelscope.cn/profile/gongjy)
+- **Online Demo**: [ModelScope Studio](https://www.modelscope.cn/studios/gongjy/MiniMind)
diff --git a/docs/quickstart.en.md b/docs/quickstart.en.md
deleted file mode 100644
index c3e0ade..0000000
--- a/docs/quickstart.en.md
+++ /dev/null
@@ -1,114 +0,0 @@
-# Quick Start
-
-This page will help you quickly get started with the MiniMind project.
-
-## 📋 Requirements
-
-- **Python**: 3.10+
-- **PyTorch**: 1.12+
-- **CUDA**: 12.2+ (optional, for GPU acceleration)
-- **VRAM**: At least 8GB (24GB recommended)
-
-!!! tip "Hardware Configuration Reference"
- - CPU: Intel i9-10980XE @ 3.00GHz
- - RAM: 128 GB
- - GPU: NVIDIA GeForce RTX 3090 (24GB)
-
-## 🚀 Testing Existing Models
-
-### 1. Clone the Project
-
-```bash
-git clone https://github.com/jingyaogong/minimind.git
-cd minimind
-```
-
-### 2. Install Dependencies
-
-```bash
-pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
-```
-
-!!! warning "Torch CUDA Check"
- After installation, test if Torch can use CUDA:
- ```python
- import torch
- print(torch.cuda.is_available())
- ```
-
-### 3. Download Model
-
-Download pretrained models from HuggingFace or ModelScope:
-
-```bash
-# From HuggingFace
-git clone https://huggingface.co/jingyaogong/MiniMind2
-
-# Or from ModelScope
-git clone https://www.modelscope.cn/models/gongjy/MiniMind2.git
-```
-
-### 4. Command Line Q&A
-
-```bash
-# load=0: load PyTorch model, load=1: load Transformers model
-python eval_model.py --load 1 --model_mode 2
-```
-
-### 5. Start WebUI (Optional)
-
-```bash
-# Requires Python >= 3.10
-pip install streamlit
-cd scripts
-streamlit run web_demo.py
-```
-
-Visit `http://localhost:8501` to use the web interface.
-
-## 🔧 Third-party Inference Frameworks
-
-MiniMind supports multiple mainstream inference frameworks:
-
-### Ollama
-
-```bash
-ollama run jingyaogong/minimind2
-```
-
-### vLLM
-
-```bash
-vllm serve ./MiniMind2/ --served-model-name "minimind"
-```
-
-### llama.cpp
-
-```bash
-# Convert model
-python convert_hf_to_gguf.py ./MiniMind2/
-
-# Quantize model
-./build/bin/llama-quantize ./MiniMind2/MiniMind2-109M-F16.gguf ./Q4-MiniMind2.gguf Q4_K_M
-
-# Inference
-./build/bin/llama-cli -m ./Q4-MiniMind2.gguf --chat-template chatml
-```
-
-## 📝 Effect Testing
-
-```text
-👶: Hello, please introduce yourself.
-🤖️: Hello! I'm MiniMind, an AI assistant developed by Jingyao Gong.
- I interact with users through natural language processing and algorithm training.
-
-👶: What is the highest mountain in the world?
-🤖️: Mount Everest is the highest mountain in the world, located in the Himalayas,
- with an elevation of 8,848.86 meters (29,031.7 feet).
-```
-
-## 🎯 Next Steps
-
-- Check [Model Training](training.en.md) to learn how to train your own model from scratch
-- Read the source code to understand LLM implementation principles
-
diff --git a/docs/quickstart.md b/docs/quickstart.md
index a8bbf94..7eaa845 100644
--- a/docs/quickstart.md
+++ b/docs/quickstart.md
@@ -1,74 +1,74 @@
-# 快速开始
+# Quick Start
-本页面将帮助你快速上手 MiniMind 项目。
+This page will help you quickly get started with the MiniMind project.
-## 📋 环境要求
+## 📋 Requirements
- **Python**: 3.10+
- **PyTorch**: 1.12+
-- **CUDA**: 12.2+(可选,用于 GPU 加速)
-- **显存**: 至少 8GB(推荐 24GB)
+- **CUDA**: 12.2+ (optional, for GPU acceleration)
+- **VRAM**: At least 8GB (24GB recommended)
-!!! tip "硬件配置参考"
+!!! tip "Hardware Configuration Reference"
- CPU: Intel i9-10980XE @ 3.00GHz
- RAM: 128 GB
- GPU: NVIDIA GeForce RTX 3090 (24GB)
-## 🚀 测试已有模型
+## 🚀 Testing Existing Models
-### 1. 克隆项目
+### 1. Clone the Project
```bash
git clone https://github.com/jingyaogong/minimind.git
cd minimind
```
-### 2. 安装依赖
+### 2. Install Dependencies
```bash
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
```
-!!! warning "Torch CUDA 检查"
- 安装后请测试 Torch 是否可用 CUDA:
+!!! warning "Torch CUDA Check"
+ After installation, test if Torch can use CUDA:
```python
import torch
print(torch.cuda.is_available())
```
-### 3. 下载模型
+### 3. Download Model
-从 HuggingFace 或 ModelScope 下载预训练模型:
+Download pretrained models from HuggingFace or ModelScope:
```bash
-# 从 HuggingFace 下载
+# From HuggingFace
git clone https://huggingface.co/jingyaogong/MiniMind2
-# 或从 ModelScope 下载
+# Or from ModelScope
git clone https://www.modelscope.cn/models/gongjy/MiniMind2.git
```
-### 4. 命令行问答
+### 4. Command Line Q&A
```bash
-# load=0: 加载 PyTorch 模型, load=1: 加载 Transformers 模型
+# load=0: load PyTorch model, load=1: load Transformers model
python eval_model.py --load 1 --model_mode 2
```
-### 5. 启动 WebUI(可选)
+### 5. Start WebUI (Optional)
```bash
-# 需要 Python >= 3.10
+# Requires Python >= 3.10
pip install streamlit
cd scripts
streamlit run web_demo.py
```
-访问 `http://localhost:8501` 即可使用 Web 界面。
+Visit `http://localhost:8501` to use the web interface.
-## 🔧 第三方推理框架
+## 🔧 Third-party Inference Frameworks
-MiniMind 支持多种主流推理框架:
+MiniMind supports multiple mainstream inference frameworks:
### Ollama
@@ -85,30 +85,30 @@ vllm serve ./MiniMind2/ --served-model-name "minimind"
### llama.cpp
```bash
-# 转换模型
+# Convert model
python convert_hf_to_gguf.py ./MiniMind2/
-# 量化模型
+# Quantize model
./build/bin/llama-quantize ./MiniMind2/MiniMind2-109M-F16.gguf ./Q4-MiniMind2.gguf Q4_K_M
-# 推理
+# Inference
./build/bin/llama-cli -m ./Q4-MiniMind2.gguf --chat-template chatml
```
-## 📝 效果测试
+## 📝 Effect Testing
```text
-👶: 你好,请介绍一下自己。
-🤖️: 你好!我是 MiniMind,一个由 Jingyao Gong 开发的人工智能助手。
- 我通过自然语言处理和算法训练来与用户进行交互。
+👶: Hello, please introduce yourself.
+🤖️: Hello! I'm MiniMind, an AI assistant developed by Jingyao Gong.
+ I interact with users through natural language processing and algorithm training.
-👶: 世界上最高的山峰是什么?
-🤖️: 珠穆朗玛峰是世界上最高的山峰,位于喜马拉雅山脉,
- 海拔 8,848.86 米(29,031.7 英尺)。
+👶: What is the highest mountain in the world?
+🤖️: Mount Everest is the highest mountain in the world, located in the Himalayas,
+ with an elevation of 8,848.86 meters (29,031.7 feet).
```
-## 🎯 下一步
+## 🎯 Next Steps
-- 查看 [模型训练](training.md) 了解如何从 0 开始训练自己的模型
-- 阅读源码了解 LLM 的实现原理
+- Check [Model Training](training.md) to learn how to train your own model from scratch
+- Read the source code to understand LLM implementation principles
diff --git a/docs/training.en.md b/docs/training.en.md
deleted file mode 100644
index dd6624d..0000000
--- a/docs/training.en.md
+++ /dev/null
@@ -1,186 +0,0 @@
-# Model Training
-
-This page introduces how to train MiniMind language models from scratch.
-
-## 📊 Data Preparation
-
-### 1. Download Dataset
-
-Download datasets from [ModelScope](https://www.modelscope.cn/datasets/gongjy/minimind_dataset/files) or [HuggingFace](https://huggingface.co/datasets/jingyaogong/minimind_dataset).
-
-Create `./dataset` directory and place data files:
-
-```bash
-./dataset/
-├── pretrain_hq.jsonl (1.6GB, ✨Recommended)
-├── sft_mini_512.jsonl (1.2GB, ✨Recommended)
-├── sft_512.jsonl (7.5GB)
-├── sft_1024.jsonl (5.6GB)
-├── sft_2048.jsonl (9GB)
-├── dpo.jsonl (909MB)
-├── r1_mix_1024.jsonl (340MB)
-└── lora_*.jsonl
-```
-
-!!! tip "Recommended Combination"
- Fastest reproduction: `pretrain_hq.jsonl` + `sft_mini_512.jsonl`
-
- **Single 3090 only needs 2 hours + $0.5!**
-
-### 2. Data Format
-
-**Pretrain Data** (`pretrain_hq.jsonl`):
-```json
-{"text": "How to overcome procrastination? Overcoming procrastination is not easy..."}
-```
-
-**SFT Data** (`sft_*.jsonl`):
-```json
-{
- "conversations": [
- {"role": "user", "content": "Hello"},
- {"role": "assistant", "content": "Hello!"}
- ]
-}
-```
-
-## 🎯 Training Pipeline
-
-All training scripts are located in the `./trainer` directory.
-
-### 1. Pretraining
-
-The pretraining stage lets the model learn basic knowledge, the goal is to **learn word continuation**.
-
-```bash
-cd trainer
-python train_pretrain.py
-
-# Multi-GPU training
-torchrun --nproc_per_node 2 train_pretrain.py
-```
-
-Output weights: `./out/pretrain_*.pth`
-
-!!! info "Training Duration"
- - MiniMind2-Small (26M): ~1.1h (single 3090)
- - MiniMind2 (104M): ~3.9h (single 3090)
-
-### 2. Supervised Fine-Tuning (SFT)
-
-The SFT stage teaches the model conversation patterns and adapts to chat templates.
-
-```bash
-python train_full_sft.py
-
-# Multi-GPU training
-torchrun --nproc_per_node 2 train_full_sft.py
-```
-
-Output weights: `./out/full_sft_*.pth`
-
-!!! info "Training Duration"
- - MiniMind2-Small: ~1h (using sft_mini_512)
- - MiniMind2: ~3.3h (using sft_mini_512)
-
-### 3. LoRA Fine-tuning (Optional)
-
-LoRA is a parameter-efficient fine-tuning method, suitable for domain adaptation.
-
-```bash
-python train_lora.py
-```
-
-**Use Cases**:
-- Medical Q&A: use `lora_medical.jsonl`
-- Self-awareness: use `lora_identity.jsonl`
-
-Output weights: `./out/lora/lora_*.pth`
-
-### 4. DPO Reinforcement Learning (Optional)
-
-DPO is used to optimize model response quality to better align with human preferences.
-
-```bash
-python train_dpo.py
-```
-
-Output weights: `./out/rlhf_*.pth`
-
-### 5. Reasoning Model Distillation (Optional)
-
-Distill reasoning capabilities from DeepSeek-R1.
-
-```bash
-python train_distill_reason.py
-```
-
-Output weights: `./out/reason_*.pth`
-
-## 📈 Model Architecture
-
-MiniMind uses Transformer Decoder-Only architecture (similar to Llama3):
-
-
-
-### Model Parameter Configuration
-
-| Model Name | params | d_model | n_layers | kv_heads | q_heads |
-|------------|--------|---------|----------|----------|---------|
-| MiniMind2-Small | 26M | 512 | 8 | 2 | 8 |
-| MiniMind2-MoE | 145M | 640 | 8 | 2 | 8 |
-| MiniMind2 | 104M | 768 | 16 | 2 | 8 |
-
-## 🧪 Test Model
-
-```bash
-# model_mode: 0=pretrain, 1=sft, 2=rlhf, 3=reason
-python eval_model.py --model_mode 1
-
-# Test LoRA model
-python eval_model.py --lora_name 'lora_medical' --model_mode 2
-```
-
-## 🔧 Multi-GPU Training
-
-### DDP Method
-
-```bash
-torchrun --nproc_per_node N train_xxx.py
-```
-
-### DeepSpeed Method
-
-```bash
-deepspeed --master_port 29500 --num_gpus=N train_xxx.py
-```
-
-### Wandb Monitoring
-
-```bash
-# Login first
-wandb login
-
-# Enable wandb
-torchrun --nproc_per_node N train_xxx.py --use_wandb
-```
-
-## 💰 Training Cost
-
-Based on single NVIDIA 3090:
-
-| Dataset Combination | Duration | Cost | Effect |
-|-----------|------|------|------|
-| pretrain_hq + sft_mini_512 | 2.1h | ≈$0.35 | 😊😊 Basic chat |
-| Full dataset (MiniMind2-Small) | 38h | ≈$6.50 | 😊😊😊😊😊😊 Complete capabilities |
-| Full dataset (MiniMind2) | 122h | ≈$20.80 | 😊😊😊😊😊😊😊😊 Best performance |
-
-!!! success "Quick Reproduction"
- Using `pretrain_hq` + `sft_mini_512`, single 3090 only needs **2 hours + $0.5** to train a ChatBot!
-
-## 📝 Common Issues
-
-- **Out of memory**: Reduce `batch_size` or use DeepSpeed
-- **Training not converging**: Adjust learning rate or check data quality
-- **Multi-GPU training error**: Ensure all GPUs are visible and CUDA versions are consistent
-
diff --git a/docs/training.md b/docs/training.md
index c1c80d8..dd6624d 100644
--- a/docs/training.md
+++ b/docs/training.md
@@ -1,19 +1,19 @@
-# 模型训练
+# Model Training
-本页面介绍如何从 0 开始训练 MiniMind 语言模型。
+This page introduces how to train MiniMind language models from scratch.
-## 📊 数据准备
+## 📊 Data Preparation
-### 1. 下载数据集
+### 1. Download Dataset
-从 [ModelScope](https://www.modelscope.cn/datasets/gongjy/minimind_dataset/files) 或 [HuggingFace](https://huggingface.co/datasets/jingyaogong/minimind_dataset) 下载数据集。
+Download datasets from [ModelScope](https://www.modelscope.cn/datasets/gongjy/minimind_dataset/files) or [HuggingFace](https://huggingface.co/datasets/jingyaogong/minimind_dataset).
-创建 `./dataset` 目录并放入数据文件:
+Create `./dataset` directory and place data files:
```bash
./dataset/
-├── pretrain_hq.jsonl (1.6GB, ✨推荐)
-├── sft_mini_512.jsonl (1.2GB, ✨推荐)
+├── pretrain_hq.jsonl (1.6GB, ✨Recommended)
+├── sft_mini_512.jsonl (1.2GB, ✨Recommended)
├── sft_512.jsonl (7.5GB)
├── sft_1024.jsonl (5.6GB)
├── sft_2048.jsonl (9GB)
@@ -22,108 +22,108 @@
└── lora_*.jsonl
```
-!!! tip "推荐组合"
- 最快速度复现:`pretrain_hq.jsonl` + `sft_mini_512.jsonl`
+!!! tip "Recommended Combination"
+ Fastest reproduction: `pretrain_hq.jsonl` + `sft_mini_512.jsonl`
- **单卡 3090 仅需 2 小时 + 3 块钱!**
+ **Single 3090 only needs 2 hours + $0.5!**
-### 2. 数据格式
+### 2. Data Format
-**预训练数据** (`pretrain_hq.jsonl`):
+**Pretrain Data** (`pretrain_hq.jsonl`):
```json
-{"text": "如何才能摆脱拖延症?治愈拖延症并不容易..."}
+{"text": "How to overcome procrastination? Overcoming procrastination is not easy..."}
```
-**SFT 数据** (`sft_*.jsonl`):
+**SFT Data** (`sft_*.jsonl`):
```json
{
"conversations": [
- {"role": "user", "content": "你好"},
- {"role": "assistant", "content": "你好!"}
+ {"role": "user", "content": "Hello"},
+ {"role": "assistant", "content": "Hello!"}
]
}
```
-## 🎯 训练流程
+## 🎯 Training Pipeline
-所有训练脚本位于 `./trainer` 目录。
+All training scripts are located in the `./trainer` directory.
-### 1. 预训练(Pretrain)
+### 1. Pretraining
-预训练阶段让模型学习基础知识,目标是**学会词语接龙**。
+The pretraining stage lets the model learn basic knowledge, the goal is to **learn word continuation**.
```bash
cd trainer
python train_pretrain.py
-# 多卡训练
+# Multi-GPU training
torchrun --nproc_per_node 2 train_pretrain.py
```
-输出权重:`./out/pretrain_*.pth`
+Output weights: `./out/pretrain_*.pth`
-!!! info "训练时长"
- - MiniMind2-Small (26M): ~1.1h (单卡 3090)
- - MiniMind2 (104M): ~3.9h (单卡 3090)
+!!! info "Training Duration"
+ - MiniMind2-Small (26M): ~1.1h (single 3090)
+ - MiniMind2 (104M): ~3.9h (single 3090)
-### 2. 监督微调(SFT)
+### 2. Supervised Fine-Tuning (SFT)
-SFT 阶段让模型学习对话方式,适应聊天模板。
+The SFT stage teaches the model conversation patterns and adapts to chat templates.
```bash
python train_full_sft.py
-# 多卡训练
+# Multi-GPU training
torchrun --nproc_per_node 2 train_full_sft.py
```
-输出权重:`./out/full_sft_*.pth`
+Output weights: `./out/full_sft_*.pth`
-!!! info "训练时长"
- - MiniMind2-Small: ~1h (使用 sft_mini_512)
- - MiniMind2: ~3.3h (使用 sft_mini_512)
+!!! info "Training Duration"
+ - MiniMind2-Small: ~1h (using sft_mini_512)
+ - MiniMind2: ~3.3h (using sft_mini_512)
-### 3. LoRA 微调(可选)
+### 3. LoRA Fine-tuning (Optional)
-LoRA 是一种参数高效的微调方法,适合领域适配。
+LoRA is a parameter-efficient fine-tuning method, suitable for domain adaptation.
```bash
python train_lora.py
```
-**应用场景**:
-- 医疗问答:使用 `lora_medical.jsonl`
-- 自我认知:使用 `lora_identity.jsonl`
+**Use Cases**:
+- Medical Q&A: use `lora_medical.jsonl`
+- Self-awareness: use `lora_identity.jsonl`
-输出权重:`./out/lora/lora_*.pth`
+Output weights: `./out/lora/lora_*.pth`
-### 4. DPO 强化学习(可选)
+### 4. DPO Reinforcement Learning (Optional)
-DPO 用于优化模型回复质量,使其更符合人类偏好。
+DPO is used to optimize model response quality to better align with human preferences.
```bash
python train_dpo.py
```
-输出权重:`./out/rlhf_*.pth`
+Output weights: `./out/rlhf_*.pth`
-### 5. 推理模型蒸馏(可选)
+### 5. Reasoning Model Distillation (Optional)
-蒸馏 DeepSeek-R1 的推理能力。
+Distill reasoning capabilities from DeepSeek-R1.
```bash
python train_distill_reason.py
```
-输出权重:`./out/reason_*.pth`
+Output weights: `./out/reason_*.pth`
-## 📈 模型结构
+## 📈 Model Architecture
-MiniMind 使用 Transformer Decoder-Only 结构(类似 Llama3):
+MiniMind uses Transformer Decoder-Only architecture (similar to Llama3):

-### 模型参数配置
+### Model Parameter Configuration
| Model Name | params | d_model | n_layers | kv_heads | q_heads |
|------------|--------|---------|----------|----------|---------|
@@ -131,56 +131,56 @@ MiniMind 使用 Transformer Decoder-Only 结构(类似 Llama3):
| MiniMind2-MoE | 145M | 640 | 8 | 2 | 8 |
| MiniMind2 | 104M | 768 | 16 | 2 | 8 |
-## 🧪 测试模型
+## 🧪 Test Model
```bash
# model_mode: 0=pretrain, 1=sft, 2=rlhf, 3=reason
python eval_model.py --model_mode 1
-# 测试 LoRA 模型
+# Test LoRA model
python eval_model.py --lora_name 'lora_medical' --model_mode 2
```
-## 🔧 多卡训练
+## 🔧 Multi-GPU Training
-### DDP 方式
+### DDP Method
```bash
torchrun --nproc_per_node N train_xxx.py
```
-### DeepSpeed 方式
+### DeepSpeed Method
```bash
deepspeed --master_port 29500 --num_gpus=N train_xxx.py
```
-### Wandb 监控
+### Wandb Monitoring
```bash
-# 需要先登录
+# Login first
wandb login
-# 启用 wandb
+# Enable wandb
torchrun --nproc_per_node N train_xxx.py --use_wandb
```
-## 💰 训练成本
+## 💰 Training Cost
-基于单卡 NVIDIA 3090:
+Based on single NVIDIA 3090:
-| 数据集组合 | 时长 | 成本 | 效果 |
+| Dataset Combination | Duration | Cost | Effect |
|-----------|------|------|------|
-| pretrain_hq + sft_mini_512 | 2.1h | ≈2.73¥ | 😊😊 基础对话 |
-| 完整数据集 (MiniMind2-Small) | 38h | ≈49.61¥ | 😊😊😊😊😊😊 完整能力 |
-| 完整数据集 (MiniMind2) | 122h | ≈158.6¥ | 😊😊😊😊😊😊😊😊 最强性能 |
+| pretrain_hq + sft_mini_512 | 2.1h | ≈$0.35 | 😊😊 Basic chat |
+| Full dataset (MiniMind2-Small) | 38h | ≈$6.50 | 😊😊😊😊😊😊 Complete capabilities |
+| Full dataset (MiniMind2) | 122h | ≈$20.80 | 😊😊😊😊😊😊😊😊 Best performance |
-!!! success "极速复现"
- 使用 `pretrain_hq` + `sft_mini_512`,单卡 3090 仅需 **2 小时 + 3 块钱**即可训练出能对话的 ChatBot!
+!!! success "Quick Reproduction"
+ Using `pretrain_hq` + `sft_mini_512`, single 3090 only needs **2 hours + $0.5** to train a ChatBot!
-## 📝 常见问题
+## 📝 Common Issues
-- **显存不足**:减小 `batch_size` 或使用 DeepSpeed
-- **训练不收敛**:调整学习率或检查数据质量
-- **多卡训练报错**:确保所有卡都可见且 CUDA 版本一致
+- **Out of memory**: Reduce `batch_size` or use DeepSpeed
+- **Training not converging**: Adjust learning rate or check data quality
+- **Multi-GPU training error**: Ensure all GPUs are visible and CUDA versions are consistent
diff --git a/mkdocs.yml b/mkdocs.yml
index 4fab42c..7fd519c 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -3,27 +3,10 @@ site_description: MiniMind - 轻量级语言模型训练框架 / Lightweight Lan
site_author: jingyaogong
site_url: https://minimind.readthedocs.io/
-# 多语言配置
+# 搜索插件配置
plugins:
- search:
- lang:
- - zh
- - en
- - i18n:
- docs_structure: folder
- fallback_to_default: true
- reconfigure_material: true
- reconfigure_search: true
- languages:
- - locale: zh
- default: true
- name: 简体中文
- build: true
- site_name: MiniMind
- - locale: en
- name: English
- build: true
- site_name: MiniMind
+ lang: en
# 主题配置
theme:
@@ -57,13 +40,13 @@ theme:
- content.code.copy # 代码复制按钮
- toc.follow # 目录跟随
- toc.integrate # 目录集成到左侧边栏
- language: zh
+ language: en
# 导航结构
nav:
- - 首页: index.md
- - 快速开始: quickstart.md
- - 模型训练: training.md
+ - Home: index.md
+ - Quick Start: quickstart.md
+ - Model Training: training.md
# Markdown 扩展
markdown_extensions:
diff --git a/requirements.txt b/requirements.txt
index f7ff530..a007367 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,3 +1,2 @@
mkdocs>=1.5.0
mkdocs-material>=9.0.0
-mkdocs-static-i18n>=1.0.0