init minimind-v docs

.readthedocs.yaml

@@ -0,0 +1,19 @@
+# Read the Docs 配置文件
+version: 2
+
+# 构建配置
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"
+
+# MkDocs 配置
+mkdocs:
+  configuration: mkdocs.yml
+  fail_on_warning: false
+
+# Python 依赖
+python:
+  install:
+    - requirements: requirements.txt
+

MIGRATION_SUMMARY.md

@@ -0,0 +1,76 @@
+# MiniMind-V Documentation Migration Summary
+
+## Changes Made
+
+### 1. Configuration Files
+- **mkdocs.yml**: Updated site name, description, and URL from "MiniMind" to "MiniMind-V"
+
+### 2. Images
+- **Removed**: All old MiniMind LLM-related images
+- **Added**: MiniMind-V VLM-specific images from `/images/`:
+  - logo.png
+  - minimind2-v.gif
+  - VLM-structure.png
+  - VLM-structure-moe.png
+  - llava-structure.png
+  - minimind-v-input.png
+  - pretrain_loss.png
+  - sft_loss.png
+
+### 3. Documentation Files (All in English)
+
+#### index.md
+- Changed from LLM introduction to VLM introduction
+- Updated model list to show MiniMind-V variants
+- Added VLM-specific concepts and architecture
+- Updated related links to minimind-v repository
+
+#### quickstart.md
+- Updated from LLM quick start to VLM quick start
+- Added CLIP model download instructions
+- Changed evaluation script from `eval_model.py` to `eval_vlm.py`
+- Updated model architecture section with Visual Encoder details
+- Added image-text dialogue examples
+
+#### training.md
+- Changed from LLM training to VLM training
+- Added VLM dataset formats (pretrain_vlm_data, sft_vlm_data, sft_vlm_data_multi)
+- Updated training scripts to `train_pretrain_vlm.py` and `train_sft_vlm.py`
+- Added Visual Encoder and Projection layer explanations
+- Updated training pipeline with VLM-specific steps
+- Added multi-image fine-tuning section
+
+#### README.md
+- Updated project description to MiniMind-V
+- Changed documentation URL to minimind-v.readthedocs.io
+- Updated GitHub links to minimind-v repository
+
+## Key Differences: MiniMind vs MiniMind-V
+
+| Aspect | MiniMind (LLM) | MiniMind-V (VLM) |
+|--------|----------------|------------------|
+| Type | Language Model | Vision-Language Model |
+| Input | Text only | Text + Images |
+| Components | Transformer Decoder | Transformer Decoder + Visual Encoder + Projection |
+| Training Data | Text (pretrain.jsonl, sft.jsonl) | Text + Images (pretrain_vlm_data, sft_vlm_data) |
+| Eval Script | eval_model.py | eval_vlm.py |
+| Training Scripts | train_pretrain.py, train_full_sft.py | train_pretrain_vlm.py, train_sft_vlm.py |
+| Model Weights | *.pth (LLM only) | *.pth (LLM + Visual components) |
+
+## Verification
+
+All documentation now correctly references:
+- ✅ MiniMind-V project name
+- ✅ VLM-specific architecture diagrams
+- ✅ Vision-language training pipeline
+- ✅ Correct repository URLs (minimind-v)
+- ✅ VLM evaluation and inference scripts
+- ✅ Image-text dataset formats
+- ✅ English language throughout
+
+## Next Steps
+
+1. Deploy to ReadTheDocs at minimind-v.readthedocs.io
+2. Ensure .readthedocs.yaml is properly configured
+3. Test local build with `mkdocs serve`
+4. Verify all image links work correctly

README.md

@@ -0,0 +1 @@
+### MiniMind-V Docs

docs/index.md

@@ -0,0 +1,90 @@
+# <strong>Welcome to MiniMind-V!</strong>
+
+<figure markdown>
+  ![logo](images/logo.png)
+  <figcaption><strong>"The Greatest Path is the Simplest"</strong></figcaption>
+</figure>
+
+## 📌 Introduction
+
+MiniMind-V is a super-small multimodal vision-language model project trained completely from scratch, requiring **only 1.3 RMB + 1 hour** to train a **26M** parameter vision-language model!
+
+- **MiniMind-V** series is extremely lightweight, the smallest version is **1/7000** the size of GPT-3
+- **MiniMind-V** is an extension of the visual capabilities of the [MiniMind](https://github.com/jingyaogong/minimind) pure language model
+- The project open-sources the minimalist structure of VLM models, including:
+  - Dataset cleaning
+  - Pretraining
+  - Supervised Fine-Tuning (SFT)
+  - Multi-image dialogue support
+- All core algorithm code is reconstructed from scratch using native PyTorch, without relying on third-party abstract interfaces
+- This is not only a full-stage open-source reproduction of vision-language models, but also a concise tutorial for getting started with VLMs
+
+!!! note "Training Cost"
+    "1 hour" is based on NVIDIA 3090 hardware (single card) testing `1 epoch`, "1.3 RMB" refers to GPU server rental cost
+
+## ✨ Key Features
+
+- **Ultra-low cost**: Single 3090, 1 hour, 1.3 RMB to train a vision-language model from scratch
+- **Complete pipeline**: Covers Visual Encoder, Projection, Pretraining, SFT full process
+- **Education-friendly**: Clean code, suitable for learning VLM principles
+- **Ecosystem compatible**: Supports `transformers` and mainstream inference frameworks
+
+## 📊 Model List
+
+| Model (Size)            | Inference Memory (Approx.) | Release    |
+|------------------------|----------------------------|------------|
+| MiniMind2-V (104M)     | 0.6 GB                     | 2025.02.20 |
+| MiniMind2-Small-V (26M) | 1.1 GB                    | 2025.02.20 |
+| minimind-v-v1-small (27M) | 0.6 GB                  | 2024.10.04 |
+| minimind-v-v1 (109M)   | 1.1 GB                     | 2024.10.04 |
+
+## 🚀 Quick Navigation
+
+- [Quick Start](quickstart.md) - Environment setup, model download, quick testing
+- [Model Training](training.md) - Pretraining, SFT training full process
+
+## 🔗 Related Links
+
+- **GitHub**: [https://github.com/jingyaogong/minimind-v](https://github.com/jingyaogong/minimind-v)
+- **HuggingFace**: [MiniMind-V Collection](https://huggingface.co/collections/jingyaogong/minimind-v-67000833fb60b3a2e1f3597d)
+- **ModelScope**: [MiniMind-V Models](https://www.modelscope.cn/profile/gongjy)
+- **Online Demo**: [ModelScope Studio](https://www.modelscope.cn/studios/gongjy/MiniMind-V)
+
+## 🎯 Project Highlights
+
+**"Building a plane with Legos is much more exciting than flying in first class!"**
+
+Is it really as complex as imagined to build a VLM-based multimodal large model? How is the code implementation done? Is the training process difficult?
+
+Now, let's explore the answers and feel the joy of creation together!
+
+<figure markdown>
+  ![minimind2-v](images/minimind2-v.gif)
+</figure>
+
+## 📚 Core Concepts
+
+### What is a Vision-Language Model (VLM)?
+
+A Vision-Language Model is a multimodal model that can understand both images and text simultaneously. MiniMind-V achieves this through:
+
+1. Using a **Visual Encoder** (CLIP model) to convert images into feature vectors
+2. Aligning visual and text feature spaces through a **Projection layer**
+3. Injecting image features into the language model to enable image-text understanding
+
+### MiniMind-V Design Philosophy
+
+- **Minimal modifications**: Based on the MiniMind language model, only adding Visual Encoder and Projection submodules
+- **Core algorithm changes < 50 lines**: Extremely low migration difficulty
+- **Simple yet effective**: Uses linear projection for cross-modal alignment, simple but effective
+
+## 🎓 Target Audience
+
+- Beginners who want to understand vision-language model principles
+- Researchers who want to quickly build VLM prototypes
+- Developers interested in multimodal large models
+- Teams needing low-cost training of customized vision-language models
+
+## 💡 Get Started
+
+Ready? Let's start your VLM journey from the [Quick Start](quickstart.md) page!

docs/quickstart.md

@@ -0,0 +1,186 @@
+# Quick Start
+
+This page will help you quickly get started with the MiniMind-V project.
+
+## 📋 Requirements
+
+- **Python**: 3.10+
+- **PyTorch**: 1.12+
+- **CUDA**: 12.2+ (optional, for GPU acceleration)
+- **VRAM**: At least 8GB (24GB recommended)
+
+!!! tip "Hardware Configuration Reference"
+    - CPU: Intel i9-10980XE @ 3.00GHz
+    - RAM: 128 GB
+    - GPU: NVIDIA GeForce RTX 3090 (24GB)
+    - Ubuntu: 20.04
+    - CUDA: 12.2
+    - Python: 3.10.16
+
+## 🚀 Testing Existing Models
+
+### Step 0: Preparation
+
+```bash
+# Clone the repository
+git clone https://github.com/jingyaogong/minimind-v
+cd minimind-v
+```
+
+```bash
+# Download CLIP model to ./model/vision_model directory
+git clone https://huggingface.co/openai/clip-vit-base-patch16
+# or
+git clone https://www.modelscope.cn/models/openai-mirror/clip-vit-base-patch16
+```
+
+### Step 1: Install Dependencies
+
+```bash
+pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
+```
+
+!!! warning "Torch CUDA Check"
+    After installation, test if Torch can use CUDA:
+    ```python
+    import torch
+    print(torch.cuda.is_available())
+    ```
+    If unavailable, download the whl file from [torch_stable](https://download.pytorch.org/whl/torch_stable.html) for installation.
+
+### Step 2: Download Model
+
+Download pretrained models from HuggingFace or ModelScope:
+
+```bash
+# From HuggingFace
+git clone https://huggingface.co/jingyaogong/MiniMind2-V
+
+# Or from ModelScope
+git clone https://www.modelscope.cn/models/gongjy/MiniMind2-V.git
+```
+
+### Step 3: Command Line Q&A
+
+```bash
+# load=0: load PyTorch model, load=1: load Transformers model
+python eval_vlm.py --load 1
+```
+
+### Step 4: Start WebUI (Optional)
+
+```bash
+python scripts/web_demo_vlm.py
+```
+
+Visit `http://localhost:8501` to use the web interface for image-text dialogue.
+
+## 📝 Effect Testing
+
+### Single Image Dialogue Examples
+
+**Test Image 1: City Street Scene**
+
+```text
+Q: Describe the content of this image
+A: The image shows a busy city street with tall buildings on both sides of a long road. 
+   The street is packed with cars, trucks, and buses, along with many other vehicles...
+```
+
+**Test Image 2: Panda**
+
+```text
+Q: What animal is in this image?
+A: The image shows a white brown bear sitting on the grass, next to a large bear with brown spots. 
+   This bear seems shy or playful as it lies on the grass, resting...
+```
+
+### Model Performance
+
+| Model | Parameters | Inference Speed | Image Understanding |
+|-------|-----------|-----------------|---------------------|
+| MiniMind2-V | 104M | Fast | 😊😊😊😊😊😊 |
+| MiniMind2-Small-V | 26M | Very Fast | 😊😊😊😊 |
+
+## 🔧 Loading from PyTorch Weights
+
+If you want to use native PyTorch model weights (`*.pth` files):
+
+### Download Weight Files
+
+Download the required weight files from:
+
+- [HuggingFace - MiniMind2-V-PyTorch](https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch)
+- [ModelScope - MiniMind2-V-PyTorch](https://www.modelscope.cn/models/gongjy/MiniMind2-V-PyTorch)
+
+Files needed:
+- `sft_vlm_512.pth` or `sft_vlm_768.pth` (SFT model weights)
+- Optional: `pretrain_vlm_512.pth` or `pretrain_vlm_768.pth` (pretrained model weights)
+
+### Run Testing
+
+```bash
+# model_mode=0: test pretrain model, model_mode=1: test SFT model
+python eval_vlm.py --load 0 --model_mode 1
+```
+
+## 📊 Model Architecture
+
+MiniMind-V adds Visual Encoder and Projection layers on top of the MiniMind language model:
+
+![VLM-structure](images/VLM-structure.png)
+
+### Core Components
+
+1. **Visual Encoder (CLIP)**
+   - Uses `clip-vit-base-patch16` model
+   - Input image size: 224×224
+   - Output: 196×768 dimensional visual tokens
+
+2. **Projection Layer**
+   - Simple linear transformation
+   - Aligns visual tokens to text embedding space
+
+3. **Language Model (MiniMind)**
+   - Inherits from MiniMind language model
+   - Supports text generation and dialogue
+
+### Model Parameter Configuration
+
+| Model Name | Params | d_model | n_layers | kv_heads | q_heads |
+|-----------|--------|---------|----------|----------|---------|
+| MiniMind2-Small-V | 26M | 512 | 8 | 2 | 8 |
+| MiniMind2-V | 104M | 768 | 16 | 2 | 8 |
+
+## 🎯 Next Steps
+
+- Check [Model Training](training.md) to learn how to train your own vision-language model from scratch
+- Read the source code to understand VLM implementation principles
+- Try testing with your own images
+
+## ❓ Common Issues
+
+### 1. Model fails to load?
+
+Ensure all dependency files are downloaded:
+- CLIP model weights
+- MiniMind-V model weights
+- tokenizer configuration files
+
+### 2. Out of memory?
+
+- Use the 26M Small version
+- Reduce batch size
+- Use CPU inference (slower)
+
+### 3. Poor image recognition?
+
+- Ensure image quality is clear
+- Try adjusting image size
+- Use more specific question descriptions
+
+## 🔗 Related Resources
+
+- **Online Demo**: [ModelScope Studio](https://www.modelscope.cn/studios/gongjy/MiniMind-V)
+- **Video Introduction**: [Bilibili](https://www.bilibili.com/video/BV1Sh1vYBEzY)
+- **Project Home**: [GitHub](https://github.com/jingyaogong/minimind-v)

docs/training.md

@@ -0,0 +1,383 @@
+# Model Training
+
+This page introduces how to train MiniMind-V vision-language models from scratch.
+
+## 📊 Data Preparation
+
+### 1. Download Dataset
+
+Download datasets from the following addresses:
+
+- [ModelScope - minimind-v_dataset](https://www.modelscope.cn/datasets/gongjy/minimind-v_dataset)
+- [HuggingFace - minimind-v_dataset](https://huggingface.co/datasets/jingyaogong/minimind-v_dataset)
+
+Create `./dataset` directory and place data files:
+
+```bash
+./dataset/
+├── pretrain_vlm_data.jsonl        # Pretrain data (~595K samples)
+├── sft_vlm_data.jsonl             # Single-image SFT data (~300K samples)
+├── sft_vlm_data_multi.jsonl       # Multi-image SFT data (~13.6K samples)
+├── eval_images/                   # Test images
+│   ├── 城市车水马龙-city-traffic.jpg
+│   ├── 熊猫草地-Panda-Grassland.jpg
+│   └── ...
+└── sft_multi_images_trans_image.zip  # Multi-image data images (needs extraction)
+```
+
+!!! tip "Dataset Notes"
+    - `*.jsonl` are Q&A datasets
+    - `*images` are accompanying image data, need to be extracted after download
+    - Please reserve about 5GB space for the dataset
+    - If space is insufficient, try skipping pretrain and go directly to SFT training
+
+### 2. Data Format
+
+**Pretrain Data Format** (`pretrain_vlm_data.jsonl`):
+
+```json
+{
+  "conversations": [
+    {
+      "role": "user",
+      "content": "Provide a brief description of the given image.\n<image>"
+    },
+    {
+      "role": "assistant",
+      "content": "Olive oil is a healthy ingredient for free use."
+    }
+  ],
+  "image": "GCC_train_002582585.jpg"
+}
+```
+
+**Single-Image SFT Data Format** (`sft_vlm_data.jsonl`):
+
+```json
+{
+  "conversations": [
+    {
+      "role": "user",
+      "content": "What impact does the location of the alarm clock have on sleep quality?<image>"
+    },
+    {
+      "role": "assistant",
+      "content": "Place the digital alarm clock on the nightstand..."
+    }
+  ],
+  "image": "train-00000-of-00001_image_0_0.jpg"
+}
+```
+
+**Multi-Image SFT Data Format** (`sft_vlm_data_multi.jsonl`):
+
+```json
+{
+  "conversations": [
+    {
+      "role": "user",
+      "content": "context: Source Image: <image> Target Image: <image> Instruction: What is the correct image edit instruction?<image>"
+    },
+    {
+      "role": "assistant",
+      "content": "take the people out of the back in the photo..."
+    }
+  ],
+  "image": "0.jpg, 1.jpg"
+}
+```
+
+### 3. Data Source
+
+- **Pretrain Data**: [Chinese-LLaVA-Vision](https://huggingface.co/datasets/LinkSoul/Chinese-LLaVA-Vision-Instructions)
+  - Approximately 570,000 images from CC-3M and COCO 2014
+  
+- **SFT Data**: [llava-en-zh-300k](https://huggingface.co/datasets/BUAADreamer/llava-en-zh-300k)
+  - 300K instruction fine-tuning data
+  - 150K images
+  - Translated content, better Chinese support
+
+## 🎯 Training Pipeline
+
+All training scripts are located in the `./trainer` directory.
+
+### Step 0: Prepare Base Language Model
+
+Download pure language model weights to the `./out` directory (as the base language model for training VLM):
+
+```bash
+# Download 512-dim model
+wget https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/llm_512.pth
+
+# Or download 768-dim model
+wget https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/llm_768.pth
+```
+
+### Step 1: Pretraining (Learning Image Description)
+
+The pretraining stage teaches the model general image knowledge, such as a deer is a deer, a dog is a dog.
+
+```bash
+cd trainer
+python train_pretrain_vlm.py --epochs 4
+
+# Multi-GPU training
+torchrun --nproc_per_node 2 train_pretrain_vlm.py --epochs 4
+```
+
+**Output weights**: `./out/pretrain_vlm_*.pth` (* is the model dimension, default is 512)
+
+!!! info "Training Duration"
+    - MiniMind2-Small-V (26M): ~1h (single 3090)
+    - MiniMind2-V (104M): ~3h (single 3090)
+
+**Training Strategy**:
+- Freeze Visual Encoder (CLIP model) gradients
+- Only train the last layer parameters of Projection and LLM
+
+**Loss Curve**:
+
+![Pretrain Loss](images/pretrain_loss.png)
+
+### Step 2: Supervised Fine-Tuning (Learning Image-Caption Dialogue Style)
+
+The SFT stage teaches the model real image-text dialogue format, better aligning with human communication habits.
+
+```bash
+python train_sft_vlm.py --epochs 4
+
+# Multi-GPU training
+torchrun --nproc_per_node 2 train_sft_vlm.py --epochs 4
+```
+
+**Output weights**: `./out/sft_vlm_*.pth`
+
+!!! info "Training Duration"
+    - MiniMind2-Small-V: ~1h (single 3090)
+    - MiniMind2-V: ~3h (single 3090)
+
+**Training Strategy**:
+- Freeze Visual Encoder (CLIP model) gradients
+- Set all parameters of Projection and LLM as learnable
+
+**Loss Curve**:
+
+![SFT Loss](images/sft_loss.png)
+
+### Step 3 (Optional): Multi-Image Fine-Tuning
+
+Multi-image fine-tuning provides a demo example based on bird comparison dataset.
+
+```bash
+python train_sft_vlm.py --epochs 4 --use_multi_image
+```
+
+**Notes**:
+- Multi-image dataset is relatively small and contains English conversations
+- Only includes two-image comparison scenarios
+- Fine-tuning effect is limited, provided as a reference approach
+
+!!! warning "Training Notes"
+    By default, training process saves parameters every 100 steps to `./out/***.pth` (will overwrite old weight files)
+
+## 📈 Model Architecture Details
+
+MiniMind-V's structure only adds Visual Encoder and feature projection submodules:
+
+![VLM Structure](images/VLM-structure.png)
+
+### Core Components
+
+1. **Visual Encoder**
+   - Uses [clip-vit-base-patch16](https://huggingface.co/openai/clip-vit-base-patch16)
+   - Based on ViT-B/16 architecture
+   - Input image: 224×224
+   - Patch size: 16×16
+   - Output: 196×768 dimensional features (196 visual tokens)
+
+2. **Projection Layer**
+   - Simple unbiased linear transformation
+   - Functions:
+     - Aligns 768-dimensional visual tokens to LLM text tokens
+     - Maps image features to the same space as text embeddings
+   - Achieves cross-modal feature alignment
+
+3. **Language Model**
+   - Fully inherits from MiniMind
+   - Minimal modifications (core algorithm changes < 50 lines)
+
+### Input-Output Mechanism
+
+**Input Format**:
+
+In `minimind-v`, a 196-character `@@@...@@@` placeholder is used to replace the image:
+
+```text
+@@@......@@@\nWhat is this image describing?
+```
+
+Why 196 characters? Because any image is encoded by the CLIP model as 196×768 dimensional tokens.
+
+![Input Mechanism](images/minimind-v-input.png)
+
+**Multi-Image Implementation**:
+
+Achieved by injecting multiple `<image>` placeholders, no framework modification needed.
+
+### Model Parameter Configuration
+
+| Model Name | Params | d_model | n_layers | kv_heads | q_heads | Visual Token |
+|-----------|--------|---------|----------|----------|---------|--------------|
+| MiniMind2-Small-V | 26M | 512 | 8 | 2 | 8 | 196×768 |
+| MiniMind2-V | 104M | 768 | 16 | 2 | 8 | 196×768 |
+
+## 🧪 Test Model
+
+### Test Trained Model
+
+Ensure the model `*.pth` file to be tested is in the `./out/` directory.
+
+```bash
+# model_mode=0: test pretrain model
+python eval_vlm.py --model_mode 0
+
+# model_mode=1: test SFT model
+python eval_vlm.py --model_mode 1
+```
+
+### Use Pre-Trained Model
+
+You can also directly download and use pre-trained `*.pth` files:
+
+- [HuggingFace - MiniMind2-V-PyTorch](https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch)
+- [ModelScope - MiniMind2-V-PyTorch](https://www.modelscope.cn/models/gongjy/MiniMind2-V-PyTorch)
+
+## 🔧 Multi-GPU Training
+
+### DDP Method
+
+```bash
+torchrun --nproc_per_node N train_xxx.py
+```
+
+### DeepSpeed Method
+
+```bash
+deepspeed --master_port 29500 --num_gpus=N train_xxx.py
+```
+
+### Wandb Monitoring
+
+```bash
+# Login first
+wandb login
+
+# Enable wandb
+torchrun --nproc_per_node N train_xxx.py --use_wandb
+```
+
+By adding the `--use_wandb` parameter, you can log the training process. After training, you can view the process on the wandb website.
+
+You can specify the project name and run name by modifying `wandb_project` and `wandb_run_name` parameters.
+
+## 💰 Training Cost
+
+Based on single NVIDIA 3090:
+
+| Dataset Combination | Training Time | Cost (Approx.) | Effect |
+|---------------------|---------------|----------------|---------|
+| pretrain (1 epoch) + sft (1 epoch) | 2h | ≈1.3 RMB | 😊😊😊 Basic dialogue |
+| pretrain (4 epochs) + sft (4 epochs) | 8h | ≈5.4 RMB | 😊😊😊😊😊 Better effect |
+
+!!! success "Quick Reproduction"
+    Using single epoch of pretrain and SFT, single 3090 only needs **2 hours + 1.3 RMB** to train a vision-language ChatBot!
+
+## 📝 Training Tips
+
+### 1. Reduce Memory Usage
+
+- Use smaller batch size
+- Use 512-dim model instead of 768
+- Enable gradient checkpointing (if implemented)
+- Use DeepSpeed ZeRO optimization
+
+### 2. Accelerate Training
+
+- Use multi-GPU training (DDP or DeepSpeed)
+- Use mixed precision training (FP16)
+- Reduce image resolution (but may affect performance)
+
+### 3. Improve Performance
+
+- Increase training epochs
+- Use larger model (768 vs 512)
+- Use higher quality datasets
+- Adjust learning rate
+
+## 🎓 Core Principles
+
+### Why Pretraining?
+
+Pretraining teaches the model basic image description capabilities, establishing fundamental mapping relationships between image features and text.
+
+### Why SFT?
+
+SFT teaches the model real dialogue formats, making its outputs more aligned with human communication habits rather than simple image descriptions.
+
+### Why Freeze CLIP?
+
+CLIP is already a powerful pre-trained visual encoder. Freezing its parameters can:
+- Significantly reduce trainable parameters
+- Speed up training
+- Prevent overfitting
+- Lower training costs
+
+### Future Improvement Directions
+
+```text
+> More complex Projection layers (e.g., Cross-Attention) may bring better cross-modal alignment
+> Use larger CLIP models (e.g., large series) for finer-grained image features
+> Increase image resolution (currently only 224×224, dataset uses 128×128)
+> Expand multi-image datasets to support more complex multi-image understanding scenarios
+```
+
+## 📚 Related Resources
+
+- **Base Language Model**: [MiniMind](https://github.com/jingyaogong/minimind)
+- **Reference Paper**: [LlaVA](https://arxiv.org/pdf/2304.08485)
+- **Visual Encoder**: [CLIP](https://huggingface.co/openai/clip-vit-base-patch16)
+
+## ❓ Common Issues
+
+### 1. Out of memory during training?
+
+- Reduce batch_size
+- Use 512-dim model
+- Use DeepSpeed ZeRO
+- Single-GPU training instead of multi-GPU
+
+### 2. Training loss not decreasing?
+
+- Check if dataset path is correct
+- Check if learning rate is appropriate
+- Confirm base LLM model is loaded correctly
+- Check for gradient explosion/vanishing
+
+### 3. Multi-GPU training error?
+
+- Ensure all GPUs are visible and CUDA versions are consistent
+- Check if port is occupied (modify `--master_port`)
+- Try using DeepSpeed instead of DDP
+
+### 4. How to use custom dataset?
+
+Prepare jsonl files and corresponding images according to the data format above, and modify the data path in the training script.
+
+## 🎯 Next Steps
+
+Congratulations! You've learned the complete training process of MiniMind-V. Now you can:
+
+- Start training your own vision-language model
+- Try using different datasets
+- Explore the source code implementation
+- Contribute your improvements to the project

mkdocs.yml

@@ -0,0 +1,66 @@
+site_name: MiniMind-V
+site_description: MiniMind-V - 超小多模态视觉语言模型 / Super-Small Multimodal Vision-Language Model
+site_author: jingyaogong
+site_url: https://minimind-v.readthedocs.io/
+
+# 搜索插件配置
+plugins:
+  - search:
+      lang: en
+
+# 主题配置
+theme:
+  name: material
+  favicon: images/logo.png
+  icon:
+    logo: material/book-open-page-variant
+  palette:
+    # 浅色模式
+    - scheme: default
+      primary: white
+      accent: blue
+      toggle:
+        icon: material/brightness-7
+        name: 切换至深色模式
+    # 深色模式
+    - scheme: slate
+      primary: black
+      accent: blue
+      toggle:
+        icon: material/brightness-4
+        name: 切换至浅色模式
+  features:
+    # - navigation.instant    # 与多语言切换不兼容，已禁用
+    - navigation.tracking     # 锚点跟踪
+    - navigation.sections     # 导航分组
+    - navigation.expand       # 默认展开导航
+    - navigation.top          # 返回顶部按钮
+    - search.suggest          # 搜索建议
+    - search.highlight        # 搜索高亮
+    - content.code.copy       # 代码复制按钮
+    - toc.follow              # 目录跟随
+    - toc.integrate           # 目录集成到左侧边栏
+  language: en
+
+# 导航结构
+nav:
+  - Home: index.md
+  - Quick Start: quickstart.md
+  - Model Training: training.md
+
+# Markdown 扩展
+markdown_extensions:
+  - toc:
+      permalink: true
+  - admonition
+  - pymdownx.highlight:
+      anchor_linenums: true
+  - pymdownx.inlinehilite
+  - pymdownx.snippets
+  - pymdownx.superfences
+  - pymdownx.details
+  - pymdownx.tabbed:
+      alternate_style: true
+  - attr_list
+  - md_in_html
+

requirements.txt

@@ -0,0 +1,2 @@
+mkdocs>=1.5.0
+mkdocs-material>=9.0.0