diff --git a/README.md b/README.md index bfa0614..3fcf61f 100644 --- a/README.md +++ b/README.md @@ -1659,9 +1659,9 @@ MiniMind模型本身预训练数据集小的可怜，也没有针对性的对测 ## 模型转换 -* [./scripts/convert_model.py](./scripts/convert_model.py)可以实现`torch模型/transformers`模型之间的转换 +* [./scripts/convert_model.py](./scripts/convert_model.py)可以实现`torch / transformers`模型的互相转换 +* 如无特别说明，`MiniMind2`模型均默认为`Transformers`格式的模型，需提前`t2t`转换！ ---- ## 基于MiniMind-API服务接口 @@ -1670,13 +1670,12 @@ MiniMind模型本身预训练数据集小的可怜，也没有针对性的对测 * 从[Huggingface](https://huggingface.co/collections/jingyaogong/minimind-66caf8d999f5c7fa64f399e5)下载模型权重文件，文件树： ``` - (root dir) - ├─ + minimind (root dir) + ├─（例如MiniMind2） | ├── config.json | ├── generation_config.json - | ├── LMConfig.py - | ├── model.py - | ├── pytorch_model.bin + | ├── model_minimind.py or w/o + | ├── pytorch_model.bin or model.safetensors | ├── special_tokens_map.json | ├── tokenizer_config.json | ├── tokenizer.json @@ -1709,6 +1708,8 @@ MiniMind模型本身预训练数据集小的可怜，也没有针对性的对测 vLLM是极其流行的高效推理框架，支持大模型快速部署，优化显存利用与吞吐量。 +以openai-serve形式启动 minimind2： + ```bash vllm serve ./MiniMind2 --model-impl transformers --served-model-name "minimind" --port 8998 ``` @@ -1718,59 +1719,106 @@ vllm serve ./MiniMind2 --model-impl transformers --served-model-name "minimind" llama.cpp是一个C++库， 可以在命令行下直接使用，支持多线程推理，支持GPU加速。 -参考官方仓库安装后，在`convert_hf_to_gguf.py` ～760行插入 +**目录结构**：建议将llama.cpp与minimind放在同级目录下 -```text -# 添加MiniMind2 tokenizer支持 +``` +parent/ +├── minimind/ # MiniMind项目目录 +│ ├── MiniMind2/ # HuggingFace格式MiniMind2模型 (先convert_model.py生成) +│ │ ├── config.json +│ │ ├── model.safetensors +│ │ └── ... +│ ├── model/ +│ ├── trainer/ +│ └── ... +└── llama.cpp/ # llama.cpp项目目录 + ├── build/ + ├── convert_hf_to_gguf.py + └── ... +``` + +0、参考`llama.cpp`官方步骤进行install + +1、在`convert_hf_to_gguf.py`的`get_vocab_base_pre`函数最后插入： + +```python +# 添加MiniMind tokenizer支持（这里随便写一个例如qwen2即可） if res is None: - res = "smollm" + res = "qwen2" ``` -转换自定义训练的minimind模型 -> gguf +2、转换自训练的minimind模型：huggingface -> gguf ```bash -python convert_hf_to_gguf.py ../minimind/MiniMind2/ +# 在llama.cpp下执行，将生成../minimind/MiniMind2/MiniMind2-xxx.gguf +python convert_hf_to_gguf.py ../minimind/MiniMind2 ``` -量化模型 +3、量化此模型 (可选) ```bash -./build/bin/llama-quantize ../minimind/MiniMind2/MiniMind2-109M-F16.gguf ../minimind/MiniMind2/Q4-MiniMind2.gguf Q4_K_M +./build/bin/llama-quantize ../minimind/MiniMind2/MiniMind2.gguf ../minimind/MiniMind2/Q4-MiniMind2.gguf Q4_K_M ``` -命令行推理 +4、命令行推理测试 ```bash -./build/bin/llama-cli -m ../minimind/MiniMind2/MiniMind2-109M-F16.gguf --chat-template chatml +./build/bin/llama-cli -m ../minimind/MiniMind2/MiniMind2.gguf -sys "You are a helpful assistant" # system prompt必须固定 ``` ## [ollama](https://ollama.ai) ollama是本地运行大模型的工具，支持多种开源LLM，简单易用。 -通过ollama加载自定义的gguf模型，新建minimind.modelfile： +1、通过ollama加载自定义的gguf模型 + +在`MiniMind2`下新建`minimind.modelfile`，写入： ```text -FROM ./MiniMind2-109M-F16.gguf -TEMPLATE """{{ if .System }}<|im_start|>system +FROM ./Q4-MiniMind2.gguf + +SYSTEM """You are a helpful assistant""" + +TEMPLATE """<|im_start|>system {{ .System }}<|im_end|> -{{ end }}{{ if .Prompt }}<|im_start|>user +<|im_start|>user {{ .Prompt }}<|im_end|> -{{ end }}<|im_start|>assistant +<|im_start|>assistant +{{ .Response }}<|im_end|> """ ``` -加载模型并命名为`minimind2` +2、加载并命名此模型为`minimind-local` ```bash -ollama create -f minimind.modelfile minimind2 +ollama create -f minimind.modelfile minimind-local ``` -启动推理 +3、启动推理 -```text -ollama run minimind2 -> 你好，我是MiniMind2，一个基于xxxxxxxx +```bash +ollama run minimind-local +``` + +
+📤 推送你的模型到 Ollama Hub + +```bash +# 1. 为本地模型重命名为你的ollama-account/minimind的tag +ollama cp minimind-local:latest your_username/minimind:latest + +# 2. 推送模型 +ollama push your_username/minimind:latest +``` +
+
+ +⭐️ 也可直接使用我提供的ollama模型一键启动： + +```bash +ollama run jingyaogong/minimind2 # 其他可选 minimind2-r1 / minimind2-small / minimind2-small-r1 +>>> 你叫什么名字 +我是一个语言模型... ``` > 以上三方框架的更多用法请参考对应官方文档😊 diff --git a/README_en.md b/README_en.md index 012ab60..67081b5 100644 --- a/README_en.md +++ b/README_en.md @@ -1582,9 +1582,10 @@ Models generally achieve baseline performance due to small parameter scales and ## Model Conversion -* [./scripts/convert_model.py](./scripts/convert_model.py) enables conversion between `torch models/transformers` models +* [./scripts/convert_model.py](./scripts/convert_model.py) enables mutual conversion of `torch / transformers` models +* Unless otherwise specified, `MiniMind2` models are by default in `Transformers` format and require `t2t` conversion beforehand! + ---- ## OpenAI-API Based MiniMind Service Interface @@ -1592,13 +1593,12 @@ Models generally achieve baseline performance due to small parameter scales and * Download model weights from [Huggingface](https://huggingface.co/collections/jingyaogong/minimind-66caf8d999f5c7fa64f399e5), file structure: ``` - (root dir) - ├─ + minimind (root dir) + ├─ (e.g. MiniMind2) | ├── config.json | ├── generation_config.json - | ├── LMConfig.py - | ├── model.py - | ├── pytorch_model.bin + | ├── model_minimind.py or w/o + | ├── pytorch_model.bin or model.safetensors | ├── special_tokens_map.json | ├── tokenizer_config.json | ├── tokenizer.json @@ -1631,6 +1631,8 @@ Models generally achieve baseline performance due to small parameter scales and vLLM is an extremely popular efficient inference framework supporting fast deployment of large models, optimizing GPU memory usage and throughput. +Start minimind2 in openai-serve format: + ```bash vllm serve ./MiniMind2 --model-impl transformers --served-model-name "minimind" --port 8998 ``` @@ -1639,62 +1641,109 @@ vllm serve ./MiniMind2 --model-impl transformers --served-model-name "minimind" llama.cpp is a C++ library that can be used directly from command line, supports multi-threaded inference, and supports GPU acceleration. -After referencing official repository installation, insert at ~760 line in `convert_hf_to_gguf.py` +**Directory Structure**: It is recommended to place llama.cpp and minimind in the same parent directory -```text -# Add MiniMind2 tokenizer support +``` +parent/ +├── minimind/ # MiniMind project directory +│ ├── MiniMind2/ # HuggingFace format MiniMind2 model (generated by convert_model.py first) +│ │ ├── config.json +│ │ ├── model.safetensors +│ │ └── ... +│ ├── model/ +│ ├── trainer/ +│ └── ... +└── llama.cpp/ # llama.cpp project directory + ├── build/ + ├── convert_hf_to_gguf.py + └── ... +``` + +0. Follow the official `llama.cpp` installation steps + +1. Insert at the end of the `get_vocab_base_pre` function in `convert_hf_to_gguf.py`: + +```python +# Add MiniMind tokenizer support (you can use any existing one like qwen2) if res is None: - res = "smollm" + res = "qwen2" ``` -Convert custom-trained minimind model -> gguf +2. Convert your custom-trained minimind model: huggingface -> gguf ```bash -python convert_hf_to_gguf.py ../minimind/MiniMind2/ +# Execute under llama.cpp, will generate ../minimind/MiniMind2/MiniMind2-xxx.gguf +python convert_hf_to_gguf.py ../minimind/MiniMind2 ``` -Quantize model +3. Quantize the model (optional) ```bash -./build/bin/llama-quantize ../minimind/MiniMind2/MiniMind2-109M-F16.gguf ../minimind/MiniMind2/Q4-MiniMind2.gguf Q4_K_M +./build/bin/llama-quantize ../minimind/MiniMind2/MiniMind2.gguf ../minimind/MiniMind2/Q4-MiniMind2.gguf Q4_K_M ``` -Command line inference +4. Command line inference test ```bash -./build/bin/llama-cli -m ../minimind/MiniMind2/MiniMind2-109M-F16.gguf --chat-template chatml +./build/bin/llama-cli -m ../minimind/MiniMind2/MiniMind2.gguf -sys "You are a helpful assistant" # system prompt must be fixed ``` ## [ollama](https://ollama.ai) ollama is a tool for running large models locally, supports multiple open-source LLMs, simple and easy to use. -Load custom gguf model through ollama, create new minimind.modelfile: +1. Load custom gguf model through ollama + +Create `minimind.modelfile` under `MiniMind2`: ```text -FROM ./MiniMind2-109M-F16.gguf -TEMPLATE """{{ if .System }}<|im_start|>system +FROM ./Q4-MiniMind2.gguf + +SYSTEM """You are a helpful assistant""" + +TEMPLATE """<|im_start|>system {{ .System }}<|im_end|> -{{ end }}{{ if .Prompt }}<|im_start|>user +<|im_start|>user {{ .Prompt }}<|im_end|> -{{ end }}<|im_start|>assistant +<|im_start|>assistant +{{ .Response }}<|im_end|> """ ``` -Load model and name as `minimind2` +2. Load and name this model as `minimind-local` ```bash -ollama create -f minimind.modelfile minimind2 +ollama create -f minimind.modelfile minimind-local ``` -Start inference +3. Start inference -```text -ollama run minimind2 -> Hello, I am MiniMind2, a model based on xxxxxxxx +```bash +ollama run minimind-local ``` -> For more usage of the above three-party frameworks, please refer to their official documentation 😊 +
+📤 Push your model to Ollama Hub + +```bash +# 1. Rename your local model to your_username/minimind tag +ollama cp minimind-local:latest your_username/minimind:latest + +# 2. Push the model +ollama push your_username/minimind:latest +``` +
+
+ +⭐️ You can also directly use the ollama model I provided with one command: + +```bash +ollama run jingyaogong/minimind2 # Other options: minimind2-r1 / minimind2-small / minimind2-small-r1 +>>> What's your name? +I am a language model... +``` + +> For more usage of the above third-party frameworks, please refer to their official documentation 😊 # 📌 Acknowledge