[feat] update readme
parent
cc81925ecc
commit
eb96113cd4
104
README.md
104
README.md
|
|
@ -1659,9 +1659,9 @@ MiniMind模型本身预训练数据集小的可怜,也没有针对性的对测
|
||||||
|
|
||||||
## 模型转换
|
## 模型转换
|
||||||
|
|
||||||
* [./scripts/convert_model.py](./scripts/convert_model.py)可以实现`torch模型/transformers`模型之间的转换
|
* [./scripts/convert_model.py](./scripts/convert_model.py)可以实现`torch / transformers`模型的互相转换
|
||||||
|
* 如无特别说明,`MiniMind2`模型均默认为`Transformers`格式的模型,需提前`t2t`转换!
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 基于MiniMind-API服务接口
|
## 基于MiniMind-API服务接口
|
||||||
|
|
||||||
|
|
@ -1670,13 +1670,12 @@ MiniMind模型本身预训练数据集小的可怜,也没有针对性的对测
|
||||||
|
|
||||||
* 从[Huggingface](https://huggingface.co/collections/jingyaogong/minimind-66caf8d999f5c7fa64f399e5)下载模型权重文件,文件树:
|
* 从[Huggingface](https://huggingface.co/collections/jingyaogong/minimind-66caf8d999f5c7fa64f399e5)下载模型权重文件,文件树:
|
||||||
```
|
```
|
||||||
<MiniMind-Model-Name> (root dir)
|
minimind (root dir)
|
||||||
├─<MiniMind-Model-Name>
|
├─<MiniMind-Model-Name>(例如MiniMind2)
|
||||||
| ├── config.json
|
| ├── config.json
|
||||||
| ├── generation_config.json
|
| ├── generation_config.json
|
||||||
| ├── LMConfig.py
|
| ├── model_minimind.py or w/o
|
||||||
| ├── model.py
|
| ├── pytorch_model.bin or model.safetensors
|
||||||
| ├── pytorch_model.bin
|
|
||||||
| ├── special_tokens_map.json
|
| ├── special_tokens_map.json
|
||||||
| ├── tokenizer_config.json
|
| ├── tokenizer_config.json
|
||||||
| ├── tokenizer.json
|
| ├── tokenizer.json
|
||||||
|
|
@ -1709,6 +1708,8 @@ MiniMind模型本身预训练数据集小的可怜,也没有针对性的对测
|
||||||
|
|
||||||
vLLM是极其流行的高效推理框架,支持大模型快速部署,优化显存利用与吞吐量。
|
vLLM是极其流行的高效推理框架,支持大模型快速部署,优化显存利用与吞吐量。
|
||||||
|
|
||||||
|
以openai-serve形式启动 minimind2:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
vllm serve ./MiniMind2 --model-impl transformers --served-model-name "minimind" --port 8998
|
vllm serve ./MiniMind2 --model-impl transformers --served-model-name "minimind" --port 8998
|
||||||
```
|
```
|
||||||
|
|
@ -1718,59 +1719,106 @@ vllm serve ./MiniMind2 --model-impl transformers --served-model-name "minimind"
|
||||||
llama.cpp是一个C++库,
|
llama.cpp是一个C++库,
|
||||||
可以在命令行下直接使用,支持多线程推理,支持GPU加速。
|
可以在命令行下直接使用,支持多线程推理,支持GPU加速。
|
||||||
|
|
||||||
参考官方仓库安装后,在`convert_hf_to_gguf.py` ~760行插入
|
**目录结构**:建议将llama.cpp与minimind放在同级目录下
|
||||||
|
|
||||||
```text
|
```
|
||||||
# 添加MiniMind2 tokenizer支持
|
parent/
|
||||||
|
├── minimind/ # MiniMind项目目录
|
||||||
|
│ ├── MiniMind2/ # HuggingFace格式MiniMind2模型 (先convert_model.py生成)
|
||||||
|
│ │ ├── config.json
|
||||||
|
│ │ ├── model.safetensors
|
||||||
|
│ │ └── ...
|
||||||
|
│ ├── model/
|
||||||
|
│ ├── trainer/
|
||||||
|
│ └── ...
|
||||||
|
└── llama.cpp/ # llama.cpp项目目录
|
||||||
|
├── build/
|
||||||
|
├── convert_hf_to_gguf.py
|
||||||
|
└── ...
|
||||||
|
```
|
||||||
|
|
||||||
|
0、参考`llama.cpp`官方步骤进行install
|
||||||
|
|
||||||
|
1、在`convert_hf_to_gguf.py`的`get_vocab_base_pre`函数最后插入:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# 添加MiniMind tokenizer支持(这里随便写一个例如qwen2即可)
|
||||||
if res is None:
|
if res is None:
|
||||||
res = "smollm"
|
res = "qwen2"
|
||||||
```
|
```
|
||||||
|
|
||||||
转换自定义训练的minimind模型 -> gguf
|
2、转换自训练的minimind模型:huggingface -> gguf
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python convert_hf_to_gguf.py ../minimind/MiniMind2/
|
# 在llama.cpp下执行,将生成../minimind/MiniMind2/MiniMind2-xxx.gguf
|
||||||
|
python convert_hf_to_gguf.py ../minimind/MiniMind2
|
||||||
```
|
```
|
||||||
|
|
||||||
量化模型
|
3、量化此模型 (可选)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build/bin/llama-quantize ../minimind/MiniMind2/MiniMind2-109M-F16.gguf ../minimind/MiniMind2/Q4-MiniMind2.gguf Q4_K_M
|
./build/bin/llama-quantize ../minimind/MiniMind2/MiniMind2.gguf ../minimind/MiniMind2/Q4-MiniMind2.gguf Q4_K_M
|
||||||
```
|
```
|
||||||
|
|
||||||
命令行推理
|
4、命令行推理测试
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build/bin/llama-cli -m ../minimind/MiniMind2/MiniMind2-109M-F16.gguf --chat-template chatml
|
./build/bin/llama-cli -m ../minimind/MiniMind2/MiniMind2.gguf -sys "You are a helpful assistant" # system prompt必须固定
|
||||||
```
|
```
|
||||||
|
|
||||||
## <img src="https://ollama.com/public/cloud.png" height="28" style="vertical-align: middle;"/> [ollama](https://ollama.ai)
|
## <img src="https://ollama.com/public/cloud.png" height="28" style="vertical-align: middle;"/> [ollama](https://ollama.ai)
|
||||||
|
|
||||||
ollama是本地运行大模型的工具,支持多种开源LLM,简单易用。
|
ollama是本地运行大模型的工具,支持多种开源LLM,简单易用。
|
||||||
|
|
||||||
通过ollama加载自定义的gguf模型,新建minimind.modelfile:
|
1、通过ollama加载自定义的gguf模型
|
||||||
|
|
||||||
|
在`MiniMind2`下新建`minimind.modelfile`,写入:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
FROM ./MiniMind2-109M-F16.gguf
|
FROM ./Q4-MiniMind2.gguf
|
||||||
TEMPLATE """{{ if .System }}<|im_start|>system
|
|
||||||
|
SYSTEM """You are a helpful assistant"""
|
||||||
|
|
||||||
|
TEMPLATE """<|im_start|>system
|
||||||
{{ .System }}<|im_end|>
|
{{ .System }}<|im_end|>
|
||||||
{{ end }}{{ if .Prompt }}<|im_start|>user
|
<|im_start|>user
|
||||||
{{ .Prompt }}<|im_end|>
|
{{ .Prompt }}<|im_end|>
|
||||||
{{ end }}<|im_start|>assistant
|
<|im_start|>assistant
|
||||||
|
{{ .Response }}<|im_end|>
|
||||||
"""
|
"""
|
||||||
```
|
```
|
||||||
|
|
||||||
加载模型并命名为`minimind2`
|
2、加载并命名此模型为`minimind-local`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ollama create -f minimind.modelfile minimind2
|
ollama create -f minimind.modelfile minimind-local
|
||||||
```
|
```
|
||||||
|
|
||||||
启动推理
|
3、启动推理
|
||||||
|
|
||||||
```text
|
```bash
|
||||||
ollama run minimind2
|
ollama run minimind-local
|
||||||
> 你好,我是MiniMind2,一个基于xxxxxxxx
|
```
|
||||||
|
|
||||||
|
<details>
|
||||||
|
<summary>📤 推送你的模型到 Ollama Hub</summary>
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. 为本地模型重命名为你的ollama-account/minimind的tag
|
||||||
|
ollama cp minimind-local:latest your_username/minimind:latest
|
||||||
|
|
||||||
|
# 2. 推送模型
|
||||||
|
ollama push your_username/minimind:latest
|
||||||
|
```
|
||||||
|
</details>
|
||||||
|
<br/>
|
||||||
|
|
||||||
|
⭐️ 也可直接使用我提供的ollama模型一键启动:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ollama run jingyaogong/minimind2 # 其他可选 minimind2-r1 / minimind2-small / minimind2-small-r1
|
||||||
|
>>> 你叫什么名字
|
||||||
|
我是一个语言模型...
|
||||||
```
|
```
|
||||||
|
|
||||||
> 以上三方框架的更多用法请参考对应官方文档😊
|
> 以上三方框架的更多用法请参考对应官方文档😊
|
||||||
|
|
|
||||||
107
README_en.md
107
README_en.md
|
|
@ -1582,9 +1582,10 @@ Models generally achieve baseline performance due to small parameter scales and
|
||||||
|
|
||||||
## Model Conversion
|
## Model Conversion
|
||||||
|
|
||||||
* [./scripts/convert_model.py](./scripts/convert_model.py) enables conversion between `torch models/transformers` models
|
* [./scripts/convert_model.py](./scripts/convert_model.py) enables mutual conversion of `torch / transformers` models
|
||||||
|
* Unless otherwise specified, `MiniMind2` models are by default in `Transformers` format and require `t2t` conversion beforehand!
|
||||||
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## OpenAI-API Based MiniMind Service Interface
|
## OpenAI-API Based MiniMind Service Interface
|
||||||
|
|
||||||
|
|
@ -1592,13 +1593,12 @@ Models generally achieve baseline performance due to small parameter scales and
|
||||||
|
|
||||||
* Download model weights from [Huggingface](https://huggingface.co/collections/jingyaogong/minimind-66caf8d999f5c7fa64f399e5), file structure:
|
* Download model weights from [Huggingface](https://huggingface.co/collections/jingyaogong/minimind-66caf8d999f5c7fa64f399e5), file structure:
|
||||||
```
|
```
|
||||||
<MiniMind-Model-Name> (root dir)
|
minimind (root dir)
|
||||||
├─<MiniMind-Model-Name>
|
├─<MiniMind-Model-Name> (e.g. MiniMind2)
|
||||||
| ├── config.json
|
| ├── config.json
|
||||||
| ├── generation_config.json
|
| ├── generation_config.json
|
||||||
| ├── LMConfig.py
|
| ├── model_minimind.py or w/o
|
||||||
| ├── model.py
|
| ├── pytorch_model.bin or model.safetensors
|
||||||
| ├── pytorch_model.bin
|
|
||||||
| ├── special_tokens_map.json
|
| ├── special_tokens_map.json
|
||||||
| ├── tokenizer_config.json
|
| ├── tokenizer_config.json
|
||||||
| ├── tokenizer.json
|
| ├── tokenizer.json
|
||||||
|
|
@ -1631,6 +1631,8 @@ Models generally achieve baseline performance due to small parameter scales and
|
||||||
|
|
||||||
vLLM is an extremely popular efficient inference framework supporting fast deployment of large models, optimizing GPU memory usage and throughput.
|
vLLM is an extremely popular efficient inference framework supporting fast deployment of large models, optimizing GPU memory usage and throughput.
|
||||||
|
|
||||||
|
Start minimind2 in openai-serve format:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
vllm serve ./MiniMind2 --model-impl transformers --served-model-name "minimind" --port 8998
|
vllm serve ./MiniMind2 --model-impl transformers --served-model-name "minimind" --port 8998
|
||||||
```
|
```
|
||||||
|
|
@ -1639,62 +1641,109 @@ vllm serve ./MiniMind2 --model-impl transformers --served-model-name "minimind"
|
||||||
|
|
||||||
llama.cpp is a C++ library that can be used directly from command line, supports multi-threaded inference, and supports GPU acceleration.
|
llama.cpp is a C++ library that can be used directly from command line, supports multi-threaded inference, and supports GPU acceleration.
|
||||||
|
|
||||||
After referencing official repository installation, insert at ~760 line in `convert_hf_to_gguf.py`
|
**Directory Structure**: It is recommended to place llama.cpp and minimind in the same parent directory
|
||||||
|
|
||||||
```text
|
```
|
||||||
# Add MiniMind2 tokenizer support
|
parent/
|
||||||
|
├── minimind/ # MiniMind project directory
|
||||||
|
│ ├── MiniMind2/ # HuggingFace format MiniMind2 model (generated by convert_model.py first)
|
||||||
|
│ │ ├── config.json
|
||||||
|
│ │ ├── model.safetensors
|
||||||
|
│ │ └── ...
|
||||||
|
│ ├── model/
|
||||||
|
│ ├── trainer/
|
||||||
|
│ └── ...
|
||||||
|
└── llama.cpp/ # llama.cpp project directory
|
||||||
|
├── build/
|
||||||
|
├── convert_hf_to_gguf.py
|
||||||
|
└── ...
|
||||||
|
```
|
||||||
|
|
||||||
|
0. Follow the official `llama.cpp` installation steps
|
||||||
|
|
||||||
|
1. Insert at the end of the `get_vocab_base_pre` function in `convert_hf_to_gguf.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Add MiniMind tokenizer support (you can use any existing one like qwen2)
|
||||||
if res is None:
|
if res is None:
|
||||||
res = "smollm"
|
res = "qwen2"
|
||||||
```
|
```
|
||||||
|
|
||||||
Convert custom-trained minimind model -> gguf
|
2. Convert your custom-trained minimind model: huggingface -> gguf
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python convert_hf_to_gguf.py ../minimind/MiniMind2/
|
# Execute under llama.cpp, will generate ../minimind/MiniMind2/MiniMind2-xxx.gguf
|
||||||
|
python convert_hf_to_gguf.py ../minimind/MiniMind2
|
||||||
```
|
```
|
||||||
|
|
||||||
Quantize model
|
3. Quantize the model (optional)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build/bin/llama-quantize ../minimind/MiniMind2/MiniMind2-109M-F16.gguf ../minimind/MiniMind2/Q4-MiniMind2.gguf Q4_K_M
|
./build/bin/llama-quantize ../minimind/MiniMind2/MiniMind2.gguf ../minimind/MiniMind2/Q4-MiniMind2.gguf Q4_K_M
|
||||||
```
|
```
|
||||||
|
|
||||||
Command line inference
|
4. Command line inference test
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build/bin/llama-cli -m ../minimind/MiniMind2/MiniMind2-109M-F16.gguf --chat-template chatml
|
./build/bin/llama-cli -m ../minimind/MiniMind2/MiniMind2.gguf -sys "You are a helpful assistant" # system prompt must be fixed
|
||||||
```
|
```
|
||||||
|
|
||||||
## <img src="https://ollama.com/public/cloud.png" height="28" style="vertical-align: middle;"/> [ollama](https://ollama.ai)
|
## <img src="https://ollama.com/public/cloud.png" height="28" style="vertical-align: middle;"/> [ollama](https://ollama.ai)
|
||||||
|
|
||||||
ollama is a tool for running large models locally, supports multiple open-source LLMs, simple and easy to use.
|
ollama is a tool for running large models locally, supports multiple open-source LLMs, simple and easy to use.
|
||||||
|
|
||||||
Load custom gguf model through ollama, create new minimind.modelfile:
|
1. Load custom gguf model through ollama
|
||||||
|
|
||||||
|
Create `minimind.modelfile` under `MiniMind2`:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
FROM ./MiniMind2-109M-F16.gguf
|
FROM ./Q4-MiniMind2.gguf
|
||||||
TEMPLATE """{{ if .System }}<|im_start|>system
|
|
||||||
|
SYSTEM """You are a helpful assistant"""
|
||||||
|
|
||||||
|
TEMPLATE """<|im_start|>system
|
||||||
{{ .System }}<|im_end|>
|
{{ .System }}<|im_end|>
|
||||||
{{ end }}{{ if .Prompt }}<|im_start|>user
|
<|im_start|>user
|
||||||
{{ .Prompt }}<|im_end|>
|
{{ .Prompt }}<|im_end|>
|
||||||
{{ end }}<|im_start|>assistant
|
<|im_start|>assistant
|
||||||
|
{{ .Response }}<|im_end|>
|
||||||
"""
|
"""
|
||||||
```
|
```
|
||||||
|
|
||||||
Load model and name as `minimind2`
|
2. Load and name this model as `minimind-local`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ollama create -f minimind.modelfile minimind2
|
ollama create -f minimind.modelfile minimind-local
|
||||||
```
|
```
|
||||||
|
|
||||||
Start inference
|
3. Start inference
|
||||||
|
|
||||||
```text
|
```bash
|
||||||
ollama run minimind2
|
ollama run minimind-local
|
||||||
> Hello, I am MiniMind2, a model based on xxxxxxxx
|
|
||||||
```
|
```
|
||||||
|
|
||||||
> For more usage of the above three-party frameworks, please refer to their official documentation 😊
|
<details>
|
||||||
|
<summary>📤 Push your model to Ollama Hub</summary>
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Rename your local model to your_username/minimind tag
|
||||||
|
ollama cp minimind-local:latest your_username/minimind:latest
|
||||||
|
|
||||||
|
# 2. Push the model
|
||||||
|
ollama push your_username/minimind:latest
|
||||||
|
```
|
||||||
|
</details>
|
||||||
|
<br/>
|
||||||
|
|
||||||
|
⭐️ You can also directly use the ollama model I provided with one command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ollama run jingyaogong/minimind2 # Other options: minimind2-r1 / minimind2-small / minimind2-small-r1
|
||||||
|
>>> What's your name?
|
||||||
|
I am a language model...
|
||||||
|
```
|
||||||
|
|
||||||
|
> For more usage of the above third-party frameworks, please refer to their official documentation 😊
|
||||||
|
|
||||||
# 📌 Acknowledge
|
# 📌 Acknowledge
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue