github
/
minimind-v
mirror of https://github.com/jingyaogong/minimind-v.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[feat] update readme

pull/107/head
jingyaogong 2025-10-21 22:53:00 +08:00
parent 974a666741
commit 8bed2ec00b
2 changed files with 6 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -128,9 +128,9 @@ git clone https://www.modelscope.cn/models/openai-mirror/clip-vit-base-patch16
```bash ```bash
# 下载纯语言模型权重到 ./out 目录下（作为训练VLM的基座语言模型） # 下载纯语言模型权重到 ./out 目录下（作为训练VLM的基座语言模型）
https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/lm_512.pth https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/llm_512.pth
# or # or
https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/lm_768.pth https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/llm_768.pth
``` ```
## Ⅰ 测试已有模型效果 ## Ⅰ 测试已有模型效果
@ -347,7 +347,7 @@ GPT模型根据现有token预测输出下一个下下一个下下下一个token
下面，我们简单讨论MiniMind-V的外部输入输出的变化。 下面，我们简单讨论MiniMind-V的外部输入输出的变化。
VLM的输入依然是一段文本，其中包含特殊的<image>占位符。 VLM的输入依然是一段文本，其中包含特殊的`<image>`占位符。
在计算文本嵌入后，可以将图像编码器生成的向量投影到该占位符对应的嵌入部分，替换掉原先的占位符embedding。 在计算文本嵌入后，可以将图像编码器生成的向量投影到该占位符对应的嵌入部分，替换掉原先的占位符embedding。
例如： 例如：

6
README_en.md
View File

@ -134,9 +134,9 @@ git clone https://www.modelscope.cn/models/openai-mirror/clip-vit-base-patch16
```bash ```bash
# Download the pure language model to the ./out directory (as the base language model for training VLM): # Download the pure language model to the ./out directory (as the base language model for training VLM):
https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/lm_512.pth https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/llm_512.pth
# or # or
https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/lm_768.pth https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/llm_768.pth
``` ```
@ -377,7 +377,7 @@ With that, the internal structural changes of MiniMind-V are now fully presented
Next, let's briefly discuss the changes in the external input and output of MiniMind-V. Next, let's briefly discuss the changes in the external input and output of MiniMind-V.
The input to the VLM is still a segment of text containing special <image> placeholders. The input to the VLM is still a segment of text containing special `<image>` placeholders.
After computing the text embedding, the vector generated by the image encoder can be projected onto the corresponding After computing the text embedding, the vector generated by the image encoder can be projected onto the corresponding
embedding part of the placeholder, replacing the original placeholder embedding. embedding part of the placeholder, replacing the original placeholder embedding.
For example: For example: