github
/
minimind-v
mirror of https://github.com/jingyaogong/minimind-v.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge aefaaedd5c into 974a666741

pull/81/merge
shanfenglan 2025-10-21 14:11:35 +08:00 committed by GitHub
parent 974a666741 aefaaedd5c
commit a9374500c5
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 8 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
/model/__pycache__
/dataset
/out

6
README.md
View File

@ -127,10 +127,8 @@ git clone https://www.modelscope.cn/models/openai-mirror/clip-vit-base-patch16
``` ```
```bash ```bash
# 下载纯语言模型权重到 ./out 目录下（作为训练VLM的基座语言模型） # 下载纯语言模型权重到 ./ 目录下（作为训练VLM的基座语言模型）
https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/lm_512.pth git clone https://huggingface.co/jingyaogong/MiniMind2-V
# or
https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/lm_768.pth
``` ```
## Ⅰ 测试已有模型效果 ## Ⅰ 测试已有模型效果

6
README_en.md
View File

@ -133,10 +133,8 @@ git clone https://www.modelscope.cn/models/openai-mirror/clip-vit-base-patch16
``` ```
```bash ```bash
# Download the pure language model to the ./out directory (as the base language model for training VLM): # Download the pure language model to the ./ directory (as the base language model for training VLM):
https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/lm_512.pth git clone https://huggingface.co/jingyaogong/MiniMind2-V
# or
https://huggingface.co/jingyaogong/MiniMind2-V-PyTorch/blob/main/lm_768.pth
``` ```

2
eval_vlm.py
View File

@ -25,7 +25,7 @@ def init_model(lm_config, device):
state_dict = torch.load(ckp, map_location=device) state_dict = torch.load(ckp, map_location=device)
model.load_state_dict({k: v for k, v in state_dict.items() if 'mask' not in k}, strict=False) model.load_state_dict({k: v for k, v in state_dict.items() if 'mask' not in k}, strict=False)
else: else:
transformers_model_path = 'MiniMind2-Small-V' transformers_model_path = 'MiniMind2-V'
tokenizer = AutoTokenizer.from_pretrained(transformers_model_path) tokenizer = AutoTokenizer.from_pretrained(transformers_model_path)
model = AutoModelForCausalLM.from_pretrained(transformers_model_path, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained(transformers_model_path, trust_remote_code=True)
model.vision_encoder, model.processor = MiniMindVLM.get_vision_model("./model/vision_model/clip-vit-base-patch16") model.vision_encoder, model.processor = MiniMindVLM.get_vision_model("./model/vision_model/clip-vit-base-patch16")