From bb0ec7a420b3bce5a84ab99ed099b4669b9e3913 Mon Sep 17 00:00:00 2001 From: jingyaogong Date: Wed, 9 Apr 2025 17:03:13 +0800 Subject: [PATCH] update readme --- README.md | 2 +- README_en.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 61b626f..aeff2f4 100644 --- a/README.md +++ b/README.md @@ -311,7 +311,7 @@ GPT模型根据现有token预测输出下一个下下一个下下下一个token 和LlaVA、Qwen-VL等视觉语言模型类似，MiniMind-V同样选用开源Clip系列模型作为Visual Encoder。 具体使用[clip-vit-base-patch16](https://huggingface.co/openai/clip-vit-base-patch16)， 一种基于 ViT-B/16 架构的经典Visual Encoder用于描述图像文本信息。 -输入的图像尺寸为224x224，因为划分的Patch是16×16，所以会产生16*16=196个token作为encoder编码层的输入， +输入的图像尺寸为224x224，因为划分的Patch是16×16，所以会产生14*14=196个token作为encoder编码层的输入， 最终产生1×768维的嵌入向量用于和文本对计算误差。 我们并不需要最终嵌入表示，因此只取encoder层的输出，也就是VIT核心主干的输出特征即可。 它拿到前一层维度196×768大小的特征，我们把它作为196个visual token输入MiniMind-V。 diff --git a/README_en.md b/README_en.md index 66989f4..4deebfc 100644 --- a/README_en.md +++ b/README_en.md @@ -338,7 +338,7 @@ Like LlaVA, Qwen-VL, and other visual language models, MiniMind-V also uses the Visual Encoder. Specifically, we use [clip-vit-base-patch16](https://huggingface.co/openai/clip-vit-base-patch16), a classic Visual Encoder based on the ViT-B/16 architecture for describing image-text information. -The input image size is 224x224, and because the Patch size is 16×16, it generates 16*16=196 tokens as the input to the +The input image size is 224x224, and because the Patch size is 16×16, it generates 14*14=196 tokens as the input to the encoder layer, which produces a 1×768 dimensional embedding vector for calculating error with the text. We don’t need the final embedding representation, so we only take the output from the encoder layer, which is the output feature from the core ViT backbone.