From a932399071b468171305691eec257a07f965c13f Mon Sep 17 00:00:00 2001
From: gongjy <2474590974@qq.com>
Date: Sat, 5 Oct 2024 01:06:18 +0800
Subject: [PATCH] update readme

---
 README.md    | 12 +++++++++++-
 README_en.md | 12 +++++++++++-
 2 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/README.md b/README.md
index 57fe6bf..5219423 100644
--- a/README.md
+++ b/README.md
@@ -497,13 +497,23 @@ python web_server.py
     - **描述全面且具体**：该模型的描述是四个模型中最详细和精确的。比如，描述熊的图像时提到了“站在一片开放的草地上，周围有树木和草丛，还有一只背包”，能够准确捕捉到多个图像元素。
     - **具备更强的理解力**：该模型能够识别图像的场景和背景，提供合理的解释和推测。例如，描述“家庭聚会”或“庆祝活动”，这些解释让图像更具上下文联系。
 
-### 总结：
+### GPT-4o总结：
 
 - **512_pretrain**的表现最差，描述简单且不准确。
 - **512_sft**的描述详细度有所提升，但偶尔出现不相关信息。
 - **768_pretrain**信息连贯性差，但在某些方面能提供基本描述。
 - **768_sft**表现最佳，能够给出详细、准确的描述，并且能够很好地推测图像的上下文。
 
+### 个人小结：
+
+- 视觉信号对于LLM是一种特殊的外语，因此“学习外语”的能力高低，很大程度上取决于LLM的能力。
+- LLM性能越强，对应的VLM必然越强，且效果增益很明显。
+- 值得改进的方面：
+  - 更简单的Projection的跨模态特征对齐方式，相较于Cross-Attention会带来更大的性能损失。
+  - Clip模型可以尝试更大性能更强的large系列，用更具细粒度的token表征图像特征，目前仍非常粗糙。
+  - 分辨率不高，理论上只有224*224（minimind-v数据集为节省空间，仅设定为128*128）。
+  - ...
+
 ---
 
 # 📌 Acknowledge
diff --git a/README_en.md b/README_en.md
index f801850..d255daf 100644
--- a/README_en.md
+++ b/README_en.md
@@ -483,13 +483,23 @@ Based on the provided table data, the performance of the four models can be summ
     - **Comprehensive and specific descriptions**: This model's descriptions are the most detailed and precise among the four. For instance, when describing the bear image, it mentions "standing in an open field of grass, surrounded by trees and bushes, with a backpack," capturing multiple elements of the image.
     - **Stronger comprehension ability**: This model can identify the scene and context of the image, providing reasonable interpretations and speculations. For example, describing a "family gathering" or "celebration" gives the image a more contextual connection.
 
-### Summary:
+### GPT=4o Summary:
 
 - **512_pretrain** performs the worst, with simple and inaccurate descriptions.
 - **512_sft** has improved detail in descriptions but occasionally includes irrelevant information.
 - **768_pretrain** has poor coherence in information, yet provides basic descriptions in some aspects.
 - **768_sft** performs the best, offering detailed, accurate descriptions, and is able to make good context-based inferences.
 
+### Personal Summary:
+
+- Visual signals are a special kind of foreign language for LLMs, so the ability to "learn foreign languages" largely depends on the capabilities of the LLM.
+- The stronger the performance of the LLM, the stronger the corresponding VLM will be, and the performance gain will be significant.
+- Areas for improvement:
+  - Simpler projection methods for cross-modal feature alignment result in greater performance loss compared to Cross-Attention.
+  - Larger and more powerful large-series Clip models can be tried, using more fine-grained token representations for image features, which are currently very rough.
+  - The resolution is not high, theoretically only 224*224 (minimind-v dataset is set to 128*128 to save space).
+  - ...
+
 ---
 
 # 📌 Acknowledge