62c41b050c
Merge 2a65e6a675 into b6a2532801
2026-05-03 10:38:37 +08:00
b6a2532801
[update] readme
2026-05-02 00:35:40 +08:00
1e86f201a2
[update] readme
2026-05-01 12:11:48 +08:00
6fa6127b69
[update] readme
2026-04-28 17:26:02 +08:00
0a9c06fa68
[update] readme
2026-04-27 10:57:35 +08:00
fb38500054
[update] readme
2026-04-27 10:51:46 +08:00
ff81f88833
[update] minimind-3v
2026-04-20 16:54:49 +08:00
92ae3f963a
[update] readme
2026-04-20 00:06:27 +08:00
9c3343dd88
[update] minimind-3v
2026-04-19 23:51:48 +08:00
333654d6e9
[update] image
2026-04-04 14:57:45 +08:00
f74124f6b9
[fix] bugs
2026-04-04 12:43:13 +08:00
5098d67d9f
[fix] issue#114
2026-04-04 12:13:28 +08:00
41c6d9e2a5
Merge pull request #115 from CgggggY/fix-single-batch-multi-image-shape
...
修复 bs=1 多图输入时 vision_tensors 维度不一致的问题
2026-04-04 12:06:04 +08:00
9e59a77080
Fix vision_tensors shape for bs=1 multi-image inputs
2026-04-04 11:32:38 +08:00
c840cdea35
Personal study notes for model_vlm (local only)
2026-04-04 11:22:27 +08:00
4bc6524ac2
[fix] sync minimind
2026-04-02 16:34:38 +08:00
887a42d9cb
[update] model
2026-04-02 15:29:50 +08:00
28acd77d42
[fix] dummy grad
2026-04-01 17:56:44 +08:00
48adb0fda0
[fix] pr113
2026-04-01 15:40:25 +08:00
ac4c506eeb
Merge pull request #113 from JX446/master
...
fix: 修复 eval_vlm.py 中 f-string 引号嵌套导致的语法错误
2026-04-01 15:31:33 +08:00
eac29745da
fix: 修复 eval_vlm.py 中 f-string 引号嵌套导致的语法错误
...
- 修正第 66 行 print 语句中单双引号嵌套冲突的问题。
- 解决在部分 Python 版本下因反斜杠处理导致的 SyntaxError。
2026-04-01 15:11:38 +08:00
14cfa88629
[update] requirement
2026-03-30 11:51:18 +08:00
9089310ede
[update] fp16 inference
2026-03-27 16:22:47 +08:00
1a6d650e9a
[update] readme
2026-03-26 11:46:05 +08:00
e51dcf0fda
[update] minimind-3v
2026-03-26 00:09:02 +08:00
2a65e6a675
[eval] Add DPO preference accuracy evaluation for DriveMind-V
...
- eval_dpo_driving.py: compares SFT vs DPO model on all 56 driving
preference pairs across 8 safety categories; uses length-normalized
log-probs (per-token mean) to eliminate bias toward shorter rejected
responses; reports per-category accuracy, reward margin distribution,
and qualitative generation samples
- eval_dpo_report.txt: captured results — DPO improves preference
accuracy from 33.9% to 67.9% (+33.9 pp, +51% relative), median
per-token margin flips from -1.0 to +0.4; speed and traffic_signals
categories reach 100%
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 00:26:12 +00:00
823ce1e4fc
[feat] Implement DPO training for driving safety alignment
...
- dataset/driving_preferences.jsonl: 56 (prompt, chosen, rejected) pairs
across 8 safety categories: pedestrian, speed, lane change, weather,
following distance, traffic signals, emergency vehicles, night driving
- scripts/generate_driving_preferences.py: generator script for the dataset
- trainer/train_dpo_driving.py: custom DPO trainer
* implements DPO loss from scratch (no trl dependency)
* frozen reference model + LoRA policy (1.07% trainable params)
* logs preference accuracy per step
* saves LoRA adapters or full weights
* 1 epoch on 56 pairs: loss 0.11, pref_acc 96.4%
Usage:
uv run python trainer/train_dpo_driving.py --data_path dataset/driving_preferences.jsonl --use_lora --epochs 3
2026-03-06 05:00:42 +00:00
62f47fb844
[fix] LoRA/RAG: correct target modules, vlm_checkpoint call, and gitignore
...
- lora_utils.py: fix target_modules to match MiniMind arch (q/k/v/o_proj
instead of GPT-2 style c_attn/c_proj/c_fc)
- train_sft_vlm_lora.py: fix broken vlm_checkpoint call in train_epoch
(was passing model/epoch/step/args instead of vlm_config); now saves
inline like train_sft_vlm.py; remove unused SkipBatchSampler import
- .gitignore: add __pycache__, .claude/, dataset/sft_vlm.parquet
2026-03-06 04:49:40 +00:00
a86648ed66
[feat] Implement LoRA + RAG for DriveMind-V
...
Implemented Day 1 of DriveMind-V plan: parameter-efficient training with LoRA and retrieval-augmented generation with FAISS.
## LoRA (Low-Rank Adaptation)
- Created trainer/lora_utils.py with LoRA configuration helpers
- Created trainer/train_sft_vlm_lora.py for LoRA-enabled SFT training
- Reduces trainable parameters by 98-99% (r=16 → ~1% trainable)
- Memory savings: ~50% during training
- Checkpoint size: 2-5MB vs 100-400MB for full model
## RAG (Retrieval-Augmented Generation)
- Created rag/ package with vector search infrastructure
- rag/vector_store.py: FAISS-based vector storage
- rag/retriever.py: CLIP ViT-B/16 based image retrieval
- rag/rag_vlm.py: RAG-enhanced VLM wrapper
- scripts/build_knowledge_base.py: Build FAISS index from images
- scripts/demo_rag.py: Interactive retrieval demo
## Demo & Testing
- Built demo knowledge base from 8 eval images
- Tested retrieval: perfect match at distance 0.0
- Fixed tensor shape bug in batch encoding
## Stats
- Time: ~1.5 hours
- Files created: 11 new files
- Dependencies added: faiss-cpu==1.13.2
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-05 01:13:59 +00:00
56ab6ebd13
[update] readme
2026-02-05 00:23:20 +08:00
33db76fe10
[update] dataset name
2026-02-05 00:22:58 +08:00
b28715e6fd
[update] vlm params
2026-02-04 16:41:17 +08:00
7e094afa59
[update] shuffle data
2026-01-18 17:02:24 +08:00
326b24727e
[update] sync minimind
2026-01-15 21:51:58 +08:00
d2bdff3269
[feat] test parquet
2025-12-27 10:56:07 +08:00
fef284776f
[feat] delete images
2025-12-27 10:53:38 +08:00
3684285a91
[feat] support parquet data
2025-12-27 10:49:50 +08:00
1863743ee9
[feat] update readme
2025-12-27 10:48:54 +08:00
32cf4c5c01
[fix] add gc
2025-12-27 01:09:33 +08:00
b7cb98c136
[fix] reduce aux_loss_alpha
2025-12-05 23:07:55 +08:00
13c3a2c093
[fix] cuda memory
2025-12-01 16:35:05 +08:00
e03af46d12
[feat] update yarn
2025-12-01 16:34:00 +08:00
a157b66ace
[feat] del cuda memory
2025-11-27 19:37:39 +08:00
e613613a13
[feat] update model
2025-11-27 19:36:41 +08:00
d698d98dc5
[feat] update readme
2025-11-11 11:12:09 +08:00
92ca9959bb
[feat] update readme
2025-10-30 23:32:12 +08:00
ca15b0440f
[feat] update readme
2025-10-30 23:31:19 +08:00
2e548ba08e
[feat] update datasets
2025-10-30 11:08:30 +08:00
25d25383e7
[feat] update requirements
2025-10-30 11:06:20 +08:00
7605850e1d
[feat] update args
2025-10-30 10:50:54 +08:00
dynorth
jingyaogong
jingyaogong
CgggggY
DJY
MiniMind-V User