github
/
minimind-v
mirror of https://github.com/jingyaogong/minimind-v.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
175 Commits
2 Branches
2 Tags
34 MiB
Commit Graph

9 Commits (7c4e935bde56ff126b772f174b6556f04f9a3be6)

Author SHA1 Message Date
dynorth 7c4e935bde
Merge 2a65e6a675 into 2b94dcb3d0 2026-05-14 11:27:57 +08:00
jingyaogong ff81f88833 [update] minimind-3v 2026-04-20 16:54:49 +08:00
jingyaogong 9c3343dd88 [update] minimind-3v 2026-04-19 23:51:48 +08:00
jingyaogong 9089310ede [update] fp16 inference 2026-03-27 16:22:47 +08:00
jingyaogong e51dcf0fda [update] minimind-3v 2026-03-26 00:09:02 +08:00
MiniMind-V User 823ce1e4fc [feat] Implement DPO training for driving safety alignment 
- dataset/driving_preferences.jsonl: 56 (prompt, chosen, rejected) pairs
  across 8 safety categories: pedestrian, speed, lane change, weather,
  following distance, traffic signals, emergency vehicles, night driving
- scripts/generate_driving_preferences.py: generator script for the dataset
- trainer/train_dpo_driving.py: custom DPO trainer
  * implements DPO loss from scratch (no trl dependency)
  * frozen reference model + LoRA policy (1.07% trainable params)
  * logs preference accuracy per step
  * saves LoRA adapters or full weights
  * 1 epoch on 56 pairs: loss 0.11, pref_acc 96.4%

Usage:
  uv run python trainer/train_dpo_driving.py     --data_path dataset/driving_preferences.jsonl     --use_lora --epochs 3
 2026-03-06 05:00:42 +00:00
MiniMind-V User a86648ed66 [feat] Implement LoRA + RAG for DriveMind-V 
Implemented Day 1 of DriveMind-V plan: parameter-efficient training with LoRA and retrieval-augmented generation with FAISS.

## LoRA (Low-Rank Adaptation)
- Created trainer/lora_utils.py with LoRA configuration helpers
- Created trainer/train_sft_vlm_lora.py for LoRA-enabled SFT training
- Reduces trainable parameters by 98-99% (r=16 → ~1% trainable)
- Memory savings: ~50% during training
- Checkpoint size: 2-5MB vs 100-400MB for full model

## RAG (Retrieval-Augmented Generation)
- Created rag/ package with vector search infrastructure
- rag/vector_store.py: FAISS-based vector storage
- rag/retriever.py: CLIP ViT-B/16 based image retrieval
- rag/rag_vlm.py: RAG-enhanced VLM wrapper
- scripts/build_knowledge_base.py: Build FAISS index from images
- scripts/demo_rag.py: Interactive retrieval demo

## Demo & Testing
- Built demo knowledge base from 8 eval images
- Tested retrieval: perfect match at distance 0.0
- Fixed tensor shape bug in batch encoding

## Stats
- Time: ~1.5 hours
- Files created: 11 new files
- Dependencies added: faiss-cpu==1.13.2

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-03-05 01:13:59 +00:00
jingyaogong 7605850e1d [feat] update args 2025-10-30 10:50:54 +08:00
jingyaogong 623a130c34 250427 2025-04-27 18:13:22 +08:00