MiniMind-V User
|
2a65e6a675
|
[eval] Add DPO preference accuracy evaluation for DriveMind-V
- eval_dpo_driving.py: compares SFT vs DPO model on all 56 driving
preference pairs across 8 safety categories; uses length-normalized
log-probs (per-token mean) to eliminate bias toward shorter rejected
responses; reports per-category accuracy, reward margin distribution,
and qualitative generation samples
- eval_dpo_report.txt: captured results — DPO improves preference
accuracy from 33.9% to 67.9% (+33.9 pp, +51% relative), median
per-token margin flips from -1.0 to +0.4; speed and traffic_signals
categories reach 100%
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-03-07 00:26:12 +00:00 |