367838379a
The generate() method divides all previously-seen token logits by repetition_penalty uniformly. For tokens with negative logits (which represent less-likely tokens), dividing by penalty > 1.0 moves the logit toward zero, which *increases* the token's softmax probability. This is the exact opposite of the intended repetition suppression. Example with penalty=1.5: logit -3.0 → -3.0/1.5 = -2.0 → probability INCREASES (bug) logit -3.0 → -3.0*1.5 = -4.5 → probability decreases (correct) Fix: apply the penalty conditionally based on logit sign: - positive logits: divide by penalty (push toward zero) - negative logits: multiply by penalty (push away from zero) Both directions reduce probability for previously-seen tokens, matching the behavior of Hugging Face Transformers' RepetitionPenaltyLogitsProcessor. Added 14 tests covering: direction correctness for positive/negative logits, penalty=1.0 no-op, batch independence, duplicate token handling, large penalties, edge cases, and a regression test proving the old code boosted negative-logit token probabilities. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| model_lora.py | ||
| model_minimind.py | ||
| tokenizer.json | ||
| tokenizer_config.json | ||