The generate() method divides all previously-seen token logits by
repetition_penalty uniformly. For tokens with negative logits (which
represent less-likely tokens), dividing by penalty > 1.0 moves the
logit toward zero, which *increases* the token's softmax probability.
This is the exact opposite of the intended repetition suppression.
Example with penalty=1.5:
logit -3.0 → -3.0/1.5 = -2.0 → probability INCREASES (bug)
logit -3.0 → -3.0*1.5 = -4.5 → probability decreases (correct)
Fix: apply the penalty conditionally based on logit sign:
- positive logits: divide by penalty (push toward zero)
- negative logits: multiply by penalty (push away from zero)
Both directions reduce probability for previously-seen tokens, matching
the behavior of Hugging Face Transformers' RepetitionPenaltyLogitsProcessor.
Added 14 tests covering: direction correctness for positive/negative logits,
penalty=1.0 no-op, batch independence, duplicate token handling, large
penalties, edge cases, and a regression test proving the old code boosted
negative-logit token probabilities.
d 🔹
jingyaogong
yuyu5333