swaylq 3108ee24b5 fix: don't char-slice the prompt by max_new_tokens / max_tokens ... web_demo.py rendered the full user message in the chat bubble but stored prompt[-max_new_tokens:] in the message history, so the model received the user's input front-truncated by *character* count using the generation-length slider — the UI and the model saw different text, and the beginning of long questions was silently dropped. serve_openai_api.py applied the same [-max_tokens:] slice to the rendered chat template, which corrupts the <|im_start|> structure (e.g. drops the system block / cuts mid special-token) whenever a client sends a small max_tokens. max_tokens is the *output* budget and has nothing to do with trimming the input. Store the full prompt instead. Input context is already bounded at the token level by tokenizer(..., truncation=True) (model_max_length=131072). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>		2026-06-17 04:43:13 +08:00
..
chat_api.py	[update] minimind-3	2026-03-25 23:57:45 +08:00
convert_model.py	[update] tie embedding	2026-04-19 21:57:28 +08:00
eval_toolcall.py	[update] fp16 inference	2026-03-27 16:29:46 +08:00
serve_openai_api.py	fix: don't char-slice the prompt by max_new_tokens / max_tokens	2026-06-17 04:43:13 +08:00
web_demo.py	fix: don't char-slice the prompt by max_new_tokens / max_tokens	2026-06-17 04:43:13 +08:00