Only end </think> tag but no start <think> tag.
I used vllm to deploy Qwen/Qwen3-4B-Thinking-2507 model, and used langchain framework to develop an agent with workflow. However, whether I use simple chat flow or the workflow, the returned responses always have no start tag but end tag.
Why is that?
If the response message is too long and exceeds Max_Context_Length, the beginning of the message may be truncated and output.
or maybe this
NOTE: This model supports only thinking mode. Meanwhile, specifying enable_thinking=True is no longer required.
Additionally, to enforce model thinking, the default chat template automatically includes . Therefore, it is normal for the model's output to contain only without an explicit opening tag.
how can i prevent this part from printing? i need only the output
I created a simple qwen_chat_template.jinja file, and deployed the LLM with --chat-template /model/qwen_chat_template.jinja parameter. Then the issue was solved. There are start tag and end tag of 'think'.
how can i prevent this part from printing? i need only the output
I think you can write code to filter the thinking part.
@zhangziji1021 can you please share the chat template that solved this problem
This is the simple template I use.
{% for message in messages %}
{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}
{% endfor %}
{% if add_generation_prompt %}
{{ '<|im_start|>assistant\n' }}
{% endif %}
Also, I found that sometimes the response will have no think tag. When I strictly added rules in the system prompt like "wrap your thinking content by tags", the problem seems to be solved. I need to do more test rounds on it.
Curious, why'd ya'll decide to leave out the initial tag?
I get that it's an implicit token because this model is a thinking model, but for model providers, this is a problem because all other models are following the convention of including it.
Which means providers have to one off patch it.
It also breaks LLM Studio and Cherry Studio's thinking functionality because it isn't present.
Anyway you guys could revert this change and have it return that first token?