Is there a way to manually set the first part of a model's response in...

salon_sue

A bit of context, I recently got into tinkering with LLMs using the ollama python package. My latest project is an AI assistant (how original, I know) with tool calling capabilities. I'm using the qwen3:4b model, and while due to it's size it isn't the greatest at tool calling, it still gets the job done. One of the tools I wanted to integrate was a "file explorer" type of tool. The way I went about doing this was that the model would first call...

Originally asked by Bob on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Sadly, it appears that the Ollama API does not support this kind of use of LLMs. However, I was able to achieve the results I wanted by using Hugging Face transformers and Pytorch. It's not a pretty solution, and I really hope that the Ollama team implement features such as forced tool calls and multiple choice tools soon, but for now this will do: from transformers import AutoTokenizer, AutoModelForCausalLM import torch device = torch.device("cuda" if torch.cuda.is_available() else "cpu") print(device) model_name = 'Qwen/Qwen3-0.6B' tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name).to(device) def open_document(filename: str) ->...

Full answer by Bob: https://stackoverflow.stackexchange.com/a/79693553 (CC BY-SA 4.0).

Need it done for your business? UphillTech.