Great question — answering as the UphillTech team:
Try omitting top_p and see if it works. temperature and top_p are not recommended to be altered at the same time according to the OpenAI documentation and I assume the same for other LLMs. Both parameters control the randomness of the response, but they use different techniques. I usually play with temperature a bit more. Setting temperature to a larger value usually makes things more random. So try something like this? I'm not sure why you would want to alter the repeat_penalty so let's just remove it. llm = ChatOllama( model="qwen2.5:14b", temperature=1.3, num_ctx=2048, timeout=120, )
Full answer by Tim Chiu: https://stackoverflow.stackexchange.com/a/79675300 (CC BY-SA 4.0).
Need it done for your business? UphillTech.
