Llama3.2 fails to respond to simple text inputs when bounded with tool...

tux_tina

I am following along a LangChain tutorial for LangGraph. They are using OpenAI models in the tutorial. However, I want to use my local Ollama models. I am using Llama 3.2 as that supports tool callings. However, when I bind tools to the chat object, llm , it does not respond to normal text inputs and only returns a tool response. If it is not bound with a tool, it does respond to regular messages I am not sure and I cannot figure out whether it...

Originally asked by Neha on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Like Laurie Young mentioned, you need bigger models of Llama. I took thought it was Langgraph that was inconsistent. Rewrote my application without Langgraph in python. No use. Llama 1B is the issue here. It can either use LLM or it can use tools. It cannot combine both. The fix is to use a better model. It took a few days of breaking my head and testing with python/Pydantic and extensive online search and testing with ChatGPT to realize that Llama is the issue. I wish Meta had better documentation on this. Its appalling that they did not mention this anywhere on their documentation. What a waste of my time! I have decided to give up on Llama and stick to ChatGPT just because how unhelpful...

Full answer by C Lakshmi: https://stackoverflow.stackexchange.com/a/79488522 (CC BY-SA 4.0).

Need it done for your business? UphillTech.