I have Qwen 2.5 7B installed on an EC2 instance - currently a C7i.8Xlarge - and am running a series of data driven prompts through it. The variables from the prompts are pulled from a dataframe. I am using Ollama invoke to call the model. As the script iterates through the rows in the dataframe, each call to Qwen typically takes 20 seconds - sometimes as low as 6 seconds and sometimes as long as 57 seconds. And then sometimes it will hang for 1...
Originally asked by 29Clyde on Stack Exchange (stackoverflow) (CC BY-SA 4.0).
