I am running quantized llama-2 model from here . I am using 2 different machines. 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz 2.80 GHz 16.0 GB (15.8 GB usable) Inference time on this machine is pretty good. I get my desired response in 3-4 minutes Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz 2.20 GHz (2 processors) 224 GB Inference time on this machine is very long. It takes around half an hour to give unsatisfactory response. It even has an...
Originally asked by Muhammad Burhan on Stack Exchange (stackoverflow) (CC BY-SA 4.0).
