Inferencing LLAMA-2 13B

grill_gus

I have been inferencing LLAMA-13B and suddenly following error arose, --------------------------------------------------------------------------- AssertionError Traceback (most recent call last) <ipython-input-5-7820a34f7358> in <cell line: 3>() 1 # GPU 2 lcpp_llm = None ----> 3 lcpp_llm = Llama( 4 model_path=model_path, 5 # n_gqa = 8, /usr/local/lib/python3.10/dist-packages/llama_cpp/llama.py in __init__(self, model_path, n_ctx, n_parts,...

Originally asked by Malik on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Good thread on this — worth a read before you pay anyone to build it.

Discussion: https://stackoverflow.com/questions/76981470/inferencing-llama-2-13b.

Want it set up for you? UphillTech.