llama-index: multiple calls to query_engine.query always gives "Empty...

garage_greg

I have the following code that works as expected model_url = "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGUF/resolve/main/llama-2-13b-chat.Q4_0.gguf" llm = LlamaCPP(model_url=model_url,temperature=0.1,max_new_tokens=256,context_window=3900,generate_kwargs={},model_kwargs={"n_gpu_layers": 1},messages_to_prompt=messages_to_prompt,completion_to_prompt=completion_to_prompt,verbose=True,) reader =...

Originally asked by Jamie Dixon on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Good thread on this — worth a read before you pay anyone to build it.

Discussion: https://stackoverflow.com/questions/77477656/llama-index-multiple-calls-to-query-engine-query-always-gives-empty-response.

Want it set up for you? UphillTech.