Great question — answering as the UphillTech team:
The code you have written here is a little old/erroneous. But the main error is the service context setup with llm_predictor=llm . You can just pass the llm in directly as a kwarg. Using the latest version (v0.7.22 at the time of writing) I would re-write your service context like so: service_context = ServiceContext.from_defaults( chunk_size= 1024, llm=llm, # this is updated prompt_helper=prompt_helper, embed_model=embed_model ) Source: https://gpt-index.readthedocs.io/en/stable/core_modules/model_modules/llms/usage_custom.html#example-changing-the-underlying-llm For reference, if you pass in an llm from langchain like this, the service context detects this and wraps it with our langchain...
Full answer by Logan: https://stackoverflow.stackexchange.com/a/76868784 (CC BY-SA 4.0).
Need it done for your business? UphillTech.
