I am trying to use a local LLM and langchain to create agents. I have observe that the loading time for the transformer pipeline differs in these two codes: t1 = time.time() pipe = pipeline( "text-generation", model=model_id, ) t2 = time.time() print(t2-t1) ## extreamly fast llm1 = HuggingFacePipeline(pipeline=pipe) Initial_prompt = PromptTemplate( input_variables=..., template=... ) Agent = Initial_prompt | llm1 | StrOutputParser()...
Originally asked by Etrom on Stack Exchange (stackoverflow) (CC BY-SA 4.0).
