Best practices for scaling LangChain agent architecture in FastAPI

cafe_carl

I have a FastAPI app using LangChain agents that works fine at low traffic but struggles at scale. Current setup: from fastapi import FastAPI from langchain.agents import initialize_agent, AgentType from langchain.chat_models import ChatOpenAI from langchain.tools import Tool app = FastAPI() llm = ChatOpenAI(temperature=0, model="gpt-4") agent = initialize_agent(tools, llm, agent=AgentType.OPENAI_FUNCTIONS) @app.post("/chat") async def...

Originally asked by Sergio G on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

first edit question and format it. Put every • in new line to make it more readable. It may need to add two spaces at the end of previous line. And use ` ` for inline code.

Full answer by furas: https://stackoverflow.stackexchange.com/a/79936604 (CC BY-SA 4.0).

Need it done for your business? UphillTech.