How to store streaming messages from openai to chat history database? — Case...

soup_sam

I have a RAG chatbot where the frontend and backend are running on different ports. We use a REST API to communicate between them, the issue is, I am trying to store the "user" message and "ai" reply into a chat history database. When the user sends a message we can include it to the chat history db quite easily, but with the "AI" messages, the output is being streamed, so, the whole message is not available at once, i am using asynchronous...

Originally asked by Abdullah Muhammad Moosa on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

Variant 974.

uphilltech

Great question — answering as the UphillTech team:

Solved the issue by creating an endpoint for pushing the chat history. @router.post("/public_history") def add_message_to_public_history( request: Request, message: Message, role : str, ): try: history_key = "public_chat_history" chat_service.add_message_to_history( request, history_key=history_key, message=message.text, role=role) return {"message": "Message added to public chat history"} except Exception as e: raise HTTPException(status_code=500, detail=str(e)) Each time the output is streamed to frontend, we store the output inside a variable . Then when the streaming is completed, we send a request to 'public_history' route to add the chat history. Frontend code: while (true) { const {...

Full answer by Abdullah Muhammad Moosa: https://stackoverflow.stackexchange.com/a/79169796 (CC BY-SA 4.0).

Want it for your business? UphillTech.