use embeddings stored in vector db to reduce work for LLM generating response

spark_sara

I'm trying to understand what the correct strategy is for storing and using embeddings in a vector database, to be used with an LLM. If my goal is to reduce the amount of work the LLM has to do when generating a response, (So you can think of a RAG implementation where I've stored text, embeddings I've created using an LLM, and metadata about the text.) I'm then trying to generate responses using say openai model from queries about the data, and...

Originally asked by user3476463 on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Yes, you're thinking about it right. When a user asks a question, you use the same model you used to store your documents in the first place to turn that question into an embedding. The vector database then determines which text passages are the most similar by comparing that query embedding to the ones that are stored. That's what we're looking for. Since LLMs comprehend text and not number vectors, you then send the LLM the actual text, not the embeddings. So the straightforward flow is: Question → Embedding → Locate related stored text → Forward that text and question to LLM → Receive response You only have to make embeddings for your documents once. The goal of RAG is to keep things...

Full answer by Aradhya: https://stackoverflow.stackexchange.com/a/79889294 (CC BY-SA 4.0).

Need it done for your business? UphillTech.