Limit tokens per minute in LangChain, using OpenAI-embeddings and Chroma...

chef_carol

I am looking for a way to limit the tokens per minute when saving embeddings in a Chroma vector store. Here is my code: [...] # split the documents into chunks text_splitter = CharacterTextSplitter(chunk_size=1500, chunk_overlap=0) texts = text_splitter.split_documents(documents) # select which embeddings we want to use embeddings = OpenAIEmbeddings() # create the vectorestore to use as the index db = Chroma.from_documents(texts, embeddings)...

Originally asked by Heka on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

This happens because you can send a limited number of tokens to OpenAI. The solution I found is to feed it to OpenAI slowly. I expected Chroma to have a rate limiter, but I could not find such thing. Below code did it for me After you created your database for splitted_document in all_splits: vectorstore.from_documents(documents=[splitted_document], embedding=OpenAIEmbeddings(), persist_directory=base_path) time.sleep(60)

Full answer by Aaron Shayesteh: https://stackoverflow.stackexchange.com/a/76890712 (CC BY-SA 4.0).

Need it done for your business? UphillTech.