chromadb persistent client crashes when attempting to add a record to collection

penguin_pam

I created a persistent chromadb client and added a collection to it. collection = client.create_collection( name="TEST-COLLECTION", embedding_function=OpenAIEmbeddingFunction( api_key=config.OPEN_AI_EMBEDDINGS_API_KEY, model_name="text-embedding-3-small" ) ) In a separate script, I'm trying to populate this collection with embeddings. Each document is a string representation of a JSON client = chromadb.PersistentClient(config.CHROMADB_PATH)...

Originally asked by ska on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

The abrupt process exit is typical of HNSW issue (e.g. seg faults). These errors are more frequent (as per our observations) on Windows systems but Linux-based systems are not immune to this problem. I can't tell from the above which version of Chroma and what OS/arch this error occurs on. A workaround, users have found useful is to run Chroma in a docker container and then access it via the http client. docker run -p 8000:8000 chromadb/chroma:1.0.15 Then your python code: client = chromadb.HttpClient(host='localhost', port=8000) collection = client.get_collection(name="TEST-COLLECTION") doc_id = ... doc = ... metadate = ... collection.add( ids=[doc_id], documents=[json.dumps(doc)],...

Full answer by Trayan Azarov: https://stackoverflow.stackexchange.com/a/79722469 (CC BY-SA 4.0).

Need it done for your business? UphillTech.