How to start ollama with docker compose with specific LLM model

taco_truck_tom

I have a docker compose where a dummy python app is using ollama LMs on the background for some tasks. I want to be able to tell ollama somehow which model to download and use on the app deployment. Now my docker compose is like this: version: '3.8' services: elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.10.0 container_name: elasticsearch environment: - discovery.type=single-node - "ES_JAVA_OPTS=-Xms512m -Xmx512m" ports:...

Originally asked by KZiovas on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Ok so I show somewhere that we can call ollama to install the model we want after the service starts. So I now specify the model with an env variable called LLM_MODEL_VERSION . After my ollama service starts I have an entry point bash script called ollama_entrypoint.sh : #!/bin/bash echo "Starting Ollama..." /bin/ollama serve & pid=$! echo "Waiting for Ollama to start..." until ollama list > /dev/null 2>&1; do sleep 2 done MODEL_NAME=${LLM_MODEL_VERSION} if ollama list | grep -q "$MODEL_NAME"; then echo "Model $MODEL_NAME already available." else echo "Retrieving model: $MODEL_NAME" ollama pull "$MODEL_NAME" echo "Model $MODEL_NAME is ready!" fi wait $pid So my new yaml is: version: '3.8'...

Full answer by KZiovas: https://stackoverflow.stackexchange.com/a/79393538 (CC BY-SA 4.0).

Need it done for your business? UphillTech.