Running Ollama in Docker File for Python Langchain Application

ubuntu_ulysses

Background Info I have a python application that uses langchain and Ollama. Running this locally works perfectly fine because I have the Ollama client running on my machine. What I want to do is host this application on a serverless platform (like GCR, for example) and in order to do this I need to containerize the application. This is easy for the python side of the application but I am struggling to get the Ollama part right. I need the Ollama...

Originally asked by Brent on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Create a start_service.sh file: #!/bin/sh # Start Ollama in the background ollama serve & # Wait for Ollama to start sleep 5 # Pull and run <YOUR_MODEL_NAME> ollama run <YOUR_MODEL_NAME> In your Dockerfile: EXPOSE 11434 RUN chmod +x /app/start_services.sh # Run .sh file CMD ["/app/start_services.sh"] Hope this may help you. This is how it worked for me,

Full answer by Die Hard: https://stackoverflow.stackexchange.com/a/78539748 (CC BY-SA 4.0).

Need it done for your business? UphillTech.