Is it possible to include Ollama directly in my Python project?

spark_sara

I'd like to create a Python program with Ollama (llama3.2, or 3.3) included in my package, so that the user doesn't need to install it separately when installing my program. I've done it with Tesseract-OCR, but it doesn't seem to work there... Anyone knows if it is possible and how to do it? I've tried adding a folder containing all the Ollama files and dependencies, as you would for a Windows installation, to my Python project. I've done the...

Originally asked by alex_pythooon on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Yes, It's possible to bundle ollama with your Python package. It's more complex than tessracth tho. You should use docker as it's the best approach here for cross platform working. You can package Ollama with Llama3 and your python. Use Ollama image, Pull the llama3, Install python and dependecies, Copy your app into the container, EXPOSE ollama's default port, Finally start ollama as a background process and run your python app.

Full answer by Cem: https://stackoverflow.stackexchange.com/a/79451633 (CC BY-SA 4.0).

Need it done for your business? UphillTech.