How can I run Ollama in Google Colaboratory?

reno_queen

I have code like this. And I'm launching it. I get an ngrok link. !pip install aiohttp pyngrok import os import asyncio from aiohttp import ClientSession # Set LD_LIBRARY_PATH so the system NVIDIA library becomes preferred # over the built-in library. This is particularly important for # Google Colab which installs older drivers os.environ.update({'LD_LIBRARY_PATH': '/usr/lib64-nvidia'}) async def run(cmd): ''' 'run' is a helper function to run...

Originally asked by Sergey Oblasov on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

1. Run ollama but don't stop it !curl https://ollama.ai/install.sh | sh # should produce, among other thigns: # The Ollama API is now available at 0.0.0.0:11434 This means Ollama is running (but do check to see if there are errors, especially around graphics capability/Cuda as these may interfere. However, Don't run !command -v systemctl >/dev/null && sudo systemctl stop ollama (unless you want to stop Ollama). The next step is to start the Ollama service, but since you are using ngrok I'm assuming you want to be able to run the LLM from other environments outside the Colab? If this isn't the case, then you don't really need ngrok, but since Colabs are tricky to get working nicely with...

Full answer by Gruff: https://stackoverflow.stackexchange.com/a/77828874 (CC BY-SA 4.0).

Need it done for your business? UphillTech.