Great question — answering as the UphillTech team:
@Vishnu Priya Vangipuram 's answer runs the model llama3 , not your custom model with ./Modelfile , while @Jinna Baalu 's omits the step of loading a model. Here is the setup you can use when you need to make inference with your custom model using Modelfile . Modelfile and the GGUF file file_name.gguf should both be located in the local directory ./model_files , and compose.yaml should be in the parent directory. compose.yaml services: ollama: # New service for running the Dockerfile in /ollama image: ollama/ollama:latest pull_policy: always container_name: ollama ports: ["11435:11434"] # will be accessible in http://localhost:11435 volumes: - ./model_files:/model_files # Mount the...
Full answer by Sayyor Y: https://stackoverflow.stackexchange.com/a/79028150 (CC BY-SA 4.0).
Need it done for your business? UphillTech.
