llama-cpp-python installing for x86_64 instead of arm64

grill_gus

I am trying to set up local, high speed NLP but am failing to install the arm64 version of llama-cpp-python. Even when I run CMAKE_ARGS="-DLLAMA_METAL=on -DLLAMA_METAL_EMBED_LIBRARY=on" \ pip install llama-cpp-python --no-binary llama-cpp-python --force-reinstall --no-cache-dir I still get the same error when I try to import it: RuntimeError: Failed to load shared library...

Originally asked by Dennis Losett on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Good thread on this — worth a read before you pay anyone to build it.

Discussion: https://stackoverflow.com/questions/79675288/llama-cpp-python-installing-for-x86-64-instead-of-arm64.

Want it set up for you? UphillTech.