How can I install llama-cpp-python with cuBLAS using poetry?

reno_queen

I can install llama cpp with cuBLAS using pip as below: CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python However, I don't know how to install it with cuBLAS when using poetry. Installation is possible, but cuBLAS Acceleration is not available. I checked that I can use cuBLAS when I installed it with pip in my environment. I added llama-cpp-python dependency to the pyproject.toml file as below: [tool.poetry.dependencies]...

Originally asked by KimuGenie on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

I encountered a similar issue and found a workaround. While Poetry doesn't directly support passing environment variables like pip, I used poetry run pip install as a temporary solution. This approach involves setting the necessary environment variables and then running: poetry run pip install llama-cpp-python --upgrade --force-reinstall --no-cache-dir This method allowed me to install llama-cpp-python with CU-BLAS support, which I couldn't achieve solely with Poetry. It's important to note that this bypasses Poetry's dependency resolution, so use it cautiously and document it in your project. I tried using https://github.com/volopivoshenko/poetry-plugin-dotenv/ , but was still not getting...

Full answer by Luis Antunes: https://stackoverflow.stackexchange.com/a/77622933 (CC BY-SA 4.0).

Need it done for your business? UphillTech.