I have installed ctransformers using - pip install ctransformers[cuda] I am trying following piece of code - from langchain.llms import CTransformers config = {'max_new_tokens': 512, 'repetition_penalty': 1.1, 'context_length': 8000, 'temperature':0, 'gpu_layers':50} llm = CTransformers(model = "./codellama-7b.Q4_0.gguf", model_type = "llama", gpu_layers=50, config=config) Here gpu_layers parameter is specified still gpu is not being used and...
Originally asked by Yogesh Tembe on Stack Exchange (stackoverflow) (CC BY-SA 4.0).
