assert self.ctx is not None AssertionError - TheBloke/Manticore-13B-GGML

plumb_pete

So i found this repo, and i've been trying to make it work in google colab. https://github.com/ecliipt/personal-assistant/tree/main i am using the Manticore-13B.ggmlv2.q5_1.bin model, i've already tried using ggmlv3 but got the same error. i'm using torch 2.0.1 on cpu and i also changed some things from the original repo's code. here's the error that i get when running retrievalQA.py (py 3.10.12): Loading the Manticore-13B.ggmlv2.q5_1.bin...

Originally asked by eclipt on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Good thread on this — worth a read before you pay anyone to build it.

Discussion: https://stackoverflow.com/questions/76533716/assert-self-ctx-is-not-none-assertionerror-thebloke-manticore-13b-ggml.

Want it set up for you? UphillTech.