Using CalderaAI/Lazrus for CausalLM on a local server

fixit_fiona

I need to run a LLM on a local server and need to download different model to experiment. I am trying to follow this guide from HuggingFace https://huggingface.co/docs/transformers/installation#offline-mode To begin with, I picked "CalderaAI/30B-Lazarus". I ran the script from transformers import AutoTokenizer, AutoModelForSeq2SeqLM model_name="CalderaAI/30B-Lazarus" tokenizer = AutoTokenizer.from_pretrained(model_name) model =...

Originally asked by zoomraider on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Try using this: model = AutoModelForCausalLM.from_pretrained("CalderaAI/30B-Lazarus")

Full answer by Harshad Patil: https://stackoverflow.stackexchange.com/a/76495723 (CC BY-SA 4.0).

Need it done for your business? UphillTech.