LLAMA-2 model download is failing

plumb_pete

I've got the token from huggingface and I was able to login with huggingface-cli. I also got approval to download 7B-hf model. When I started inference run, model download started well but it is now stuck at Map and it is not completing. What did I miss? Loading checkpoint shards: 100%| . . . | 2/2 [00:01<00:00, 1.83it/s] Data type of the model: torch.float32 Map: 0%| . . .| 0/2662 [00:00<?, ? examples/s]

Originally asked by RedFox on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Steps: Get approval from Meta for the model (e.g. meta-llama/Llama-2-70b-chat-hf) -> check in huggingface in model's page how to get Meta's authorization Create a read token from here : https://huggingface.co/settings/tokens pip install transformers Execute huggingface-cli login and provide the read token -> note the token is not shown in terminal but is there when u paste Run the script, should work Seems you've got the auth and the token, but try another model. For me the model meta-llama/Llama-2-70b-chat-hf worked but the model meta-llama/Llama-2-7b-chat-hf got stuck forever in one of the downloads.

Full answer by user5623611: https://stackoverflow.stackexchange.com/a/77980636 (CC BY-SA 4.0).

Need it done for your business? UphillTech.