How to get Llama-2 Rotary Embeddings?

kernel_kate

I want to get the Llama-2 rotary embeddings. I do print(model) and get the following output: In the picture I highlight the rotary embeddings. How can get the rotary embeddings and how can I interpret the output? What means 32x LLamaDecoderLayer and in its round brakets are four layer plus LlamaRotaryEmbeddings? It's possible to get the embeddings as the first hidden-state hidden_state[0] and I want to know, which hidden-state represents the...

Originally asked by Christian01 on Stack Exchange (ai) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

The 32x LLamaDecoderLayer refers to a specific configuration of the LLamaDecoder model. It consists of 32 layers, each of which includes LlamaRotaryEmbeddings. The round brackets indicate that there are four layers in total, including the LlamaRotaryEmbeddings layer. Regarding the hidden states, the first hidden state hidden_state[0] represents the output of the first layer in the model. Each subsequent hidden state hidden_state[i] represents the output of the i-th layer in the model. Therefore, hidden_state[0] does not specifically represent the rotary embeddings, but rather the output of the first layer in general. There can be multiple rotary embeddings depending on the specific...

Full answer by user76624: https://ai.stackexchange.com/a/42251 (CC BY-SA 4.0).

Need it done for your business? UphillTech.