I provided test for arc-easy results for Llama 2 7B. I use lm evaluation harness, I get: acc: 75.51 acc_norm: 73.86 In the Llama 2 technical report, arc-easy is reported as 75.2. But in some quantization papers I saw lower numbers, for example around 69.3, and in another table even 53.5 for acc_norm. I want to understand which part of the evaluation protocol causes this difference.
Originally asked by danlee on Stack Exchange (ai) (CC BY-SA 4.0).
