Comment by msoad

Phi-3 blow this out of the water.

                      Benchmark  |  Gemma 2 (9B)  |  Phi-3 Small (7B)
    -----------------------------|----------------|-------------------
                  MMLU (5-Shot)  |       63.6     |       75.7
             HellaSwag (5-Shot)  |       49.8     |       77.0
                  ANLI (7-Shot)  |       48.7     |       58.1
           GSM-8K (8-Shot; CoT)  |       59.8     |       89.6
                 MedQA (2-Shot)  |       49.6     |       65.4
               AGIEval (0-Shot)  |       42.1     |       45.1
              TriviaQA (5-Shot)  |       72.3     |       58.1
                Arc-C (10-Shot)  |       78.3     |       90.7
                Arc-E (10-Shot)  |       91.4     |       97.0
                  PIQA (5-Shot)  |       78.1     |       86.9
                SociQA (5-Shot)  |       65.5     |       79.2
    BigBench-Hard (3-Shot; CoT)  |       59.6     |       79.1
            WinoGrande (5-Shot)  |       55.6     |       81.5
           OpenBookQA (10-Shot)  |       78.6     |       88.0
                 BoolQ (2-Shot)  |       66.0     |       84.8
        CommonSenseQA (10-Shot)  |       76.2     |       80.0
      TruthfulQA (10-Shot; MC2)  |       52.1     |       70.2
             HumanEval (0-Shot)  |       34.1     |       61.0
                  MBPP (3-Shot)  |       51.5     |       71.7