Well that is lamer than expected. The RLHF censorship was expected, but no 30B model, and single digit benchmark improvements with 40% more data? Wat. Some of the community fine tunes managed better than that. The 4k context length is nice, but RoPE makes it irrelevant anyway.
Edit: Ah wait, it seems like there is a 34B model as per the paper: "We are releasing variants of Llama 2 with
7B, 13B, and 70B parameters. We have also trained 34B variants, which we report on in this paper
but are not releasing due to a lack of time to sufficiently red team."
Edit: Ah wait, it seems like there is a 34B model as per the paper: "We are releasing variants of Llama 2 with 7B, 13B, and 70B parameters. We have also trained 34B variants, which we report on in this paper but are not releasing due to a lack of time to sufficiently red team."