Comment by espadrine

How big of a perf improvement would result from using the architectural tweaks that Llama3 and others have put in place since GPT-2?