Comment by snats

I also did a couple of experiments with pruning LLMs[1] using genetic algorithms and you can just keep removing a surprising amount of layers in big models before they start to have a stroke.

[1]https://snats.xyz/pages/articles/pruningg.html