Comment by cjjuice

Right now I am just restricting it to devices with 8GB RAM. I plan to start testing older phones soon to see if I can get it running well on them. MoE with low active params is interesting. I wonder if there is smaller models based on specific pairs (En/Es, ect.)