One thing I don't see anyone mention is that the M1Max is the probably the cheapest GPU memory you can buy. The only other way to get >=64Gb GPU memory is with the A100 (?) which is like 20k by itself.
So this would be great specifically for finetuning large transformer models like GPT-J, which requires a lot of memory but not a lot of compute. Just hoping for pytorch support soon..
So this would be great specifically for finetuning large transformer models like GPT-J, which requires a lot of memory but not a lot of compute. Just hoping for pytorch support soon..