For some time I have a feeling that Apple actually IS playing the hardware game in the age of AI. Even though they are not actively innovating on the AI software or shipping products with AI, their hardware (especially the unified memory) is great for running large models locally.
You can't get a consumer-grade GPU with enough VRAM to run a large model, but you can do so with macbooks.
I wonder if doubling down on that and shipping devices that let you run third party AI models locally and privately will be their path.
If only they made their unified memory faster as that seems to be the biggest bottleneck regarding LLMs and their tk/s performance.
You can't get a consumer-grade GPU with enough VRAM to run a large model, but you can do so with macbooks.
I wonder if doubling down on that and shipping devices that let you run third party AI models locally and privately will be their path.
If only they made their unified memory faster as that seems to be the biggest bottleneck regarding LLMs and their tk/s performance.