Comment by mleo

Apple has the mlx framework. Most/all major software for running models locally support it. Apple also has RDMA for interconnecting multiple machines across Thunderbolt connections.