The M-series are ARM chips based on the A series from the iPhones. Therefore Apple had ten years of chip design (plus the prior decades of ARM history) to rely on. The GPUs and modems were a much more recent effort.
(My take. happy to be corrected by someone with chip design experience who can comment.)
(My take. happy to be corrected by someone with chip design experience who can comment.)