It does incrementally make a difference. You want all your libraries to stay under a certain budget if you don't want to hit your users with a 500 KB bundle size.
It is not only about how long it takes to download, it is also about parsing that amount of code in low end phones.
It is not only about how long it takes to download, it is also about parsing that amount of code in low end phones.