M1: Each CPU has four memory channels, each node has 16 DIMMs (8/CPU, 2 DIMMs/channel) which means that you can use 1 DIMM per channel -> maximum memory bandwith and speed with RDIMMs. Using 64GB or 128GB LRDIMMs with E5v4 CPUs won't affect your bandwith or speed as long as you populate all the channels.[0]
Your memory options are:
1TB - 16x64GB / 8x128GB
2TB - 16x128GB
[0] - SuperMicro X10DRT-PT Motherboard manual page 35(2-13).