In my experience working with Debian, the rebuild is sensitive to disk usage. If you start using the disks, the rebuild will slow down intentionally so as to not choke the drives. This could explain why it was fast for you but not for the parent.
Edit: To clarify, I was using mdadm, not hardware raid.
Edit: To clarify, I was using mdadm, not hardware raid.