Discussion

How to Move 8PB from NYC to SJC?

I have been asked to cooy 8PB from a customers datacenter in NYC to our Datacenter in SJC. I expect to start in 2 weeks and I want to be done 30 days later.

NYC dc is currently unknown (!). I should learn this next week. The data is a collection of files size 100k to 2MB Ave size 500k.

What's the best way to move it?

Options:

1 - Ship my storage servers out there. 24 core controller with 128g ram, SAS card with 2xHGST 60 disk 10tb JBOD and 40gig nic. We use Freenas. I could ship it out, copy the data and ship it back. I could either just ship the disks or the whole storage system.

Option 2 - aws snowball. I figure I can get 3gbps based on testing so the copy would take 30 days.

Option 3 - aws snowmobile - overkill for this?

Option 4 - find 1 or more temp 10g or 100g circuits

Option 5 - run it through aws direct connect assuming I can get this at the east coast dc. I have high bw aws dx in the sjc facility.

Option 6 - something else?

Your thoughts suggestions recommendations and experience appreciated!

5 points throwbigdata 9 years ago 11 comments

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton