Comment

Comment by samptemp

I'm kinda a little disappointed at the answers provided here.

1) I downloaded a 49.6 MB TIFF file from here: http://hubblesite.org/newscenter/archive/releases/2004/32/im...

2) From the OSX terminal:

$ time sips -Z 150 hs-2004-32-d-full_tif.tif

Resulting file size is 150x150px and 32KB.

Time to run process: 1.48s user 0.15s system 98% cpu 1.663 total

The superpower computer required for this incredible speed is a 3 year old MacBook Pro (with SSD) ;-)

UPDATE:

I created 10 copies of the image. Total size: 520.1 MB

$ time sips -Z 150 .tif

Time to process 10 images: 14.98s user 1.44s system 94% cpu 17.384 total (approximately 1.498 per image).

2000000 images 1.663 seconds = 38 days.

Note: This is using larger images (49.6MB vs the 25-30MB images you have) and is using a single MacBook Pro. Divide amongst a few machines and be done in a week. GraphicsMagick could possibly be even faster.

ANOTHER UPDATE:

Since your files are smaller (20-30MB), I found a 30MB jpg sample here: http://sto-rvlt-01.sys.comcast.net/speedtest/random4000x4000...

Time to process was much faster: 0.37s user 0.04s system 97% cpu 0.418 total

2000000 images * 0.418 seconds = 9.67 days

At that speed we are talking under 10 days. On a single 3 year old MacBook Pro. Using built-in software. Without any optimizations. Find 5 computers in the NY Public Library and you'll be done in 2 days.

I am not saying this is the fastest solution. I posted this because it appears that people are over-engineering this problem or proposing solutions which will cost a lot of money (Amazon, bandwidth, shipping hard drive fees, etc).

parent samptemp 12 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton