sfoskett

OP here. Thanks for the comments and suggestions! I updated the post to include the full text of the email as well as a link to this discussion.

I wonder if anyone has any suggestions about what we can do to stop this kind of thing. Should we complain to someone or just warn people online? Is there anything that can be done to head off this kind of thing?

Great idea! I just updated the post with the entire text (unlinked)!

I'm glad to see so many folks listening/watching! The On-Premise IT Roundtable selects a controversial premise to discuss on each episode, and it appears that this was a good one!

If you'd like to join in a future discussion, I'd love to have you join us. I'm @SFoskett on Twitter.

Find me these “properly programmed devices”! This is the core of the issue: bad assumptions and poor QA let bugs like this have an outsized impact. It’s our fault, folks.

Thank you! I appreciate both tips!

There kinda isn't such a thing at this point. I sure wish there was! And this is the point behind the article: Wouldn't it be awesome if ZFS had continued developing as the super filesystem it looked like last decade? It would do all this and more! Frankly if ZFS supported re-balancing ("re-RAID-ing") and hybrid pools, it would be pretty much everything we all need. I do hope ZFS or Btrfs or something gets there.

I guess I'm an old storage guy. I call it the FAT on everything! :-)

I'm sorry, I was referring to hybrid storage as a technology category, not the hybrid disk drives like Seagate Momentus XT. You're right that a 2-drive hybrid (Fusion Drive) will mathematically be less reliable than a non-hybrid one and that those "hybrid disks" haven't lived up to the hype.

Pretty much every enterprise storage solution designed today is hybrid (SSD plus HDD) or all-flash and includes lots of advanced availability features.

I didn't even consider this. Thanks for the explanation! Makes tons of sense!

Incidentally, many modern filesystems (including NTFS) store very small files in the FAT rather than taking up a whole block for this very reason!

Yeah, as mentioned in the post, you can indeed just add a vdev to the pool whenever you want. But the risk, apart from performance, is data protection If you add one disk, your whole pool is at risk. So of course you have to add two or more disks as a mirror or RAIDZ but lots of home users don't want to do this. They want that Drobo thing where they throw another log on the fire and everything gets rebalanced. It's sad that ZFS can't do this, and that it can't "upgrade" RAID-Z1 to RAID-Z2. It's hard to do, but would be so much nicer.

Hybrid storage combines flash and disk for better performance and lower cost. Most hybrid storage is tiered, meaning that data can "live" on either flash or disk, but there are other approaches that look a lot more like a cache (see Nimble Storage, for example). True hybrid storage would work like Apple Fusion Drive or Hybrid Storage Spaces Direct - A single pool with SSD and HDD where data can reside wherever is best.

L2ARC is only ever a cache and although it can be on SSD, ZFS generally won't use much more than a few tens of GB. ZIL isn't even a cache and is only used for synchronous writes. Check any ZFS tuning guide and the gist will be "just buy more RAM or create an all-SSD pool" rather than trying to wedge an SSD into L2ARC or ZIL.

Enterprise Storage == Real Storage Arrays

Basically every storage array is years beyond ZFS in terms of features and capabilities. Even the ones based on ZFS! And I don't think anyone would argue that. After all, Dell EMC, HPE, Pure Storage, HDS, etc spend many millions a year developing advanced storage technologies.