I was planning on a straight RAID 6 until I got enthused by ZFS…

Commodity desk top drives might fail slightly more than the enterprise ones but they cost less so I can get a few extras and in this configuration they should work fine.

Thanks for the interesting read!

If you look at this paper:

http://www.emulex.com/products/white/fcswitch/Advancing_Storage_Reliability_wp.pdf

you will see the typical claims for enhanced SBOD reliability through monitoring.

Here are the claims:

1) JBODs use serially connected FC-AL topologies. A rouge drive on an FC-AL segment may cause one or both FC-AL channels to become unavailable, potentially making all drives unavailable. SBOD can isolate a rogue drive more effectively.

2) SBOD allows a newly inserted drive to be dynamically tested before it is ever placed into service.

3) SBOD controllers are better set up to monitor traffic trends to see potential bottlenecks going into individual drives, which may reveal drives that are about to go bad, but do not yet report CRC errors.

4) SBOD controllers examine words on FC links to each drive looking for CRC errors. In JBOD, the source of CRC errors on the FC-AL can be elusive because of the shared, serial nature of the topology. SBOD can track down the exact port and Source ALPA of frames with CRC errors.

5) SBOD can perform a clock check comparing the relative frequency of attached devices (both drives and initiators). Identifying devices with out-of-tolerance clock frequencies allows them to be replaced under optimal system conditions before failure.

6) SBOD provides enhanced MTTRs because it can tell you right away which cabling error occured during repair.

The question is, does all that really matter, and if so, how much?

Yep, I agree that a drive won’t last 100 years. MTBF is just a statistical measurement that becomes invalid after so many years.

Regarding predictive failure analysis, also known as PFA, or SMART:

A vendor saying that RAID-5 with SMART is better than RAID-6 is a little misleading. Of course RAID-6 can also take advantage of SMART.

However I think I see the point they’re trying to make. Using SMART you can detect a bad drive before it fails and replace it, hopefully avoiding the condition of two simultaneous failures and thereby not requiring RAID-6.

But that’s not why we should use RAID-6. The main purpose of RAID-6 is to protect against bit errors detected during a rebuild. SMART can’t protect against that.

BTW, you mention an SBOD switch detecting SMART errors. I’ve never seen a switch that does that. RAID is done in an HBA or external controller. Why would a switch be checking for SMART errors, and if it saw one, wouldn’t it just inform the RAID stack anyway? Or maybe I’ve misunderstood how they’re defining an SBOD. I usually think of a SBOD as being a JBOD with a switch - and therefore it’s typically FC and soon SAS.

If you can tell me which vendor you’re talking about, I’d love to look into what they have.

Thanks for reading. I hope you find the posts useful.

TT