Is PCI RAID clustering finally dead?
Posted in Storage Applications, Storage Interconnects & RAID, Advisor - Tom by Tom TreadwayI certainly hope so. Talk to any PCI RAID vendor (or customer) who has implemented clustering based on parallel SCSI and I guarantee that they will roll their eyes at you. And make sure you bring your lunch because they’re going to have a LOT of war stories about just how hard it was to pass Microsoft’s WHQL logo. Hopefully all that is about to change. (No, not the old boring stories - just the need for new ones going forward.) Microsoft has been putting out feelers to customers and vendors over the last several months regarding dropping logo testing in the Vista timeframe for PCI RAID clustering. Given the timeframe, it’s obvious that this is primarily a statement against PCI RAID SAS clustering.
I think you’ll see most RAID vendors, me included, dancing in the streets at this news. But it’s a pity since SAS will solve all the cabling nightmares of parallel SCSI. Many of the issues with clustering over parallel SCSI were caused by the terminators and repeaters strategically placed along the signal path. Just imagine connecting two hosts and a bunch of drives with auto-terminating cables and impedance mismatches galore. Yikes.
Obviously, the concept of clustering isn’t going away, so what replaces it? The answer is that it’s basically the same model that we see with FC today - the RAID needs to move out of the host and into external controllers. So instead of each host have a RAID controller, the hosts will have simpler non-RAID SAS port(s), which may someday be embedded for free in server chipsets. The RAID controller will move to the drive enclosure in the form of a SAS-to-SAS bridge controller. The functionality of the RAID controller is basically the same, but the PCI host interface will be replaced with a SAS host interface. Also, for redundancy, you should expect to have two of these controllers functioning in either an active/active or active/passive mode with automatic failover. So for a 4-node clustering system, the total cost could be expect to go down because the user is moving from 4 PCI RAID controllers to just 2 external RAID controllers.
There are also significant advantages in how the storage is provided to the hosts without the restraints of shared-none models and disk sets. But I’ll save those details for a later post if there’s enough interest.
The bottom-line is that Microsoft’s rejection of PCI RAID SAS clustering is the tough love we’ve all needed. Clustering has always been a critical technology, but it was never able to take off with the burden of parallel SCSI. Users were forced to implement much more expensive FC-based clustering solutions. That will finally change with the introduction of much cheaper external SAS solutions, as well as external iSCSI solutions, in 2006.
TT
March 14th, 2006 at 1:29 pm
Tom,
Does this mean that the Adaptec SAS RAID 4805SAS controllers won’t share a RAID volume for HA in a linux environment ? I can’t find it anywhere in the doc’s.
For us it would be a prime reason to choose SAS for a HA linux DB cluster. This would offer more performance then an iSCSI solution (pricewise).
Regards,
Dennis
March 24th, 2006 at 11:38 am
Dennis,
Just to be sure we’re talking about the same thing, I assume you’re talking about two or more Linux machines, each with a 4805 controller, all talking to the same set of disks. I also assume that each drive is physically owned by just one controller, i.e., there aren’t multiple RAID cards all trying to share the same array.
This is where the question gets a little tricky because in this weblog we try to avoid discussions on product features - especially features that are not yet announced.
However one way to build the configuration I described above is to use SAS switches with zoning. I don’t think these parts are shipping yet, but I think we should start seeing them in 2006. This would allow each SAS drive to be assigned to just one RAID controller. During failover, some management software magic would occur to move ownership of those drives to another controller.
Would that work for you?
TT
March 30th, 2006 at 11:23 am
Tom,
Yes that’s exactly what I meant. Each controller would ‘own’ it’s own set of disks and in cases of a server failure the standby server or designated failover server would take over the ‘orphaned’ set of disks.
I’ve made the decision of combining an iSCSI head server with an external SAS JBOD and a coldspare server. This would still keep the price down compared to FC and keep the door open for further growth with SAS.
This is very much a wait and see on what kind of SAS products will emerge in the comming year. I would love to see external cabinets with all the RAID logic internally and a connection to the SAS domain announcing itself as a RAID volume / multiple RAID volumes.
As I’ve said in reply to your other article with some imagination there are some very nice setups that will be made possible with the right SAS gear.
Regards,
Dennis
April 15th, 2006 at 4:30 pm
I too am implementing and ISCSI head server wit an external SAS JBOD. Since every diagram of an SAS domain shows that is it capable of supporting multiple initators, I’ve been searching for information on just how that works. Can you please explain the requires to support this environment? I’m not specifically speaking about clustering, but rather the ability to connect multiple initators to the the storage resources within the same domain. Of course each host would have it’s own resources.
Regarding clustering and SAS, here is some great info right from the horse’s mouth as it were:
http://download.microsoft.com/download/5/7/7/577a5684-8a83-43ae-9272-ff260a9c20e2/WinHECwhitepaper2003-LSI.doc
April 17th, 2006 at 6:31 am
Errol,
First, starting with the whitepaper: Thanks for sending the link. It’s a good write-up on MPIO and clustering from our friends at LSI. Note that in some examples they show a PCI SAS RAID card, but never in a clustering configuration - only in an MPIO configuration. So I don’t think the paper contradicts the basic premise that Microsoft won’t be certifying PCI SAS RAID in a clustering environment.
Regarding multiple initiators in the same domain: The easiest way to do that is with SAS expanders supporting zoning. The zoning will allow a shared-none model, i.e., each initiator can have sole access to a subset of the drives. No other initiator will be able to access those drives. Of course the trick is figuring out how to configure the zoning. This configuration is done in-band via SMP, but I’m not aware of any native support in Windows (or any other OS) to configure these expanders. PMC-Sierra seems to have a head-start on zoning. You can probably find something on their website regarding how to configure a shared storage domain.
TT
October 14th, 2006 at 1:56 am
Of course each host would have it’s own resources and they have not to be dependable at all.