Comments on: Tom’s got nice SAS http://storageadvisors.adaptec.com/2006/04/10/toms-hardware-loves-sas/ Storage Solutions for Real World IT Professionals Mon, 15 Mar 2010 17:56:08 +0000 http://wordpress.org/?v=1.5.2 Comment on Tom’s got nice SAS by: Tom http://storageadvisors.adaptec.com/2006/04/10/toms-hardware-loves-sas/#comment-28844 Fri, 26 Jan 2007 19:33:09 +0000 http://storageadvisors.adaptec.com/2006/04/10/toms-hardware-loves-sas/#comment-28844 Wow! At first I assumed that it was just using InfiniBand for inter-controller communication, but this is a real InfiniBand connection to the host. Cool. Thanks for sharing. BTW, did you mean to post this in the <a href="http://storageadvisors.adaptec.com/2005/12/03/infiniband-history/">InfiniBand thread</a>?. No big deal either way. Like you said, both threads are pretty stale. But extra points to you for finding it and commenting! :-) Wow! At first I assumed that it was just using InfiniBand for inter-controller communication, but this is a real InfiniBand connection to the host. Cool. Thanks for sharing.

BTW, did you mean to post this in the InfiniBand thread?. No big deal either way. Like you said, both threads are pretty stale. But extra points to you for finding it and commenting! :-)

]]> Comment on Tom’s got nice SAS by: Trevor http://storageadvisors.adaptec.com/2006/04/10/toms-hardware-loves-sas/#comment-28841 Fri, 26 Jan 2007 18:34:23 +0000 http://storageadvisors.adaptec.com/2006/04/10/toms-hardware-loves-sas/#comment-28841 This thread is a little stale, but I'll tack on anyway. Well, it looks like your wish has come true XiraNet now has a device that supports both 10 GB and Infiniband. http://www.xiranet.com/43.0.html This thread is a little stale, but I’ll tack on anyway.
Well, it looks like your wish has come true XiraNet now has a device that supports both 10 GB and Infiniband.

http://www.xiranet.com/43.0.html

]]> Comment on Tom’s got nice SAS by: Tom http://storageadvisors.adaptec.com/2006/04/10/toms-hardware-loves-sas/#comment-460 Tue, 25 Apr 2006 14:32:14 +0000 http://storageadvisors.adaptec.com/2006/04/10/toms-hardware-loves-sas/#comment-460 Are you talking about a SAS-to-SAS RAID controller that also has an Ethernet option? If so, I agree completely. That's exactly how remote replication should be done. SAS doesn't have the required cable lengths to even get out of the rack, and FC is just rediculously expensive. iSCSI, or some other proprietary protocol, is the way to go. Unfortunately it requires either a TOE or a big improvement in the SAS RAID CPU to get decent performance. But as x86 prices come down, and the x86 CPUs and chipsets become more integrated, even the cheapest controller will have a a ton of GHz to throw at the problem. TT Are you talking about a SAS-to-SAS RAID controller that also has an Ethernet option? If so, I agree completely. That’s exactly how remote replication should be done. SAS doesn’t have the required cable lengths to even get out of the rack, and FC is just rediculously expensive. iSCSI, or some other proprietary protocol, is the way to go. Unfortunately it requires either a TOE or a big improvement in the SAS RAID CPU to get decent performance. But as x86 prices come down, and the x86 CPUs and chipsets become more integrated, even the cheapest controller will have a a ton of GHz to throw at the problem.

TT

]]> Comment on Tom’s got nice SAS by: Allen http://storageadvisors.adaptec.com/2006/04/10/toms-hardware-loves-sas/#comment-435 Mon, 24 Apr 2006 23:14:09 +0000 http://storageadvisors.adaptec.com/2006/04/10/toms-hardware-loves-sas/#comment-435 &#62;&#62; a dual-ported JBOD enclosure is probably most interesting &#62;&#62; behind a dual-controller RAID enclosure That's the killer app for SAS. The high throughput/low latency and low cost of U320 with the natural and easy fault tolerance of FC and the flexibility of iSCSI [1]. Whichever tier 1 storage vendor builds it first will make their year early. It was said this way in another article: "SAS is the SAN people really want." I know I do. [1] iSCSI is here to stay regardless of SAS for two reasons, WAN being only the most obvious. Less obvious: true block level access for Virtual Machines. There's no other (good) way to do this with without bending things a lot. Google this to see what I mean: iscsi site:vmware.com Quarter TiB .vhd files are cool in a perverse sort of way, but they are effectively opaque to the rest of the system. Is there a reason a SAS RAID head/enclosure couldn't also have a few (optional, user installable, thank you) Ethernet ports to leverage the perfectly good RAID engine...? >> a dual-ported JBOD enclosure is probably most interesting
>> behind a dual-controller RAID enclosure

That’s the killer app for SAS. The high throughput/low latency and low cost of U320 with the natural and easy fault tolerance of FC and the flexibility of iSCSI [1]. Whichever tier 1 storage vendor builds it first will make their year early.

It was said this way in another article: “SAS is the SAN people really want.” I know I do.

[1] iSCSI is here to stay regardless of SAS for two reasons, WAN being only the most obvious. Less obvious: true block level access for Virtual Machines. There’s no other (good) way to do this with without bending things a lot. Google this to see what I mean:

iscsi site:vmware.com

Quarter TiB .vhd files are cool in a perverse sort of way, but they are effectively opaque to the rest of the system. Is there a reason a SAS RAID head/enclosure couldn’t also have a few (optional, user installable, thank you) Ethernet ports to leverage the perfectly good RAID engine…?

]]> Comment on Tom’s got nice SAS by: Tom http://storageadvisors.adaptec.com/2006/04/10/toms-hardware-loves-sas/#comment-411 Mon, 24 Apr 2006 13:24:06 +0000 http://storageadvisors.adaptec.com/2006/04/10/toms-hardware-loves-sas/#comment-411 Allen, IMHO, I see three ways to use a JBOD box with redundant IO modules: (1) Single controller in the host, connected via two redundant cables to the enclosure. This allows cable failover. (2) Dual controllers in the host, each connected via a single cable to the enclosure. This allows controller failover. (3) Dual servers, each with a controller connected via a single cable (or perhaps dual) to the enclosure. This allows server failover. Starting in reverse order, the dual server approach smells like clustering. And Microsoft is not interested in propogating support of PCI RAID clustering with SAS. Pity, since the cabling is so much nicer than SCSI. I made a <a href="http://storageadvisors.adaptec.com/2005/10/24/is-pci-raid-clustering-finally-dead/">post</a> on this topic earlier. Dual controllers in a server is interesting, but probably overkill. Controllers don't/shouldn't go bad. Servers (due to power supplies, fans, etc.) and cables go bad. So I'm not crazy about this solution. Plus, it requires an MPIO framework under Windows, and I'm not sure what framework under Linux. So that leaves us with the first usage, which I think is the most interesting. Cables do fail and they do fall out. Or someome yanks the wrong cable. And it's not hard to support this feature - the RAID controller "just" needs to support detecting the same storage via multiple paths, and then avoid presenting twice the amount of real storage to the host. If a cable fails, then the controller either "remembers" the other path to the storage, or re-detects the topology and finds the remaining path. Another cool use of the dual paths is for added performance, i.e., using both paths to actively transfer data. But this is probably a little over the top since a 4X SAS cable can already support 1.2GB/s and a TON of random IOPs. Lastly, and I think you referred to this in your comment, a dual-ported JBOD enclosure is probably most interesting behind a dual-controller RAID enclosure, such as a FC-to-SAS or iSCSI-to-SAS enclosure. (And someday a SAS-to-SAS enclosure for great performance and the lowest cost point.) In this configuration, each RAID controller would connect to the dual-ported JBOD enclosure for cable and controller redundancy. This is all great stuff, because I believe that most people with multiple servers go down the expensive FC SAN road because that was their only choice. SAS makes a lot more sense - especially if you can use cheap <a href="http://storageadvisors.adaptec.com/2005/12/16/can-sata-breach-the-chasm/">SATA drives protected by RAID-6</a>. Thanks for reading. TT Allen,

IMHO, I see three ways to use a JBOD box with redundant IO modules:

(1) Single controller in the host, connected via two redundant cables to the enclosure. This allows cable failover.

(2) Dual controllers in the host, each connected via a single cable to the enclosure. This allows controller failover.

(3) Dual servers, each with a controller connected via a single cable (or perhaps dual) to the enclosure. This allows server failover.

Starting in reverse order, the dual server approach smells like clustering. And Microsoft is not interested in propogating support of PCI RAID clustering with SAS. Pity, since the cabling is so much nicer than SCSI. I made a post on this topic earlier.

Dual controllers in a server is interesting, but probably overkill. Controllers don’t/shouldn’t go bad. Servers (due to power supplies, fans, etc.) and cables go bad. So I’m not crazy about this solution. Plus, it requires an MPIO framework under Windows, and I’m not sure what framework under Linux.

So that leaves us with the first usage, which I think is the most interesting. Cables do fail and they do fall out. Or someome yanks the wrong cable. And it’s not hard to support this feature - the RAID controller “just” needs to support detecting the same storage via multiple paths, and then avoid presenting twice the amount of real storage to the host. If a cable fails, then the controller either “remembers” the other path to the storage, or re-detects the topology and finds the remaining path.

Another cool use of the dual paths is for added performance, i.e., using both paths to actively transfer data. But this is probably a little over the top since a 4X SAS cable can already support 1.2GB/s and a TON of random IOPs.

Lastly, and I think you referred to this in your comment, a dual-ported JBOD enclosure is probably most interesting behind a dual-controller RAID enclosure, such as a FC-to-SAS or iSCSI-to-SAS enclosure. (And someday a SAS-to-SAS enclosure for great performance and the lowest cost point.) In this configuration, each RAID controller would connect to the dual-ported JBOD enclosure for cable and controller redundancy.

This is all great stuff, because I believe that most people with multiple servers go down the expensive FC SAN road because that was their only choice. SAS makes a lot more sense - especially if you can use cheap SATA drives protected by RAID-6.

Thanks for reading.

TT

]]>