Question to the Storage Advisors, from Muneeb: We are trying to build a data streaming system for data acquisition purposes. It requires sustained writing capability at a rate of at least 100 MB/s. We have tried PCI and motherboard based RAID (SATA I/II), and even PCI based SCSI U320, but there is always a bottleneck at around 75 ~ 80 MB/s. What would be the best Adaptec (or another) controller to use (PCIe??) for sustained writing (reading speed can be slower) at 100 MB/s for at least 500 GB of data?

Muneeb, good question. And a common one at that. There are a lot of factors to take into account.

First, calculate the maximum write capability of your drives. Each drive should be able to sustain 50-100MB/s to and from the medium. Note that throughput to the outer section of the disk (lower block numbers) is substantially faster than throughput to the inner section, so make sure you use the smallest numbers provided in the drive’s spec sheet. Then, assuming that each drive has it’s own port to the controller or motherboard, simply add up all those throughputs and you’ll have the maximum throughput supported by your drive stack. You won’t ever get to that throughput, but it’s good to know what the high water mark is.

Next, make sure you’re using a RAID controller that doesn’t have something ancient like an ISA interface. I doubt you are, but if so, you’d be limimted to sub-100MB/s throughputs.

Next, pick your RAID level. RAID-0 performance will approach the high water mark that you calculated earlier. RAID-1 will be approximately half that. And RAID-5 will be something completely different and complicated that we’ll dive into next.

RAID-5 is can be written in two different ways. If the writes from the host are short, then each write will be converted to two disk reads and two disk writes. Performance will be at BEST 25% of your high water mark. And it will probably be much less than that.

However if the host writes are long and sequential, then the performance can theoretically approach that of RAID-0. But it’s tough to completely hide the overhead of RAID-5, so getting in the 80% ballpark is pretty good.

All of these RAID performance numbers are also affected by the controller or OS cache. For example, short sequential RAID-5 writes are typically very slow (because of the four IOs described earlier) but if they can combine in a write-back cache to create long sequential RAID-5 writes, then performance can easily jump 10X.

You will also need to characterize the access pattern from your host OS or application. Obviously if the access is mostly random you’d be lucky to be get 10MB/s. In fact, you should be measuring IO/sec instead of MB/s. And you also need to see what the transfer size is. If it’s short it will be hard to get efficient disk access. But if there is a write cache, as previously described, then those inefficient short IOs can become efficient long IOs.

As part of that access pattern check, you will also need to determine the host command queue depth. For example, if the host is issuing just one command at a time then only one drive at a time may be accessed, reducing throughput dramatically. But, again, a write cache can really help that situation.

BTW, drives also have a write cache that you may want to turn on. But beware that a drive’s cache doesn’t have a battery, and if you lose power you will lose data. A good RAID controller will have a battery option. If you care about your data, buy the battery option.

So this is a lot of info that I threw at you. And you may have trouble answering these questions. But rest assured that a good RAID controller can easily exceed 200MB/s. I hope this helps.

Good luck.

TT

This entry was posted on Friday, November 3rd, 2006 and is filed under Storage Interconnects & RAID, Advisor - Tom Treadway. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.