SATA RAID with ICH5 on FreeBSD

Intro

After days of floundering around with hardware RAID and vinum under FreeBSD, and a month or more of wound licking, I opted to revisit the topic of RAID on my new SuperMicro servers with the latest 5.2-RELEASE. The results were successful, but not in a way that I had originally expected.

So the primary ingredients in this mix are:

The Problem

About the SuperServer 5013C-T: In case you don't follow the link, this is a 1U Pentium 4 based server. It is using some of the latest in Intel chipsets and supports Hyperthreading CPUs up to 3.2GHz, Serial ATA (SATA) drives, and other fun stuff. The SATA drives are in swappable bays and the system has two (count 'em, 2, Sparky) Intel Gigabit ethernet interfaces. For drives, I am using the Western Digital 36GB Raptor drives (5 year warrantee).

One of the selling point of the Intel 875 chipset the server uses is that it has their great new ICH5 SATA hardware RAID controller. Unfortunately, this has really crappo support right now in the realm of Open Source OSes. No one seems to have good driver support for it at all yet. Still, you have a good box with a built-in SATA for two drives, so the next step was to try the software RAID offerings.

The longstanding offering from FreeBSD for doing various RAID-like activities has been vinum. The documentation on vinum is, well... painful. There is lots of it out there, but it all seems to be pretty much slightly varied takes on the original docs. Also, I won't get into the details, but the steps to get a system installed with mirrored booting and root file systems are quite odious. Perhaps I am just impatient, or unattentive... either way it was too much work. I gave up and tried to forget the pain.

...time passed...

The Solution

So... FreeBSD 5.2 has just been released, and I thought, "What the hey... I'll give the latest release a try and see if they improved support for the ICH5 SATA controller!"

Well, there has been no obvious major progress... but while searching a bit on the topic I came across someone else's query on the topic, to which someone said something to the effect of, "Why not use atacontrol?"

atacontrol is a tool that has been kicking around in FreeBSD for a while, but I hadn't tried it because it seemed to be directed at managing other hardware ATA RAID setups. However, it ends up that this works like a charm (just about) with the built in goods. Here is how I did it:

  1. The first thing I had to do was change the mode that the SATA controller was in. There are around 5 modes that it can be in, I won't get into their details, but the one I wanted was "Combined Mode" where each SATA port was considered to be its own IDE channel by the BIOS. SO there is one channel for each of the two the legacy parallel ATA interfaces and one channel for each SATA drive.
  2. Next I had to create the RAID container to install into. The way that it worked for me, which was the easiest option based on the media I had available, was that I did a really minimal install on the first SATA drive, ad4, and rebooted the system off of it. I could then say atacontrol create RAID1 ad4 ad6 which creates a new (but completely uninitialized) RAID container, ar0 consisting of my first and second SATA drives.
  3. Then I could reboot off of my install CD and, besides recognizing drives at ad4 and ad6, ar0 was offered as a destination for the install. So I installed. And it was good.

So now I have a software based RAID mirror of my two SATA drives. It is sort of hot swappable... I can't just yank an arbitray drive (I have tried this, the system just hangs), but using atacontrol detach # I can take down an individual drive and then pull it from the live system. The RAID continues to run in a degraded mode. I can then re-insert the same drive, do an atacontrol attach # and atacontrol rebuild 0, and my RAID will rebuild itself in the background. The system is also quite capable of booting when wither one of the drives is removed. (Note: The drives were labeled in "dangerously dedicated" mode.)

So, that is about it, I am happy with my results with it, and hopefully you will find this tidbit useful.... and the one thing that I neglected to mention was what the cost of the system is, about $1600 at the time of orignal purchase (Aug 2003) with a 2.8GHz CPU and 512MB RAM. Not too shabby. 

techno-obscura : delgado : notes