{"id":370,"date":"2012-02-26T18:14:25","date_gmt":"2012-02-26T18:14:25","guid":{"rendered":"http:\/\/joelinoff.com\/blog\/?page_id=370"},"modified":"2012-02-26T21:51:51","modified_gmt":"2012-02-26T21:51:51","slug":"supermicro-6036st-6lr-lsi-sas2008-it-hba-mode-for-zfs","status":"publish","type":"page","link":"https:\/\/joelinoff.com\/blog\/?page_id=370","title":{"rendered":"SuperMicro 6036ST-6LR LSI SAS2008 IT HBA Mode for ZFS"},"content":{"rendered":"

I am building a filer based on OpenIndiana\/ZFS using the SuperMicro 6036ST-6LR super server because I need High Availability (HA). To use ZFS efficiently I want to use the SAS controllers as I\/T HBA’s.<\/p>\n

1 The Problem<\/h1>\n

I flashed the controllers using PH11 from ftp:\/\/ftp.supermicro.com\/driver\/SAS\/…8\/IT\/Firmware\/<\/a> and everything seemed to work fine. The flash completed successfully but I could not see the SAS expander and the 8 SAS drives in the SAS BIOS topology list when I rebooted. I also expected to see them on the Boot page of the MB BIOS as well. The MB BIOS did show the SAS controller (but no drives).<\/p>\n

Note that I also did not see the drives with the original version of the firmware.<\/p><\/blockquote>\n

1.1 Screen Shot – LSI SAS 2008 Controller Config Utility<\/h2>\n

During the boot process I type Ctrl-C and the controller was recognized.<\/p>\n

\"\"<\/a><\/p>\n

1.2 Screen Shot – LSI SAS 2008 Controller Topology<\/h2>\n

I then navigated to the topology screen and saw this. As you can see, there are no devices and no SAS expander. Yesterday, following the same procedure, I saw the SAS controller in the boot menu (but with no devices). I have no idea why it didn’t show up this time.<\/p>\n

\"\"<\/a><\/p>\n

1.3 Screen Short – BIOS Boot Menu<\/h2>\n

I then looked at the BIOS boot menu and the only recognized device was the USB I had plugged in.
\n\"\"<\/a><\/p>\n

1.4 It should work<\/h2>\n

This setup should work.<\/p>\n

There are vendors out there like Nexenta and VA technologies that run ZFS on these chassis’s. I am sure they are not using RAID because ZFS runs better without it. <\/p>\n

2 Where am I looking for help<\/h1>\n

This is a brief summary things I tried after reading all of the technical documentation that I could find, reseated the boards, checked the components, talked to friends, tried booting different OS’s and everything else that I could think of.<\/p>\n

2.1 SuperMicro<\/h1>\n

This is the obvious first place. I spoke to a customer service person and sent them email on Friday. They were very helpful but they only offer service on weekdays (even though the SuperServer hot-line says 24 hours which I thought meant 24\/7 not 24\/5).<\/p>\n

2.2 ServeTheHome<\/h2>\n

Here is another link to where I am trying to get help: http:\/\/forums.servethehome.com\/showthread.php?505-Supermicro-6036ST-6LR-SAS-2008-controller-IT-mode&p=3598#post3598<\/a>.<\/p>\n

They were very helpful. Their feedback helped confirm that I am not losing my mind.<\/p>\n

2.3 Google\/Bing<\/h2>\n

Searching did not really help with this problem much but it did eventually point me to ServeTheHome.<\/p>\n

3 Parts List<\/h1>\n

Here is my parts list:<\/p>\n\n\n\n\t\n\t\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t
Part<\/th>Qty    <\/th>Description<\/th>\n\t<\/tr>\n<\/thead>\n
SuperMicro 6036ST-6LR<\/td>1<\/td>High end storage bridge bay (SBB) comprised of the SC937 chassis and two X*DTS-F dual processor serverboards.
\r\n
\r\nIt also has a SAS mid-plane (BPN-SAS-937-O-P) and sixteen HDD carriers.
\r\n
\r\nTwo SAS controller mezanine cards (AOM-SAS2-L8).<\/td>\n\t<\/tr>\n\t
HDD<\/td>8<\/td>Seagate Constellation ST1000NM0001 ES SAS 6GB\/s 1TB
\r\n<\/td>\n\t<\/tr>\n\t
CPU<\/td>2<\/td>Intel Xeon E5645<\/td>\n\t<\/tr>\n\t
RAM<\/td>6<\/td>Kingston KST KVR1333D3\/2GR R 2GB<\/td>\n\t<\/tr>\n\t
Ethernet Card<\/td>1<\/td>SuperMicro AOC-SG-I2
\r\n2 port Gbps ethernet LAN card<\/td>\n\t<\/tr>\n<\/tbody>\n<\/table>\n\n

4 Firmware Download<\/h1>\n

Here is the script I used to download the firmware flash files:<\/p>\n

[crayon lang=”bash” toolbar=”always” title=”Firmware\/BIOS Flash Data”]
\n#!\/bin\/bash
\numask 0<\/p>\n

FTPROOT=’ftp:\/\/ftp.supermicro.com\/driver\/SAS\/LSI\/2008\/IT\/Firmware’<\/p>\n

if [ ! -d ‘fw’ ] ; then
\n mkdir fw
\nfi<\/p>\n

cd fw<\/p>\n

if [ ! -e PH10.zip ] ; then
\n echo wget ${FTPROOT}\/PH10.zip
\n wget ${FTPROOT}\/PH10.zip
\n rm -rf PH10
\nfi<\/p>\n

if [ ! -d PH10 ] ; then
\n unzip PH10.zip
\nfi<\/p>\n

if [ ! -e PH11.zip ] ; then
\n echo wget ${FTPROOT}\/PH11.zip
\n wget ${FTPROOT}\/PH11.zip
\n rm -rf PH11
\nfi<\/p>\n

if [ ! -d PH11 ] ; then
\n unzip PH11.zip
\nfi<\/p>\n

pdfs=(
\n ‘SAS2BIOS_PH10-7.19.00.00.pdf’
\n ‘SAS2BIOS_PH11-7.21.00.00.pdf’
\n ‘SAS2FW_Phase10-10.00.02.00.pdf’
\n ‘SAS2FW_Phase11-11.00.00.00.pdf’
\n)
\nfor pdf in ${pdfs[@]} ; do
\n if [ ! -e $pdf ] ; then
\n\techo wget ${FTPROOT}\/$pdf
\n\twget ${FTPROOT}\/$pdf
\n fi
\ndone
\n[\/crayon]<\/p>\n

5 USB Flash Update<\/h1>\n

I then mounted by USB drive and copied the files over.<\/p>\n

[crayon lang=”bash” toolbar=”always” title=”Update my USB Boot Drive”]
\n% # Linux: Ubuntu 11.10
\n% # Plug in the USB drive (FreeDOS created using unetbootin)
\n% mkdir -p \/mnt\/usb
\n% ls -l \/dev\/disk\/by-id\/usb*
\n% mount -f vfat \/dev\/sdf1 \/mnt\/usb
\n% cd \/mnt\/usb
\n% mkdir SMC
\n% cd SMC
\n% mkdir IT
\n% cd IT
\n% unzip ~\/work\/SuperMicro\/SAS-LSI-2008-IT\/fw\/PH11.zip
\n% unzip ~\/work\/SuperMicro\/SAS-LSI-2008-IT\/fw\/PH10.zip
\n[\/crayon]<\/p>\n

6 Flashed the Controller Firmware<\/h1>\n

After updating the USB drive, I inserted into the server board KVM port using an adapter and rebooted. I then executed the following DOS commands:<\/p>\n

[crayon lang=”dos” toolbar=”always” title=”Flash Firmware”]
\n> C:
\n> cd SMC
\n> cd IT
\n> cd PH11
\n> SMC2008.bat
\n[\/crayon]<\/p>\n

It took about 5 minutes to successfully flash the firmware.<\/p>\n

7 Eratta<\/h1>\n

This is a collection of random thoughts that will be used later.<\/p>\n

The power consumption of this setup was pretty good. I was measuring about 360W with everything turned on but with no activity.<\/p>\n","protected":false},"excerpt":{"rendered":"

I am building a filer based on OpenIndiana\/ZFS using the SuperMicro 6036ST-6LR super server because I need High Availability (HA). To use ZFS efficiently I want to use the SAS controllers as I\/T HBA’s. 1 The Problem I flashed the controllers using PH11 from ftp:\/\/ftp.supermicro.com\/driver\/SAS\/…8\/IT\/Firmware\/ and everything seemed to work fine. The flash completed successfully … Continue reading SuperMicro 6036ST-6LR LSI SAS2008 IT HBA Mode for ZFS<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"open","template":"","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"_links":{"self":[{"href":"https:\/\/joelinoff.com\/blog\/index.php?rest_route=\/wp\/v2\/pages\/370"}],"collection":[{"href":"https:\/\/joelinoff.com\/blog\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/joelinoff.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/joelinoff.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/joelinoff.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=370"}],"version-history":[{"count":20,"href":"https:\/\/joelinoff.com\/blog\/index.php?rest_route=\/wp\/v2\/pages\/370\/revisions"}],"predecessor-version":[{"id":395,"href":"https:\/\/joelinoff.com\/blog\/index.php?rest_route=\/wp\/v2\/pages\/370\/revisions\/395"}],"wp:attachment":[{"href":"https:\/\/joelinoff.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=370"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}