LSI 9211 (SAS2008) BIOS problems, invalid PCI slot

[P Orrifolius]

Hello.

I have an LSI 9211-8i controller which is running the IR firmware. My
intention is to flash it with the IT firmware and use it for an md
RAID array.

However I don't think it is playing nicely with my motherboard BIOS.


On first installing it, with no drives attached, the LSI BIOS runs and
then the motherboard BIOS goes through an AHCI drive detection phase.
After detecting the drives (2xHD 1xOptical, connected to SATA ports on
MB) the boot hung and required a hard reset.

The LSI MPT2BIOS version is 7.29.0.0 and the controller firmware is 15.0.0.0-IR.
The motherboard is a Gigabyte GA-MA78G-DS3H (revision 1) running the F3 BIOS.

I can enter the LSI configuration utility during the controller boot
process and I see that it has a PCI slot of 'ff' which, according to
the utility, indicates an invalid PCI slot.


I've updated to the latest motherboard BIOS (F9) and now the boot
process follows the same sequence but continues after the AHCI
detection and eventually brings up linux. However the controller
still reports a PCI slot of 'ff'.

I can see it in linux but if I have a drive connected I don't see it
in the boot configuration utility (not sure if I should) and I don't
see it as a /dev/sd? device (not sure if this is where I should find
it). So I suspect it's not really working.

Here is the output from lspci -v:

02:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic
SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
Subsystem: LSI Logic / Symbios Logic Device 3020
Flags: bus master, fast devsel, latency 0, IRQ 18
I/O ports at de00 [size=256]
Memory at fdcfc000 (64-bit, non-prefetchable) [size=16K]
Memory at fdc80000 (64-bit, non-prefetchable) [size=256K]
[virtual] Expansion ROM at fdb00000 [disabled] [size=512K]
Capabilities: [50] Power Management version 3
Capabilities: [68] Express Endpoint, MSI 00
Capabilities: [d0] Vital Product Data
Capabilities: [a8] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [c0] MSI-X: Enable+ Count=15 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [138] Power Budgeting <?>
Capabilities: [150] Single Root I/O Virtualization (SR-IOV)
Capabilities: [190] Alternative Routing-ID Interpretation (ARI)
Kernel driver in use: mpt2sas
Kernel modules: mpt2sas


Anybody run into this or have any advice? Is this really a problem?

I'm reluctant to risk bricking it during a firmware flash if it truly
is misbehaving.


Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Stan Hoeppner]

Post by P Orrifolius
I have an LSI 9211-8i controller which is running the IR firmware. My
intention is to flash it with the IT firmware and use it for an md
RAID array.
However I don't think it is playing nicely with my motherboard BIOS.
On first installing it, with no drives attached, the LSI BIOS runs and
then the motherboard BIOS goes through an AHCI drive detection phase.
After detecting the drives (2xHD 1xOptical, connected to SATA ports on
MB) the boot hung and required a hard reset.
The LSI MPT2BIOS version is 7.29.0.0 and the controller firmware is 15.0.0.0-IR.
The motherboard is a Gigabyte GA-MA78G-DS3H (revision 1) running the F3 BIOS.
I can enter the LSI configuration utility during the controller boot
process and I see that it has a PCI slot of 'ff' which, according to
the utility, indicates an invalid PCI slot.

Invalid in what way? The motherboard assigns PCI resources to expansion
boards, not the other way round. So I'm curious as to what the board is
saying is invalid about the PCI slot configuration.

Post by P Orrifolius
I've updated to the latest motherboard BIOS (F9) and now the boot
process follows the same sequence but continues after the AHCI
detection and eventually brings up linux. However the controller
still reports a PCI slot of 'ff'.

Do the drives show up under the device identifier, such as in this img:
Loading Image...

Post by P Orrifolius
I can see it in linux but if I have a drive connected I don't see it
in the boot configuration utility (not sure if I should) and I don't
see it as a /dev/sd? device (not sure if this is where I should find
it). So I suspect it's not really working.

If you have a drive connected and powered up it should show up in the
LSI BIOS config util as above, and should also be enumerated by the LSI
boot BIOS during the POST sequence. If the drive isn't showing up in
the controller utility then you likely have:

1. Bad or not fully connected data or power cable(s)
2. Bad drive
3. Bad HBA

Post by P Orrifolius
02:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic
SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
Subsystem: LSI Logic / Symbios Logic Device 3020
Flags: bus master, fast devsel, latency 0, IRQ 18
I/O ports at de00 [size=256]
Memory at fdcfc000 (64-bit, non-prefetchable) [size=16K]
Memory at fdc80000 (64-bit, non-prefetchable) [size=256K]
[virtual] Expansion ROM at fdb00000 [disabled] [size=512K]
Capabilities: [50] Power Management version 3
Capabilities: [68] Express Endpoint, MSI 00
Capabilities: [d0] Vital Product Data
Capabilities: [a8] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [c0] MSI-X: Enable+ Count=15 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [138] Power Budgeting <?>
Capabilities: [150] Single Root I/O Virtualization (SR-IOV)
Capabilities: [190] Alternative Routing-ID Interpretation (ARI)
Kernel driver in use: mpt2sas
Kernel modules: mpt2sas

That's a good sign. The driver loads and detects the board, and shows
no errors.

Post by P Orrifolius
Anybody run into this or have any advice? Is this really a problem?
I'm reluctant to risk bricking it during a firmware flash if it truly
is misbehaving.

The firmware rev isn't going to prevent a connected drive from being
detected and listed in the utility. If the drive doesn't appear it's
almost certainly a data/power cabling issue. You're using one 8087
breakout cable and one SATA connector on that cable correct?

[P Orrifolius]

The executive summary:

Connected drive now being detected and made available in linux. But
the PCI Slot is still being listed as 'ff' and boot fails with some
controller settings.

More details below.

Post by Stan Hoeppner

Post by P Orrifolius
I can enter the LSI configuration utility during the controller boot
process and I see that it has a PCI slot of 'ff' which, according to
the utility, indicates an invalid PCI slot.

Invalid in what way? The motherboard assigns PCI resources to expansion
boards, not the other way round. So I'm curious as to what the board is
saying is invalid about the PCI slot configuration.

If I highlight the 'ff' value for PCI Slot in the configuration
utility and then hit F1 (Help) it says:

PCI Slot
The PCI Slot number assigned by the System BIOS to an adapter. If the
value displayed is FF it indicates invalid slot number.

I can't find any information about it, other than this comment to a
post: http://www.servethehome.com/howto-flash-supermicro-x8si6f-lsi-sas-2008-controller-lsi-firmware/#comment-30607

Post by Stan Hoeppner
http://i.imgur.com/weKsd.jpg

No.

Post by Stan Hoeppner
If you have a drive connected and powered up it should show up in the
LSI BIOS config util as above, and should also be enumerated by the LSI
boot BIOS during the POST sequence. If the drive isn't showing up in
1. Bad or not fully connected data or power cable(s)
2. Bad drive
3. Bad HBA

I connected the drive direct to the motherboard and it works fine.

I connected the drive to the controller via different SATA breakouts,
different 8087->4xSATA cables and different ports on the controller...
still no dice.

In every case I could tell by touch that the drive wasn't spinning up.


However...

I changed the controller Boot Support setting to Disabled and the
Drive Spin-up Delay from 2 -> 0 after my failures, without touching
any cabling.

And, lo! On reboot the drive spun-up immediately and the controller
detected it. I booted into linux and it appeared as a sd? device.

Weird thing is I set the controller options back as they were yet the
drive still spins-up... I swear I didn't touch the cabling.


So current status is:
- Controller, and detected connected drive, reports PCI Slot ff which
is, it claims, invalid.

- When the controller Boot Support is set to 'Enabled BIOS and OS' or
'Enabled BIOS only' the AHCI drive detection on boot takes ages then
eventually outputs 'Warning - something wrong with your hardware!'
Hard reset is then needed. This prevents linux booting of course.
The connected drive is detected and listed after the controller
initialises. Presumably the MPT2 boot ROM that the controller says it
installed doesn't work well with my motherboard.

- If Boot Support is set to 'Disabled' or 'Enabled OS only' then the
controller initialisation simply states Adapter(s) disabled. Then the
AHCI drive detection happens and linux boots. Happily the connected
drive is visible in linux.


I can live with the mystery and I don't need to boot from drives
connected to the controller, so my only questions really are:

- will flashing be safe given the 'ff' business?

and

- will the controller being 'Disabled' lose me any capabilities
beyond booting from controller connected devices?


Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Stan Hoeppner]

...

Post by P Orrifolius
PCI Slot
The PCI Slot number assigned by the System BIOS to an adapter. If the
value displayed is FF it indicates invalid slot number.

This is irrelevant, harmless. The "PCI slot number" is assigned so a
human can find a card in a physical slot in the machine. Cheap mobo
BIOS often doesn't assign slot numbers. Enterprise targeted mobos and
vendor systems most certainly do. Having a missing PCI slot number does
NOT affect the functionality of the card. This is a convenience feature
only. For more on this see:

"Core BIOS and BIOS configuration utility will display "FF" as the PCI
slot number when proper slot information is not available."

http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5090849

...

Post by P Orrifolius
I changed the controller Boot Support setting to Disabled and the
Drive Spin-up Delay from 2 -> 0 after my failures, without touching
any cabling.

When using the LSI controllers that have BIOS/firmware RAID and can boot
the system, it's always best to disable the controller BIOS to prevent
compatibility problems with the host BIOS. If you aren't booting from a
drive on the LSI, disable the BIOS.

Post by P Orrifolius
And, lo! On reboot the drive spun-up immediately and the controller
detected it. I booted into linux and it appeared as a sd? device.
Weird thing is I set the controller options back as they were yet the
drive still spins-up... I swear I didn't touch the cabling.

Good, making some progress. Keep in mind these enterprise HBAs are
designed primarily for SAS. SCSI drives have supported spinup delay for
decades. SATA drives often do not, or the implementation is poor.
Simply disabling the delay may be the key.

...

Post by P Orrifolius
- If Boot Support is set to 'Disabled' or 'Enabled OS only' then the
controller initialisation simply states Adapter(s) disabled. Then the
AHCI drive detection happens and linux boots. Happily the connected
drive is visible in linux.

So the problems are solved. Good.

Note that enterprise RAID/HBA cards often cause problems with consumer
mobos because the latter vendors simply never do QC with such
aftermarket storage controllers. LSI publishes an extensive list of
systems, server, and workstation mobos that have been tested with their
HBAs, including any errata for each, such as specific settings required
for the HBA to work.

Post by P Orrifolius
I can live with the mystery and I don't need to boot from drives
- will flashing be safe given the 'ff' business?

Flashing firmware, whether mobo or add in card, is never a guaranteed
proposition. But any safety factor has nothing to do with slot number
assignment, which I explained above. The flash program should find the
card without problems. Whether the flash process is successful is
another matter.

That said, I highly recommend you do not flash the board with the
Initiator Target (IT) firmware. The IT firmware simply removes RAID
capability, and adds support for SAS expanders and up to 256 connected
devices. Other than that, AFA Linux is concerned, there is no
-functional- difference between the IR firmware in HBA (BIOS disabled)
mode, and the IT firmware. If you do not plan to use an expander, don't
flash the board. Just leave the BIOS disabled.

Post by P Orrifolius
- will the controller being 'Disabled' lose me any capabilities
beyond booting from controller connected devices?

The controller isn't disabled. Only the BIOS and RAID firmware are
disabled. You simply lose the firmware based RAID capability. It
should be possible to boot a connected drive even with the RAID function
disabled. Play with it some more.
--
Stan




--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[P Orrifolius]

Post by Stan Hoeppner
That said, I highly recommend you do not flash the board with the
Initiator Target (IT) firmware. The IT firmware simply removes RAID
capability, and adds support for SAS expanders and up to 256 connected
devices. Other than that, AFA Linux is concerned, there is no
-functional- difference between the IR firmware in HBA (BIOS disabled)
mode, and the IT firmware. If you do not plan to use an expander, don't
flash the board. Just leave the BIOS disabled.

Post by P Orrifolius
- will the controller being 'Disabled' lose me any capabilities
beyond booting from controller connected devices?

The controller isn't disabled. Only the BIOS and RAID firmware are
disabled. You simply lose the firmware based RAID capability. It
should be possible to boot a connected drive even with the RAID function
disabled. Play with it some more.

Yeah, it was apparent that it wasn't really disabled, though that is
what the config utility calls it.

All the information I'd seen suggested flashing to IT was
required/ideal in order to use as JBOD to avoid the controller messing
with things, though I obviously started to doubt that when I saw the
drive in linux.

Anyway, I don't need the expanders or more drives so, as you say, I'll
have a play with it 'Disabled' on the strength of your comments. I'd
rather not flash the firmware.

Thanks for the help.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html