#DE6600 with IOM12

1 messages · Page 1 of 1 (latest)

fresh lance
#

Hello,

We have tried to upgrade our DE6600 storage shelves to SAS3, as the previous SAS2 modules (100120-113 FRU 45822-00 ) have reached their bandwidth limit. We have therefore gotten IOM12 modules (X5720A P/N 111-02850). We were able to operate the shelf with the old modules as a DAS JBOD on an HBA without any problems. With the IOM12 modules, only the amber warning LEDs of the IOMs turn on and the fans go to 100%.
Is a firmware update necessary at some point or are the IOM12 modules incompatible?

Thanks for any help in advance.

arctic wyvern
#

Hi there! Did you also post about this on NetApp communities?

arctic wyvern
#

also, just confirming, do any of the drawers or IOM12s have a thick blue line on them?

fresh lance
fresh lance
torn drift
#

I'm pretty sure you cannot use NetApp IOM12 modules in the DE6600 because it's not the same as the DS460 shelf although it looks the same... (there must be some firmware differences on the shelf it self) I think I saw someone on YT try this with the same results as you have... The "IOM" modules for the DE6600 are called ESM modules and only 6G is supported...

#

What you want is the DE460C or the DS460C shelfs... they shoule have 12G ports

arctic wyvern
#

The DE460C is converged design but yeah, must be specific codes for the IOMs.

glad coyote
#

actualy this does work, at least with a E5700. As soon as you connect the shelf, the E-Series will detect it as incompatible (because of the wrong firmware on the module) but after a few minutes it will have automatically updated IOM firmware to the E-Series compatible one and everything will work

torn drift
#

Yeah but does the DE6600 support 12G SAS? (isn't is the same as if you plugged IOM12 into a DS4246, as the disks will still only link up with 6G, but the IOM12 will have 12G out) ? Also will this "trick" also work if you were to plug a DS6600 or DE460C to an ONTAP Controller?

glad coyote
#

The DS460 was only available with IOM12 modules and 12G backplane, so yes, it will support 12G SAS. And we have not yet tested the reverse, but I would assume that a simple storage shelf firmware update will fix the firmware again to the ONTAP one

#

the old E-Series specific IO modules might not have supported 12G SAS, that I don't know from the top of my head, but I would be surprised if they didn't. The E-Series has historically always been the system with the "faster" IO path, so I would think they adopted 12G SAS as early as possible

fresh lance
#

We do not necessarily want to achieve 12G. What we essentially want is a higher bandwidth at the I/O modules.

As we understand it, we have a maximum of 2 modules x2 ports x4 links x6GBit = 96GBit available. That is 12GByte/s. So with 60 drives per drive only 200MB/s.

50018183_1.png
#

What we also know is that the DELL MD3460 modules work. These would be 12G and come with 4 x 12G inputs each. However, you have to use arrays and dell disk firmwares in this case.

dell_md3460_12g_sas_controller_module_4mchf_front_dsc8348.png
glad coyote
#

IOM12 modules have 4 SAS3 ports each, so 2x4x4x12=256gbit/s. But that's purely theoretical, I doubt you can reach even close to that. Feel free to prove me wrong, I'd love to see such a setup running

torn drift
#

You are right about the 2ports with x4 6Gbit etc... but your host needs to handle all this and "balance" the paths between the links, because it will see all disks from every port.. on the ONTAP controllers there is/was a command to balance the paths (back in 7mode)... I am not so sure how you will manage this with say Linux mpath driver... I would suggest using SAS disks instead of SATA because only SAS adds two data paths to each disk... that being said we have several FAS2700 systems with 2 x DS460 shelfs and 24 internal disks, but only two SAS loops for everything.. that's pretty close to the limit... guess this is why the system is limited to 144 disks 😉

glad coyote
#

Linux mpath will discover the paths automatically and you can use a "least queue length"-type policy that balances traffic out pretty well. Also SAS vs SATA does not matter much since for maximum sequential performance you only want to access the disk through one path at the same time (otherwise the head movement will kill the performance). But still for 60 disks to do maximum throughput you need some beefy CPUs and many PCI lanes dedicated to the SAS HBA (and the Linux drivers are not known to be the fastest, FreeBSD has IMHO better drivers at least for SAS HBAs)

torn drift
#

We have a system with 2 x DS212C with only one IOM12 module in each shelf, connected to a dual port HBA.. this is all 20TB SAS drives installed, and works great. And with ZFS you can even use alias to match the drive names to their hba/shelf/slot... so it's much like ontap syntax... there are also tools avaliable that can "control" the LEDs on the shelf via the SG driver... but we have not tested with the DS460 yet

glad coyote
#

yeah I have a DS4246 with 24x 10TB drives from an old E-series in it, running ZFS on FreeBSD (because Linux didn't like the SAS HBA and kept hanging during boot, and I didn't feel like troubleshooting much 😉 )
The multipathing on FreeBSD works but for some reason you have to tell it manually which paths go to the same drive, it somehow can't figure this out from the drive's S/N. So I didn't bother (yet) to do that, I just run everything over a single path. This is just for fun anyways, no production data on the disks