#DS460C shelf difference (DCM, DCM2, DCM3) and IOM12/IOM12B

1 messages · Page 1 of 1 (latest)

safe tartan
#

Hi... I have been looking on the support site, and cannot find anywhere that describes the differences between the 3 different DS460C shelfs.. Does anyone know if there are any performance differences? Thay all have two IOM modules installed, so my guess is not? But there there are IOM12B modules 😉 So if someone have any insights into this, it would be appreciated 🙂

silk violet
#

there is no functional difference between the IOM modules. The new ones were introduced because of supply chain issues

dark lance
#

except that the IOM12 (X5720A) goes EOS 30-Sep-2028

safe tartan
#

OK thanks... would you possentially get a bit more performance if you did a quad-path setup, using all 4 12G ports in the IOM12 module? (we are talking NL-SAS disks)... but two DK460C in one stack..

velvet urchin
#

Don't think so. I've not yet seen any quad-path setup in the wild. 🤷
Maybe there are reproducible performance-improvements if you go up to ~240x NL-SAS disks in a stack. Or with 96x SSDs. But usually the controller will be the bottleneck in your system and not the available bandwidth to the disks.

#

I mean you have 48Gb/s with multipath SAS3, should be enough for most HDD aggrs.

#

The SAS SE guide says: "Quad-Path HA cabling recommended for large block, 64k or larger, sequential read workloads. Multipath HA sufficient for other workloads."

#

If you have the cables (and some spare-time) you can try the quad-path cabling but I don't think you will see much improvements, if any.

dark lance
#

It helps a lot more in SSD-only environments, especially when you limit the SSDs to one shelf per stack..at most two.

#

ONTAP will normally try to "even" out communications to the SAS attached drive.

#

I have never done it with spinning disks.

safe tartan
#

I did one setup with an A700 and 12 shelfs... and too long SAS cables (mostly 5M) ... it was a total mess to look at 😉

solar viper
#

There are different drawer control module versions: DCM, DCM2, and DCM3. They are identifiable by different color stripes on the front of the drawer. DCM2 requires an IOM12 with firmware 3.00 or later, or an IOM12B. DCM3 requires an IOM12 with firmware 4.01 or later, or an IOM12B with firmware 2.02 or later.

On the IOM there are two uplink ports on the A side, and two uplink ports on the B side. You can use both ports on the A side for uplink to a controller to aggregate the bandwidth, called a SAS wide port connection, however you will need to connect both connections to the same controller chip servicing both PHY’s.

If you have one SAS2 or SATA3 disk element, the whole SAS chain decreases to the lowest common speed of 6.0 Gb/s. I use these shelves on a 3rd party controller and with SATA3 spinning rust drives I easily saturate the total bandwidth of a wide port configuration with just 30 of the 60 bays filled.

safe tartan
#

I also have a test system with four DS212 shelfs connected to each own SAS3 HBA with only one path.. I was actually thinking of replacing them with one DS460C shelf, but I am worried about the performance with 60 SAS3 disks on one loop... I have been thinking of using mpath together with the ZFS pool that is on the disks, but not sure how that mixes with loadbalance etc.. and it is best to use one path from each IOM12 module?

solar viper
#

If I could do it over again, I wouldn't use these DS460C's, I would use DS212C's. I didn't know the particularities until I put them in to service. I got them for next to nothing, so that was the primary reason, however they a massive, are almost too large to move empty, and definitely immovable when loaded. The require 200-240v power, they are loud, and I am actually using two of these shelves with only 30 drives each so that I can get most of the performance each drive element can provide. If you are using SSD's or over 30 spinners in these shelves, don't expect to get anything near theoretical maximum. I use ZFS as well, and the hallmark of that filesystem is scrubs and resilvers, which requires reading all data fairly often to check and fix bit rot, and I don't want to wait days for completion due to storage topology bottleneck. One thing I did do with my ZFS setup was build a special VDEV using SSD's to store metadata and small blocks.

#

The SAS wide port configuration to the SAS controller works great. I don't use multipath since the disto/appliance I use doesn't natively support it, so I am stuck with one fabric. Even if multipath was supported in software, the SATA3 drives I use only have one controller interface and I don't even know if the expander (IOM's) would support that due to NetApp magic. That is the only thing that kind of sucks about using NetApp hardware for third party software is that NetApp is a black box.

safe tartan
# solar viper If I could do it over again, I wouldn't use these DS460C's, I would use DS212C's...

Well, in "some" contries 240V is standard 😉 guess I'm one of the lucky ones 😉 But you are right for "homelabbing" the DS460 is a mouthful, but this is all placed in a professional datacenter, where less mRACK Units are more important than the sound it makes 🙂 We have a 4 node ONTAP Cluster with 4 DS460 connected and it has been working flawlessly... Funny thing is, if you look at the watt numbers per shelf, the DS460 actually uses a bit more power than five DS212 shelfs... But we only use SAS disks in our systems, even the one with four DS212 and ZFS... and we use dRAID with special megadata cache etc. etc. It has been great and I would argue that we get better performance out of the same disks than ONTAP. Of cause this can be because of much more CPU and memory with ZFS... and we don't have to deal with the NVRAM like ONTAP has to...