#Getting Flex Group to work the best with CIFS or NFS?

1 messages · Page 1 of 1 (latest)

peak bison
#

A Flex Group is more or less just a set of volumes that are "merged" together into one namespace... the underlaying volume can be on different aggregates belonging to different nodes in the cluster.
So as you write to your Flex Group volume, the files are written to alternate volumes and ONTAP tries to balance the capacity over the member volumes...
My question is if there are a way to make the copy of many files to a Flex Group utilize this... in our first test setup we can see that using CIFS the clients "lock" onto an IP and writes everything to the node that owns this LIF, and it then uses the cluster links to write to the other aggregate every 2nd file... we have tried to use "rclone" to run two copies at a time, hoping that we could utilize both underlying volumes, so to have the performance of both aggregates... but as metioned it uses the cluster links which seems to slow down the overall transfers and ever so often it tries to write two files at the same time to the same underlaying volume... so not very efficient... this is smaller systems FAS27xx with spinning rust... and the files we are copying are from 20G-80G in size... Is there any way this can be optimised? (we only have one underlaying volume on each aggregate right now..)

tribal escarp
peak bison
#

Yeah as expected there isn't really anything you can do... this would not be a problem if this was larger controllers with 100G cluster links, but as we LACP 4 10G links together on each node the backend of 2 x 10G can become the bottleneck... we where hoping that ONTAP could somehow control the load via multichannel SMB or similar on NFS (pNFS)... but doesn't look like it... I think this was just a wishful thinking and this case is also somewhat special and as soon as we have moved all the files the workloads should benefit from files placed on different aggregates... sometimes... if they are lucky 😉

dusk ridge
#

in our first test setup we can see that using CIFS the clients "lock" onto an IP
This is unfortunately by design. The only thing you can do against that is using DNS loadbalancing (either simple roundrobin or using the ONTAP-included DNS as forwarder) and two different clients... or use IP addresses instead of names for copying.
Note that you can have DFS redirects at volume junction points if that option is enabled, so that as soon as the client crosses to another volume it will get redirected to the IP where that volume resides... but that probably doesn't work well with FlexGroups since ONTAP decides where the data will be stored

fierce drift
#

With cifs, take a look at odx. It might help. Never used it. Just have heard about it

peak bison
# fierce drift With cifs, take a look at odx. It might help. Never used it. Just have heard abo...

I think it does more or less the same as rclone does... (run multiple copies) but as long as ONTAP cannot control the destination or the TCP/IP connection it doesn't make much sense... It looks like one copy streem is just as fast as two, even when it happens to copy one file to each underlying volume... it looks like the traffic through the cluster links has some kind of effect on the direct copy running... not sure... that's just what I can see using nabox and sysstat on each node... maybe this will be better once we scale up the number of disks from 12 to 28 en each aggregate...

#

I am a bit surprised, because we have a LIF on each node for CIFS, so shouldn't it present both IP adresses to the client? And the client then uses the LIF that is local to the files it want to use? Or maybe this only works on reads?

dusk ridge
muted star
#

pNFS would work because the connection can be transferred to the local nblade of the disks. CIFS with Referrals I don't think does this, but maybe it does?