#Changing max-dirsize of a volume, is this non disruptive process?

1 messages · Page 1 of 1 (latest)

strange plume
#

Can we change max-dirsize of a volume without having any impact to existing sessions?

quick trench
#

Yes.

fickle elk
#

yes, you can. we have done it multiple times

strange plume
#

Thanks

dusky ore
#

Curious as tp what you are increasing it to and what is prompting the increase? How many files per directory are you expecting? There are some side effects of increasing this value (which is why we don't increase it by default)

fickle elk
#

we have some volumes with well over 120 million files.
I have made our product team split folder sizes up to no more than 100k per folder
but, still a ton of files

#

most are small files as well, under 7k

#

example, one of our current production volumes
Maximum files 1,232,505,960
Used files 551,036,921

midnight flame
#

...today in "Help, my devs think a filesystem is a database" 😉

fickle elk
#

dont get me started on our dba group, oye

#

some of our ms sql clusters have 300+ luns assigned to them because they think each database needs it's own drive and log drive

midnight flame
#

100k files in each directory will notiveceably impact performance even on an AFF system. If they require a flat hierarchy for storing small data chunks they should look at S3 or document databases instead of just dumping everything to disk. But that won't help you here and now I guess 🙂

#

then again, for 100k files, the default directory size should usually be enough (unless the filenames are reeeeally long)

fickle elk
#

this thing has been like this since i started working here, 18+ years ago.
They REFUSE to change the way anything works

#

at least now they do split things up, it used to all be in 3 folders

midnight flame
#

well, as long as the performance seems to be sufficient for them I guess it works 🤷‍♂️

fickle elk
#

performance is fine, and we do tier most of that to blob in azure (no s3), all CVO instances now

midnight flame
# fickle elk example, one of our current production volumes Maximum files 1,232,505,960 U...

number of files in total is never an issue... number of files in a single directory might be a problem. on every "ls" from the client, ONTAP has to read the complete directory (say, 200-500 megs, depending on "maxdirsize") into RAM and send it to the client (via NFS for example). This is everything but efficient, and it will thrash the read cache if you have dozens or hundreds of those large directories, especially on smaller systems with less RAM. But as long as the users don't complain it won't be an issue for ONTAP itself

fickle elk
#

all data is served by http and the files/folders aren't really scanned, except by AV, because the database keeps track of the file location.
So when a file is requested, the db simply pushes the path it was saved as back to the client
We have a client SLA for page load times that aren't supposed to exceed 3sec, and so far we haven't failed that

midnight flame
#

still, if you open a file by name, ONTAP has to scan the entire directory for that file, and if it is somewhere "at the end of the list", it will have to read 200-500 megs to get to the file's inode number

#

but SSDs are so fast that the 3 seconds SLA is probably never an issue 🙂

#

it is more noticeable if you work on those directories interactively

covert sail
#

Maxdir size is totally different than maximum files.

Max dir size has to do with the directory size (like an LS -l) which if I recall is defaulted to 320MB. That roughly equates (again if I recall correctly) to about 300,000 files following the 8.3 naming convention. Longer file names reduce the number of allowed items in any given directory.

If you are referring to the total number of files that’s different. That’s just maxfiles if I recall

midnight flame
#

maxfiles is per volume. usually the number of inodes is set to (I think) 1 inode per 32kb or thereabouts.
maxdirsize directly correlates to the size of the "directory file", i.e. more files per directory -> larger directory, and if you have longer file names then it fills up faster (it relates to the number of bytes used for each file name entry plus some metadata...)
300k files with "regular" file name, unicode name (the old create_ucode/convert_ucode flags) and 8.3 name can easily reach 300 mb, depending on how long the filenames are

#

there's a diag mode command that lets you look at and explore those on-disk data structures (inodes, directory files, buftrees, etc.)... really interesting to see how a directory for example is laid out in WAFL

tawdry mist
fickle elk
#

should be
:> volume modify -maxdir-size XXX

tawdry mist
#

That is clear I mean more how the explore the structure like Darkstar mentioned

midnight flame
# tawdry mist Can you share the command?🙈

I'm a bit reluctant to share it because when I started to play around with it I managed to panic the filer a few times just by trying to read some inodes and stuff 😄 it's definitely not a harmless command even without using the "write" functionality ...

tawdry mist
#

I see this sounds adventurous 😅

hasty wave
#

I always wondered why there isn't an autosize feature for maxfiles and maxdirs?

midnight flame
#

there is no "maxdirs", but for maxfiles it boils down to "legacy" I guess (fan-in SnapVault to a single volume was a thing in 7-mode, and you didn't want to suddenly run out of inodes on your SV destination, for example, and you can't decrease maxfiles as long as you have one of the "high" inodes in use)

#

as for maxdirsize, since that can negatively impact performance especially on spinning-disks, they probably didn't want that to silently grow

hasty wave
#

I know you cannot decrease them, but maxfiles has been the single most issue for "downtime" that I can think of 😉 It would be nice with an option to autoincrease it by x (and trigger an autosupport)

midnight flame
#

hm, I guess if you work with so many small files just set it to the maximum when creating the volume then?

hasty wave
#

because: users... 🙂

fickle elk
#

we have a monitor that runs and checks inode status, when it gets to 85% we get a notice, then another at 90%, once it hits 95% a playbook runs to increase them by 10%

covert sail
#

For “maximum files” there is this option : files-set-maximum
Once set it maximizes the file count. If you ever grow the volume it will update the file count

#

I’ve had to have customers set it on zone flexgroup volumes.

fringe arch
strange plume
#

Can we monitor this in OCUM

fickle elk
#

yes, both can be monitored in OCUM/AIQ

strange plume
#

Cool thanks