Problem adding cache to aggregate on 9.3P21 | NetApp | Page 1

peak mica Aug 9, 2023, 3:19 PM

#

Hello,
our customer had added SSDs (cache1) to two a 2-node cluster and split the cache size between the two nodes of the same aggregate (aggr1). Now he got more SSDs (different drives, cache2) and wanted to extend aggr1 but the job failed when trying to add the second cache.
The job failed with: Disk addition failed, internal error 'Limit: 800.00 GB. Attempted size: 930.21 GB

The idea was to split cache2 again like cache1 in 2 tranches and add it to the aggr1_FSAS_01 + aggr1_FSAS_02.

Why is it failing?
Would the additional cache bring performance benefits or what would be the adequate action to do in this case?

Thx,
ksh

tired stratus Aug 9, 2023, 3:20 PM

#

Are the SSDs the same size?

#

Maybe try RAID4 for the cache?

#

Also I'd recommend upgrading ONTAP as 9.3 is very old.

peak mica Aug 9, 2023, 3:28 PM

#

The SSDs are not the same size. of course not. thats why a second pool was anticipated.
It's an old FAS2240. So no more upgrades possible.

What's also bothering me, is that there are 5 disks in the cache2 and 2 are used as spares. I would rather have 1 as spare.

#

When creating the pool I cant exclude the hot spares. maybe works manually

tired stratus Aug 9, 2023, 3:32 PM

#

Try through CLI

peak mica Aug 9, 2023, 3:37 PM

#

Via CLI works. However, does it even make sense to have that much cache? the two FSAS on aggr1 with 24.09TB and 33.73TB should be supported by cache

tired stratus Aug 9, 2023, 3:38 PM

#

I don't know.

#

What are you using it for?

#

2240s only hvae 4 cores from 2010 so it isn't gonna be powerful.

peak mica Aug 9, 2023, 3:42 PM

#

VMware mostly

#

NFS3 and vVols

tired stratus Aug 9, 2023, 4:32 PM

#

Should be ok.

peak mica Aug 9, 2023, 4:36 PM

#

alright then. So you say the cache is going to be used?

#

I will make another attempt now and see if I can add the disks now

#

tired stratus Aug 9, 2023, 4:43 PM

#

Looks like it's working.

steady radish Aug 9, 2023, 4:45 PM

#

the 800GB limit is a system limit of the FAS2240

tired stratus Aug 9, 2023, 4:46 PM

#

steady radish the 800GB limit is a system limit of the FAS2240

Ah good find.

peak mica Aug 9, 2023, 4:49 PM

#

oha. is that for the entire cluster?

steady radish Aug 9, 2023, 4:50 PM

#

AFAIK it's per-node

peak mica Aug 9, 2023, 4:51 PM

#

`
FAS2240::*> storage aggregate show -fields hybrid-cache-size-total
aggregate hybrid-cache-size-total

aggr0_SAS_01 0B
aggr1_FSAS_01
371.9GB
aggr1_FSAS_02
371.9GB
`

#

that should leave me about 400GB per node

peak mica Aug 9, 2023, 5:18 PM

#

this should work then. However, I see the error again stating 930.21.

tired stratus Aug 9, 2023, 5:19 PM

#

Well, then your limit is being hit...

peak mica Aug 9, 2023, 5:27 PM

#

puh. I guess so. Is there any way I can use the 5 SSD's and do something useful with it? When creating an aggr at least 3 are required but 2 must be spares.

raid.min_spare_count is set to 1 already.

Any d*rty way I can create a raid with all 5 SSD's or at least 4 to squeeze out the last bytes?

tired stratus Aug 9, 2023, 5:57 PM

#

set it to raid4?

peak mica Aug 9, 2023, 6:01 PM

#

see the screenshot. -raidtype 4

steady radish Aug 9, 2023, 6:33 PM

#

disks assigned to the wrong node? it says you don't have 5 disks, only 3

peak mica Aug 9, 2023, 9:25 PM

#

ahhh facepalm Will try that

#Problem adding cache to aggregate on 9.3P21