#┊・storage

1 messages · Page 1 of 1 (latest)

fluid halo
#

8.2.5 In mode 7, is there a way to force a user (session) using NFS to drop? I don't know even if I look at the options with "manfs."

fluid halo
edgy marsh
#

NFS exports (at least up to v3) give access to hosts.
User are then given access on the host (NFS client) side.
Could you remove the file level access for that user?

#

Or do the user have his own host?

fluid halo
#

I want to keep the service and make sure that only certain bands of sessions are unmounted.

#

For example, I want to umount a Linux server called [192.192.192.123] that is using [192.192.192.55:/vol/NFS.]

edgy marsh
#

What does exportfs show for that path?

unique spade
#

Hi Team,

We are currently facing an expansion of a 2Node AFF A300 cluster.

We currently have just 1 x DS224C with 24x960GBxSSD disks with ADP(root-data-data) and P1 partitions assigned to node1 an P2 partitions assigned to node o two different data aggregates.

A new DS224C with 12x960GBxSSD has been purchased and we would like to confirm (we understand is possible like a regular FAS system) if it's possible to assign the new 12 disks without ADP to a single node of the AFF system and create a new aggregate without ADP (9+2+1). We known that on AFF A300 first two shelves supports root-data-data partitioning, but on the new one we wouldnt like to use it. On NetApp fusion this configuration appears as supported but we would like to be sure that once we create the new aggr the new disks wont be partitioned.. Does anyone has faced a similar scenario?

PS: We wouldn't like to expand existing partitioned aggregates cause it's not a BestPractice to have on a same aggregates different raid sizes of composing rgs.
Thanks guys 🙂

uneven roost
#

Drives will only be partitioned when added to a raid group of already partitioned drives, so yes, it will work as you hope

unique spade
#

Thanks alex for your clear response 🙂

tawny basalt
#

Additionally that best practice is really geared towards spinning media like hdds.

#

If you are running current ontap, it may auto partition the drives when you attach them. This is fuzzy as I have seen some version auto partition and others leave them whole.

#

***note my previous comment may be the behavior with internal ssd/nvme like on the a800. I don’t recall which platform I saw this on

edgy marsh
#

I've had the 2nd 224x-shelf autopartition when connected. I've sometimes added all data-partitions to the same controller / aggregate if thats where I needed space. Assigning both data partitions on a drive to the same controller might be an alternative to running un-partitioned. Not sure if that is best practise, but I had no problems

foggy narwhal
#

Do we have documentation or KB regarding disk space auto reclaim on 7 mode systems? We have a 6tb LUN on a windows host. We already maximize the 6TB and deleted some files lowering it to 4TB (2TB free space) but on the storage side, is still shows that the disk is completely used. Are there ways to reclaim the space on the netapp storage?

Version: 8.2.4 7 Mode
Space auto reclaim was introduce on CDOT

elfin igloo
#

Hey @foggy narwhal I see your issue, most of our KB's and Docs are about ONTAP CDOT 9. I did find this https://library.netapp.com/ecmdocs/ECMP1196995/html/GUID-93D78975-6911-4EF5-BA4E-80E64B922D09.html

How host operating systems can automatically reclaim space and keep...

Starting with Data ONTAP 8.2, you can use the space_alloc option to reclaim space and notify the host when a thinly provisioned LUN cannot accept writes. The space allocation option enables the Logical Block Provisioning feature as defined in the SCSI SBC-3 standard.

uneven roost
#

but it's a batch job, not an in-line thing

elfin igloo
#

You always know who the long-timer NetApp folks are as they know the 7mode answers! Me on the other hand... clueless. 😆

cedar raptor
uneven roost
#

I’d really rather forget 🤣

#

I mean it made me a lot of money.. but my brain needs extra space these days 😉

edgy marsh
#

... 7-mode is the new mode, with support for aggregates and flexvols, right? 😇 😂

dark sphinx
uneven roost
#

I don't think 8.1 7mode does the zero detect does it?

#

looks like 8.3? https://community.netapp.com/t5/ONTAP-Discussions/how-can-i-turn-on-Inline-zero-block-deduplication-without-Inline-deduplication/td-p/134492

how can i turn on Inline zero-block deduplication without Inline de...

Hi.     The following TR recommends to enable "Inline zero-block deduplication" for file services on HDD, but unlike other type of workloads and disks not "Inline deduplication" https://www.netapp.com/us/media/tr-4476.pdf   The only problem. is that i can;'t find how to do it. without enabling inlin...

foggy narwhal
#

Current version is 8.2.3P4 7 Mode. I am not quite familiar on the reclaim it using the snapdrive or the SCSI unmap. The LUN is directly mounted on the windows server. Without reclaiming the LUN is like set as Thick Provisioned

olive umbra
#

i always follow this KB

#

also this "fsutil behavior query disabledeletenotify" command on windows must return 0 to allow the unmap to work

rose thicket
#

Anyone know why ONTAP 9.9 and UM efficiency ratios appear to be off by a wide margin?

cedar raptor
#

can you share an example?

#

Most of the time that's related to the way snapshots get factored in to eff ratios

rose thicket
#

Well, my average per aggregate is about 65:1. I think I may have found the bug. If you look at the data in SM, all of the numbers are in GB's. i.e look at Tiers, select more information; Capacity is shown on top, and on the bottom the Data Reduction, in this particular case 68.9 to 1; directly underneath, it says 16PB logical used.

#

Same in CLI, 'storage aggregate show-efficiency': Aggregate xxx: Total Storage Efficiency Ratio: 62.43:1; Total Data Reduction w/o Snaps 1.77:1; Total Data Reduction w/o Snapshots & FlexClones 1.77:1

dense chasm
#

on FlexCache how do i see the actual size of the flexcache vol that is created on destination cluster on the linux mounts not on the cluster but on linux host doing a df -h

#

currently when performing a df it show the size of the source vol

cedar raptor
cedar raptor
#

What's the bug though that youre thinking?

rose thicket
#

Are you saying it counts the snapshot space as logical?

cedar raptor
#

the high ratio? yeah.

#

here's my 2552

unknown.png
rose thicket
#

Doesn't that defeat the purpose of the number? I see a 4.4PBUsed and Reserved, 1.55PB available, and 64.5 to 1 Data Reduction. That is a very mis-leading set of numbers.

cedar raptor
#

are you able to post a screen shot of your SM?

rose thicket
#

No, I'm sorry I am in a dark site. And I apparently can't see4 the BURT, but I was able to see the KB so it sounds like the may do something to show a more physical number, rather than logical.

#

Thank you

dense chasm
cedar raptor
#

Correct.

#

Typically folks will size FC for 10-15% of the source. I've seen people do 100% but wasn't worth it.

stoic jolt
#

i have a 8040 and a 2650 in a 4 node cluster with ADP running on both HA pairs, and i need to sanitize the drives so we can send it back to NetApp.. from reading the documents it seems you need remove partitioning to do this. Given every drive is partitioned is there an updated document which explains how to do this

stoic jolt
#

would i almost get the same result if i create some new aggregates.. blow them away and then zero-spares.. do that 5 times ?

uneven roost
#

reboot, special boot menu, option 9 to remove partitioning, after that is done, unseat all but 6 drives, option 4, basic setup, then reseat remaining drives, run sanitize on them, then reboot, unseat all but a different 6 drives, option 4 and basic setup again, then sanitize the 6

cedar raptor
stoic jolt
#

yeah i think i might need to just blow everything away.. initialize and create some new 3 disk aggrs for root aggrs and then everything else will be spares with no ADP and then run the sanitize

#

we need a new command.. send_back_to_netapp which does all this for us 🙂

dense chasm
#

is there anyway to to change admin passwords automatically every quarter

fallen halo
#

Tie the Netapp to AD and implement the password policy through AD?

dense chasm
#

but they want to change it without having to be prompted when AD policy asks to change

#

environment is too big to be doing one controller at a time

cedar raptor
#

Automating an admin password change sounds.... risky?

patent flame
#

I believe there is a force reset on next login option?

#

Im just not sure how to trigger that externally

dark sphinx
#

I'd say tie ONTAP into an IDM program.

dense chasm
#

i have pointed them toward CyberArk which they use

#

i am sure it has ability to execute that and save the new password automatically

dark sphinx
#

Yeah that'd be better to use something outside of ONTAP to manage that so it's one place only.

uneven roost
#

I have a couple of customers using cyberark for it

stable grove
#

A short question regarding IDR (Inactive Data Reporting) scans:
For a customer I've updated ONTAP to 9.8 and enabled "-is-inactive-data-reporting-enabled" for all data-aggrs. I think I didn't modify the "tiering-minimum-cooling-days" value so according to docs it should default to reporting 31 days of inactive data (if the tiering-policy is on "auto" which I hope it is).
My question: If we decide to change the value to like 21 days will the calculations/monitoring start anew and we have to wait another 21 days? Or can we shorten the reporting without losing anything?

#

The appointment with the customer to report some nice inactivity values is 29 days after I've activated IDR 😐 🤦‍♂️

cedar raptor
#

i'm pretty sure it restarts the counter. Let me check through docs later though.

crystal solar
#

it does

stable grove
#

☹️

#

ok, then I will postpone that meeting for 3 days

#

thanks

dense chasm
#

I would wait and make sure you can see the data in ActiveIQ before setting up the meeting, IDR is very finicky and it has no warning when it's not working

stable grove
#

omg... the "Volume Tiering Policy" is currently set to none for all volumes according to the ASUPs...

#

do I need to change that to "auto" so that IDR will report anything?

cedar raptor
#

for none defaults to 31 days

dense chasm
#

to my knowledge the policy change is only required when you really want to tier to Object storage

cedar raptor
#

For an aggregate that is not used for FabricPool, inactive data reporting uses the tiering policy to decide how much data to report as cold.

For the none tiering policy, 31 days is used.

stable grove
#

phew

#

ok, thanks

dark sphinx
#

You have to wait 21 days.

#

Or whatever.

fallen halo
fluid halo
#

hi guys.

i have 3 aggregate (45tb / 45tb / 26tb)
and wanna make flexgroup volume.

so i typed below commands but volume create was failed.
what should i do?


Notice: The FlexGroup volume "vol01" will be created with the following number
        of constituents of size 7.19TB: 16.
Do you want to continue? {y|n}: y

Error: command failed: Request to provision FlexGroup volume "vol01" failed
       because the requested size of 115TB (126443837194240B) is greater than
       the maximum size. The maximum possible size is 51.38TB
       (56488197406720B).

Does the aggregate have to have the same capacity to create a flexgroup?

#

please let me know...

# 
 --------- -------- --------- ----- ------- ------ ---------------- ------------
aggr01     45.75TB   45.50TB    1% online       5 FSSTORAGE101-01  raid_dp,
                                                                normal
aggr02     45.75TB   45.50TB    1% online       4 FSSTORAGE101-02  raid_dp,
                                                           normal
aggr03     25.71TB   25.71TB    0% online       0 FSSTORAGE101-01  raid_dp,
 normal```
crystal solar
#

you typed the command wrong... it's 'vol flexgroup create ...'

#

you've asked the system to create a normal volume larger than 100TB, and that it won't do.

fluid halo
#

Omg

#

I will try it agian

#

@crystal solar
bro documents said my command is right.

#

am i wrong to create flexgroup?

crystal solar
#

well, the error is reacting to the size of attempting to create a traditional volume... i guess i would try adding flexgroup (or just flex) to see if that works better... it always has for me

fluid halo
#

i will try it tomorrow but, if i failed to create flex group whatshould i do?

crystal solar
#

read the error message and adjust your command, i guess

fallen halo
#

Are you not able to just create the vol in system manager?

#

Believe the issue is just syntax

fluid halo
#

I will try it tommorow brothers

#

Regards

#

: )

cedar raptor
#

try -

vol create -volume vol01 -aggr-list aggr01,aggr02,aggr03 -aggr-list-multiplier 4 -size 115tb -state online -policy default -space-guarantee none

tawny basalt
#

Mike has it

#

You were specifying thin provisioning and didn’t have enough space

#

Using the space guarantee of none enables thin provisioning and should work

tawny basalt
crystal solar
#

i would believe that any attempt to make a volume larger than 100TB without the 'flexgroup' option should fail... except on the few platforms and licenses that allow larger volumes

foggy narwhal
#

Anyone tried to integrate their netapp storages to Zabbix monitoring tool?

dark sphinx
cedar raptor
#

they're 300TB for the ASA platform only for large luns.

uneven roost
fallen halo
#

Is there any documentation out there for LUN volume creation? I have a customer who refuses to break up sql luns into data/log/sysdb, etc and only wants 1 lun per environment to house all of the sql databases.
Ideally, most ppl split these out in case 1 fills, it doesn’t bring down everything, but I can’t find documentation to back this up.

#

Besides… common sense. Which doesn’t seem to be high up on their list for this.. 😳

dark sphinx
#

There's a KB about single lun performance...

fallen halo
#

Found it. 4-6 vols is recommended for luns and ideally 8-16

#

Tr4080

thorny maple
#

httpsdocs netapp comus enontapencryption

dark sphinx
fallen halo
#

p.34-p.35

dark sphinx
#

Here's a couple KBs outlining a few issues I've seen and written about in these articles with others in perf support.

fallen halo
#

Gracias sir 🙏🏾

cedar raptor
#

That was pretty common config with early versions of snapcenter i think. I saw a few customers like that.

crystal solar
#

most get into the problem of how many "drives" a server will be able to support, so they try to keep it "windows simple"

#

they don't understand mount point volumes on windows... that many luns can be mapped under a "drive" like you easily could do on unix/linux systems

fluid halo
#

Hi Team.

Can configure Direct attach with FAS <-> Server using for NFS?

sample FAS2720 e0e (10gb) <-- 10GB SFP --> Server HBA

Directly to the server without going through the switch.

cedar raptor
#

Won't survive a failover event.

#

i had a customer insist on this a while back when deploying a flexpod mini (to avoid iscsi) (which is supported)).
After it was configured. I did an HA test - (takeover/giveback) . and... boom.
The lif needs to be able to migrate to the over node.

fluid halo
#

hahah Regards

cedar raptor
#

iscsi works really well for small direct attached deployments though 🙂

#

(assuming vmware)

uneven roost
#

Though afaik neither is supported - fas doesn’t support direct attach and e-series does

#

(Last I checked)

cedar raptor
#

FlexPod Mini

#

and others.

icy wyvern
#

ONTAP version and requirement for specific firmware for Brocade SAN switches: is it documented anywhere? HWU? What I want to know - what is the minimum version of Brocade firmware required for running ONTAP 9.8, 9.10, etc

cedar raptor
#

IMT

orchid pier
#

Hi!
I am looking for a way to find out the storage efficiency rate or how much been saved by the storage efficiency for the entire cluster, using CLI/Active IQ UM/System Manager, if you can please help me out? Thank you!

vocal monolith
#

Is this a cluster that’s sending ASUP? There’s a nice easy report on aiq.netapp.com if it is. Otherwise I’ll have to go look out the exact places to check in the other interfaces, for System Manager there are some details on the dashboard/home page (depending on what version of ONTAP) and then more information on the tiers page (I think that’s the right name, don’t have one in front of me right now). AIQUM I’d have to take a look, so someone else might have a better answer.

drowsy dust
#

HOKAY.
Probably a question that will end with support, but while I wait with that I'll badger here.

I've got an unfortunate amount of systems utilizing NFS4.1 and local name-mapping on a cluster to do cross platform access from rhel/aix boxes and ntfs paths.

My issue is. I've got a local user with UID/GID 1001 that is operated by one set of messes that's mapped to Oracle for ETL processes.

I've got another system that wants to pull keys with a local account as well for automation purposes but also wants to utilize UID/GID 1001 for reasons

#

Both have been in prod for awhile and nobody wants to budge

#

Utilizing this guide again as a reference -> https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/Understanding_name-mapping_in_a_multiprotocol_environment

I pretty much have narrowed my access issues down to not having a local user for the second account.

Without forcing one of them to either
A) new SVM because stupid.
B) new UID/GID for one of them...

Is there anything else I can do to kick this can down the road?

NetApp Knowledge Base
Understanding name-mapping in a multiprotocol environment
foggy narwhal
#

Hi Everyone, does netapp have a tool to collect system inventory for Cdot? Back then we used Inventory Collection Tool for 7 Mode system. Is there a counterpart of that tool for CDOT?

elfin igloo
foggy narwhal
#

Thanks! I was reading it a while ago. Does netappdocs also function the same?

elfin igloo
tawny basalt
#

There is the OneCollect utility on the tools download area. This basically collects the same data that ConfigAdvisor collects. NetAppDocs uses a subset of the data. NetAppDocs is only available to employees and partners. If requested, e can give the NetAppDocs-Lite version to customers. This only collects data into a raw XML file.

elfin igloo
fluid halo
#

Guys, I forced a single cluster from two node clusters to do an HA test today, but no takeover was done.

I just don't understand why.

And when a failover occurs with node 1 -> node 2, the storage becomes an immediate failover, but the server times out for about 1 minute and causes a hang!
(10gb full-mesh configuration)

dark sphinx
#

Any EMS log output?

#

This is usually the kind of thing a case is good for.

fluid halo
#

I opened the case, but the HA test didn't go as I wanted, so I told here 😛

dark sphinx
#

Case #?

fluid halo
#

Every time I do HA test, autosupport is flying, so my mailbox is about to explode!

#

maybe i guess......... Case#2009310812

dark sphinx
#

HA Group Notification (REBOOT (power on)) NOTICE

#

Weird. I don't remember seeing those ASUPs during a regular TO/GB

#

But that may be my bad memory.

#

SHJSG2231000015.7: NETAPP You've got a shelf ID that is duplicate on the external shelf it looks like.

#

Or maybe both shelves.

fluid halo
#

oh i fixed that problem

#

i just rebooted all of shelfs before ha test.

#

all of shelfs got new id 00 / 01 / 02 / 03

dark sphinx
#

You're right.

#

Discrepancies exist between VLDB(Volume Location Database) and WAFL (Write Anywhere File Layout).

fluid halo
#

Oh no, did the problem happen because I didn't have enough time and shut down too fast?

dark sphinx
#

Your cluster LIF isn't home either. That may make things difficult.

#

Fix that first.

fluid halo
#

I thought the cluster was recovered because the status was normal when I said "::> show", ":> show cluster show", "::> show node show".

dark sphinx
#

Start there and see what you find.

fluid halo
#

What do you mean? Cluster LIFs each own an e0M of home node.

#

well i will check it just give me a sec

dark sphinx
#

<vif>(redacted)-01_clus2</vif>
<service_policy>default-cluster</service_policy>
<services>
asup:list
asup:licluster-core</asup:li>
</asup:list>
</services>
<role>cluster</role>
<data_protocol>
asup:list
asup:linone</asup:li>
</asup:list>
</data_protocol>
<home_node>(redacted)-01</home_node>
<home_port>e0b</home_port>
<curr_node>(redacted)-01</curr_node>
<status_oper>up</status_oper>
<is_home>false</is_home>

fluid halo
#

@dark sphinx i just read kb but i can't collect that commands right now... kk Does autosupport seem to have a lot of configuration problems?

dark sphinx
#

It would be quicker to run through those commands to rule anything out. That was all I saw right off.

#

The cluster LIF seems most likely to be the issue at first diagnosis.

fluid halo
#

hmm... i will check it tomorrow.
well many thanks for reply !

#

enjoy u r day 😄

dark sphinx
#

Yw.

#

BTW you may get contacted tonight since you opened the case late in your time zone.

fluid halo
#

The Dalian office will contact me at 9 a.m. tomorrow. It's not urgent, but I came home and created a case by organizing my backlog.

#

lol many thanks i love this community

dark sphinx
#

Yw.

#

Good night!

fluid halo
#

thanks you too

granite charm
#

I have a unix security style volume shared as both NFS and CIFS. Is it possible to change the unix permissions though the CIFS interface on a windows box?

#

I figure I can't do this directly and might need to revert to using a powershell netapp module and/or REST API on the filer to set ACL but I thought I would ask.

dark sphinx
#

Yes it is.

#

The interface isn't as extensive as a NTFS security style volume, but honestly I think this is the cleanest way. NTFS and accessing by NFS/Linux is harder IMO.

#

(nfsv3 anyway)

granite charm
#

Can you point me to some docs to help me do this?

fallen halo
#

Looking for some general help
Issue: Moving a vol/lun to a node that was recently added into my existing cluster and im wanting to ensure that all paths to the disk are properly mapped on the sql hosts to avoid a blip or outage. I know alua usually takes care of this, but wanting to be super careful here.
I've added all of the ip's under the iscsi initiator discover tab and they all show up and connect correctly. The target pool has the ip's of the new nodes in them correctly.
However when i go to add the sessions for the node under the mapped iqn of where it will be migrating to, i cannot add the session and create the friendly target. they are not listed in the session pool to be able to select.
Is this a networking issue or something that needs to be configured via the netapp/server side?

fallen halo
#

Nvm figured this one out. It was a networking setting. Everything is now lining up!

solid cedar
#

Welcome @viscid robin
I believe this will be a good channel for the question you posted elsewhere.

viscid robin
#

Hi,
Anyone there have used XCP for NFS migration?
I have a case where the linux host for XCP is running out of memory in the copy process. - We have tried to expand the memory to 64GB, but it still fails after a short amount of time. - Are there some additional performance parameter we can set to avoid this?

dark sphinx
#

They can also get in touch with the devs to help you fix it.

minor acorn
#

xcp help will give you recommended configuration settings.
We have used XCP a lot - like tens of thousands of times, both for migrations and for massive tree deletes. For xcp copies, we try to use physical servers, typically with 384GB of RAM. For xcp deletes, we can get away with a lot less and have a pool of 5 VMs deleting files all day long. When we first migrated to NetApp from a competitor, we used XCP to migrate over a petabyte over a long weekend. That was fun...

Make sure you are running a current copy of XCP - we're still on the latest 1.8.something but I think 1.9 is out.

patent flame
#

Ed have you and your team been intro’ed to XPack yet? If not, I wanna hook that up

drowsy dust
#

XCP is pretty good, we attempted to use it for a prior migration but ran into massive issues due to weird DACL's from source (old vnx5300).
It wasn't a failing of XCP, or Robocopy before that but instead legacy settings that made it improbable to migrate successfully until we found that odd setting.

That said, XCP can be a memory hog... It's a feature though! 🙂

fallen halo
#

Robcopy for us with the unbuffered io flag

patent flame
drowsy dust
#

Absolutely will

vocal monolith
#

But it will certainly move a lot of data very quickly

#

I've used it extensively for PB's of data and while it has its quirks at time, its a very robust tool

#

The dev team are great as well

stoic jolt
#

is there any trick to make a .snapshot directory visible under RHEL ?

#

ive followed this but it seems its still not visible

#

i can cd into .snapshot and do a ls and it lists all the snapshots but if i go back to the mount point and run a ls -la i cannot see .snapshot

#

i have also changed this on the SVM so you dont see the .snapshot mounted with a df

#

vserver nfs modify -vserver $name -v3-fsid-change disabled

dark sphinx
#

is there any trick to make a snapshot

stoic jolt
#

@dark sphinx ahh ok.. so been doing a bit more digging and it appears its being mounted NFS V4 (default with RHEL when you type mount) .. soon as i mount it with a -o vers=3 i can see the .snapshot.. so with V4 you will not see the .snapshot but you can cd into it

uneven roost
stoic jolt
#

so why does NFS V4 hide\not display it ? it does not hide other .folders which are created ?

dark sphinx
dark sphinx
#

Apparently newer Linux kernels aren't affected because they said different FSIDs. (I'm getting info from a NAS EE)

orchid pier
#

Hi! I need to reattach an aggr from one S3 bucket to the other, what are commands or steps to do that? It used to work by the command "storage aggregate object-store replace -aggregate agg_name -new-object-store-name new_obj_name", but it is no longer working. Please le me know your ideas.

dark sphinx
orchid pier
#

yes, I tried this link. and got the errors: *> storage aggregate object-store mirror -aggregate node-x-aggr1 -object-store-name object_x

Error: command failed: Cannot verify availability of the object store from node node-x. Reason: Invalid access key.

dark sphinx
#

Do you know why it is saying invalid access key?

orchid pier
#

No, I don't. The bucket object_x used to work before. so, the access key should work. Anyway, how can I change / view the access key if it is not correct?

#

My understanding was the access key has been already put in objiect-store configuration, so, the same access key that used to be working should still work now, I assume

dark sphinx
#

Maybe try readding them?

orchid pier
#

how?

dark sphinx
#

::>set diag
::>diag object-store client bucket modify -node node1 -handle 0 -access-key XXXXXXXXX -secret-key XXXXXXXXXXX
::>diag object-store client bucket modify -node node2 -handle 0 -access-key XXXXXXXXX -secret-key XXXXXXXXXXX

orchid pier
#

So, the command is going to update the access key and secret-key, and not to view, right?

#

that is going to replace whatever the currently access key and secret key, right?

dark sphinx
#

Yes. Do it on all nodes of the cluster.

#

Right.

#

You can reuse the existing one I'd imagine on the S3 side.

orchid pier
#

OK. I need to get keys from AWS admin, right?

#

@dark sphinx I got the access key, how do i continue with the command storage aggregate object-store mirror -aggregate node-x-aggr1 -object-store-name object_x?

#

because it doesn't seem accepting any more paramenters, if I tab it.

#

"command is complete"

#

In addition, I got the access-key by running " storage aggregate object-store config show -object-store-name object_x", so, I don't know if it makes sense by enter the same access-key again...

dark sphinx
#

Are you in diag?

orchid pier
#

here is my question:
cluster1::> storage aggregate object-store attach -aggregate aggr1 -name my-store-1
for this command, "my-store-1" would be my source object store, right?

#

my-store-2 in the second command would be the destination store?

dark sphinx
#

The one with the error.

orchid pier
#

cluster1::> storage aggregate object-store attach -aggregate aggr1 -name my-store-1
cluster1::> storage aggregate object-store mirror -aggregate aggr1 -name my-store-2
My questions was, 'my-store-1" is the source object store, and "my-store-2" is the destination store.

#

Corrrect?

dark sphinx
#

Looks correct.

#

But it should need another parameter on the second command.

#

No guess not.

inner remnant
#

I am have enabled CIFS auditing on one of my vservers. Afterwards I accessed the vserver over CIFS and created a directory, created a file, deleted the file and then deleted the directory I created. I figured there would be events in the audit log showing that i did this, however when I search for my AD username in the audit log I am not seeing any activity for my user at all. Should I expect to see logs that show that I performed the steps I outlined in the CIFS audit log? I currently have the following event types enabled file-ops,cifs-logon-logoff,cap-staging,file-share,user-account,security-group,authorization-policy-change,audit-policy-change. I figured the file ops event type would have included that sort of information. I do see activity from other hosts that are joined to AD in the cifs audit log, but nothing from my own user.. Any ideas?

rose thicket
#

Anyone that can help with SnapCenter plugin and SVM-DR?

pearl coral
#

iirc you can combine them but snapcenter will only restore from a vol based snapmirror not from svm-dr

fallen halo
#

Anyone know a way to output from cli volume to lun mappings with igroups or initiator associations?

cedar raptor
#

lun mapping show -fields igroup,initiators

#

?

fallen halo
#

I mean a way to show from 1 output vol, associated lun and initiator

#

The above doesn’t show vol the luns are under

#

Only option I can think of would be to try ps cli and try to pipe the 2 cmds together

#

Or I break out the path to get the vol info

cedar raptor
#

lun mapping show -volume <volumename> -fields igroup,initiators

#

?

#

or you can add volume to the fields list.

#

if that's not it. i'd need to see an example.

brisk swan
#

Hi, could some advise the best approach to this scenario.

We have aggregates that have FabricPool tiering assigned. This is all working fine. But need to enable NAE, this will require the volumes be encrypted NVE first, and then NAE afterwards.
We don't have a huge amount of free space to play with.

If we run vol move or vol encryption conversion commands to satisfy the above, will this reinflate the volume and pull all the data back from the FP tier?

elfin igloo
#

Hey @brisk swan I think this KB article might answer your question - <edit to remove incorrect KB link>

#

If not, then given your space concerns, a support case might be advisable so support can double check the expected behaviour before you go changing things 🙂

brisk swan
elfin igloo
brisk swan
stoic jolt
#

if you have StorageGRID setup to Archive/Tier data off to an Cloud Storage Pool eg.. AWS S3 bucket using ILM rules..What is the process to recall that data back OnPrem into the GRID ?

fluid halo
#

Hi guys.

I'm looking FlexGroup Snapshot autodelete scripts.
I just checked NetApp Ontap GitHub but I'm new to python so i can't create python script 😂😂😂

Does anyone has Snapshot autodelete scripts ?

** FlexGroup Volume dosen't support snapshot autodelete

Please DM me or tag me.

storm spade
#

Hey guys; just wondering if anyone can provide some guidance with DataPriv?
Is there an option to Make Traverse Permissions for existing groups?

stoic jolt
#

im trying to wipe an older 2650 which has NSE drives so i ran the command 'storage encryption disk sanitize' which wiped the drives but now all 24 drives are appearing as broken '1.10.23 1.63TB 10 23 SAS broken -'.. how do i move these out of broken state so i can create a new AGGR ?

fallen halo
#

If they are nse drives, I believe you have to 0x0 both the fips key and data key
Storage encryption disk modify -disk * -fips-key-id 0x0
Storage encryption disk modify -disk * -data-key-id 0x0

stoic jolt
#

looks like it must be common for them to be marked as broken

#

storage disk unfail -disk disk_name -s -q

#

your commands the the unfail seemed to bring them back

uneven roost
icy wyvern
#

lun share from 7-mode days. 7-mode transition guide says lun share is not supported in Clustered ONTAP. Is there any other way to export FC/iSCSI LUN as NFS share, so it can be mounted as loopback mount on Linux server for inspection?

inner remnant
#

I noticed that there is an option to redirect the audit logging to a different SVM on our NetApp. When I tried to create the audit logging redirection, I am getting the following error ntap3::*> vserver audit audit-log-redirect create -vserver svm1 Error: command failed: Audit Log Redirect configuration not available. Is there some other way I need to configure the audit logging redirection in order to have all of the audit logs get forwarded to a separate SVM that does nothing but hold my audit logs? This would be preferable over having to configure audit logging directories on each SVM.

dusk lava
#

this is a bit of a stretch, but: has anyone successfully configured OCUM and/or AIQUM with a non-admin user? I can't get it to work for love or money; specifically, it seems to choke when the cluster is added and I get a "Cannot add datasource" error

devout tiger
#

do you mean using a non admin account to authenticate to the cluster?
aiqum needs write permissions for several apis. admin is the supported role. you can add with admin and switch to a limited account after the initial acquisition, if you're ok with a few things not working in aiqum properly

dusk lava
devout tiger
#

give me a few to refresh my memory on that. it's been a while since i looked at the cert error when adding a cluster.

dusk lava
#

thanks Dawn! I've checked the clocks are all synced, time zones are the same, and the cluster isn't being monitored by any other instances

#

(those are things called out by the error message, but I think that's actually a red herring)

devout tiger
#

Does UM's common/SAN name in the certificate match it's name in DNS? the command used to install the um cert on the cluster should be the 'security certificate install' command. assuming that has write, that should be fine, and mean it's an issue with the cert itself.

orchid pier
#

Hi, We have a volume which is used for storing the backups of a DB2 DB. The DB size is only 2TB. But, the size of single snapshot sometime took about 8-9TB. It happened not every time, but very often. My understanding is that the maximum size of a snapshot could be 2TB in the worst case when all blocks got overwritten or changed. But, how come in this case, we got 9TB snapshot. Can you please logically explain why?

stable grove
#

How did you identify that the snapshot takes 9TB?

orchid pier
#

By using "snapshot show volume_name -instance", the output would show you the size on every snapshot.

stable grove
#

Use it on the "9TB" snapshot, I doubt that it will spit out 9TB

foggy narwhal
#

Does the NetApp 7 mode Migration Tool compatiblr with the latest version release? (ONTAP 9.11)

stable grove
#

Nope, officially it's only supported until 9.9.1

vocal monolith
#

Watch this space

dry dirge
#

Hi All

I have a question about the predefined "vsadmin" and "Administrator" SVM accounts. I don't think we use them. Can they be deleted ?

dusk lava
stable grove
dusk lava
#

thank you!

vocal monolith
#

There’s a new version dropping soon, I just don’t have a date for you. So keep an eye on it.

dusk lava
#

is there a way to tell, from the ONTAP CLI, if a cluster is being managed by OCUM/AIQUM? and manually remove the monitoring UM instance?

from memory there is, but that memory is really hazy. the primary use-case was something like "my UM instance caught fire and my cluster still thinks it's being managed by a zombie"

dark sphinx
#

Diag level

#

And yes you can remove it.

turbid vale
#

i'have updated NetAppDocs from 3.5 to 4.0, and get a nice error when trying to import the module.. As i'm not an expert ( first time user ) , can someone point me to the direction which might solve this issue ?

Schermafbeelding_2022-11-11_om_08.21.35.png
tawny basalt
#

Wasn’t there some .NET dependency as well?

Try installing and using the latest power shell 7.

I use that all the time now without a problem

vocal monolith
# 
Windows PowerShell 5.1+ and PowerShell 7.0.3+.
Microsoft .NET Framework 4.5 and higher.```
#

Make sure the pre-reqs are met, you may need to enable .NET or update the version in use, also check the PowerShell version

#

If this is a recent version of Windows 11, 22H2 for example, some versions of .NET are disabled by default

foggy narwhal
#

Hey guys, anyone experience creating a datastore on a vmware host. The FC disk can be detected, we also checked the zoning configuration and it was logged in. But when creating a datastore from that device. We encounter an error:

Failed to create VMFS datastore Gelo_test - Operation failed, diagnostics report: Unable to create Filesystem, please see VMkernel log for more details: Failed to create VMFS on device naa.600a098038314a44653f546e55555130:1

uneven roost
foggy narwhal
uneven roost
#

and the volumes the luns are in?

foggy narwhal
# uneven roost and the volumes the luns are in?

Found the issue. It seems there has been incompatibility between the NetApp and the FI. The SFP on the netapp can support up to 16GB. but the SFP on the FI only has 8GB, we changed the SFP on the FI with a 16GB one, and we check the previous logs and we got a lot of Discards on the FI logs. After the replacement of SFP we got 0 Discards and can create datastores from the FC devices

uneven roost
#

oh interesting!

#

thanks for the followup

tawny basalt
#

For what it’s worth, you may have been able to force the speed on the Netapp side down to 8. That might have helped also. Better off with full 16 in my opinion

carmine trout
#

I am running 9.9.1p3, if I have a volume called "vol_clone", and when I run "vol clone show -flexclone vol_clone", I didn't get output back. Does that mean the "vol_clone" has been already separated from the volume "vol" and already become an independent volume? which also means that I am no longer able to identify what was it's parent-volume or parent-snapshot?

cedar raptor
#

if it's missing from "vol clone show", it's no longer a clone. and was split.

#

you can also check the vol in question by -fields clone-volume

#

WOPR::> vol show -volume NFS_test_vol01 -fields clone-volume
vserver volume clone-volume

NFS NFS_test_vol01 false

carmine trout
#

Thanks!

carmine trout
#

I have one more question about a clone and split. If vol_clone is spit from it parent volume, will it also contain those data in the parent volume in addition to the new writes to the clone?

cedar raptor
#

yes. a clone itself cannot exist without the parent. So when you split it, it copies everything with it that it needs to surive and be on it's own.

carmine trout
#

The clone was created for an application temporarily, then after split, "vol_clone" became a production volume for the app, most part of data were copied from the parent volume and don't needed by the app. To clean it up, we would have to migrate data that the app needs on "vol_clone" mounted on the client to a new mounted volume, then remove vol_clone. Right?

elfin furnace
#

Is it possible to generate ontap capacity report , which reports inactive data older than 30, 90 and 120 days, in a system with fabric pool enabled ?

cedar raptor
carmine trout
#

👍

sand flint
#

Hi there,
I would like question you the following:
I would like to know how can I find out which are the servers which are mounting a specific share presented from the storage cabin
the share it`s a CIFS

turbid jasper
sand flint
#

Many thanks, for your great support @turbid jasper

latent flume
#

Hi, TIA for those who will answer: Is there a way to Migrate our 2tb Windows file share with almost a hundred paths inside to NetApp NAS_CIFS?

#

Thanks..

cedar raptor
#

various ways. -
Some folks do use robocopy
There's also software like datadobi and storageX that's used, but costs $ to use.
NetApp has a tool called XCP - Which is free for customers.

patent flame
#

Komprise is also a great partner of ours, knows and integrates with all our stuff, and specialize in orchestrated migrations.

#

It really depends on how much hand-holding you want. If you’re savvy/advanced enough, the right Robocopy parameters will get it done. If you want something a little more white glove and UI focused, I’d recommend Komprise.

latent flume
#

Thanks..

eager marsh
#

Hope this is not the wrong channel. But I curious if netapp filers do not support sending data tiered down to any other S3 platform apart from storage grid

uneven roost
proper matrix
#

hey, does anyone have any idea how to enable server-side copy on a SMB share on a FAS ONTAP 9.10.1P7

crystal solar
#

you mean copy-offload? and you want to enable or disable it?

proper matrix
olive umbra
proper matrix
#

Just wondering why deduplication is needed.. is it on per default?

dark sphinx
proper matrix
#

so i should be able to turn on ODX. No reboot, no restart of SMB, and each new SMB connection would be able to use ODX

stable grove
#

Any S3 / FabricPool experts in here?
I'm struggling to understand which LIFs communicate with each other when using ONTAP S3 as a capacity tier.
So my scenario: 2x clusters (one AFF, one FAS), both in the same DC, maybe even on the same switches. (It's no 4-node cluster, but 2x 2-node-clusters).

According to the docs and KB, the following network config is needed:
On my source AFF (performance tier) I need a intercluster LIF per node.
On my destination FAS (capacity tier):

  • I need a S3 data LIF in my S3 SVM (best practice is for both nodes but not necessary)
  • I somehow also need intercluster LIFs here (I don't get why...)

I also don't need to peer both clusters.

What I don't understand is how the source communicates with the destination.
Is it: source-intercluster-LIF --> destination-intercluster-LIF ?
Or: source-intercluster-LIF --> destination-s3-data-LIF ?
Or even both? 🤨

wise reef
#

Good afternoon netapp ppl - does anyone have any good resources for CLI management of an FAS2040 running Release 8.1.4P1 7-Mode? I've been attempting to track down resources on this and it's a legacy debt item which I need to safely decomm

#

anywhoo, if anyone out there sees this and can advise/assist with a resource or URL, would be much appreciated. Otherwise my search continues

image.png
stable grove
wise reef
dark sphinx
stable grove
#

That's clear, but why do I need IC LIFs on the destination?
The data LIF inside the S3 SVM is my S3 server IP.
The IC LIFs on my source cluster are my S3 client IPs.

#

Why do I also need IC LIFs on my destination server?

dark sphinx
#

Because FabricPool uses the IC LIFs.

stable grove
#

Well, FabricPool is only really active on the source side.

#

On the destination it's a simple S3 server. Just like in AWS or StorageGRID. The S3 server on my destination cluster does not know that the S3 client is FabricPool. It could be any other S3 client.

#

So why should I need IC LIFs on the destination?

dark sphinx
#

Because that's how ONTAP works.

stable grove
#

Paul, I know all these links 😬
but ok, I'm just gonna accept this ONTAP quirk

What I'm still missing: Do I need to put all these LIFs on the same network?

  • IC LIFs on source cluster
  • IC LIFs on destination cluster
  • S3 data LIF on destination SVM
dark sphinx
#

IC LIFs need ethernet connectivity, but I don't think the data LIF needs to be on the same as the IC LIFs.

stable grove
#

ooohhhh ok wait a minute

#

So all the FabricPool traffic goes between the IC LIFs on source and destination?
And the S3 data LIF is only really needed to.... talk to DNS?

dark sphinx
#

S3 data LIF assumes you aren't talking to just ONTAP for FabricPool but any S3 client needs.

stable grove
#

I'm gonna need to try this out, but this would explain so much....

#

basically I put the S3 data LIF on my mgmt network

dark sphinx
#

Yeah that makes sense why you were getting confused. 😄

#

ONTAP S3 server != FabricPool only.

stable grove
#

and all the FabricPool LIFs on their own network (in their own VLAN and IPspace)

stable grove
dark sphinx
#

Oh wait...

#

Derp.

stable grove
#

😄

dark sphinx
#

You might be right. I was messing it up.

stable grove
#

I'm still totally unsure if I am 🙈

dark sphinx
stable grove
#

Maybe FabricPool uses the S3 data LIF for the initial connection to the bucket and then afterwards the IC LIFs for the traffic?

dark sphinx
#

No I had it backwards.

#

I told wrong.

stable grove
# dark sphinx

yeah but this does not explain why the docs says I should also create IC LIFs on the destionation 😄

dark sphinx
#

The FP S3 client (frontend node) uses IC, the backend uses data LIF.

stable grove
#

It even says "cluster peering not required"

#

which obviously means IC LIFs on both source and destination

stable grove
#

so this KB does not mention at all that IC LIFs might be needed on the destination too...

dark sphinx
#

I see why you're confused.

dark sphinx
#

I'd suggest a feedback on this KB as a start.

stoic jolt
#

how do i add a route to a SVM through system manager these days ?

tawny pollen
#

regarding the S3 conversation, maybe when you try to retrieve tiered-off data from the capacity S3 nodes, it pulls it via S3 protocol rather than through intercluster LIFs? Just my guess why you might need both?

#

Could always throw a few temporary unlicensed ONTAP Select instances out there and test how it really works. I think you can apply the S3 master key to them to enable it, even if the nodes are on eval licenses.

stable grove
#

I'm currently consulting another team which already implemented over 10x FabricPool configs how they done it. My wild guess would be: No IC-LIFs needed on S3 server cluster.

#

In our lab I have a running config currently where the destination cluster has no IC-LIFs at all and it works without issues. There's no real workload on it but the Profiler ran perfectly fine PUTing and GETing blocks and objects...

spark pewter
#

Quick question on data layout best practices. I have 2 AGGR on a AFF8080. Should I be trying to put all my VM's on one, and SQL on another, in order to maximize deduplication?

cedar raptor
#

yeah. generally if you know certain data sets have simalier data, put them on the same aggr (with AFF*).
note: I would also not forget about performace either.

spark pewter
#

OK makes sense. What do you mean by not forget about performance? similar loads perform better together?

cedar raptor
#

balance the workload. you don't want to over drive one controller and leaving the other not doing much of anything

spark pewter
#

right right of course, i've got pretty even loads between controllers, fortunately

#

While I'm here, I want to take a few shelves from an old system, and move them to a new one. One shelf has data i MUST keep. The other shelves I can start from scratch.

What's the best guide to this? I see guides for moving shelves that have system vols, but I don't need that. I also want to make sure I add those "empty" drives to my system in teh most optimal way, including utilizing AFP

cedar raptor
#

in the 7mode days it was decently straight forward. Alex's Comment here is still valid, if you want to do it and preserve data -https://community.netapp.com/t5/ONTAP-Hardware/Physically-moving-an-aggregate-composed-of-disks/m-p/139623#M8770

Physically moving an aggregate composed of disks

Hello,   we have a FAS8040 with two DS4486 shelves, Ontap 9.1P11 and a FAS8060 Ontap 9.1P11. We want to remove these two shelves with existing aggregates and datas, and then add to FAS8060, is the below kb OK for that, or we should do more than that?   https://library.netapp.com/ecmdocs/ECMP1196986/...

#

IMHO, you're better off to use snapmirror to move it.

spark pewter
#

the problem is, i don't' have the capacity on the new system to move it over. I'm trying to migrate over to my new system as safely as possible, but i have to do it a bit by bit because i don't have enough space to simply snapmirror all over

#

what about the empty disks, when they get mounted and added ot the existing aggr they will automatically get cut in half and utilized for AFP?

cedar raptor
#

ADP? it depends?

#

ADP is only done in initilization of the system.

#

but if you add disks later on, and they fit with in the RG. they'll be paritioned.
also, check HWU, there is a max supported ADP disks per model.

uneven roost
cedar raptor
#

yeah. shame he took down his blog though. i get a not found.

spark pewter
#

ok ya, i had done this several times in the old days, but never on 9.x. as you said, it used to be very easy

uneven roost
uneven roost
spark pewter
#

ok, maybe i'll swing that 20tb over to another file server, move my shelves, then copy back

spark pewter
#

Migrating from an 8.3 release to 9.7, there's no way to copy a LUN/VOL over WITHOUT SnapMirror between two separate controllers is there? I can't use SnapMirror, because they are incompatible versions

#

otherwise i have to copy millions of files inside the LUN with something like a robocopy job

cedar raptor
#

what's the 8.3 box? model #

#

otherwise. yeah. copy.
xcp/rsync/robocopy from the NAS side.
LUN... inside the host copy from old disk to new disk would probably be the easist.

#

or FLI.. Can't recall the verion compatibility off the top of my head on that.

spark pewter
#

8040 to 8080, right now I have no better plan than to present the source and destination luns to the same machine, then start the copy

cedar raptor
#

can you not upgrade the 8.3 box? looks like you could snapmirror if you can get it to 9.3

spark pewter
#

I cannot unfortunately

dark sphinx
#

Why not?

#

Entitlements?

spark pewter
#

No longer have any support, I buy my gear on eBay so I am stuck with what I have 😦

uneven roost
#

I commend you for almost uncommon adherence to our software license in that regard, though it’s worth considering that unless you somehow got us to transfer ownership and support, you’re likely out of technical compliance with it anyway.. so.. govern yourself accordingly? We are here to deal with technical questions, not legal ones however.

#

So don’t feel bad asking I guess is what I’m saying

spark pewter
#

Ha thanks Alex! I don't even have a way to download it so it's a moot point unfortunately. I love having access to this group though, huge bonus for someone who's used Netapp for 20 years

spark pewter
#

I create all my resource groups and snapshot/protection policies in SnapCenter. In On Command, all those VOLs show unprotected, even though they all have snapshots.

Is that expected behavior? You'd think the two would be integrated so that if there was a snapshot policy in SnapCenter it satisfied the protection requirement

stable grove
#

Nah, unfortunately not. ONTAP has no idea that some other backup application will regularly trigger snapshots according to its own schedule.

#

Integrating that info into System Manager would be possible though since SC can login to ONTAP. I guess what's missing is the other way around: ONTAP needs to be certain that SC is actually still alive and is actively creating snapshots before it could say "this volume is protected" with some confidence.

#

But that's sounds like a good RFE for me 😉

spark pewter
#

So what do people do. Have some very basic snapshot job run on the NetApp directly?

stable grove
#

You can do that if no application- or crash-consistency is needed for your backups.

#

The notice in System Manager if a volume is "protected" or not is only for visibility. If you let other backup-applications trigger your snapshots that's fine too. Many backup-vendors have storage-integrations with their software. Like Veeam, CommVault, etc.

dark summit
#

Hi there, we have 2*24 SSD Shelfs with ADP2. 2 Spare partitons = 46 usable "Disks" Is it better to create one RG with 24 disks and one with 22 disks. Or should we use two 23 Disk Raidgroups?

#

First Plex

RAID Group rg0, 24 disks (block checksum, raid_dp)
Usable Physical
Position Disk Type Size Size
---------- ------------------------- ---------- -------- --------
shared 13.40.6 SSD-NVM - -
shared 13.41.6 SSD-NVM - -
shared 13.40.0 SSD-NVM 3.49TB 3.49TB
shared 13.41.0 SSD-NVM 3.49TB 3.49TB
shared 13.40.7 SSD-NVM 3.49TB 3.49TB
shared 13.41.7 SSD-NVM 3.49TB 3.49TB
shared 13.40.1 SSD-NVM 3.49TB 3.49TB
shared 13.41.1 SSD-NVM 3.49TB 3.49TB
shared 13.40.8 SSD-NVM 3.49TB 3.49TB
shared 13.41.8 SSD-NVM 3.49TB 3.49TB
shared 13.40.2 SSD-NVM 3.49TB 3.49TB
shared 13.41.2 SSD-NVM 3.49TB 3.49TB
shared 13.40.9 SSD-NVM 3.49TB 3.49TB
shared 13.41.9 SSD-NVM 3.49TB 3.49TB
shared 13.40.3 SSD-NVM 3.49TB 3.49TB
shared 13.41.3 SSD-NVM 3.49TB 3.49TB
shared 13.40.10 SSD-NVM 3.49TB 3.49TB
shared 13.41.10 SSD-NVM 3.49TB 3.49TB
shared 13.40.4 SSD-NVM 3.49TB 3.49TB
shared 13.41.4 SSD-NVM 3.49TB 3.49TB
shared 13.40.11 SSD-NVM 3.49TB 3.49TB
shared 13.41.11 SSD-NVM 3.49TB 3.49TB
shared 13.40.5 SSD-NVM 3.49TB 3.49TB
shared 13.41.5 SSD-NVM 3.49TB 3.49TB

#

RAID Group rg1, 22 disks (block checksum, raid_dp)
Usable Physical
Position Disk Type Size Size
---------- ------------------------- ---------- -------- --------
shared 13.40.18 SSD-NVM - -
shared 13.41.18 SSD-NVM - -
shared 13.40.12 SSD-NVM 3.49TB 3.49TB
shared 13.41.12 SSD-NVM 3.49TB 3.49TB
shared 13.40.19 SSD-NVM 3.49TB 3.49TB
shared 13.41.19 SSD-NVM 3.49TB 3.49TB
shared 13.40.13 SSD-NVM 3.49TB 3.49TB
shared 13.41.13 SSD-NVM 3.49TB 3.49TB
shared 13.40.20 SSD-NVM 3.49TB 3.49TB
shared 13.41.20 SSD-NVM 3.49TB 3.49TB
shared 13.40.14 SSD-NVM 3.49TB 3.49TB
shared 13.41.14 SSD-NVM 3.49TB 3.49TB
shared 13.40.21 SSD-NVM 3.49TB 3.49TB
shared 13.41.21 SSD-NVM 3.49TB 3.49TB
shared 13.40.15 SSD-NVM 3.49TB 3.49TB
shared 13.41.15 SSD-NVM 3.49TB 3.49TB
shared 13.40.22 SSD-NVM 3.49TB 3.49TB
shared 13.40.16 SSD-NVM 3.49TB 3.49TB
shared 13.41.16 SSD-NVM 3.49TB 3.49TB
shared 13.40.23 SSD-NVM 3.49TB 3.49TB
shared 13.40.17 SSD-NVM 3.49TB 3.49TB
shared 13.41.17 SSD-NVM 3.49TB 3.49TB

Aggregate capacity available for volume use would be 131.9TB.

#

or

#

RAID Group rg0, 23 disks (block checksum, raid_dp)
Usable Physical
Position Disk Type Size Size
---------- ------------------------- ---------- -------- --------
shared 13.40.6 SSD-NVM - -
shared 13.41.6 SSD-NVM - -
shared 13.40.0 SSD-NVM 3.49TB 3.49TB
shared 13.41.0 SSD-NVM 3.49TB 3.49TB
shared 13.40.7 SSD-NVM 3.49TB 3.49TB
shared 13.41.7 SSD-NVM 3.49TB 3.49TB
shared 13.40.1 SSD-NVM 3.49TB 3.49TB
shared 13.41.1 SSD-NVM 3.49TB 3.49TB
shared 13.40.8 SSD-NVM 3.49TB 3.49TB
shared 13.41.8 SSD-NVM 3.49TB 3.49TB
shared 13.40.2 SSD-NVM 3.49TB 3.49TB
shared 13.41.2 SSD-NVM 3.49TB 3.49TB
shared 13.40.9 SSD-NVM 3.49TB 3.49TB
shared 13.41.9 SSD-NVM 3.49TB 3.49TB
shared 13.40.3 SSD-NVM 3.49TB 3.49TB
shared 13.41.3 SSD-NVM 3.49TB 3.49TB
shared 13.40.10 SSD-NVM 3.49TB 3.49TB
shared 13.41.10 SSD-NVM 3.49TB 3.49TB
shared 13.40.4 SSD-NVM 3.49TB 3.49TB
shared 13.41.4 SSD-NVM 3.49TB 3.49TB
shared 13.40.11 SSD-NVM 3.49TB 3.49TB
shared 13.41.11 SSD-NVM 3.49TB 3.49TB
shared 13.40.5 SSD-NVM 3.49TB 3.49TB

stable grove
#

Performance-wise makes absolutely no noticeable difference. Especially with SSDs and even more with NVMe-SSDs. I would do 2x 23-raid-groups.

dark summit
patent flame
#

We would SM to offsite/corp, and then SV to a partner system sitting next to the offsite and a separate tape library managed by NetBackup to pull NDMP off the SV-secondary box.

#

We did that strategically to be able to fully restore applications and associated data independent of any backing storage arrays. But the bulk of our data was unstructured, or backup sets themselves (ISO repo, for example) and didn't require any consistency.

#

Exceptions were Oracle, Exchange, and MSSQL that used NetApp snaps via SMO/SMSQL/SME (now SnapCenter)

spark pewter
#

Thanks @patent flame . I had designed my system around using SnapCenter for backing up our SQL loads, but it seems to be too buggy. I've gone from being able to backup most our SQL to Snapcenter not even seeing any of the new DB's added.

leaden pike
#

Silly question, and not sure if this is the right place but, from the CLI, I am using vol show -vserver oldsvm -fields used,size to grab a listing of all the space used my volumes on old VMs. This works fine but used and size return values in different units, MB/GB/TB and so on. Is there a way to force the output to be in one unit, say MB or GB only?

cedar raptor
#

the , is helpful if you ever need to make a csv/xlsx doc.

leaden pike
#

thanks, exactly what I was looking for!

tawny basalt
#

And if you use “set diag” mode there is a truly hidden option “-sort-by” that allows multi sort. The option itself will not tab complete, but after it is typed you can use tab to see available sort fields

cedar raptor
dusk lava
#

@cedar raptor that is a great blog. I just wish there was a way to grep! Cisco NX-OS (and maybe IOS these days) lets you do | i foo which will grep the output for foo

cedar raptor
#

yeah. there is kinda though, depending on the command, you can also do "not"! i believe too still. but I get what you're saying.

back when I was an admin i'd just SSH into a system from a linux jump box and | grep as needed.

tawny basalt
#

vol show -aggregate sata_01|sata_02 -volume !vol0,!root -fields size, used, avail, aggregate -sort-by vserver, aggregate, used

minor acorn
tawny basalt
#

Agreed

waxen frost
proper matrix
#

hey guys.. is it possible to enable ODX just for some volumes?

on the Netapp Wiki "vserver cifs options modify -vserver vserver_name -copy-offload-enabled true" has no option to select the volume

my StorageVM has like 10 volumes and only 1 has dedub enabled. So when i turn on ODX for the whole SVM it will turn on ODX only on that one volume?

uneven roost
tired radish
minor acorn
#

It's a bad workaround for missing functionality.

tired radish
#

its not ideal

hearty flax
#

Hello everyone, Can anyone tell me which command I can get the view on CIFS operations as shown in the image.

image.png
dusk lava
#

it's just a guess but that looks like the output from the smb2:vserver counter?

Try this and see what you think: statistics catalog counter show -object smb2:vserver

Actually I just gathered those statistics myself and your screenshot has a different output:

# 

Object: smb2:vserver
Instance: vs1
Start-time: 12/27/2022 14:32:48
End-time: 12/27/2022 14:33:34 
Elapsed-time: 46s
Scope: FsxId0a12345678
Number of Constituents: 2 (complete_aggregation)
    Counter                                                     Value
    -------------------------------- --------------------------------
    op_pct                                                          -
                             GetAttr                                0
                             SetAttr                                0
                                Read                               18
                               Write                               18
                                Lock                                0
                          Open/Close                               36
                           Directory                                0
                               Other                               27
hearty flax
dusk lava
hearty flax
dusk lava
stable grove
#

Hi all, did anyone try out the SVM migrate feature already?

#

We have a usecase for a customer, for a techrefresh from FAS8200 to AFF A250.

#

Is this really an absolute show-stopper then?

#

Or can I start the SVM migrate with "-auto-cutover false", manually add the qtrees and then do the manual cutover?

dusk lava
orchid pier
#

We are adding 2 and half more SAS shelve(DS224C) with 60x1.8 disks onto 2 FAS8200 controllers within 4-nodes cluster. The existing aggr configurations are attached below. I am thinking to use one of two configurations below:

  1. Use all 60 disks(~90TB) to create a new and 2nd aggr on node_02. Since node_01 already has a 78TB SAS aggr, so, both node would have two SAS aggrs, and should then have SAS load balanced between them.
  2. Add all 60 disks by creating 2 or 3 new raid-groups into node_01_sas_aggr2, then it will have total of 168(78+90)TB SAS aggr. The concern is that then this node will have much more loads than the other.

Taking account of both Performance and loads balancing betwen 2 nodes, which one I should use to configure, and are there any other better solutions? Please advise.

image.png
stable grove
#

Do you have big volumes that also need to grow? They can only reside inside one aggr so increase the size of that aggr.
You could also use FlexGroups (which are volumes made up of subvolumes/constituents) which can reside on many different aggrs. So you could do your load-balancing like that. You can do that by either converting your current FlexVols to FlexGroups (which has some limitations) or by creating new FlexGroups and moving your data over.

Another consideration: Compare the average CPU load of both nodes. If the CPUs of node_02 are already very busy it makes no sense to add more disks to it (with possibly even more workload).

Also: You should check the current raid-group size of the node_01_sas_aggr2. If you want to add new raid-groups they should consist of roughly the same size than the current raid-groups.

#

If the load from the SATA aggr on node_02 is negligible I would create a new aggr on node_02 I think.

orchid pier
#

We will configure the new space without considering FlexGroups.

  1. Currently, CPU load is not too high on both controllers. "node_01" is higher with 57% and "node_02 with 37% for 95th percentile utilization. It's hard to say if CPU load would be too high after added 90TB onto the existing aggr node_01_sas_aggr2, and the same node is already with higher CPU load.
  2. Currently node_01_sas_aggr2 has 4 rg's with 23 disk in each rg. If I add 60 more disks, can I do 2 rg's with 28 disks each and 4 for spares? It's hard to determine what size of the new riad-groups, right?
    Any additional thoughts based on above please?
cedar raptor
#

@orchid pier Do any of the aggrs have high utlization?

But a second / new sas aggr on node 2 might be the direction i'd go.

orchid pier
#

We might want to look into "node unitl" not "aggr util", in my opinion. Currently, node_01 has 57% 95 percentile util. and node_02 has 37%. It's hard to forecast what util would look like after add more workloads onto any nodes. Thanks for your messages, @cedar raptor If we would go for 2nd/new aggr on node-02, I could create the new aggr with 2 x 29 raid-dp's and leave 2 for spares. Does the size of 29 disks for a raid-dp sounds alright to you?

stable grove
#

29x Disks in one RAID-DP raid-group is not possible. You need RAID-TEC for that.

#

28x Disks is maximum for RAID-DP with SAS-disks but that's a bit high for that many disks.

#

check "Supported RAID Configurations" in HWU

stable grove
cedar raptor
orchid pier
#

@cedar raptor Would you say 28 is bit too high for RAID-DP, and RAID-TEC with 3 disks for parity disks as @stable grove said?

cedar raptor
#

it's not a common thing, but keeping performace on spinning drives in mind, yeah, you're better off with 2 RGs.

orchid pier
#

Sorry, are you saying to use 2x28 RAID-DP groups?

cedar raptor
#

with 28 disk. it would be 2x 14 disks.

orchid pier
#

I have total of 60 new HDD disks, 1.8TB each. Should we create 2 RAID-DP's with 28 size and leave 4 for spares?

stable grove
#

I would do 2x RAID-TEC with 29x disks if you have a valid support contract with NetApp.

#

Two spares are enough and it does not happen really often that another disk breaks while you are waiting for the replacement disk from NetApp.

#

Keep your disk and shelf firmware up-to-date and you are good to go

cedar raptor
#

What's the current spare count?

i put the new drives into fusion -

3x RG with 20/20/18. (or 20 depending on your spare count) . gives you 76TiB

  • or -
    2x RG with 28, leaving 4 spares - 76TiB

2x RG with 29 RAID TEC leaving 2 spares - 76TiB

orchid pier
#

First of all, if we create a new aggregate, would spares for the aggr share the same spares on the other aggr?

cedar raptor
#

1.8 SAS will share across the node.

#

(typically)

#

like disks *

carmine trout
#

only if they are in the same size of SAS disks, right?

#

If that's the case, there are 7 spares

cedar raptor
#

it'll right size if it has too.

#

i.e. if you had 960s and 1.8. and no spare 960s. it would right size a 1.8

carmine trout
#

960s? what's that

cedar raptor
#

960GB disks

orchid pier
#

ok. right, there are 7 x 1.8 spares

#

So, based on the list of options you gave, 3xRG with 20/20/18 would be the same as 2xRG with 28 and 4 spares, right?

cedar raptor
#

plenty. i'd go 3RG at 20 disks each. RAID DP.

#

3x20 disks gives you 79TiB

carmine trout
#

better.

cedar raptor
#

the 18 was if you needed spares. but 7 is a good number of them.

orchid pier
#

right.

cedar raptor
#

it also lets you grow should you fill in that half a shelf.

orchid pier
#

If I run "storage disk show", I got 1.63TB usable space disks back. Those are actually 1.8TB disks, right?

#

I just wanted to make sure those new 60 HDD's are really the same size as existing spares

cedar raptor
#

what are the model numbers

#

exisitng vs new

#

storage disks show -fields model

orchid pier
#

X343_TA14E1T8A10

#

only one of them with X343_SSKBE1T8A10

cedar raptor
#

x343. same disk

#

there's your 1.8 vs 1.6 too

orchid pier
#

i'd go 3RG at 20 disks each. RAID DP, and gives me 79TiB. Great!

#

Thank you so much for your help here!

cedar raptor
#

no prob

rose thicket
#

Any #ONTAP RAID SME's here?

rose thicket
#

Can I mix different sized discs in an aggregate by creating a new raid group?

patent flame
orchid pier
#

@cedar raptor or other experts:
Will appreciate it if you can advice on following two cases about how we should configure the new space into aggregates:
**1. Adding 18 x 3.8TB SSD’s onto 2 existing A300 controllers. **
Currently, there is 24x3.49TB usable as shown from ONTAP, believe they are 3.8TB also. They are split into 2 data partitions and 1 root partition on each disk and onto 2 nodes, with 1x23 partitions RAID-DP and 2 partitions spares for data. I should create a new RG and add it into the existing aggr(not create a new aggr), but there are two different ways to create the RG:
a) Configure the new 18X3.8TB in the same way as current which is to partition them to 2 data P’s, and leave P3 unused, create a 18(Partitions) in size for the new RAID-DP.
b) Create 9 (disks) in size of the new RAID-DP

**2. Adding 24 x 15.3TB SSD onto 2 existing A700s controllers. **
Currently, there is 30x13TB usable as shown from ONTAP, believe they are 15.3TB as well. They are split into 2 data and 1 root partitions on each disk and onto 2 nodes, with 1x28 partitions RAID-DP and 2 partitions for spares. Similar to case 1, I can create a new RG and add it into the existing aggr. There are two ways to create the RG:
a) Configure the new 24X15.3TB in the same way as current which is to partition them to 2 data P’s, and leave P3 unused, create a 24(Partitions) in size for the new RAID-DP.
b) Create 12 (disks) in size of the new RAID-DP
What way would you choose or what options else can I have with above two respective cases?

sand condor
cedar raptor
stable grove
#

If you are ok with increasing the size of only one of the aggrs --> go for full disks.

orchid pier
#

@cedar raptor Would you say the same as @stable grove suggested to use partitions not whole disks to create new RG's and add them into respective existing aggr's?

cedar raptor
#

How many RG are in your current AGGR?

#

AFF are way more leanient / flexable / forgiving when it comes to aggr layout.

#

i would for sure add to existing though.

orchid pier
#

@cedar raptor 1 RG in the current existing aggr on both case 1 and case2.

cedar raptor
#

i'd go with a new RG honestly - something like this.

image.png
orchid pier
#

@cedar raptor Would you use the new RG into the existing AGGR or for creating a bran new AGGR?

cedar raptor
#

add to existing

orchid pier
#

Good. The only question then:
In Case 1, the size of the new RG would be 9 (whole disks), versus current 24 (partitions).
In Case 2, the size of the new RG would be 12(whole disks), versus current 28(partitions).
In terms of the size of RG's, they look quite different, would that be okay?

cedar raptor
#

yeah. with AFF it matters less.

#

granted. I WOULDN'T do one RG of 24 and a second of like 3 disks.

orchid pier
#

I am sorry, let me rephrase...

cedar raptor
#

you mean that some are partitioned and others arent?

#

also not an issue.

stable grove
#

With 15TB disks I think that hurts

cedar raptor
#

if it's only 1 existing RaidGroup though, you're only goint to get up to 28 in the RG.
so when adding 9 disks to 24

orchid pier
#

I am not sure I understand what you just said... @cedar raptor

cedar raptor
#

So looking at just #1.
you have 24 disks existing, that are partitioned in to D1 and D2 and Root
you have 2 aggrs, 1 per controller. with 23 disks + spare.
If you're adding 18. you're going to end up with a second RG anyway because the max disks is 28.

#

so between the HA pair. you're adding 9 per node.

orchid pier
#

correct, except one as a spare...

#

8 on one node within the HA, 9 on the other.

cedar raptor
#

if you do grow out the existing. to 28. you'd have 4/5 disks left over. which wouldn't make for a good sized RG.
you'd be better off with 2 RG -
the rg0 - existing 23 disks.
new rg1 - new 8 disks.

stable grove
#

I was talking about using the new disks, partitioning them and adding them as new RGs to the existing aggrs. Then you will only lose 2x disks for parity.

#

if you use full disks you will need 4x parity disks if you want to add them as new RGs to the existing aggrs.

cedar raptor
#

what ONTAP version?

orchid pier
#

9.9.1p3

cedar raptor
orchid pier
#

Okay, we should use partitions not whole disks for create RG's and add them into existing AGGR?

cedar raptor
#

yeah. I think i was misreading your orgional question, sorry about that.

orchid pier
#

NP, my wording...

cedar raptor
#

there's an example of the command -

orchid pier
#

So, basically,
In Case 1, we are going to partition 18 disks first, and create size 17 (partitions) RG, and add it into the existing AGGR on both nodes in HA. One disk (2 partitions) will be used for spares.

#

In case 2, size 23 (partitions) new RG and add it into the existing AGGR. One disk(2 partitions) for spares.

cedar raptor
#

that sounds like a plan.

orchid pier
#

The only question left: Can we do anything to utilize P3, that small size of 3rd partitions, or we would just have to leave them wasted?

cedar raptor
#

not really, they are spare root partitions.

orchid pier
#

Great!

#

Thank you @cedar raptor as always, and also @stable grove !

stable grove
cedar raptor
#

You'd be correct 🙂

sand condor
sand condor
orchid pier
#

This pace is a good resource to receive valuable advice

sand condor
#

If you could reinitialize the arrays after the SSD additions, then the root aggregate would be spread across all of the attached SSDs and result in larger data partition sizes.

#

That's not always the most convenient, but it is the more storage-efficient approach

tawny basalt
#

@rose thicket you can mix different size drives in same aggregate. The best practice is to act a minimum have the same size drive in each raid group.

Remember this, wafl is always looking for the best/longest stripe to send data to disks. If you have the first rg with 4t drives and then second rg with 10t drives, the performance of the aggregate will slow down once the stripes in the 4t drives are full. Generally speaking I try to keep similarly sized drives in the same aggregate. As long as the stripe is acknowledged, go ahead and add the rg with bigger (or smaller) drives

stable grove
#

Quick question: Is NVMe/TCP supported on MCC-IP?
I only find the info that beginning with ONTAP 9.12.1 NVMe/FC is supported on 4-node MCC-IP. But nothing about NVMe/TCP...

uneven roost
#

although it then says "Requires any network card".. so probably not 🙂

stable grove
#

but I guess the page is not yet updated for 9.12.1

stable grove
uneven roost
#

"MetroCluster configurations do not support front-end NVMe host connectivity." is pretty blanket

#

I'd say that's most accurate..

sick palm
elfin igloo
elfin igloo
sick palm
elfin igloo
solid cedar
#

Am I the support site team you want to send it to, Ross?

elfin igloo
solid cedar
#

I got it

spark pewter
#

Hello all looking for some expert opinions. I have an 8080 AFF that was initialized and running with 48x 750 SSDs. I've added 72 more 750s, and 24 3.5tb's. They are all sitting as spares

Running 9.7, everything was initialized with ADPv2, so all my 48 drives have shared partitions.

When i simulate adding X more drives to one of my two data aggregates, it only partitions the drive added to the nearly full raid group, and adds full drives to the next raidgroup(s).

Is this recommended? Or should I go through the effort of specifying a disk list, so that the additional drives are also partitioned upon addition? I find conflicting recommendations over the years, as at one point 48 was the max amount of drives supported for having a shared drive

dark sphinx
#

@cedar raptor this may be a question more up your alley 🙂

#

It kinda depends on your use case for the system too.

spark pewter
#

Fair enough. Right now I have two aggregates, tehy are about equally full and similar IO. HyperV and SQL

cedar raptor
#

I believe more than 48 ADP disks isn't supported. (check HWU).

spark pewter
#

it seemed 48 was at one point the limit. Now that's the limit for system initialization and can go higher if you add drives later, as I am

cedar raptor
#

@sand condor what's the offical on more than 48 disks ADPed?

#

But the filling out remaining RG and create a whole disk RG sounds normal/standard.

spark pewter
#

ya it is normal, because I have to follow that above guide to "trick" it into partitioning the next RG

#

and I try to stay away from tricks

#

but I didn't want to miss a significant performance boost by not doing my homework

#

My other question was, Should I be using the 3.5tb drives in a different RG or even Aggr than the 750's

sand condor
#

48x is the limit for ADP to be implemented during system initialization only

#

you can grow beyond that using various methods, though

spark pewter
#

Thanks @sand condor that's what I had read changed at some point. Trying to find where I read you can grow it

sand condor
#

that is, as long as you are careful in what you're doing

spark pewter
#

but SHOUDL you grow it. or just leave it as is. I only have 6GB IOM's so I'm thinking that's going to end up being a limit

sand condor
#

I know that that answer stinks, but it is true

#

some customers prefer expansion with whole drives while others do not

spark pewter
#

why would anyone prefer other than the max performance, which I assume is partitioned?

#

simpler recovery i suppose

sand condor
#

not having to work too hard while expanding their aggregate(s)

#

it's more predictable to some. Others like the idea of one drive shared between 2x controllers.

spark pewter
#

is it fair to think that adding 48 full drives, would be half the speed of 96 partitioned drives?

sand condor
#

well, an SSD is an SSD......the whole idea of speed of response is almost moot

#

with spindle drives, this was a far greater issue

dark sphinx
#

^^^ This.

#

Reallocate is not supported on AFF for this reason.

sand condor
#

Using exclusively spindle drives, I never recommend a RAID group of less than 11x

spark pewter
#

ok, sounds like i'm better with KISS

sand condor
#

I'm a big fan of keeping it simple

spark pewter
#

Should I be using the 3.5tb drives in a different RG or even Aggr than the 750's

sand condor
#

so you have 5x total shelves of 800GB and 1x shelf of 3.8TB

#

nice config

spark pewter
#

yep correct

#

on 3 strings

sand condor
#

so then I'm guessing: one stack of 2x shelves, one stack of 3x shelves, and the 3.8TB shelf in its own stack?

spark pewter
#

unfortunately no. We followed the install guide when we had just 2 shelves. so those each got their own stack. The 3 shelves added after are on a 3rd stack

sand condor
#

Ah, I see

spark pewter
#

I didn't want to take a maintenance window and rejigger everything

sand condor
#

since they're all spares, if you're brave enough, you can do this all online

cedar raptor
#

Also - i'd go with the 3.8 in their own aggr.

spark pewter
#

I am not brave

#

🙂

#

Thanks @cedar raptor that was my thought

sand condor
#

but in this case, I'd recommend the 3.8TB drives in a single aggregate and the newer added 800GB shelves added into new RAID groups on the existing aggregates

spark pewter
#

you guys are awesome thanks

#

Keep the RG size at the default, 24?

sand condor
#

yes

spark pewter
#

I guess one last question while I'm here.

Why did 9 of the 96 drives I attached, not auto assign?

sand condor
#

rev of ONTAP?

spark pewter
#

9.7p3

sand condor
#

make certain that you have the latest Disk Qualification Package and firmware installed

#

that is non-disruptive

spark pewter
#

hmm, perhaps those spares are not supported in current rev?

#

currently those 3.8tb drives are owned 50/50 by each node. I guess I need to move ownership all to a single node. if I want to create a single aggregate with all those drives?

sand condor
#

correct

solid cedar
median steeple
#

Sorry for posting the question right away when I first came in discord channel. I've read threads and internet documentation, but I don't understand, so I ask a question. thank you. Please understand that there is currently no understanding of ONTAP. main question is

"Can I put a custom SSD on the FAS2240-2? (like samsung evo ssd or something)"

on storagereview.com post says

"NetApp uses a proprietary sled and interposer card to support only qualified storage devices." so i'm a little worried about it.

my current situation is

  1. Currently using DS4246 as JBOD. Direct connect to Linux server with LSI 9200 raid card without a FAS controller
  2. Purchased FAS2240-2 to access from multiple servers + to use ONTAP + to expand additional storage.

thank you.

reguards

uneven roost
median steeple
uneven roost
#

ONTAP support for SSDs is based on which version it is running and which drives. Hwu.NetApp.com details this

#

If you’re using ontap, you may only use drives supplied originally from NetApp

median steeple
#

Thank you for your kind reply!

calm maple
#

Hi all,
did you know if i can move aggregates from one datastore to another in Ontap Select ? i can't find any documentation about that. it's because we are removing a storage class and we need to move everything to other storage in vmware .

carmine trout
#

We accidently mounted a NFS volume on Linux as "ext4" type instead of "nfs", and then the Linux system was not able to be brought up. I know we shouldn't do that, but conceptually how we can explain the cause like in-depth explanations?

dusk lava
#

this is a really helpful command to see how much I/O is being served by Flash Cache:

statistics start -sample-id test -object ext_cache_obj -counter inserts|hit_percent

does anyone know of an equivalent command that will show how much I/O is being served from RAM?

dark sphinx
dusk lava
dark sphinx
#

Have you started here?

dark sphinx
dark sphinx
carmine trout
#

Yes, I was just seeking an in-depth explanations on why, since it should be more like understanding to both NFS and EXT4 different file systems.

dark sphinx
dusk lava
dark sphinx
#

Actually us TSEs update KBs, not doc folks.

#

You can leave feedback as well and it should be updated within a day or so. If you have a case the TSE can update sooner.

#

We have several KCS folks in here who can do updates.

dusk lava
#

well whomever it is, please tell them thank you 🙂

dark sphinx
#

It was me this time. I can't update every time but I needed to update the answer because it needed the command.

orchid pier
#

We are running system version 7.0(3)I7(8) and RCF v1.8 on NX3132Q-V InterCluster switches, ONTAP 9.9.1P3. Also, replacing A300 with A400, can some expert here please advise if we will have to upgrade NX and RCF version before I can do heads swapping? I know I could find such infor from HWU but spent quite some time still not able to figure it out myself.

dark sphinx
#

I believe you want either KB or IMT...

orchid pier
#

Yeah, I do, but for now, I would like to know if I have to upgrade the switche's version first for NX3132q-v working with A400, before I do heads swapping.

uneven roost
minor acorn
#

We use FabricPool extensively (>10PB tiered) from multiple clusters. Is there way to tell how much user data is retrieved from StorageGRID? I don't know if there's a way to separate the routine traffic (snapshot expiration cleanup, etc.) from the user-requested blocks. It's so transparent it's almost too transparent 🙂

copper oyster
#

Hello, I am not sure if this is the right channel to ask, but I have a question about ACC. We recently set up an on prem install of ACC as part of a POC, and we are having an issue with 22.11. Even though we set telemetry to false, the telemetry pods were still deployed, and are failing to start. We are getting a 404 when trying to connect to ACC, and we are wondering if the two things are related. Is there any documentation regarding what endpoints we need to whitelist on our firewall for telemetry? I haven't been able to find anything.

valid basalt
stoic jolt
#

i think i know the answer to this but i figure i ask anyway.. we have a lot of file data sitting in a few ISCSI lun in a SVM.. We want to move this from and ISCSI lun into a native CIFS SVM with volumes.. there is no secret backdoor way to do this copy on the NetApp cluster is there ? if not I will pretty much have to use a host and robocopy the data over to the CIFS SVM

minor acorn
#

If it's a LUN, then the filer has no idea what the filesystem is - it is just a collection of blocks. You'll need a host. XCP might be faster than robocopy - you should test it. We use XCP extensively for NFS but haven't tested the Windows functionality yet.

patent flame
#

Ballpark how much data are we talking about?

copper oyster
valid basalt
#

It would be a neat trick if you could mount an iSCSI LUN to an SMB SVM as a volume and keep the move inside the NetApp, but no it is not possible.

On the XCP topic, it's wicked fast, if memory serves it'll do one thread per core available, so don't cheap out on core assignment.

stoic jolt
stoic jolt
valid basalt
tawny basalt
# orchid pier We are running system version 7.0(3)I7(8) and RCF v1.8 on NX3132Q-V InterCluster...

Not sure about the exact versioning.

Me personally, I’d follow the switch upgrade docs (which I have gotten properly corrected within the past year!). Although the docs say one thing: I do it this way:

  1. Upgrade code to 9.3(8) or the latest supported version on both switches, one at a time!
  2. If needed, update the rcf on both. I believe I saw you had 1.8. You can probably skip this step the latest code for these is 1.8A which has banner typos corrected if I recall, nothing significant.

There are a couple of bugs to look out for:
The RCF does not properly apply on 9.3 code unless you are at 9.3(8) or higher. Follow the directions. If needed, Wipe the switch, do basic setup, apply the rcf, reboot, apply the rcf (yes a second time!)
2. Image verification for NXOS upgrade: if it fails to apply the image, you need to disable Feature. I think the command is something like:
conf
No feature signature-verification

Then do the upgrade. This is a bug in the 7x code that I’ve hit too many times to count

spark pewter
#

Apologies in the advance for the very n00b question

I understand the fundamentals of SnapMirror. What I don't understand, is the proper way to sync any changes done on the source volume after the last snapshot.

In my environment, i have hourly asynchronous replication in snapmirror. If that sync happens at 1, data changes at 115, and at 130 I break the mirror and present the LUN up on the Destination filer, it will be missing that data that changed at 115.

What is the proper way to make sure that when I flip it over, I have every last change?

pliant charm
spark pewter
#

OK got it. That of course worked, I figured this was such a common thing that there must be some proper automated flow for it

pliant charm
#

Your schedule determines your RPO with asynchronous snapmirror; so having hourly means you are willing to lose up to ~1hr of data is a disaster scenario. If that's not acceptable you may need to reconsider your RPO targets.

#

This KB might be helpful with respect to DR testing and SnapMirror
https://kb.netapp.com/Advice_and_Troubleshooting/Data_Protection_and_Security/SnapMirror/Guidelines_for_SnapMirror_Disaster_Recovery_(DR)_testing_and_reverse_SnapMirror_in_ONTAP

NetApp Knowledge Base
Guidelines for SnapMirror Disaster Recovery (DR) testing and revers...

This KB is a walkthrough for a basic SnapMirror Disaster Recovery (DR) Test and reversal in clustered Data ONTAP and ONTAP 9. The procedure of this KB can be also used for SVM-DR. This KB can also …

spark pewter
#

Awesome, thanks. Does everything here have a UI equivalent.?

pliant charm
#

The GUI will bundle some steps together, but yes, you should be able to do this via the GUI also.

spark pewter
#

Is there an advantage to doing an SVM replication vs individual volume? For example, does it copy the iSCSI initiator target configs over automatically?

cedar raptor
#

Nothing iscsi is moved over with SVM Snapmirror.

pliant charm
spark pewter
#

fantastic, thanks. I only use iSCSI so I think i'll keep it simple and replicate volume by volume

orchid pier
tawny basalt
#

The code upgrade is super easy an no issues for downtime if the directions are properly followed.

I think the 3132 currently tops out at 9.3(10).

Minor complications arise when you need to do the rcf update as that is a full wipe and restart on the switches (one at a time)

tawny basalt
#

So for the NX3132Q-V, the REC versions are 9.3(9)/9.3(10) and 9.3(10), 9.3(9), 9.3(8), 7.0(3)I7(10), 7.0(3)I7(9) are all listed as supported.

#

the difference between 1.8a and 1.8 versions is that additional Installation details are added to the 1.8a Banner

minor acorn
orchid pier
#

I have a tiering-policy=snapshot-only and with tiering-minimum-cooling-days=5 on a lot of volumes. Is there any commands or any other ways to display how much snapshots data are left in the performance tier, and how much snapshots data being tiered to a object storage?

minor acorn
orchid pier
# minor acorn I think you're looking for volume show-footprint

Not really, because "volume show-footprint" only shows you how much data or snapshots data in the capacity tier, but cannot tell you how much snapshots data in the performance tier. The amount shown in the performance tier contains both active data and snapshots data.

minor acorn
orchid pier
# minor acorn with a snapshot policy of snapshot-only, the math appears straight-forward. Yo...

By using "df -h volume" I can get the total size of the snapshots, then deduct the amount in the capacity tier from "volume show-footprint". True. these are steps that I can get the amount of snapshots in the performance tier for a volume. But, I have a lot of such volumes. To get such amount on all these volumes, it requires a script if using same steps. But I am not so good in scripting. So, I am looking for a command or a few straightforward commands?

echo terrace
#

Hello All, question on Windows ACL on File System, the SYSTEM group, does this have to be applied? Does the underlying cDOT OS rely on it to gain insights into the data on the volume?

elfin igloo
stoic jolt
#

whats the hold up with 9.12.1 GA release 9.12.1.RC1P1 looks funny

elfin igloo
# stoic jolt whats the hold up with 9.12.1 GA release 9.12.1.RC1P1 looks funny

Hey Greg, check out the most recent ONAIR episode where we have a PM talk about ONTAP 9.12. Briefly touches on some contributing factors to why the GA isn't out yet. https://www.youtube.com/watch?v=sT3h5wb6jVA&t=153s

YouTube
NetApp
ONTAP 9.12 Updates with Keith Aasen | NetApp ONAIR

Keith Aasen, Snr Prod Mgr ONTAP, joins the show to go over all the updates with ONTAP 9.12, and any new developments since it was announced at Insight 2022 back in November! Join us LIVE to get your questions answered LIVE on the show!

0:00 Green Room
9:50 Welcome & Introductions
17:18 ONTAP 9.12.1
1:07:52 Closing Thoughts

👏 Thanks for Watch...

▶ Play video
minor acorn
uneven roost
#

We have done RC2/RC3 in the past, I can only assume there’s a metric of what has been changed which means it’s a P release of the RC

#

Those metrics themselves are subject to change too 😉 I remember running 7.2.5.1 on a system many moons ago

minor acorn
uneven roost
#

it was a pretty popular one

dusk lava
#

are they going to skip ONTAP 10 and go straight to 11*? I seem to recall that the old GX product was known as ONTAP 10

  • no Spinal Tap references, please
cedar raptor
#

But yeah. that was what I heard years back when i made some passing comment about "ONTAP X"

sand condor
#

future numbering is TBD

#

The situation is similar to why Microsoft never offered a Windows 9. The logic was that the '9' might, due to typos, intermingle with the old Windows 9x (95/98) code lines and reek havoc during compilation.

viscid robin
#

Hi,
Does anyone here know if Lenovo also manipulate with the ONTAP OS in the DM Series?

I have a DM system running 9.11.1P3 where I have configured a SVM with LDAP and Kerberos. The SVM is running both NFS v3 and NFS v4 shares. When a non-root user try's to access a NFS v3 share, they get a NFS Stale File Handle / Permissions denied. (in the event log, I can see "try to autheticate user with ID..." error)

With the exact same setup on a native NetApp ONTAP 9.11.1P3, we don't get any permissions denied, and everything works as expected.

We have tested from the same client against both systems.

dark sphinx
#

It may be a setting that got missed.

cedar raptor
#

yeah as far as i know, it's just a rebrand. nothing changed like back in the N Series days.

umbral stratus
viscid robin
dark sphinx
#

Access denied can be hard. Without tracing or doing sectrace, going through logs, etc., it's hard to say.

patent flame
orchid pier
#

Sure, moved

stoic jolt
#

moved

patent flame
#

Hey Greg, drop this into the #1062049169520476220 forum channel so we can keep things organized. These are discussion channels intended to just talk about industry trends and such.

sand flint
#

Hi - I'm looking at a Netapp FAS500f as a replacement for a Pure Storage C60. Am I right to think that the FAS500f is able to run dedupe/compression inline (like the AFFs) or will it do data reduction post-process?

dark sphinx
sand flint
patent flame
#

It's an AFF with big boi QLC drives. Here are the details/breakdown... https://youtu.be/qh92RffdL-k

YouTube
DatacenterDude
Cabling up my NetApp storage system

After last week's unboxing video, there is a shocking revelation as we go through just how simple it is to cable up a NetApp storage array! Make sure you tune in next week as we walk through all of the firmware and remote management software updating procedures!

👏 Thanks for Watching and 🔴 Subscribing! New videos coming each week!
-----------...

▶ Play video
sand flint
analog mist
sand flint
uneven roost
#

(I have no information suggesting this is going to change, but I can only advise for this point in time, as licensing does change periodically)

sand flint
carmine oracle
#

Hello. I'm new to NetApp, any idea if this card "NetApp 111-00341+G0" can be installed in Ubuntu?

drowsy dust
#

Hey mongo,
You're looking at a homelab using ds2246's/ds4246's right?

If so, yes you can absolutely get that thing to recognize with Ubuntu from what I've read.

elfin igloo
carmine oracle
#

Thanks. I was actually looking for a solution for connecting Hitachi JBOD that comes with a QSFP port rather SAS port. So figured, there is a cable that has one end QSFP and other SAS. All good

low dune
#

I am not sure I post in correct room or not, if incorrect please I say sorry. I my role is server admin and I use NetApp on my server and I like to understand more about NetApp. What is WAFL? I surf in Internet and some said it is filesystem but some said it is not filesystem. For my opinion I like to consider it into 2 layers, lower layer is hierarchical filesystem like Unix filesystem and upper layer is NetApp filesystem. The upper layer is NetApp intellectual property to place storage data and their metadata and NVRAM and caching to improve performance better than native Unix filesystem. I am not sure, am I right or not? Can anyone give me more clarification or hint me about any document?

cedar raptor
#

It’s a file system. “Write anywhere file layout”. Ontap is the operating system. That could fit into your 2 layer thought.

#

It’s BSD based. And both WAFL and ontap are proprietary

low dune
#

@cedar raptor thanks

pale cradle
#

hey guys, quick question. we are planning to expand our fas2620 with another ds212c shelf filled with 12x 10TB 7.2k disks. to met raid tec requirements they need to be assigned to one controller instead of splitting 6 disks to ctrlA and 6disks to ctrlB. any flaws of having all disks in shelf assigned to one controller? or it will just work as intended

cedar raptor
#

That's fine. Verify all the cables are correct though. The shelf will be cabled to both controllers. so during a takeover it's partner will gain ownership of the drives.

pale cradle
sand condor
#

No issues with this configuration

orchid pier
#

I am not sure if this is a right place to post my question about FlexPod.
We heard an incompatibility issue from VMware team which is not met with Interoperability Matrix between vmware 7 version and NetApp.(We are running 9.9.1p3). It is due to outdated UCS firmware. I know these information are not so detailed. The issue could cause unpredictable result when failover/failback. I have following questions:

Could outdated UCS firmware cause issues with storage failover/failback? Because I thought conceptually failover/failback should be controlled by NetApp. UCS is more on the back side.

Are there any documents out there to describe how UCS or UCS fabric interconnect etc. work with NetApp cluster in a lecturing way? I have some, but still could not understand how they work together.

stable grove
#

Did your VMware team tell you anything more? A bug ID or anything? Maybe something about using NFSv4 for your datastores?

#

What protocol are using for your datastores? And how is the storage connected to the FIs? Directly or are there Nexus switches involved?

hybrid ridge
#

Find out about some of the great new SnapMirror features that just shipped with NetApp ONTAP 9.12.1 in my new Tech ONTAP blog:

https://lnkd.in/gFrkc__n

drowsy dust
#

Great write up!

patent flame
stiff maple
#

Who here does nfs with a unix ldap and has accomplished a more than 16 group capability?

vocal monolith
stiff maple
# vocal monolith https://docs.netapp.com/us-en/ontap/nfs-admin/configure-number-group-ids-allowed...

Yeah but the article leaves it unclear. You enable this and then any user or group that is only on the client and not in your name source is then denied. I made a mistake and found out the hard way.

https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/How_does_AUTH_SYS_Extended_Groups_change_NFS_authentication

NetApp Knowledge Base
AUTH_SYS Extended Groups changes for NFS authentication for ONTAP 9
amber compass
#

Hello everyone, I have the Storage DS2246 but I can factory reset everything and I saw need the NetApp Sustem Manager but I cannot find anymore.

Anyone have this software to provide, please?

cedar raptor
#

A DS2246 is a JBOD (disk shelf)

#

There’s nothing system manager can do.

#

People will buy them off eBay and hook them to Linux boxes. I’d recommend checking out /r/homelab

amber compass
#

because I can see 2 ports opened, 22 and 50000 and also I can connect via serial.

cedar raptor
#

Can you post a pic

#

Of the rear of the device

amber compass
cedar raptor
#

That looks more like a FAS2552

#

Which uses the 2246 as a chassis.

#

Software downloads and licenses require a valid support contract as well.

amber compass
cedar raptor
#

Those end caps are inter changeable.

#

Someone probably just had one laying around.

amber compass
#

can I ask you if you have some software to try to connect an reset?

cedar raptor
#

Serial cable / terminal connection.

#

And ssh.

#

Also to note if you wipe it make sure you have the licenses.

amber compass
#

I'm connected via serial, but ssh the default user/password dont work

i tried:

root/netapp!123
admin/netapp!123
root/root
admin/root
root/admin

#

Im stuck on this... I can see all boot in progress but stop in this

IMG_1068.jpg
cedar raptor
#

if it keeps boot looping in to that chances are you don't have an OS on the box.

#

Can you create a post here -

image.png
amber compass
#

ok. I will do right now

stiff maple
sand condor
vocal monolith
sand condor
sand condor
amber compass
#

I just have this options

Screenshot_2023-02-06_at_18.50.46.png
#

some one have the system manager to install here and try? I cannot download from anywhere

cedar raptor
#

It’s ran from Loader>

#

If it keeps looping. There’s no OS

kind prairie
#

You’re in the boot loader, not the OS. If you can’t reach the boot menu you’ll have to NetBoot.

amber compass
#

do you believe we can load the os from this screen?
or with system manager will work?

kind prairie
#

System manager has been an onbox part of the OS since version 9. If you wanted to run 8.something it still would not do anything for you until the OS was fully operational

#

From this screen:
boot_ontap

#

When prompted press control-c repeatedly until the boot menu appear.

#

If it boot loops before prompting then see kb for instructions on netboot

#

You’ll need a valid support account to get this system operational

amber compass
#

huuummm ok, I'm trying to contact the support, but no success. I already have created my account but without access to downloads.

kind prairie
#

Downloads access will need the system to have current maintenance

#

But at least you be able to read the kb now.

amber compass
#

OK Sean, I will try to find some information... thank you

kind prairie
#

Good luck!

cedar raptor
solid cedar
wheat shale
#

what is the actual difference between IOM12 and IOM12B, I understand these can be mixed in the same stack, but didnt know what the B signified?

#

just newer model?

pearl coral
#

B is for Betta 🙂 ¯_(ツ)_/¯

vivid gull
#

IOM12B is just newer model due to supply chain issues causing a redesign. Can be mixed in the same stack as IOM12 (now referred to as IOM12A in some spots).

pearl coral
#

next version will be MB = Mo Betta 🙂

uneven roost
#

It also has a blue stripe on it. You can't mix IOM12A and B in the same shelf, but you can in the same stack as others have said

wheat shale
#

Yep saw that in the documentation, thanks

stable grove
#

Make sure you are on an ONTAP version which supports IOM12 though: "Minimum ONTAP support for IOM12B modules: 9.7+, 9.6P7+, 9.5P13+"

#

I would also recommend to update the shelf firmware of the shelves before adding them to the stack.

sand condor
#

no mixing IOM12 and IOM12B in the same shelf; same stack is OK though

waxen frost
#

Hi all, does someone has any experiance with NVMe/TCP and VMware? I dont have any idea from where I get the transport port number, and the digest parameter for the controller

sand condor
#

Section 3.1.2 of the RFC calls out these specific details regarding TCP ports.

wheat shale
#

what is the actual maximum number of partitioned NVMe disks for an A900 system, I know its 48 at system initialisation but need to know post initialisation? struggling to find on HWU

sand condor
#

During initialization, the max is 48x which is documented in HWU. That is because the root aggregates (per node) can only be spread across a maximum of 24x drives. You can grow well beyond that, though without limitations.

#

and that is true for all systems

cedar raptor
#

I'd think there is a point though were it doesn't make practical sense because of the unused space that ends up in the root partition.

wheat shale
cedar raptor
#

and growing aggrs like this is fine in the AFF world.

image.png
sand condor
#

Exactly

#

Growing beyond 48x partitions is doable, but depending on the size of the existing root partitions, you're essentially throwing away usable disk space by doing so.

little geode
#

Does Santricity do bit rot protection at the block level?

solid cedar
#

@little geode - This has already been asked in #1062049169520476220. Please allow some time for others to see it and respond.

little geode
#

Sorry 🙂 @solid cedar

#

Its just such a simple question

#

Which I think most people here who use Netapp devices probably know the answer to.

#

I just dont see why it wouldnt.

#

It's such an important part of storage.

#

Otherwise their Santricity boxes are just a collection of disks.

drowsy dust
little geode
#

I just wish they had bitrot protection

#

Im considering going with linux with dm-integrity instead

uneven roost
drowsy dust
#

Yeah they're not JBODs

little geode
#

@uneven roost Hmm, it says media errors. Maybe that means physicsl drive errors and not bitrot?

uneven roost
#

What would your definition of bit rot be, if not data on disk changing from when you wrote it?

little geode
#

I dont know tbh. Ive just read enough to think that maybe there is some difference. I have also had a Lenovo employee tell me that Santricity doesnt do bit rot detection.

uneven roost
#

🤔

#

Well, in as much as Lenovo are valued partners, in this instance I think we’d suggest that person do some more investigation of the technology they have licensed

#

Bit rot is when the data is not “durable”, ie, not what you wrote to disk when it is read back.

#

And RAID and DDP media scrubs make sure that doesn’t occur

#

Despite the name, media scrub isn’t just looking for errors accessing an LBA - it does check the data against the redundancy information by default. These checks can be turned off, but by default they are on

little geode
#

Since thats the case, what makes Ontap more valuable than Santricity? @uneven roost

#

I want to contact Lenovo about getting a quote on one of these: https://www.lenovo.com/us/en/p/servers-storage/storage/storage-area-network/de-hybrid-flash-array/thinksystem-de6400h-hybrid-flash-array/len22ts0014

Lenovo ThinkSystem DE6400H Hybrid Flash Array | SAN Storage

A ThinkSystem DE Series Hybrid Flash Array is the perfect combination of performance and economics to handle your most demanding enterprise workloads, such, databases, data analytics, technical computing, and AI applications.

#

Can anyone give me their guess on how much this would cost? Would it be under $20k you think?

uneven roost
#

ONTAP does block and file, and multitenancy

#

E-Series/Santricity is block only

little geode
#

Ok. Thank you. Santricity seems like what I need. If I could get NVME Santricity for under $20k, that would be perfect. But I may be dreaming.

uneven roost
#

I can't talk pricing, sorry

#

As for NVMe.. some E-Series do NVMe drives, some just do SAS SSDs. I'm not super familiar with Lenovo's lineup, so I couldn't say which do/don't, if any

little geode
#

The data assurance feature provides controller-to-drive data integrity protection through the SCSI direct-
access block device protection information model. This model protects user data by appending protection
information to each block of user data. The protection model is sometimes referred to as data integrity
field protection or T10 PI. This model makes sure that an I/O has completed without any bad blocks
written to or read from disk. It protects against displacement errors, data corruption resulting from
hardware or software errors, bit flips, and silent drive errors, such as when the drive delivers the wrong
data on a read request or writes to the wrong location.

#

bit flips = bit rot

#

What are the benefits of having an Ontap box for NAS services compared to having a Santricity box and using a secondary linux machine to serve up NAS services?

#

(With the linux box serving the NAS from the iscsi of the Santricity san)

patent flame
#

How much time you got?

#

🙂

little geode
#

A kit

#

a lot

#

lol, hit the wrong keys shifted to the left, it spelled a word. 🙂

#

I am very curious about it all. I am new to enterprise storage. Right now I am on Truenas

#

I hate not being able to expand my pools.

#

@patent flame hit me with it

little geode
#

Would having an all SSD san (2u) bottle neck a NVMe over RoCE connection?

#

It would be great if we could use a local NVME as cache for the SAN/iscsi

lone island
#

Hi. We are using e-series 2800. Can someone help me understand what Drive Channel is ? Per documentation it is a path between a drive and controler. But is it logical or physical ? Each drive have separate channel ?

patent flame
uneven roost
#

I think we might have to have this in here - we don't have a santricity forum yet 😉

uneven roost
uneven roost
#

each drive has it's own seperate channel.. but there may be portions of it shared, like SAS cables. SAS cables have four lanes, each drive gets put on one of the four lanes at drive initialisation time

lone island
#

thanks for explanation 🙂

sand condor
little geode
#

I dont understand how DAS machines can be connected to multiple servers at one time.

#

Wouldnt they conflict with each other?

sand condor
cedar raptor
#

VMware/esxi is good with it. I've done several flexpod mini's that were 2x UCS pizza boxes SAS connected to an eseries.

little geode
#

What is the price range of NVME santricity boxes?

elfin igloo
#

Apologies for that.

uneven roost
#

the logical units, or LUNs, mappings are kept track of by the DAS enclosure

little geode
#

Ok heres a question you guys can answer. 🙂 Can an all SSD san saturate an NVME over fiber or RDMA connection? What about a fiber channel connection?

#

@uneven roost @elfin igloo

uneven roost
#

honestly it depends. FC's at 32Gbit now right? that would be a bit of a challenge

#

because it's not just drive to FC, it has to calculate RAID for writes, deal with PCIe bus limitations etc

little geode
#

What about an all nvme san? Does the fiber channel not bottle neck the bandwith?

#

I imagine the big selling point of nvme sans is latency? @uneven roost

uneven roost
#

latency and throughput. SAS and SATA are limited to 12 and 6Gbit links, while NVMe is PCIe speeds (or 100Gbit shelves)

little geode
#

yes but isnt fiber channel the bottle neck?

#

Actually, nm. I see that FC can go to 100gb

uneven roost
#

it can be, haven't seen much 64Gbit+ FC though

#

and most of the time those speeds are for trunks, not edge connectivity

#

and FC is.. not very common anymore

#

I have done about 100 netapp installs.. probably 10 had FC

little geode
#

What is common then?

#

FC = FiberChannel

#

Is FCoE the more common approach?

#

@uneven roost

uneven roost
#

iSCSI honestly

#

for SANs, and NFS/SMB for NAS

#

zero of the installs I've done have used FCoE

little geode
#

I have read that fiber channel has better data protection.

uneven roost
#

so you're doing a lot of reading 🙂 what problem are you actually trying to solve?

little geode
#

data protection for a small business

#

choosing what SAN to buy

elfin igloo
#

Correct me if I am a bit wrong on the details here @uneven roost but there is really two different conversations here,

  1. Decision around using what protocol for hosts to talk to storage (i.e FC, FCoE or ISCSI)
  2. What data protection capabilities the storage has to protect data once its written to storage
#

For the first part as Alex has said, ISCSI tends to be the most common these days as it has more flexibility in terms of scale, using pre-existing ethernet networks and a bunch of other benefits in terms of future proofing for later growth , tech refreshes etc.

#

For the second part, the data protection features / capabilities will vary between E-Series and our other main solution FAS/AFF.

#

If data protection is critical then capabilities like snapshots, snapmirror, backups etc are all factors to consider. Our ONTAP based products like FAS/AFF might be a better fit. You may want to consider reaching out to a local NetApp partner or our sales team as they can obviously walk you through the the benefits that would suit your busines needs.

wheat shale
#

In an A900 with 3 fully populated SSD shelves and one shelf with 18 disks, is there any particular location for the 18 disks on the last shelf, or can they populate slots 0-17?

#

or should this follow the outside in rule, so populate 0-8 and then 15-23?

elfin igloo
#

Hey @wheat shale you will likely get a better response by dropping your AFF question in our dedicated AFF channel #1062049107096633454

solid cedar
#

Thanks 🙂

cedar raptor
little geode
#

I am still in the research phase but Im leaning toward a new Lenovo server with Starwinds San & Nas. Can anyone sell me on the benefits of a netapp/lenovo san compared to this setup?

cedar raptor
#

had to look up Starwind. but intially i'd say -
AFF (A or C) or FAS running ONTAP -
AFF looks simplier to deploy, manage and update.
You get all the ONTAP features - Storage Efficies , replicaion (Snapmirror), Cloud intergration. VMware plugs. etc.

eSeries - doens't have all the bells and whistle as ONTAP, but they are great boxes.

pearl coral
# little geode I am still in the research phase but Im leaning toward a new Lenovo server with ...

the protocol...in this case FC that you're referring to will offer no data protection. from what i briefly saw you posted you'd want to look at a small ONTAP filer and use something like NFS or iSCSI which will make things a ton easier going forward then if theres not enough budget for another small ONTAP filer then just replicate that data with native ONTAP system tools to our cloud services in BlueXP

little geode
#

What CPU does the eseries use?

pearl coral
# little geode What CPU does the eseries use?

E-Series and other ONTAP products utilize all kinds of processors. you can not install anything on top of these systems and shouldn't be a determining factor in picking one over another

#

its not a dinky arm cpu like in a synology etc if thats what your asking

little geode
#

Do you have to use Netapp's pre-selected hard drives in their systems or can you use other enterprise hard drives?

cedar raptor
#

NetApp supported harddrives. There's specific firmware and formating for netapp drives.

little geode
#

Ok. And what is the average wattage of an all flash san?

cedar raptor
#

depends.

#

on the model.

little geode
#

Can you give me a range

#

Under 200watts?

cedar raptor
#

Do you have an idea of Capacity and throughput and IOPs you're looking at

little geode
#

Not sure.

#

But im also looking at zfs

#

So im trying to compare my options

cedar raptor
#

So one of our smaller system - C190 - with 24x 960GB drives -
2RU -
Typical Amps - 3.24
Typical Watts - 317
Typical BTU hr - 1082

That's also 2 controllers in an HA pair.

dark sphinx
#

3 Amps? That's actually really good.

little geode
#

300 watts is pretty good for that many drives.

#

tbh, the main reason I am even looking at enterprise storage is because I dont trust zfs that much

cedar raptor
#

What are you looking for in a storage system?

little geode
#

Reliability. Wont destroy my data because of a bug.

cedar raptor
#

That's a given :).

Do you need DR? Or have a DR plan?
What's the workload?

little geode
#

DR?

cedar raptor
#

Disaster Recovery

#

e.g. your primary site burns to the ground. What's next?

little geode
#

I have online backups for that

cedar raptor
#

"online"? like backblaze or something to the cloud?

#

what's your current setup?

little geode
stoic jolt
#

where can i post an eseries technical question ? i can only see ONTAP and StorageGRID support areas ?

drowsy dust
elfin igloo
#

Some folks still do choose to stay with FC due to the pre-existing networking infrascture already being in place for it. Does cost to over-haul if you're already knee deep in FC land.

elfin igloo
drowsy dust