Fabricpool with SGRID - Archive Node

[sour umbra]

Is fabric pool ever going to support an archive node in the grid to glacier instant access? Have a customer with multi PB of data, Ontap and Grid needing this badly

[solar blade]

I thought S3 archive tiering is deprecated in favor of Cloud Storage Pools... 🤔

[visual quail]

The Archive Node has been deprecated for a few releases now, and I believe will be completely unsupported in the next release of StorageGRID. SG 11.8 introduced the capability to decommission an Archive Node.

[visual quail]

Cloud Storage Pools supports Glacier, however is not supported with FabricPool. The reason is that retrievals are slow from Glacier, and ONTAP expects an immediate response when recalling/accessing tiered objects.

As per this, https://docs.netapp.com/us-en/storagegrid-118/ilm/considerations-for-cloud-storage-pools.html, "Using Cloud Storage Pools with FabricPool is not supported because of the added latency to retrieve an object from the Cloud Storage Pool target."

I recommended lodging a feature request to get visibility into the demand for this.

[fickle gull]

The idea that there should be PB of data behind NAS controllers is basically a refusal to evolve, imho. If you don't have internal development teams moving data to S3 natively, things are just going to get very messy. If you have to rely on commercial software offerings, things there are terribly bleak as far as native S3 support, incredibly enough.

[sour umbra]

Well it is what it is... but keep in mind I an talking about glacier instant access... Same sla as anything S3... So if S3 were to be supported in CSP with fabripool; then glacier instant access would be as well... As far as your comment @fickle gull; When you have a global drug development company with tons of MNA; these scenarios are always going to be present... They are doing data transformation but nothing is going to solve for decades worth of data developing drugs under many different companies...

[visual quail]

We do move data to S3 natively ... its called FabricPool. I have several customers with 10PB+ each tiered to S3 via FabricPool. Some are using StorageGRID, others are using hyperscaler offerings. They all work. But they all get an instant response from the S3 servivce. All we're saying is ONTAP expects instant response and recall (streaming) of object data to commence immediately. Traditional Glacier can't do that. It can take extended periods to recall objects. Can your NAS or Block front-end client workloads wait days for a single file to open, and how long should ONTAP sit and wait for the object tier?

As I said. Feature Requests. That's the way to get attention and priority.

[solar blade]

there's also the option of simply using Cloud Sync or rclone to get rid of all the old data and move it (as objects) into S3

[fickle gull]

Yeah, no. Having applications that store data directly to S3 is infinitely different than tiering behind NAS. NAS requires huge amounts of administration with file system rights, directory structures and migration actvities. S3 has huge advantages as far as dynamic granular access rights and basically 0 actual directory structure, not to mention the ease of self-service within tenants. I get why FabricPool is necessary in reality. I use it myself, but it is also just keeping everyone using technology that is ill-suited to large amounts of data.

[fickle gull]

"nothing" would seem to be a stretch. There are obviously many technological solutions, but data management is not usually a high priority task until it becomes a problem.

[sour umbra]

nothing immediately... and also this company has no cohesion because it is a collection of fragments

[fickle gull]

yeah, sounds familiar. As long as everything is still profitable and it's not a dumpster fire, nothing changes

[rustic mauve]

I know newer PACS applications are supporting S3 now, but applications that are older haven't introduced S3 support yet.

FabricPool is meant for infrequently accessed data, so if you need more frequently accessed data with lower response times then FabricPool may not be the right solution vs. a structured NAS solution that is say HDD or C series only.

You can use FP as a second tier if you're ok with the added latency.

[sour umbra]

this isn't PACS... It is plain old unstructured data... If FP can use Glacier Instant Retrieval (it is absolutely supported https://www.netapp.com/media/17239-tr-4598.pdf) then why can I just setup up this tier in the grid backending the current fabricpool with a cloud storage pool and call it a day... everyone would win... Specifically the customer not having to fork out the $ for by the drip FP licensing... Probably why NetApp says no

[solar blade]

dunno what to tell you mate. Chris already linked you to a document explaining the reasoning, and a way for you to try and get it supported.
If you want to do it anyways feel free to do so, it will probably work in most cases... worst case you will have to pull all objects back into your grid 🤷‍♂️
And FP doesn't need a license when tiering to StorageGRID

[drifting matrix]

i mean the biggest issue is in my Opinion, that you have to test how the GRID is responding to the Client (so Ontap in this Case) if the Cloud Storage Pool is not able to retrieve the Object you tiered out... if the GRID says something like "object not Found" you'll maybe get a corrupted WAFL :/

and if this is the behavior, it makes sense to not support Tiering of a tiering, because the Client will not get noticed of the unavailable of the Cold Archive tier.

[rocky dove]

but you do know that many companies rely on dated apps and services which only know about NAF (SMB/NFS) and their modernization will take years ... for very little business value....