IBM MQ "remote I/O" errors during snapmirror quiesce

[round herald]

It's observed that IBM MQ application observes "remote I/O errors" and faces outage while reading/writing to mounted NFS (v4.1/v4.2) storage whenever, snapmirror quiesce is performed.
The NFS exported volumes are snapmirrored via Snapmirror Sync (not strict Sync).

During these issues EMS messages record "sms.status.out.of.sync:error" and at the same time, the application is not able to perform any read/write requests on the Snapmirror Source volumes.

I understand if Snapmirror Sync relationship is not strict sync then the read/write operations on the file system will not encounter any issues”, however, we have now seen this multiple times.

Has anyone faced such issues before and have a possible solution?

[compact smelt]

The common snapshot may be inducing latency. I suspect when a quiesce happens it does a common snapshot update.

In my experience, IBM MQ is really latency sensitive.

[snow moth]

Um, have you tried the stateless nfs version 3?

With v4 being stateful, the client is likely barfing when ONTAP has any hiccup

[round herald]

We enquired about using NFS v3, however, IBM's requirement for "multi-instance queue managers" require the use of NFS v4

https://www.ibm.com/docs/en/ibm-mq/9.2?topic=multiplatforms-requirements-shared-file-systems

https://www.ibm.com/support/pages/testing-statement-ibm-mq-multi-instance-queue-manager-file-systems

[snow moth]

You did see this footnote?

It has been found that server delegation must be disabled to prevent I/O errors under certain conditions with NFSv4. Mount the filesystem with 'nfsv4delegation=NONE' to disable server delegation.

[round herald]

We have both read/write delegations "disabled" on the NFS SVM. Will setting the mount option 'nfsv4delegation=NONE' still make any difference?

[compact smelt]

That's an IBM question.

If it happens again, getting packet traces would likely help.