Why can't I update `setQueryData` mutably?

[opal charm]

            queryClient.setQueryData<{ pages: Message[][] }>(
              queryKeys.channelMessages(channelId),
              (ctx) => {
                const shouldAdd =
                  (ctx?.pages.length === 1 &&
                    (ctx.pages[0]?.length ?? 0) < 100) ||
                  ((ctx?.pages.length ?? 0) > 1 &&
                    (ctx?.pages[0]?.length ?? 0) < 50);

                shouldAdd
                  ? data.forEach((msg) => ctx?.pages[0]?.unshift(msg))
                  : data.forEach((message) => ctx?.pages.unshift([message]));

                return ctx;
              }
            );

Ik the docs say you can't but why not? if i return the updated object after mutating then it should work no?

im trying to avoid cloning these arrays cause they can get very large