GuildMember.fetch loading 15s-30min but fetching manually using REST works instantly

[heady silo]

Hey, I am currently having the issue that on very busy servers (100-800k members), my Discord bot is having issues fetching members properly.
Executing js member.fetch() sometimes takes ages while js fetch(`${apiURL}/guilds/${guildId}/members/${memberId}`) resolves instantly with a rate limit bucket of 4 of 5 left. Is this an issue with discord.js' internal rate limit handling or am I missing something? Any help is appreciated!

[slow spire]

• What's your exact discord.js npm list discord.js and node node -v version?
• Post the full error stack trace, not just the top part!
• Show your code!
• Explain what exactly your issue is.
• Not a discord.js issue? Check out #useful-servers.

[hollow oak]

Are you fetching 1 member with id or all members?

[hard storm]

This fetching all members through the Gateway. Members are sent with chunks by 1000 in each. That's why it can take a long time on large servers.

await <Guild>.members.fetch();

This fetching only 1 member through the REST API with the route that you specified in your question.

await <Guild>.members.fetch('id');

[tepid juniper]

Did you test the direct API fetching with the same load your bot has? Or did you just fetch one single member and concluded it works perfectly while your bot will fetch several members in a short timeframe? Because it’s most definitely ratelimit related

[heady silo]

I am fetching a single member, not multiple

[tepid juniper]

But only one during the whole runtime of your bot? Or one member per command/interaction?

Or where/when do you fetch?

[heady silo]

A server reported that they are having issues with banning, warning, muting users but also with the userinfo command. So I checked if my bot got rate limited fetching users using client.users.fetch(userId) but that resolved instantly. So I did guild.members.fetch({ user: userId, force: true }) and it did not resolve after a few seconds. While this operation was still running, I could easily fetch the member using node-fetch("/api/.../members/userId"). After ~8 minutes, the guild.members.fetch thingy finally resolved. This fixes itself after a while but mostly happens when the server is very active

I also debugged the guild.members.fetch function in the discord.js code and it was stuck at

const data = await this.client.api.guilds(this.guild.id).members(user).get();

[hollow oak]

because fetch by default caches fetched data, and next time when you are trying to fetch someone it takes from cache. unless its force fetch. getting from cache won't take a lot time

[heady silo]

I did use force: true and the node-fetch package does not cache data to my knowledge.

[hollow oak]

node-fetch doesn't, discord.js does

[tepid juniper]

So it had requests queued that it was going through one by one. Your direct fetch call circumvented that and basically cut line saying I‘m important, fetch me now! making your other requests be delayed even further

[heady silo]

Yeah, but discord.js took 8 minutes while node-fetch resolves instantly

[heady silo]

But why are the rate limit headers are then saying that I have 4 of 5 requests left when using node-fetch? Shouldn't it be anything less than 4 since discord.js is fetching other members right now?

[tepid juniper]

Because your node-fetch probably was the first of a pack of 5 then

[heady silo]

Alright. Any way to see how many members are currently in the fetch queue?

[hollow oak]

Its when you are force-fetching it, right? I just wouldn't recommend make EACH fetch forced.

[hard storm]

btw do you use prefix commands or slash commands?

[heady silo]

For testing purposes I force fetched a single member. But usually I don't use the force option

[hollow oak]

oh, single

[heady silo]

My bot supports both, the server reported the issue used slash commands.

Is there any way to deal with this on a large server?

[tepid juniper]

Not fetch that often? Add cooldowns, use cache as much as possible, …

[hard storm]

So what user do you fetch? Users from the user option are always resolved.

[heady silo]

I am not really fetching that often.. just when using moderation commands which are used a lot in the server. But only members who already wrote something in the past few minutes are usually punished, so they should be in the cache already.
Maybe as a side note: This server did not use the main bot but a custom branded version of it with the Presence Intent enabled, so almost all members are cached.

Just checked and the custom branded bot which is just in this single server currently has 160k members in its cache.

[tepid juniper]

And do you call member.fetch() or guild.members.fetch(id) to fetch the single member?

Because the former effectively is a force fetch. Just don’t do that at all, you already have the member right there

[heady silo]

Just checked, I always do member.fetch(false) or guild.members.fetch(id)

[tepid juniper]

Why do you do member.fetch at all? You obviously already have a GuildMember object, why would you fetch it again?

[heady silo]

The only time I am using member.fetch(false) is when assigning roles through Reaction Roles as it waits 5 seconds to assign the roles and I want to make sure it is still in the cache (so the member did not leave). But the server that is having the issues is not using Reaction Roles at all.

[tepid juniper]

so that would reject when they left. If you didn’t fetch then the role adding would reject. So nothing gained imho

[heady silo]

Member fetch rejected / member not in cache -> do nothing
Role could not be assigned -> Tell server admins that my bot is unable to add roles because of err.message

But this is not the thing causing issues here. I am still trying to figure out why it gets stuck on fetching a member that is already cached when using guild.members.fetch(id) with force: false

[tepid juniper]

It shouldn’t, that should resolve immediately

Do you have makeCache or sweepers settings ?

[heady silo]

Nope

[tepid juniper]

What djs version are you on exactly?

[heady silo]

13.11.0

[tepid juniper]

Then the member either isn’t cached or is a partial member

[heady silo]

Is there any way to check what fetch requests are queued? So the next time it happens, I can check if a full queue is the issue?

[tepid juniper]

Only by digging deep into the RESTManager‘s inner working, not by documented properties

You can however listen for the ratelimit event

[hot tartan]

client.rest.requestManager.handlers is a collection of handlers, each having a property called #asyncQueue, but idk how far you will get with that

[tepid juniper]

That’s for v14 and the # marks it as very private

[hot tartan]

time to get rid of v13

[heady silo]

Hey again, turns out that it was not the member fetching but member editing that caused the delay when using moderation commands. The server that is having these issues is currently getting +10-30k members every day which is also the reason for their high use of moderation commands. Is Discord increasing rate limits for editing members (for adding/removing roles or timeouts) in large servers? Thank you all for the help btw.

[hollow oak]

I think rate limit is same for each server(large or small), if there would be rate-limit, ig, it will be decreased for large server.
But I can't be sure, I can't find any useful rate-limit docs/guides/notes