caching

[devout sand]

i have a bot with a task that fetches all members from a guild, and stores them in a database. The server has less than 1000 members, but the bot uses more and more time to process the members.
i use await guild.gateway_chunk() to fetch the members, and then iterate over them to see if anyone has joined or left.

After enabling logging i see that the number of members processed just keeps increasing, the same does the time. Is this intended behaviour? Why are we saving the same members in the cache several times?

bot-1 | 2024-08-20 18:05:44,609 - interactions - INFO - Cached members for 888378227679195147 in 2.10 seconds
bot-1 | 2024-08-20 18:06:42,531 - interactions - INFO - Processing 45650 members for 888378227679195147
bot-1 | 2024-08-20 18:06:44,679 - interactions - INFO - Cached members for 888378227679195147 in 2.15 seconds
bot-1 | 2024-08-20 18:07:42,521 - interactions - INFO - Processing 46563 members for 888378227679195147
bot-1 | 2024-08-20 18:07:44,723 - interactions - INFO - Cached members for 888378227679195147 in 2.20 seconds

[storm tapir]

Hey! Once your issue is solved, press the button below to close this thread!

[devout sand]

I also tried to manaually modify the cache size after reading a thread here on discord, but it does not seem to change anything:

bot = Client(intents=Intents.GUILD_MEMBERS | Intents.GUILD_MESSAGES | Intents.DEFAULT, member_cache=create_cache(5,2000,2000))

[devout sand]

running over night, it now shows:

bot-1  | 2024-08-21 08:10:43,073 - interactions - INFO - Processing 816222 members for 888378227679195147
bot-1  | 2024-08-21 08:11:21,405 - interactions - INFO - Cached members for 888378227679195147 in 38.33 seconds

[devout sand]

bump. This is still an issue

[devout sand]

seems like the guild._chunk_cache keeps growing with "empty" users.

hack:

   guild._chunk_cache.clear()
   await guild.gateway_chunk(presences=False)