archive-sharding

[tranquil bone]

How to check message guild shard id

message.guild.shard.id 

[cedar zodiac]

message.guild.shardId

how can i import my client into ShardingManager file?

[nimble mist]

You don’t, what’s the use case?

The clients (yes, plural, one per shard) are instances of the Client class and only exist in the childprocesses (or worker threads) that got spawned by the ShardingManager (named shards). You can only get plain JSON objects from there, never class instances

[cedar zodiac]

I managed to do what I wanted using client.on('shardReady'... in my index.js

[tranquil bone]

This is working

client.shard.id

[nimble mist]

Depends on if they use internal or traditional sharding

[hybrid hearth]

what's a shard? 😳

[fierce compass]

https://discordjs.guide/sharding/

[hybrid hearth]

oh, ok, I'm doing it by myself with child_process :p

[hybrid hearth]

thank you!

[scenic hollow]

So I'm trying to host my sharded bot using railway but discord send me a msg saying my bot trying logging in 1k times in a short period of time so they reset the token, anyone knows why that's happening?

Here's the code if u need it

const { ShardingManager } = require("discord.js");

const manager = new ShardingManager("./bot.js", {
    totalShards: "auto",
    token: process.env.token
});

manager.on('shardCreate', async (shard) => {
  console.log(`${shard.id} shards launched`)
  shard.on('error', (error) => {
     console.log(error)
  })
})

manager.spawn();

[nimble mist]

What version of djs?

[scenic hollow]

13.2.8 ig

[nimble mist]

Update to 13.8.1

[scenic hollow]

Ok

Ig that's gonna be a bit of rewriting then

[nimble mist]

Shouldn’t be much, since it‘s only minor versions, no huge breaking changes in djs itself, but might be some changes if you use builders

[terse cairn]

i need help 🥲 it won’t sent you the channel

client.shard.broadcastEval((c, { channelId, newGuild }) => {
const ch = c.channels.cache.get(channelId)
if (ch) {
ch.send({ embeds: [newGuild] })
}
}, { context: { channelId: logChannel, newGuild } })

[nimble mist]

Did you console.log the context you pass in there to check it is what you expect?

[soft mango]

For cluster

[nimble mist]

Yes?

[soft mango]

How to cluster

[nimble mist]

By starting different ShardingManager on different machines and passing them arrays containing the shard ids they should spawn. You’d need to implement your own way to communicate between these different machines though, as djs only supports IPC directly

[terse cairn]

i found the issue and it was because the logChannel was invalid but thank you

[edgy lake]

Can I someone confirm something rq

If a guild's events aren't handled by my current shard, I can still .fetch my way through to said guild?

[nimble mist]

No, you shouldn’t fetch guilds that aren’t on your current shard. Use broadcastEval to get information from a guild on another shard

[terse cairn]

How can do cluster in bot

[nimble mist]

Do you already do traditional sharding with ShardingManager?

[terse cairn]

Yes

[nimble mist]

Then you‘d pass shardList to the ShardingManager containing a list of the shard IDs that ShardingManager should spawn. Also set totalShards to the amount of all shards that all ShardingManager use.
The hard part is that you then need to implement your own way for cross-shard communication across the different machines, if you need anything like broadcastEval or fetchClientValues yet.

So one machine could spawn shardList: [0,1,2,3,4,5,6] while the next does 7-12 etc.

[terse cairn]

How can setup it? any example

[nimble mist]

What more example do you need? You need several VPS to run your code on, each one having a subset of shardList

[terse cairn]

Okay

How can I setup both machines for cluster etc stuff

Like any package require or held by djs

[nimble mist]

You‘d install your current bot on both and literally only change the shardList and totalShards property to not be 'auto' (in case they were)

And maybe add the inter-guild communication I talked about if you need it

[terse cairn]

Oh host bot in 2 machines

1 to 6 shard I'm one 7 to etc in second

[nimble mist]

Depends on resources you have available and what your bot uses of course. 1-6 was just an example

[terse cairn]

How can setup shard like half - half to total servers

Like what if both shard launch in same server at same time

[nimble mist]

By finding out how many shards you need and then assign 1-half to the first and half+1 to last to the other server

[terse cairn]

Can you explain by a short src example

[icy totem]

Simple and smart. Thx.

[nimble mist]

Don’t really see which part of this you struggle with. I named the properties inside the ShardingManager options you need to pass, I named an example for their values. If that doesn’t suffice for you to get it working I‘m wondering how you managed to get a bot running needing clustering in the first place…

[molten cape]

Hi, why is it going ready https://media.discordapp.net/attachments/927630453932970034/993365142400147521/IMG_1752.png

It constantly doing it, its not normal

It normally don’t say ready only reconnecting then resumed

[snow sierra]

what your djs version? there is was a bug that shard re-identify on resuming

[dreamy mason]

What is worker and process ? and what the deference of them

...

[snow sierra]

why dont you actually use search bar ? ^

[zealous steppe]

Hey, when I start my bot it uses 36mb of ram bot when it starts the shard it shoots to 144mb of ram and doesn't go back down. Does anyone know how to fix this?

[nimble mist]

@solid walrus You don’t get guilds from other shards. Since you won’t be able to use them outside of their shard anyway. What is your use-case. (Move sharding related conversations here)

[solid walrus]

I want to get the guild and send it back as is

[nimble mist]

To do what with it? Because it will be plain JSON data, not a class instance anymore… see pins in this channel

[solid walrus]

I have an API for my Dashboard and to get the discord server name, etc... I get the server with the bot and return the response as is

[nimble mist]

Then just use client.guilds.cache.get(context) not fetch, that way only the shard having that guild will return something that‘s not undefined

Or even better use the shard: property in broadcastEval‘s second parameter to only ask that shard to begin with.

[proud mortar]

Documentation suggestion for @solid walrus:
(static) ShardClientUtil.shardIdForGuildId()
Get the shard id for a given guild id.

[solid walrus]

Like this ?

const guildId = "123153351351513"
client.shard.broadcastEval(async (c, context) => {
  return await c.guilds.fetch(context).catch(() => {
    return null
  })
}, { context: guildId, shard: client.shard.shardIdForGuildId(guildId) }).then(v => {
  if (v[0] == null) return reply.send({error: 401, message: "Incorrect GuildID"})
  return reply.send(v[0])
})

[molten cape]

"discord.js": "^13.5.0"

update to "discord.js": "^13.8.1" ?

[nimble mist]

shareIdForGuildId gets the total amount of shards as second parameter. Pass client.shard.count for that

And it’s a static method, so ShardClientUtil. not client.shard.

[nimble mist]

And you still fetch when you should cache.get

[solid walrus]

without broadcast ?

[solid walrus]

Mmmmh 🤔 OK

[nimble mist]

No, the broadcastEval stays

[molten cape]

How do i get the shard number for the interacting guild

[nimble mist]

Traditional or internal sharding?

[molten cape]

internal

[nimble mist]

interaction.guild.shardId

[molten cape]

ty

to get total shards its :client.options.shards.length?

[nimble mist]

shardCount

[molten cape]

ok

and for my bot status i want it display...
Shard: currentShardOfGuild / TotalShards

[nimble mist]

With internal sharding I don’t know if you can set different status‘ for different shards… gotta double check

[molten cape]

ok

[nimble mist]

Didn’t find anything supporting that

[molten cape]

ohk

[solid walrus]

@nimble mist

client.shard.broadcastEval(async (c, context) => {
  return c.guilds.cache.get(context['guildId'])
}, { context: request.params }).then(v => {
  if (!v[0]) return reply.send({error: 401, message: "Incorrect GuildID"})
  return reply.send(v[0])
}).catch(() => {
  return reply.send({error: 401, message: "Incorrect GuildID"})
})

Same error

[nimble mist]

Where do you run that code?

Because it looks like you have an issue with IPC not working as it should

[nimble mist]

What version of djs? And did you enhance ShardingManager or Shard in any way or use a Third party library for it?

[solid walrus]

On my server 🤔

[solid walrus]

13.8.1

I only use discord.js even for shards

[nimble mist]

Show your ShardingManager file please

[solid walrus]

const Discord = require('discord.js');
const config = require('./setup/config.json')

// ------------- Shard Management ------------------
const manager = new Discord.ShardingManager("./index.js", {
    totalShards: "auto",
    token: config.token
})

manager.spawn([this.totalShards, 15000, 60000])
    .then(shards => {
        shards.forEach(async shard => {
            const gNumber = await shard.eval('this.guilds.cache.size')
            console.log(`[SHARD] Shard ID #${shard.id} -> ${gNumber} guilds`)
        });
    })
    .catch(console.error);

[molten cape]

how to fix bot starting twice. I trying to move from internal sharding to traditional

[molten cape]

starting 2 instances on startup*

[nimble mist]

A) pass the number 2 to the spawn, not the ShardingManager constructor
B) don‘t pass any shards: property to the Client constructor

[molten cape]

Do you have an example?

[rigid ermine]

client.shard.ids

[nimble mist]

Umm, spawn takes an object with amount, timeout, delay properties. Not an array. I‘m confused why that even works…

[solid walrus]

I just looked on discordjs.guide and I'm going to take a base from 0 and we'll see if it prefers

[solid walrus]

Shard:

const Discord = require('discord.js');
const config = require('./setup/config.json')

// ------------- Shard Management ------------------
const { ShardingManager } = require('discord.js');

const manager = new ShardingManager('./index.js', { token: config.token });

manager.on('shardCreate', shard => console.log(`[SHARD] Shard ID ${shard.id}`));

manager.spawn();

Code:

client.shard.broadcastEval((c, context) => {
  let gui = c.guilds.cache.get(context['guildId'])
  if (gui) {
    return gui
  }
  return null
}, { context: request.params }).then(v => {
  if (v[0] == null) return reply.send({error: 401, message: "Incorrect GuildID"})
  return reply.send(v[0])
}).catch((err) => {
  return reply.send({error: 401, message: "Incorrect GuildID"})
})```

It's OK but guilds returned are unavailable 🤔

it's not OK..... 😐

[nimble mist]

Is that your only eval you call on shards? Or could it be another call causing that error?

[solid walrus]

It's this call because when I remove this part I don't get an error...

[nimble mist]

I‘m just confused because neither ShardingManager, Shard, ShardClientUtil or Client have a replace call outside of the login function (which won‘t get called upon broadcastEval)

And your code doesn’t either.

[solid walrus]

That's why I don't understand either 🤔

[solid walrus]

@nimble mist I found !

if I put this, I have an error:

return gui

But if I put for example :

return gui.id

It's OK

[solid walrus]

I found @nimble mist all problem come from Discord Intents

I forget to put Discord.Intents.GUILD
Thank you very much for your help

[molten cape]

how do i spawn 2 shards manually? traditional sharding

[nimble mist]

Pass {amount:2} to the manager.spawn call

[molten cape]

const { ShardingManager } = require("discord.js");
const config = require("./src/config.js");
const manager = new ShardingManager('./src/bot.js', {
    token: config.token
});
manager.spawn({amount: 2});

like this?

Also how do I get total shards and current shards on guild

[nimble mist]

client.shard.ids[0] and client.shard.count

[molten cape]

ok

In internal sharding can i limit how much guilds can go to 1 shard?

[coral dome]

hello, how can i send dms messages to a user using sharding manager?

[nimble mist]

Why do you suddenly switch to internal sharding? Which guild goes to which shard is based solely on guildId and total amount of shards. So you can only limit that by increasing the amount of shards.

[nimble mist]

You can send DMs from any shard (no need to go through the ShardingManager). Their response will only reach shard 0 though…

[molten cape]

I’m working on 2 different bots

[coral dome]

it's still working if not using broadcastEval while sending DMs?

[nimble mist]

Yes

[coral dome]

oh okay, but if the user isn't in shard 0, it's still working for DMs the user?

[nimble mist]

Users don’t belong to a specific shard, Guilds do

messageCreate Events for DMs will be sent to shard 0 though

So messageCollectors will only work in DMs on shard 0 too

[coral dome]

ah i see

thanks a lot for the information

[solid walrus]

Hello
How put a function in broadcastEval ?

function test() {
  return "hello world"
}

client.shard.broadcastEval((clientShard, context) => {
  return context.test()
}, context: { test: test }) // doesnt work (ERROR: context.test() is not un function)

Thanks

[dense cape]

Put the function inside of the broadcastEval

You can’t pass functions via broadcastEval bc it’s not json serializable. And even if you do, it can’t use the variables you defined in the upper scope

[marble trellis]

when using ShardingManager, how many shards will be spawned per subprocess?

[nimble mist]

One

[terse cairn]

Will Mongodb will be an ideal database for traditional shading?

[terse cairn]

also is there a way to get shard id from client object?

or maybe client.guilds.cache.first().shardId?

[nimble mist]

client.shard.ids is an array of ids in that client’s process (usually only one id with traditional sharding only)

[terse cairn]

oh okay

thanks

[terse cairn]

how do i assign only selected guilds to a shard?

[dense cape]

Not possible, only Discord can change the default guild-shard assignment behavior

And they only do that for larger bots, if necessary

[terse cairn]

oh, got it

[terse cairn]

When we spwn a shard process using sharding. 1 shard takes 1 thread?
So if I have a CPU with 6 Cores and 12 Threads. It takes 1 thread to spawn a shard.
So I can spawn 11 shards and 1 host?

or maybe 10 shards and 1 host and 1 for system use?

[nimble mist]

You can run as many processes as you want on that CPU (at least I hope you have an OS that supports more than 12 processes running simultaneously). A shard is a child process or a worker thread. Not a CPU core blocking magic thing

[terse cairn]

I am using Debian. Oh so, a worker thread is something else? hmmm.

[bold orbit]

Hey friends how can i fix the SHARDING_READY_TIMEOUT error even the infinity wait doesn't seem to work out for me and its still giving this error

[proud mortar]

Tag suggestion for @bold orbit:
Please add the following code to your code base outside of any other event listeners and provide the full log output relevant to your issue.

client
    .on("debug", console.log)
    .on("warn", console.log)

• Note: if you initialize your Client as bot or other identifiers you need to use these instead of client
• If the output is too long to post consider using a bin instead: gist | hasteb.in | sourceb.in | hastebin

[bold orbit]

@nimble mist thanks for the log tip , here more details on this same issue

[dense cape]

What's ur d.js version? npm ls discord.js

[bold orbit]

npm ls discord.js
00-discordbot-api@1.0.0 D:\te-ordenen\BrawlStars-game-files\00-discordbot-api
└── discord.js@13.8.1

i think this is the last version you can have

[nimble mist]

Looks like you should increase your timeout in manager.spawn of your sharding file. Since your client does seem to become ready but only after that timeout happens

[bold orbit]

-1 or infinity should disable the timeout , but it doesn't . also tried putting it rly high like 9999999 but that also didn't work

[vital oriole]

I also got this issue today. How can I disable the timeout?

[vital oriole]

manager.spawn({ timeout: -1 })

Was the solution ;D
All 7 Shards are alive again

[bold orbit]

yeah this works , life saver !

[vital oriole]

Yeah! But its strange. The last months it working fine until today where I restarted the bot for an code change

[bold orbit]

yeah same just from this week it started to act like this. then i was checking what was going on and the timeout value was not used , checked github and it was not there. so we indeed needed this object to define it ourself

[nimble mist]

Where did you define the timeout value if not there?

[bold orbit]

in the shardmanager like it was before

[nimble mist]

By before you mean v12? Because it’s been like this since v13.0.0

[terse cairn]

can I create a single db connection in the main file and not in shard file? So all shards only access a single db instance.

or do I have to create new connections on every shard?
I am using mongodb

[nimble mist]

While you can that would mean all DB requests would have to go through the IPC channel and same for the results.

[terse cairn]

what's IPC?

[nimble mist]

inter process communication

[terse cairn]

oh so the link between processes/shards?

[nimble mist]

The communication link, yes. What’s being used for broadcastEval etc

[terse cairn]

oh okay Thanks, so it's best to create new connection for each shard?

[bold orbit]

no as in v13dev

[nimble mist]

Ah, so before April 2021…

[nimble mist]

It’s what I do, yes. Else the db connection Is too slow

[terse cairn]

Got it
Thanks :)

[terse cairn]

Hey, when sharding. Is each shard is a worker thread? If yes then can I spawn more threads inside the shard?

I tried spawning a worker and it worked. So each shard is a new process?

[nimble mist]

You can set the mode in ShardingManager to either process or worker

[terse cairn]

ohh okay thanks

[pale sapphire]

Whenever I start my bot, I have Shard ID 0 logging undefined servers.

bot.shard.broadcastEval(client => console.log('Shard ID', client.shard.ids[0], client.guilds.cache.map(guild => guild.name)));

[rough heron]

Hi, I have this error Shards are still being spawned. when I launch the bot, I try to console.log(client.guilds.cache.size) because the bot is connecting to all shards. Is there a way to wait for the end of all the connections and then display in the console the number of guilds?

[nimble mist]

When do you send that broadcastEval?

[pale sapphire]

In the ready event.

[nimble mist]

The way I do it is attach a listener to each shard‘s ready event and add their id to a Set. When the Set‘s size reaches the count of shards I send an 'allShardsReady' message to all shards. Then they know they can start doing sharding stuff

[nimble mist]

Of which shard?

[pale sapphire]

Shard 0.

[nimble mist]

Well, shard 0 will be ready before all other shards will be (since it gets spawned first), so the other shards won‘t have all their guilds sent to them yet

[pale sapphire]

I only have one shard. Error [SHARDING_READY_TIMEOUT]: Shard 0's Client took too long to become ready.

[rough heron]

Do you have an example please because I don't really understand how it works

[nimble mist]

Why do you have a broadcastEval etc. in there if you only have one shard?

[pale sapphire]

Cause I sometimes test it out with two shards.

However, why would it sometimes result with undefined servers?

[nimble mist]

const readyShards = new Set();

manager.on("shardCreate", (shard) => shard.on("message", (message) =>
{

   if (message.fullyLoaded && startTime)
   {

      readyShards.add(shard.id);

      if (readyShards.size === manager.totalShards)
      { … }}}));

That’s what my ShardingManager has

[nimble mist]

Because the API didn’t send those Guild‘s payload yet

[rough heron]

Okay, thanks, I understand better

[pale sapphire]

Is it possible that I could wait, like, 10 seconds before executing the code inside the ready.js event?

Maybe that would fix the issue while I figure out on my canary bot?

[nimble mist]

Sure, setTimeout exists

[pale sapphire]

Well, that didn't work. Keeps erroring right and left.

I still can't start my bot. :/

[nimble mist]

Just checking: you start through a ShardingManager? But only 1 shard?

[pale sapphire]

Yes.

[nimble mist]

Then add timeout:-1 in the object passed to <ShardingManager>.spawn

[pale sapphire]

manager.spawn({ delay: 10000, timeout: -1 });

[nimble mist]

Yup

[pale sapphire]

Well, let's hope it works locally so I can then try on the VPS.

Damn, still didn't start yet.

[plush cape]

Hi, is it impossible to use the shardsManager system in ES6?

[dense cape]

You mean ESM?

No, just set type to module in the package.json or use the mjs extension

[plush cape]

Ah yes, perfect, thank you

Is there any way to detect when all the shards are connected? Because when my bot starts, it should update its status but I get an error because not all shards are connected and I have to add a timeout.

[dense cape]

You can await the manager.spawn call

[plush cape]

Perfect, thank you

[nimble mist]

That only works with timeout not set to -1 or Infinity btw.

[lunar seal]

Hello, is it possible to have the list of all the servers? Because when I do a function to get the full list, I only get the current shard servers. If I use await client.shard.fetchClientValues("guilds.cache.size");, I get an error because not all shards are connected yet.

[nimble mist]

Then call that function only once all shards are connected 🤷‍♂️

[lunar seal]

Ok, but I can't use to know when all shards are connected in the bot.js file like in the tuto https://discordjs.guide/sharding/#fetchclientvalues

I get this error: This session would have handled too many guilds - Sharding is required. because I export a function that returns the number of guilds. If I delete the function, I don't have the problem anymore

[nimble mist]

You can either await the promise of ShardingManager.spawn() or you can simply catch the rejection you get and retry in a setTimeout until you don’t get that rejection

[lunar seal]

Okay, I understand. But if I want to make a promise, how can I do that? Because I can't export my discord client

[nimble mist]

You need to await in the ShardingManager and then send a broadcastEval to all shards to inform them of the finished login

[lunar seal]

I send a broadcastEval in my bot.js file? Because if I do that, I get this error This session would have handled too many guilds - Sharding is required. because in my index file, I import the function and even if I import it without using it, I get the error

[nimble mist]

Where do you export (and import) that function?

[nimble mist]

You can’t import your ShardingManager in the Client‘s file. You need to use broadcastEval/fetchClientValues or process.send(…) directly to communicate with manager/other shards

[lunar seal]

Export my function to the bot.js file (This is where my discord client is created.) and then import the function to the index file (shardsManager file)

[lunar seal]

Okay, is there any example or documentation on this subject?

[nimble mist]

Yeah, don’t do that. ShardingManager and Client run in seperate processes. Importing one into the other means you create more Clients

[nimble mist]

Which part? BroadcastEval and fetchClientValues are explained in the sharding portion of the guide (see pins of this channel). process.send is a node standard function

[lunar seal]

Okay, thanks for your help, I'll look into it

[ionic frigate]

    ErrorCaptureStackTrace(err);
    ^

Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
    at new NodeError (node:internal/errors:372:5)
    at process.target.send (node:internal/child_process:741:16)
    at /root/GolyatsecGuardProd/node_modules/discord.js/src/sharding/ShardClientUtil.js:85:17
    at new Promise (<anonymous>)
    at ShardClientUtil.send (/root/GolyatsecGuardProd/node_modules/discord.js/src/sharding/ShardClientUtil.js:83:12)
    at /root/GolyatsecGuardProd/node_modules/discord.js/src/sharding/ShardClientUtil.js:121:12
    at new Promise (<anonymous>)
    at ShardClientUtil.fetchClientValues (/root/GolyatsecGuardProd/node_modules/discord.js/src/sharding/ShardClientUtil.js:108:12)
    at _loop (/root/GolyatsecGuardProd/codebase/web/index.js:143:55)
    at Timeout._onTimeout (/root/GolyatsecGuardProd/codebase/web/index.js:136:17) {
  code: 'ERR_IPC_CHANNEL_CLOSED'
}

I am getting this error when using ShardingManager. Can anyone help?

[nimble mist]

Did your ShardingManager keep on running after that and respawn the shard? Because it looks like the ShardingManager died first… or your host doesn’t support IPC somehow

[ionic frigate]

Yep my shard manager keeps running after crash and tries again and again.

[ionic frigate]

node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

Error [SHARDING_IN_PROCESS]: Shards are still being spawned.
    at ShardingManager._performOnShards (/root/GolyatsecGuardProd/node_modules/discord.js/src/sharding/ShardingManager.js:285:75)
    at ShardingManager.fetchClientValues (/root/GolyatsecGuardProd/node_modules/discord.js/src/sharding/ShardingManager.js:266:17)
    at Shard._handleMessage (/root/GolyatsecGuardProd/node_modules/discord.js/src/sharding/Shard.js:359:22)
    at ChildProcess.emit (node:events:527:28)
    at emit (node:internal/child_process:938:14)
    at processTicksAndRejections (node:internal/process/task_queues:84:21)
[8:27:10 PM] [Discord] Shard 0 died.
[8:27:10 PM] [Discord] Shard 0 spawned.
/root/GolyatsecGuardProd/node_modules/discord.js/src/sharding/Shard.js:161
        reject(new Error('SHARDING_READY_DIED', this.id));
               ^

Error [SHARDING_READY_DIED]: Shard 0's process exited before its Client became ready.
    at Shard.onDeath (/root/GolyatsecGuardProd/node_modules/discord.js/src/sharding/Shard.js:161:16)
    at Object.onceWrapper (node:events:642:26)
    at Shard.emit (node:events:539:35)
    at Shard._handleExit (/root/GolyatsecGuardProd/node_modules/discord.js/src/sharding/Shard.js:407:10)
    at ChildProcess.emit (node:events:527:28)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12) {
  [Symbol(code)]: 'SHARDING_READY_DIED'

[nimble mist]

Wait, that error looks like it comes from a dashboard or some other webservice running in the same process. Does that check if all shards are spawned before calling fetchClientValues ?

[ionic frigate]

Hmm wait ill look into that. Could this have something to do with Contoba? (VPS)

[nimble mist]

Theoretically, sure. I don’t know them, but if it works locally but not on your VPS that should be a strong indication

[ionic frigate]

Yep works in local environment.

[nimble mist]

Then it’s most definitely an issue with your VPS and how it does/doesn’t allow childprocesses and IPC channels

[ionic frigate]

Thank you for your help. That issue definitely due to our VPS.

[molten cape]

When my shard error whats all the possible reason for it

[nimble mist]

Be more specific. what error does your shard produce?

[molten cape]

gonna have to come back when it happens again, the console is overwritten

[wispy citrus]

Hi 👋 , I'm trying to get the memory usage of my shards, but I can't, I have my code here, did I do something wrong?

const promise = await client.shard.broadcastEval(`[this.uptime, process.memoryUsage().heapUsed]`)

error:

reject(new TypeError('SHARDING_INVALID_EVAL_BROADCAST'));
TypeError: SHARDING_INVALID_EVAL_BROADCAST

[next plaza]

You have to use functions, instead of strings (depriciated)

[ionic frigate]

Update: I migrated from Contabo to Google servers. But there was no solution. I kept getting the error. Now I got api limit blocked from Discord. My bot is not active. I will wait 1 day. My project only works locally. Very interesting. As a last resort I will be using a Windows machine.

[faint coyote]

if i set a global variable as the shardID, would it overwrite in process mode to the other shards?

[dense cape]

No

Processes don’t share memory

[faint coyote]

ok thank you

[dense cape]

Not that you should be using global variables

[faint coyote]

ik

[rough heron]

Hi, someone has an example to get the whole server list via client.shar.fetchClientValues('')

[proud mortar]

Suggestion for @rough heron:
Sharding: FetchClientValues
read more

[nimble mist]

If you want the actual list and not just the count you‘d need to use broadcastEval instead.

[wispy citrus]

ho ok, do you have one example ? because I can't find in the documentation (for the memory) c:

[proud mortar]

Documentation suggestion for @wispy citrus:
ShardClientUtil#broadcastEval()
Evaluates a script or function on all shards, or a given shard, in the context of the [Client](<https://discord.js.org/#/docs/discord.js/stable/class/Client>)s.

[neat pewter]

it has an example

it also has examples in guide, take a look at it

[wispy citrus]

ok thanks !

[lusty nest]

Hi everyone, does anyone know how I can resolve that :

1|Anonymous  | [2022-07-11T20:46:23.968Z] Error [SHARDING_READY_TIMEOUT]: Shard 1's Client took too long to become ready.
1|Anonymous  |     at Timeout.onTimeout (/home/bot/anonymous-bot/node_modules/discord.js/src/sharding/Shard.js:166:16)
1|Anonymous  |     at listOnTimeout (node:internal/timers:559:17)
1|Anonymous  |     at processTimers (node:internal/timers:502:7) {
1|Anonymous  |   [Symbol(code)]: 'SHARDING_READY_TIMEOUT'
1|Anonymous  | }

It's when I would like to restart my bot

[pale sapphire]

[SHARDS]: Launched shard 0
Error [SHARDING_IN_PROCESS]: Shards are still being spawned.
[SHARDS]: Launched shard 1

Whenever I have two shards, it errors this.

[nimble mist]

Do you try to broadcastEval or fetchClientValues before all shards are spawned?

[pale sapphire]

Wait, so my ready.js event fires from the very first shard getting spawned?

[nimble mist]

It fires for each shard individually, yes

The manager.spawn(…) resolves when all shards are ready. Use that to make your shards do stuff that needs broadcastEval etc

[pale sapphire]

Is there no resolve parameter?

[nimble mist]

Where?

[pale sapphire]

In the shardCreate event or something.

[nimble mist]

Umm, what should that parameter do exactly?

[pale sapphire]

Nevermind, my bad.

require('dotenv').config();
const { ShardingManager } = require('discord.js');
const manager = new ShardingManager('./src/index.js', { token: process.env.TOKEN, totalShards: 2 });

manager.on('shardCreate', shard => {
    shard.on('ready', async () => {
        if (manager.shards.size === manager.totalShards && manager.shards.every(x => x.ready)) await manager.broadcast('allReady');
    });
    console.log(`[SHARDS]: Launched shard ${shard.id}`);
});

manager.spawn({ delay: 10000, timeout: -1 });

I decided to make something like this, but it never really broadcasted allReady.

[pale sapphire]

For some reason, I'm facing this issue:

console.log(manager.shards.every(x => x.ready)); // true
console.log(manager.shards.size === manager.totalShards); // false It's `1 2`.

[nimble mist]

And did you try logging both values you compare there seperately?

[pale sapphire]

Indeed.

Check the updated code for the logs.

[nimble mist]

Then your second shard didn’t get spawned… listen to the debug event to find out more

[pale sapphire]

Is it possible to wait it out?

[nimble mist]

As I said: the <ShardingManager>.spawn() call‘s Promise resolves when all shards are ready

[pale sapphire]

Is it possible to get the status of the promise?

[nimble mist]

We‘re entering basic JS territory now… #rules 2

[pale sapphire]

Let's do it.. alright. 👍

[proud mortar]

Tag suggestion for @pale sapphire:
Resources to understand Promise:
• MDN: learn more
• Guide: learn more
• JavaScript info: learn more

[pale sapphire]

I got it solved. 👋

[brazen herald]

Hi, what should I put in the "select redirect url" box?

@brazen herald

[nimble mist]

Nothing, what do you need guilds scope for?

[rough heron]

Hello, my bot should send a message when a Twitch streamer starts a stream but it can't send the message because the server is not in the right shards. Do you have an idea to solve this problem?

I tried to use broadcastEval but I get this error: Cannot read properties of null (reading 'broadcastEval')

[dense cape]

What did you call broadcastEval on?

[rough heron]

discord.shard.broadcastEval(`this.channels.cache.get('123..').send({ embeds: ${embed.toJSON()} })`)

[dense cape]

How is discord defined?

[rough heron]

import { Client } from 'discord.js';

import {} from 'dotenv/config';

const discord = new Client({
  intents: [
    'GUILD_MEMBERS',
    'GUILD_PRESENCES',
    'GUILDS',
    'GUILD_INTEGRATIONS',
    'GUILD_MESSAGE_REACTIONS',
    'GUILD_MESSAGES',
    'DIRECT_MESSAGES'
  ],
  restRequestTimeout: 60000,
  waitGuildTimeout: 1000,
  partials: ['CHANNEL']
});

discord.login(process.env.DISCORD_Token);

discord.once('ready', async () => {
  discord.shard.broadcastEval('this.channels.cache.get("123..").send({ content: "test message ...." })');
});

This is my code for testing but it doesn't work. I load my file with the ShardingManager like on the doc

@dense cape

[dense cape]

There’s got be smth funky going on bc it can only be null or a ShardClientUtil, not undefined

Oh wait, it does say null

How did you startup the bot?

[rough heron]

I made this tuto https://discordjs.guide/sharding/#when-to-shard .

So my file is bot.js but I launch my bot with index.js

[terse cairn]

does broadcastEval takes an async function?

[nimble mist]

It can, yes

[terse cairn]

thanks :)

[pale sapphire]

I have this broadcastEval() code, however:

bot.shard.broadcastEval(client => console.log('Shard ID', client.shard.ids[0], client.guilds.cache.map(guild => guild.name)));

When it comes to the results of the guild names, I can see, supposedly, guild called ABC in shard 0 and shard 1? What does this mean and is there something I should be doing?

[terse cairn]

i wanted to know the eta for launching @discordjs/sharder by kyra

[nimble mist]

Seems like you either have two different guilds names ABC or you tried to fetch ABC guild on a shard that shouldn’t handle it

Or you have shard overlap from wrongly configuring shards

[pale sapphire]

Different guild names ABC, I checked, that's not the case.

[pale sapphire]

How can I be sure on the second case?

[nimble mist]

Include the guild.shardId in your console.log (and if both claim it belongs to them log ShardClientUtil.shardIdForGuildId(guild.id, client.shard.count) to find out who‘s right

[pale sapphire]

It goes with the greater number.

[nimble mist]

Either you‘re german like me or I didn’t get what you mean

[pale sapphire]

I figured out the issue. 👋 Apparently, when you fetch for a specific server, the guild gets added on all shards.

[nimble mist]

Huh? Why do you fetch a server? Guilds are always cached and shouldn’t be fetched

[pale sapphire]

Yes, my bad on such so.

[pale sapphire]

However, when we have multiple shards, and supposedly, we would want to get a specific user...

I would have to use fetch during that case, correct?

[nimble mist]

Users yes, members only on the guild‘s shard

[pale sapphire]

Wait, how for members? Example.. ?

[nimble mist]

Just remember that Dm Messages will all reach shard 0

[nimble mist]

<Guild>.members.fetch() should only be done on the shard that guild belongs to

[pale sapphire]

For sure.

[nimble mist]

client.users.fetch() you can do on any shard.

Just be aware that messages received by your bot in DMs will always reach shard 0 only

[pale sapphire]

Indeed, for sure. Understandable.

Lastly.. you know, depending on the amount of shards, it equals to the amount of times my index.js file would run.

For example, if I have 4 shards, this code would run 4 times: https://cdn.hamoodihajjiri.com/1CH5DwGBsm .

[terse cairn]

Hello, I have a bot that is on more than 2500 servers, so I follow the instructions https://discordjs.guide/sharding/#how-does-sharding-work.

But when I want to send a message on a channel (client.channels.cache.get('1234567890'), I get an error because the server is not on the right shards (if I understand correctly). So I use the broadcastEval function but I get this error: Canot read properties of null (reading 'broadcastEval').

[pale sapphire]

Apparently, having multiple shards and looping through guild cache, my particular server isn't cached, basically, not all servers are.

[nimble mist]

Yes. Only the guilds that shard handles are cached. And you shouldn’t try to access other shard‘s guilds through cache but use broadcastEval instead

[nimble mist]

Show me how you tried to use broadcastEval (and where)

[pale sapphire]

How about this case?

[nimble mist]

Each shard spawns a Client, yes. Each of your shards will run your file in their own process, so you‘d have 4 top.gg posters, etc. top.gg supports being run in ShardingManager, don’t know about the others

[pale sapphire]

Oh, damn.

[pale sapphire]

Meaning creating and setting commands will run 4 times?

[nimble mist]

If you do that in your client file yes. But you shouldn’t do that in the first place anyway. Only deploy them when they changed, not on every restart

[pale sapphire]

This is where I'm confused, how can I do that?

[proud mortar]

Suggestion for @pale sapphire:
Creating Your Bot: Registering commands - Command deployment script
read more

[pale sapphire]

Oh, moment.

[pale sapphire]

Lastly, in order to use such so, I need to wait for all shards to be spawned, whereas, I can just wait for them to be spawned before I can post slash commands, is that possible?

I have 4 shards, wait for all to be spawned, then set slash commands.

[nimble mist]

You can post slashcommands without spawning a single shard

[pale sapphire]

Fair, fair. 👍

[terse cairn]

client.shard.broadcastEval('this.channels.cache.get("968262845601038436").send({ content: "TEST" })')

[nimble mist]

And where do you call that?

[terse cairn]

In my client.on('ready')

[nimble mist]

Also broadcastEval gets passed a function, not a string. And you need to do the sending in there, can’t get a Channel out of it (check this channel‘s pins)

[nimble mist]

If client.shard is null in your client‘s ready event then you don’t have a ShardingManager instantiating your client‘s file

[terse cairn]

Yes, I use ShardingManager in another file.

I did like in the discord.js tuto

[nimble mist]

And which file do you start?

[terse cairn]

I run my index.js file which is linked to bot.js

[nimble mist]

Sorry, had to run some errands. What’s in your index.js, what in bot.js and how are they linked?

[terse cairn]

index.js

bot.js

[nimble mist]

Okay, then you only need to change what you pass to broadcastEval to be a function, not a string. So (client) => and remove the ''

But your error of client.shard being null makes no sense if you run index.js like that…

What djs Version?

[terse cairn]

I don't see where I should do it.

"discord.js": "^13.8.1"

[proud mortar]

Sharding: BroadcastEval
read more

[nimble mist]

For an example where the function should be

[terse cairn]

Perfect, it works, thank you very much

[long night]

I'm trying to get an entire shard's data using broadcastEval. When i log it to the console it works fine, if i try to use .then() to get it and send it to discord, it hates me for it.

this works:

await interaction.client.shard.broadcastEval((client) => console.log(client.ws.shards), { shard: shardId });

this doesn't:

interaction.client.shard.broadcastEval((client) => client.ws.shards, { shard: shardId })
  .then((results) => {
    const shards = results.map((result) => result);
    console.log(shards);
  })
  .catch((err) => {
    console.log(err);
  });

It just gives the the error in the screenshot. I'm sure im just being stupid. But first day really working with shards. Usually other team members do this stuff. Any help and guidance is much appreciated.

[small nebula]

after setting up sharding as in the example docs i get the following error

node:internal/url:1413
  return decodeURIComponent(pathname);
         ^

URIError: URI malformed
    at decodeURIComponent (<anonymous>)
    at getPathFromURLPosix (node:internal/url:1413:10)
    at fileURLToPath (node:internal/url:1423:50)
    at finalizeResolution (node:internal/modules/esm/resolve:403:14)
    at moduleResolve (node:internal/modules/esm/resolve:1009:10)
    at defaultResolve (node:internal/modules/esm/resolve:1218:11)
    at ESMLoader.resolve (node:internal/modules/esm/loader:580:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:294:18)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:80:40)
    at link (node:internal/modules/esm/module_job:78:36)
/home/christopher/Code/dice-witch/node_modules/discord.js/src/sharding/Shard.js:161
        reject(new Error('SHARDING_READY_DIED', this.id));

here's my bot.ts https://pastebin.com/RENuXUTU

no idea what the error message means and not sure where to start debugging 😢

[azure prism]

the first thing i noticed is URI malformed

and in this case, pathname is the URI

[small nebula]

right, but that's in node internal/url, so it doesn't really help

[azure prism]

it starts at discord.js/src/sharding/Shard.js:161

i'm also curious about reject(new Error('SHARDING_READY_DIED', this.id));

is that the complete stack trace (error message)?

i was expecting to see your code some where in there.

that Shard.js file is in the discord.js module.

[small nebula]

here's the rest of it

Error [SHARDING_READY_DIED]: Shard 0's process exited before its Client became ready.
    at Shard.onDeath (/home/christopher/Code/dice-witch/node_modules/discord.js/src/sharding/Shard.js:161:16)
    at Object.onceWrapper (node:events:642:26)
    at Shard.emit (node:events:527:28)
    at Shard._handleExit (/home/christopher/Code/dice-witch/node_modules/discord.js/src/sharding/Shard.js:407:10)
    at ChildProcess.emit (node:events:527:28)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12) {
  [Symbol(code)]: 'SHARDING_READY_DIED'
}```

so we know the child process died but we don't know why, i'm sure there's a way to log it but i'm coming up blank for how to do it

[azure prism]

i don't know what in the code you linked to is causing the "sharding" in the first place.

i see some console.logging in your code, do you see that output on the "terminal"?

are you running this locally? on your dev box? or somewhere else?

[small nebula]

this is my index.ts

import { Shard, ShardingManager } from "discord.js";
import config  from "../config.js"
const { discordToken } = config;
const manager = new ShardingManager('./build/src/bot.js', { token: discordToken  });

manager.on('shardCreate', (shard: Shard)  => console.log(`Launched shard ${shard.id}`));
manager.spawn();

[azure prism]

i'm learning about sharding now.

https://gavwin.gitbooks.io/sharding-in-discord-js/content/

[small nebula]

the official docs are here: https://discordjs.guide/sharding/

[azure prism]

my next suggestion is to delete all but necessary functionality in your bot.js app and see if the spawn manager will span the simplest of apps.

also, what node -v are you using?

[small nebula]

16.6.0, the minimum req for djs v13

[azure prism]

i just got your code running on my machine using node -v v18.2.0

i'm on a Mac M1, fresh install of your code dependencies.

    "axios": "^0.27.2",
    "discord.js": "^13.8.1",
    "toad-scheduler": "^1.6.1"
  },

[small nebula]

interesting!! thanks for doing that. i just switched over to node v18.2.0, updated to those deps, deleted node_modules, reinstalled and am getting the same error

[azure prism]

yw. man. are you just running node index.js?

[nimble mist]

broadcastEval can’t return class instances, as they get send over an IPC channel between processes as JSON. So you won’t ever get instances from it, only plain JSON (and JSONified instance objects)

[small nebula]

i'm using typescript so in my dist folder but otherwise yes

[azure prism]

there's sample code in this thread that sends the --trace-warnings which might log more info when running: https://github.com/discordjs/discord.js/issues/5364

[nimble mist]

So you start the shards from build/src/bot.js then what was the index.ts you showed earlier?

[small nebula]

sorry that's my bot.ts, lol, i misspoke, let me edit the comment so i don't confuse anyone else

pastebin is the bot.ts, what i pasted above is the index.ts

[azure prism]

does manager.spawn(); need to be await manager.spawn();?

because it returns a Promise

https://github.com/discordjs/discord.js/blob/16390efe6e4f2035aa0fe365c8d593366beca7ce/packages/discord.js/src/sharding/Shard.js#L141

[small nebula]

not sure about when we'd want to await it since i'm a noob to sharding, but i just tried and awaiting it doesn't resolve the issue 😢

[azure prism]

did you add the trace-warnings cli argument?

    execArgv: ['--trace-warnings'],
    token: discordToken,
});```

[small nebula]

doesn't seem to produce any additional output unfortunately

[nimble mist]

Only if you want to do something after it resolves. If you don’t care for the promise you don’t need to await it

[nimble mist]

For sharding move the content of your startServer function to top-level. There are some issues when the login call isn’t in top level for sharding

[small nebula]

thanks for the tip! still doesn't resolve the issue

[proud mortar]

Tag suggestion for @small nebula:
Please add the following code to your code base outside of any other event listeners and provide the full log output relevant to your issue.

client
    .on("debug", console.log)
    .on("warn", console.log)

• Note: if you initialize your Client as bot or other identifiers you need to use these instead of client
• If the output is too long to post consider using a bin instead: gist | hasteb.in | sourceb.in | hastebin

[small nebula]

☝️ the crash comes before i even declare the client, but if these methods were on ShardManager instead of Client that would be exactly what i need to get some insight here

[azure prism]

i'm available to to pair on this if you still need help.

[long night]

Stupid question, but could I just stringify it and then convert it back after?

Like with request payloads?

[long night]

i figured it out. after a much needed break lol. stringify'd it and then parse it on return.

[molten cape]

Hi, uhh how do i fixed this issue

[visual fable]

My shards keep disconnecting and reconnecting making the bot irresponsive in multiple servers. can anyone help or guide me what to do?
My bot is in 12k servers 12 shard

[nimble mist]

Do you have debug logs of those dis-/reconnects already?

[visual fable]

no shard error thrown

[nimble mist]

That wasn’t what I asked.

[proud mortar]

Tag suggestion for @visual fable:
Please add the following code to your code base outside of any other event listeners and provide the full log output relevant to your issue.

client
    .on("debug", console.log)
    .on("warn", console.log)

• Note: if you initialize your Client as bot or other identifiers you need to use these instead of client
• If the output is too long to post consider using a bin instead: gist | hasteb.in | sourceb.in | hastebin

[visual fable]

I see, I'm not listening to debug and warn events

[nimble mist]

Those are just files. You don’t need an extra file to add the debugging, just add those lines temporarily anywhere outside of any event

[visual fable]

the logs are spammed with [Function: bound]

is it normal ?

[nimble mist]

Uh, no?

Where do you log it to?

[visual fable]

console

from debug

[nimble mist]

Show the output please

Wait, show how you added the debug handler actually

[faint coyote]

since i shareded my bot, it keeps loosing its status message on the profile

  const guildCountsArr = await client.shard.fetchClientValues('guilds.cache.size');
  const guildCounts = guildCountsArr.reduce((previousCount, currentCount) => previousCount + currentCount, 0);
  await client.user.setActivity(`${guildCounts} servers.`, { type: 'LISTENING' });

so.. should i run it in a look? and how often?

[nimble mist]

It might be enough to run it in shardReady event…

[faint coyote]

so. its not supposed to be in the shard, but in the shardmanager

[nimble mist]

It’s most probably an issue with the WebSocketShard reconnecting, not the actual Shard process

[faint coyote]

Ah. Alright

[terse cairn]

How can I check which shard has which guilds assigned to it?

[proud mortar]

Documentation suggestion for @terse cairn:
(static) ShardClientUtil.shardIdForGuildId()
Get the shard id for a given guild id.

[nimble mist]

It’s based upon guildID, bitshifted and then modulo the totalAmount of shards. Unless you have a really big bot but then discord would have contacted you and told you about it😉

Because I felt like it, this is the actual formula used by discord

const shard = Number(BigInt(guildId) >> 22n) % shardCount;```

[terse cairn]

oh
okay thanks

btw how can I listen to broadcast messages on shards?

@nimble mist

[nimble mist]

process.on('message' if it‘s child process mode

[terse cairn]

oh got it Thanks bro :)

It is a static property so this should work but is manager.shards.size is accurate?

@nimble mist Also which approach is better? I have some startup tasks for some guilds that are stored in a db.
Should I run startup tasks of guilds on each shards and check of that shard has that guild by checking the cache?
or I should check for the shard a guild should be in and then send a process broadcast to that shard and then do stuff?

[nimble mist]

I‘d do it in each guild shard and let them filter those that are on their shard (with client.guilds.cache). And for the amount I‘d use manager.totalShards rather (makes it work even if you decide to switch to cluster sharding later)

[terse cairn]

oh, so you say it's better to do it in each shard?

[nimble mist]

Oh yeah, that’s what I meant. Fixed it

[terse cairn]

Ah Thanks

[devout vine]

I have a kinda complex question about sharding and the global rate limit. I've had issues in the past with hitting the global rate limit when running a job (which needs to post messages on every server) on multiple shards simultaneously, but I also know discord.js queues up actions to avoid rate limits, and I think the reason I hit the global rate limit is because there's no cross-shard communication on rate limits. I'm thinking of redesigning this job so it goes one shard at time to avoid the rate limit, but my question is this: can I run the job for many guilds all at once within the same shard, waiting for that shard to complete, then run the next shard, while discord.js avoid the rate limit / queue up those actions?

[nimble mist]

I‘d more rethink what you‘re doing. If you know your actions are resulting in ratelimits you shouldn’t do that so much…

[carmine oyster]

where can i find sharding events docs? like all the names of events in sharding

[proud mortar]

Shard
A self-contained shard created by the [ShardingManager](<https://discord.js.org/#/docs/discord.js/stable/class/ShardingManager>). Each one has a ChildProcess that contains an instance of the bot and its [Client](<https://discord.js.org/#/docs/discord.js/stable/class/Client>). When its child process/worker exits for any reason, the shard will spawn a new one to replace it as necessary.

ShardingManager
This is a utility class that makes multi-process sharding of a bot an easy and painless experience. It works by spawning a self-contained ChildProcess or Worker for each individual shard, each containing its own instance of your bot's [Client](<https://discord.js.org/#/docs/discord.js/stable/class/Client>). They all have a line of communication with the master process, and there are several useful methods that utilise it in order to simplify tasks that are normally difficult with sharding. It can spawn a specific number of shards or the amount that Discord suggests for the bot, and takes a path to your main bot script to launch for each one.

Client (extends BaseClient)
The main hub for interacting with the Discord API, and the starting point for any bot.

[nimble mist]

In these files and their respective event sections 👆

And next time read #rules 6

[sinful sapphire]

manager.broadcastEval((c) => {
  c.user?.setPresence({
    status: "online",
    activities: [
      {
        name: `Online | Shard ${c.shard?.ids}`,
        type: ActivityType.Watching,
      },
    ],
  });
});

is there a way to get the shard ID from broadcast eval?

[terse cairn]

return c.shard.ids[0] in the broadcasteval

[acoustic epoch]

Archived to focus conversation on #986520997006032896 (our shiny forum channel) and kept read-only to facilitate in-app search (CTRL+F) for solutions to problems that have already been asked.
If your issue has not been solved yet, feel free to re-post in the new channel