archive-sharding

[misty cosmos]

should i do sharding although i'm only at 250 servers?

[abstract lichen]

no

maybe if you expect your bot to grow extremely fast.
but I wouldnt look into it until 1k servers, and if you go over 2k you can buy some time with internal sharding

[valid gale]

It really doesn't matter, you're just gonna run with one shard, which is essentially the same as not sharding, you can do what you want really

[terse cairn]

Do I need to edit my whole code to shard?

My bot is reaching 2.5k+ soon

[acoustic epoch]

we have some general information here (and following chapters): https://discordjs.guide/sharding/
the gist is that instead of starting but a single client you start a sharding file, which orchestrates multiple interlinked clients to be instantiated instead

any code that needs to interact with multiple guilds (global counts etc.) will need to retrieve and accumulate that information from the various shards, though (see the broadcast eval section)

[fleet brook]

Official resource: https://discordjs.guide/sharding/
Unofficial resource: https://anidiots.guide/understanding/sharding

[broken sequoia]

Maybe this should go in the channel topic ^

[abstract lichen]

you could also use internal sharding, which requires barely any code changes, but is less scalable in the long run.
afaik you can just specify 'auto' for the ClientOptions#shards property
https://discord.js.org/#/docs/main/stable/typedef/ClientOptions

[acoustic epoch]

Traditional sharding with discord.js
• Guide: https://discordjs.guide/sharding/
• When do i need to shard? At 2,500 guilds
• One process per shard
• Requires communication between processes (broadcast eval is needed to execute code on all shards, if you aren't sure which shard (= process) information can be found on

Internal sharding
• In-process (hence "internal") sharding
• removes the need of inter process communication via broadcast eval calls
• If your bot grows large you might find it necessary to use process based "traditional" sharding instead

// Internally sharding automatically
const Discord = require('discord.js');
const client = new Discord.Client({ shards: 'auto' });

// Interally sharding with specific shards
const Discord = require('discord.js');
const client = new Discord.Client({ shards: [0, 1] });

Mixing both approaches
• If you need X shards per process on N processes

[devout vine]

I run a fairly large bot (18k servers), and am currently using Traditional sharding + worker threads for the shards, so it is all essentially the same root process. I have also created "clusters" where each cluster or root process is handling 16 shards each.

However, I am running into problems with high memory usage, and was thinking about switching to internal sharding to save on memory. Would switching to internal sharding at 16 shards each root process not be recommended? I'm trying to think what downsides there could be to what I'm currently doing.

[wet topaz]

biggest difference between sharding manager and internal sharding is that sharding manager spawns separate processes that are copies of the main code, and each of them handles only part of the guilds at once.
internal sharding however only mantains multiple websocket connections, everything else runs on exactly same process, so while you can theoretically save small amounts of RAM by not having duplicated users between shards (as there is only one cache in this case), but that also comes in with the fact that all of the events are on the same event loop (normally you could not be able to get messages from more than 2500 guilds on one process, but on internal sharding you have all guilds on same process), and while that probably is not going to be a problem with bot of your size, it can probably be one later on.

only point on which you could save memory is by having single cache for users. everything else would be exactly same, since channels, guilds, members etc were completely separate between processes

[devout vine]

I think I see, so I would be basically having more load on that single process

[wet topaz]

yes

[abstract lichen]

you can mix it to have e.g. each process doing internal sharding on 5 or 10 shards

but the memory difference is probably not that big

[wet topaz]

in such case all the profit you could get that i mentioned is probably gonna be non-existent

since you again split the caches

[devout vine]

Even with worker threads they each have separate caches?

Thank you for the help btw, this is all good to know

[devout vine]

I did some quick tests and switching to internal sharding cut my memory usage in half, but CPU usage on 1 core tripled from ~10% to ~30%

[abstract lichen]

I mean you also save yourself the overhead each nodejs process needs and stuff like that

[devout vine]

I've been using worker thread mode, so that should share 1 node.js process I believe

Does worker thread mode automatically spread the load among CPU cores? It seems to be doing so based on what I'm seeing

[royal beacon]

Aha finally it here

[next plaza]

same lol

[wet topaz]

how did you measure that

[devout vine]

htop on debian... I changed my production bot temporarily lol

[wet topaz]

so you just booted the bot and that's that?

[devout vine]

I measured both after a fresh boot, yes

[languid sage]

ah hi wise

[gray meteor]

all my shards stopped working

[sick musk]

.

[lofty plover]

please read the pins

[rotund bridge]

How can you check if there is a bot on a specific server if shards are used?
client.guilds.cache.get("GuildID") - not working...

[royal beacon]

@rotund bridge try to read this first https://discordjs.guide/sharding/

[acoustic epoch]

if you use internal sharding that should work
(see pins)

[royal beacon]

I can sure you can find solution here if not I will help you with some code hint

[rotund bridge]

@royal beacon I studied all the information on this, but I could not find a solution ...

[rotund bridge]

Oh, snx! I try this!

[acoustic epoch]

with traditional sharding: have a more detailed look at broadcasteval - you don't necessarily want to get a functioning Guild structure back though, but instead do whatever you have to do inside the broadcast eval and return data, instead of structures

[royal beacon]

I never heard that we have another way to shards ? can I have more information about it

[acoustic epoch]

updated, can also do a hybrid of both, but need some missing information from mr. internal sharding before i add that

[devout vine]

So it is possible to do a hybrid approach with the current library?

[jaunty valve]

What is sharding?

[strange echo]

see pins

[lusty smelt]

Woa finally. So quick question, how do you restart websocket shards?

[white raptor]

You can respawn a shard which will kill it and spawn it again.

[royal beacon]

some stupid question from me can I using both tranditional and internal sharding together ?

[next plaza]

what does nearly mean?
on shard status

[next plaza]

nope, you cant, but someone is working on it, to make it possible

[next plaza]

internal sharding, client.ws.shards.get(shardid).destroy()

Sharding manager ```js
client.shard.broadcastEval(if(this.shard.ids.includes(shardid) process.exit()))

[lusty smelt]

destroying restarts it?

[next plaza]

yes, just the shard

[hollow gorge]

When the shardDisconnect event is fired, is it safe to spawn another shard or should it just be left alone?

[mild shore]

Leave it alone. shardDisconnect is fired whenever the shard disconnects due to discord asking it to or a network error. Djs will try to resume the session if the error is not fatal.

[mild shore]

https://github.com/discordjs/discord.js/blob/ee5bc1a5c4e218ea1f16992694a2bff86c86d2d2/src/client/websocket/WebSocketShard.js#L269 Only ever referenced here. Seems to be when the connection is open by the client is not 100% ready yet

[mild shore]

No destroying just closes the connection gracefully without restarting it.

[novel mantle]

So just for sharding, we need to add shards: "auto" in client?

[abstract lichen]

for internal sharding, yes

[wanton raft]

Since client.shard.id is gone in v12, what should I do to get the shardID of the specific instance? Or should I just access the guild cache every time to get the shardID value?

[abstract lichen]

isnt there client.shard.ids

[wanton raft]

it's an array of IDs right?

[abstract lichen]

yeah

[wanton raft]

would it be the same value? or does one instance cover multiple shards?

[abstract lichen]

I would assume that it should only contain the ids the actual client handles, but I dont use traditional sharding

[wanton raft]

I guess I'll scuff my way through and just take the first element

[abstract lichen]

and yes a client can cover multiple shards, afaik

[wanton raft]

so what happens when you restart a specific shard

[abstract lichen]

just log it and I would guess its only one value

[wanton raft]

Aight

[abstract lichen]

I dunno, sorry I'm only familiar with the theory, I use internal sharding

[wanton raft]

Aight aight

if anyone has experience with this please explain it to my monke brain thancc

oh nvm it's an array with one element, problem solved

[mild shore]

If you use the sharding manager client.shard.ids will be an array of 1 element that element being the shard that the client is handling.

client.shard is only available if you use the sharding manager if you use internal sharding it will return undefine

[wanton raft]

Yep I do use the manager

So that solved everything,
thanks

[split nimbus]

Anyone have a good sharding tutorial?

[abstract lichen]

the pins should do

if you have extra questions try to answer them yourself or just ask here

[split nimbus]

Ok. Thanks!

[valid gale]

@wispy verge stop shitposting.

[spare cairn]

bruh

[valid gale]

@strange echo help please

[strange echo]

?ban @wispy verge returned to shitpost in support

[icy crow]

You sure you want me to ban this [no gender specified]?

[strange echo]

y

[icy crow]

Successfully banned Lmao#0001

[dapper quiver]

I have a cuestion, how Can i sens a message to a specific channel when its on anther shard?

[next plaza]

one sec, code is coming

let channelid = "the channelid";
let yourmessage = "";///add some thing
client.shard.broadcastEval(`this.channels.cache.has("${channelid}") ? this.channels.cache.get("${channelid}").send("${yourmessage}") : null`).then(console.log);

[dapper quiver]

thx i will try it soon

[green hill]

Does this.users.cache.get(id) in a broadcastEval() return a Discord.js User type or just a plain object without methods like .send()?

[mild shore]

No that is a poor implementation. Construct the channel and send it from there.

const guild = new Discord.Guild(client, { id: "1234567890" })
const channel = new Discord.TextChannel(guild, { id: "1234567890" })

channel.send(embed)```

Doesn't have any broadcast evals and is super fast. Typed on my phone so apologies for errors

[mild shore]

It will return a discord user type

[wet pawn]

Plain object without methods

It wouldn't be of type User

[grand lagoon]

When should one consider sharding?

[royal beacon]

check the pin

[grand lagoon]

thx

[wet topaz]

@wet pawn @green hill that depends where are they "returning" that object out.
inside of the broadcastEval(), it will be proper User object. ```js
broadcastEval('this.users.cache.first().constructor.name')
// [ 'User', 'User', 'User' ]

taken *outside* of the `broadcastEval()` , you are not going to get proper `User` object back. ```js
broadcastEval('this.users.cache.random()').then(results => results.map(obj => obj.constructor.name))
// [ 'Object', 'Object', 'Object' ]

[wet pawn]

Thought they mentioned when it's "outside" of bEval

[wet topaz]

they didn't say anything specific to aything

so since they got two different answers now, clarified

[wet pawn]

Yep

[wet topaz]

@mild shore as well, didn't ping above

[next plaza]

I already knew this Solution, but for a beginner, wouldnt be it better, that they check if there is a channel?

As far as I know, this will send the request directly to the gateway

[mild shore]

Yeah sorry I assume they ment inside. Personally I do logic inside for exactly that reason. Sorry for the confusion!

[mild shore]

Well not necessarily. I don't think that solution is any easier to understand and is far more complicated and uses more memory and CPU. If you prefer it ofc do it but constructing them is more efficient by a mile

[acoustic epoch]

i would very heavily suggest you don't manually instantiate channels, or any discord.js structure, for that matter. They won't work as you expect them to and you may or may not be entering invalid structures into cache which can cause all sorts of unwanted and very hard to debug behaviour further down the line

(and please don't recommend that as superior solution for the sake of efficiency.)

[devout vine]

What I do, and maybe this is not the best option, but I extend the DJS Client class and make a CustomClient, and add extension methods for anything custom I want to do in a broadcastEval(). This way the method is being interfaced, so if the DJS implementation changes, I can still get build errors.

Extended client class:

export class CustomClient extends Client {
    constructor(clientOptions: ClientOptions) {
        super(clientOptions);
    }

    public async setPresence(type: ActivityType, name: string, url: string): Promise<Presence> {
        return await this.user?.setPresence({
            activity: {
                type,
                name,
                url,
            },
        });
    }
}

Calling extended method in broadcastEval:

await this.shardManager.broadcastEval(`
    (async () => {
        return await this.setPresence('${reqBody.type}', '${reqBody.name}', '${reqBody.url}');
    })();
`);

This is typescript, but I'm sure the same thing can be done with JavaScript.

[next plaza]

yeah! my solution won lol

[quaint beacon]

whats the difference between internal sharding and traditional sharding?

[abstract lichen]

traditional sharding spawns multiple child processes (each process taking one shard) and you have to make quite a few code changes (depends on what ur bot does)

internal sharding keeps all the shards in a single nodejs process, you barley need code changes (usually just two lines)

internal sharding is not as scalable, but depending on your bot it can take quite a lot of shards

[prisma dew]

I'm posting here as a last resort... I'm not doing anything with shards myself, but I am encountering an error that involves one.
I'm attempting to join the voice channel of the user executing a command. In this one specific server attempting to do so results in the following error (see attachment). Code is simply as follows:

connections[msg.guild.id] = await msg.member.voice.channel.join();

Why would a guild's shard be undefined?

[acoustic epoch]

@prisma dew not too sure why you think this is sharding related or where this points to a guild's shard. this is very likely related to the new stage channels, which aren't supported in discord.js yet.
if you are still on 12.5.1 please update to 12.5.3, we fixed some things since then to prevent hard crashes.

[prisma dew]

It's not a stage channel. I traced the callstack and it lead back to a line in the VoiceConnection class where it attempts to call send on the guild shard

[acoustic epoch]

why did i look at 183... good question

[prisma dew]

Error occurs on this line of VoiceConnection:

[acoustic epoch]

can you log client.ws.shards.keyArry() as well as the guild's <Guild>.shardID please?

[prisma dew]

I'll log the first one here momentarily

I already know that the guild's shardID is also undefined

[acoustic epoch]

what

[prisma dew]

The keyArry() returns the following array: [ 0 ]

And, confirmed. msg.guild.shardID is undefined

[fair cairn]

is that a typo

[prisma dew]

Is what a typo....

[fair cairn]

keyArray

[prisma dew]

It was, my intellisense autocompleted to keyArray()

[acoustic epoch]

yes, it is, but they found it - so irrelevant

Guild#shardID should very definitely not be undefined, ever

[prisma dew]

I figured not. Guild is not undefined, but the shard, and the shardID are

[acoustic epoch]

@prisma dew which exact version of discord.js are you on, before i do a deep dive into the code base? npm list discord.js

[prisma dew]

12.5.3

[green hill]

Ah alright. Thanks! 👍

[prisma dew]

@acoustic epoch previously I was on 12.5.1, due to your suggestion I updated and tried all the same tests again, with the same results

[acoustic epoch]

yeah, we didn't change anything in that area, 12.5.1 -> 12.5.3 was just stage related things

[green hill]

I'm having some issues with fetching a user across all shards and sending them a DM, getting an error that .send is not a valid method. Am I not getting a type User back?

const fetchUser = async (message, id) => {
  const user = await message.client.shard.broadcastEval(
    `this.users.cache.filter(user => user.id === "${id}")`
  );
  console.log(user); // logs a user object 
  return user[0];
};

[acoustic epoch]

no, broadcastEval returns an array of data results from each shard the eval was executed on.
the returned structures (array) won't be functioning User instances either, but a JSON-sendable version of them (no methods, just the data)
you should do these things in the eval itself, rather than trying to retrieve the user to the outside scope

the base idea is to just return simple data from the b-eval calls and doing things with the structure inside the eval, so it gets executed on the shard directly

[wet topaz]

literally just after you mentioned me with explanation

[green hill]

Ohh alright, thanks will try that!

[acoustic epoch]

@prisma dew msg.client.shard is undefined too?

and do you use any client options in the client constructor?

[prisma dew]

Let me check...

Nothing passed to the client constructor

msg.client.shard appears to be (specifically) null

[mild shore]

Oh really? It seems so much cleaner and I've never had any issues with it. It obviously is way more efficient than fetching the correct channel just to get access to the .send method. What is the reccomended way? ( For the record I don't know how I could be entering invalid data is I just construct the channel with an ID )

[sour wagon]

whats of shard's benefits?

[icy crow]

const { ShardingManager } = require('discord.js');
const Manager = new ShardingManager('./bot.js', { token: 'your-token' });
Manager.spawn();

https://discord.js.org/#/docs/main/stable/class/ShardingManager
For an in-depth guide: https://discordjs.guide/sharding/

[wet topaz]

sharding splits the load between different processes, and is enforced by discord when you reach 2500 guilds

[sour wagon]

is it good thing or ?

[wet topaz]

well, considering it's enforced, yes?

there's no point in sharding before 2000 guilds though

[dapper quiver]

Any one knows how to get total servers in all shards using const

Or

console.log(`SERVERS: ${idk how}`)```

[wet topaz]

Fairly sure that this is explained in the guide

[devout vine]

You can use fetchClientValues(), there is an example in the guide here:
https://discordjs.guide/sharding/#fetchclientvalues

[prisma dew]

@acoustic epoch sorry for pinging you, but were you able to get any more insight into what may have been going on?

[acoustic epoch]

only update i have is that someone else ran into it as well, so it's definitely a thing, no idea what causes a guild to not have a shard yet

[prisma dew]

I see. Thanks for your help nonetheless

[mild shore]

Could someone please explain this btw? ^

[quiet idol]

why it didnt work ?

[wet topaz]

What are you even trying to do

Cause that looks like a wrong way

[valid gale]

I don't see what more you want explained. Souji went into full detail explaining why instantiating djs structures is not something you should do.

[mild shore]

I mean I guess I don't believe him. I've looked pretty thoroughly at the code for channel constructors and u don't see how what he says could happen could happen? By providing just and ID you're not entering data and it twill still have all the same methods as a normal channel? I can't understand why there is a downside to it?

[valid gale]

Because you're passing invalid guild data to the constructor? If you don't know how to properly construct a guild e.g. to mock it in unit testing, there is no reason to use the internal constructors as you will just add invalid structures to the cache as Souji already said.

[mild shore]

So I understand how there could be I valid data but if the structure already exists in cache it won't get overridden? How is it suggested to send a message to a different channel with sharding?

[dapper quiver]

Anyone help me in making shards status like servers, users etc.

[next plaza]

use fetchClientValues and broadcaats eval

[tacit spoke]

How can I get a specific variable from a particular shard, for example get the number of connections to vc from shard 1?

[lusty smelt]

why is this happening when I start my shards? (D.js version 12.5.3)

[next plaza]

client.shard.broadcastEval(`if(this.shard.ids.includes("${shardid}")) getvcconnections()`)

[tacit spoke]

That's because that guild have an stage channel

[lusty smelt]

Well how do I suppress them?

those are really annoying lol

[sudden sequoia]

Hmm, dont the latest version fix those? I thought it did

[next plaza]

yes

[lusty smelt]

Hasnt, they spam all the way along

[dapper quiver]

i made from tutorial and it returns null

[next plaza]

did you updated your package?

[lusty smelt]

^

After the update, the spam increased.

[next plaza]

show me the code

[dapper quiver]

kk

// For each shard, get the shard ID and the number of guilds it owns
let values = await client.shard.broadcastEval(`
    [
        this.shard.id,
        this.guilds.size
    ]
`);
// Make a final string which will be sent in the channel
let finalString = "**SHARD STATUS**\n\n";
// For each shard data
values.forEach((value) => {
    // Add the shard infos to the final string
    finalString += "• SHARD #"+value[0]+" | ServerCount: "+value[1]+"\n";
});
// Send the final string in the channel
message.channel.send(finalString);

[next plaza]

are they different guilds?

[valid gale]

Well the shard that holds that one guild is offline meanwhile the shard that holds the other one isn't

[wet topaz]

@mild shore do it in broadcastEval, by getting the channel. One of the shards has it

[mild shore]

How would you implement a command to send to all guilds? Seems super heavy weight when really all you are doing is sending a message?

[valid gale]

You just grab the guild/channel and send the message as tip said above?

[mild shore]

So if I sent to 20k channels I have to broadcast eval 20k times?!?!?!

[marsh finch]

No

[mild shore]

I've never had a problem with just constructing the channel and the only alternative methods I can think of all require super hacky ( and useless ) ways of doing things? One alternative is just to send a straight request?

[marsh finch]

You could let Discord do all this work by using a news channel.

[mild shore]

Needs to be custom messages + mentioning

[valid gale]

space's point still stands

[mild shore]

How? he didn't even make a point? News channels won't work for my use case

[marsh finch]

What is your use case?

[valid gale]

You could just be less obnoxious. We're trying to help you.

[mild shore]

Didn't mean to be obnoxious apologies just don't understand why mu current method is incorrect. My use is alerting users when a virtual item hits a certain price. Messages need to mention custom roles + have custom currencies

[wet topaz]

Nothing stops you from having a central guild that has channels for that. If someone is interested, they will most likely join that guild

[mild shore]

So I did think about having 1 news channel for each currency but I have like 30 odd different currencies so each requires a separate channel. Is an option though but still doesn't solve the mentioning problem

People really like @ here and custom roles for certain mentions

Wish discord would support mention propagation insdead of having @craggy bluff-role ( sorry for ping whoops ) even still doesn't support custom roles for channels

[marsh finch]

I'm not sure why you wouldn't keep your pre-sharding approach.

[mild shore]

Explain sorry?

[marsh finch]

Just send to all appropriate channels available on the current client, do this on all shards and you got all channels covered. No broadcasting necessary.

[mild shore]

Well I did think of that option but it fucks rate limits. Sometimes if things break I can get destroyed by the rate limits and if I have each shard sending the messages it gets fucked over even more

[marsh finch]

Oh, the global one? That'd make sense, as currently the clients don't synchronize anything rate limit-wise.
You could try looking into webhooks then, as those are not tied to your bot/application.

[mild shore]

Yeah webhooks were another option but it presents the problem that I never had the create webhook permission when I started ( and still don't ) so chances are most guilds won't allow me to create them

So I would need to go through all guilds and ask them to enable create webhook perms :/ kinda a pain and should have thought about it when I started but I was a noob + I had no idea it would become this popular

That would make lots of things simpler though but I dread having to convert everyone 😦

[next plaza]

20000 messages in a second @mild shore ?

you want to send this amount, so fast?
whats the maximal time all messages should be sent?

[next plaza]

you are using discord.js-light
then just do client.channels.forge(chanellid).send(message) (⚠️JUST DISCORD.JS-LIGHT)
this does not require any broadcastEVal and sends the request directly to the gateway

alternativly you could use client.api.get......

[next plaza]

thats a good solution, then you have to save the webhook id and the webhook token in your db

[fathom olive]

Yes

[fathom olive]

yes

[fathom olive]

@valid gale

[stoic tartan]

@fathom olive you are arabic bro and me is arabic

[acoustic epoch]

#archive-offtopic for handshakes and brofists please

[proven iris]

Greetings! It's very funny, but my bot has disconnected on several servers, but I get information about all shards, and they are all online) is it possible to somehow synchronize the bot with these servers without restarting the rest?

https://i.imgur.com/Ako2F21.png

[proud mortar]

Shard#respawn()
Kills and restarts the shard's process/worker.

[terse cairn]

do u also need to do the broadcastEval for the VOiceStateUpdate event?

[marsh finch]

I don't see how these two are related.
I'm going with "no".

What do you mean with "sharding process"?

[terse cairn]

but without it it wont work

if someone joins a channel in shard 3 i wanna join with the bot there not work

[marsh finch]

You need to elaborate on the "not work" part.

[marsh finch]

I don't understand, discord.js does not prevent you from exiting the process using process.exit.

[astral gale]

I have a problem, from time to time the shards seem to come out of the ShardManager, which has the effect of creating double shards, i.e. duplicate commands. It was rare, but now it happens quite often. If someone has already had a problem like this, please contact me, it would be really nice of you.

[marsh finch]

Yes, that's correct.

You can use Shard#kill from within the manager process to not have the child process restart.
https://discord.js.org/#/docs/main/stable/class/Shard?scrollTo=kill

Client#shard is not a Shard.

You need to send a message to the manager process (using ShardClientUtil#send, that's what Client#shard).
This message needs to be received and handled by you from within the manager process using the Shard#messages event.
There you can then kill the shard.

[devout vine]

What kind of sharding are you using? Internal? Traditional + process shards? Traditional + worker thread shards?

If you use worker threads you could potentially kill the whole bot at once

?docs ShardingManagerMode

[devout vine]

When you create the ShardingManger, one of the options is a "mode", you can specify "worker" if you'd like to use worker threads. I believe it defaults to separate processes

Then when you do process.exit, it should kill the worker threads as well I believe

That's one option, if you want to keep using traditional sharding with processes, I think you'll have to kill each shard separately

[valid gale]

Then follow the steps that space laid out for you

[marsh finch]

If you just want to restart the process, you already got that going, since d.js does that automatically.

[astral gale]

no one ? I can't run my bot this make double commande quickly and i don't know from where it come

[devout vine]

How are you spawning shards? Do you use ShardingManager.shardList?

Double commands can happen if you have 2 instances of the same shard ID running at the same time

[astral gale]

I use sahrdmanager.spawn()

[rotund bridge]

How can you check if there is a bot on a specific server if shards are used?

[fathom olive]

How do I use the load balancer in my bot Do you have a tutorial for Discord.js library?

[winter garnet]

client.shard.fetchClientValues()

[rotund bridge]

It's doesn't work

[winter garnet]

Can you show the full code and console output?

[rotund bridge]

This method only displays the number of users, servers, etc.

[tall spade]

Can someone explain to me the difference between ShardingManager and pm2

[valid gale]

The sharding manager manages the shards/subprocesses of the sharded bot, and pm2 is a node process manager

[tall spade]

can i run my bot from multi host like 100 server in one host and 100 in other host ?

[abstract lichen]

not with raw discord.js, you would have to implement the actual shard balancing yourself (you can tell each client what shards it should run on)

and honestly why do that at such a small scale

[tall spade]

i have a hosting problem . my server have 16core and 32gb but the bot so lagy
in 2900 guilds

[abstract lichen]

what the fuck

what is your bot doing

[tall spade]

muisc bot

[abstract lichen]

Maybe look into lavalink.
My bot has ~5k guilds and my 4 core 16GB server barley cares

oh and does it actually use all the resources?

if it lags in voice channels it's probably not going to be solved by having more hosts with less shards running on

[terse cairn]

Is creating clusters part of sharding?

[tiny grove]

How do I check total number of shards with internal sharding?

[fierce compass]

client.shard.count

[tiny grove]

"internal sharding" client.shard will be null

[abstract lichen]

client.ws.shards.size

[terse cairn]

Checkout this guide on sharding: https://discordjs.guide/sharding/

[lunar parcel]

so i've been starting to have shard disconnect and refuse to reconnect automatically. i'm at 12 shards right now. do i need to start looking into clustering?

[young totem]

https://github.com/discordjs/discord.js/pull/5222 This PR says that I should be able to broadcasteval to a specific shard while the bot is booting, yet it still throws an error when I do that.

[terse cairn]

When i run this file with node shard.js i get no error. I'm not exactly sure whats wrong other then that it is in the top.gg api area. If someone could help that would be awesome

[terse cairn]

Nvm missed a ;

[young totem]

Still need help w this

[buoyant oracle]

can i shard without 2000 servers

[devout vine]

Yes

It's required when you hit 2,500 servers, but you can start it any time before that

[buoyant oracle]

just for testing ?

[buoyant oracle]

but will it harm any of my things in my project

cause i don't want my rpoject to get glicthed or destroy something like that ;-;

[ocean mesa]

no, it wont harm anything, just follow the guide

[torn plover]

I sharded when my bot was on 20 servers, for testing

[buoyant oracle]

ohh nice

[torn plover]

Just read the docs as it'll tell you why sharding can be both good and bad, sharding is good for multiple servers as it splits to many clusters which have different pings. Although, this does drag down the bot and make it slower slightly, but is beneficial for 2k + bots

[wet topaz]

Slower?

🤔

[rotund bridge]

How can you check if there is a bot on a specific server if shards are used? Pls, help

[wet topaz]

what are you trying to do?

[rotund bridge]

client.guilds.cache.get();

[wet topaz]

im asking more generally

what do you want from that guild

[rotund bridge]

Check if there is a bot on this server

[wet topaz]

use client.shard.broadcastEval()

https://discordjs.guide/sharding/#broadcasteval

[young totem]

Can someone help with this?

[wet topaz]

can you provide any more info?

[young totem]

Turns out the error was something else, disregard my issue :)

[buoyant oracle]

does sharding have many functions ? i mean how much data can we fetch using vars with sharding ..

i mean should i need to learn all that variables for getting all that data ?

[wet topaz]

you cannot take out full objects out of the broadcastEval

this should explain <#archive-sharding message>

[finite vale]

Anybody know little about this?

Reinstall the OS of the vps, install the dependencies and when you want to start it I get this

[abstract lichen]

I would guess one of your shards errors and then the sharding manager complains because it isnt ready (because it threw an error)

[sacred sparrow]

Can shard X fetch a channel from shard Y, and send a message to it, without going through broadcastEval?

[finite vale]

How I could fix it

[abstract lichen]

"error: Cannot load default config file"
is maybe the error?

[finite vale]

Maybe, but I don't know what that mistake might be, it doesn't say where it comes from

[remote frigate]

how do I make a slash command ephemeral

[strange echo]

that is not a question related to sharding, nor does the library support interactions (yet), please see #useful-servers for an invite to dapi

[remote frigate]

is there a ETA?

[lime sluice]

[finite vale]

Hi, what is this error?

Channel closed

[unborn cairn]

I have:

const manager = new ShardingManager('./src/bot.js', {
  token: Config.Bot.token,
});
manager.on('shardCreate', (shard) => console.log(`Launched shard ${shard.id}`));
manager.spawn();

After about a day, my 4th shard normally goes down without any type of error surfacing. I'm beginning to honestly think it's a RAM issue. But, my RAM only sits at around 62%. Any thoughts?

process.on('uncaughtException', (error) => {
  console.error(error);
});

process.on('unhandledRejection', (error) => {
  console.error(error);
});

client.login(Config.Bot.token);

Nothing is showcased from the client itself

[devout vine]

Do broadcastEval(), fetchClientValue(), etc have a timeout? I'm running into a weird scenario, where (what I think is going on) one of my shards is unresponsive, which causes broadcastEval(), fetchClientValue(), etc to not return and seem to just be waiting for a response forever.

[small kayak]

it may have something to do with your bot loading before the shard manager. This wan an error with my bot, so just throw an async / await func and it should fix the problem.

[dawn moss]

Does not make it slower, infact it makes it faster, BUT sharding does increase memory usage

[dapper ibex]

why am i getting Error [SHARDING_READY_TIMEOUT]: Shard 0's Client took too long to become ready. everytime i start the bot?

[crude crypt]

Do you have the fetchAllMembers option enabled?

[dapper ibex]

no

[crude crypt]

Default timeouts for everything?

[dapper ibex]

yes

could it be caused by a slow connection?

[crude crypt]

It could be

I'd recommend logging the debug event and seeing what's going on

[dapper ibex]

aight

[dapper ibex]

when i pass an array through broadcastEval only the first element is carried over, how to pass the entirety of the array?

nvm, i passed it on the JSON.stringify function first to solve it

[sacred sparrow]

Wy are await Promise.all(...) not allowed in a broadcastEval script ?

[crude crypt]

They are?

[crude crypt]

Oh yeah, its not wrapped in an async function, you'll need to wrap it yourself

[sacred sparrow]

(i am in an async function ofc)

[abstract lichen]

in the eval context probably not

[sacred sparrow]

uhh, i never did that, do you have any examples ?

[crude crypt]

(async () => { /* my async code */ })()

[sacred sparrow]

that's what i did

[abstract lichen]

in the eval?
just show your code smh

[abstract lichen]

hmmm

[sacred sparrow]

no ideas then ?

[crude crypt]

async => { is not a valid arrow func

[sacred sparrow]

oh

indeed, thank you

[crude crypt]

🙇

[terse cairn]

Hello, how can I wait until all shards are loaded?

[sacred sparrow]

What are you trying to do ?

[terse cairn]

<client>.fetchClientValues gives error when all shards aren't loaded

Error: [SHARDING_IN_PROGRESS]

[sacred sparrow]

I personally do const res = <client>.shard.fetchClientValues('something').catch(() => undefined);,
and then if (res.filter(r => r === undefined).length != 0) return message.reply('Not all shards ready');

it basically returns undefined if a shard is not ready, and then checks if any shard returned undefined

[terse cairn]

thx, in your case i'd some changed your code from .catch(() => undefined) to .catch(() => {}) cuz func by default returns undefined, but i think that you're way is easier to understand for you

[split nimbus]

Can someone help with my issue? https://github.com/Heptagram-Bot/Heptagram/issues/74

[dense cape]

?shardreadytimeout

[icy crow]

Got this error: Error [SHARDING_READY_TIMEOUT]: Shard 0's Client took too long to become ready.?
The spawnTimeout can be set to -1 or Infinity to prevent future errors:
https://discord.js.org/#/docs/main/stable/class/ShardingManager?scrollTo=spawn. This can also happen if you call ShardingManager#spawn before attaching a listener to the shardCreate event which could create a race condition possibly preventing the shard 0 to perform a successful launch.

[dense cape]

@split nimbus

[lone light]

ups

[buoyant oracle]

i tried sharding but its not spawing any shards

why ? my bot is in 1 server that's why ?

it should spawn 1 shard atleast then ?🤔

should i add my bot in more server so that it will shard

[plain sapphire]

This really should not be something to worry about until your client is in 2,000 guilds.

[distant acorn]

@buoyant oracle You dont need to shard till atleast 2000

[buoyant oracle]

but i wanted to test

at 20 servers it won't shard ?

[valid gale]

unnecessary. even when you reach 2K guilds you can just turn on internal sharding til you reach 20k, so no need to test anything at 20 guilds

[buoyant oracle]

oh

[iron solstice]

is there a reason broadcastEval takes a string instead of a function? it seems much more intuitive to pass a function directly instead of evaling from a string, isn't it?

[valid gale]

you can do that, as outlined in the sharding section of the guide

[abstract lichen]

you can force it to shard with totalShards

[devout vine]

You can pass an actual function into broadcastEval? I thought it has to be a string?

[abstract lichen]

it has to be a string, but

http://images.chilo.space/Pk5g

[crude crypt]

It can be a function -- if it is, we'll stringify the function and wrap it in an iife

[jaunty valve]

What is sharding, and why do I need it?

[devout vine]

Interesting, had no idea that function.toString() was a thing

[abstract lichen]

.toString exists for all objects in javascript ;)
(but sometimes behave different)

[abstract lichen]

see pinned messages.
as soon as you hit 2.5k Guilds you need sharding, should probably look into it before that

[valid gale]

No. You can pass an actual function.

[abstract lichen]

probably gets converted to a string somewhere, thats what I meant with that

script = typeof script === 'function' ? `(${script})(this)` : script;

https://github.com/discordjs/discord.js/blob/stable/src/sharding/ShardClientUtil.js#L139
guess d.js can handle functions, converts them to strings and calls them

@devout vine fyi

[valid gale]

You have to wrap it in template strings, but the function itself doesn't have to be a string, again this is all outlined in the sharding guide.

client.shard.broadcastEval(`(${funcName})('${arg}')`);

[oak glade]

how do I display the total guild count in the status?

I'm having problems with promises

[jaunty valve]

This doesn't answer my question of why sharding is necessary

[abstract lichen]

load balancing

[jaunty valve]

what is that?

[abstract lichen]

your balance your load.
imagine you have millions of guilds, you dont run that off a single server, you spread the load among many servers.
each shard handles x (reccomended is 2k - 2.5) guilds and each shard has its own websocket connection.

for more questions:
https://discord.com/developers/docs/topics/gateway#sharding
or the discord api server

[jaunty valve]

Ohhh ok, ty

[strange echo]

?warn @rugged sentinel please do not post invites here, especially not in support channels

[icy crow]

Successfully warned misheutter#6147

[hidden rune]

Hello please help. I have a ratelimit i how to fix this?

i waiting for 1 hours

[craggy shoal]

?shardreadytimeout

[icy crow]

Got this error: Error [SHARDING_READY_TIMEOUT]: Shard 0's Client took too long to become ready.?
The spawnTimeout can be set to -1 or Infinity to prevent future errors:
https://discord.js.org/#/docs/main/stable/class/ShardingManager?scrollTo=spawn. This can also happen if you call ShardingManager#spawn before attaching a listener to the shardCreate event which could create a race condition possibly preventing the shard 0 to perform a successful launch.

[craggy shoal]

and for the ratelimit, well you just have to wait for now until you can send requests again

[terse cairn]

how many shards i need for 3.5k servers?

[terse cairn]

Generally speaking, shards will have around 1000-1500 servers, so I'd say 3, but if you use the default settings, the recommendend amount will be used

ok 👍

[buoyant oracle]

ik i did it as 15 shards

[valid gale]

each shard can hold 2.5k guilds.

[terse cairn]

Of course, but that's not relevant here.

[floral root]

Is there anyway to test sharding, I mean like can i make a sharding with a bot in 3 guilds or so, I'm just eager to know how the sharding would be, so its possible?

[abstract lichen]

yes, just set the shardCount to whatever you want

for internal sharding its shardCount, with the shardingmanager its totalShards

[terse cairn]

sup, my bot is in 1800 servers and growing, should i start using shards?

[wet topaz]

most likely

[terse cairn]

so it's running for example index.js for every shard?

[wet topaz]

sharding with sharding manager runs copies of your code for every shard you are running

[terse cairn]

pain

[abstract lichen]

another alternative is internal sharding, you need to change 1-3 lines of code that's it

not as scalable, but depending on what your bot does, it can probably take quite some shards (I currently run 5k guilds with internal sharding but I would guess 20k or more is doable)

Check the pins

@terse cairn bump

[terse cairn]

gonna use normal shards anyway probably

[abstract lichen]

alright, just wanted to let you know of the alternative. In the worst case scenario you can buy yourself some time with internal sharding

[golden wasp]

I let a bot that makes miencraft kill, thanks to @terse cairn

[half flint]

no clue why you pinged me, but I have no clue what you're talking about nor have i helped you with anything

[golden wasp]

sorry I missed with the tags :v

[acoustic epoch]

please keep this category to questions and problems, feel free to share accomplishments in #archive-offtopic though

[upbeat island]

i think you must use javascript let user = message.channel.mentions.users.first();

[edgy edge]

no

[upbeat island]

?

[edgy edge]

Not sure why you even answered to that, since they didn't ask for help, but your answer is also wrong.
But this channel is no place to discuss that.

[chrome kindle]

Hello, I have a question around sharding that i'm not able to really find anywhere. So it looks like traditional sharding is the most recommended for busy bots. However it also appears that traditional sharding simply separates the incoming messages over multiple processes on a single server.

How would traditional sharding work for say 10-20 ECS instances within a VPC?

[wet topaz]

traditional sharding on its own doesn't really care about much. all it does is spawn n instances of same code, and provides some communication between shards it runs

we don't really have any built-in way for supporting multiple sharding manager processes, but nothing stops you from having a separate api, service, redis or whatever you want to have some inter communication

though indeed, you can spawn only some shards on one sharding manager, and other shards on other one, sharding manager can take a list of shards to spawn afterall

[chrome kindle]

So am i right in thinking – the purpose of sharding is not to directly split guilds, but instead split the load of requests over multiple processes. And so all shards will need access to the same cache in order to be able to fulfil a request?

I suppose secondly to that, my views on scalability are auto-scaling within AWS and it can scale based on CPU.

The alternative could be to have a single service that pushes the incoming requests into an SQS queue to be consumed and handled via the Discord API.

So neither of those can be achieved using Gateway, which unless i'm mistaken is the only method available to me?

[wet topaz]

with sharding manager there is no shared cache

each process is just a copy of your main file

only thing that is "shared" is way to communicate between the shards, mainly with broadcastEval or fetchclientValues

and you can't really change shard count on the fly that easily

[chrome kindle]

Ah ok. so it basically sounds like it would be impossible for me to simply run my bot on multiple ECS instances.

[wet topaz]

well, not exactly impossible, but definitely not supported out of the box

you just can't really have it auto scale

you can have some shards on one instance, and other on second instance

but that's that

[chrome kindle]

That was my worry 😦 then i suppose realistically how does bots like mee6 run?

[wet topaz]

by doing what i suggested

they have ways to communicate between servers, most likely with some form of internal api

[chrome kindle]

I guess i'm cautious of a single server, how big would it need to be? how much ram/cpu will i need? what happens when i hit a physical limit

[wet topaz]

depends on the server and what the bot does

[chrome kindle]

This may be my lack of understanding, but I can't see any physical way around scaling other than throwing more physical resource at something and load-balancing it. So in say software engineering one would simply set desireCount and run the same service and either consume from a message queue or the load-balancer would feed requests

[wet topaz]

yeah, but this isn't really possible from discord api standpoint

[chrome kindle]

I would expect something similar for the bot, just i don't think Discord support that.

[wet topaz]

api in general needs your shard count when you identify

and you can't scale that out that easily without restarting

[chrome kindle]

I see, so i'm struggling to see the alternative scaling method is all. I would rather auto-scale via AWS as that's more cost effective, scale when it needs and only pay for the resource i need

[wet topaz]

+large bots have certain specific shard counts they have to follow

[chrome kindle]

Ah okay, that at least gives me something to think about.

[wet topaz]

unless your bot is extremely large or resource intensive, there is no point in auto-scaling it much

if your bot does something intensive, like image processing for example, you could think about separating that part out to its own server

then you don't have to worry that much about bot's main server running out of cpu

[chrome kindle]

Oh i've got you. yeah i guess the sharded instance could be a single instance with multiple processes that pushes into SQS

then i can handle those from consumers and that way it frees cpu

consumers can scale based on SQS message number/size

[wet topaz]

could also use a separate gateway that pushes things to some queue, but this isn't really discord.js anymore at this point (d.js doesn't have separate gateway, and that won't come to it for quite a while)

[chrome kindle]

yeah, that's true. I appreciate this info. I mean discojs is fine for my current bot and the bits im doing. It's my next bot/project... this has been very helpful

[wet topaz]

[green hill]

Is it possible to “break out” of bEval whenever i.e. a user has been found?

[terse cairn]

You should avoid to search for a user in a bEval in the first place I guess, what is it you'd like exactly? Getting a user is just client.users.fetch for an arbitrary shard.
When getting member info, you could only fetch the member in de bEval after a guild.id check from example

[green hill]

I’m fetching a user from cache and trying to DM them

[terse cairn]

You can just client.users.fetch(id) and the user.send from any shard you'd like, no bEval required
Note that his response will always be received on shard ID 0

[green hill]

How does d.js handle fetching? Does it check the cache across shards first before actually making a request to Discord?

[terse cairn]

No it only checks the cache on its own shard before fetching

But checking the cache on all shards sounds very inefficient

[green hill]

I see, I’m just trying to keep requests to Discord to a minimum. I’ll try your suggestion out, thanks!

[wary roost]

hello! i have bot written in TypeScript & discord.js, i tried sharding so my bot can handle more than 2500 servers but i have some issues with dashboard. Dashboard runs normally because i made it starts with shardingmanager not with shard but my dashboard fetches data from client and here problem begins.

Luke.guilds.cache.get((guild.g.id as string))?.channels.cache || []

i tried to rewrite this but how? Actually

await Luke.shard?.broadcastEval(`this.guilds.cache.get(${guild.g.id})`)

doesn't work

Dashboard router without sharding: https://github.com/Nimplex/Luke/blob/main/src/dashboard/router.ts

[wary roost]

it's running on 1 shard

but it sent two messages

[green hill]

What exactly does "run normally" imply? Also I'm not sure you need to figure out sharding just yet

[wary roost]

I'll need later, run normally means execute command one time instead of 2-4 (or trigger event)

[wet topaz]

you cannot return full structures outside of broadcastEval

[wary roost]

that's not problem anymore

the problem is thst bot is responding multiple times

[blazing lintel]

Hello, could someone explain to me what the shards are for please? (Mp)

[terse cairn]

When your bot reaches 2500 servers, discord cant send all the data using 1 WS connection anymore, so you distribute the server over multiple shards

[wary roost]

https://discord.com/developers/docs/topics/gateway#sharding

[coarse arch]

is there a way to fetch the guild cache collection object from every shard? I've tried everything with both broadcastEval and fetchClientValues

[hollow gorge]

If you have access to ShardingManager/ShardClientUtil, you can use <Sharder>.broadcastEval() to broadcast a script to all of the shards.

If you wish to kill all of your shards, might as well shut down the whole bot.

It takes a normal line of code, just in a string.

But yes, process.kill() will shutdown that process.

[proud mortar]

ShardClientUtil#broadcastEval()
Evaluates a script or function on all shards, or a given shard, in the context of the {@link Client}s.

[hollow gorge]

Forgot broadcastEval() has two arguments. You can provide the code string and a shard number.

shardDisconnect is fired when the websocket for the shard dies when the shard is in the ready state. So no, it won't fire when the shard itself is forceably killed.

It would be on the Shard itself.

[proud mortar]

(event) Shard#death
Emitted upon the shard's child process/worker exiting.

[hollow gorge]

The event itself is called death, and it is on the Shard.

[trail frost]

what'S the maximum amount of guilds each shard?

[abstract lichen]

2500

[wet topaz]

but note that if you pass it, your bot will not be able to log in

[valid gale]

You will not receive help with snipe commands, also this channel is for issues related to the sharding ecosystem

[wary roost]

how to get guild using shardingmanager?

[terse cairn]

You don't actually, you cannot share D.JS structures between shards

[wary roost]

then how to do this with multiple shards? client.guilds.cache.get(guild.g.id)

[terse cairn]

What is it youd like to do with the given guild?

[wary roost]

fetch data.
name, channels, id, owner, (bot member)

[terse cairn]

Okay so that is information you can retrieve

Like broadcastEval const g = this.guilds.cache.get('id'); g && ({ name: g.name, id: g.id })

[wary roost]

mhm i'll ty

[terse cairn]

Actually, I would broadcastEval this.guilds.cache.get('id')?.toJSON()

[terse cairn]

static void UpdatePresence()
{
    DiscordRichPresence discordPresence;
    memset(&discordPresence, 0, sizeof(discordPresence));
    discordPresence.details = "setting up";
    discordPresence.largeImageText = "Numbani";
    discordPresence.smallImageText = "Rogue - Level 100";
    discordPresence.partyId = "fff";
    Discord_UpdatePresence(&discordPresence);
}

[abstract lichen]

pretty sure that isnt javascript (nor typescript)

[terse cairn]

it is my rp sinko

[abstract lichen]

damn thats crazy, but why are you sending it into this channel again?

[terse cairn]

because no one answered on help xd

[dim canyon]

wrong language genius

[terse cairn]

what lang

[dim canyon]

this is not a c#(?) server

[abstract lichen]

this is a server for a javascript/typescript library, so unless you dont use one of those, wrong server

[terse cairn]

it is js

[dim canyon]

no its not

its c#

[terse cairn]

idk c#

[abstract lichen]

either way, its not any language that we support here, please stop, thank you

[terse cairn]

lk py,html,java and some js

[valid gale]

@terse cairn keep offtopic discussions out of the support channels.

[terse cairn]

Shard 1's Client took too long to become ready.

Any idea why this happens?

[devout vine]

The shards client took too long to become ready lol

You can increase the time the ShardingManager waits for the client to spawn with the spawnTimeout option

?docs ShardingManager#spawn

[golden moth]

Hi, when testing sharding how do I make a bot with 2 servers and property 'totalShards: 2' assign each server to a different shard?
Right now my sharding manager spawns two shards but but both servers are on shard 1 and shard 0 is empty 🤔

[minor nova]

Alright. Me and 3 other people are all hosting the same bot on our virtual private servers. We used the built in sharding manager and all took several shards.

Recently I had to restart my vps and just everything broke. With the same code and everything, my vps takes all shards and when restarting all hosts, the two other hosts both take shard 0 and mine all the shards again.

Does anyone have a clue what could cause that?

[idle saffron]

bot goes offline right after it is online
and i ain't sharding soo idk?

[abstract lichen]

and thats related to sharding because?

[terse cairn]

I know, but the error continues without reason

Shard 1 refuses to start

[terse cairn]

Could anyone help me? I have had the same problem for a while and I can't find any errors.

[devout vine]

Shard 1 or Shard 0 (the first shard?)

And is it always the same shard? Do other shards start up normally?

[terse cairn]

Yes, the other shards start normal.

But within minutes they go off for no reason.

[late drum]

how do i check the ram usage of a shard

[strange echo]

specify if you're using internal or traditional sharding, please

[late drum]

using const { ShardingManager } = require("discord.js"); assuming that means traditional

[fierce compass]

https://discordjs.guide/sharding/additional-information.html#specific-shards

but instead of exiting the process get the ram usage

[late drum]

tnx

[devout vine]

Still running into this weird issue, but now have a little more info since I've added some debug logging.

I'm using Traditional sharding + worker thread mode (instead of process mode).

It appears that after my bot has been running awhile, one of the shard's clients (Shard 17 in my last case) becomes unresponsive. I get reports from some servers where the bot appears offline for them. I have added logging and can see that the specific client that fails no longer receives the broadcastEval command, which I've set to console.log() temporarily. I get no log from Shard 17. All other shards do log this event. And the ShardingManager's broadcastEval() never returns. It seems to be stuck waiting for that broken client to respond.

Is there any additional info I could log to see what's going on with the shard's client? fetchClientValue, broadcastEval, etc calls to the shard will run forever seemingly

Or any ideas what might be causing this?

[blazing crescent]

Quick question about sharding. When it comes time to need to spawn a new shard, do I have to restart my bot for it to spawn or will djs do it automatically?

[strange echo]

There's no need to spawn new shards while running, you can go over the 2500 guilds per shard during run time. If you set shards to auto, it will make the correct amount next time you start

[minor nova]

Alright so I am sharding my bot. It is in about 8.5k servers yet the sharding manager is only spawning 1 shard. Is there a reason why that could happen?

[terse cairn]

At how many servers does a bot need to start sharding?

[terse cairn]

thanks

[valid gale]

@terse cairn please read the pinned messages next time.

[strange sky]

I still get this error while using broadcastEval(), which was said to resolve this over fetchClientVaules() at ready event.

[sour pendant]

it connects does not respond to the event, the status is not set since this afternoon
It's really weird and I think it's because of a server that it doesn't work anymore
discord.js version : 12.5.3

[sour pendant]

and have this

@pearl cove

[pearl cove]

sorry, but I have never used sharding, so I don't think I can help you much

[broken sequoia]

What version of djs are you on?

[sour pendant]

12.5.3

[broken sequoia]

When did the issue start?

[sour pendant]

today

17h france

[broken sequoia]

If you’ve sharped before without problems it’s probably just discord

[sour pendant]

but my bot is not working anymore because of this and I didn't change my code it just happened all of a sudden

[abstract lichen]

that error is because of stage channels iirc

I have that too, besides those warnings I dont have problems with it

[pliant tiger]

Stage channels make fetchInvite bug

[valid gale]

@pliant tiger this channel is for issues related to the sharding functionality of the lib.

[pliant tiger]

Ok 😅 where is the bug report channel ?

[terse cairn]

on github

[austere osprey]

Hi all, sometimes my bot double-posts when responding to commands. It seems this is due to the sharding. After the bot has been online for a while, stopping the shard manager process leaves some instances of the bot running, which indicates that some extras are being spawned somehow. Is there a solution for this?

In the meantime I've had to use cron to stop the shard manager process, kill all extra instances and restart the shard manager

Please help me out here, I've been recieving tons of bug reports for the past month and I've got no idea how to resolve this 😦

[abstract lichen]

Isn't this a regression? I remember a ticket that had the same problem... shards running after the parent dies

I don't know and can't really look rn, an Idea would have is to use client.shard.send and something in the master process to have a heartbeat, and as soon as those don't arrive anymore you can assume the parent process is gone

Assuming this is really an issue with orphan processes

[austere osprey]

Isn't this a regression?
It is a regression, but the reason my bot didn't exhbitit this behaviour before is because it was on DJS 11. This happened after porting it to 12. The sharding code is very simple though.

Assuming this is really an issue with orphan processes
Fairly certain it is. If I run pm2 stop bot to kill the shard manager, then run htop I can still see the orphaned processes running

..master process to have a heartbeat
This heartbeat idea is interesting, I'll look into it. The double posting is driving me crazy

[abstract lichen]

Nevermind, seems like it was never merged, so not a regression

https://github.com/discordjs/discord.js/pull/4153

You would have to experiment but you can probably simplify this by checking if the IPC channel is not disconnected yet

but the heartbeat approach would be more reliable

So yeah, try the following:

[proud mortar]

ShardingManager#shards
A collection of shards that this manager has spawned

[abstract lichen]

iterate through that and regularly send a heartbeat to shards, shards should be able to check with

[proud mortar]

ShardClientUtil#parentPort
Message port for the master process (only when {@link ShardClientUtil#mode} is worker)

[abstract lichen]

should have a message event, I don't know how the messages will look, you just need to try it out

a simpler alternative would be (if this doesn't work) to check when the parentPort closes (it has a close event and various other events), in theory it should close when the parent dies, but I don't know if this is actually true. Just test a bit around

@austere osprey bump

prolly should not test in production lol

[hollow gorge]

I actually made a PR about this but totally forgot to update it. I'll open up a new PR that fixes ghost processes.

[abstract lichen]

||isn't it orphan tho||

[hollow gorge]

well, yes, wasn't thinking

[craggy tundra]

What would be the proper method of getting client.commands

[hollow gorge]

No idea, as it isn't a djs problem.

[craggy tundra]

it has to do with sharding...

if I do a regular eval, it returns it correctly

[craggy tundra]

fixed, doing .array() fixed it

[austere osprey]

Thank you ❤️

[idle saffron]

bot goes offline right after it is online
and i ain't sharding soo idk?
even if its sharding how do i know?

[umbral cradle]

Does bot.on("guildCreate", guild => { }) work always for all shards?
or only works in 1 shard?

[wet topaz]

A guild can only be on one shard at the time

[umbral cradle]

I mean, i just want a log, that will notify me whenever my bot joins any server in any shard.

[wet topaz]

Ok?

That doesn't change the answer

@idle saffron #archive-sharding message

[umbral cradle]

Sorry, i think you didnt get my question, thats my fault coz i couldnt describe the question properly.

Will this event emit whenever my bot joins any server in any shard?

[wet topaz]

are you using sharding manager?

[umbral cradle]

bot.on("guildCreate", guild => {

  const added = new MessageEmbed()
    .setColor(color)
    .setTitle(`${guild.name} - ${guild.id}`)
    .addField("Guild Members", guild.memberCount)
    .setImage(guild.icon)
    .setThumbnail(
      "https://cdn.discordapp.com/emojis/467569397981118495.png?v=1"
    );


  const channelId = "692711072444055623";

  bot.shard.broadcastEval(`const channel = this.channels.cache.get("${channelId}"); if (channel) channel.send(added)`);

    })```

[wet topaz]

yeah, you are

sharding manager runs multiple copies of the main file

[umbral cradle]

And, wanted to know, if this is how the broadcastEval() works

[wet topaz]

each copy is one shard

[umbral cradle]

script parameter as a string

[wet topaz]

should work, apart from the fact that you don't have added as a string

or as a variable

[umbral cradle]

added is my embed

[wet topaz]

no, it's a string

[umbral cradle]

so i have to use it as channel.send("added") ?

[wet topaz]

that will send added as a string, yes

but as you can even see in the codeblock itself, it's just some letters in the string

[umbral cradle]

I dont want to be sent as a string, added is defined as my embed

[wet topaz]

it's not the embed

look at the difference between the channelId and added

the channelId is a template literal and will get resolved to a variable

the added there is just a meaningless part of the string

[umbral cradle]

channel.send("${added}")

[wet topaz]

yes. and this is where the hard part starts

you can't really put full discord.js structures in or out of the broadcastEval

[umbral cradle]

if it would be outside the broadCastEval()

we usually do channel.send(added) simply, but when its inside broadCastEval() why its different?

[wet topaz]

cause you're passing a string

[umbral cradle]

oh

[wet topaz]

send('9touhglihglkidfhdilhbvf') sends a string

broadcastEval on its own takes a string

so you have to use template literals to fill in variables

isntead of just having a srting

[idle saffron]

solved it, not cuz of sharding. and i dont shard private bots

[umbral cradle]

hmm got it

[wet topaz]

anyway

you can't put embed like that

[umbral cradle]

  const channelId = "692711072444055623";

  bot.shard.broadcastEval(`const channel = this.channels.cache.get("${channelId}"); if (channel) channel.send("${added}")`);``` 

should work fine now?

[wet topaz]

not really

[umbral cradle]

damn do i have to define the embed variable inside the broadCastEval() then?

[wet topaz]

let me write full message

[umbral cradle]

ok

[wet topaz]

since broadcastEval takes a string, you have to pass a string. template literals basically stringify whatever gets passed to it. MessageEmbed isn't a string, so doing MessageEmbed.toString() results in [object Object] and not your embed - this means it will not work.
to actually have an embed passed from outside of broadcastEval, you have to do ${JSON.stringify(added.toJSON())} - this will turn MessageEmbed into plain JSON object, and stringify it, making it possible to be used in the broadcastEval.
alternatively, you can just create that embed inside of the broadcastEval and use it like you would normally.

similiar thing goes for taking things outside of broadcastEval - you can't take out most of library structures out like this - look here #archive-sharding message

[umbral cradle]

wow, thanks for the well explanation 👏

bot.on("guildMemberRemove", async member => {
  if (member.guild.id !== "692047883062738975") return;
  let embed = new MessageEmbed()
    .setTitle(`Bye bye ${member.user.username}`)
    .setDescription("Bye from Corona Support Server")
    .setColor(color);

    bot.shard.broadcastEval(`  var byechannel = bot.channels.resolve("692711072444055623");
    byechannel.send({"${JSON.stringify(embed.toJSON())}"});`)

});```

did i just get it now?

[wet topaz]

without the "s, but should do, yes

your sending embed, not a string after all (send(embed), not send('embed'))

note that resolve doesn't always return the channel, so the part you had before was more error resistant

[umbral cradle]

const channelId = "692711072444055623";

  bot.shard.broadcastEval(`const channel = this.channels.cache.get("${channelId}"); if (channel) channel.send("${JSON.stringify(added.toJSON())}")`);

this part?

it should be channel.send("${JSON.stringify(added.toJSON())}")

or channel.send(${JSON.stringify(added.toJSON())})

[wet topaz]

actually, wait

mm

you have to do send({ embed: ${JSON.stringify(added.toJSON())} }), since it's an object and not MessageEmbed

and i was talking about const channel = this.channels.cache.get(); if (channel) channel.send()

[umbral cradle]

why sharded bot is hard af 😂

[wet topaz]

it really isn't

this here is p much just related to making a string out of MessageEmbed you can use somewhere else

[umbral cradle]

  bot.shard.broadcastEval(`const channel = this.channels.cache.get("${channelId}"); if (channel) channel.send({ embed: ${JSON.stringify(added.toJSON())} )`);

[wet topaz]

should work

[umbral cradle]

bot.on("guildMemberAdd", async member => {

  if (member.guild.id !== "692047883062738975") return;
  let embed = new MessageEmbed()
    .setTitle(`Welcome ${member.user.username}`)
    .setDescription("Welcome to Corona Support Server")
    .setColor(color);

    const channelID = "692711072444055623"
  bot.shard.broadcastEval(`var welcomechannel = this.channels.cache.get("${channelID}");
  welcomechannel.send({embed: ${JSON.stringify(embed.toJSON())}});`)

});```

[wet topaz]

i just mentioned why the previous way of getting the channel was better

[umbral cradle]

oh yes!

edited

5|x | logged in as Corona Stats#4983
5|x | Error [SHARDING_IN_PROCESS]: Shards are still being spawned.
5|x |     at ShardingManager._performOnShards (/root/covid/node_modules/discord.js/src/sharding/ShardingManager.js:258:75)
5|x |     at ShardingManager.fetchClientValues (/root/covid/node_modules/discord.js/src/sharding/ShardingManager.js:245:17)
5|x |     at Shard._handleMessage (/root/covid/node_modules/discord.js/src/sharding/Shard.js:338:22)
5|x |     at ChildProcess.emit (events.js:314:20)
5|x |     at emit (internal/child_process.js:877:12)
5|x |     at processTicksAndRejections (internal/process/task_queues.js:85:21)
5|x | Launched shard 1```

any idea why i always get this error? low ram ?

[abstract lichen]

probably fetching client values while shards are still spawning

[umbral cradle]

do i have to run guild.owner.send() inside broadCastEval() as well?

[wet topaz]

if you want

[umbral cradle]

x | (node:24262) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'send' of null

that means i have to right?

 bot.shard.broadcastEval(`guild.owner.send("Thanks for adding me in ${guild.name} ! You can use .help to discover commands. here is our support server link: \n  , you can support us by voting: https://top.gg/bot/691936549243191346/vote");`)```

?

[minor nova]

guild.owner is undefined

[abstract lichen]

*null
try fetching the user

[proud mortar]

Guild#ownerID
The user ID of this guild's owner

[vital aspen]

you may not have intents

[valid gale]

No. Guild#owner is a nullable property. you should fetch it before accesing it as laid out above.

[vital aspen]

if you have not intents you can't fetch ;P

[dense cape]

?owner

[icy crow]

Guild#owner is a nullable property which relies on cache. Fetch the owner through GuildMemberManager#fetch() using Guild#ownerID.

• https://discord.js.org/#/docs/main/stable/class/Guild?scrollTo=ownerID
• https://discord.js.org/#/docs/main/stable/class/GuildMemberManager?scrollTo=fetch

[dense cape]

If ur just dming, then you can just fetch them as a user

[vital aspen]

@dense cape @valid gale Wait.... But if i do not have guild member intents i can not fetch users 🤔 so...

[dense cape]

What?

[vital aspen]

@dense cape adrian reacted no

[dense cape]

Guild members intent doesn’t affect users

[vital aspen]

wait what is that

So i can fetch the user?

[dense cape]

Yes

[vital aspen]

WE ngrioqwg n[rowign rw0iodcgnqpwiemfcopqemxfi;

Brain fucked now

[wraith thicket]

Hey! So I was editing the webpacks because discord.stable.min.js wasn't working, and I almost got it working. However, now I'm facing the error of Uncaught Error: The shardCount option must be a number greater than or equal to 1 when I try to start the bot via the browser.

Nvm, fixed the error.

[next plaza]

hihi, I suggested this xD #archive-guide-discussion message

[wet topaz]

Been using it before

[dreamy plume]

what even is sharding?

[abstract lichen]

check the pinned messages

[sturdy ember]

Why I cannot get channels that are in other shards?

Ping me when you see this pls

[terse cairn]

That's how sharing works, you create instances of your client that only handle a part of the total guilds. You can use broadcast eval tho to for example send a message in a channel you can access

[sturdy ember]

can u show me how?

i have already tried but I couldn't firgured out how it works

[terse cairn]

client.shard.broadcastEval("client.channels.get('some ID')?.send('some message')")

For example

[fervent dragon]

Up to how many shards can u have?

[wet topaz]

there is not such thing as shard limit

only barrier is identify limit

[covert whale]

why does the lib throw if a shard isn't ready "in time", why doesn't it continue with the remaining shards in queue?

also, why doesn't the lib respect the spawn delay? why does it need for the previous shard to mark ready? I would expect the sharding manager to spawn shards every 5 seconds if i tell it to for an example, but it spawns the next shard until the previous marks ready regardless of the spawn delay i set

[valid gale]

When using optional chaining please make clear that it requires node 14+

[sturdy ember]

Thanks

[terse cairn]

hello, after I changed the shards number to 2 shards after my bot rached 2.5k servers, the ram consumption tripled, is it normal or I'm missing a configuration

[hollow gorge]

When you shard, it spawns an instance per shard.

So if you had 1 shard before that used 250mb of RAM, once you spawn another shard, it can double to 500mb of RAM.

[terse cairn]

is there a way to split the cache ? each shard caches only the servers it has

[junior tulip]

For internal sharding, the cache is shared. Process/worker-thread sharding have separate cache because they run in separate threads or processes, and they only cache guilds they have

[sturdy ember]

this should work when trying to find a user with broadcast eval right?

let member = client.shard.broadcastEval("client.users.cache.get(args[0])")

[terse cairn]

It would not, you cannot share D.JS strucutes between shards, so you cannot get the full User object, you can get properties form the user object, or execute methods on the user object inside the broadcastEval

[sturdy ember]

and this? this should work right?

const channel = client.shard.broadcastEval("client.channels.cache.get('832169994141302794')")

[dapper ibex]

quick question, do each spawned shard execute the passed file on the ShardManager constructor to instantiate each shard's client?

[abstract lichen]

did you actually read the answer above you

[sturdy ember]

he was talking about the user object

[abstract lichen]

It would not, you cannot share D.JS strucutes between shards

what do you think a channel is

[broken sequoia]

Can't you fetch a channel though

[winged bobcat]

Ok so you see bots creating "clusters" with sharding, is this done with the option shardLists ?

[abstract lichen]

Disclaimer: I haven't used any of this, I'm just explaining how I think it works and from looking at code
I would guess they mix internal sharding (ClientOptions#shards) with custom process sharding,
e.g. you have 12 shards
you spawn 3 processes and each process handles 4 shards (with internal sharding).
a library that implements this, is https://github.com/DevYukine/Kurasuta

[vital aspen]

Why we can not pass function in broadcastEval ?

[abstract lichen]

you can, but it will simply get converted to a string, and evaled by the other process

it should work, depending on the function you pass in (or its content)

[vital aspen]

So why it is recommended to put string?

[abstract lichen]

is it?

[vital aspen]

idk, everyone using string version

[abstract lichen]

maybe because they dont know you can pass functions, idk

[round hound]

Error: ENOENT: no such file or directory, stat 'C:\Users\Silver\Desktop\A_Learn\Cyspace\cyspace' at Object.statSync (fs.js:1016:3) at new ShardingManager (C:\Users\Silver\Desktop\A_Learn\Cyspace\node_modules\discord.js\src\sharding\ShardingManager.js:61:22) at Object.<anonymous> (C:\Users\Silver\Desktop\A_Learn\Cyspace\dist\sharding.js:4:17) at Module._compile (internal/modules/cjs/loader.js:1137:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1157:10) at Module.load (internal/modules/cjs/loader.js:985:32) at Function.Module._load (internal/modules/cjs/loader.js:878:14) at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12) at internal/main/run_main_module.js:17:47 { errno: -4058, syscall: 'stat', code: 'ENOENT', path: 'C:\\Users\\Silver\\Desktop\\A_Learn\\Cyspace\\cyspace' }

my sharding file

const { ShardingManager } = require('discord.js');
const { TOKENS } = require('./config');
const { info } = require('./lib/logger/logger');
const manager = new ShardingManager('./cyspace', { token:  TOKENS.BOT_TOKEN});

manager.on('shardCreate', shard => info(`Launched shard ${shard.id}`));
manager.spawn();

My main file

const Discord = require('discord.js');
const { onReady } = require('./lib/events/onReady')
const { TOKENS } = require('./config.js');


const client = new Discord.Client();


client.on('ready', (message) => { onReady(client) });
client.login(TOKENS.BOT_TOKEN)

[bright lynx]

you sure that your bot file is called cyspace and not cyspace.js?

[round hound]

a js file without js extension?

[bright lynx]

you're passing './cyspace' without an extension

[round hound]

same error after passing cyspace.js

my file structure

[bright lynx]

is the file in a folder or in the root of the project?

[round hound]

in a folder named dist

[bright lynx]

then you also need to add that to the path

[round hound]

ok one quick question
can i use sharding on my new bot

[bright lynx]

you can, there's just no reason to if your bot is in a small amount of guilds

[round hound]

thanks

[dapper ibex]

is it possible to use a GuildEmoji from a guild from another shard to react on a message on a different shard?

[wet topaz]

you can just use id to react, but if you really need to have full object for whatever reason, https://discordjs.guide/sharding/extended.html#using-functions-continued

[dapper ibex]

oh, thanks. I just need the ID. Didn't know (or i forgot) it was possible.

👍

[winged bobcat]

[covert vortex]

Hello, my bot shard 2 just keeps starting up
https://cdn.discordapp.com/attachments/653572279485333505/832341209485279242/unknown.png

do you know why ?

[dense cape]

If ur on node v. 15, unhandled rejections are deprecated and will crash

[covert vortex]

I'm in v12.18.0 of nodejs

which version do I install ?

[dense cape]

That should be fine. Hard to say what's killing ur shard

[covert vortex]

hm

[wet topaz]

is there any more info? d.js version at least?

[crystal merlin]

we made the update of Node.Js

[hazy hinge]

How do I find what shard is the bot in? Like there is shard 0, 1, 2, 3, 4, 5 so for example, im in shard 1 and have this stats command. How will the bot tell me that im in shard 1 (ofc through a message but how do I display the shard its on?)

[violet compass]

message.guild.shardID```

[hazy hinge]

thx

[jaunty valve]

What does manager.broadcast do, and how can I recieve the messages

[real lava]

how can i do multi-host sharding? i've looked on the docs and guide and have found nothing.

[abstract lichen]

You would need to implement this on your own, 98% of d.js users don't need this so writing a guide on it is wasted lifetime

or there is a library that helps with that, but I only know of same host sharding

[real lava]

alright

[valid gale]

It's fairly simple, you just run multiple instances on multiple hosts and tell each instance to only handle a certain set of shards

[real lava]

so like i could have a docker service, all running the bot container, all on different servers though, and all those bot containers have sharding enabled?

[abstract lichen]

yeah, but you have to tell each instance what shards to run

[real lava]

right, so i'd need some kind of master bot / controller

[abstract lichen]

depends, could probably have each host communicate with each other over a database so you don't have a master, depends how you wanna setup/program your infrastructur

[real lava]

yeah ofc

[real lava]

@abstract lichen (sorry for ping) I took your advice and used Kurasuta but it doesn't seem to log in / ready (I get Error: Cluster 0 failed to start), the GH page said if that happens, that the API never emits READY. any ideas?

[abstract lichen]

Nope, never used it and not really djs related.
I would try just copy pasting the exact example and if that doesn't work... I don't know, open a gh issue or something

[real lava]

alright, sounds good

[crisp coyote]

are you using a custom client?

make sure to pass options along in your constructor

[terse cairn]

should we do sharding of bot is in 150 guilds

[strange echo]

It's unnecessary at that size

[real lava]

yeah i'm using akairo

[crisp coyote]

@real lava https://github.com/DevYukine/Kurasuta/issues/360

[real lava]

i saw that issue and i tried it, i’ll send some code here later when i’m back at my PC

[real lava]

alright here's the code

client:

class NyxxClient extends AkairoClient {
  ...
  constructor(clientOptions: ClientOptions) {
    super({
      ownerID: ['401792058970603539', '162305223589756928'],
    }, {
      ...clientOptions,
    });

    ...
  }

  async init(): Promise<this> {
    ...
  }
}

cluster:

export class NyxxCluster extends BaseCluster {
  public client!: NyxxClient;

  logger: typeof Log

  constructor(...args: [ShardingManager]) {
    super(...args);

    this.client.cluster = this;
    Log.setSettings({
      prefix: [`Cluster ${this.id}`],
    });
    this.logger = Log;
  }

  async launch(): Promise<void> {
    this.logger.info('Initializing Client...');

    this.client.init().then(() => {
      this.logger.info('Client initialized!');
    }).catch((err) => {
      this.logger.fatal('Failed to initialize client', err);
      throw err;
    });

    this.logger.info('Logging in...');

    this.client.login(process.env.BOT_TOKEN).then(() => {
      this.logger.info('Client Logged in!');
    }).catch((err) => {
      this.logger.fatal('Failed to log in', err);
      throw err;
    });
  }
}

index:

const Sharder = new ShardingManager(join(__dirname, 'struct', 'Cluster'), {
  client: NyxxClient,
  development: (process.env.NODE_ENV !== 'production'),
  respawn: (process.env.NODE_ENV === 'production'),
  retry: true,
  token: process.env.BOT_TOKEN,
  shardCount: 1,
  timeout: 120000,
});

Sharder.on(SharderEvents.DEBUG, (message: string) => {
  Log.debug(message);
});

Sharder.spawn().catch((err) => {
  Log.fatal(err);
  throw err;
});

[crisp coyote]

what is your error? @real lava

[real lava]

after 30 seconds, the default timeout, it errors just by saying "Cluster 0 failed to start". extending the timeout does nothing.

full err trace

(node:30584) UnhandledPromiseRejectionWarning: Error: Cluster 0 failed to start
    at ShardingManager.spawn (D:\Code\Nyxx\node_modules\kurasuta\src\Sharding\ShardingManager.ts:117:37)

[real lava]

@crisp coyote any ideas? jw

[crisp coyote]

not really stuff looks fine

can you log the custom client options which kurasuta is passing?

[solid lake]

help-me

[dense cape]

Update d.js to 12.5.3

[solid lake]

Ok thanks

[real lava]

sure

<ref *1> ShardingManager {
  _events: [Object: null prototype] { debug: [Function (anonymous)] },
  _eventsCount: 1,
  _maxListeners: undefined,
  path: 'D:\\Code\\Nyxx\\src\\struct\\NyxxCluster',
  clusters: Map(0) {},
  clusterCount: 16,
  guildsPerShard: 1000,
  clientOptions: {},
  development: true,
  shardCount: 1,
  client: [class NyxxClient extends AkairoClient],
  respawn: false,
  ipcSocket: 9999,
  retry: true,
  timeout: 30000,
  token: '...',
  nodeArgs: undefined,
  ipc: MasterIPC {
    _events: [Object: null prototype] {
      debug: [Function (anonymous)],
      error: [Function (anonymous)]
    },
    _eventsCount: 2,
    _maxListeners: undefined,
    manager: [Circular *1],
    server: Server {
      _events: [Object: null prototype],
      _eventsCount: 4,
      _maxListeners: undefined,
      sockets: Map(0) {},
      status: 3,
      name: 'Master',
      server: [Server],
      [Symbol(kCapture)]: false
    },
    [Symbol(kCapture)]: false
  },
  [Symbol(kCapture)]: false
}

that's in the constructor, ClientOptions

[digital jasper]

hi, I'm working on sharding for my bot, and I have some information I want all shards to be able to access and modify (if a shard modifies the information, I need it to be modified for each shard).

Is there a way I could do this easily? I've tried a few things that haven't worked so far, and I'm a complete beginner when it comes to sharding so I have no experience.
Any help would be much appreciated!

[abstract lichen]

a database

preferable a database server (postgresql, mongo, redis, whatever you want)

[digital jasper]

hmmm that's what I was trying to avoid lol. Oh well, if it's the only solution it'll have to do

[abstract lichen]

Your message describes the exact use case for a database. I don't know what other option there would be?

[mild granite]

UnhandledPromiseRejectionWarning: Error [SHARDING_READY_TIMEOUT]: Shard 0's Client took too 
long to become ready.```
Any workaround for this?

[terse cairn]

?shardreadytimeout

[icy crow]

Got this error: Error [SHARDING_READY_TIMEOUT]: Shard 0's Client took too long to become ready.?
The spawnTimeout can be set to -1 or Infinity to prevent future errors:
https://discord.js.org/#/docs/main/stable/class/ShardingManager?scrollTo=spawn. This can also happen if you call ShardingManager#spawn before attaching a listener to the shardCreate event which could create a race condition possibly preventing the shard 0 to perform a successful launch.

[coral sluice]

?shard

[icy crow]

const { ShardingManager } = require('discord.js');
const Manager = new ShardingManager('./bot.js', { token: 'your-token' });
Manager.spawn();

https://discord.js.org/#/docs/main/stable/class/ShardingManager
For an in-depth guide: https://discordjs.guide/sharding/

[digital jasper]

the issue is a it's scores that then need to be sorted, so I'd need to pull from the DB and then sort all the items, pretty easy to do but can pretty slow (takes about 2 secs atm, and that time will only get worse)

Ideally I'd want some sort of pre sorted list/array in memory I could just pull the corresponding elements from, but I can't find a way to do that (if it even exists)

[abstract lichen]

have you considered using a db that can sort

or use redis

[digital jasper]

hmmm what DB would be able to do that? I haven't thought of it at all

[abstract lichen]

any sql db? order by

[digital jasper]

so does that mean the DB stores the items already sorted, so runs a sorting algorithm when you call the DB?

[abstract lichen]

you can use redis to have in memory arrays/lists

[digital jasper]

looking into it now, looks like it's what I need

[abstract lichen]

or use a db that can sort before it returns your data, I bet most database servers do heavy optimizations on repetetive queries

but yeah, if you are looking for fast in memory operations, look into redis

[digital jasper]

the one I use (sequelize, from the discord.js tutorial, I never bothered to learn a new one), takes about 1s to order the data, need it needs to be sorted. I haven't really looked into having it sorted before hand (didn't think it was possible)
But yea the bulk of the time is pulling from the DB, so I'd say redis is the way to go

ty for the info 🙂

[abstract lichen]

sequelize with what database?

because sequelize is just an orm, it uses whatever database you tell it to, assuming sequelize is not completely stupid, its not important, its important how fast the database behind it can be

[digital jasper]

hmmm I didn't know that, haven't looked into it at all.
Is sqlite3 a database? I had to install that with sequelize when I followed the tutorial

[abstract lichen]

change that then asap

sqlite3 is not ideal in a sharded enviroment

as far as I know

[digital jasper]

ah okay, so something like mysql would be much better?

[abstract lichen]

I'm not a fan of mysql, but it would do. I would recommend postgresql

[digital jasper]

okay more stuff to look into lol.
I'm doing a lot of rewriting before sharding anyways, so it's 100% the time to do this kind of stuff

[abstract lichen]

okay so see #archive-offtopic
sqlite can probably handle more concurrent than I think, I just often had bad performance with it

if you have the time, just compare both, imo

[digital jasper]

Hmmm okay.
Well I'm not asking to much. Writing some data a few times a min, and if I implement memory caching I won't be reading huge amounts of data at once.
Also I'm used to using it which as a definite advantage, but I'll be sure to check out postgresql

[abstract lichen]

Also I'm used to using it
wdym?

you can just tell sequelize to connect to a postgres server instead, it probably takes barley any code changes at all

only thing you might have to worry about is migrating the data

[digital jasper]

Oh okay, now I get what sequelize is!
Migrating the data shouldn't be to hard with a simple for loop ig, even if there are better ways it's only a 1 time thing so I won't think about it too hard

[solid lake]

how to tag a discord channel by id? command in Js

[dense cape]

Don’t crosspost

[spare cairn]

What's the best option between process and worker for ShardingManagerMode?

I might not be djs-related, but I don't get the difference between the two options

[abstract lichen]

worker uses worker_threads

[proud mortar]

Unable to parse received data, but maybe this is useful: https://nodejs.org/api/worker_threads.html

[spare cairn]

I'm already reading that

The only diff is that they share memory?

[abstract lichen]

yeah, but dunno how big the difference is

[dense cape]

Workers are just threads running on a single process iirc

[spare cairn]

I don't need my shards to interact so i think i'll use process

[abstract lichen]

uh ok?

[spare cairn]

yeah wdc sorry

[abstract lichen]

processes still interact with each other over ipc channels but ok

[spare cairn]

shard.on("death", (error) => {
     console.log(chalk.red(`SHARD #${shard.id} DIED!`));
     console.error(error);
     shard.respawn()
          .then(() => console.log(chalk.green(`SHARD #${shard.id} WAS RESPAWED!`)))
          .catch(() => console.log(chalk.red(`SHARD #${shard.id} WAS NOT RESPAWNED!`)));
 });

Is this a good way of handling death/errors on shards? Should I respawn the shard right after the error was caught, should I wait (more than 500ms) or should I leave the shard dead to prevent further errors from happening?

[icy steeple]

shard

[hollow gorge]

Shard respawns are automatic if you have the respawn option enabled in the ShardingManager options. There shouldn't be a reason for you to be manually handling the respawns.

[real lava]

@crisp coyote hate to bother you, but are you still able to help me with this?

[spare cairn]

'kay i'll remove this part

[terse cairn]

?shardreadytimeout

[icy crow]

Got this error: Error [SHARDING_READY_TIMEOUT]: Shard 0's Client took too long to become ready.?
The spawnTimeout can be set to -1 or Infinity to prevent future errors:
https://discord.js.org/#/docs/main/stable/class/ShardingManager?scrollTo=spawn. This can also happen if you call ShardingManager#spawn before attaching a listener to the shardCreate event which could create a race condition possibly preventing the shard 0 to perform a successful launch.

[static spindle]

stop.ts I was going to help them :(

[terse cairn]

there is a tag

[valid gale]

Please keep the support channels on topic.

[unkempt hornet]

whats the easiest way to find the guild count of a sharded bot?

[valid gale]

the sharding section of the guide does this with detailed explanations

[unkempt hornet]

oh, djs has an inbuilt sharding manager?

thats really nice
thanks, found the link

[worthy thunder]

if (message.content === prefix +'help')

how do i make that work, so that i can use prefix

[spare cairn]

Wrong channel

[worthy thunder]

pls help{

[spare cairn]

?r4

[icy crow]

4. When asking for help, make sure to provide as much detail as possible. Simply saying, "it's broke yo", or "how 2 make music bot" certainly won't get either of us anywhere.

[spare cairn]

When is shardError emitted? I tried to manually throw an error in my client's code, the shard dies then is respawned, but I don't receive any shardError event.

[real lava]

i'm using Kurasuta as some here have recommended for sharding, but my bot never starts. says "Cluster 0 failed to start" and it hit the timeout so it "took too long". setting the timeout to -1 or Infinity did nothing. if i remove the sharding and just use the bot normally (since i'm using discord-akairo) the bot starts fine, so it's not an issue with the bot token. what do i do?

[little bluff]

Error [SHARDING_NO_CHILD_EXISTS]: Shard 32 has no active process or worker.
This will happen after a few days. Because it takes so long, and happens so randomly, I can't figure out the cause. No errors are thrown on 32 to knock it offline. :-/

[jade venture]

https://github.com/discordjs/discord.js/issues/3956

Anybody know how to fix it in v12? My bot cant login from any IP/PC for about 2 days

[jade venture]

dont help

[craggy tundra]

I get this error randomly when starting my bot

[solid lake]

hello guys, I'm trying to create a bot and I want him to do the following thing, when a new member enters the server he will check the day the account is created and if the account is less than 7 days old for example I want him to a special position for her, how can I do that? please send me a code example.

[wet topaz]

that's not really related to sharding

and please don't crosspost

[spare cairn]

Are ratelimits per application/token or per shard?

[broken sequoia]

No, it’s per bot user

[spare cairn]

Is it possible to force a guild to be in the shard #0 so that I can start sending messages to that guild even if the other shards are not loaded yet?

[wispy ocean]

is there a minimum required amount of servers before you can start sharding? I'm interested in messing around with it for several reasons, but wanted to be sure if it'd even work with only ~1400 servers

[real lava]

technically no iirc @wispy ocean

[wispy ocean]

thanks

[real lava]

yup!

[little bluff]

What may be happening is that shard is running into an error, because of the undefined guild, and either exiting the process or preventing it from signalling a ready state..

An easy way to check if it's an error causing your shard from spawning, throw this at the top of the file that initiates each client (not the sharding manager):

process
    .on( 'uncaughtException',  console.error )
    .on( 'unhandledRejection', console.error );

const bot = new Discord.Client( {
//... Etc...

Important note, do not rely on this as a lazy means of catching errors. You should always have proper error handling/logging. However, it should get your shards to ready, and therefore get you to where you can better debug the issue.

Also, not sure if it'll help, consider upping the timeout.

[next plaza]

it can be bc of insufficient ressources....
The Error means that Shard 32 lost their worker/process ===> you have to create a new worker_thread or child_process, when you have respawn enabled ==> it willl create a new worker/process

[little bluff]

Resources in terms of what? It's running on a very costly VPS, so it's not low on memory, storage, etc. My current solution is to just have the sharding manager respawn it when it happens, but I want to prevent it from happening.

[next plaza]

you can't, thats probably a node problem, respawn is the only way to keep them alive

[dusty sapphire]

is it possible to make a specific shard online or offline ?

[mellow burrow]

at how many guilds should one switch from internal to traditional sharding

[terse cairn]

Hello !

My Discord bot has reached 2 500 servers, so I have to use the shards system.
Here is what I do:

const { Client, Collection } = require("discord.js");
const { readdir } = require("fs");
const Enmap = require("enmap");
const express = require("express");
const bodyParser = require("body-parser");
const { bot, settingsDbl } = require("./config.json");
const { reportError, setClient } = require("./Utils/loggingManager.js");
const client = new Client({ DisableMentionType: "all" });
client.commands = new Collection();

client.database = { };

client.database.servers = new Enmap({ name: "servers" });
client.database.msp = new Enmap({ name: "msp" });

const app = express();
app.use(bodyParser.json());

readdir("./Commands/", (error, files) => {
  if (error) {
    reportError(error);
  }

  let commands = files.filter(f => f.split(".").pop() === "js");

  if (commands.length == 0) {
    reportError("0 Commands!");
    process.exit(1);
  }

  commands.forEach(commandFile => {
    let command = require(`./Commands/${commandFile}`);
    console.log(`Loaded ${commandFile}!`);
    client.commands.set(command.help.name, command);
  });
});

readdir("./Events/", (error, files) => {
  if (error) {
    reportError(error);
  }

  let events = files.filter(f => f.split(".").pop() === "js");

  if (events.length == 0) {
    reportError("0 Events!");
    process.exit(1);
  }

  events.forEach(eventFile => {
    let events = require(`./Events/${eventFile}`);
    let event = eventFile.split(".")[0];
    client.on(event, events.bind(null, client));
  });
});

setClient(client);

my msp.js file the file I start :

const { bot } = require("./config.json");
const { ShardingManager } = require("discord.js");

const manager = new ShardingManager("./index.js", {
    totalShards: 'auto',
    token: bot.token
});

manager.on('shardCreate', (shard) => console.log(`Shard ${shard.id} launched`));

manager.spawn();

The problem is that I get this error: (node:13896) UnhandledPromiseRejectionWarning: Error [SHARDING_READY_TIMEOUT]: Shard 0's Client took too long to become ready.

I don't know how to solve this problem, can someone help me please?

You can mention me!

[dense cape]

Do you even have the client.login call?

@terse cairn

[terse cairn]

no

because I thought it was already doing it with the shards system

I have to make it I think

[dense cape]

Nah, that’s to get the sharding info, like number of shards to spawn

[terse cairn]

ah yes thank you it's work !

[dense cape]

Also gotta be careful with hosting webserver bc it’s will run the index.js file once for every shard

Can’t have multiple servers on the same port, tho it doesn’t seem like ur using it atm

[terse cairn]

ahhh yes !

[dense cape]

Better off putting it on the sharding manager file

[terse cairn]

yes 👍

thank you for this precision

[dense cape]

Np

[terse cairn]

@dense cape how I can export the client in my msp file pls ?

[dense cape]

What?

[terse cairn]

const { bot, settingsDbl } = require("./config.json");
const { ShardingManager } = require("discord.js");
const express = require("express");
const bodyParser = require("body-parser");

const manager = new ShardingManager("./index.js", {
    totalShards: "auto",
    token: bot.token
});

manager.on("shardCreate", (shard) => console.log(`Shard ${shard.id} launched`));

manager.spawn();

const app = express();
app.use(bodyParser.json());

app.post("/webhooks_bot", (req, res) => {
res.sendStatus(200);

if (req.headers.authorization === settingsDbl.authorization_bot) {
const user = req.body.user;

client.channels.cache.get(settingsDbl.channel_bot).send(`:arrow_up: | <@${user}> (\`${user}\`) has just voted for **${client.user.username}**, thank you! https://top.gg/bot/${client.user.id}/vote`);
}});

app.listen(settingsDbl.port, () => {
console.log("The server is online !");
})

[dense cape]

?sharding

[icy crow]

const { ShardingManager } = require('discord.js');
const Manager = new ShardingManager('./bot.js', { token: 'your-token' });
Manager.spawn();

https://discord.js.org/#/docs/main/stable/class/ShardingManager
For an in-depth guide: https://discordjs.guide/sharding/

[terse cairn]

I need it to send a message

[dense cape]

Use Manager.broadcastEval

[terse cairn]

👍

Nice, thx

@dense cape Hum, I have an error :

manager.broadcastEval("this.channels.cache.get(settingsDbl.channel_bot).send(`:arrow_up: | <@${user}> (\`${user}\`) has just voted for **${this.user.username}**, thank you! https://top.gg/bot/${this.user.id}/vote`)", 0);

(node:16119) UnhandledPromiseRejectionWarning: SyntaxError: missing ) after argument list

it's a code error, but what ?

you can ping me 🙂

[terse cairn]

okay nobody

const { MessageEmbed } = require("discord.js");
const { bot, settingsLogs, settingsEmbed, settingsEmojis } = require("../config");

module.exports = async(client, guild) => {
  
    const database = client.database.servers;

    if (database.get(guild.id)) {
    } else {
    database.set(guild.id, {
        prefix: bot.defaultPrefix
    });
    }
  
    const channel = client.shard.broadcastEval("this.channels.cache.get(\"767016401361043456\")");
    const vocal = client.shard.broadcastEval("this.channels.cache.get(\"768867402204577792\")");
    let servers = await client.shard.broadcastEval("this.guilds.cache.size");
    const reducer = (accumulator, currentValue) => accumulator + currentValue;
    servers = servers.reduce(reducer);

  vocal.edit({ name: `📡 ${servers} servers` }, `i joined ${guild.name} (${guild.id})`);

}

Otherwise, I don't understand how I can get a channel

event : guildCreate

everything is more complicated with shards

how I can get my channel ?

const channel = client.channels.cache.get("ID");

now it's not this bc I use shards

I am very confused, I don't know how it works and my bot doesn't work :CCCC

ping me pls ^^

[dense cape]

You can’t receive a proper channel across shards

Well, you can’t make any api calls with it

[terse cairn]

ah ???

It is no longer possible to get a channel?

There must be a way, no ?

ahh okay

client.shard.broadcastEval(`
    (async () => {
        let channel = this.channels.cache.get('id');
        let msg;
        if (channel) {
            msg = await channel.messages.fetch('id').then(m => m.id);
        }
        return msg;
    })();
`);

so :

    const channel = await client.shard.broadcastEval(`
    (async () => {
      let channel = await this.channels.cache.get("767016401361043456");
      return channel;
      })();
    `);
    console.log(channel);
    const vocal = await client.shard.broadcastEval(`
    (async () => {
      let channel = await this.channels.cache.get("768867402204577792");
      return channel;
      })();
    `);
    let servers = await client.shard.broadcastEval("this.guilds.cache.size");
    const reducer = (accumulator, currentValue) => accumulator + currentValue;
    servers = servers.reduce(reducer);

    vocal.edit({ name: `📡 ${servers} servers` }, `i joined ${guild.name} (${guild.id})`);

It's don't work, I log channel, and it's an array :C

[
  {
    type: 'text',
    deleted: false,
    id: '767016401361043456',
    name: 'join',
    rawPosition: 9,
    parentID: '767016289994145802',
    permissionOverwrites: [
      '676860457935437863',
      '676860851948355616',
      '764580414391058483'
    ],
    topic: 'ℹ️ When a person invite the bot, a message ' +
      'will appear here! Link: ' +
      'https://bit.ly/bot-msp',
    nsfw: false,
    lastMessageID: '833634163063259183',
    rateLimitPerUser: 0,
    lastPinTimestamp: null,
    guild: '676860457935437863',
    messages: [],
    createdTimestamp: 1602941360560
  }
]

an idea pls ?

ping me

[sterile musk]

Just get the first index?

[terse cairn]

like channel[0] ?

no, don't work

[next plaza]

child processes can share the same port !?

[dense cape]

webserver

[next plaza]

oh

[dense cape]

They were abt to run webservers on the shard file

[next plaza]

a question, how do I set different presences on internal shards?
do I have to do a for loop on client.on('ready') ?
https://discord.js.org/#/docs/main/stable/typedef/PresenceData
the links shows it.
But isnt a looping required?

bc client.ws.shard is a collection of ws shards

[proud mortar]

(event) Client#shardReady
Emitted when a shard turns ready.

[next plaza]

A question, when I just edit the status ....., without providing a shardid, will it edit it globally on the client?

[dense cape]

All shards the client has

@next plaza

[next plaza]

will it overwrite the current activity.name, when they are different?

[dense cape]

Yes

[next plaza]

So there is no possibility to make somethink like "Bot is Starting " and then "Playing cmd | Shard 0"
Set timeout is the only way to do it

[dense cape]

You can’t have a presence before the bot becomes ready

[real lava]

i mean you TECHNICALLY could if you used the websocket connection direcetly, no djs, just connect and set the status, but you should never to that tbh

[dense cape]

So you would have to run two instances per shard

[real lava]

i've been stuck on the same sharding issue for days lol but im using akairo so i don't think many can help me here

[next plaza]

state your question

[real lava]

so tl;dr i'm using akairo with Kurasuta (sharding on multiple cores) and it keeps erroring after the timeout period of 30 seconds saying "Cluster 0 took too long to start", setting the timeout to -1 or Infinity does nothing. if i start the bot normally, it works fine and logs in.

worst case scenario i can just use djs' sharding but i'd rather not since Kurasuta is mutlicore

[next plaza]

discord js sharding manager is multicore too

both do the same thing

I saw their code and they do the same thing, start worker_threads or child_processes

[real lava]

hm did not know that

so i guess ill use that lol

[next plaza]

use the normal sharding manager, when your bot is over 2500 guilds. WHen your bot uses less cpu....then you can use internal sharding

Note: Sharding has no effect on bots smaller than 2k guilds

[next plaza]

do you them as an array of objects?

[wanton raft]

So I get an error on one of my last few shards saying it's timed out

and then the rest of them don't spawn

[next plaza]

this will give [guild1, guild2, guild3, guild4......]

updated it

to learn? you to know?