#broadcastEval serder changes

34 messages · Page 1 of 1 (latest)

topaz narwhal

Making a thread since it's a complex topic that needs a lot of planning and explaining

mellow kernel

@upper brook It is a broadcastEval response, so a djs structure as object without getters and methods

topaz narwhal

v8's serder is basically an alternative to JSON that's more compressed and can hydrate a lot more complex types, such as maps

mellow kernel

ah okay

topaz narwhal

However, it seems it can't hydrate user classes (such as Guild), so we'd need to go for another approach

Your proposal will probably be considered and implemented for v14, although with small changes, you'd do new Guild(client, data) instead.

mellow kernel
topaz narwhal

There's a series of changes we'd need to do first, starting with Souji's PR at...

bronze hemlockBOT
topaz narwhal

Same changes need to be done everywhere else that relies on extras/additional context/cache

The other change is to make each structure's toJSON return the de-transformed data in Discord's format, so we would be able to do (in theory) new Guild(client, guild.toJSON()) to clone a Guild instance. Once that's done, we can easily deserialize objects into richer datatypes.

mellow kernel

we could do this:

Discord.GuildManager.prototype.hydrate = function(guildobject) {
    const rawobject = turnGuildObjectToDiscordObject(guildobject)
    return this._add(rawobject, false);
};

can be accessed with client.guild.hydrate(guildobject)

topaz narwhal
topaz narwhal
upper brook

I think making toJSON methods return the Discord "version" of the structure... a cleaner idea

topaz narwhal

And that's basically bloat, plus complicates the deserialization of GuildMember, Role, GuildEmoji, GuildChannel, etc, so not really doable

upper brook

It wouldn't be that difficult to implement (just properties to adapt) and wouldn't require any more code

mellow kernel
topaz narwhal

Well, I brought a point on how we need to strip the structures's constructors from extra context, your proposal requires extra context

Mind you, the hydration will most likely not be automatic, because the amount of pitfalls from it (data structured in formats that are complicated to transverse, e.g. objects), but hopefully it's a single line to change in most cases for you

mellow kernel

Lets take a user as example:
Discord Structure:

{
  "id": "691288534375596062",
  "username": "Meister",
  "discriminator": "9667",
  "avatar": "bf11ac5979583c05b29ee9ae1f9ae005",
  "flags": 64,
  "premium_type": 1,
  "public_flags": 64
}

Discord.js Structure:

User {
  id: '691288534375596062',
  bot: false,
  system: false,
  flags: null,
  lastMessageID: null,
  lastMessageChannelID: null,
  username: 'Meister',
  discriminator: '9667',
  avatar: 'bf11ac5979583c05b29ee9ae1f9ae005'
}

So on easy Structure such as #role, #user , it will require less changes. But the Guild Object is a little bit nested on djs-side

tepid charm

Feel free to open a PR if its easy enough

tame ferry

I'm not sure on that, maybe just for "simple" structures?

For guilds this might (read: will, people already try this a lot) incentivize bad practice: e.g. Fetching all guilds including all their data (see below) from all shards to do something that does not need all that data, or rather hardly any of that. (e.g. find a channel / emoji / sticker by name, instead of doing that in-place and only aggregate / final filter this back in the calling process.)

Also how deep would we go? Guilds are quite large: They have emojis, stickers, roles, members, presences, voice states, stage instances, channels, permissions and probably a lot more that I couldn't think of right now.

mellow kernel
topaz narwhal

IMO it's the user's responsibility to handle that. In an ideal world, JS would support GraphQL OOTB so we don't need to worry about serializing fields we don't need

topaz narwhal

Come think of it, toJSON emitting raw data would be basically almost zero-cost if we carry out our plans for v14 of storing a portion of the raw data, Crawl

And also solves @tame ferry's concern, as you'd be able to do guild.data to get just the guild data, without the channels, emojis, roles, members, etc

mellow kernel
topaz narwhal

That's essentially what we had in mind for v14

mellow kernel
topaz narwhal

Mind you, that's something we discussed internally only, there are many details we want to polish, as well as a lot of investigation (specially determining whether this helps reducing memory usage, if it's feasible and doesn't worsen the library's performance, etc)

mellow kernel
topaz narwhal

We wouldn't mutate the raw data at all, just omit them via object destructure