Any way to serialise discord objects to JSON

[hazy pawn]

So in the context of APIs its knowledgable that they use json to pass to the user data like sort of payloads. But i want on 1 scenario to use json and kinda translate back and fourth from json to a embed, message.... etc.

Since i wanna work on a very powerful feature thats sort of channel snapshots where the bot gets the last 100 messages including pinned ones, serialises the data to json like embeds, components, files..... And when i lets say want to use a command called /snapshot recreate i can just pass in the json file the bot gave me. And it will create all messages sent with webhooks, attach the permissions and all other attributes which can deal with scenarios like nukes and hacked admins and ofc you can save snapshots manually by /snapshot save which gets the objects and converts them into json

The question is should i invent the wheel on how to transform the objects into json or is there a built in way to do so?

I thought of stuff like databases but it can be a memory issue when there are a lot of channels, lets hypothetically say i have 105 messages stored in each channel the 5 are pinned and 100 are newest. If i have a server where there are 35 channels and has the average said above. It would be 3675 records(not counting permissions, if the channel is stage, voice, announcement.....) for each server which might be a issue if i wanna make it public

[novel wing]

If you make a class that has all that data you could serialize it with pickle, woudnt be json but it takes a lot less storage

[hazy pawn]

original solution would be to give to the admin the snapshot

that is json

that way no worries of storage

and they could input that json file to recreate the channel

although 2 problems, 1 the admin could change stuff like messages if they know how to do so or might accidentally mess up and 2 i will have to check for syntax errors.....

something like this

small example

// type 0 is text channel
{
  "name": "general",
  "type": 0,
  "permissions": [
     "@everyone": {
        read_message_history: true,
        manage_messages: true
        // ...
     },
     "admin": {
        read_message_history: true,
        manage_messages: false
        //...
     },
  ]
    "messages": [
       {  
          "content": "Hello World"
          // ...
       }
    ]
}

[hazy pawn]

there is also to_dict

i realised

[hazy pawn]

also forgot to mention this

but idk if pickle is secure enough

i heard rumors it has some secuirity risks

i haven't gone tho to full research mode yet

[velvet thunder]

The question is should i invent the wheel on how to transform the objects into json or is there a built in way to do so?
Yes because you can choose what to save and what to not
For example iirc bots can't send embed with type other than 4, so you don't need to save type for every embed
to_dict exists for most stuff, but I still recommend to use your own structs to add some additional info

I also think storage is not the problem here
Even if 1 message == ~10kb (which is most of the time is not)
1 channel will be ~1mb and 1 guild ~35mb
You can easily store ~ 100 servers with 4gb of space without optimisations
Simple zip algorithm can make files from x1,2 to x10 times smaller
(btw, don't forget to make limits and add premium)

BUT, I'm really concerned that saving user's message content without their permissions is really against the discord tos
Afaik every member must know that someone can copy/store their messages without them even knowing that
Yes, logs exists, but most bots don't store messages outside of discord
If someone deleted the message, then they had a reason to do that

[hazy pawn]

hm?

[hazy pawn]

also regardless message deleting can be tracked as well as edits. I kinda have to log those im forced since people can edit stuff from innocent to lets say gore and vice versa

and well tracking behaviour that will probably be harmful to the server

will be a great tool to have

[velvet thunder]

Well, editing to something bad is so rare, that you most likely will never see that
There are no reasons rn to do that lol

[hazy pawn]

its still to have just in case the 1 out of 1 million comes out

[velvet thunder]

What what will happen? lol

Get message owner, ban him and delete the message

Or just... Delete the message

[hazy pawn]

there is https://www.youtube.com/watch?v=oCff80qSLUw&ab_channel=NoTextToSpeech

this

which im kinda afraid

[velvet thunder]

Lmao

[hazy pawn]

a simple logging ofc is not gonna cut it

[velvet thunder]

1. Afaik ntts mistakenly took random servers bans after new system as "new exploit" (which is most likely was a joke imo)
   Yes, exploit maybe existed in first 2 days, but if the exploit really worked like that, then it's just fucking ridiculous and you can't still be safe from embed method lol
2. Afaik discord just doesn't ban servers for bad content posted by users anymore, I heard it was a case in the past, but now I never heard it

[jaunty pewter]

i would take pretty much everything this guy says with a grain of salt

[hazy pawn]

LMAOOOO

[hazy pawn]

oki dokie

[hazy pawn]

well i kinda doubt the report system is good tho

like you get 100 messages every second being reported

what you are gonna check jefferey's humble server

and find "Oh its not a NSFW server and someone is false reporting"

meanwhile messages clutter and clutter

(i mean its not 1 person working ik but you get the idea)

[velvet thunder]

Discord sometimes actually learns from mistakes
iirc discord now doesn't ban you for "i'm 12" type of messages

i'm not 12 btw lmao

[hazy pawn]

ok

but its sometimes

the discord support was kinda mid

like its been for a while

[velvet thunder]

Because sometimes something cannot be undone or changed so easily

[hazy pawn]

true

i agree something cannot be altered easily and in the case of support it might cost money

[jaunty pewter]

discord probably gets thousands of support requests daily

[hazy pawn]

yeh

[velvet thunder]

So... Don't worry about random discord stuff lol
Think more about how to store as little info as possible to provide functionality efficiently and better to user

[jaunty pewter]

and a team of no more than 50 to deal with them

[hazy pawn]

they should scale

but its way

easier said than done

or

my approach to the problem

[velvet thunder]

I sent application for privileged intents and discord rejected my application in first hour, but I still don't have any info about my second application after a week

[hazy pawn]

could be to use ai as a actual tool and not develop some Clyde bot that is stupid

and what i mean is use for filtering messages

[velvet thunder]

afaik discord outsources moderation

[hazy pawn]

oh

i mean

there can be still hope

like it would be clever for them to have some malicious link detector

[velvet thunder]

Nah, it's impossible imo

[hazy pawn]

something like this https://www.ipqualityscore.com/documentation/malicious-url-scanner-api/overview

[jaunty pewter]

for dev support queries they basically always have to be manually reviewed and processed, and it's (probably) an entirely different team from t&s

[velvet thunder]

It's like anti-cheat problem

[jaunty pewter]

a much smaller team

[hazy pawn]

wdym?

[velvet thunder]

Treadmill problem
You can watch a really good gdc about that topic from valve

[velvet thunder]

If you create anti-cheat, people will create anti-cheat bypassers

[hazy pawn]

kinda reminds me of the mc java chat report

yeh its gonna snowball

true on that

[jaunty pewter]

yeah, the one where people freaked out about it and nothing actually happened

[hazy pawn]

i mean they kinda did for good reason. But bypasses(mods) were developed

[velvet thunder]

50/50
mc java chat report actually had A LOT of technical problems and with community help mojang fixed some of the most critical problems

For example... People actually found a way to report messages sent from a command block lmao

[hazy pawn]

but lets speak discord mostly

for the concered part of you

like storing messages without the user's permission

it can be a worse issue when you realise that the snapshot could be edited

like say i sent "Hello World"

"content": "Hello World"
``` the content would be this

if the admin knowns json

he can make it like

purely random example

"content": "One could imagine mcbrincie happy"

and when the admin regenerates the snapshot. The webhooks will see it and not recognise it was edited and thats not the original content

no worries as i have some solutions. Maybe even doing it in binary

like obfuscating it

that way it won't be too obvious what it is

perhaps hashes, base-64.....

[velvet thunder]

lol

someone can still send message with webhooks

and iirc even edit messages

You just can't trust bots and webhooks

[hazy pawn]

well

its kinda funny from one question

from going to json, then objects back and fourth

we went through a Sisyphus meme, minecraft, a malicious url scanner, the discord team.....

[hazy pawn]

tbh idk what do i do

[hazy pawn]

can i ask also a quick question

the discord.Member.global_name what is it exactly

like if the user has a nickname it returns a nickname?

if the user doesn't have a nickname it returns display name

its a lil confusing

[velvet thunder]

.pomelo

[native crypt]

Before Pomelo-

member.name          -> username
member.nick          -> guild nick name
member.display_name  -> member.nick OR member.name

After Pomelo-

member.name          -> username
member.global_name   -> global display name (global nick name)
member.nick          -> guild nick name
member.display_name  -> member.nick OR member.global_name OR member.name

[hazy pawn]

noice

thx

[velvet thunder]

.close

[native crypt]

Done with your help thread?

Please close your own help thread by using </close:1009144375709814897> with @proven yew.

Backup bot: </solved:1109625445990793246> (or .solved) with @native crypt.

[proven yew]

This thread was archived by the user that opened it.

[hazy pawn]

ik i shouldn't message even if the thread is archived, but quick note that i forgot to mention for the concern. The bot delivers a file that is encrypted with fernet within the cryptography module. And each user has their key which is unique from the rest and only the user that created snapshot can recreate it back into a channel, their key is not revealed to them and only exists within the bot's database and regenerates. So admins cannot have the ability to look for stored messages in the snapshot. But they can get revealed when creating a new snapshot which means they can kinda sort of preview the last 100 messages before the storage and pinned ones. But thats about it, old ones are kinda lost if the channel is deleted. The feature is mostly made for recovering purposes of lost channels so yeh

Again sorry if i have to open it a bit just for this message

[proven yew]

This thread was archived by the user that opened it.