development

[deft wolf]

Apparently it's a legacy thing

As far as I know, it was previously used to let the client know that the member has a nickname

[pearl trail]

oh, now it doesn't seem like it

[bitter granite]

Scam?

[pearl trail]

🤑

[stoic dock]

Hello, my bot was verified on top.gg and I'm setting up the voting system but it's my first time using webhooks in python

can anybody guide me / point me in the right direction, I'm completely new

[bitter granite]

@rugged dawn wanan learn

[rugged dawn]

-ban @high notch dropshitting

[gilded plank]

lo57888#0 was successfully banned.

[fair lantern]

[prime cliff]

Reworking my 2FA account page
Much better info and space usage.

Will have backup, auth app and passkey status under 2FA status for that empty space.

[wheat mesa]

man, you're productive af

[prime cliff]

Yup gonna get a beta release for this in January so :p

[wheat mesa]

My current company has had a project going for 4 years and it's set to release in production this january, and it's not even as good looking as this

It's probably more complex in the backend but considering I'm assuming you're the only one working on it, that's impressive

[prime cliff]

Yup

[wheat mesa]

Ours has had a team of 10+ working on this project for the last 4 years

Damn man

I need that level of motivation in my life

Blazor and ASP.NET?

[prime cliff]

Yup asp.net blazor with asp.net identity, mongodb, full 2FA app and passkey support with future oauth google/x/discord login.

[wheat mesa]

damn

I've been fiddling around with just google auth on my asp.net app for the past month lol

(Granted I've put like 4 hours into it in the past month, but still)

[prime cliff]

Month oh god

[wheat mesa]

Yeah

Haven't had much time between work and uni

But after my DSA exam on tuesday I'll be off on break from uni

So I'll work on it more

I think I'm starting to get the hang of more modern ASP practices

My work has poisoned me with using ASP.NET Boilerplate & jquery with MVC

[prime cliff]

I've switched to fully using blazor for all my stuff it's pretty nice to use and handy when you can reuse multiple components, dialogs, etc in a bunch of pages and isn't single modeled like MVC

[wheat mesa]

god single model is the bane of my existence

our viewmodels have 200+ props in some pages

[prime cliff]

🙂

[wheat mesa]

Yeah looks much nicer

I'd send a screenshot of a file but I think I could get in trouble for that

Let's just say, it's a constructor with 30+ params

For a single VM

[prime cliff]

Blazor components is similar with params especially if you use a component library they can get pretty wild

xD

[wheat mesa]

This project we're working on could've been massively simplified by using either modern blazor or a SPA framework like Vue

[wheat mesa]

that's still nothing tbf

this is blurry enough to send

https://cdn.discordapp.com/attachments/780497429707358258/1315383714389164122/image.png?ex=6757360c&is=6755e48c&hm=28241543ba39c1bb3dac2e19f9fdce29db0d7bfd5a7347430a1d8a4953a6e9bb&

All props

For one viewmodel

[prime cliff]

Lol

[wheat mesa]

I think the C# portion of our codebase is well over 200k lines

And honestly it does NOT need to be that complex

But honestly given that it's my first professional experience, I think it's good that I now know how to navigate a large and messy codebase

[long marsh]

For discord applications, I technically don't need a "bot" right? As it stands, my application is only initiated via slash commands and doesn't have a physical bot. Will this hurt me long term?

There are some advantages to having an actual bot affiliated with it: ie. can have a presence in the server with a status (for advertising purposes).

Before my app gets too popular (I'm still sub 100 servers right now), just thinking through having a bot or not.

[lament rock]

If your application doesn't need to do any gateway actions or respond to anything from the gateway against a guild, this is perfectly fine. For gateway actions, this is voice state update, presence update as you mentioned, and fetch guild members to receive GUILD_MEMBERS_CHUNK

[long marsh]

Gateway actions, in this case, are other things unrelated to receiving interactions and responding to them?

[lament rock]

Correct

[long marsh]

Thank you for your help on this.

[prime cliff]

@sharp geyser @wheat mesa Want to try out the first test release of my DevSpace project

If you already have a linux server with nginx and SSL cert already done.
Optionally cloudflare and docker.

[wheat mesa]

I don’t have a Linux server 😔

[sharp geyser]

How much space does it need

[wheat mesa]

I’d be happy to give it a try once I find a use case for a VPS

Atm though I have nothing to host

Though I have a REST API in the works which might be something to host

[prime cliff]

Not sure about space haven't fully tested that, but hey at least it's not electron

[long marsh]

Got it. So, I've built an application with a gateway that receives and responds to interactions. I have a "global" leaderboard and I don't intend on needing to fetch users, invites, webhooks, etc. I'm assuming I'll still be able to see how many guilds the application is installed into as well.

I assume since I'm still gateway-based, I'll need to implement sharding once I hit that threshold ... and not having a bot shouldn't prevent that since I have the guild intent?

[sharp geyser]

I will gladly test it builderb

send me all the info over

I use portainer rn

Always happy for an alternative

[wheat mesa]

I should learn more about docker

Idk basically anything

[sharp geyser]

I know enough to get by

[long marsh]

I'm a professional software engineer that deals with docker everyday ... it's definitely a must-have at the enterprise level.

[wheat mesa]

I haven’t even had the chance to work with it really

[sharp geyser]

I use it often

[wheat mesa]

I know what it is but I just don’t have any use cases for it yet :c

[sharp geyser]

I have found myself containerizing my stuff

[long marsh]

Do you have a bot?

[wheat mesa]

Not anymore

Was a pet project for when I was first learning

[lament rock]

So, you're misunderstanding the term gateway. By gateway, I was referring to a Discord websocket connection for real time events. Discord refers to this WS as the gateway.
Since you don't have one, you don't need to concern yourself with sharding. The only design considerations you must do is load balancing your HTTP interactions

[wheat mesa]

Haven’t done discord dev in ages

[sharp geyser]

Oh btw waffle

I got accepted to a uni

[wheat mesa]

Congrats!!

[long marsh]

You're certain that I'm confusing this? I'm also referring to the web socket connection for real time events. I haven't enabled the REST / HTTP event gateway either.

[wheat mesa]

Knew you’d make it

[sharp geyser]

I start january at one of their perimeter colleges to get my GPA up

I gotta get a 2.0 in 30 credit hours before I can transfer to the main campus

[wheat mesa]

Easy peasy

[sharp geyser]

I chose to do it online

[wheat mesa]

Online is so much better. I honestly hate in person college

At least for the uni I’m at

Not much to do

[sharp geyser]

I wanna go in person and live on campus

Experience the uni life

[long marsh]

I would definitely recommend doing so. (Sorry for butting into this conversation, haha)

[prime cliff]

Friend me Aaron

[wheat mesa]

Wish I could do the same, my college does not really have much of a “university” life tho

[long marsh]

Networking in person / making connections is what got me my first job out of college

[wheat mesa]

We’re in the top 10 most depressed colleges in the US list 💪

[sharp geyser]

Mine is n the heart of downtown atlanta

Uni life is every day

[wheat mesa]

God I hate networking

It feels like a front for “yeah I’m gonna use you as a stepping stone to get a job”

It just feels so wrong

[long marsh]

I used too as well ... but I've changed on it

[wheat mesa]

I mean it’s how you play the game

[sharp geyser]

Fuck morals I need a paycheck

[wheat mesa]

I just really dislike it

[long marsh]

This is a negative connotation to it, but you're not necessarily wrong.

I genuinely want to make a connection with the person to understand their perspective either in adjacent industries or by offering my own experience.

[wheat mesa]

Thankfully I landed my current internship without networking and my next one without networking 💪

[sharp geyser]

I am hoping to get my degree asap

[wheat mesa]

I’m fine with networking as in “I know you and worked with you, I’d love to give you a referral” or something, but career fairs and such just feel weird

[long marsh]

Oh yeah, agreed here. Career fairs are meh

They're great for talking to businesses though

[long marsh]

In what?

[wheat mesa]

Yeah

[sharp geyser]

Comp Sci

[wheat mesa]

Same

[long marsh]

Nice – that's what I got

[sharp geyser]

I did the dumb mistake of selecting the wrong degree on my application

hopefully they let me change it

🙏

[wheat mesa]

I’ll finish my CS degree 3 semesters from now, I’ll have completed college in 6 semesters total

[long marsh]

Are you a first year? Or are you coming in with credits?

[sharp geyser]

First

[sharp geyser]

Gawt dam

Remember me when you make bank man

[long marsh]

Yeah youre good ... the degree you selected on your application doesn't matter haha

[sharp geyser]

Dont forget your origins

[wheat mesa]

I get to work with Berry soon 😉

(Not really, same company tho)

[sharp geyser]

lmao

google?

[wheat mesa]

Chase

[sharp geyser]

Become a valve developer

[lament rock]

To receive interactions, you can either have them POSTed to an HTTP endpoint or over the Discord WS and those interactions will be received on the shard that covers the guild the interaction is from if any, otherwise they're pushed to shard 0 (I do not personally recommend this approach and would actively advise against it despite it being essentially free load balancing)

You do not need a WS connection depending on what data you need. If you NEED an accurate guild count within your code, you will need the WS to process GUILD_CREATE and GUILD_DELETE events. Otherwise, you can check on growth via the install app page or application directory listing on Discord if you apply for it there. If all you do is respond to interactions via HTTP responses, then do not add complexity with a gateway as the gateway can push A LOT of events and inflating the volume of messages can be expensive on the CPU assuming you try to put all shards onto the same machine

[sharp geyser]

annual average salary is 1.2m

[wheat mesa]

Nah my goal is eventually quant finance

(SWE at quant)

[long marsh]

Thanks for the breakdown. I haven't configured an HTTP endpoint, but I also don't have a bot. Am I cooked?

[lament rock]

How are you receiving interactions then

[long marsh]

If I go to my server right now, there isn't a bot at all for my application; however, I receive events via a gateway.

[lament rock]

Gateway as in the ws connection to Discord?

[long marsh]

yeah the websocket

Let me clarify I guess: I don't have a bot listed in the members. It does show up as installed in the guild.

I have done the REST way before and I needed to setup ngrok and configure a reverse proxy to hit localhost. With the way I'm doing it now with discordjs, I don't have to do that.

[lament rock]

So, the app doesnt need the bot presence to receive INTERACTION_CREATE for the guild it's installed to

[prime cliff]

Wut...

[long marsh]

Right

[prime cliff]

Presence intent means game/rpc data and online status

[lament rock]

but I was under the assumption you had not created a bot user in the app page

[long marsh]

I haven't

[lament rock]

In this context, it means appearing in the member list which only occurs when the bot is invited with the bot scope

[lament rock]

Impossible as you must have created a bot user to login to the gateway

[long marsh]

This is why I'm confused haha

[prime cliff]

You're confusing it by saying "bot presence" though just say that it's installed on the server xD

[long marsh]

I know what you're saying though

[lament rock]

The presence can be anything including offline

[long marsh]

Papi, if you had a second, I'd love to show you where my confusion lies.

But understandable if you don't.

[lament rock]

but it has to have the bot scope to actually push that presence

[lament rock]

It depends on how long this will take. I am about to play games with my girlfriend

[prime cliff]

Technically also yes/no if you have interaction url set it will appear always online 🙂

[long marsh]

I don't have an interaction url set

I don't have a bot and I don't have an interaction url set.

[lament rock]

Does it appear in the member list even if only authorized with application.commands?

[long marsh]

It doesn't for me

[lament rock]

So I was correct

[long marsh]

right

I'm assuming that I'll still need to shard though? But I won't know how many guilds my application is installed in?

[prime cliff]

It would still need the bot scope because it's not actually on the server or has roles to show in members yea

[lament rock]

It's only if the bot has posted application commands does it get that always online status. It must also have been authorized with the bot scope

Otherwise, no member list appearance, no presence, no gateway events EXCEPT INTERACTION_CREATE

[lament rock]

Yes you will need to shard now knowing that you receive events over the gateway. You must have 1 shard per 2,500 guilds

[long marsh]

To reiterate once more (to make sure I'm on the same thinking):

• My application is initiated through the INTERACTION_CREATE event through a WS Gateway.
• My application does not have the bot scope nor a bot in the members list
• My application does not have a HTTP interaction url set

Therefore, my questions are now:

1. How will I know how many servers my bot is installed into?
2. I assume I'll need to shard?
3. Should I truly consider the HTTP interaction method instead?
4. Should I consider adding a bot just for the hell of it / having a bot for later use?

[surreal sage]

https://cdn.discordapp.com/attachments/1258446506675929098/1311202536749011006/togif.gif

[lament rock]

Your points are fine. To address your questions:

1. You cannot currently unless you also authorize your app with the bot scope to guilds, have the GUILDS intent and process THE GUILD_CREATE and GUILD_DELETE events
2. Yes
3. The problem with receiving INTERACTION_CREATE over the gateway is that the shard is sometimes asked to drop its connection and reinitiate it which the library will handle on its own, no intervention from you. However, interactions expire after 2 seconds and if the connection isn't re-established within those 2 seconds an interaction is initiated, you won't receive the INTERACTION_CREATE until your shard resumes and trying to respond to that now expired interaction will return an API error. With receiving events over HTTP, you will not have any reconnects, but special setup is required for load balancing at scale. If you are content with interactions sometimes failing for your users, then continue on.
4. It depends™️

[delicate zephyr]

for the first point

• Afaik there is an endpoint on the public discord api (if your application is in discovery) to check the exact install count
• Otherwise you can get the approximate_user_install_count and approximate_guild_count via the discord api https://discord.com/developers/docs/resources/application#application-object

[delicate zephyr]

Point 3 is very valid, for interactions HTTP server based is a lot better unless you need presence based stuff

[lament rock]

I should have put exact count my mistake

[delicate zephyr]

it's a Kinda for exact count

since if it's on discovery you can fetch it via the api

[lament rock]

Let me check this as I was unaware of this endpoint. I may consider implementing

[delicate zephyr]

it may have super heavy ratelimits

i havent looked into it myself

I also know you can get the exact count inthe discord dev portal

https://discord.com/developers/docs/change-log#user-app-install-count

[lament rock]

I cannot seem to find it under Application or Guild where it would make the most sense so I guess it doesn't exist

[delicate zephyr]

May be undocumented then, so more than likely a no go

[long marsh]

Got it. I'd honestly use a serverless approach with API Gateway + Lambda. At scale, I could get around those limitations. And the load balancing would still be an issue with gateway WS at scale as well

I'm honestly fine with the interaction create failing every so often. As long as it doesn't fail over 0.5 - 1% of the time.

[delicate zephyr]

there's also the fact gateway will use up more resources

assuming you're using JS

[long marsh]

Which gateway?

REST interaction or WS?

[delicate zephyr]

WS

[long marsh]

Ah true, agreed

I've tried my best to disable all the cache though

[delicate zephyr]

Only real benefit for WS is if you want a bot in the future its not a pain to add it

if you're going interactions only

realistically using HTTP is the best way to go

and thats from experience, the WS is great for presence based stuff, anything other than that the API can generally do

it's literally just scaling a web app tbf

[long marsh]

I believe, so far, that I'm exclusively going to use interactions only ... but I can't be certain of that. I do have plans to send messages to servers that are uninitiated from an interaction.

[delicate zephyr]

you can do that without WS

[long marsh]

I didn't know that!

[delicate zephyr]

the WS literally only sends you events, you send nothing back really

[long marsh]

ahhhh

Gotcha

That just clicked

You're still responding via REST to discord to complete the interaction?

[delicate zephyr]

You create an API for discord to call

[delicate zephyr]

yup

[long marsh]

Gotcha.

hmm...

Good to know.

[delicate zephyr]

its literally either REST <- (WS -> REST)

[long marsh]

got it.

[delicate zephyr]

or REST <-> REST

so

[long marsh]

Yep

[delicate zephyr]

if you're going interaction only mate, i'd go HTTP

also means you can restart with literally 0 downtime easier

since you dont need to wait for identifies

especially at scale

[long marsh]

Hmm... good point. I'll consider that for sure. For local development testing, I assume you simply use ngrok or something?

[delicate zephyr]

Yea, or you just setup cloudflare zero trust with cloudflare vpn

[long marsh]

Let me ask you one more thing:

• HTTP latency can be slower than WS in my testing with other applications.

Do you see that? Or not really?

Also, if I'm doing HTTP, I understand that I can literally use anything in the world; however, can I swap over my current logic using DiscordJS to interaction-based url?

[delicate zephyr]

scaling HTTP with discord is also 10x easier

for example imagine you have 4 shards

in order to scale from 4 -> 8 with zero downtime (has to be double as to scale shards properly you need 2*(current shard count) (2 for every 1 existing)

you need a way to store you cache externally from the library. then have a way to ignore events temporarily until the new shards are ready and then kill off the olds ones as they're spawning

with HTTP its literally just add new process and include in loadbalancer done

[long marsh]

And with serverless-based architecture, it's even easier.

[delicate zephyr]

exactly

[delicate zephyr]

depends, it's negligable and depends on server location just like WS

could also just use cloudflare really to mitigate that afaik

[long marsh]

Got it. You use DiscordJS or no?

[delicate zephyr]

Have done

I hate the lib but it's the simplest to use

[long marsh]

Ironically, I used GO originally and switched over to discordjs lmao

[delicate zephyr]

so cant bash it

GoLang is cool

[long marsh]

I used GO with the HTTP endpoint

[delicate zephyr]

there's a learning curve but it's mint

[long marsh]

GO is awesome, simply put

[delicate zephyr]

going from JS to Go is honestly hilariously easy tho

[long marsh]

The tooling is phenomenal around the language

[delicate zephyr]

yea agreed

the only reason I wont bash Discord.JS is cause it's built around people learning

[long marsh]

I'm a professional software engineer haha ... going from JS to any language is trivial at this point. Unless we're talking functional ... it's a whole different ballgame

[delicate zephyr]

Same, just trying to get a job to be able to immigrate from the UK to germany

Had a shit year last year caused me to lose my job in january, only just got back on my feet

so it's a fucking ballache atm cause no one is really hiring lmao

[long marsh]

Oh wow! How is the job market out there for software engineers? Are you a software engineer?

[long marsh]

Ah nvm

[delicate zephyr]

I'm in the UK atm

it's finding someone willing to help me immigrate

like I wanna move to be able to live with my partner

[long marsh]

I know an employer in the UK, but not sure I can help out with the referral process as much

[delicate zephyr]

Mate, I'll send my CV to you if you want and you tell me if it's worth me sending them an email

Any shot is better than no shot rn

[dry imp]

anyone can help me with this? been a while since i touched next.js, i think this only happened on the newer ver of next.js, it worked before i updated it

[delicate zephyr]

you need blurDataURL it applies a blur to an existing image so that it can fetch the image as the page loads

[dry imp]

but shouldnt it automatically create one if its a static import?

i mean it works on the older version

[delicate zephyr]

if it's a static import

then absolutely no idea

[dry imp]

like its so weird, i cant even do this anymore

[delicate zephyr]

checked the docs?

[dry imp]

i didnt see anything useful to fix the issue, like i think that should work but somehow it doesnt work based on the docs

[delicate zephyr]

https://nextjs.org/docs/pages/building-your-application/optimizing/static-assets

this should help

if it's in your public folder

then you can use Next/Image

[dry imp]

oh right

missed that part

[delicate zephyr]

and just do /<url> where url will be converted to /public/<url> internally

hopefully that helps

[dry imp]

nvm still doesnt work omg

[solemn latch]

React 19 really changed things

[dry imp]

ah shid

[solemn latch]

The way a lot of things were handled are no longer the recommended way. It's a good time to catch up before working on anything serious.

[dry imp]

ahhh okay

ill try to learn it first then before blowing my head figuring out whats wrong

[tacit estuary]

Anybody here struggling to switch over to slash commands since apparently discord wanted to make ctx and interaction completely different and you can't simply change the words and have to go through every instance of ctx and carefully change it? Not a big deal except for very large programs.

[lyric mountain]

It all depends on how you structure your handlers

If you're feeling something requires too much manual fiddling then it might be better to rewrite it and save yourself time in the future

You can pretty much make both message commands and slashes compatible with eachother by making a compat layer

[tacit estuary]

Alrighty. I use python and every program I have is simple enough to change except the massive almost 2,000 lines game. I do like the idea of just rewriting it and copying/pasting the stuff along.

[neat mauve]

mhhm just recently I gave someone a proper explaination about slash commands and gave them a simple structure to build onto it. even though its not many lines, it took like 2 hours to explain everything and all pitfalls. But they were starting out and I immediately redirected them to slash commands over prefix stuff.

I'd suggest you to build a simple slash command, the simplest you can get... and then just try to translate each of your prefixes one by one... it all probably depends on how you implemented it in the first place.

[long marsh]

Does anyone here use the HTTP interaction endpoint option vs. the web socket?

I'm curious how the latency is vs the websocket.

This is how it is currently is if I'm understanding correctly:

Discord Webhook -> (WS -> Discord API)

This is how it will be after the HTTP interaction:

Discord Webhook -> API

The critical difference here is that discord doesn't require a callback for the interaction response on the HTTP interaction (like it does with the WS). I'm curious if this will actually lessen the latency.

[lyric mountain]

latency will be effectively the same

as it's a function of distance to the server

[prime cliff]

Not the same, websocket is low latency so it is faster also you have to factor in other issues like not having a cache so you have to specifically fetch data which adds more delay

@long marsh if you have an ok enough vps server that dosen't have network issues contabo you wont really notice the delay much

Websocket will always be faster at responding though so it depends if your bot needs cache/events or just standard messages and data

[long marsh]

That's the interesting part though... I think HTTP interaction will be faster since I don't have to make a REST callback to Discord. The response from the HTTP interaction is enough.

[lyric mountain]

also you have to factor in other issues like not having a cache
this is on programmer's side, unrelated to connection type

[long marsh]

Agreed, I'm strictly referring to network latency.

Discord Webhook -> REST API ✅

vs.

Discord Webhook -> WS -> Discord API

[prime cliff]

Actually the rest call depends on how you want to use it because if you need loading state (more than 5 seconds) you will have to respond once for loading and second time for actual message.

You also need to make http calls to fetch certain data

If you have a low ram vps or for example an azure function app that loads on demand then interaction will be the best choice.
If you need or frequently use cache data like fetching other members/servers/channels or actual server/message events like mod bots then websocket will be a lot better.

[long marsh]

For my case, I typically won't need too

Is it 5 seconds?

That's easy

[prime cliff]

Ah wait nvm it's 3 seconds

[long marsh]

3 seconds is a little tighter, but I'm averaging 200ms - 500ms per call over the WS

And that's with the the callback to discord.

My particular setup:

Discord -> API Gateway (~100ms) -> VPS (~200ms)

[frosty gale]

discord developers having to rewrite their entire bot every year because discord changed something

new year new bot aah

[neat mauve]

what network problems does contabo have? i see people complain about it here all the time... I have gotten good experience with them so far

[prime cliff]

I tried contabo before had network issues and latency not really good for latency sensitive apps

[neat mauve]

works fine for my bot :c

got some other gameservers running there... it may gets occasionally (every 5-30 secs) bad latency of up to 5000ms, but this might be due to packages lost and the game calculating it awefully. though idk really.

[lyric mountain]

contabo had a some troubles regarding infra in the past where they often oversold servers, often leading to busy neighbor problems

nowadays they're mostly fine

[neat mauve]

thanks for the infos

[long marsh]

Thanks everyone for helping me! I switched over to the HTTP interactions endpoint and have seen drastic improvement!

Makes for easier development + infra

[delicate zephyr]

Nice!!

glad to hear it worked out well

[lament rock]

Was only a matter of time tbh

If you can eliminate your dependence on the gateway, do it lmao

@neat mauve If you ever consider contabo, ensure it isn't for any critical infra as my bot once suffered a major outage which lasted for 9 hours and support was very unresponsive and very unhelpful when they did actually respond

I had multiple VPS' within the same datacenter and my main bot cluster's VM was experiencing the outage. So, nothing insane like internet backbone failure was occurring or their datacenter wasn't on fire. When support responded, they said everything was fine on their end and they found my VPS running perfectly fine. (It was still down and unreachable and I hadn't made any changes or touched the VPS in a while)

Otherwise, I hadn't had any other major issues. Insanely good VPS for the price point. Hadn't had any issues with neighbors. The remote management is clunky at best but serviceable.

[neat mauve]

thanks for the infos! 😮

I will watch out to not go down that path. even though they seem to have gotten better?

[neat mauve]

Yeah, thats true and my main concern currently... getting good value for price... even though i can afford it, but its a running cost, which i want to minimize and everyone needs a couple srvers... so it will only get bigger ':D

Before contabo i had a different host for around double the price and i think only 2/3 the power...

[lament rock]

Depending on what you need, I purchased myself an orange pi 4 or 5 idr which, but it is an 8 core ARM chip with 16GB of SOC memory. And those can be had for around $100 USD at the time I purchased it. You shouldn't run any websites off your home network, but I host my private bots off that thing and it has some power

Though it should be mentioned that I try to optimize the crap out of my bots

[river crypt]

K

[neat mauve]

mhhm I see... I have also gotten a lot of pi's... But havent touched them since years... Have shut down all of them... Thats due to my kids have been messing around with them and i wasnt able to get them to a place where they would be safe to setup.

They getting older now though, but still not safe enough... maybe in 1-2 years I can go for it again.

But thanks for this awesome reminder! I should definitely shift to selfhosting in my raspis

[amber rose]

Why is discord.js preferred over discord.py?

I can already see the hatred from people reading this

[deft wolf]

I don't understand in what sense it is more "preferred". These are two different languages ​​and two different libraries

If someone chooses javascript, then discord.js is preferred because it is the most popular library and is rather accessible to novices (and you can't really use discord.py in your javascript project)

[solemn latch]

I think js is just preferred over python

Space based syntax is awful

[lyric mountain]

and you can't really use discord.py in your javascript project
not with that attitude

[earnest phoenix]

Because some people prefer JS to Python

[amber rose]

:/

[earnest phoenix]

Can't compare something that isn't used for the same purpose or by the same target audience

[wheat mesa]

I despise both javascript and python!

[solemn latch]

good choice

[lament rock]

This is an incredibly sensitive topic as developers have their preferences in their language and what their libs offer and how they're offered. Both JS and PY are high level languages meaning beginner friendly/the languages do the heavy lifting of safety. If we compare both discord.js and discord.py for what they are versus what their underlying language offers subjectively, I would say your answer is too broad. A better question would have been why it's preferred to beginners.

Discord.py takes a more functional approach while discord.js takes a more oop approach and oop tends to make more sense to beginners until they can grasp the concept of types. Functional programming can be more powerful as sometimes information is hidden behind too many layers of abstraction, but it comes at the cost of a higher learning curve.

Discord.js might actually be safer in terms of performing actions against Discord as compared to discord.py, what I mean by this and I will use an example I have personally experienced is that discord.js will not emit reaction events like reaction add or reaction remove unless the message reacted to is cached or partials is enabled which allows users to work with partial Discord data.

Discord.js has documentation which is very beginner friendly.

That's about it imo

[remote mango]

finally own bot monitoring what you guys think, is a good idea? i would be happy to get some feedback. (i mostly did it for me private, to get better overview for all my own bots). and if a bot crashes, u get a Message directly on Desktop ^^ 😛

https://media.discordapp.net/attachments/110374153562886144/1316197921619710003/image.png?ex=675a2c56&is=6758dad6&hm=95a8411ab79dc01f802db28a71d5f03ee0bef285726333fc1e0e79e67fd49767&=&format=webp&quality=lossless&width=839&height=630

[solemn latch]

I prefer using email for private status checking, just so you get notified on any device. But still really neat ^-^

[remote mango]

Yea thats actually fine with email, but a Bigger Monitoring isnt bad at all haha, Btw i got a problem, well more a thing i didnt read carefully - Tokens can't show a status Offline or Online (directoy for bot), well do you got any idea how i can do it ?

I mean i dont want to setup a bot, which checking the statuses always for my own bots. and simplier idea?

[lament rock]

You cannot see the online status of bots unless you share a guild with the bot. The only other way would be bot devs to use an API to submit their status or lack of status

[remote mango]

Yea thats what i actually thought, but im to lazy to make api for my own bots. i just thought there were easier ways haha.

But i mean, creating a bot, which is in the same as the rest of my bots, can be easier to catch the on or offline statuses.

[tacit estuary]

I use discord.py I think and I am really new at it even though I been using python for over 3 years and even took a college class for intro to python programming. I still struggle with understanding "files" in python, but I'm able to figure them out. I still struggle badly with "class" and "discord cogs". Also is it possible to use both prefix and slash commands cause I want to switch over to slash commands, but I can only change the basic commands and not the 1,729 lines in one major program.

TLDR: I'm still a newbie at coding, but I got basic python down pretty nicely.

[amber rose]

You can use hybrid commands

System for both, prefix commands and app commands

I think classes are just object Oriented programming for Python, think of it like this, a class that makes cars for you, you can give the car some values, and you can add functions to the classes, so when you use the car object, you can tell it to drive or paint it, that's why I love OOP.

[neat mauve]

https://discordpy.readthedocs.io/en/stable/api.html

[lyric mountain]

"bot account" and "token" are 2 things that dont belong there

the former will always be true, unless you plan to selfbot which would get you banned

the latter not only is irrelevant, but you're literally removing all safety layers protecting the tokens

[bitter granite]

That bot token is so unnessesery

[harsh nova]

you have a lot of free horizontal space. Could probably space out the information within that space to make it easier to read

[remote mango]

Thsnks for the feedback guys, ill change few things👍

[jagged egret]

Hello

[frosty gale]

contabo users reaction after getting hit with fees and taxes right before you pay for a server

so many times i went "thats actually a good deal" before i got to the checkout page 💀

[solemn latch]

you mean the setup fee?

[sharp geyser]

tbh, the price for contabo isn't bad

the servers are just kinda shite

It's still hell of a lot cheaper than ovh, do, vultur, and those other mainstream ones

[lament rock]

GalaxyGate has an unmetered connection which my bot needs

$22/m isnt super based though

[spark flint]

what specs?

[lament rock]

print screen is funky in vr sry

[spark flint]

Mid specs

[lament rock]

its alr

[sharp geyser]

idk man

I cant warrant spending 20$ for so little ram

Then again most of my servers are game servers

[solemn latch]

you also dont really want to host game servers in new york. too expensive

[sharp geyser]

yea

if I host game servers its in central locations

[bitter granite]

I use contabo for like 3 month
Its not the best imo

[sharp geyser]

For its price

you get what you pay for

I buy em when I need to quickly host something for testing

[bitter granite]

Its not the quality that is the issue for me
The issue for me is that they ask my id every start of The billing cycle

[delicate zephyr]

I had this for a while, emailed support and they said "oh okay" and stopped asking

[bitter granite]

is this dumb? just wanna know

[pearl trail]

hey, will discord's timestamp convert utc to each user's timezone?

[sharp geyser]

Uhm

I would say so

If you find yourself doing the same query multiple times in different places, i'd think about if you need to start making helper methods

rather than "bandaging" the problem

[bitter granite]

Whats that Btw?

[sharp geyser]

Whats what

[bitter granite]

Helper method

[sharp geyser]

Its just abstracting stuff out into functions that you can call multiple times to accomplish the same goal

So rather than writing
db.findOne({ user: "1234" })
multiple places, and doing all your like validations and what not multiple times

you'd bundle all that into a function

What they are doing, does not solve any real problem. All it does is group together their query strings

[radiant kraken]

no iirc

[radiant kraken]

yeah like how my helper trait failed horribly

sobbing

[sharp geyser]

Nah

It helped

[radiant kraken]

nuh uh!

[sharp geyser]

traits are just limiting

💀

[long marsh]

Is there a way to @ a user via their discord id?

I'm implementing a global leaderboard, but don't store the user name of the player :/. I assumed it would be accessible without having to make a call.

[sharp geyser]

Yes

<@ID>

[long marsh]

thank god

Will that work even if the user isn't in the guild?

[sharp geyser]

Nope

It will show up as unknown-user

[long marsh]

So for "global" leaderboards, I need to store the username of the user?

[sharp geyser]

Yep

Careful though

Displaying their username and such globally isn't always the best idea

Make it an opt in/out

[radiant kraken]

oh right yeah

[long marsh]

Interesting. I've seen plenty of games showcase leaderboards of user names globally. By what you're saying, this is discouraged now?

I assume due to data privacy reasons?

[sharp geyser]

Well

Typically they have it opt in

Some people don't want their usernames on blast for people to add em

even now since discord got rid of discrims

[long marsh]

Gotcha. So is there a recommended process for this? I assume I shouldn't showcase a leaderboard of just numbers?

What is typically out in the wild?

[sharp geyser]

You can still do usernames, but like I said I personally advise an opt in/out so they can choose to stay off it

Then again its not really against discord TOS, so you can do what you want

[long marsh]

Damn this leaderboard is a pain in the ass so far haha

I could start out by showcasing where their personal rank is and what the top numbers are by other players...

[long marsh]

Thanks for your help!

[bitter granite]

also they usualy have it in their Tos and PP

[long marsh]

I think, for now, I'll just tell the user where they're ranked (if they're top 5,000) ... then display the leading scores.

[pearl trail]

alright thanks

[bitter granite]

np

[tacit estuary]

I made a leaderboard/top scores on my bot, however it's not globally and only specific to every server. And yes, they are stored as id numbers and converted back to names when a user checks the high score listings.

For globally, if I did such a thing, I would add a special "0 or 1" number to their data, that way if a user does not want their names to be shown globally, they can have it not be shown and the number will change to 1 indicating their name will not be shown.

I hope that helps. I'm kinda a newbie, but I been working heavy on high score listings and got it down pretty great.

[bitter granite]

If you do, please do not show server in global

[tacit estuary]

Thanks. I am aware of that. Keep it minimal.

[small tangle]

Hey @radiant kraken i wrapped some external errors in my rust program into my error type and convert them with the From<> trait, so far it works good. But now i want to write unit tests but im running into an issue that i cant create that external error type for comparison since one if its struct fields are private, any idea how i can solve that?

enum AnalyzerError {
    // wrapped errors
    IOError(ErrorKind),
    LexerError(LexerError),
    ParserError(ParseError<usize>),
}

And one of the fields of ParseError is private .-.

assert!(matches!(error, AnalyzerError::ParserError(..)));

i found that and matches! doesnt care about the concrete values but that doesnt feel right

[radiant kraken]

make them pub(crate)

pub(crate) is private, but public throughout the entire crate's scope

[small tangle]

But ParseError is a type of an external lib i dont have control over that do i?

[radiant kraken]

oh

[radiant kraken]

also, that's not a struct, that's an enum

[small tangle]

oh true

Because i created a test file with an expected parsing error and want to compare the return value of my parsing function against the expected parser error

And i dont see a way of how to create that expected ParseError

[radiant kraken]

why is that so?

i mean, it works

[small tangle]

Why is what so?

[radiant kraken]

you just want to assert that error is AnalyzerError::ParserError, right?

[small tangle]

Yes and ideally the fields like reported location and expected token

[radiant kraken]

wait

like how

[small tangle]

one sec

[radiant kraken]

can you show me the ideal assertion?

[small tangle]

// arrange
let expected_parser_error = ParseError {
    location: 5,
    expected: ExpectedSet {
        expected: // this is private...
    }
}
let expected_error = AnalyzerError::from(expected_parser_error);

// act
let error = parse_tokens(tokens).unwrap_err();

// assert
assert_eq!(error, expected_error);
assert_eq!(error.location, 5);
assert_eq!(error.expected, Token::LeftParenthesis);

ideally something like this if that makes sense

[radiant kraken]

oh i seee

so, expected is private in what terms?

[small tangle]

#[derive(PartialEq, Eq, Debug, Clone)]
pub struct ExpectedSet {
    expected: BTreeSet<&'static str>,
}

its defined like this in the lib

[radiant kraken]

is BTreeSet a private struct in your lib?

[small tangle]

No thats all from the lib

[radiant kraken]

oh, ExpectedSet is also a part of a third-party library?

[small tangle]

Yes

[radiant kraken]

hmmmm

can i know what lib it is?

[small tangle]

https://docs.rs/peg/latest/peg/

Its a parser generator lib

[radiant kraken]

huh, how does ExpectedSet match with Token::LeftParenthesis

[small tangle]

oh that was more sudo code

[radiant kraken]

what do you mean?

[small tangle]

like "along the lines of" instead of concrete implementation

[radiant kraken]

i may need to know how you do that

[small tangle]

do what?

[radiant kraken]

this

[small tangle]

let expected_parser_error = ParseError {
    location: 5,
    expected: ExpectedSet {
        expected: [Token::LeftParenthesis].into()
    }
};

i guess that would work?

[radiant kraken]

ah icic

how do you... do that

expected is a private property in ExpectedSet

[small tangle]

Yes thats still my problem

That doesnt compile

[radiant kraken]

have you tried using tokens() and matching it with (?

[small tangle]

uhh no

[radiant kraken]

try it

that's probably one of the only ways you can access self.expected, albeit indirectly

[small tangle]

but i still need an expectedset instance no?

[radiant kraken]

OH

uhm

what about assert_eq!(error.expected.tokens(), "(");?

oh wait

its an iterator

assert_eq!(error.expected.tokens().collect::<Vec<_>>(), ["("]);
``` try this

you should not need a separate ExpectedSet instance if it's like this

[small tangle]

the error of the parse_tokens returns an AnalyzerError tho

[radiant kraken]

can't you retrieve the inner error from AnalyzerError?

[small tangle]

Not currently

I need to implement that then

[radiant kraken]

you should add a way to retrieve it

yeah

if you dont want to make it public, just make it pub(crate)

[small tangle]

The thing is AnalyzerError contains both external wrapped errors and my own errors how would that work with such a function?

[radiant kraken]

https://doc.rust-lang.org/std/error/trait.Error.html#method.source

it uses a dyn reference tho, but i think it's possible

[small tangle]

impl Error for AnalyzerError {
    fn source(&self) -> Option<&(dyn Error + 'static)> {
        match self {
            AnalyzerError::IOError(error) => Some(error),
            AnalyzerError::LexerError(error) => Some(error),
            AnalyzerError::ParserError(error) => Some(error),
            _ => None,
        }
    }
}

i did it like this now

[radiant kraken]

mhm

i don't think you need the _ => None tho

unless your AnalyzerError was marked with #[non_exhaustive]

[small tangle]

but the match isnt exhaustive otherwise?

[radiant kraken]

i don't think so...

[radiant kraken]

but this... 😭

[small tangle]

oh that was only a part of my type, thats only the wrapped errors. Sorry for the confusion

[radiant kraken]

oh, my bad hahaha

then yeah, your code looks good

[small tangle]

But i would still need a way to retrieve the ParserError variant specifically

[radiant kraken]

one sec

@small tangle try this! ```rs
assert_eq!(error.source(), Some(&inner) if inner.location == 5 && matches!(inner.expected.tokens().collect::<Vec<_>>(), ["("]));

[small tangle]

my ide says thats not valid syntax

[radiant kraken]

wtf which part 😭

oh wait

i think i got this wrong

try again @small tangle

[radiant kraken]

wth

it worked in the rust playground

wait

maybe it just doesn't work with assert_eq!?

yup

@small tangle try making it ```rs
assert!(matches!(error.source(), Some(&inner) if inner.location == 5 && matches!(inner.expected.tokens().collect::<Vec<_>>(), ["("])));

[small tangle]

inner is a dyn Error tho, not a ParserError

source() returns that

[radiant kraken]

hmmmm

[small tangle]

And i realized another issue My AnalyzerError derives PartialEq for the asserts but the wrapped std::io::Error doesnt implement that

[radiant kraken]

what

😭

[small tangle]

i dont want no more

[radiant kraken]

wtf 😭

[small tangle]

okay i removed the PartialEq from my ErrorType but now need to write my tests like that:

assert!(match error {
    AnalyzerError::RepeatedAcquisition {
        line,
        lock_id,
        thread_id,
    } => {
        assert_eq!(line, 7);
        assert_eq!(lock_id, 9);
        assert_eq!(thread_id, 7);

        true
    }
    _ => false,
});

[radiant kraken]

what about this? ```rs
assert_eq!(error, AnalyzerError::RepeatedAcquisition {
line: 7,
lock_id: 9,
thread_id: 7
});

[small tangle]

uuh

[radiant kraken]

WHAT

😭

OH

it's assert_eq

my bad

i am so sorry

😭

[small tangle]

No, for assert_eq i need the partialeq trait

[radiant kraken]

OH

dsfsdufhsudfhdf

oh screw you std::io::Error

[small tangle]

I appreciate your effort tho

[radiant kraken]

i dont wanna make u sad lol

[small tangle]

No u dont

ooooo null

assert!(match error {
    AnalyzerError::ParserError(inner) => {
        assert_eq!(inner.location, 3);
        println!("{:?}", inner.expected.tokens().collect::<Vec<_>>());

        assert_eq!(inner.expected.tokens(), ["("]);

        true
    },
    _ => false,
});

the first assert is working

now i need to figure out the tokens

[radiant kraken]

SLAY

so glad

😭

the solutions are ugly but ig at least it works??

[small tangle]

yea

all bc it doesnt implement PartialEq

[radiant kraken]

stupid io::Error

[small tangle]

I mean i understand why thats the case but it still sucks

[radiant kraken]

mhm!

[small tangle]

But thank you very much for your time

[radiant kraken]

i am so sorry if i am not much of a help today 💔 hahaha

[small tangle]

No, youre good

Im happy that i can ask you such questions

[radiant kraken]

i'm very glad!

feel free to ask me anything

[small tangle]

Thas so weird, inner.expected.tokens().collect::<Vec<_>>() returns ["[LeftParenthesis]"]

[radiant kraken]

well... then match it with ["[LeftParenthesis]"]!

lol

[small tangle]

assert!(match error {
    AnalyzerError::ParserError(inner) => {
        assert_eq!(inner.location, 3);
        assert_eq!(inner.expected.tokens().collect::<Vec<_>>(), vec!["[LeftParenthesis]"]);

        true
    },
    _ => false,
});

[radiant kraken]

SWAGGG

[small tangle]

Ahh the inner [] comes due how the parser macro works

rule event() -> Event
  = [ThreadIdentifier(thread_identifier)] [Pipe] operation:operation() [LeftParenthesis] operand:operand() [RightParenthesis] [Pipe] [LineNumber(loc)] {
    Event { thread_identifier, operation, operand, loc }
  }

you define the parsing rules like this

and then ofc the parser reports [LeftParenthesis] or at least thats my guess

Enough rust for now

[radiant kraken]

interesting!

[radiant kraken]

it's alright! hahahaha

[toxic trellis]

Could anyone help me I can’t figure out why my bot can’t read messages

[deft wolf]

The most common reason is that message intent is not enabled

[sharp geyser]

https://tenor.com/bnsEx.gif

[toxic trellis]

😭

[deft wolf]

And in your code?

[toxic trellis]

What u wanna see exactly

[deft wolf]

I want to see if you have message intent enabled in your code

[toxic trellis]

U mean this?:

intents = discord.Intents.default()
intents.messages = True
bot = commands.Bot(command_prefix="!", intents=intents)

[deft wolf]

You are missing message_content intent

[toxic trellis]

Oh where should I put it?

[deft wolf]

Just add

intents.message_content = True

[lyric mountain]

you shouldn't enable all intents unless you need them btw

[toxic trellis]

Alr

[lyric mountain]

presence intent is very heavy on bandwidth

[toxic trellis]

I just tried every button there to see if it would’ve fixed it

Wdym with it?

Thx

[spark bobcat]

I have two bots with 100 servers, but they are unverified as of now. I want to ask about how I can develop them further.
This isn't about developing/debugging code for bot, but about the general development of the bot.
To verify, each bot is required to have a privacy policy and a TOS.
I am thinking about buying a domain and having a subdomain for each of the bots, which will host their specific home pages, dashboards, privacy policy and tos.
I would also want to monetise them and would like advice/tips on that.
I understand that this channel may be about development only, and would delete this msg if so.

[lyric mountain]

well, first step would be to get them verified

[spark bobcat]

that requires tos and privacy policy, which i plan to host on the website

[long marsh]

How did you do a server-based leaderboard?

[lyric mountain]

you can have an api endpoint or use a github repository to host it

doesn't need to be anything fancy, just accessible

[spark bobcat]

true

[lyric mountain]

breaking the 100-limit allows it to keep growing while you work on the site/monetization

[spark bobcat]

yes

ok i will make a temporary tos and privacy policy and host it on github to get the bot verified

i can change the link to them later right?

[lyric mountain]

yep, on the dashboard

[spark bobcat]

ok thanks

how should i go about making a tos/privacy policy?
i was thinking about finding a gpt related to legal stuff and ask it to generate somethings
and modify those until they feel like they cover everything

that and reading tos and pp of other bots

[lyric mountain]

it's not a good idea to let stuff generate PP and ToS for you, there are a bunch of tutos online on how to write one

PP is basically to protect your users, ToS is to protect you

[spark bobcat]

ok, i'll research more on that then

[spark bobcat]

this should help with that right?

[lyric mountain]

yes, for sure

browse some open-source repositories to see how they wrote theirs

[spark bobcat]

another question: ik this would depend more on which country im in but
i want to make multiple more bots and monetize them as well
and i would want it to be clear that these bots are made by the same entity, but i dont want my name associated with it

could i just use a random name?

like could i just make a website called jojosbots.com and have multiple bots on there, which are monetised

i cant monetise through discord cause its not available in my country yet

[lyric mountain]

don't think I can answer that, would require a better legal knowledge

[spark bobcat]

and would probably depend on my country and its laws i assume?

[lyric mountain]

likely

[spark bobcat]

you have a bot yourself which has a website

have you monetised it?

[lyric mountain]

not yet, as I haven't found a decent service with discord integration that supports my currency atm

I will, eventually

[spark bobcat]

oh okay

[earnest phoenix]

discord markdown parsing 101: prevent people from sending spoilered codeblocks that contain || - Done 👍

||```
left || right

[astral robin]

Thank you to everyone

[deft wolf]

Ok?

[astral robin]

No

Dead inside

[sacred sand]

hi

[astral robin]

Hello how are you

[sacred sand]

iam good

[astral robin]

That’s great

[sacred sand]

were are you from

[astral robin]

I’m from Cambodia 🇰🇭

How about you?

[sacred sand]

iam from Egypt

[astral robin]

Nice to meet you

[sacred sand]

me to

[astral robin]

Welcome to our server

[sacred sand]

thanks

what are you want to play

[astral robin]

Excuse me!

Play what

More details please

[astral robin]

My pleasure

[slender wagon]

So guys i figured out something weird when fetching a certain api.
I made a simple fetch request using bun and the same request using reqwest from rust.
The rust one ends up blocked by cf but not the bun one? Do u guys have any explanation of this

[brave shuttle]

reqwest does not handle cf challenges, bun does

[slender wagon]

do u happen to know similar libraries for rust 👉 👈

[brave shuttle]

idk

but try using cloudscraper

or adding the user-agent

and allowcookies

[slender wagon]

im already using fantoccini but it is way slower than what bun does

and i basically tried everything with reqwest

[brave shuttle]

In Rust there is no library for that at this moment

python handles cf challenges

automatically

[slender wagon]

never touching python

[brave shuttle]

or try using a headless web browser

like

playwright

[slender wagon]

i jumped into rust cuz it is way faster than bun

for other stuf

[slender wagon]

im literally only fetching api's that's too much

[brave shuttle]

soo

yeah

python

[slender wagon]

nah, i'll do my calculations again for what my app does

and see the overall speed of bun

vs rust

currently on a small vps for the average regex functions it takes like 8ms

[slender wagon]

rust

or i could try mixing them up

but i feel like that would be too much

[brave shuttle]

you could try mixing cloudscraper from python and integrate it on rust

[slender wagon]

oh

but that's for advanced cf challenges

mine are just stupid

[brave shuttle]

i wish u luck

[slender wagon]

ty fr fr

[long marsh]

I had this conversation yesterday, but wanted to circle back to make sure.

When building out my global leaderboard, I need some way to identify users. What should I do? Can I display the usernames of individuals without too much trouble? For now, I've only stored ids.

Maybe a shortened version of everyone's username?

[vivid fulcrum]

you should only really store ids

when displaying your leaderboard you can fetch the users via REST and (suggested) keep them in a cache

[solemn latch]

Usernames change, ids dont

[long marsh]

That's fine for my case, I'd just refresh it upon the next update

[long marsh]

Let me think this one through...

1. A user "clicks", the player cache is updated, then the redis sorted set is inserted into (leaderboard in this case).

Upon fetch of the leaderboard, do a REST API call to the discord API for each username? How long would I keep the username's in cache for instance?

The leaderboard is updated every 2.5 minutes .. would I then do a bulk cache invalidate / revalidate for the leaderboard every morning or something?

[lyric mountain]

You definitely don't want to sort a cache

[long marsh]

Are you familiar with Redis Sorted Sets? That's exactly the intention of that

[lyric mountain]

Yes, wait are you using redis as a db?

[long marsh]

No – just the leaderboard

But yeah, it is long-lived

[lyric mountain]

Would be better to use sqlite then

Redis is better suited as a cache, it was kinda made for that

[long marsh]

I've thought about that as well – SQLite would run into concurrent issues at scale right? If I'm writing in bulk every 2.5 minutes for all the players that have recently interacted, would the SQLite database be locked for each write?

The sorted sets are cool because you can use ZREMRANGEBYRANK to remove everything past a certain mark (I was thinking of using the cache for a long-lived leaderboard for the top 10,000 players)

[lyric mountain]

Not really, sqlite is only limited to 1 write op, but can do many reads concurrently

On top of that there are layers to queue writes so that it's transparent to u

The advantage of sql in general is that it was made for such task, that and relationships

And u can do range deletes on it too

[long marsh]

I'm still feeling as though Redis would be more suited for this use-case. I see what you're saying ... because, philosophically, a cache should be short-lived and not used as a database (I've heard of people making this mistake before). Redis is already multi-shard for me (using Amazon Elasticache) and I would need to figure out SQLite backups / concurrent connections at scale, etc. Is there a managed solution for SQLite that I can tap into? I know about Turso ...

[lyric mountain]

Well, no, sqlite is local

You create a file and that's it, all you need is a lib to access it

Backup would be just copying the file

And u don't need to manage concurrent connections, WAL takes care of that, just have a single process manage it to avoid UB

[long marsh]

Not in my case. I'm running a serverless web application and interact via HTTP Endpoint.

[lyric mountain]

Ah, serverless

[long marsh]

I'd need to spin up a server just to host the database (ironically enough)

[lyric mountain]

Using a third party service would be the only way then yeah, sqlite over network loses most benefits

[long marsh]

There is some caching of connections, but, realistically, the SQLite database would need to handle tens of hundreds or connections at once as people are playing.

[lyric mountain]

That wouldn't be an issue if they're just reading

U said the cache would dump to database every 2.5 mins, so that'd be the only actual write

I use it on at job for a high data flow app, still didn't get db performance issues

Network always ended up being the bottleneck

[long marsh]

I've heard of SQLite not performing well with 100's of concurrent reads

But it's been a while since I was in that world

[long marsh]

How much data we talking? How many active readers per second?

Leaderboard data is relatively low in memory; however, storing potentially hundreds to hundreds of thousands of records will slow the SQLite database right?

[lyric mountain]

It runs a synchronizer every 2 minutes to keep two databases (remote and local) in sync, and some tables go into thousand rows range. Not sure how many reads, that'd be the synchronizer itself doing the heavy work.

[long marsh]

Nice, that's pretty cool

What's the use-case?

If you tell me leaderboard, I'm going to laugh haha

[lyric mountain]

Company sales app

But well, tim uses sqlite for his bot iirc, his use-case is probably the most demanding around here

[long marsh]

Tim? As in Tim who maintained discordjs-light for a while?

I used that back in the day if so ... really appreciated their work on it.

[lyric mountain]

Yep, also dev of astrobot (think it was named that)

[long marsh]

Would love to hear the take on SQLite from them as well 🤷‍♂️

But I don't know them haha

[lyric mountain]

@quartz kindle when ur free

[long marsh]

What is your take on something like Turso?

On the free plan, I get 1 billion row reads + 25 million row writes per month

[lyric mountain]

Never hear of it tbh

But 1b sounds a lot for a free plan

That's about 33m reads per day

[long marsh]

I wouldn't need anywhere NEAR that much haha

[long marsh]

Btw, how have others done their "server" / "global" leaderboard schemas?

[delicate zephyr]

I used to do it like such when I was programming a discord bot

• User

  • id
  • globalXP

• guildXp

  • userID - PK / FK to user.id
  • guildID
  • xp

ofc I have a guilds table too but that was generally the idea

levels were based on total XP and calculated in real time

[long marsh]

What happens when a user leaves a guild?

[delicate zephyr]

You have one of two choices

keep the data in the guild in the case that they may rejoin

or, delete the data from the guildXp table

[long marsh]

So if the top-rated player in your game started hopping from server to server by entering and exiting, their username would persist if you did the 1st choice. Therefore, would always be ranked in that server even if they're not in it

[delicate zephyr]

then I would delete them or mark them as inGuild = false

[long marsh]

Which means, you'd likely need to hook into guild members intent right? And listen for that event?

[delicate zephyr]

yep

[tacit estuary]

The way I did it was make a folder that is dedicated to server leaderboard listings. Every server has its own file which is automatically saved as the server id number. Whenever a user access the leaderboard, it looks up the file by server id of the server. Hopefully that helps.

Also from what I can tell a server id and user id is 19 characters long. Some are 18 long.

[lament rock]

Don’t count on this as snowflakes will exceed this

Ideally never hardcode snowflake character length

some people make horrible mistakes of checking lengths in regex

keep in mind that even 0 is technically a valid snowflake

[tacit estuary]

Well poopy. I hope to not get any servers or users over 19 characters long as that will break the listing.

[lament rock]

https://tenor.com/view/cat-cats-cat-eating-eating-eating-cat-gif-8459125914348971806

[slender wagon]

hello, so i made an application in rust. It seems to be working just fine but i have a problem with some of it's libraries. Which i was able to solve with bun. Would making a local endpoint from rust to the bun app add much latency?

Cuz speed is a big deal in my app

[lapis condor]

The bot now is 214 server

How 48?

[spark flint]

you need to POST the server count via the API

[warped summit]

Im currently implementing this in my upcoming bot

Is it cool?

Auto editing every 5min

[deft wolf]

This is one message with 4 embeds?

[amber rose]

It looks very fine

[warped summit]

Yes

You gize like it separated?

[warped summit]

Sml ty

[amber rose]

Which programming language?

[warped summit]

Discord.js v14

[amber rose]

Ok

:/

[warped summit]

What? 😭

Think I should switch?

[amber rose]

Nope

Don't wanna argue

[warped summit]

No no speak your mind lol

[amber rose]

Everyone always uses Discord.js but I prefer Discord.py

like most of the time

[warped summit]

Ooo

I don’t know Discord py

[amber rose]

But it's all good

[amber rose]

JavaScript is a language for creating backend

and website stuff

[warped summit]

My bot heavily relies on real time events 🥲 which is better for that?

[amber rose]

Discord.py has support for that too

[warped summit]

Python look so hard mayne 😭

[amber rose]

:/

It's one of the easiest

[warped summit]

What have you created?

[amber rose]

Accept my friend request

[warped summit]

Alr

[lyric mountain]

any language will perform relatively similar for a discord bot

network is the major bottleneck in such scenario

[digital swan]

is a hash index any better than b tree for discord ids?

[lyric mountain]

hash is potentially faster if there are few hash collisions

[digital swan]

any benefit for memory usage?

[lyric mountain]

hash might be slightly higher as it has to store the hashes

btree just relies on sorting

[astral robin]

[pearl trail]

is it me or what, the error isn't clear. what must be between 1 and 25

nvm it's my skill issue

[lyric mountain]

huh, c# errors dont show line & column numbers in stacktraces?

never noticed that

[wheat mesa]

Hash is definitely much faster for higher input sizes. For small input sizes, hash may potentially be slower depending on the cost of the hash function & size of hash table and the balancing of the b tree

[wheat mesa]

Yes that is what happens when you open top for the first few seconds

[surreal sage]

so silly

[digital swan]

Input size being the column data?

[lyric mountain]

btree is O(log n), whereas hash is O(1) if no hash collisions

[wheat mesa]

Also, you have to decide what your use cases are

If you want to maintain order (which I suppose doesn’t make sense for discord IDs), you need a b tree. If you want fast lookup and insertion, hash maps are the way to go

[prime cliff]

It does but it depends on some factors such as if the dll/lib you're using has debug info or symbols disabled or if you're building in production and have the pdb file disabled or excluded

Also that one shown says inner error which means there's a patent error with more info scrolled below it?

[quartz kindle]

depends on how you access it, ie language and lib/adapter
for example on js using better-sqlite3, an average indexed query takes less than 1ms for me, which translates to over 1000 per second

but of course it depends on a lot of other things as well

if your priority is read performance, something like lmbd would be better

[long marsh]

Concurrent reads as well?

[long marsh]

I also don’t have a VPS - it’s pure serverless at this point.

I’d have to spin up a server just to have access to a SQLite haha. Which defeats the purpose.

What is your take on using Redis Sorted Sets for longterm leaderboards?

[prime cliff]

For any asp.net blazor nerds 👀
Working on a dynamic form library that creates forms based on a C# model with design and validation attributes (asp.net data annotations).
https://dynamicform.fluxpoint.dev/

[pearl trail]

oh, it was hot reloaded

[pearl trail]

i wonder why this permission still existed lmao

[quartz kindle]

sqlite itself does do concurrent reads, but js doesnt for example

you can open the same sqlite db and do concurrent reads from multiple nodejs processes

but a single nodejs process wont be able to do it because its single threaded

[quartz kindle]

if youre going serverless then yeah you need a database that spins a server like redis or pg

both options are fine, redis should be faster for simple queries, pg might be faster for complex queries

there are a few redis "alternatives" that claim to be even faster but i have never tried them

[quartz kindle]

afaik you can skip hashing altogether because discord ids are already unique
if you can use a hash table but bypass the hashing step that would be even faster

[static barn]

How to handle if the ops for the bot is too high?

[frosty gale]

hash index will almost always be faster than a btree, the only disadvantage of a hashmap is when it gets filled up and needs to be expanded which is extremely expensive

in a b tree youd just move a few keys/nodes around and create a new one, not too bad

[eternal osprey]

yoo this is sick i just created my own version of flappy bird lmao

christmas themed n allat

[deft wolf]

It's all good but why are you collecting bananas in a Christmas theme?

[eternal osprey]

Tf can i else collect

i thought birds love fruitshahahaha

good enough

I don't celebrate christmas myself so idk, should i add a fucking turkey leg then?

[small tangle]

You could collect christmas tree ornaments

[eternal osprey]

yeah sure that could be cool

letmme add that

[rustic spire]

hard life when my first successful and first verified bot took the least amount of time compared to my other bots 😔

[small tangle]

Why? You probably had experience from previous projects so it took the least amount of time

[rustic spire]

well I guess thats true

though this bot is a lot more simplistic

[digital swan]

so i assume hashes are best on columns where each row is unique? wouldn't be best on a table like this for example

(most queries filter by userid or user id and item)

[wheat mesa]

Why bother using hashes for this? That’s the job of your database to figure out

They are usually optimized enough to where you shouldn’t have to make decisions on hashing keys yourself and such

[digital swan]

a hash index i mean

[wheat mesa]

My point still stands

In prisma you have to specify how you want your database to index things? Ew

I wouldn’t worry about it that much, unless you have a TON of users. At which point, your bottleneck will probably be javascript, not your database

[digital swan]

the database is becoming the bottleneck actually

i'm using digital ocean managed db and it gets pretty expensive pretty quick

[wheat mesa]

How many queries are you throwing at it?

[digital swan]

about 400,000/hour on average

but i think it's mostly memory usage that's the problem

[wheat mesa]

I was gonna say, 400k is a lot but for a simple table like this, a database should be able to handle it

[digital swan]

oh it's not just that table

that was just 1 example i had that is used frequently

[wheat mesa]

Many databases handle tens of thousands of requests per second, you’re only throwing 111 per second at it on average

Have you benchmarked or profiled and determined that the issue is your database queries taking too long?

[digital swan]

yeah i've got logs that record when queries take too long

a couple weeks ago i noticed that small ones could be spiking and taking a while

and some errors for connection failures

[wheat mesa]

Are you sure it’s not your queries being unoptimized?

[digital swan]

i upgraded to 2gb/1cpu and a lot of it went away for now

[digital swan]

some of them are probably quite bad but the most ran ones are pretty simple

assuming prisma isn't doing anything stupid

[wheat mesa]

I’d look into the queries before you start blaming the database itself

It’s very unlikely that your db cannot handle 100req/sec

Unless something is pulling way more data than it needs

[digital swan]

yeah i do there's query stats that show the most ran queries and the timing etc

i've got this graph that shows average time per query and queries/hour and you can see around early december it gets higher

on the 7th i upgraded the db and it calmed down

but there were more queries there too (grey line)

[wheat mesa]

Though I haven’t had much experience at scale, your database should be able to handle 1k queries per second, ideally way more than that at peak loads

[digital swan]

even with 2gb 1 cpu?

[wheat mesa]

You can look into optimizing the indices, but overall most db performance problems are the result of poorly optimized queries

[wheat mesa]

Yes

I would say be careful with making any sort of PK a long string. Your PK should ideally just be an auto increment ID or something of that sort. Long strings aren’t great at being indexed efficiently for use as a PK

[digital swan]

majority of them are discord ids

if not then it's an auto increment

[wheat mesa]

But your database still sees them as strings no?

[digital swan]

yeah they're not super long though are they

[frosty gale]

depending on the query thats easy load for any db even mongo

[wheat mesa]

I’d still prefer using a number for PK rather than a string. They’re not SUPER long but they fit into a bigint if your database supports it

[digital swan]

would that really be better?

[wheat mesa]

I’d give it a shot. Strings don’t perform well as primary keys

[frosty gale]

after they are inserted string column lookups should perform basically just as well as numerical lookups since they should be hashed

[wheat mesa]

There’s always a cost to hashing a string

At scale it can be detrimental

[frosty gale]

hardly anything significant ive noticed

[digital swan]

i mean currently all the indexes are b tree assuming postgres uses that all the time for default

[frosty gale]

depending on the algorithm, especially in this case where collisions are acceptable

[wheat mesa]

Numbers would still be the faster choice imo

[digital swan]

how much of a difference are we talking here though

cause then js has to deal with bigint conversions

[wheat mesa]

It depends. If you have some sort of load testing that you could do, that’s probably the best way to check. But that’s probably not the case for a discord bot I’m assuming

[frosty gale]

only thing id worry about when using strings is more storage use per record, dbs are really good at handling this sort of stuff to the point where you dont even notice the performance penalty

i doubt its contributing to your performance issues

but definitely do a benchmark if you have the time

[digital swan]

just my main concern is the memory usage

like on the old db it was 1gb of ram

and it was constantly at like 80-90%

and then obv started failing to connect with cpu spikes

and now with 2gb it's still hovering at 75-80% memory usage

[frosty gale]

thats just cache, it will always fill up no matter what you do with your data

you need to limit it because by default databases will use as much as it can

the cache gets filled up probably by records/data you fetch maybe once every few hours and are waiting to be replaced by something new

[digital swan]

Ahhh so I guess just ignore that and pay attention to the timings

[frosty gale]

well if its taking up 90% of your memory you should limit the amount of ram it can use as cache

go through this its very quick https://www.enterprisedb.com/postgres-tutorials/how-tune-postgresql-memory

set it to an ok amount, it doesnt need to be anything extreme especially for small traffic projects

[digital swan]

I assume that’s already done for me since I’m paying for a managed db

[frosty gale]

hold on is the db hosted on the same server or is it a separate instance that you connect to?

[digital swan]

Separate

[frosty gale]

if its the second then dont touch anything the high memory usage is to be expected, youd waste your money if you set it to use less

[digital swan]

Yeah I thought so

Is there anything that would cause connection timeouts though?

[frosty gale]

how do you know they are timing out?

do you mean theres an active connection and it suddenly drops off or you time out when connecting

[digital swan]

this error

it would happen lots before i upgraded the memory a week ago

[frosty gale]

i may be wrong but i have a feeling its just a bad error for you maxing out on the database's connection limit

[digital swan]

well i'm using pgbouncer

[frosty gale]

check this article out, i would try increasing the connection limit and see if it solves your problem https://www.prisma.io/docs/orm/prisma-client/setup-and-configuration/databases-connections#optimizing-the-connection-pool

but this is more of a temporary fix, id try to investigate why its maxing out in the first place if its indeed the problem

^this articles the recommended fix for timeout errors for pools

[digital swan]

there's errors before that with a larger connection pool, errors just stopped happening when i upgraded the server

[frosty gale]

how big of a connection pool is prisma trying to acquire? also worth noting, often times the DB keeps around dead connections because the interval for clearing out dead connections is very long

so they may be taking up your connection limit

[digital swan]

I thought that would pretty much be negated with pgbouncer though

[frosty gale]

can't lie i've never used pgbouncer or postgres as it is so i can't talk much about it

i rely on the fact that sql rdms systems all basically have the same concepts behind them™️

plus prisma might be doing additional screwing in the process further complicating things

[static barn]

Isn't this problem caused by lack of permission?

[coral pagoda]

can the bot see the channel

[pearl trail]

that doesn't matter if it's slash command

the error seems like the bot trying to send "thanks for inviting me" message to a channel

and then it got no permission, it crashes/stalled

then when executing the command, nothing's happened

[coral pagoda]

it says missing access though?

[pearl trail]

different error

that's my assumption

[coral pagoda]

fair

[pearl trail]

unless you're telling user to use help command when they're already using the help command

[radiant kraken]

ooooo

icic

i feel like thanks for inviting messages are lowkey unnecessary

[pearl trail]

sorta

kinda annoying imo

[radiant kraken]

yup

[pearl trail]

a simple .catch(() => void 0) would fix that problem

[static barn]

Why 😭

You want to add svg instead hehe

[static barn]

not crashed, it logged error in the console

[pearl trail]

well that error you should catch

also that error isn't the reason why your help command is not working

that's a different error about your bot can't send a message to a channel

[static barn]

You are different 🙏🏻

[pearl trail]

wdym