#development | Top.gg | Page 231

frosty gale Jun 28, 2024, 5:05 PM

#

you have sequential arrays which are full of elements

#

and arrays which may have holes in between of elements

frosty gale Jun 28, 2024, 5:06 PM

#

frosty gale and arrays which may have holes in between of elements

id assume this one is stored as a linked list to save memory

frosty gale Jun 28, 2024, 5:06 PM

#

frosty gale you have sequential arrays which are full of elements

and this one just as any other array/memory block for performance

quartz kindle Jun 28, 2024, 5:25 PM

#

theres like 8 different types

#

6 actually

#

sharp geyser Jun 28, 2024, 5:26 PM

#

hello tim

quartz kindle Jun 28, 2024, 5:26 PM

#

general kenobi

sharp geyser Jun 28, 2024, 5:27 PM

#

do you know how VOD (Video On Demand) works?

#

Its different from live streaming

quartz kindle Jun 28, 2024, 5:27 PM

#

i mean

sharp geyser Jun 28, 2024, 5:27 PM

#

As you upload the files, and then encode them, and then stream them out to people through HLS/DASH

quartz kindle Jun 28, 2024, 5:27 PM

#

vod is like youtube video

sharp geyser Jun 28, 2024, 5:27 PM

#

yuh

#

Im just trying to figure out if I can do this myself or nah

#

💀

quartz kindle Jun 28, 2024, 5:28 PM

#

stream is the same thing, but the file is not stored on disk, just buffered in memory

#

or likely both

#

on livestream you have an input, the recording, then as the input comes in, you both save it to disk, and redirect it to some network output

sharp geyser Jun 28, 2024, 5:28 PM

#

If I can do this myself and not use a service like bunny.net then I 100% will, it'd likely be cheaper

quartz kindle Jun 28, 2024, 5:41 PM

#

sharp geyser If I can do this myself and not use a service like bunny.net then I 100% will, i...

ye if you wanna go public with it, you definitely want a cdn

#

livestream is uploaded to cdn in real time, then viewers connect to the cdn to stream it

sharp geyser Jun 28, 2024, 5:43 PM

#

quartz kindle ye if you wanna go public with it, you definitely want a cdn

Can i not route it through cloudflare cdn?

#

I also have to encrypt/decrypt the files myself also 😔

quartz kindle Jun 28, 2024, 5:43 PM

#

probably yeah, but video cdn is like a separate service no?

#

becaude video is expensive

sharp geyser Jun 28, 2024, 5:44 PM

#

yea my only problem is

#

I dont want to pay for their storage if I can avoid it

#

I'd rather pay 4$ / tb rather than 10-20

#

or even 30

#

I get that it's replicating the files too those locations and making it faster for the cdn but still

#

up to 30$ a month per tb

quartz kindle Jun 28, 2024, 5:45 PM

#

livestream to youtube/vimeo upload

#

free cdn

#

:^)

sharp geyser Jun 28, 2024, 5:45 PM

#

I can't do that

#

😭

#

I mean realisitcally its a pay as you go system, so you only pay for what you use so it wont be that much to start out with

#

but if it gets that high ima be fucked if im not bringing in revenue

quartz kindle Jun 28, 2024, 5:51 PM

#

ye video is expensive

#

itis a lot of data

sharp geyser Jun 28, 2024, 5:53 PM

#

Sadge

#

I guess ima have to suck it up

#

bunny is the least expensive option

#

5$ /tb of bandwidth isn't the worst

#

The only sucky part is the storage but its whatever, hopefully it will bring in some revenue if I code this shit properly

dawn socket Jun 28, 2024, 6:00 PM

#

like buttons a span svg etc

sharp geyser Jun 28, 2024, 6:04 PM

#

quartz kindle ye video is expensive

Wait question

#

is DRM support mandatory?

#

If I enable DRM, yes it stops video downloads

#

but it also limits how I can play said videos

#

There is only selected number of players (all of them look absolutely terrible) that support playing DRM protected content

#

and the ones that exist only work on x browsers, and sometimes doesn't work on x devices

#

I notice even youtube doesn't protect all their videos via DRM

#

only the paid ones

frosty gale Jun 28, 2024, 6:07 PM

#

quartz kindle

arent all numbers in js doubles? why is there one for normal elements and one for doubles

#

i would guess the other one is for arrays which contain things like objects

frosty gale Jun 28, 2024, 6:10 PM

#

sharp geyser If I enable DRM, yes it stops video downloads

drm is probably the most anticonsumer thing ever and it isnt even effective a lot of the time

#

i wouldnt use it unless theres a good reason to

sharp geyser Jun 28, 2024, 6:10 PM

#

Gotcha

#

I was looking at bunny.net's DRM and I was like erm no thanks

#

their enterprise DRM is like expensive asf too

#

Each time someone plays a video it charges you 0.005 a license

frosty gale Jun 28, 2024, 6:11 PM

#

some DRM also depends on the hardware and OS and will refuse to play if you dont have satisfactory hardware like HDCP which requires the gpu and monitor and drivers to support them

sharp geyser Jun 28, 2024, 6:12 PM

#

WHAT

#

Bruh

#

Thats kinda wack

frosty gale Jun 28, 2024, 6:12 PM

#

louis rossman did a whole video on netflix doing it

#

they dont allow you to stream HD on unsupported hardware

sharp geyser Jun 28, 2024, 6:12 PM

#

That's bogus

#

Alright note to self no drm

#

iara_nod

#

That saves me another headache

quartz kindle Jun 28, 2024, 6:13 PM

#

frosty gale arent all numbers in js doubles? why is there one for normal elements and one fo...

yea SMI = small integer

#

basically 1byte ints

#

so you have array of int/char, array of double and array of pointer

#

this is on the v8 side btw, on the js side all numbers are doubles yes

rigid maple Jun 28, 2024, 6:42 PM

#

I use svelte and tailwind, but tailwind does not work when I try to design a homepage.

#

tailwind's output.css is in the head section

#

It works when I give it as a style instead of the tailwind class.

surreal sage Jun 28, 2024, 6:54 PM

#

rate the jsdoc

neon leaf Jun 28, 2024, 6:54 PM

#

wow

digital swan Jun 28, 2024, 7:13 PM

#

rigid maple It works when I give it as a style instead of the tailwind class.

did you set tailwind up properly?

https://tailwindcss.com/docs/guides/sveltekit

civic scroll Jun 28, 2024, 7:23 PM

#

dawn socket like buttons a span svg etc

Ctrl + F

rigid maple Jun 28, 2024, 7:51 PM

#

digital swan did you set tailwind up properly? https://tailwindcss.com/docs/guides/sveltekit

Yes, it works on /dashboard/ path. It only didnt works on the home page

digital swan Jun 28, 2024, 7:54 PM

#

are you importing the css on the correct layout?

rigid maple Jun 28, 2024, 8:21 PM

#

yes i imported css from app.html

sharp geyser Jun 28, 2024, 8:27 PM

#

excuse me what
Error: Io(Os { code: 54, kind: ConnectionReset, message: "Connection reset by peer" })

lyric mountain Jun 28, 2024, 8:27 PM

#

the other side dropped connection

sharp geyser Jun 28, 2024, 8:27 PM

#

... I am the client

#

Like I am starting the webserver

lyric mountain Jun 28, 2024, 8:27 PM

#

the other side

sharp geyser Jun 28, 2024, 8:27 PM

#

I am the other side

#

💀

#

I am simply running an api that literally does nothing rn but return hello world at /

lyric mountain Jun 28, 2024, 8:28 PM

#

no, like, if u receive that on server it's cuz client dropped, same for the other way around

sharp geyser Jun 28, 2024, 8:28 PM

#

right

#

but here's the thing

#

How is it dropping if its happening when I start the webserver

lyric mountain Jun 28, 2024, 8:29 PM

#

does it appear when u access the url or when u literally start?

sharp geyser Jun 28, 2024, 8:29 PM

#

#

quite literally when I start it

#

I already authorized it on my macbook

#

so that can't be it

lyric mountain Jun 28, 2024, 8:29 PM

#

hm, are u using any service?

#

or api, database, whatever

sharp geyser Jun 28, 2024, 8:30 PM

#

Oh wait

#

indeed I am

#

💀

lyric mountain Jun 28, 2024, 8:30 PM

#

then that's where the issue lies

sharp geyser Jun 28, 2024, 8:30 PM

#

Forgot that I was

#

ngl

#

postgres://asdasd:asdasdads@localhost:5432/vidcord does this look correct?

lyric mountain Jun 28, 2024, 8:31 PM

#

is that supposed to be a jdbc url?

#

if so, it's incorrect

sharp geyser Jun 28, 2024, 8:33 PM

#

nah

#

doesn't have to be jdbc

lyric mountain Jun 28, 2024, 8:33 PM

#

ok but does it use the jdbc standard?

sharp geyser Jun 28, 2024, 8:33 PM

#

If it does it doesn't ask for it

lyric mountain Jun 28, 2024, 8:34 PM

#

the standard database url is driver://url:port/database

sharp geyser Jun 28, 2024, 8:34 PM

#

🦐

#

I got it working

#

When I opened the ssh tunnel I used the wrong ip

#

I put 0.1 instead of 1.1 as the last 2

#

val_WaaGone

lyric mountain Jun 28, 2024, 8:35 PM

#

skill issue

sharp geyser Jun 28, 2024, 8:36 PM

#

Also

#

I have to url encode my redirect uris for discord oauth2 right

lyric mountain Jun 28, 2024, 8:39 PM

#

well, not specifically for oauth2, u have to do it to have a valid url to begin with mmLol

#

non encoded urls aren't valid at all

surreal sage Jun 28, 2024, 8:55 PM

#

is *.* a valid wildcard in dns

#

or does * already do that

#

asking cuz cloudflare doesnt supply ssl for anything but first level sub domains

#

nvm not possible

lyric mountain Jun 28, 2024, 9:00 PM

#

never saw a url that has 2 levels of sub domain

quartz kindle Jun 28, 2024, 9:08 PM

#

lyric mountain never saw a url that has 2 levels of sub domain

https://docs.aws.amazon.com

quartz kindle Jun 28, 2024, 9:11 PM

#

surreal sage is `*.*` a valid wildcard in dns

surreal sage Jun 28, 2024, 9:13 PM

#

#

heh

quartz kindle Jun 28, 2024, 9:14 PM

#

surreal sage

dude broke ts

#

:^)

surreal sage Jun 28, 2024, 9:15 PM

#

does it happen if u have too many jsx elements

#

or

#

not even a typescript.tv description

quartz kindle Jun 28, 2024, 9:16 PM

#

surreal sage does it happen if u have too many jsx elements

SO suggests its a typescript issue and recommends updating typescript and/or vsc typescript extensions

surreal sage Jun 28, 2024, 9:16 PM

#

im already on the latest

#

Skull

quartz kindle Jun 28, 2024, 9:17 PM

#

some people say they fixed by rolling back to v4

#

lmao

surreal sage Jun 28, 2024, 9:17 PM

#

fuck that

#

was just trying out threejs but ig not

quartz kindle Jun 28, 2024, 9:17 PM

#

idk try any of these

#

https://stackoverflow.com/questions/74847053/how-to-fix-expression-produces-a-union-type-that-is-too-complex-to-represent-t

surreal sage Jun 28, 2024, 9:17 PM

#

its alr i deleted my code anyways

quartz kindle Jun 28, 2024, 9:18 PM

#

nooo if you do that then ts wins

#

ts must not win

surreal sage Jun 28, 2024, 9:18 PM

#

ts won

quartz kindle Jun 28, 2024, 9:18 PM

#

noooo

#

js my love, it doesnt matter if ts won, js will always be the winner in my heart

#

:^)

lyric mountain Jun 28, 2024, 9:22 PM

#

quartz kindle <https://docs.aws.amazon.com>

Damn pesky aws engineers

dawn socket Jun 28, 2024, 9:40 PM

#

civic scroll Ctrl + F

how to i know the name of the that element?

#

how long discription have character limit?

#

i'm unable to save it

civic scroll Jun 28, 2024, 9:41 PM

#

dawn socket how to i know the name of the that element?

you can also Ctrl + Shift + C in devtools then click on desired element you want to inspect in the page

dawn socket Jun 28, 2024, 9:42 PM

#

civic scroll you can also Ctrl + Shift + C in devtools then click on desired element you want...

but then i have to manually click on the class and copy it's name

dawn socket Jun 28, 2024, 9:42 PM

#

dawn socket how long discription have character limit?

any help?

civic scroll Jun 28, 2024, 9:44 PM

#

dawn socket but then i have to manually click on the class and copy it's name

what are you trying to do exactly

#

what do you wish to aquire via topgg elements

lyric mountain Jun 28, 2024, 9:53 PM

#

worth to note, it's useless to use any of top.gg classnames

#

except for a few that are fixed, they change on every page load

dawn socket Jun 28, 2024, 9:54 PM

#

civic scroll what do you wish to aquire via topgg elements

i am trying to add elements like button

#

where user can interact with it

#

can i execute my bot command in the description

lyric mountain Jun 28, 2024, 9:55 PM

#

you can't use js in long desc

civic scroll Jun 28, 2024, 9:55 PM

#

dawn socket i am trying to add elements like button

then just add it?
assuming topgg allows acripts

lyric mountain Jun 28, 2024, 9:55 PM

#

it doesn't, only html + css

civic scroll Jun 28, 2024, 9:55 PM

#

but even so that is not recommended

#

i don't see the point of having interactive bot preview

#

because if anything they can try out in private servers, or your support server

#

also hi kuu

lyric mountain Jun 28, 2024, 9:56 PM

#

hi sa-yun

#

I did add an interactive preview, but just cuz there was no way of showcasing 20 cards without becoming a mess

#

it's just a roulette tho

civic scroll Jun 28, 2024, 9:57 PM

#

i meant in context of topgg desc

#

that would just hog perf so much

lyric mountain Jun 28, 2024, 9:58 PM

#

civic scroll i meant in context of topgg desc

KEKW

civic scroll Jun 28, 2024, 9:59 PM

#

o

#

yugioh style

dawn socket Jun 28, 2024, 10:11 PM

#

lyric mountain it doesn't, only html + css

d9_alonerain

#

we can use iframe right?

lyric mountain Jun 28, 2024, 10:15 PM

#

yep

sharp geyser Jun 28, 2024, 10:17 PM

#

Whats the base url for discord's api

#

I swear I can never remember it

#

😭

#

is it https://discord.com/api/v10/?

lyric mountain Jun 28, 2024, 10:18 PM

#

I think so

sharp geyser Jun 28, 2024, 10:18 PM

#

I can never find it on the docs

quartz kindle Jun 28, 2024, 10:20 PM

#

sharp geyser I can never find it on the docs

its in reference

#

sharp geyser Jun 28, 2024, 10:20 PM

#

Ah

#

ty

quartz kindle Jun 28, 2024, 10:21 PM

#

also why is v6 still default

#

wtf

#

lament rock Jun 28, 2024, 10:28 PM

#

It's still supported and what the majority of bots were on

#

There might be some lobbying from larger bots to keep it alive

#

I don't think they'd have extended its life time this much if that wasn't the case

quartz kindle Jun 28, 2024, 10:31 PM

#

i mean, supported sure, but default is weird, especially how pushy discord was with v9 an v10 things

#

i guess its for compatibility with old libs who dont explicitly set a version in the url lol

lament rock Jun 28, 2024, 10:33 PM

#

Maybe. It's weird

#

A lot of old bots will die, that's for sure

#

I wonder what kind of message bots will receive when their gateway connection gets terminated for good looks like

quartz kindle Jun 28, 2024, 10:44 PM

#

lament rock I wonder what kind of message bots will receive when their gateway connection ge...

"request to discord returned funny result"

frosty gale Jun 28, 2024, 10:49 PM

#

cant wait to have to update discord.js for a discord bot i have running to track some data that i never touch

#

and then have to deal with djs breaking changes as a result

#

"just dont update djs" tell that to the discord gateway

quartz kindle Jun 28, 2024, 10:50 PM

#

djs is like apple, planned obsolete

#

force people into getting the new one every year

#

to keep the project running

#

lmao

past field Jun 28, 2024, 11:04 PM

#

is there a way to prevent the bot from crashing when someone tries to use it without the correct perms/view channel access? when i get “missing access” error it shuts the bot down

quartz kindle Jun 28, 2024, 11:05 PM

#

past field is there a way to prevent the bot from crashing when someone tries to use it wit...

handle the errors

past field Jun 28, 2024, 11:08 PM

#

quartz kindle handle the errors

it’s hard to pin point what guild is causing that error😭

quartz kindle Jun 28, 2024, 11:09 PM

#

past field it’s hard to pin point what guild is causing that error😭

i mean, in your code, you can catch those errors, and prevent them from crashing

sharp geyser Jun 28, 2024, 11:09 PM

#

hey tim

#

have you ever heard of discord's oauth2 token endpoint returning an html page

#

the status code is 200 so I assume it was successful

#

but instead of giving me the access token and what not it returns an html page

quartz kindle Jun 28, 2024, 11:10 PM

#

its a cloudflare page im guessing

#

i've seen that happen before

#

returns some rate limit information or something, but in an html page lol

sharp geyser Jun 28, 2024, 11:10 PM

#

wait a sec

#

did I just find a vulnerability

#

why does it give me their stripe keys

#

👀

quartz kindle Jun 28, 2024, 11:11 PM

#

its probably just stuff related to your session

sharp geyser Jun 28, 2024, 11:11 PM

#

but I don't have anything related to stripe?

quartz kindle Jun 28, 2024, 11:11 PM

#

well, a lot of services create sessions of everything, including visitors

sharp geyser Jun 28, 2024, 11:12 PM

#

but do thye put it in in window.Global_Env?

sharp geyser Jun 28, 2024, 11:27 PM

#

Response: "{\"message\": \"Invalid Form Body\", \"code\": 50035, \"errors\": {\"redirect_uri\": {\"_errors\": [{\"code\": \"URL_TYPE_INVALID_URL\", \"message\": \"Not a well formed URL.\"}]}}}"

#

Hmm

#

Uhm but its the same exact url I used when authorizing

#

thonk

quartz kindle Jun 28, 2024, 11:35 PM

#

why do people give you formatted json responses

#

just have the user format the json themselves

#

its not meant to be transmitted in its formatted form lmao

sharp geyser Jun 28, 2024, 11:35 PM

#

wut

#

What are you talkin bout

#

Thats a console log buddy

#

👀

quartz kindle Jun 28, 2024, 11:35 PM

#

ah lol

#

nvm then xD

sharp geyser Jun 28, 2024, 11:36 PM

#

lol

#

still tho

#

wtf is discord cooking

quartz kindle Jun 28, 2024, 11:36 PM

#

bread

sharp geyser Jun 28, 2024, 11:36 PM

#

I gave them the same exact redirect uri as I did when authorizing

#

so how is it not well formed

#

http%3A%2F%2Flocalhost%3A3000%2Fapi%2Fauth%2Fdiscord%2Fcallback

#

this is how it looks

#

Is this not correct?!?!?!

#

wait a sec

#

do I not need to url form encode the urls?

#

I guess not

#

noted

frosty gale Jun 29, 2024, 12:31 AM

#

im writing a feature where you can search for users by username and it gives you the top x users depending on the likeness to the username you typed in

#

its an sql database and i am not sure if there is any way i can really optimise this for a large amount of users

#

was thinking of doing the levenshtein distance but that definitely defeats any hope of optimisation so maybe ill check by first letters in the usernames

quartz kindle Jun 29, 2024, 12:33 AM

#

frosty gale was thinking of doing the levenshtein distance but that definitely defeats any h...

what i usually do is first filter and then order

#

if usernames are indexed, get all usernames that match string in any position, then order result by levenshtein

#

but if you want something more fuzzy, then its more complicated

frosty gale Jun 29, 2024, 12:35 AM

#

i dont think theres really a way to avoid a full table scan in this case

#

assuming there is a lot of users

quartz kindle Jun 29, 2024, 12:35 AM

#

depends on your index

#

i think normal indexes support LIKE query with prefix (basically startswith)

#

for LIKE queries with both prefix and postfix you probably need a different index, like trigram

frosty gale Jun 29, 2024, 12:38 AM

#

ah i didnt know you could still do an indexed search with anything other than exact matches

#

i think in this case for best optimisation ill do searches by prefixes

quartz kindle Jun 29, 2024, 12:38 AM

#

frosty gale Jun 29, 2024, 12:39 AM

#

quartz kindle

i see why now its very interesting

#

for a lot of indexes b-trees are used

#

the nodes for each string are ordered by their character order

#

so it can traverse down the tree if you search by prefix of suffix easily

#

so you get an o(log(n)) search since each search of prefix/suffix halves the possible amount of combiantions

quartz kindle Jun 29, 2024, 12:41 AM

#

ye

frosty gale Jun 29, 2024, 12:44 AM

#

im not sure if i can insert a template inside of a like statement but its worth a try

#

you cannot

#

you have to concatenate them

eternal osprey Jun 29, 2024, 1:24 AM

#

I am trying to remove really small outliers using the IQR but found that it sets the threshold to like a big minus. I know my calculation is correct though...

#

Is it just how iqr behaves? Is it skewed towards removing small value outliers instead of big value ones?

quartz kindle Jun 29, 2024, 2:01 AM

#

frosty gale im not sure if i can insert a template inside of a like statement but its worth ...

you can insert it in the variable

#

"... LIKE ?").all(`${username}%`)

sharp geyser Jun 29, 2024, 2:05 AM

#

when defining a varchar(x), x is the amount of bytes right, not like character length?

#

so I can do varchar(16) for a 16 byte string of text (GUID)

quartz kindle Jun 29, 2024, 2:51 AM

#

sharp geyser when defining a varchar(x), x is the amount of bytes right, not like character l...

appaaaarently

#

its db dependent

#

lmao

sharp geyser Jun 29, 2024, 2:52 AM

#

val_WaaGone

#

I put 36 just in case

quartz kindle Jun 29, 2024, 2:52 AM

#

sharp geyser Jun 29, 2024, 2:52 AM

#

since GUID is typically 36 characters long

#

idk if postgres does it

#

but if so oh well now there is a 36 byte roof

#

💀

quartz kindle Jun 29, 2024, 2:52 AM

#

if you use varchar it will use strings anyways

#

and count the bytes in the string

sharp geyser Jun 29, 2024, 2:52 AM

#

akashrug

#

I put 36 cuz honestly doesn't really matter

#

its reasonable enough

#

it wont cause performance issues

quartz kindle Jun 29, 2024, 2:53 AM

#

if you want to save only 16 bytes then you need either the UUID type or binary type

sharp geyser Jun 29, 2024, 2:53 AM

#

😔

#

I really dont care at this point

quartz kindle Jun 29, 2024, 2:53 AM

#

understandable

sharp geyser Jun 29, 2024, 2:54 AM

#

mmLol

#

but wont the uuid type conflict when inserting a guid?

quartz kindle Jun 29, 2024, 2:54 AM

#

idk

sharp geyser Jun 29, 2024, 2:54 AM

#

Hmm

#

ah ok

#

so it is characters

#

looking at the docs for pg

#

varchar(n) where n is the number of characters (not bytes)

lyric mountain Jun 29, 2024, 3:35 AM

#

I mean, it's kinda the same thing

#

As chars and bytes have the same size

#

Varchar is 2 + n bytes

dawn socket Jun 29, 2024, 3:37 AM

#

i'n unable to delete announcement

civic scroll Jun 29, 2024, 3:42 AM

#

dawn socket i'n unable to delete announcement

what

#

check if you have permission to delete it

dawn socket Jun 29, 2024, 4:30 AM

#

civic scroll check if you have permission to delete it

i'm the owner

#

how can i fix this?

civic scroll Jun 29, 2024, 4:31 AM

#

oh, the website

#

just hit Esc

dawn socket Jun 29, 2024, 4:31 AM

#

civic scroll what

it was network issue

dawn socket Jun 29, 2024, 4:32 AM

#

civic scroll just hit Esc

no when the element is not in focus it make the element visible

#

how can i make it invisible?

#

i'm editing my page

#

d9_awkward

neon flicker Jun 29, 2024, 11:56 AM

#

TypeError: client.commands.get is not a function

Is it related to my Discord.js version?

#

Beacuse in the updated documents, it seems to be a valid function

solemn latch Jun 29, 2024, 12:02 PM

#

neon flicker ``` TypeError: client.commands.get is not a function ``` Is it related to my Dis...

iirc client.commands is a custom property of client.

Related to this section of the guide.
https://discordjs.guide/creating-your-bot/command-handling.html#loading-command-files

discord.js Guide

Imagine a guide... that explores the many possibilities for your discord.js bot.

past field Jun 29, 2024, 4:33 PM

#

anyone able to look at this and tell me why it's showing this timestamp?

#

const { SlashCommandBuilder } = require('@discordjs/builders');
const db = require('../db');

const DAILY_COINS = 30; // Number of daily coins a player can claim
const TWELVE_HOURS = 12 * 60 * 60; // Time period in seconds (12 hours)

module.exports = {
    data: new SlashCommandBuilder()
        .setName('daily')
        .setDescription('Claim your daily Click War Coins.'),
    async execute(interaction) {
        const playerId = interaction.user.id; // Fetching the player's Discord ID
        const now = Math.floor(Date.now() / 1000); // Current time in seconds

        try {
            // Fetching player stats from the database
            const stmt = db.prepare('SELECT click_war_coins, last_daily FROM players WHERE id = ?');
            const playerStats = stmt.get(playerId);

            if (!playerStats) {
                // If the player is not in the database, add them
                const insertStmt = db.prepare('INSERT INTO players (id, click_war_coins, last_daily) VALUES (?, ?, ?)');
                insertStmt.run(playerId, DAILY_COINS, now);

                const nextClaimTime = now + TWELVE_HOURS;
                await interaction.reply({ content: `You have claimed your daily ${DAILY_COINS} Click War Coins! Your next claim is available <t:${nextClaimTime}:R> at <t:${nextClaimTime}:t>.`, ephemeral: true });
            } else {
                const { click_war_coins, last_daily } = playerStats;

                if (now - last_daily < TWELVE_HOURS) {
                    // If the player has already claimed their daily coins within the last 12 hours
                    const nextClaimTime = last_daily + TWELVE_HOURS;
                    await interaction.reply({ content: `You can claim your daily Click War Coins <t:${nextClaimTime}:R>. Your next claim is available at <t:${nextClaimTime}:t>.`, ephemeral: true });
                } else {
                    // If the player can claim their daily coins
                    const updateStmt = db.prepare('UPDATE players SET click_war_coins = click_war_coins + ?, last_daily = ? WHERE id = ?');
                    updateStmt.run(DAILY_COINS, now, playerId);

                    const nextClaimTime = now + TWELVE_HOURS;
                    await interaction.reply({ content: `You have claimed your daily ${DAILY_COINS} Click War Coins! Your next claim is available <t:${nextClaimTime}:R> at <t:${nextClaimTime}:t>.`, ephemeral: true });
                }
            }
        } catch (error) {
            // Error handling for database operations
            console.error('Database operation failed:', error);
            await interaction.reply({ content: 'An error occurred while processing your request. Please try again later.', ephemeral: true });
        }
    }
};

neon flicker Jun 29, 2024, 4:33 PM

#

Sure, send the code please

#

What exactly is the issue? Like what does the message that it sends look like?

past field Jun 29, 2024, 4:34 PM

#

neon flicker Jun 29, 2024, 4:35 PM

#

Because Discord's built-in timestamping does not use milliseconds AFAIK, so you should try dividing it by 1000 before showing it up

#

I might be wrong as well, sorry if I am

past field Jun 29, 2024, 4:37 PM

#

it does already

#

const now = Math.floor(Date.now() / 1000);

neon flicker Jun 29, 2024, 4:38 PM

#

Okay, let me check the code again

past field Jun 29, 2024, 4:39 PM

#

maybe it's the database

neon flicker Jun 29, 2024, 4:40 PM

#

I don't really have much experience with SQL so it might be, please try printing what it returns so we may see the problem

past field Jun 29, 2024, 4:41 PM

#

i’ve just added some debugging to see what it’s calculating

#

it won't let me post it

#

#

ohhhh i see what it is

deft wolf Jun 29, 2024, 4:44 PM

#

Automod probably detected it as a phone number

past field Jun 29, 2024, 4:45 PM

#

last_daily is stored in milliseconds, not seconds

#

ok it's fixed

solemn latch Jun 29, 2024, 4:45 PM

#

deft wolf Automod probably detected it as a phone number

yeah ^-^

#

Honestly, I dont know why we have that one, I've not seen a phone number posted ever. Or I dont remember that

past field Jun 29, 2024, 4:49 PM

#

deft wolf Jun 29, 2024, 4:50 PM

#

Good job

real rose Jun 29, 2024, 6:54 PM

#

solemn latch Honestly, I dont know why we have that one, I've not seen a phone number posted ...

The whatsapp scams use them

#

and dont have any invite links or pings in the messages

#

iirc

solemn latch Jun 29, 2024, 6:55 PM

#

ah

frosty gale Jun 29, 2024, 8:39 PM

#

@lyric mountain mr nginx i have routes like /api/users/:userid/delete, can you match routes like /api/users/*/delete or is it more complicated

sharp geyser Jun 29, 2024, 8:45 PM

#

why are you doing this at nginx level?

#

Is there something specific you want mapped at those routes?

lyric mountain Jun 29, 2024, 8:57 PM

#

frosty gale <@350836145921327115> mr nginx i have routes like `/api/users/:userid/delete`, c...

yeah no, that's definitely not something ur supposed to do at nginx level

#

u probably can, but shouldn't

surreal sage Jun 29, 2024, 9:05 PM

#

docker try not to download images at 0.1 mb/s level impossible

#

https://astrid.email/u/WindowsTerminal_MRVvhXCdsY.gif

quartz kindle Jun 29, 2024, 9:29 PM

#

frosty gale <@350836145921327115> mr nginx i have routes like `/api/users/:userid/delete`, c...

https://stackoverflow.com/questions/76396334/nginx-regex-match-the-middle-of-a-url-and-also-the-ending

Stack Overflow

nginx: Regex match the middle of a URL and also the ending?

I have an app on a domain that is set by a developer to proxy at certain URLs:
example.com/browser/123foo0/stuff.js

for example, where 123foo0 is some random key. The key may also change length in

frosty gale Jun 29, 2024, 10:28 PM

#

sharp geyser Is there something specific you want mapped at those routes?

i would want to handle ratelimits on the nginx level and other stuff like upload limits and accepted data for specific routes

#

but nginx no no wanna support modern url conventions

frosty gale Jun 29, 2024, 10:31 PM

#

quartz kindle https://stackoverflow.com/questions/76396334/nginx-regex-match-the-middle-of-a-u...

that is not a pretty solution and im not sure how i feel about regex

#

wonder if theres a module that adds support for this

sharp geyser Jun 29, 2024, 10:32 PM

#

frosty gale i would want to handle ratelimits on the nginx level and other stuff like upload...

WHAT

#

Why on the nginx level?

#

I mean without using regex idk what else you can do

#

any module will likely also use regex

frosty gale Jun 29, 2024, 11:28 PM

#

sharp geyser Why on the nginx level?

why wouldnt you

#

i would think nginx would be able to handle things like this much more optimally than any server can

sharp geyser Jun 29, 2024, 11:28 PM

#

why would you want that on proxy level though

frosty gale Jun 29, 2024, 11:28 PM

#

fast

sharp geyser Jun 29, 2024, 11:29 PM

#

That's something your app should handle no?

#

or even cloudflare

frosty gale Jun 29, 2024, 11:29 PM

#

if i wanna rate limit someone for example

#

i can make nginx go bye bye on connection instantly

#

if the app handled that it would have to proxy it through and process the request again on its end

sharp geyser Jun 30, 2024, 1:40 AM

#

Should I expire session cookies if I am using my own auth system?

#

or should I just remove the sessions when they log out

#

I feel like so long as they don't log out themselves I see no reason to log them out for them at a certain time

quartz kindle Jun 30, 2024, 2:55 AM

#

personal preference pretty much

sharp geyser Jun 30, 2024, 3:01 AM

#

icic

pearl trail Jun 30, 2024, 3:35 AM

#

maybe just put 6 months or so

sharp geyser Jun 30, 2024, 3:35 AM

#

eh

#

I wont bother

pearl trail Jun 30, 2024, 3:35 AM

#

KEK

sharp geyser Jun 30, 2024, 3:35 AM

#

If they dont logout themselves why expire them

#

if they are dumb enough to give out their session id cookie sucks to be them

#

💀

pearl trail Jun 30, 2024, 3:38 AM

#

well if your data is very important, you should expire them

#

https://security.stackexchange.com/questions/90979/should-cookies-expire see the answer

Information Security Stack Exchange

Should cookies expire?

Let's say I have an account on a blog. I'm using https on every single page of that blog through web server rules, so there is no way to visit the page in the clear text. I'm also using a laptop that

#

attackers steal cookies from websites to gain access to user's account

civic scroll Jun 30, 2024, 3:57 AM

#

sharp geyser if they are dumb enough to give out their session id cookie sucks to be them

log out when user starts with css

sharp geyser Jun 30, 2024, 4:00 AM

#

pearl trail attackers steal cookies from websites to gain access to user's account

like I said

#

Not my problem if someone allows their cookie to be stolen

pearl trail Jun 30, 2024, 4:02 AM

#

then sure yeah

sharp geyser Jun 30, 2024, 4:02 AM

#

I can expire them sure

#

but thats more headache for me

pearl trail Jun 30, 2024, 4:02 AM

#

eh, well attackers don't ask users for permission to access their cookies or not

#

but whatever

sharp geyser Jun 30, 2024, 4:02 AM

#

I mean...

#

They don't need permission when they just give it out

#

💀

#

"Here click this link and install it" "Ok sure"

#

"Use this public wifi without a vpn" "Ok sure"

pearl trail Jun 30, 2024, 4:03 AM

#

@civic scroll can I handle db like insert on nextjs?

sharp geyser Jun 30, 2024, 4:03 AM

#

If I tried to protect people from their stupidity I would spend more time on that then actual features

sharp geyser Jun 30, 2024, 4:04 AM

#

pearl trail <@520964894279860224> can I handle db like insert on nextjs?

Wym

#

You should not be making any direct db calls from nextjs

#

You should have an API you call to that handles that for you

civic scroll Jun 30, 2024, 4:04 AM

#

pearl trail <@520964894279860224> can I handle db like insert on nextjs?

nextjs had serverside code

#

which means you can setup api routes

sharp geyser Jun 30, 2024, 4:05 AM

#

do people actually use that tho?

civic scroll Jun 30, 2024, 4:05 AM

#

why not?

sharp geyser Jun 30, 2024, 4:05 AM

#

Well, sure if its something small

#

but anything big I advise breaking it off

pearl trail Jun 30, 2024, 4:05 AM

#

civic scroll nextjs had serverside code

yeah, it won't appear on the client's "application" right?

civic scroll Jun 30, 2024, 4:05 AM

#

you have types usable by both frontend and backend

pearl trail Jun 30, 2024, 4:06 AM

#

well in react you can disable that on build

civic scroll Jun 30, 2024, 4:06 AM

#

pearl trail yeah, it won't appear on the client's "application" right?

yes

sharp geyser Jun 30, 2024, 4:06 AM

#

right, but thats only helpful if its a small api

#

anything big and complex is a bad idea to use that

civic scroll Jun 30, 2024, 4:06 AM

#

you can break it out

sharp geyser Jun 30, 2024, 4:06 AM

#

not to mention any big api should not be written in js

civic scroll Jun 30, 2024, 4:06 AM

#

in fact iirc nextjs has a separated foldersfor api endpoints

sharp geyser Jun 30, 2024, 4:06 AM

#

it does indeed

civic scroll Jun 30, 2024, 4:07 AM

#

sharp geyser not to mention any big api should not be written in js

nestjs: omE

sharp geyser Jun 30, 2024, 4:07 AM

#

yea sure nestjs is cool

#

and its fine for something small scale

pearl trail Jun 30, 2024, 4:07 AM

#

civic scroll yes

oki, last question, can I do anything on the nextjs server? like I can install any npm that runs on node

civic scroll Jun 30, 2024, 4:07 AM

#

there is elysia
the creator is a cosplayer

#

ExuStare

sharp geyser Jun 30, 2024, 4:07 AM

#

but there's a reason companies don't use js for enterprise level backend apis

pearl trail Jun 30, 2024, 4:07 AM

#

maybe like canvas, jimp, sharp

sharp geyser Jun 30, 2024, 4:07 AM

#

its not efficient

civic scroll Jun 30, 2024, 4:07 AM

#

pearl trail oki, last question, can I do anything on the nextjs server? like I can install a...

again, nestjs is server-first

pearl trail Jun 30, 2024, 4:08 AM

#

okokokok

civic scroll Jun 30, 2024, 4:08 AM

#

in SSR and and ISR configurations, you code runs on the server

pearl trail Jun 30, 2024, 4:08 AM

#

thanks

sharp geyser Jun 30, 2024, 4:08 AM

#

civic scroll there is elysia the creator is a cosplayer

uses bun

#

💀

civic scroll Jun 30, 2024, 4:08 AM

#

fuck.

#

CluelessChatting

sharp geyser Jun 30, 2024, 4:08 AM

#

so not node

civic scroll Jun 30, 2024, 4:08 AM

#

ima make elysia in node and then cosplay

sharp geyser Jun 30, 2024, 4:09 AM

#

😭

civic scroll Jun 30, 2024, 4:09 AM

#

comptetitor™

sharp geyser Jun 30, 2024, 4:09 AM

#

I stand by that js is terrible for enterprise level abckend apis

#

its not efficient and wont hold up to that amount of traffic

civic scroll Jun 30, 2024, 4:10 AM

#

yes BUT

#

tbh no enterprise has only one person for backend

#

EXCEPT

#

THE USS ENTERPRISE 🦅🇺🇸🚢

pearl trail Jun 30, 2024, 4:16 AM

#

https://tenor.com/view/azur-lane-enterprise-anime-belfast-uss-enterprise-gif-17744347

Tenor

#

thanks misty, now I'm interested in using ruby on rails for rest api

sharp geyser Jun 30, 2024, 5:28 AM

#

pearl trail thanks misty, now I'm interested in using ruby on rails for rest api

thats not a bad choice if you like ruby

#

My only thing is, if its something small js can definitely do it

#

but if you are getting thousands or hundreds of thousands of people hitting your backend api it becomes a problem

#

js isn't meant to handle that, and to make it even get close to being able to handle something like that is more work than its worth

pearl trail Jun 30, 2024, 6:01 AM

#

sharp geyser thats not a bad choice if you like ruby

yeah i want to learn it too

#

looks fast 🚀

radiant kraken Jun 30, 2024, 6:50 AM

#

pearl trail thanks misty, now I'm interested in using ruby on rails for rest api

what api are you making

pearl trail Jun 30, 2024, 7:04 AM

#

radiant kraken what api are you making

🤫

radiant kraken Jun 30, 2024, 7:09 AM

#

pearl trail 🤫

tell smh

warm surge Jun 30, 2024, 7:22 AM

#

  if (n < 1e3) return n;
  if (n >= 1e3 && n < 1e6) return +(n / 1e3).toFixed(1) + "K";
  if (n >= 1e6 && n < 1e9) return +(n / 1e6).toFixed(1) + "M";
  if (n >= 1e9 && n < 1e12) return +(n / 1e9).toFixed(1) + "B";
  if (n >= 1e12) return +(n / 1e12).toFixed(1) + "T";
};

warm surge Jun 30, 2024, 7:22 AM

#

warm surge ```function Formatter (n) { if (n < 1e3) return n; if (n >= 1e3 && n < 1e6) ...

for pixxiebot

sharp geyser Jun 30, 2024, 7:22 AM

#

WHAT

#

what even is this

warm surge Jun 30, 2024, 7:22 AM

#

sharp geyser what even is this

formats numbers

sharp geyser Jun 30, 2024, 7:24 AM

#

Yea

#

I understand that

#

but why

warm surge Jun 30, 2024, 7:24 AM

#

its just suggestions for his rpc

rose warren Jun 30, 2024, 7:25 AM

#

My rpc is how I want it to be mayaLaugh

warm surge Jun 30, 2024, 7:25 AM

#

rose warren My rpc is how I want it to be <:mayaLaugh:1091251004101644308>

oki

sharp geyser Jun 30, 2024, 8:27 AM

#

@pearl trail question

#

<html lang='en'>
      <body>
        <div className='flex flex-row'>
          <Drawer />
          <Navbar />
          {children}
        </div>
      </body>
    </html>

#

not an optimal result

#

the Hello all the way at the right is the {children}

#

Hmm

pearl trail Jun 30, 2024, 8:28 AM

#

where do you want to have the {children}

sharp geyser Jun 30, 2024, 8:29 AM

#

I guess in the empty whitespace

#

Idk how to describe it

#

Ideally the content of each page should be surrounded by the navbar and drawer

pearl trail Jun 30, 2024, 8:30 AM

#

<html lang='en'>
      <body>
        <div className='flex flex-row'>
          <Drawer />
+          <div className="w-full">
            <Navbar />
            {children}
+          </div>
        </div>
      </body>
</html>

#

hope the width works

sharp geyser Jun 30, 2024, 8:30 AM

#

:o

#

why tf does that work

surreal sage Jun 30, 2024, 8:31 AM

#

thought i had a hdd failure just now, just wasn't plugged in right..

pearl trail Jun 30, 2024, 8:32 AM

#

before, you have

- drawer
- navbar
- hello

those 3 are in 1 flex-row, so you gotta do

- drawer
- div
  - navbar
  - hello

there are only 2 that will be "flexed", drawer and div with navbar and hello

sharp geyser Jun 30, 2024, 8:34 AM

#

ah shit

#

I just realized that w-full is likely going to interfear with me resizing the drawer's width no?

#

oh no wait

#

its me doing w-screen inside navbar

pearl trail Jun 30, 2024, 8:34 AM

#

yes

#

w-full is 100%

sharp geyser Jun 30, 2024, 8:36 AM

#

yea but that wasn't what was stopping me

#

it was me doing w-screen inside navbar's main element

#

I 100% will need to change the color scheme at a certain point

#

but for now i cba

#

I just want the stuff there

#

I can fuck with the color scheme later

surreal sage Jun 30, 2024, 9:29 AM

#

sudo give me purpose

neon leaf Jun 30, 2024, 9:30 AM

#

alias fucking='sudo'
fucking apt update && fucking apt full-upgrade -y

sharp geyser Jun 30, 2024, 9:30 AM

#

pearl trail yes

im doing away with the entire drawer + navbar thing

#

navbar only

#

sunglas

#

the drawer looks UGLY

#

at least open 24/7

surreal sage Jun 30, 2024, 9:36 AM

#

neon leaf alias fucking='sudo' fucking apt update && fucking apt full-upgrade -y

im doing this holy shit

quartz kindle Jun 30, 2024, 9:59 AM

#

warm surge ```function Formatter (n) { if (n < 1e3) return n; if (n >= 1e3 && n < 1e6) ...

no need to repeat conditions that were already eliminated before

function Formatter (n) {
  if (n < 1e3) return n;
  if (n < 1e6) return (n / 1e3).toFixed(1) + "K";
  if (n < 1e9) return (n / 1e6).toFixed(1) + "M";
  if (n < 1e12) return (n / 1e9).toFixed(1) + "B";
  return (n / 1e12).toFixed(1) + "T";
};

sharp geyser Jun 30, 2024, 11:48 AM

#

What kind of algorithm should I use to show the user most liked videos in two scenarios

They are not subscribed to anyone, show them most liked videos on topics they choose
They are subscribed to people, show them their subscribed content creator's videos first, and then any other videos that are similar

past field Jun 30, 2024, 12:07 PM

#

ok question

#

i have a game that i’ve created called click war

#

and idk if this makes sense

#

but is there a way to add every game state to the database? incase the bot crashes while the lobby is open? or crashes during a game in progress? so it’ll continue when the bot is turned back on

quartz kindle Jun 30, 2024, 12:09 PM

#

past field but is there a way to add every game state to the database? incase the bot crash...

yes

neon flicker Jun 30, 2024, 12:16 PM

#

In order to initialize both the dashboard and the bot, I should prepare a mutual main .js file which both sets up the bot and configurate the routing right?

pearl trail Jun 30, 2024, 12:25 PM

#

yes

#

just make sure both bot and webserver can access each other if you want your webserver to access some data from bot

sharp geyser Jun 30, 2024, 12:30 PM

#

quartz kindle yes

gm tim

#

:D

past field Jun 30, 2024, 12:55 PM

#

quartz kindle yes

thank you!

quartz kindle Jun 30, 2024, 1:02 PM

#

sharp geyser gm tim

yes

sharp geyser Jun 30, 2024, 1:03 PM

#

;c

deft wolf Jun 30, 2024, 1:05 PM

#

neon flicker Jun 30, 2024, 2:07 PM

#

<p className="fixed left-0 top-0 flex w-full justify-center border-b border-gray-300 bg-gradient-to-b from-zinc-200 pb-6 pt-8 backdrop-blur-2xl dark:border-neutral-800 dark:bg-zinc-800/30 dark:from-inherit lg:static lg:w-auto  lg:rounded-xl lg:border lg:bg-gray-200 lg:p-4 lg:dark:bg-zinc-800/30">

I'm a little confused with those long-named class names, are they Next.js classes? If so, where can I view the other classes as well?
It is from the default Next.js project

#

Oh, it's tailwind

warm surge Jun 30, 2024, 2:53 PM

#

quartz kindle no need to repeat conditions that were already eliminated before ```js function ...

what?

surreal sage Jun 30, 2024, 2:54 PM

#

interesting

#

36 bytes per second 🔥

quartz kindle Jun 30, 2024, 3:11 PM

#

warm surge what?

you return if n < 1e3, so afterwards you dont need to check if n >= 1e3 because you know for sure it is, otherwise it would have returned

sharp geyser Jun 30, 2024, 3:18 PM

#

right guys

#

this seems very empty

#

Sadge

warm surge Jun 30, 2024, 3:23 PM

#

quartz kindle you return if n < 1e3, so afterwards you dont need to check if n >= 1e3 because ...

Oh

deft wolf Jun 30, 2024, 3:23 PM

#

sharp geyser this seems very empty

Add some ads weirdsip

neon leaf Jun 30, 2024, 3:32 PM

#

dam

#

my private bot used in 2 servers got quarantined

neon flicker Jun 30, 2024, 3:32 PM

#

Lol for what?

neon leaf Jun 30, 2024, 3:32 PM

#

neon flicker Jun 30, 2024, 3:33 PM

#

What might have caused it tho?

neon leaf Jun 30, 2024, 3:33 PM

#

i mean

#

only thing the bot does that could possibly cause this would be the demo system, a user runs the command, after 1 hour he gets a dm that demo expired and that command has a 24h cooldown

#

may be that its sus for discord when the app had like 5 people get dms on the day the bot was made

#

because people tried demo

solemn latch Jun 30, 2024, 3:34 PM

#

Join messages/dm? 👀

neon leaf Jun 30, 2024, 3:34 PM

#

#

only place where dms are sent

solemn latch Jun 30, 2024, 3:35 PM

#

Thats odd

#

Discords anti spam is too strong

neon flicker Jun 30, 2024, 3:35 PM

#

@solemn latch may I please ask a question like:
Do I have to set up the routing in my main JS file or does Next.js do it for me automatically depending on the folder rotation?

#

Or do I somehow need to initialize it?

sharp geyser Jun 30, 2024, 3:36 PM

#

nextjs if you are using the app router does it all for you

#

it goes based off folder name

solemn latch Jun 30, 2024, 3:36 PM

#

yeah

sharp geyser Jun 30, 2024, 3:36 PM

#

src
--app
---about
----page.tsx
---page.tsx

solemn latch Jun 30, 2024, 3:36 PM

#

https://nextjs.org/docs/app/building-your-application/routing/pages-and-layouts

Routing: Pages and Layouts | Next.js

Create your first page and shared layout with the App Router.

sharp geyser Jun 30, 2024, 3:36 PM

#

the page.tsx file in about folder is that folder's "index.html"

#

the page.tsx file in the app folder is your root / "index.html"

#

example.com -> "app/page.tsx"
example.com/about -> "about/page.tsx"

sharp geyser Jun 30, 2024, 3:37 PM

#

deft wolf Add some ads <:weirdsip:585884223441862656>

no

#

Not on the login/join page

#

😭

solemn latch Jun 30, 2024, 3:38 PM

#

sharp geyser right guys

Add a dark grey container around the login stuff 😄

sharp geyser Jun 30, 2024, 3:38 PM

#

#

Here is my updated stuff

solemn latch Jun 30, 2024, 3:39 PM

#

👀 not using shadcn? angeryBOYE

neon flicker Jun 30, 2024, 3:39 PM

#

Thank you, tho I have some non-routing related questions yet like
I'm confused how to create some sort of "search" box, like when they search something through it how will I create components that will show the results?

solemn latch Jun 30, 2024, 3:40 PM

#

sharp geyser

wait, that is shadcn isnt it?

solemn latch Jun 30, 2024, 3:40 PM

#

neon flicker Thank you, tho I have some non-routing related questions yet like I'm confused h...

Have you done your general layout yet?

neon flicker Jun 30, 2024, 3:40 PM

#

Not really

solemn latch Jun 30, 2024, 3:41 PM

#

I would wait to worry about search stuff until later.

neon flicker Jun 30, 2024, 3:41 PM

#

You're right but it's such a major question mark in my head

#

But yes I think I should start with making the layout first

sharp geyser Jun 30, 2024, 3:43 PM

#

solemn latch 👀 not using shadcn? <:angeryBOYE:588492565712601109>

using nextui

solemn latch Jun 30, 2024, 3:44 PM

#

neon flicker You're right but it's such a major question mark in my head

Once you play with layout, and general stuff you'll learn a few things along the way like states and useEffect, which will help with how search will work.

Out of all the things in your dashboard search will probably be the most complicated, but it will be way easier to understand after you've displayed information and whatnot.

neon flicker Jun 30, 2024, 3:45 PM

#

Thank you, it will help me

solemn latch Jun 30, 2024, 3:47 PM

#

I'm not sure if this approach is right for you right now.

But a component library might be useful.
When I was learning all of this stuff I didn't use component libraries first, I learned everything else then came back to component libraries.
Its up to you what way you want to go.
https://ui.shadcn.com/
https://nextui.org/

shadcn/ui

Beautifully designed components that you can copy and paste into your apps. Accessible. Customizable. Open Source.

NextUI

NextUI - Beautiful, fast and modern React UI Library

Beautiful, fast and modern React UI Library

#

I'm a big fan of shadcn, but I've only heard good things about nextui too.

neon flicker Jun 30, 2024, 3:49 PM

#

Thank you a lot, I really appreciate it again

celest copper Jun 30, 2024, 3:52 PM

#

Well, What that total revenue actual indicate in the 1st image is it of user

solemn latch Jun 30, 2024, 3:53 PM

#

Thats just a reference image to what a card looks like in shadcn

neon flicker Jun 30, 2024, 3:53 PM

#

Where may I view tailwind classes such as this?

solemn latch Jun 30, 2024, 3:54 PM

#

This is what I use.
https://tailwindcomponents.com/cheatsheet/

Tailwind CSS CheatSheet for Beginners and Not Only

Find quickly all the class names and CSS properties with this interactive cheat sheet. The only Tailwind CheatSheet you will ever need!

#

I pretty much always have it open on an extra monitor when working in tailwind

neon flicker Jun 30, 2024, 3:54 PM

#

Wow thank you

solemn latch Jun 30, 2024, 3:55 PM

#

If you use Visual studio code I think theres an extension for tailwind too.

#

for autocomplete

neon flicker Jun 30, 2024, 3:55 PM

#

I'd really appreciate it

solemn latch Jun 30, 2024, 3:56 PM

#

https://marketplace.visualstudio.com/items?itemName=bradlc.vscode-tailwindcss

neon flicker Jun 30, 2024, 3:56 PM

#

Is it called Tailwind CSS IntelliSense`?

solemn latch Jun 30, 2024, 3:56 PM

#

yep!

neon flicker Jun 30, 2024, 3:57 PM

#

I'm starting to build the project, I'll start with a basic design

#

As "development" points out that I can always make it better with time so

neon flicker Jun 30, 2024, 4:16 PM

#

Good, I started getting confused with JSX already

solemn latch Jun 30, 2024, 4:16 PM

#

whats got you confused?

neon flicker Jun 30, 2024, 4:18 PM

#

Hydration failed because the initial UI does not match what was rendered on the server.

<body>
<ul style ={{position:'sticky'}}>
  
</ul>
</body>

I just started yet, lol

#

Or I just have lack of HTML knowledge yet

solemn latch Jun 30, 2024, 4:19 PM

#

Do you have a layout file?

neon flicker Jun 30, 2024, 4:20 PM

#

Yes, this is what it looks like:

import { Inter } from "next/font/google";
import "./globals.css";

const inter = Inter({ subsets: ["latin"] });

export const metadata = {
  title: "Page",
  description: "Generated by create next app",
};

export default function RootLayout({ children }) {
  return (
    <html lang="en">
      <body className={inter.className}>{children}</body>
    </html>
  );
}

solemn latch Jun 30, 2024, 4:20 PM

#

It might be because you already have a body

neon flicker Jun 30, 2024, 4:21 PM

#

Oh, you're right

solemn latch Jun 30, 2024, 4:21 PM

#

Hydration errors are funky right now, in the next 6 months ish they should get better

neon flicker Jun 30, 2024, 4:22 PM

#

It's weird tho, should I remove the body from the layout file?

#

Ah yes it worked to do so

solemn latch Jun 30, 2024, 4:22 PM

#

You want your body to be in the layout, as that should be consistent across all pages.

neon flicker Jun 30, 2024, 4:23 PM

#

So the layout isn't supposed to be included in every single page right?

#

Okay, I thought Next.js required it to do it

solemn latch Jun 30, 2024, 4:24 PM

#

Yeah, your root layout will be across all pages

#

https://nextjs.org/docs/app/building-your-application/routing/pages-and-layouts#root-layout-required

Routing: Pages and Layouts | Next.js

Create your first page and shared layout with the App Router.

neon flicker Jun 30, 2024, 4:26 PM

#

Ah thank you

#

Yes, I think I have a lot more to worry about before making a search box

#

I don't even know why those rectangle backgrounds are appearing around

eternal osprey Jun 30, 2024, 5:07 PM

#

 // Send email
    await transporter.sendMail(mailOptions1);
     await transporter.sendMail(mailOptions2);
     await transporter.sendMail(mailOptions3);
 return res.status(200).send({ message: "Received successfully" });```
i have a problem.. when i send the mails with await it is basically making the user wait. I don't want to do that.
Is there a way to send one email, then directly return the status code so that the user doesn't have to wait and then send the other 2 mails (these are admin mails).

#

there's currently a delay of like 5 seconds or something

#

i am deadass r3tarded cuh

#

simply remove the return brev KEKW

#

or i implement a loader y aknow that's also a possibility.

pearl trail Jun 30, 2024, 5:19 PM

#

ayo chat is this real

eternal osprey Jun 30, 2024, 5:20 PM

#

Guys i have a service running on port 8443, when i use pm2 restart does it autoamtically release the port or will it block the fucking port?

#

in any case, how can i release the port gracefully?

pearl trail Jun 30, 2024, 5:20 PM

#

it's not docker so it's open to host

eternal osprey Jun 30, 2024, 5:20 PM

#

wym gang

pearl trail Jun 30, 2024, 5:21 PM

#

wdym release the port

eternal osprey Jun 30, 2024, 5:21 PM

#

i have a process running on port 8443. When i use pm2 restart, it creates a new instance of that process and closes the old one.

#

I am scared that it will return me that the port is still occupied.

pale vessel Jun 30, 2024, 5:21 PM

#

are you using the same port every time?

#

if it restarts then it won't be occupied until it uses it again

eternal osprey Jun 30, 2024, 5:22 PM

#

i have one process using 8443 yeah

pearl trail Jun 30, 2024, 5:22 PM

#

on sigterm, do server.close()

#

iirc it's sigterm

#

or sigkill

pearl trail Jun 30, 2024, 5:23 PM

#

pale vessel if it restarts then it won't be occupied until it uses it again

and also this ^

#

kinda rare stopped process still have the port occupied

eternal osprey Jun 30, 2024, 5:24 PM

#

ahh hell nah:

29|EMRE_OFFERTE  | Error: listen EADDRINUSE: address already in use :::8443
29|EMRE_OFFERTE  |     at Server.setupListenHandle [as _listen2] (node:net:1751:16)
29|EMRE_OFFERTE  |     at listenInCluster (node:net:1799:12)
29|EMRE_OFFERTE  |     at Server.listen (node:net:1887:7)
29|EMRE_OFFERTE  |     at Object.<anonymous> (/root/site/offerte.js:267:13)
29|EMRE_OFFERTE  |     at Module._compile (node:internal/modules/cjs/loader:1256:14)
29|EMRE_OFFERTE  |     at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
29|EMRE_OFFERTE  |     at Module.load (node:internal/modules/cjs/loader:1119:32)
29|EMRE_OFFERTE  |     at Module._load (node:internal/modules/cjs/loader:960:12)
29|EMRE_OFFERTE  |     at Object.<anonymous> (/usr/lib/node_modules/pm2/lib/ProcessContainerFork.js:33:23)
29|EMRE_OFFERTE  |     at Module._compile (node:internal/modules/cjs/loader:1256:14) {
29|EMRE_OFFERTE  |   code: 'EADDRINUSE',
29|EMRE_OFFERTE  |   errno: -98,
29|EMRE_OFFERTE  |   syscall: 'listen',
29|EMRE_OFFERTE  |   address: '::',
29|EMRE_OFFERTE  |   port: 8443
29|EMRE_OFFERTE  | }```

pale vessel Jun 30, 2024, 5:27 PM

#

what shows up if you run pm2 list?

#

just that one same process?

eternal osprey Jun 30, 2024, 5:28 PM

#

root@awsomecordServices:~/site# ps aux | grep 'Z'
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     3310215  0.0  0.0   8900   740 pts/0    S+   19:27   0:00 grep --color=auto Z```
yeah, but as i though pm2 restart does not close the ports, but it closes the process. So the process is closed, but the host still has it bound to the port.

#

it's a zombie process now

pale vessel Jun 30, 2024, 5:28 PM

#

i thought zombie proceses can't hold ports hmm

pearl trail Jun 30, 2024, 5:29 PM

#

🧟

pale vessel Jun 30, 2024, 5:29 PM

#

what happens when you do pm2 stop first and then pm2 restart?

eternal osprey Jun 30, 2024, 5:29 PM

#

same thing

pale vessel Jun 30, 2024, 5:29 PM

#

did you do that when it worked the first time?

#

dunno tbh, pm2 restart should work the same as stopping and starting

eternal osprey Jun 30, 2024, 5:30 PM

#

oh shit

#

i apparently assigned like 4 containers to one index file

#

so when i did pm2 restart index -> it started all 4 containers

pale vessel Jun 30, 2024, 5:31 PM

#

LUL

#

may as well use docker if you're doing containers

eternal osprey Jun 30, 2024, 5:33 PM

#

yeah fr

pearl trail Jun 30, 2024, 5:36 PM

#

#

jokes aside, I love docker

eternal osprey Jun 30, 2024, 5:36 PM

#

i got some bad memories in terms of using docker so i kinda hate it

neon flicker Jun 30, 2024, 5:38 PM

#

neon flicker Yes, I think I have a lot more to worry about before making a search box

Does anybody have any idea why it's occurring?

#

Okay, I found it out

quartz kindle Jun 30, 2024, 5:54 PM

#

pearl trail ayo chat is this real

:^)

neon flicker Jun 30, 2024, 5:55 PM

#

So I didn't figure it out

pearl trail Jun 30, 2024, 5:55 PM

#

quartz kindle :^)

uh 💀 is that execution time/task?

quartz kindle Jun 30, 2024, 5:55 PM

#

pearl trail uh 💀 is that execution time/task?

ye

pearl trail Jun 30, 2024, 5:56 PM

#

unbelievable 💀💀💀

earnest phoenix Jun 30, 2024, 5:56 PM

#

time to use fastify trolley

quartz kindle Jun 30, 2024, 5:56 PM

#

i mean, you cant always trust random benchmarks like that

#

but here is the source

#

https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/node-ruby.html

Node.js vs Ruby yjit - Which programs are fastest? (Benchmarks Game...

Node.js Ruby yjit - Which programs have fastest performance?

#

some of them went in ruby's favor

#

#

took them 2.5gb of ram tho

#

:^)

pearl trail Jun 30, 2024, 5:58 PM

#

uhmmmm don't make me use rust yet

#

earnest phoenix Jun 30, 2024, 6:00 PM

#

pearl trail uhmmmm don't make me use rust yet

shrimply use js for everything

#

actually why dont you use c# for backend too

pearl trail Jun 30, 2024, 6:01 PM

#

I want to try smth different

#

Performance:

Express is faster than Rails probably because of its ability to process asynchronous I/O calls
However, tasks that entail more cpu intensive requirements generally perform better with Rails

#

icic alr

#

imma do my own benchmark later

quartz kindle Jun 30, 2024, 6:02 PM

#

why is the comparison express and not node

#

:^)

pearl trail Jun 30, 2024, 6:02 PM

#

mmLul because that's what I can do

quartz kindle Jun 30, 2024, 6:03 PM

#

express is thresh

pearl trail Jun 30, 2024, 6:05 PM

#

well ngl what you said is not entirely wrong

#

KEK

surreal sage Jun 30, 2024, 6:06 PM

#

pearl trail uhmmmm don't make me use rust yet

pls no rust

#

the learning curve

#

is beyond long

pearl trail Jun 30, 2024, 6:07 PM

#

lmao

#

that's my last choice dw

charred nest Jun 30, 2024, 6:07 PM

#

ive been lurking over waiting for another cpu spike and todays the day lolSPIN i did this, the only thing ive noticed is that one feature gradually slows down when its ran (goes from 0.1 ms and gradually increases to 1400+ ms)

pearl trail Jun 30, 2024, 6:15 PM

#

ok I decided to use golang, thanks Tim

sharp geyser Jun 30, 2024, 6:57 PM

#

neon flicker Yes, I think I have a lot more to worry about before making a search box

You have a root layout, and then you can set layouts for each page as well if you want

neon flicker Jun 30, 2024, 7:13 PM

#

I think I'm still confused, I don't know which property causes it

#

But I somehow fixed it so I think there's nothing to worry about for now

surreal sage Jun 30, 2024, 7:18 PM

#

its not that deep docker

sharp geyser Jun 30, 2024, 7:18 PM

#

apparently it is

surreal sage Jun 30, 2024, 7:18 PM

#

2 minutes now

#

im guessing its copying my entire 8gb dir

#

damn

#

shoutout to rust for that...

#

#

im so confused rn

#

#

#

damn keyboard double pressed u again

neon leaf Jun 30, 2024, 7:35 PM

#

idk why but double presses have been going up over the last few weeks for me

surreal sage Jun 30, 2024, 7:36 PM

#

only happens like every 30th time for me

#

might be just that my keyboard is hella dirty

neon leaf Jun 30, 2024, 7:38 PM

#

im blaming windows

#

my keyboard is brand new

#

its mostly on c

#

wtf

surreal sage Jun 30, 2024, 7:38 PM

#

wtffff

neon leaf Jun 30, 2024, 7:38 PM

#

it just did a triple press

surreal sage Jun 30, 2024, 7:39 PM

#

new record

neon flicker Jun 30, 2024, 8:12 PM

#

Is there a way to use to App Router to seperate url.com/home and url.com/home#updates?

lament rock Jun 30, 2024, 8:14 PM

#

# is usually a markdown thing that just jumps you to a part of the document. I wouldn't recommend intentionally breaking that convention

wheat mesa Jun 30, 2024, 8:15 PM

#

that was aggressive

surreal sage Jun 30, 2024, 8:15 PM

#

WHAT??

#

docker??!?!?

#

😭

lament rock Jun 30, 2024, 8:15 PM

#

Look. I forgot # did that in Discord

surreal sage Jun 30, 2024, 8:15 PM

#

https://tenor.com/view/breaking-bad-walter-walter-white-sus-face-app-gif-23028540

Tenor

neon flicker Jun 30, 2024, 8:15 PM

#

So should I use url.com/home/updates instead?

lament rock Jun 30, 2024, 8:16 PM

#

Yeah

wheat mesa Jun 30, 2024, 8:16 PM

#

neon flicker Is there a way to use to App Router to seperate `url.com/home` and `url.com/home...

This is something you would just use another route for. home/updates

lament rock Jun 30, 2024, 8:16 PM

#

or just /updates

wheat mesa Jun 30, 2024, 8:16 PM

#

Yea

neon flicker Jun 30, 2024, 8:19 PM

#

Well, it's going to be hard to create that dashboard

neon flicker Jun 30, 2024, 8:35 PM

#

Error: Hydration failed because the initial UI does not match what was rendered on the server.
Expected server HTML to contain a matching <ul> in <body>.
This error is occuring again, it seems to fix it to remove the body from root layout, but then it breaks the whole site

dawn socket Jun 30, 2024, 8:37 PM

#

how can i add discord support server rather than top.gg server

neon flicker Jun 30, 2024, 8:43 PM

#

neon flicker `Error: Hydration failed because the initial UI does not match what was rendered...

Oh my God, the error message was telling me what to do whole the time and I just didn't notice it

#

Programming moment, anyways

#

Nevermind, it just sometimes works and it sometimes doesn't

dawn socket Jun 30, 2024, 8:45 PM

#

dawn socket how can i add discord support server rather than top.gg server

any help

surreal sage Jun 30, 2024, 8:52 PM

#

neon flicker `Error: Hydration failed because the initial UI does not match what was rendered...

react strictmode

neon flicker Jun 30, 2024, 8:53 PM

#

surreal sage react strictmode

Well, what's funny is that there's nothing wrong in the surface

green kestrel Jun 30, 2024, 8:57 PM

#

you know your bot might be using too many emojis when:

digital swan Jun 30, 2024, 8:58 PM

#

same

green kestrel Jun 30, 2024, 8:59 PM

#

lol, theres also three more servers im using for the bots emojis but those are my main support server, and two other bots support servers

green kestrel Jun 30, 2024, 9:00 PM

#

digital swan same

what does your bot do

digital swan Jun 30, 2024, 9:00 PM

#

multi purpose 🤓 but all the emojis are for economy stuff

#

and wordle

deft wolf Jun 30, 2024, 9:02 PM

#

green kestrel you know your bot might be using too many emojis when:

I remember some time ago someone wanted to add a Pokemon emote in the embed with its name for every possible Pokemon in the Pokedex

#

rotund slate Jun 30, 2024, 9:04 PM

#

idk why my bot server and shards count isnt increasing

#

result = await self.get_stats()
            data = {"server_count": int(result["servers"]), "shard_count": int(result["shards"])}

            headers = headers = {
                "Authorization": "secret",
            }
            URL = API_BASE_URL + "/bots/{}/stats".format(self.bot.user.id)
            await self.bot.session.post(URL, headers=headers, data=data)

#

here is my code

frosty gale Jun 30, 2024, 9:05 PM

#

deft wolf

i would make a bot that automatically uploads the emojis for everything as i create new servers

rotund slate Jun 30, 2024, 9:05 PM

#

this is my get_stats function code

neon flicker Jun 30, 2024, 9:06 PM

#

<div> with 100% width doesn't cover the whole X axis

#

I feel like I'm going to die

rotund slate Jun 30, 2024, 9:06 PM

#

#

and that code is working perfect idk why its not getting update

surreal sage Jun 30, 2024, 9:08 PM

#

docker try not to ignore the cache level impossible

rotund slate Jun 30, 2024, 9:09 PM

#

?

dawn socket Jun 30, 2024, 9:13 PM

#

dawn socket how can i add discord support server rather than top.gg server

no help?

primal lynx Jun 30, 2024, 9:16 PM

#

green kestrel you know your bot might be using too many emojis when:

Same

quartz kindle Jun 30, 2024, 9:30 PM

#

rotund slate idk why my bot server and shards count isnt increasing

when was the last time it updated on the site? could be the site cache displaying old data

#

try accessing on another device

surreal sage Jun 30, 2024, 9:31 PM

#

ctrl f5 and incognito (+ network tab "disable cache") my beloved

quartz kindle Jun 30, 2024, 9:31 PM

#

otherwise try manually making a post request to the website and see if it works

rotund slate Jun 30, 2024, 9:31 PM

#

quartz kindle when was the last time it updated on the site? could be the site cache displayin...

Its very long time ago

#

Last year i think

quartz kindle Jun 30, 2024, 9:32 PM

#

try to make a manual request with postman or reqbin or something

rotund slate Jun 30, 2024, 9:32 PM

#

Ohk let me try

surreal sage Jun 30, 2024, 9:40 PM

#

WHY IS TS TWEAKING FOR SOME REASON

#

ts extension rn: bye

frosty gale Jun 30, 2024, 9:56 PM

#

surreal sage WHY IS TS TWEAKING FOR SOME REASON

the problem is that you are using react

#

it is the only times i experience issues with my extensions or compilation

#

is when i use react

#

it has more dependencies than a drug addict

quartz kindle Jun 30, 2024, 9:58 PM

#

react is so slow and bulky that it doesnt actually react to anything in time

frosty gale Jun 30, 2024, 10:05 PM

#

lol

#

it just feels wrong using it

#

maintainers must be the same people installing is-odd on npm

surreal sage Jun 30, 2024, 10:29 PM

#

ok docker tweaking?

#

multi layer image

build: node:22-alpine
package: node:22-slim
324 mb

or

build: node:22
package: node:22-alpine
265mb

quartz kindle Jun 30, 2024, 10:36 PM

#

lol what is the actual difference betwen those?

surreal sage Jun 30, 2024, 10:36 PM

#

alpine is a way smaller version of normal

#

slim is just node

#

like bare bones

#

slim should be smaller

#

so i dont get it

#

node:22 > node:22-alpine > node:22-slim

#

in sizes (big to small)

quartz kindle Jun 30, 2024, 10:37 PM

#

whats the difference between build and package tho

surreal sage Jun 30, 2024, 10:37 PM

#

oh multi layer

#

build just builds the project

#

package just minimizes it completely

#

eternal osprey Jun 30, 2024, 10:37 PM

#

i was using spark to view github code and classify them based on language, but one thing that i really noticed was that many people in js in particular tend to do this:

if(condition1){
return ...
}else{
codeblock1
}

but can't you instead use this format to make it more readable/compact?

if(condition1){
return ...
}
codeblock1

surreal sage Jun 30, 2024, 10:38 PM

#

yes...

#

you dont even need the brackets for the if either 😭 (should use if it's only a return ...

quartz kindle Jun 30, 2024, 10:38 PM

#

eternal osprey i was using spark to view github code and classify them based on language, but o...

i often do the first one as well at first, makes it easier to think in the head, but later on when refactoring i change it to the second one

eternal osprey Jun 30, 2024, 10:38 PM

#

no like deadass 8gb of html data got me tripping.All of em use the first construct

eternal osprey Jun 30, 2024, 10:39 PM

#

surreal sage you dont even need the brackets for the if either 😭 (should use if it's only a ...

i know

eternal osprey Jun 30, 2024, 10:39 PM

#

quartz kindle i often do the first one as well at first, makes it easier to _think_ in the hea...

ah i see

quartz kindle Jun 30, 2024, 10:40 PM

#

surreal sage multi layer image build: node:22-alpine package: node:22-slim 324 mb or build...

cant you do both build and package node:22-alpine?

surreal sage Jun 30, 2024, 10:40 PM

#

hmmmmmm

quartz kindle Jun 30, 2024, 10:40 PM

#

or both slim

surreal sage Jun 30, 2024, 10:40 PM

#

slim big no

#

its just an empty system with node

quartz kindle Jun 30, 2024, 10:41 PM

#

sure, run node as a bootloader and ditch the os completely

#

:^)

surreal sage Jun 30, 2024, 10:41 PM

#

ong

#

ong

#

ong

#

hardcoded iso just to run 1 thing

surreal sage Jun 30, 2024, 10:43 PM

#

quartz kindle cant you do both build and package node:22-alpine?

size doesnt change

#

only the package layer matters

quartz kindle Jun 30, 2024, 10:43 PM

#

so what OS does node slim use?

surreal sage Jun 30, 2024, 10:44 PM

#

alpine i think

quartz kindle Jun 30, 2024, 10:44 PM

#

debian

#

theres your answer

surreal sage Jun 30, 2024, 10:44 PM

#

p sure node:x is debian

#

and not node:x-slim and x-alpine

quartz kindle Jun 30, 2024, 10:44 PM

#

surreal sage Jun 30, 2024, 10:45 PM

#

oh wtf

eternal osprey Jun 30, 2024, 10:48 PM

#

I do have a problem though guys.
I have around 8tb of data.
I am looking to reduce it to like, 800gb?
So what i came up with was using bloom filters (basically hashes) to check whether certain prominent features of languages fit in a category.
However as of now, my data gets tripled for some reason. I went from 140k entries to 7 million and i have no clue why:
Below snippet is the prominent feature extractor..

val filteredLinks = whs.filter { case (title, _, _) =>
  val lowercaseTitle = title.toLowerCase
  broadcastedToken.value.exists(coin => lowercaseTitle.contains(coin)) &&
  broadcastedLanguages.value.exists(keyword => lowercaseTitle.contains(keyword))
}

val uniqueLinks = filteredLinks.flatMap { case (title, url, body) =>
  val lowercaseTitle = title.toLowerCase
  val mentionedCoins = broadcastedToken.value
    .filter(coin => lowercaseTitle.contains(coin))
    .toSet

  mentionedCoins.map(coin => (coin, (url, body)))
}.groupByKey()
  .mapValues { iter =>
    val aggregatedCount = iter.flatMap { case (_, body) =>
      body.toLowerCase
        .replaceAll("[^a-z0-9\\s]", "")
        .split("\\s+")
        .groupBy(identity).mapValues(_.length).filter { case (word, counter) => word.length > 5 }
    }.groupBy(_._1).mapValues(_.map(_._2).sum)

    aggregatedCount.toSeq.sortBy(-_._2).take(30)
  }

uniqueLinks.foreachPartition { partition =>
  partition.foreach { case (coin, wordCounts) =>
    println(s"\nCoin: $coin, Found Link Count: ${wordCounts.size}")

    val topWords = wordCounts.toList.sortBy(-_._2).take(30)

    topWords.foreach { case (word, count) =>
      println(s"  - $word: $count")
    }
  }
}

spark.stop()

quartz kindle Jun 30, 2024, 10:48 PM

#

eternal osprey I do have a problem though guys. I have around 8tb of data. I am looking to redu...

we dont speak rocket science here

surreal sage Jun 30, 2024, 10:49 PM

#

you're infiltrating junior territory with that senior stuff

quartz kindle Jun 30, 2024, 10:49 PM

#

thats not senior, thats harvard phd

surreal sage Jun 30, 2024, 10:49 PM

#

go write your own lossless compression lib 😭

eternal osprey Jun 30, 2024, 10:49 PM

#

trust this is junior data science 😭

quartz kindle Jun 30, 2024, 10:49 PM

#

how do you even store 8tb

surreal sage Jun 30, 2024, 10:49 PM

#

raid

quartz kindle Jun 30, 2024, 10:49 PM

#

do you have a 20tb drive?

surreal sage Jun 30, 2024, 10:49 PM

#

hard drives

eternal osprey Jun 30, 2024, 10:49 PM

#

using hdfs

#

the hadoop distributed file system 🤓

surreal sage Jun 30, 2024, 10:50 PM

#

or that

quartz kindle Jun 30, 2024, 10:50 PM

#

lmao

eternal osprey Jun 30, 2024, 10:50 PM

#

but i continously get disk errors cuz my data went from 140k -> 7.6 million entries in 2 days.

#

EAACH QUERY TAKES 2 DAYS LMAO 😭

surreal sage Jun 30, 2024, 10:50 PM

#

man what are you storing

eternal osprey Jun 30, 2024, 10:51 PM

#

surreal sage man what are you storing

web crawls.

surreal sage Jun 30, 2024, 10:51 PM

#

bye 😭

#

couldve said web scrolls

#

📜

quartz kindle Jun 30, 2024, 10:51 PM

#

elder scrolls

eternal osprey Jun 30, 2024, 10:52 PM

#

it's over for me

quartz kindle Jun 30, 2024, 10:52 PM

#

what lang is even that?

eternal osprey Jun 30, 2024, 10:52 PM

#

ii think that i am somehow not removing or deleting the intermediate webcrawls after i processed them.

eternal osprey Jun 30, 2024, 10:52 PM

#

quartz kindle what lang is even that?

scala

quartz kindle Jun 30, 2024, 10:52 PM

#

never seen scala code before lmao

quartz kindle Jun 30, 2024, 10:53 PM

#

eternal osprey ii think that i am somehow not removing or deleting the intermediate webcrawls a...

what part of that code is in memory, and what part is disk based?

#

like how are you even operating here, ddo yu have 8tb of files and you process one file at a time?

surreal sage Jun 30, 2024, 10:54 PM

#

multi threadingn't

eternal osprey Jun 30, 2024, 10:54 PM

#

quartz kindle what part of that code is in memory, and what part is disk based?

Everything is disk based, but as soon as we process, we will load them into memory. Then send the data back to disk,

#

It basically processes it in batches ya know.

eternal osprey Jun 30, 2024, 10:55 PM

#

quartz kindle like how are you even operating here, ddo yu have 8tb of files and you process o...

We have A LOOOOOT of files

quartz kindle Jun 30, 2024, 10:55 PM

#

how much is loaded into memory at once?

eternal osprey Jun 30, 2024, 10:55 PM

#

these files are distributed over 128mb blocks across the cluster (=10 machines at the time).
Each machine starts a seperate mapper instance to execute the code on the individual blocks stored there, in paralllel.

#

So each instance has roughly 128mb in memory at the time, and the blocks are evenly stored across the cluster. So as of now, i utilize around 80% of the RAM of each machine but i sometimes tend to get heap errors as well..

#

What i think that the issue is, is that for each web scrawl file that i obtain, i crawl and process all the links that are on that site which creates loops if i don't keep track of the visited links..
What's the best way to quickly check memberships? HashMaps right?

quartz kindle Jun 30, 2024, 10:58 PM

#

if you are crawling an entire website, then yeah you need to record visited links

#

a hash map is fine

#

or even a set

eternal osprey Jun 30, 2024, 10:59 PM

#

i see. This explains a lot.

#

the queries used to take 6h but now they take like 2 days.

#

i am processing a lot of data so i don't think i can use HashMaps.. i will have to reserve a lot of memory for the bitvector.

quartz kindle Jun 30, 2024, 11:01 PM

#

i mean, you would only do it per website

#

once the website is done with, you can delete it

#

or do you have cross links between websites?

#

you can limit crawling within the domain only

eternal osprey Jun 30, 2024, 11:02 PM

#

it's a raw crawl from like the whole web, so i think they might contain crosslinking.

quartz kindle Jun 30, 2024, 11:02 PM

#

so you have 2 sets, one holds domains, the other holds paths per domain

eternal osprey Jun 30, 2024, 11:03 PM

#

wait let me send you an image of the format

quartz kindle Jun 30, 2024, 11:03 PM

#

for each domain, check domain set to see if it was already done with

#

then for each domain create a set and add paths to it

#

for each path, check the set to see if it was already done with

#

then once all paths were exhausted, delete the set and move on to the next domain

eternal osprey Jun 30, 2024, 11:04 PM

#

Schermafbeelding_2024-07-01_om_01.03.58.png

#

so each warc file, contains these info's.

#

and in that html it can have anchor tags ofc. So the web crawler will then crawl those links again, etc.

eternal osprey Jun 30, 2024, 11:05 PM

#

quartz kindle then for each domain create a set and add paths to it

hmm, what exactly do you mean with paths?

quartz kindle Jun 30, 2024, 11:05 PM

#

remove anchors/fragments from the url

quartz kindle Jun 30, 2024, 11:05 PM

#

eternal osprey hmm, what exactly do you mean with paths?

the url

#

domain+path

eternal osprey Jun 30, 2024, 11:05 PM

#

quartz kindle remove anchors/fragments from the url

owh dam i didn't even think about that.

quartz kindle Jun 30, 2024, 11:06 PM

#

although there are some websites gthat use anchors for SSR, which is stupid

rotund slate Jun 30, 2024, 11:06 PM

#

quartz kindle try to make a manual request with postman or reqbin or something

ohk so i try updating manually and it works

#

now

#

it is showing curreint in website

eternal osprey Jun 30, 2024, 11:06 PM

#

The processing in the begin will be the same though, as the webcrawler already crawls it for me and i can't really make it ignore the anchor tags, but the processing that comes after it will definitely be easier.

rotund slate Jun 30, 2024, 11:06 PM

#

but what is the issue through code/bot

eternal osprey Jun 30, 2024, 11:06 PM

#

Thank you tim!

quartz kindle Jun 30, 2024, 11:07 PM

#

rotund slate but what is the issue through code/bot

print the response from the post request inside the bot

surreal sage Jun 30, 2024, 11:07 PM

#

im recursively converting images in a dir to a webp
doing this on a single thread atm

do you think it's smarter to multi thread it, e.g. 4 workers

cpu usage with single thread:

#

or is that just dumb if it's alr begging for more cpu

quartz kindle Jun 30, 2024, 11:08 PM

#

surreal sage im recursively converting images in a dir to a webp doing this on a single threa...

it will definitely be faster, as long as you have spare cores

#

if you only have 1 core, dont bother

surreal sage Jun 30, 2024, 11:09 PM

#

i have 4 cores

quartz kindle Jun 30, 2024, 11:09 PM

#

eternal osprey Thank you tim!

yw (although i didnt do anything)

surreal sage Jun 30, 2024, 11:09 PM

#

surreal sage im recursively converting images in a dir to a webp doing this on a single threa...

i believe this is full server cpu usage

#

not one thread

#

server provider:

quartz kindle Jun 30, 2024, 11:10 PM

#

surreal sage server provider:

well are you sure its single threaded then?

#

how are you converting?

surreal sage Jun 30, 2024, 11:10 PM

#

sharp

#

for await loop with klaw

#

#

(do you see any possible optimizations here)

rotund slate Jun 30, 2024, 11:11 PM

#

quartz kindle print the response from the post request inside the bot

am getting 400 Bad Request through bot

#

idk why

quartz kindle Jun 30, 2024, 11:11 PM

#

surreal sage sharp

apparently sharp is multithreaded already

surreal sage Jun 30, 2024, 11:11 PM

#

yea its a binary and not a node script

#

i believe

quartz kindle Jun 30, 2024, 11:11 PM

#

yes

surreal sage Jun 30, 2024, 11:12 PM

#

surreal sage

gonna see if effort 6 is too much effort

quartz kindle Jun 30, 2024, 11:13 PM

#

rotund slate am getting 400 Bad Request through bot

what are you using for the request? maybe you need to json strinfigy the data first

rotund slate Jun 30, 2024, 11:14 PM

#

quartz kindle what are you using for the request? maybe you need to json strinfigy the data fi...

#

this is my code

#

aiohttp.ClientSession

#

am using that

quartz kindle Jun 30, 2024, 11:15 PM

#

rotund slate

use json, not data

   await session.post(url, json=data)

surreal sage Jun 30, 2024, 11:15 PM

#

effort 6 = 9.71 mb, 5.68s
effort 5 = 9.94 mb, 3.12s
effort 4 = 10 mb, 2.91s
effort 3 = 10 mb, 2.73s

#

melt_down

rotund slate Jun 30, 2024, 11:16 PM

#

quartz kindle use json, not data ```py await session.post(url, json=data) ```

oh ohk working

#

thanks for the help ❤️

quartz kindle Jun 30, 2024, 11:16 PM

#

surreal sage effort 6 = 9.71 mb, 5.68s effort 5 = 9.94 mb, 3.12s effort 4 = 10 mb, 2.91s effo...

what is effort? cpus used?

surreal sage Jun 30, 2024, 11:16 PM

#

effort to make file size smaller

quartz kindle Jun 30, 2024, 11:16 PM

#

ah

#

ye not worth it

surreal sage Jun 30, 2024, 11:17 PM

#

37.5 mb turned down to 10 mb with effort 3

#

so ya 3 it is

quartz kindle Jun 30, 2024, 11:17 PM

#

ye

#

webp be good

surreal sage Jun 30, 2024, 11:17 PM

#

love it

quartz kindle Jun 30, 2024, 11:18 PM

#

ever messed with jpeg2000?

surreal sage Jun 30, 2024, 11:18 PM

#

not sure if it's a good replacement for png even with quality = 100

surreal sage Jun 30, 2024, 11:18 PM

#

quartz kindle ever messed with jpeg2000?

👁️

quartz kindle Jun 30, 2024, 11:18 PM

#

i tried it once, was surprisingly good, like 30-40% smaller than jpeg

#

but browser support eeeeeeeeee

#

xD

surreal sage Jun 30, 2024, 11:18 PM

#

yea then no 😭

quartz kindle Jun 30, 2024, 11:19 PM

#

surreal sage not sure if it's a good replacement for png even with quality = 100

there is lossless webp no?

surreal sage Jun 30, 2024, 11:19 PM

#

i think quality: 100 is that

#

not sure though

#

i was using vercel and the first loaded image was 1.59 mb

#

my data transfer usage skyrocketed

#

i started using webp

#

like 70% reduction

quartz kindle Jun 30, 2024, 11:20 PM

#

nice

surreal sage Jun 30, 2024, 11:20 PM

#

not even using vercel anymore

#

only good for hobby projects

#

once you get thousands of visitors/day it's ouch

quartz kindle Jun 30, 2024, 11:20 PM

#

ye nothing really beats running your own vps

surreal sage Jun 30, 2024, 11:21 PM

#

using coolify with docker

#

but the image sizes EAT the ssd

#

it was at 50gb after a week, when using nixpacks

lament rock Jun 30, 2024, 11:21 PM

#

Come down this rabbit hole of pure optimization with us. Say no to bloated frameworks and say hello to pure HTML and JS

#

It's not that bad actually tbh

surreal sage Jun 30, 2024, 11:22 PM

#

template frameworks

lament rock Jun 30, 2024, 11:22 PM

#

https://github.com/AmandaDiscord/Amanda/tree/restart/packages/runtime-website-client/src

surreal sage Jun 30, 2024, 11:22 PM

#

full SSR is cool though

lament rock Jun 30, 2024, 11:22 PM

#

I wrote my dashboard is all pure HTML and JS. Well, TypeScript

surreal sage Jun 30, 2024, 11:22 PM

#

not always possible

quartz kindle Jun 30, 2024, 11:23 PM

#

why not

surreal sage Jun 30, 2024, 11:24 PM

#

ok tbh it's just difficult

#

maybe not impossible

#

are you a print width 80 or 120 coder think

quartz kindle Jun 30, 2024, 11:27 PM

#

i think its more of a habbit thing

#

if youre used to frameworks, its hard not to use them

lament rock Jun 30, 2024, 11:28 PM

#

There are some cases where stuff is just better off using something like snowflakes are annoying if you do them all manually and the Discord API requires you to decode the snowflakes to get timestamps

quartz kindle Jun 30, 2024, 11:29 PM

#

lament rock There are some cases where stuff is just better off using something like snowfla...

isnt that like 3 lines of code

#

lel

surreal sage Jun 30, 2024, 11:29 PM

#

2

#

export const DISCORD_EPOCH = 1420070400000

lament rock Jun 30, 2024, 11:29 PM

#

:)

frosty gale Jun 30, 2024, 11:30 PM

#

i use uuids for my IDs so you cant even get timestamps

lament rock Jun 30, 2024, 11:30 PM

#

I mean, some people might want other metadata but yeah

frosty gale Jun 30, 2024, 11:30 PM

#

(they are unnecessarily long and slow but they look cool)

surreal sage Jun 30, 2024, 11:30 PM

#

i remember ordering something off a company

quartz kindle Jun 30, 2024, 11:30 PM

#

frosty gale (they are unnecessarily long and slow but they look cool)

indeed, im using them for my api keys

surreal sage Jun 30, 2024, 11:30 PM

#

the invoice had a mongo objectid as order id

frosty gale Jun 30, 2024, 11:30 PM

#

quartz kindle indeed, im using them for my api keys

but i think im starting to get performance issues because of them so i might switch to numeric shorter IDs but keep uuids for other things

quartz kindle Jun 30, 2024, 11:31 PM

#

frosty gale but i think im starting to get performance issues because of them so i might swi...

how about discord auth tokens and webhook tokens

#

they are liek 100 chars long

#

xD

surreal sage Jun 30, 2024, 11:31 PM

#

auth tokens have multiple segments

#

one of which being plain json