Addon suggestions / issues

[cerulean shuttle]

Hi @spice otter, this looks like a pretty cool addon in terms of gameplay. I'm also a big fan of global leaderboards, however I believe your implementation is not safe.

Your leaderboard fetches the latest data as a formatted lua table info and then RunString's the returned content.

This potentially has the following issues:

• If your server got hacked, or (not that I think you would, but you have to consider the view point of people installing your addon) if you decided to misuse the addon, it would essentially allow for a remote backdoor into any server running it. In general, RunString should be avoided, but especially if you don't know where its coming from (again, from perspective of server owners).
• Since its using a HTTP connection, it is also susceptible to a man in the middle attack since it does not have any encryption. (In most cases would require the attacker to already have network control of the server, but still)

I recommend instead:

• Instead of returning code that creates a table, just return the table contents directly as JSON and use util.JSONToTable to convert it to a table when received on the client. This elimates any potential RCE concerns entirely.
• Use a HTTPs connection. Even if you buy a cheap xyz domain and use cloudflare SSL, it'd be better than no encryption at all.

Hopefully you'll fix these issues and then you'll have a great safer addon.

[blissful hawk]

👆

[cerulean shuttle]

you might not believe it but you can actually help other developers with constructive feedback, not everything has to be a passive agressive comment ❤️

[blissful hawk]

I'm not going to play the "it's your fault if you feel offended" because that's a game without winners but losers, we are a bunch of idiots and trust me you never should feel like anyone it's going to have fun of you, but we try to create a joky environment with all the members

Except Frenchies, fuck thay

[cerulean shuttle]

nah man the joke was i was being passive agressive in a comment about not being passive agressive

idc i just want the RCE fixed

the less backdoors on the workshop the better

[crimson vessel]

also could be used as an ip logger clientside, everyone is butthurt enough about datae

[blissful hawk]

datae just wanted to live the american dream

[spice otter]

Ip logger?
Man I'm not collecting any kind of player's personal datas

[crimson vessel]

did you see the word "could" ?

[spice otter]

Yeah, but I'll never be like that, so don't worry

also whos datae
Just wondering

[crimson vessel]

some guy who made a backdoored rhythm gamemode

[spice otter]

tf?

[crimson vessel]

google "datae beatrun"

[spice otter]

Ohh I've heard that

I only know that was a paid addon

Also to @cerulean shuttle
Server won't be hacked since it's hosted by myself, it's not an VPS from any vps provider.

Also why I use table.insert Is because I can append datas more easily and efficiently
If I had to use util.JSONToTable which means I'll make each player's score into a file (apparently you can't simply append texts on it), also you might need to fetch multiple files to fully read leaderboard, there's a reason why I use this weird way to achieve global leaderboard

Anyway I'll buy a .xyz domain soon

I appreciate for the suggestion you just made

[cerulean shuttle]

just because you self host doesn't mean you can't get hacked

and you can still use util.JSONToTable, you don't have to write anything to a file

[spice otter]

You'll had to fetch multiple files to fully read whole leaderboard

[cerulean shuttle]

there are a couple of issues with your addon but they can all be fixed

[spice otter]

Just tell me, I'll try my best to fix that

[cerulean shuttle]

currently your file is collecting all the leaderboards and putting it into one file. each line in your table is formatted as a table.insert

[spice otter]

Yeah exactly

[cerulean shuttle]

since you have all the leaderboard data there, just use ```php

echo json_encode([
$firstLeaderboard,
$secondLeaderboard,
$thirdLeaderboard
]);
die();

i think you're using php but if not you just have to return json

[spice otter]

Ohh that's cool

Well I don't know that since I just touched php for 2 days

[cerulean shuttle]

no exactly thats fair

its an easy fix

[spice otter]

Lemme startup my PC, gimme a second

Just woke up

[cerulean shuttle]

a second potential issue

it looks like you're sending the score to your leaderboard from the client

[spice otter]

Yeah

[cerulean shuttle]

it would be quite easy for someone to send fake data

[spice otter]

Well, it's actually hard to prevent that

For me atleast

[cerulean shuttle]

you'd have to network the score to the server, and then only have the server send those requests with an auth token too for added security

but unfortunately you couldnt really just send the score at the end of the round, since cheaters would just send fake scores

so the only way really is to move the score logic to the server honestly

[spice otter]

Won't it affected by latency/packet loss?

Remember I got a accuracy and combo system, they can be desyned, If I made everything serversided which means you'll have to sync these things in realtime

[cerulean shuttle]

net is fairly fast but you dont really wanna spam it

[spice otter]

That's what I'm about to say

using net.start every tick or frame is not good tho

[cerulean shuttle]

idk how fast StartCommand is on the server tbf

and itd be ping dependent

[spice otter]

Also for high ping servers score/datas can be delayed, you don't wanna see "Perfect" appear after ~0.1 to 0.2 seconds right?

[cerulean shuttle]

no I agree

Im just saying that having the score entirely calculated on client without any form of server validation will lead to cheaters fucking up the leaderboard

you could just contantly ban them, but youd have no way to distinguish them from real ones if they're not dumb

[spice otter]

I can make a ban system if it's needed

[cerulean shuttle]

unfortunately anything you make will be at one point looked at by some guy trying to find some vulnerability in it, and this is one sadly

what if they just pumped a bunch of average scores etc

theres no way to differentiate

the server has to validate the score at some point

[spice otter]

I know you really cared about security / cheater related thing, and I appreciate for that
If I had to valid scores I had to
Record player's input and re-calculate it, and it need to be done on serverside, although input can still be faked if someone really want to cheat, and it might lead to some inaccuracy calculation if you know how beatmaps works

[cerulean shuttle]

hold on im cooking

[glass widget]

[spice otter]

elo

[spice otter]

when did you get here lol

[glass widget]

I'm also just in this server too.

[cerulean shuttle]

okay one hypothetical:

client:

• client can basically keep doing what its doing for score checking even though people could modify its input
• as the client plays, store the SysTime for when each bar/note is spawned from the top. also store when each keystroke was.
• at the end of each round, send this list of timestamps to the server

server:

• calculatehow long each bar is going to take from top to the "score line" (idk what its called).
• "client presses the key, the current systime" - "when the bar started systime" = "how long before the client hit the note."
• then just make sure the expected scoring time matches how long it took them to press the key on a note.
• send requests from server

then, you'd have full server score validation so you can have a hypothetically safer leaderboard system. Although I 1000% understand that it is quite a bit, its just a random idea

christ i cant spell

moral of the story though, you need server validation for scores really. but more importantly you should change to using json for your API, and at bare min just send the client calculated score at the end of each round so the server can send the requests

[blissful hawk]

I've just played it, downloaded this morning, pretty amazing project, do you think there's a change to make a github repo? I would happily improve UI and even would add effects, i understand your approach of making it the most lightweight possible, but i grew up playing guitar hero and that sugar fx it's just sugar 🤑

And please, get rid of the RunString, it just doesn't feel right

[crimson vessel]

It would be very hard to ban users who fake the data anyway via this method imo, better packing it to the server with an auth token to validate, this way they must be ingame too

[blissful hawk]

yeah, i ever wanted to make some kind of stuff that would connect to my host, but honestly people it's just shit and cannot be trusted

[spice otter]

Yeah I want to get rid of that too, but I don't really have an idea for that yet, I don't really want to make new files for each score

If 10 players played same beatmap, then you'll have to fetch 10 files to fully read entire leaderboard

So I still need to write everything into one file

[crimson vessel]

Websocket leaderboards have already shown client abuse in s&box, you could pack and send this at the end of their run, I would personally only let them ask your api what the leaderboards are, and run any score updates through the server.

How are you storing these then if you must do multiple requests? Can't you just make a mysql table with column for beat maps, userid and score, one query to return them all? I imagine you're only storing their best time/score?

[cerulean shuttle]

you should bare minimum remove runstring as a main priority

[blissful hawk]

It's not that you would do anything stupid, it's just that's a huge could that i don't (and i bet many others) don't feel comfortable about

[cerulean shuttle]

if you need assistance with the PHP id be happy to help

this can all be done in one file

the way to think of it here

if you can do it in multiple files, you can do it in one

[crimson vessel]

It can, I also partake in php delights

[spice otter]

I just have an idea for that, I could still save them into multiple files, that's how I do on serverside leaderboard

Can php return stuff?

[cerulean shuttle]

do you mean you're using textfiles/jsonfiles for leaderboard storage?

[crimson vessel]

of course, but you probably mean echoing json

And yeah, it sound like he's flatfile storing

[blissful hawk]

open source it

[cerulean shuttle]

you could try to use a database. its not that terrible in PHP when you look through the docs. it may be okay if you're running SSD with good hardware honestly but for a public addon you should consider it

[crimson vessel]

Would save yourself the hassle with php pdo and a database like mariadb

[cerulean shuttle]

yeah

[spice otter]

I do have a MySQL databse running on my pc, but I don't really understand how to use it lol

[cerulean shuttle]

thats fair

all i can really recommend is tutorials honestly

[crimson vessel]

flatfile storage always becomes slower as time progresses, unlike mysql

[spice otter]

I'm already searching it, I'd prefer learning it by myself instead of directly asking for help

[blissful hawk]

I completely forgot it runs on his computer, that's cool but easily will fall apart

[crimson vessel]

nearly all of the time the queries you send will come from ram and not disk because mysql caching is good

without mentioning speed and scalability

[blissful hawk]

if it's about caching redis will kick ass

not sure if php plays well with redis tho, it does with mysql

[cerulean shuttle]

you should really consider buying a vpn. id happily offer you access to one of mine if you'd like since it doesnt seem like a very intensive application but still id really recommend you go to OVH and purchase one of the really cheap servers

[crimson vessel]

my memcached server is eating 2gb ram

[cerulean shuttle]

your PC isnt terrible but its not as stable as a VPS

[blissful hawk]

router turns off/ resets

[spice otter]

oh it does that sometimes lol

[blissful hawk]

time to push an update

[cerulean shuttle]

it just depends how stable you want your leaderboard ig

[blissful hawk]

bad people will exists :/

[crimson vessel]

I had to make a firewall updater for ovhs api as I kept constantly locking myself out when my router rebooted and ip changed 😂

[cerulean shuttle]

simply cache each leaderboad result in the clients data, so when the router resets it just uses last

or just get a VPS you can literally get one for £0.81 on OVH and that'd still probably be fine for your usecase (+cloudflare but you can go free too)

[spice otter]

0.81??

[blissful hawk]

Until a streamer finds it out and your vps just dies

[cerulean shuttle]

yeah true

[spice otter]

Wat\

I don't really get it

[blissful hawk]

0.81$ gives a 0.81$ vps, if it gets famous, that vps will just get ddosed

[crimson vessel]

he's talking about ddos attacks

[cerulean shuttle]

alot of requests = more server resources

cloudflare is your only hope

[crimson vessel]

They do a really good job at caching content too

[spice otter]

Alright, time to buy a domain

[blissful hawk]

Oh i didn't mean like actual ddos attacks, more like getting popular and it just shitting itself trying to return leaderboard, ofc this the lb it's not trivial, but if it gets too much traffic, then you will have a bad time

[cerulean shuttle]

one funny one you could do

[crimson vessel]

Ah, fair enough. I didn't consider that as I use these technologies by default these days

[spice otter]

Don't really think an rhythm game addon will get popular lol, popular stuff in gmod is always meme related things

[blissful hawk]

it's not getting popular now but it's a really cool addon, played few songs and i loved it

ofc it's not my music taste, but this could easily be used by other addons to add more songs

like packages, and people can publish in workshop any stupid song they want

[cerulean shuttle]

im like 80% sure you can still do, i did it at least.
oracle, the cloud provider, has a "always free" tier, and you get an AMD 24GB DDR4, 4 vcore with great network controls and protection

its literally a free server

[crimson vessel]

Where is this

[cerulean shuttle]

ive had mine for 8 months

[blissful hawk]

i would like to help you to do a pretty cool improvement to the UI and also help you with the extendable part since i really like rhythmic games like guitar hero

[cerulean shuttle]

https://www.oracle.com/uk/cloud/free/

thats uk but you could find yours

[blissful hawk]

thanks babe

[cerulean shuttle]

its literally insane

[crimson vessel]

I'm British What connection speeds?

[blissful hawk]

ahhh i don't know about free

[spice otter]

You can already do that by outputing beatmap as a lua beatmap

[cerulean shuttle]

lemme go test

[spice otter]

i'll check that out

[cerulean shuttle]

oh shitalso

[blissful hawk]

Ahh fine then

[cerulean shuttle]

they run out of stock quite a bit and its sometimes limited supply, so i used this https://github.com/hitrov/oci-arm-host-capacity

theres literally an automation tool to get you the most powerful free server

[crimson vessel]

🫠 I wonder if I'd break 18000gb hours in a month

[cerulean shuttle]

idk what to tell you i assure you its real i have one

tbf i dont even know what its network capabilities are i always just used it for processing

[crimson vessel]

probably limited for constant uptime, probably not applicable for vpns etc

[spice otter]

Jeez I don't know how to use Linux

It's gonna be a hard time

[blissful hawk]

Yeah, i would like to have a db provider i want to trust and without having to get a vps, but that kind of solution costs way more than a vps

[blissful hawk]

you will break it a lot, get used to it

[cerulean shuttle]

ive never had uptime issues

ubuntu@instance-20230112-2036:~$ uptime
 01:37:53 up 8 days,  3:39,  1 user,  load average: 0.09, 0.06, 0.02

that was me fixing something

idk what their trickery is here either

[crimson vessel]

My ovh box uptime has been since March 11 2021 when the old one burned in their fire

[cerulean shuttle]

oh yeah

[blissful hawk]

mine got shut down

[crimson vessel]

Unfortunately they locked them to 250mbps so it isn't applicable as I'd eat it all on gbps

[blissful hawk]

Dang, i really should up my droplet, this shit it's nut

[cerulean shuttle]

okay i couldnt think of the tool to use so heres speedtest cli

Retrieving speedtest.net configuration...
Testing from Oracle Cloud (xx.238.xx.93)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Orbital Net (London) [39.19 km]: 1.897 ms
Testing download speed................................................................................
Download: 2264.76 Mbit/s
Testing upload speed......................................................................................................
Upload: 1480.14 Mbit/s

[crimson vessel]

Sweet 2.5gbit nics

[cerulean shuttle]

yea man i have no idea why its free or how it works

[blissful hawk]

I could swear it was a lot more, also don't kink shame by using root

[crimson vessel]

who cares if sshd/22 is blocked

[blissful hawk]

rite?

[spice otter]

Guess I'll use my own server for now, And get rid of RunString

[crimson vessel]

Go check your auth log, bet its full of chinese root checkers heh

[spice otter]

My server is running in the company of my parents working on
So it's not very unstable I think

[blissful hawk]

wait how do i do that

[crimson vessel]

I was talking to gonzo sorry lol

[crimson vessel]

probably var/log/auth.log or similar

[cerulean shuttle]

yeah but are they working on it locally?

like on the same network

[glass widget]

[spice otter]

Yea

[cerulean shuttle]

thats LAN then, since you're hosting publically its now WAN so its subject to your internet connection

[blissful hawk]

[cerulean shuttle]

if your internet goes out, your parents can still work on that server since its in your network

but obviously it cant be accessed so it'd be offline for everyone else

[crimson vessel]

Go nano that file, or access.log, something with this name in that folder

[cerulean shuttle]

also yea that exactly

[spice otter]

what? it's running on their network

[blissful hawk]

Yes, i'm here, i will ASAP do something about it

This file it's endless...

[crimson vessel]

My memory is flaky at 3am, but I would just restrict 22 to your ip only

[spice otter]

They have a DVR server or something like that on there

[crimson vessel]

The main issue with root is because 50% is guessed for you

They can bruteforce depending on your ratelimits

[blissful hawk]

28mb

i'm like 10kb of that

[cerulean shuttle]

so. the server is in your house right, so its plugged in to your home network. that means that anyone inside your home network (your parents) can access the server directly, so even if the internet goes out they can keep using it since they have a direct connection.

however, if the internet does go out then we (people outside of your home network) wouldn't be able to connect. so it wouldn't work

[spice otter]

no

[cerulean shuttle]

christ

[spice otter]

^

It's in their server room

[cerulean shuttle]

ahh

maybe it'll be fine then tbf

for some reason i was under the impression it was running on your PC, i think confused from when you said you had MySQL on it

[crimson vessel]

I had the same thought

[spice otter]

nah It's my problem, My English isn't really good lol

[cerulean shuttle]

no problem

im gonna go off now, but i hope you fix those couple issues and look into the others

other than that though, good addon

[spice otter]

thanks

great
*Remove these name servers

[cerulean shuttle]

what registrar did you use

[spice otter]

nvm I found a settings for it lol

Changing nameservers need 24 hours?

[cerulean shuttle]

It can take up to 24 sometimes

But usually faster

Just depends how long it takes for it to propagate

[crimson vessel]

depends on the default ttl of the service you buy from imo

[spice otter]

Then it might be 1 hour

[cerulean shuttle]

I use https://www.whatsmydns.net

Pretty useful

Basically track the update across the world it’s kinda cool to see live

[spice otter]

Also I have a question @cerulean shuttle
Does this possible in php?

[crimson vessel]

I spam ipconfig /flushdns and chrome processes lol

[spice otter]

Basically write every score into a json

[cerulean shuttle]

Yeah

If you want I’d happily make you the php to connect to a database and retrieve the leaderboard scores it wouldn’t take long

Unless you wanna do it yourself which is also fair but just know the offer is there

[crimson vessel]

It pains me to mention looping and file_get_contents / string concatenation

[cerulean shuttle]

Yeah

It’s not ideal but it would work for a while

[crimson vessel]

Until critical mass

[cerulean shuttle]

Just depends on expected usage and lifespan

[spice otter]

It's not ideal, what does that mean

[cerulean shuttle]

A database would be far more performant

[crimson vessel]

It me as you will likely have issues with it down the line as time progresses

[cerulean shuttle]

It’s not too bad when it’s just a couple files but when you get to the point of reading 100 files for a request you’re going to slow down

[crimson vessel]

It will get way slower the more data it has this way, but not with a database (even without indexes)

[spice otter]

ah that make sense

guess I'll stick with database

[cerulean shuttle]

It’ll make up for the effort over time

[crimson vessel]

It's just a schema/structure and inserting data into those/selecting it, if you are OK with php tables and/or storing data in lua tables you shouldn't find it too hard

[cerulean shuttle]

Yeah there’s plenty of good tutorials out there if you go looking

[spice otter]

Well I have nearly 0 experience on MySQL, It's gonna take a while

[cerulean shuttle]

It’s really not as bad as you may think

[crimson vessel]

It's just syntax sugar, SELECT user, score FROM player_scores WHERE beatmap = ?

[spice otter]

cool, gonna try that later

Also I appreciate for the help you guys just gave me

[crimson vessel]

For that you'd want your table schema to have columns by name of id(autoinc), beatmap, user, score; you'll learn how to use primary and foreign keys to restrict duplicates etc and update their best scores only, it depends on your needs, but it is probably a bit early to say this, it will mean more to you later 🙂

[spice otter]

@crimson vessel Sorry for the ping, I didn't see any function related to "connecting" to my own sql databse on wiki

Only Query stuff

[crimson vessel]

https://www.php.net/manual/en/pdo.connections.php

[spice otter]

oh thanks

[crimson vessel]

If you need more help, it'll have to be tomorrow as its past my bedtime

[spice otter]

alright, good night

[spice otter]

To @cerulean shuttle
I'm currently already figured out MySQL table creation and php stuff (kind of), so RunString problem should be fixed really quickly

For scores, I'll let server to upload it (It'll upload player's steamid too, so steamid can't be spoofed, can used to ban cheaters), and server will compare user's input and notes (if input amount is less than total notes then there's definitely have some problems)

I'm going to university right now, I'll continue the work when I get back to home

[cerulean shuttle]

Apologies I included the URL in the message originally because I thought being in the addon source was public enough, wasn’t my intention to possibly lead skids towards it

Make sure you put cloudflare on your domain too, and install fail2ban or something on your server

[spice otter]

Yeah already

[spice otter]

It's fine

[cerulean shuttle]

Addon suggestions / issues

[spice otter]

@crimson vessel @cerulean shuttle I've finished leaderboard, now it's working without runstring

I'm appreciate for the help you guys just gave me

[glass widget]

That dude is back again to report abuse.

[red garnet]

and everything got tooken down again

[blissful hawk]

Guy got his PC shut down 💀

[red garnet]

?