limber cliff
I've been wanting to make an elo leaderboard for our Twitch chatters since the legendary elo stream, but have never gotten around to it because I'm busy (and lazy). For the past week or so, I've built a system that triggers automatically after each stream, pulls the chat, and does some arbitrary scoring to finally compute an elo score. This score updates every stream, and I intend to reset it regularly (i.e. seasonal resets).
You can tell that it is not very polished; if you look at the source code: https://github.com/vanorsigma/neuro-chat-elo/tree/main, you might genuinely call an exorcist. Despite that, I decided to do a public alpha release to gauge community interest and feedback, because I am concerned that it may incentivise negative behaviour (like spamming chat, making chat toxic, etc). If the project is deemed unneeded, I would rather not spend too much effort to continue its development.
Some features I wish to eventually implement are:
During the public alpha period, I may occasionally reset the leaderboard. I will try my best to backfill as much as possible, but things like elo calculation may change, which will affect rankings.
If you have any feedback, suggestion, or would just like to leave a comment, please feel free to do so.
Thank you 🙂
P.S. These leaderboards were computed on today's stream
open sleet
remote condor
that's pretty interesting
limber cliff
top overall chatters today
limber cliff
oh, you overtook val!
open sleet
do you have top bits/gifted subs yet?
limber cliff
should be able to add them fairly easily. will be there by next stream
open sleet
also, does the elo subtract emoji spam?
is it normal messages (text, text with emotes) or does it also calculate messages with only emotes
limber cliff
at the moment, i'm only factoring emoji spam into copypasta (i.e. only the chain leader of the emoji spam gets extra points). Otherwise, emojis have the lowest weightage out of all the other metrics (e.g. text, bits, etc).
overall score is calculated with text, emotes, bits and subs given
limber cliff
same 3 chatters
also added subs and bits-only leaderboards
limber cliff
we have this now
remote condor
good. it's more interesting when there are multiple categories
open sleet
we have this now
val's pfp kills me...
limber cliff
non-vip top chatters have a new rival!
the rest of the positions in all other leaderboards are classic, astor1es taking 4 categories
limber cliff
limber cliff
Today's leaderboards are still cooking (they're being processed atm), but:
I probably need to do a massive elo update eventually, we have people going from 3 digit elo to 45k elo
void schooner
lazy loading
cool how it can also apply to UIs
limber cliff
yh when i saw this happening to the UI I said enough was enough
void schooner
is that time to load in milliseconds?
limber cliff
that is the number of entries loaded into the leaderboards
each entry takes around half a second to load (with the badges)
void schooner
damn i thought it would be about max ccv (~10k). not ten times that
limber cliff
it would have been ~30k if i chose to save the leaderboards in a clever-er way
limber cliff
Only Chat Messages leaderboard, holy
calm cloak
Timeout leaderboard when
limber cliff
Can't think of a way to do it without having an IRC client online, but I'll add it to the list 👍
open sleet
i am currently working on that
problem is; normal timeouts mixed with neuro's timeouts
well im not working on it, its finished
works perfectly and well
limber cliff
neuro's timeouts probably all come from vedal987, so that'll likely be a good enough indicator
open sleet
neuro's timeouts probably all come from vedal987, so that'll likely be a good en...
all of them come from vedal987
non-mods don't get information
IRC Client throws mutes only from vedal987
limber cliff
, i'll have a poke nearer the weekend to check for any additional metadata
open sleet
and another problem; bans count as mutes
mutes & bans are shared thru the same tag
BAN-DURATION
or info idk
void schooner
only count 120s timeouts?
sure some regular timeouts will get included, but it'll probably be good enough
limber cliff
(will be resetting elo today)
limber cliff
Currently trialing a new elo system that is (hopefully) fairer than before
Details:
(picture: new elo system backfilled to past 6 streams, the podium is not well-tuned for so many rank 1s)
May update this slightly tomorrow to use percentiles than 100 randomly sampled chatters & decrease the importance of chat message lengths (this is done)
rigged
limber cliff
hi just a small update to let everyone know that I'm still working on it 🙂 (also to keep thread alive)
We have someone currently rewriting the calculation in rust, which means you get leaderboards faster after a stream 🙂
I have an idea for timeout leaderboards that I am currently working on, but it may take some time. It involves wakewords and such, so it'll be a while before I can get some results 👍
limber cliff
oh god i hope i can finish the trial program before the stream
limber cliff
erm
might not be able to get it out in time :^), getting skill issued by rust async
ping me after today if there is another timeout stream, surely i'll get it done after today
void schooner
no
limber cliff
in theory the timeout calculation program is ready for field testing. plz ping me whenever she starts timing out people 
limber cliff
The Rust PR is here! https://github.com/vanorsigma/neuro-chat-elo/pull/1
courtesy of @stoic wadi 🙂
limber cliff
👀
void schooner
discord and twitch chat monitoring
open sleet
amazing
limber cliff
just thought this was an interesting note, the elo landscape has changed
i don't even know what the meta is anymore
limber cliff
not really sure what I expected
it'll probably get more interested from here on out
(also glatiator has been pretty consistent as a #livestream-chat chatter 👀 ). got an idea to maybe rank consistent chatters asw
digital mango
I've been wanting to make an elo leaderboard for our Twitch chatters since the l...
just stumbled on this, pretty cool project! 
past bone
i fell off
limber cliff
you're topping this leaderboard though 
partners
and this is the bilibili leaderboard. as you can see this is held together by hopes and dreams (documentation for this is sparse and extremely difficult to work with + bilibili doesn't allow me to serve avatars by hotlinking 
)
void schooner
and this is the bilibili leaderboard. as you can see this is held together by ho...
🧎 your determination knows no bounds
limber cliff
i got ratelimited after 1k requests, but there we go (same guy is #1 for bilibili again)
yo fossabot popping off
(glatiator is top1 on discord again. also seems like i introduced a bug where the wrong profile photo will load if you search, but it should be fixed by next stream)
limber cliff
this definitely didn't take a lot of time at all, nope (functionally still wonky, the scrolling esp)
limber cliff
heir is #1 on livestream-chat (as an additional note you can now scroll this part properly)
next granite
it didn't break yet 
limber cliff
🔬
stoic wadi
👀
limber cliff
Reveal got updated! (lots of familiar faces on overall today) (also i should have fully fixed bilibili images now)
We're slowly creeping towards Beta, with a feature (opt-outs) landing on main pretty soon. This means we get to choose a domain (ideally something that can end up on CloudFlare)
I'm not particularly creative when it comes to naming the domain, but here are some suggestions:
Lend me your brain power for some creative names
limber cliff
I had to run it again to check I'm not going crazy. I think we need a min elo floor
limber cliff
This is 1000% because of the starting elo being 1200, and negative elo doing some shenanigans
cc devs pls fix
limber cliff
deployed a tiny fix, discord chat should be relatively accurate now (heir you got your crown back). i've identified the root cause as well, but it'll take slightly longer to fix
limber cliff
uh oh it's getting worse (also i didn't expect <3 to be a username 
)
will be trailing a new revised elo system on Friday which will hopefully fix this ™️
waxen nacelle
im still somehow winning
void schooner
the least bad chatter
limber cliff
when debugging some elo related thing, i really went "ah ravaxo is top 2, surely fine"
neuro has some well-known chatters
limber cliff
goes hand-in-hand with "rachi is top in subs and heir is top in #livestream-chat, can't go wrong there"
waxen nacelle
pinning the top post for ya
limber cliff
aite looks like the revised elo needs more time to cook (surely by tomorrow ), so today's elo will still use the same old scuffed algo
waxen nacelle
my ELO is still in the negatives for no reason 
i dont need to be on top but i can see the problem when i get droped by 31 behind people I didnt even see in the livestream chat
limber cliff
yep it's not representative and defo something i need to fix asap. It's good that we're catching this now while this project is still in alpha
waxen nacelle
ill help you however i can
limber cliff
Leaderboards with the in-progress algorithm is up! I've done an elo reset, so these elo are calculated over the last 5 streams + today's amazing ✈️ stream. (Bilibili will only get this update starting next stream)
Hopefully the ranks look a lot more sane now
waxen nacelle
caught me slacking
limber cliff
Thanks to Kotgedev and their team for letting us collaborate with them - we now have a Neuro Adventures leaderboard! 🎉
(check out #1184879711399190649 )
limber cliff
why is eliv ranked #6 on an eliv stream smh my head
waxen nacelle
you fought well 👏
im always surprised i can push ahead
limber cliff
today's news: jamgie won 1st for emotes (idk how), twitch top chatters are absolutely dominating in their rankings, the same b2 chatter is really dedicated, and the top 2 discord chatters remain undisputed.
copypasta leaders have two new top contesters
(also, we're going to be moving domains soon 👀 )
limber cliff
Hi! Some slight updates:
(wow)
also Heir, you won't believe this but you changing your profile photo made me discover a bug
try visiting the site on mobile, the experience is much nicer! (definitely intentional and not at all an accidental boon)
waxen nacelle
also Heir, you won't believe this but you changing your profile photo made me di...
wait how
limber cliff
wait how
old profile photo urls from discord are invalidated, and our leaderboard generation code didn't pick up the new profile pics until the fix
so for a while, we had a transparent image representing #1 for livestream-chat
waxen nacelle
glad to help
limber cliff
After almost nuking my repository, we now have the ironmouse leaderboard!
https://neuroranks.chat/?index=11&search=
At the moment, the implementation is basic and fairly buggy, which means you may not see your name on the leaderboard yet. There are some conditions to fulfil:
)Eventually, I'll link it to Discord tags as well, so do set your public discord username.
Anyone who appears on this derivative leaderboard will get +100 base elo on all resets in the future! Also, you get this border around your name through all the other leaderboards until I get around to remove it
limber cliff
(correct as of 4pm-ish today)
(if you aren't on here, make sure to use the template + chat in tomorrow's stream)
waxen nacelle
After almost nuking my repository, we now have the ironmouse leaderboard!
https...
Where’s Atorin on this
Internets real slow so I can’t load it
Oh this isn’t fir discord yet
Ah
Or is it 
my brains not working
limber cliff
the scores seen there is the number of pixels placed per user. that'll also be true for the casual pxls canvas whenever that rolls around (it's already there, but hidden from the menus)
The difference is that the ironmouse pxls is linked to twitch users, while the casual pxls is linked to discord users
But good idea. Once the #ironmouse-canvas gets archived, I'll run one calculation for that whole channel
limber cliff
👀 (small update while i'm away atm)
If you have a big screen, the podium & leaderboard rank items should scale correctly now
past bone
looks aight
waxen nacelle
if i could request something
I would love to see a graph mapping ELO dips and rises per day
limber cliff
sounds like a good idea. Maybe a per-user stats page or something.
right now that's not possible because we don't store elo history, and we're working on making elo update live. i'll add it to the roadmap after we roll around to beta 👍
waxen nacelle
oh wow my ELO actuaslly DROPPED today
dunno how that happened
waxen nacelle
okay so
i need to pay better attention to my numbers but my ELO i dont think actually moved at all this stream??
other peoples ELO's are moving
is it because i changed my icon again
i dont super care, im just confused 
limber cliff
the latest automatic generation got stuck, i'll rerun the numbers. last update was still this
limber cliff
bruh
waxen nacelle
<:NeuroClueless:1072901352902103100> bruh
bruh?
wait how did i go even lower
there literally wasnt even chatting
limber cliff
this generation was for yesterday's stream, and yeah, seems incorrect to me. trying to figure out why
waxen nacelle
its literally just my number just dropped
potat and glatiator stayed the same
what
limber cliff
perfect code with no bugs whatsoever
limber cliff
I think I found the cause, will regenerate soon
digital mango
the bug in question:
chilly dragon
if let Some(user) = DiscordLivestreamChat.state.get_mut("heiroflife") {
user.elo -= 100.0;
}
If it helps with isolating bugs, my ELO dropped between the message I’m replying to and now from 311 to 304
It decayed while literally noone in top 3 did I don’t think
limber cliff
yeah it's really weird. potat makes sense, since they really did send tons of messages last stream. If we remove consistency, you should have been 2nd
the elo algorithm atm takes into account of your previous elo to calculate the next elo to reward consistent chatters, but clearly that is causing some issues here
waxen nacelle
Meanwhile I KNOW glatiator had 318 when I last looked at it
And he still does
It’s so weird…
limber cliff
i'll continue looking for the bug. it's nice to have actual users since it does narrow down what i need to look for
thank you 
waxen nacelle
No prob!
I think it’s worth screenshotting top 10 and checking them after every stream to make sure no nonsense is happening
limber cliff
Ah, the problem is related to the length of the messages. Emojis & Gif links all count towards the score, and they are notoriously long
The thingy that's measuring text length is reused for Twitch & Discord. In twitch's case it makes sense because rewarding longer messages => more high quality messages
but in discord's case, where pasting GIF links is common, this stops making sense
I'll update this to simply reward equally for every message on discord
waxen nacelle
its no wonder this started happening after people started spamming
and such
i would suggest it ignore length explictly on messages starting with "https://"
since people post them from their saved gifs
waxen nacelle
Looks like its working for me again!
waxen nacelle
i should have kept track of the scores fro top 10 to see the changes
i forgot to
this stream i notibly jumped from I THINK 313 to 324 though, which put me in first from 5th.
limber cliff
Hi all! There are two new things to look at for live-elo:
limber cliff
tiny update: happy halloween!
waxen nacelle
waxen nacelle
@limber cliff it hasnt updated in a couple days
just letting you know!
limber cliff
oop, my bad. have been working on the live version recently, so i neglected the old one. seems like it's currently stuck, i'll give it a small push
limber cliff
ah, 7tv's still down. i'll regenerate tomorrow morning
safe mica
not sure if you're aware but the casual pixels leaderboard is still showing the standings from the previous canvas (the canvas gets reset each month, approximately, and so does the canvas pixel count for every user)
limber cliff
thanks for telling me! i thought i designed it to match the pxls leaderboard, but I think i accidentally clamped the values to the maximum each user has scored
i'll fix this by today's generation
limber cliff
signs of things clearly working 
limber cliff
Alright, it's been fixed!
Some quick notes:
Feel free to give any feedback / suggestions / observations 👍
limber cliff
👀 @waxen nacelle
You've seen them around, and probably know that NeuroBot now consumes cookies 🍪 ; I present you: the cookies leaderboard! If you don't see your name here, head over to #swarm and send a cookie; your score will be there by tomorrows generation.
(Technical details: This is the first in-production leaderboard processed by the new live-elo program, but published via the old elo system. We'll be slowly iteratively replacing leaderboard generation until they're mostly generated by the new live-elo program, before completely swapping over 
)
waxen nacelle
NICE
rustic gladeBOT
🍪 Om nom nom
You've given me 16 cookies! | I've received 11616 cookies total!
🍪 Om nom nom
You've given me 32 cookies! | I've received 11618 cookies total!
limber cliff
(you were there at some point and just vanished, i thought it's cuz you disassociated your twitch tag . I'll have a look soon)
quartz tapir
the site is offline so idk how i'd do that
ripe bear
So it have no logical value except for fun, which I endorse dw, Funny project, please continue~ 
limber cliff
https://neuroranks.chat/?index=11&search= @quartz tapir you're back here again
limber cliff
Community Chat Elo (Beta)
Hey all!
After some (a lot) of work, we're finally ready to bring https://neuroranks.chat to Beta 🎉
The leaderboard now updates live to all chat messages; you can see it go up and down with very pretty animations 
(look at the bottom left to see the connection status)
Some leaderboards and features still have not been fully implemented yet; we're hard at work to get those online, but we figured the live part is good enough for public consumption.
You can keep it open on a tab somewhere as you watch the subathon. That being said, happy subathon
Thanks to @stoic wadi and @obtuse fern for the work you've put in!
mossy obsidian
erm... is negative elo a thing? I don't think we are that bad of a community.
limber cliff
the scoring algorithm used is completely borked because it couldn't properly handle elo changes live 
i'll be deploying a fix shortly. subs & bits will have the right leaders after this, but until then, please stay patient
mossy obsidian
just letting you know in case you didn't see it yourself
limber cliff
tysm for the report
limber cliff
should be fixed now
limber cliff
it's been run fairly discretely
(am aware discord-cookies are broken 
)
limber cliff
fixed cookies, switched subs / bits over to use fixed scoring and adjusted the elo algorithm. as a result i've reset some leaderboards, but it should be stable enough to be left alone for the entire subathon
magic nacelle
Search function seems to be missing 
limber cliff
it is! we had to rewrite lots of stuff, and forgor search was a thing
it'll be there one day ™️
magic nacelle
Soon™️
vernal sluice
is the websocket open to all sites 
limber cliff
permission-wise, 
(my poor server will explode). technical-wise, have already implemented CORS, so will be limiting that shortly 
vernal sluice
permission-wise, <a:nopers:1301608619346890762> (my poor server will explode). t...
mind if i test the websocket to see if it still does?
limber cliff
sure, although there's not much I can do if someone spoofs the headers 😅
i'll reconfig some point after today's stream
waxen nacelle
limber cliff
might be a cache issue! try refreshing
waxen nacelle
top left button no work either
limber cliff
, that's odd, it works on my mobile devices. are you using safari? if it's a browser issue i have to investigate it further
young coral
