development

[hollow basalt]

On the chance that the IDE is actually wrong, you can feedback them and they'd be happy to help

[warm sleet]

i doubt it

this is user-error

[hollow basalt]

happened to me in phpstorm

they were nice to help

[warm sleet]

I've reported quite a few errors with IDEA before

especially the smart linting they do with java code

[hollow basalt]

you have the idea then

[warm sleet]

most of them got fixed

But I doubt this is a bug in pycharm

[hollow basalt]

doubt it

[hidden lagoon]

How hard would it be to make a piece of code
that would act as a shortcut to run a game ?
I want to launch overwatch with one click, so I dont have to click play again in the battle net launcher

I dont know if this is the right place to send my question

[hollow basalt]

doesn't battlenet have that shortcut already

I bet that feature is already present @hidden lagoon

or just yoink a shortcut for the game

i bet it opens battlenet anyways

[hidden lagoon]

it opens the launcher

I still have to press play

even the shortcut feature within battle net

still opens overwatch

I wanna add it to an elgato stream deck

so it opens straight away

I found a couple threads on reddit, but none of them work anymore around 2/4 years old

https://www.reddit.com/r/Blizzard/comments/7aywrt/all_current_shortcut_codes_for_launching_games/

[hidden lagoon]

@hollow basalt apparently the battle net has an launching option of --exec="launch GAMENAME"

I am guessing that might help with creating a bat file

[hollow basalt]

or a regular lnk should be fine

(shortcut)

[hidden lagoon]

just a regular windows shortcut ?

[hollow basalt]

you could add parameters, yea

[hidden lagoon]

I am guessing it will need to open the battle net app first

then add the launch option after

[midnight wind]

no it will just launch the program with cli options

in the windows shortcut

[hollow basalt]

probably bootstrap battlenet then the game

I really don't know battlenet

but I'm basing it off steam

[hidden lagoon]

done

"C:\Program Files (x86)\Battle.net\Battle.net Launcher.exe" --exec="launch Pro"

That was what I put inside the shortcut

Pro is the code for overwatch

[warm sleet]

Those are just shell commands

[dense scroll]

I was wondering that is it possible to change <a style='orange' herf="https://example.com"> so that the orange is replaced with a hex value? I tried just putting one there but it didn't change the color.

[midnight wind]

what's in the style string is css

just use css

so like style='color: #ffffff;

@dense scroll

[dense scroll]

I don't have CSS

[midnight wind]

@dense scroll you put css in the style tag

Look at my example

[dense scroll]

It works now

[quaint palm]

anyone here

[frozen pulsar]

Sir yes sir @quaint palm

[umbral saffron]

Ok so if I have a 40 pin screen can I plug it into an extension cable and connect that to an rpi 0 wh

How would I if I can

[bleak breach]

Depends on what the pinout is and if the Pi has the right drivers for it

[umbral saffron]

https://www.adafruit.com/product/1680

This is the screen

How do I connect it to an rpi 0 wh

But I need an extension thing

@bleak breach

[bleak breach]

@umbral saffron yeah if you can find an adapter and driver that’d be easiest. Otherwise you’ll have to read the Pinout docs and adapt another driver yourself

[umbral saffron]

So

This

https://www.adafruit.com/product/1590

To this

https://www.adafruit.com/product/862

To the pi?

@bleak breach

[umbral saffron]

Would there be a way to get a cable to extend the screen from the gpio board

I need to Make it so the screen is in one place, and the gpio board

So can u find me a cable to extend the screen cable to connect to the gpio board

[umbral saffron]

What I need

What would solve all my problems

Is a cable that I insert my screen cable into

And the other end is a 20 pin female cable

[tall kernel]

i just made a new website

https://aalbert.tech/

is it cool

[ivory bear]

This page isn’t working
aalbert.tech redirected you too many times.
ERR_TOO_MANY_REDIRECTS

[tall kernel]

i swear it worked 15 minutes ago

[tall kernel]

its working now!!!!!!!!!!

i guess-

YES

ITS WORKING

FINALLY

[ivory bear]

ok now it works

[tall kernel]

yeeeeeeeeeeeet

i want to make a dark mode button too but i dont find svg's for that

[ivory bear]

it's basically, a text with color
also, you can just change the svg color from the color value that you inserted?

[zealous tangle]

it's really funny sometimes. I have friends in tech discords asking why im going nvidia over amd when it comes to gpus, and it's a really obvious, I need CUDA for ml work

[late plank]

Hi,
So im trying to execute a Python script from my php code in laravel. Ive managed to call the script eith the right parameters through the shell_exec(), but i quickly got problems. My imports didnt work. Turns out that laravel is somehow running python 3.8.2 whilest my packages are installed for 3.9

Any clue on how to fix this? Should i try to get python 3.9 running for laravel or install python 3.8.2?

[late plank]

Im on macos but ive got 3.8.6 and 3.9 running

Do i need to install 3.8.2 aside it?

If i run
Print(sys.version) through my laravel application via shell_exec, it outputs 3.8.2 but how can it run that if i dont even have that version installed?

[sacred pumice]

@late plank likely means laravel is running a different python executable, you could run which python to find out the full path to the python executable you're running, and then tell laravel to execute that one specifically.

[late plank]

i had to set a direct path, so not the same command as the terminal has (python3 .....), but /Library/Frameworks/Python.framework/Versions/3.9/bin/python3 ....

[sacred pumice]

👍

[silk mesa]

hello guys I need help wit ha web dev thingy

when I open my mobile menu I still can scroll and I need to disable the scroll without using JQuery if possible

[fervent grail]

Try hiding the overflow?

[umbral saffron]

Can someone find me a cable to extend https://www.adafruit.com/product/1680 that screen to https://www.adafruit.com/product/1590 this gpio board

[umbral saffron]

can smeone help me
im making my own ai assistant
and i need to code the gui
its tkinter on pycharm

[hollow basalt]

[umbral saffron]

the end goal is a search engine that searches wolframalpha, sends the results back to u in a pretty gui, and has voice to text and text to voice

[hollow basalt]

That's like

Wolframalpha

[umbral saffron]

If is but it has it’s own customized gui on a little screen and has voice to text

Oh nice Thankyou

[sacred pumice]

I will probably end up using mycroft in my home soon, tried it out not too long ago on my PC and it worked great. Has home assistant integration as standard which is pretty much all I want, and the Amazon echo is starting to get really spammy.

[dense marten]

Hello, I am pretty new to JS/TS and am trying to understand how promises work. I am using axios to try and get the data from a page and this is my code so far

async function visit(url: string) {
    return axios.get(url)
        .then((res: any) => {
          return res.data;
        }).catch ((err: string) => {
          console.error(err)
        });
}

const data = visit('https://some-url-here');
console.log(data);

however when I run it I just get Promise { <pending> }
I tried doing

visit('https://some-url-here').then((res) => {
    console.log(res);
});

and this prints out the HTML of the page but I can't figure out how to get it stored in a variable, does anyone have any ideas? thanks

[midnight wind]

yeah your second thing works, because it's waiting for promise resolution with .then

res is just a variable

async function visit(url: string) {
    return axios.get(url)
        .then((res: any) => {
          return res.data;
        }).catch ((err: string) => {
          console.error(err)
        });
}
var data;
visit('https://some-url-here').then((res) => {
    console.log(res);
    data = res;
});```

bad example

but it'll work

what you could do is call another function you want to call with the data you got like this

don't know ts, writing as js:ts async function visit(url: string) { return axios.get(url) .then((res: any) => { return res.data; }).catch ((err: string) => { console.error(err) }); } function doDataStuff(data){ console.log(data); } visit('https://some-url-here').then((res) => { doDataStuff(res); });

arrow functions are useful because how they handle scoping

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Arrow_functions

[umbral saffron]

While ur here present monkey I’m reinstalling it but if that doesn’t work I downloaded python 3 on my mac and it works when I open it but all the projects are on 2.7

[midnight wind]

idk really know python and all the version stuff

what project?

your's or some project on let's say github

[umbral saffron]

Mine

When I open it the version is 2.7 and I can’t import a lot of things

Like half the library went poof

And it won’t work

[midnight wind]

yeah idk

[dense marten]

how are you opening it? if from command line are you using python3 instead of python?

[umbral saffron]

Wdym

[dense marten]

you say when you open your project it is using 2.7, what program are you using to open your project?

[umbral saffron]

Pycharm

And I tried opening it directly

And I used the IntelliJ toolbox

[midnight wind]

probobly some python eqivalent of a package.json

that specifies version and stuff

[dense marten]

So I tried what you put here and it seems like it still isn't working quite right. I did

var document = "Didn't work";

function doDataStuff(data: string){
  // console.log(data);
  document = data;
}
visit('https://some-url').then((res) => {
  doDataStuff(res);
});

console.log(document);

and when I do this it prints out the initial "Didn't work" not res as I would expect. If use console.log instead of trying to assign it then it does print properly, I'm confused why console.log seems to work but never an assignment

[midnight wind]

maybe a ts thing?

idk, I never really did ts

what does the compiled file look like?

[dense marten]

Go to file->settings and then search "interpreter" and you should be able to select a different version

[dense marten]

The axios portion is quite long, but I don't think that part is the issue since that works with console.log but the end of the code is just

var document = "Didn't work";
function doDataStuff(data) {
    // console.log(data);
    document = data;
}
visit('https://www.viz.com/shonenjump/one-piece-chapter-1/chapter/5090?action=read').then(function (res) {
    doDataStuff(res);
});
console.log(document);

the typescript is only different in that I have the variables specified as string

var document: string = "Didn't work";

function doDataStuff(data: string){

[umbral saffron]

Oh tysm that helps a lot

[lament cave]

I have this file which has contents like this: https://i.cocobot.tk/AlViNMb7.png
there are some words which are readable but most of it is random characters, is there any way I can view the actual content and not random characters? (tried switiching to different encodings but same result)

ping me/reply if you have an answer

[fallen cedar]

it logs the original "Didn't work" because the callback provided to the then is not run yet

[spring pond]

That looks like an executable file and as such you'd have to disassemble it to see the code

[fallen cedar]

when you use promises, setTimeout, setInterval, XHR requests, event handlers, etc (basically anything async), a callback is added to a queue, which will be invoked by the runtime when the call stack is empty ("nothing" is running basically)

your console.log should be in the callback, not outside of it

visit('https://www.viz.com/shonenjump/one-piece-chapter-1/chapter/5090?action=read').then(function (res) {
    doDataStuff(res);
    console.log(document);
});

using then doesn't "block" the code, so document is logged with its old value
when the callback passed to the then is "allowed" to run, it will be invoked (which happens when all your other code finished running -> call stack is empty)
i suggest watching

https://www.youtube.com/watch?v=8aGhZQkoFbQ

[silver furnace]

So every windows 10 installation (except for RT or whatever I guess) has CertUtil.exe

I just made a bat that echoes out base64 to a temp file, cert utils it and outputs to a second text file

cert util the second one and payload

It was just base64 encoded twice

Went from 24 detections on virus total to 1 by 'CAT-QuickHeal' whatever that is

Called it "JS.Nemucod.BGF"

Expected more from heuristics... >.>

[midnight wind]

@dense marten look at what they said #development message

[dense marten]

Thanks for the ping

[tall kernel]

anyone online

devs

please

someone who knows nginx and shit

i have a subdomain and an 8080 port

and i want ssl

how can i hide the port

so its just sub.domain.tld

not sub.domain.tld:8080

i cant point it to 80 because thats my webserver and 8080 is a nodejs

If you don't want a port number in your URL, you have to use the default port number for HTTP, which is 80. If it's anything other than 80, you will be required to put the port number in the URL. That's all there is to it.

ok then

how can i point it to 80(nginx)

port is already in use

[candid forge]

default ssl port is 443

[tall kernel]

i fixed it

chill

proxy_pass http://localhost:8080; fixed it

now its https://bin.aalbert.tech

instead of using aalbert.tech:8080

[lament cave]

I'm trying to reverse engineer a .exe file back into a .py file

• it uses pyinstaller
• py 3.7
• I got the exe split up into a ton of module files and the main .pyc file

Problem:
Now I can't get the .pyc file into a normal .py file for some reason
I tried normally running the main .pyc file but it errors out (same error as the next line)
I also tried running one of the module .pyc files (the base64 module) and it gives me this error: https://i.cocobot.tk/2bQszuE8.png

if anyone has any suggestions/solutions please ping/reply, thanks

[fallen cedar]

yep, i'm not a python guy, but at work other people suggested uncompyle for this as well
this can give you relatively good results, BUT don't expect a perfect 1 to 1 correspondence between the decompiled code and the original source

[bleak breach]

I thought that installing the exe would decompile the .py files on installation

[lament cave]

@fallen cedar @rancid nimbus thanks for the answers, unfortunately I already tried uncompyle6 on both the main .pyc and the module .pyc and it gives me TypeError: File base64.pyc doesn't smell like Python bytecode

[lament cave]

yeah there was pyREtic which would try to parse the .pyc files from memory when the application is running, but it doesn't work anymore (10 years old and unmaintained so thats probably why)

[fallen cedar]

can we get access to the .exe or you can't share it?

[lament cave]

yeah sure one sec

[bleak breach]

I think you also have to use the same version of python to decompile as the exe if you can find it, otherwise the bytecode won't match

[lament cave]

https://drive.google.com/drive/folders/1WltinuiLT4LR1sxjzecF9EX3KlOx5NeM?usp=sharing

it is in python 3.7 (at least I think that's what python37.dll is)

[fallen cedar]

yeah, i can imagine there are differences between ,pyc created with python2 and python3

this is a guess though 🙂 again: not python guy

[bleak breach]

Yeap

Can you look at the file itself and see if it references what minor version?

You likely need to have the exact same version

[lament cave]

I don't see any way that I know of which allows me to see the exact version unfortunately

[silver furnace]

The pyc could be in resource header

http://www.angusj.com/resourcehacker/

I don't think .pyc is always the same. Just as .luac

It's a bytecode but for what?

There's a Java implementation called Jython which compiles to JVM byte code

CPython is standard

[fallen cedar]

so i tried, i think it worked for me
installed python 3.7.9 on windows
extracted the .exe you sent with https://github.com/extremecoders-re/pyinstxtractor
then pip install uncompyle6
then uncompyle6 base64.pyc in the app.exe_extracted\PYZ-00.pyz_extracted directory and it spews out the source code

[silver furnace]

I don't think same .pyc will run across two different minor versions

[fallen cedar]

decompiled: https://pastebin.pl/view/81db9e34

[silver furnace]

I mean in theory it should always be "decompilable"

and the pyc could be made to run against new version of cpython

Oh lol you got lucky

Embedded file name: base64.py

Idk python scripts should be shipped as .py.
.pyc is what gets compiled once to save time

[fallen cedar]

also ran the decompiled code against the random file containing the string hello using python decompiled.py testfile.txt which encoded my file to base64 which i then decoded on https://www.base64decode.org/

and i got back hello, so the decompiled code works

[silver furnace]

Oh embedded file name as in just the file name?

Idk but pyc only intended for one version. If you have the original py, it's supposed to recompile it regardless. Problem is that I've even had scripts break because of wrong minor python version

def encode(input, output):
    """Encode a file; input and output are binary files."""
    while True:
        s = input.read(MAXBINSIZE)
        if not s:
            break
        while len(s) < MAXBINSIZE:
            ns = input.read(MAXBINSIZE - len(s))
            if not ns:
                break
            s += ns

What is this.

Also while true lol.

Better hope 'not s' or 'not ns' occur at some point each time 😛

[fallen cedar]

well, there are breaks :)
it's decompiled code, so it might not be like this in the original code
but as i said, i'm not a python guy

[silver furnace]

Who writes like that. Even for python

Fair enough actually

[fallen cedar]

but yeah, usually seeing this should raise some red flags for possible code smells 🙂

[silver furnace]

Yeah tbh I'm not sure. On one hand, could be compiled to that

on the other, it could be decompiler's fault?

I'd have to know cpython's byte code and how it compiles, etc

Like how does it " import re, struct, binascii "

[fallen cedar]

yeah, i honestly don't know enough about python to tell
i assume that source -> byte code in python world is not a two way street, so it's impossible to get back the original
is that a correct assumption?

as long as the byte code does the same thing behaviorally, there can be any number of optimisations applied which makes converting back to original source code impossible
just my guess

[silver furnace]

Yup. There's all sorts of scenarios. Same with gcc/g++. Various optimizations, the compiler itself and so forth.

How to compile switch statements is a popular one 🙂

[fallen cedar]

jump table 🙂

[silver furnace]

Even cpython it's some byte code for python not like actual x86 instructions right

True just had to look to remind myself lol

So the Terry Davis way 😛

[fallen cedar]

yeah, honestly i rarely if ever look at assembly (only if i stumble upon an interesting article on r/programming)
compilers are very smart nowadays 🙂 loop unrolling, inlining, branch elimination and so on

RIP Terry 🙂

[silver furnace]

Amen. RIP Terry Davis.

One man that managed to use an Intel Core like c64.

[fallen cedar]

😄 yeah
still, TempleOS is impressive
i'm curious what kind of work he did before his "more famous" days

[silver furnace]

Well impressive is an understatement. Usually you make a kernel. Then drivers (why you made the kernel).

After that, you forget about it like a bad dream

TempleOS you are the kernel

I guess not really but when you're unrestricted lol. What is a driver. Whole thing never enters restricted mode right? You're just stuck on the barest metal there is

[fallen cedar]

yeah
sounds pretty dangerous though 🙂 it was a long time ago, so i don't remember: all application code is running with highest privelege in TempleOS? if yes, then that is a footgun 🙂 just as God intended I guess

[silver furnace]

Lol Gentoo get out the way. This is the real harnessing of your pc's power

Nope it's not in real mode apparently

Ring 0 and 64 bit mode (idk maybe its set for entire usable memory)

[fallen cedar]

i see
ring 0, still pretty low level

[silver furnace]

Yeah. Idk what difference between ring 0 protected and real mode is. We still start off in real mode always. Then enable usually like 0 and 3 or whatever.

[fallen cedar]

reminded me of: beyond ring 0 🙂 https://www.youtube.com/watch?v=lR0nh-TdpVg

his talks are really mind blowing

[silver furnace]

There's a bunch more you get out of protected mode. AMD initially documented long mode from real. Now supposedly there's only documentation with protected mode

Oh...

yeah I remember something about the APIC thing. Oops.

Wait ring -2?

I'm done lol 🤣

I've heard of ring -1

[fallen cedar]

this is the typcial "we didn't foresee we'll need it"
"we have ring 0, how should we name the lower level? hmmm, just keep decrementing it"

[silver furnace]

Idk

You can max be in ring 0, the kernel one they call it

The negatives I guess you see them now. I thought it was some virtualization crap before; also only heard of -1

Intel the whole thing wasn't thought of until every year day after for like 4 decades now?

[fallen cedar]

yeah, i don't think your code can run in these modes ever
only "built-in" stuff: https://en.wikipedia.org/wiki/System_Management_Mode

[silver furnace]

Oh that answers my question a bit.

What about all those configuration address spaces

[fallen cedar]

revising my statement: your code shouldn't run in this mode, but hackers of course found flaws

[silver furnace]

Nah that's like firmware probably

Well yeah IME lol

cisc, microcodes, etc are fine

I think they just dun did it. Like why is a separate controller accessed in the address space along with everything

Oops they already had instructions for it

Idk I'm way out of line here. For some reason, I guess they needed it to go through the memory controller like everything else.

[fallen cedar]

don't worry, i'm way out of line here too 😄

sitting comfortably in the application programming space currently 😄

[silver furnace]

I know right. Also god bless x86 for being a thing and having the lion's share of soft for it now.

And all the hypervisor stuff. Metal sandbox.

Oof not even. IOMMU is a thing too right.

Another unit to map physical addresses to "guest-physical" address space.

ffs.

[warm sleet]

MMU and IOMMU do this mapping

MMU makes a mapping between virtual memory and physical memory

the IOMMU makes a mapping between virtual device addresses and physical device addresses

with IOMMU you can also group devices to allocate to a specific virtual machine

for hardware passthrough

[lament cave]

hmm interesting, so you were able to decompile base64.pyc?
also, did you try decompiling the main pyc file (should be called EditorVER1with no extension)?

I couldn’t even decompile base64.pyc lol

[lament cave]

pyinstxtractor says that it is confirmed to work on pyinstaller v2.1+, and I don’t recall any mentions on their GitHub page that there are some cases where it doesn’t work (I may be wrong)

[gilded moon]

trying to parse HTML with BeautifulSoup, but it wants to load the whole page into memory, just feels like such a clunky solution

[warm sleet]

@gilded moon lot of semantics in html

can't xpath without fixing the document first

the parser and lexer used for html is very complicated

because <script> tags use a different parser

and so does css in <p style="">

or inline css

[gilded moon]

yeah, for reference i'm just grabbing the package names from https://pypi.org/simple/ nary a script or css in sight

[warm sleet]

scraping eh

there exist libraries to normalize html

[gilded moon]

any favorites to recommend?

[warm sleet]

oof

uhm, I only wrote scrapers in java and .NET

https://html-agility-pack.net/

this was for .NET

and I believe the preffered lib for java is JSoup

though there are alternatives

[gilded moon]

I wonder if BS4 and JSoup are siblings XD

[warm sleet]

its called 'Soup' for a reason

a lot of html documents are faulty

https://en.wikipedia.org/wiki/Tag_soup

has a name ^

[gilded moon]

da souuuuup

[warm sleet]

tag soup, aka: a big whopping mess

[gilded moon]

looks longingly at the glass case labled 'Regular Expressions

it's so close

[warm sleet]

yeah regular expressions wont help you fix a broken html document

you need semantics

implement a decision tree

what to do when the parser fails at a certain point

what the fallback would be

web development is a mess

[fallen cedar]

I don't see a file with that name (or anything similar)

[lament cave]

hmm

https://i.cocobot.tk/QX97cuvh.png

oh wait

odd

when I extract it, it shows up

[fallen cedar]

i don't even see files where the extension is NOT .pyc

[jovial remnant]

Hi there. Hope someone know this. I'm converting a git repo to use lfs with bfg. Using latest version of git and lfs version 2.9.2.
I need to execute git lfs init. But that is an unknown command.

[fallen cedar]

again: used this for extracting the .exe you sent: https://github.com/extremecoders-re/pyinstxtractor
this only produced .pyc files, nothing else, no .dll or .pyd like in your screenshot

[lament cave]

interesting

yeah I have the latest version of pyinstxtractor

[warm sleet]

that script is hillarious

just guts the files directly from the binary

[fallen cedar]

ok, forget what i said lol
i was looking at the wrong directory lol

[lament cave]

oh lol

[fallen cedar]

let me try decompiling that file 😄

[lament cave]

alright, let's hope it works 🤞

[warm sleet]

https://github.com/extremecoders-re/pyinstxtractor/blob/master/pyinstxtractor.py#L257-L261

[lament cave]

ty for your help btw

[warm sleet]

^ lol

[fallen cedar]

https://pastebin.com/eV0yRM5Y

[lament cave]

OMG

TYSM

wait, just so i know next time, can yout ell me what version python you're using?

since I have one more to decompile and I don't want to keep bothering you

[fallen cedar]

i'm on windows, using Python 3.7.9 downloaded from https://www.python.org/downloads/release/python-379/ (x86-64)

[lament cave]

ahh maybe that's why

I'm using 3.7.0

ty for your help again

[fallen cedar]

no problem

[gilded moon]

peasents, 3.9 is where its at

[unkempt cloak]

Heyo, using python 3.9.1 its been a while and im trying to simulate coin flips but cant get the random module to work. it keeps returning that things like choice and random are undefined. from what ive seen online and in the in built help menu it says the random module is built in. Im a bit confused

[fallen cedar]

hi
show the code if possible and we'll help (at least try :))

[unkempt cloak]

Should add its a name error. Im a just missing something before choice

[fallen cedar]

did you import random or from random import choice?

if you show source code, it's easier to help

[unkempt cloak]

spurce code?

[fallen cedar]

yes, the code you're writing

[unkempt cloak]

numflips=4

result=choice(['heads','tails'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'choice' is not defined

Igot stuck on line 2 lol

[fallen cedar]

are you just typing things into the interpreter? or writing a script in a .py file?

[unkempt cloak]

see my problem now...

i was just typing in the interpreter

[fallen cedar]

so you'll need to import either the full random module with import random and modify choice(['heads','tails']) to random.choice(['heads','tails'])
or you can just import the choice function with from random import choice, in that case it can remain choice(['heads','tails'])

>>> from random import choice
>>> result=choice(['heads','tails'])
>>> result
'heads'

[unkempt cloak]

Thanks, this is what happens when you dont code for 5+ years and you forget which programs you used when you did

[gilded moon]

I definately vote for importing Random

that way you don't end up shadowing your own code or other libraries

also, Pastebin (or even better Github/a gist) is your friend

[midnight wind]

or hastebin, also nice

[gilded moon]

also, scraping 330K PyPI packages is never going to be fast, even if I want to sacrifice my soul to the async devil

[proper stream]

hello i have a question about raspberry pi

on the raspberry pi which im using raspbian how do i run a program at startup automaticly?

[hollow basalt]

you run it like a raspberry pi not running raspbian

[jolly arch]

Hello does anyone knows the difference between jdk 8 and jdk 15 (java development kit) ?

[warm sleet]

@proper stream you can use cron for that

@proper stream if you have a command you need to run at startup, you can use crontab

run: crontab -e

and then add the following line:
@reboot somecommand

replace 'somecommand' with whatever you want

@jolly arch java 15 is latest, java 8 is legacy, though still used

I've been stuck developing with java 8 for a couple years now

@proper stream the clean way of managing services would be through systemd or with rc.local init

[jolly arch]

i see so i downloaded jdk 15 from oracle

and want to learn java with intellij as an ide

[warm sleet]

just install IDEA and the JDK

once you've done that, you can configure the JDK inside IDEA

[jolly arch]

yeah i did it i configured the use of JDK 15

thanks man

[warm sleet]

@jolly arch It might not immediately be needed, but I highly recommend learning java using maven right away

makes managing projects a lot easier

[jolly arch]

ahh ok

[warm sleet]

maven is a project lifecycle tool

it helps manage libraries/dependencies

[jolly arch]

i saw maven in idea but didn't know its use

[warm sleet]

as well as handle compilation, packaging, testing

so you can essentially compile your entire project into a jar with 1 command

mvn package

[jolly arch]

aha

[warm sleet]

and if your project needs some additional library

all you have to do is add an entry to your pom.xml

[jolly arch]

what is the best place to learn java

[warm sleet]

no idea

lot of tutorials on the internet suck

they teach java the wrong way

https://i.imgur.com/gl3SuJS.png

see IDEA ^ can manage all these libraries

[jolly arch]

i saw a tuto on geeksfoegeeks

[jolly arch]

yeah i

[hollow basalt]

If you already know a programming language, then pretty much you only need OOP concepts to jump

[warm sleet]

@jolly arch do you have any experience with other programming language?

[jolly arch]

yes i learned python and ruby

[hollow basalt]

Then time to go OOP

[jolly arch]

but i wanted to explore java

[warm sleet]

ahh neat, so both of those share some concepts with java

[hollow basalt]

i didn't say you wouldn't use java

[crystal tundra]

a good suggestion is dont treat OOP like the only paradigm

[hollow basalt]

but you need to learn OOP first

Java is what OOP on drugs looks like

[warm sleet]

@crystal tundra my whole view of OO got flipped upside down when I started writing typescript

[crystal tundra]

??

[warm sleet]

its OO, yes. but gradually typed

[jolly arch]

ahhh ok 👌

[crystal tundra]

OOP exists in statically typed languages uno

[hollow basalt]

off you go to learn OOP

just the basics then you'll pretty much intensify the knowledge by using more java

like a minecraft plugin

or whatever your interest is

[warm sleet]

[hollow basalt]

[jolly arch]

yep

[warm sleet]

@jolly arch I taught myself java with minecraft lol

https://i.imgur.com/gj7fPod.png

[jolly arch]

[hollow basalt]

i mean that meme is wrong but

ok

[jolly arch]

hahahhaa

[warm sleet]

@hollow basalt I logged onto the minecraft server yesterday, to my surprise we had 26 people online

[hollow basalt]

Atleast in minecraft you actually implement java

[warm sleet]

I wrote over 35k lines of code for that community since 2014

[hollow basalt]

[warm sleet]

minecraft is 100% java, and hardcore too

[hollow basalt]

this guy is a real java-er

[hollow basalt]

c++ java didn't got the traction

[warm sleet]

we've even modified system classloader on your servers

to inject our own bytecode into the minecraft server xD

[hollow basalt]

C++ java

10 secs faster

didn't matter

[warm sleet]

spigot is garbage sometimes

@hollow basalt we hacked the minecraft texture pack, allowing people to have pets summoned and fly around their character

[hollow basalt]

that's also the nuance that some people didn't get to use

[warm sleet]

only issue now is

we cannot teleport a player who has one of those 'pets'

[hollow basalt]

they pretty much assume that

• create class
• load class
• profit

[warm sleet]

because those pets are technically 'riding' the player

using the same code that is used for pigs and minecarts

[hollow basalt]

so if the player is riding a pig

then it's a rider riding a ride

[warm sleet]

so now I have to intercept any player.setLocation() calls to spigot

[hollow basalt]

rider riding a ride while being rode on

[warm sleet]

probably end up forking it tomorrow lol

[hollow basalt]

[time for you to create spigot fork]

[warm sleet]

@hollow basalt well, ideally..

https://github.com/PaperMC/Paper/tree/master/Spigot-API-Patches

We can add our patch to this list, and compile that into the server

[hollow basalt]

fork the fork

[warm sleet]

since all we have to do is patch the API

not the game itself

[hollow basalt]

yea

only spigot is uhh

inconsistent

[warm sleet]

tell me about it

I've been developing plugins since beta 1.2 with Craftbukkit

[hollow basalt]

Craftbukkit

brings me memories

[warm sleet]

before it got DMCA'd

wolverness 🔫

[hollow basalt]

craftbukkit also reminds me of the old mod loadres

[warm sleet]

hMod

[hollow basalt]

before everyone went forge

[warm sleet]

its all fml now

[hollow basalt]

yea

[warm sleet]

Fabric is yet another attempt at making a modding API

[hollow basalt]

Forge (is good?) they pretty much killed the competition

[warm sleet]

Forge is great

[hollow basalt]

yea

[warm sleet]

I worked on forge briefly

I was part of the #ForgeGradle group on espernet in 2014

[hollow basalt]

Old mod loadrrs give me the PTSD of actually changing the JAR file

forge made it not insane

[warm sleet]

lol

those

@hollow basalt I had a minimap in alpha 1.4

:D

[hollow basalt]

You broke the jar file?
Congrats, redownload minecraft

Good times

[warm sleet]

The guy who made the first minimaps was a player on the server I was on at the time

RawCritics they were called

[hollow basalt]

Yea, good thing you are actually part of a server that got thru the test of time

most of the servers I used to play on, is now defunct

[warm sleet]

rawcritics is not known about much, but they were very influential

because in 2012

https://www.youtube.com/watch?v=HI5qvmVxOQk

they released this ^

First server to run multithreaded worlds seamlessly on 1 server

[hollow basalt]

Yea, multi-world is pretty much a requirement now

[warm sleet]

this was before the nether even worked in vanilla

[hollow basalt]

Such, actions that result in the whole community

[warm sleet]

I think the guy who wrote this was TyrOvC

[hollow basalt]

Haven't heard of the guy

probably this was before I even knew about minecraft plugins

[warm sleet]

he was the main developer for rawcritics at the time

while everyone ran hMod

we had our own custom server

[hollow basalt]

still sad that the community got like divided

[warm sleet]

hMod was legacy, even when it was still under development

new versions took forever to be released

and there was no uniformity in anything

no standardized permissions

@hollow basalt and ultimately the reason I started writing my own plugins was because of this

cant wait on other people to update their plugins after a new minecraft release

used to take 2-3 months

we can now, update within a month

[next igloo]

can you put <div> or <p> in <head> or outside <body>

[fallen cedar]

you can, i mean the computer won't blow up :)
but why and what do you want to achieve with it?
it won't be a valid html document (as far as i know) and don't know how different browser engines will handle it

[next igloo]

its not my code, im just trying to trouble shoot why my friends game is not working

[midnight wind]

what's not working exactly?

[next igloo]

so when i pasted the code into a diffrent code editor, (codepen to glitch) it gave me some errors and after i fixed the errors the game did not work in quite the same way

[midnight wind]

look at the official mozilla docs

[next igloo]

codepen has no error checking to my knowledge and will accept html files without <head>

[midnight wind]

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/div

[pliant siren]

You can do it, but it wouldn't be valid.

[next igloo]

ok so that probably is not the reason why his game not working

[autumn moth]

Hey, so I am making a virtual assistant and I will make a dashboard for it to make it work as an app now the thing is I dont know much about python or coding my virtual assistant is done but I am facing difficulties in integrating it in an app so if anyone is good at coding and is interested to help me please DM me. My chat with you will probably take a good amount of time so please make some time before contacting me also it is currently 2PM at morning here so please match this with your time to know if you can help me thanks a lot and this is a PI project after I make this I will transfer it to my raspberry PI so yea I am making a app for it. If anyone is interested in helping me please DM me I will really appreciate it.

[hollow basalt]

How good are you at programming in genereal

[autumn moth]

you can say i am a beginner in python i am learning thats why i coudnt integrate that

btw can u help me? i am already 70% done in making the UI just need to integrate that code in it @hollow basalt

[hollow basalt]

[quasi plover]

is there a reason to use c++ classes in separate files beside being a little organized ?

[upbeat nymph]

Possible class name collision, no?

[upbeat nymph]

^ this too. For like tiny projects, probably not worth it tbh. Especially if there's no risk of name collision.

[cloud knot]

https://omar.website/tabfs/ weirdest random thing

[umbral saffron]

Do u guys know how to put a file on a flash drive ok Mac Sierra

[umbral saffron]

New issue

The lock for read only doesn’t come up

[umbral saffron]

mhm

[umbral saffron]

it wont let me put stuff on it

its not an issue anymore but if i need to back up my laptop ill need to

[proper stream]

hello how do i fix the problem on raspberry pi cannot currently show desktop? vnc viewer

thx

[hollow basalt]

you're welcome

[proper stream]

..

[drowsy knoll]

Hi, I Built A Discord bot with a neural net inside of it, trained it to play tic-tac-toe atm but am working on training a new neural net for chatbot purposes. Its my first 'open-source' project https://github.com/HarryOC493/DiscordBot

Please Check it out, advice appreciated

[umbral saffron]

(If you have this) make sure the switch on your power cable is on. If that doesn’t work make sure the monitor is on. If the monitor turns on but the signal isn’t on the screen, take off the case if you have one and look if you see a solid red light and a blinking green light

If you don’t mess with the power cable

[umbral saffron]

anyone know why i cant install pyaudio through the mac terminal

[cyan jetty]

@umbral saffron what OS?

MacOS?

[brave tulip]

hi I am programming in cpp and I have an error

[spring pond]

you havent linked your libraries correctly

[nocturne galleon]

someone that can help me create a discord bot, ive invited it to my server but now how do i program it to do stuff?

[spring pond]

i may not be registering this correctly, but just add commands according to the API of the library that you've chosen?

[nocturne galleon]

well i want it to apply roles, like if a member sends more than 10 pictures it gets the role 'regular' assigned. When they join for the first time they get the role 'unverified' and when the accept the rules by clicking ✅ in welcome and rules channel they are assigned 'newbie'

unverfied in this case can only see that one channel until they accept..

[umbral saffron]

Yeah macos

[cyan jetty]

MacOS?

In that case.

I can't help you.

[spring pond]

what language are you using

[nocturne galleon]

English?

[spring pond]

lmao programming language

[nocturne galleon]

None?

I dont know programming

[spring pond]

so are you trying to create a bot or configure an already existing one?

[nocturne galleon]

create one

[spring pond]

well then you need to use a programming language such as JS or python

[nocturne galleon]

i can try js

is it hard?

[spring pond]

both of the languages i mentioned are very easy

[nocturne galleon]

i have worked with js before, with minecraft, not sure if that helps

[spring pond]

minecraft runs on java not javascript

but it’s good you have some experience

[nocturne galleon]

can you help me program this bot?

wouldnt know where to begin?

i suppose first having a server or some sort of host?

[spring pond]

well first you need to learn js

personally i know very little js so i wouldn’t be much help

but others in this chat can help

[regal geyser]

A bot like this will require you to setup a database and then interface with the database from your bot or a separate API that you will have to build. I would recommend looking at the docs for DiscordJs. https://discordjs.guide/

@nocturne galleon I would also recommend using Typescript over Javascript if possible. It'll be closer to Java and you'll be able to prevent the common issues with Dynamically Typed languages

[nocturne galleon]

You are flooding me with information

[regal geyser]

look into Discord.Js, that link I provided has a bunch of different tutorials that will basically cover everything that you are looking to accomplish

[nocturne galleon]

@regal geyser thanks, can I ping you if I have questions? Besides, this is a good time to learn, been sitting at home for the past 2 months...

[regal geyser]

I don't know how much I'll be able to help. But if I'm free I can take a look

[umbral saffron]

Anyone have any good python imports for speech recognition

[nocturne galleon]

i forgot about this channel lmao

yes but i can't seem to find the root of the problem ;-;

I think when i was fiddling with the date validation and stuff, thats where it got broken

[nocturne galleon]

question, why is the else statement in this code always selected, even tho i input different dates

var delDate = document.getElementById('date');
var curDate = new Date();

function dateCheck() {
    if (delDate<curDate) {
        alert('The date you selected is correct');
    }
    else{
        alert('The date you selected is invalid');
    }
}```

[nocturne galleon]

done!

[bleak breach]

You don't appear to be comparing the same type... so I'm not sure that it will work the way you expect every time.

[nocturne galleon]

its done now 😄 fixed it

function dateCheck() {
    const { date } = form;
    const body = {
        date: date.value, 
    };

    const delDate = document.getElementById("code").innerHTML = body.date;
    var curDate = new Date();
    var msec1 = Date.parse(delDate)
    var msec2 = Date.parse(curDate);

    if (msec1>msec2) {
        alert('The date you selected is correct');
    }
    else{
        alert('The date you selected is invalid');
    }```
Seems to be doing its job quite well

[upbeat nymph]

Fuck I need to figure out what’s slowing this NodeJS code down so ducking much.

I mean at scale something that’s taking other libraries seconds is taking this one 7 hours. I need to trace what call exactly is taking so ducking long.

I’m literally trying to sleep because I’ve been up for 24 hours and I’m struggling because I have this code in my head and I want to know why the fuck it isn’t working correctly

[hollow basalt]

it's node

[amber cloud]

regex isn't as hard as people say but looking back at your work without syntax highlighting would cause a stroke

[nocturne galleon]

@regal geyser hey, thanks again, ive got a source project from github installed that has a 'rolemanagement'. Bot is functioning as intended.

[nocturne galleon]

I need some help choosing what i should do for a fun project. Here are my ideas:

1. a command line shortcut helper thing. I'm thinking you could add little scrips to it, categorize them and search with keywords or just add notes about commands
2. a website where i put a simple event that has hapepend everyday in 2021
3. a desk pet or live wallpaper thing
   But idrk what would have enough things to do for it to be a long lasting/real project

Also other ideas are welcome

[fading locust]

regex makes my brain underflow

[gilded moon]

https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_pickle.html

I kid you not, my Dataframe turned itself into a Pickle, funniest shit i've ever seen

[upbeat nymph]

Yeah I get that and yes there’s limitations with Node, believe me I’ve hit some already but that’s still no excuse for my bad code and horribly optimized algorithms which takes thousands of percent longer to execute than other libraries with no real advantages

[bleak breach]

@upbeat nymph What's your code doing?

[hollow basalt]

[upbeat nymph]

Well. When I was working with Go I found a Glicko package I really liked however one with its style didn’t exist for NodeJS. So I translated it directly over. Obviously Go is a system language so multi threaded, etc. Node is not. So I knew there would be some performance hits initially but I didn’t expect it at the scale it was. So. Like here’s the translation: https://github.com/MatthewSH/go-glicko

I wrote the benchmarks for it and other libraries here: https://github.com/MatthewSH/glicko2-benchmarks

At 1m players running 3 periods of 1m each it took 7 hours while others took seconds. I’m at work right now so I can’t diagnose why, I work 3rd shift. However it could be that the other libraries are failing out. I’m just benchmarking speed, not accuracy. Although that is something I’ll need to benchmark as well.

It probably comes down to the algorithms, since Go at a deeper level is probably better suited for doing the same things much quicker than Node. I’ll be running timers or something on each step with a semi larger dataset for my own testing when I get home after work.

Keep in mind it’s like almost a 1:1 translation. I’ve made no attempt directly to optimize it but I also didn’t think it would perform that horrible.

[hollow basalt]

Obviously Go is a system language so multi threaded

what does this statement mean

[upbeat nymph]

I’m typing and working so I may have mistyped.

[hollow basalt]

so what does it mean

[upbeat nymph]

First I misused the term systems language, that’s my bad. Lmao.

So like NodeJS stills need the v8 engine (is it still on v8? I believe so. I can’t Remember) to run while Go can be compiled down to machine code. COMPILED LANGUAGE. THATS THE TERM I WAS LOOKING FOR

Also, Node is single threaded, and entire process works on one thread and one thread alone while Go, Java, etc can work naturally on more threads

[midnight wind]

@upbeat nymph you can multithread node

And for one, you need to code in a nonblocking way

Node doesn't think in threads like other languages

It will spawn threads for you iirc

Just don't block the main loop

[hollow basalt]

multi-processing takes advantage of more core's and multi-threaded just means it can jump between multiple tasks

can change because of another core
ehhh, it should be threads

uhhh what

It doesn't mean it does anything in parallel
So you mean multi-threadhing is just one task at a time?

I just want you to answer yes

this is for keeps

[upbeat nymph]

You can by spawning workers sure, but then you gotta balance calculations of the same rating period on all the workers while making sure not to update the same user more than once with nth amount of players, matches, and matches per player. Depending on the abstraction of the algorithm too could change that. I was thinking about it last night before I saw the results of the benchmark that clustering the process could make it scale horizontally nicely, possibly, but there’s other complications with that. However, the main problem again is that this code translated from Go to NodeJS is terribly unoptimizable for for a single threaded, non compiled language which is the issue I need to solve. I didn’t write the code, I just translated it from Go. But I’ll need to rewrite it, which is my goal of the benchmarks in the first place. Then I’ll have to write a test for accuracy at some point too.

[hollow basalt]

Agree

[upbeat nymph]

But I think we need to come to an agreement that the code that I wrote for the translation is well...broken. And bad. I just need to figure out how to fix it

Or the best way to fix it

[hollow basalt]

It's fundamentally slower

it should be given

[upbeat nymph]

We’re not talking about the differences between the languages at a core, because compiled is always better. We’re discussing how the exact same algorithm in go is taking thousands times longer in NodeJS compared to other libraries that do the same exact thing but just written differently. It could also be that the Go version is extremely accurate compared to the others but I haven’t created a test for that...yet.

I might do that tonight.

As well

No.

I’m comparing benchmarks with simulations up words of 4m individual objects

And running Glicko algorithms on them

Upwards*

[hollow basalt]

I haven't programmed in go, but the compiler probably optimised the hotspots

[upbeat nymph]

Damn mobile automatic fixing

Just look for the link I posted earlier

It has all of it

[upbeat nymph]

I’m not for certain, but I think that’s a safe bet. I think Rust does the same

I’d assume so...since their job is to compile to machine code. But that’s the problem with general purpose languages like JS and Java. Especially JS, inherently slower because it’s not compiled.

[upbeat nymph]

Tbh. I think my benchmarking is also flawed. All the data is random. So I’ll have to like come up with something better all together to test the system.

And something that can be used across multiple languages

Hmm but GitHub has a file limit. I’ll do something to get around it to properly test

[upbeat nymph]

yeah, and? 1m simulated players is ~500mb. Larger datasets will be more. So

I wanted to get to about 10m if possible

[nocturne galleon]

In javascript, how would you go about approaching this problem of having function4() run if and only if all func1(); func2(); func3() returned values that is required for func4() to run?

I would probably use Javascript promises for that

[upbeat nymph]

Do you have code written for it so far?

Because you can use promises, but you can also just like...not too.

[upbeat nymph]

Depends on use case and if you want the first 3 functions to run in parallel or not.

[upbeat nymph]

Huh, so digging into why...I did a broken up benchmark on a dataset of 100k players, 200k matches.

This matches (roughly) the testing of a single period of 100kp/100km (~58 s)

[upbeat nymph]

So further testing, it's the way it loops. 🤔 JS looping is so slow

[upbeat nymph]

I think it's because well, it's always recalculating the length, on top of searching the player list many thousands of times

[nocturne galleon]

Im done, i did not use promises for it

just simple

function cost() {
    if (display() && dateCheck() && timeCheck())```

[hollow basalt]

[nocturne galleon]

the functions shall return true of their outputs are correct and return false if not, it works

[upbeat nymph]

yeah. See that's fine. HOWEVER! you could use promises and complete them all at once

With Promise.all

And run it async

[nocturne galleon]

im having a hard time wrapping my head around on how to code that as i have not really learned about it

[upbeat nymph]

Then no need to dive into promises right now

[nocturne galleon]

but maybe in my next project i will

cuz promises are made to be broken

jk

[hollow basalt]

if you don't use promise in JS, you're not javascripting enough

[upbeat nymph]

@hollow basalt so yeah...this entire performance thing is...yeah. That same benchmark without without the looping ran just above the others...and that's only cause they don't track the players like I do. I'm thinking of ditching the initial loops all together and generating a unique hash for each one...but again...have to benchmark it. 🤔

[hollow basalt]

really need to remove those excessive computes somehow

[upbeat nymph]

So in that 1m benchmark it's literally looping through the players and adding them...the addPlayer method loops through the current array to see if they exist, if not...add then add them...each time....then addMatch runs addPlayer for each player...2 more times....so it's running it like...30m loops LMAO. In Go this probably wouldn't be a problem...but JS? Holy shit

[nocturne galleon]

i will learn to javascript better 😄

[hollow basalt]

sure, you would be better as go through college

[nocturne galleon]

yep, 2 months sembreak, will spend time honing skills in these programming languages

[upbeat nymph]

Just spend your months studying algorithms you don't understand just to fucking mess with the idea of matchmaking and then want to die finding unoptimized for loops. That's what I did

[upbeat nymph]

Okay...for the record...here's the current benchmark results.

# create 25,000 players and 75,000 matches.
ok ~60 ms (0 s + 60275300 ns)

# create the rating period.
ok ~31 μs (0 s + 30600 ns)

# add all players to the rating period.
ok ~250 ms (0 s + 250250800 ns)

# add all matches to rating period.
ok ~14 s (13 s + 944978900 ns)

# running calculation
ok ~401 ms (0 s + 401019900 ns)

all benchmarks completed
ok ~15 s (14 s + 656555500 ns)

Which is within 3s of the original benchmark. Same code.

Time to optimize.

wtf

this can't be real

# create 25,000 players and 75,000 matches.
ok ~98 ms (0 s + 97704000 ns)

# create the rating period.
ok ~53 μs (0 s + 52800 ns)

# add all players to the rating period.
ok ~14 ms (0 s + 14063400 ns)

# add all matches to rating period.    
ok ~23 ms (0 s + 22827100 ns)

# running calculation
ok ~163 ms (0 s + 163168800 ns)

all benchmarks completed
ok ~298 ms (0 s + 297816100 ns)

WHAT

no fucking way

Hmmm... but now the test fails...intersting

I think I fucked up the calculation loop. I just tested it...it passes now

# create 25,000 players and 75,000 matches.
ok ~104 ms (0 s + 104423800 ns)

# create the rating period.
ok ~28 μs (0 s + 28100 ns)

# add all players to the rating period.
ok ~15 ms (0 s + 14793400 ns)

# add all matches to rating period.
ok ~26 ms (0 s + 25738300 ns)

# running calculation
ok ~170 ms (0 s + 169538900 ns)

all benchmarks completed
ok ~315 ms (0 s + 314522500 ns)

yarn run v1.22.10
$ H:\Code\go-glicko\node_modules\.bin\jest
 PASS  src/Glicko2.spec.ts
  √ players should be created with proper ratings (2 ms)
  √ rating period should calculate correctly (1 ms)

Test Suites: 1 passed, 1 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        1.809 s, estimated 3 s
Ran all test suites.
Done in 2.82s.

Okay...but now the real test...1m players and 1m matches.

omg. I wanna cry..not really. but really

From an average of 6.97 hours down to an average of 8.25 seconds... I'd call that optimization.

[upbeat nymph]

however, I wonder if Map would perform better 🤔

[umbral saffron]

Imagine talking to yourself

[upbeat nymph]

# create 25,000 players and 75,000 matches.
ok ~101 ms (0 s + 100848000 ns)

# create the rating period.
ok ~37 μs (0 s + 36900 ns)

# add all players to the rating period.
ok ~6.52 ms (0 s + 6518800 ns)

# add all matches to rating period.
ok ~24 ms (0 s + 24201900 ns)

# running calculation
ok ~150 ms (0 s + 149911700 ns)

all benchmarks completed
ok ~282 ms (0 s + 281517300 ns)

Hm... a 10% performance increase.

I will continue to talk to myself

because I live alone 😦

[umbral saffron]

I just spent 10 minutes reading this whole conversation

[upbeat nymph]

lmao.

omg my cat is a crackhead.

[warm sleet]

node is total ass for high throughput calculations

begins with the lack of proper integer types

and ends with float64-all-the-things

[upbeat nymph]

yeah. whats crazy though is the reason it was slow wasn't even because of calculations

It's the loops

[warm sleet]

its interpreted

@upbeat nymph I'm not too familair with language internals

but if your loop has a context it switches between

can take a lot of time

[upbeat nymph]

Well I can tell you...that switching from looping through an array to an object took a 6.97 hour calculation down to 10 seconds....

So I know that for a fact

[warm sleet]

that sounds very odd

are you just iterating object members?

instead of iterating over an array?

[upbeat nymph]

well...let me show you the diff

https://github.com/MatthewSH/go-glicko/commit/9ceca9f0a1e850d553efe08414d6bb432b2be918?branch=9ceca9f0a1e850d553efe08414d6bb432b2be918&diff=split

And I'm about to increase the performance by ~10% (based on initial tests) by switching to maps

[warm sleet]

linear complexity is way different

the left solution scales poorly with larger set sizes

[upbeat nymph]

Without needing to convert the object keys to an array and calculate the size of that

yeah

and that's what my benchmarks showed

[warm sleet]

the right solution is always O(n)

[upbeat nymph]

mhm.

https://github.com/MatthewSH/glicko2-benchmarks#1m-players-1m-matches

This is why I benchmarked

[warm sleet]

yeah looping over an entire list to find a match is terrible.

hashtables to the rescue!

[upbeat nymph]

Well it was a direct conversion from the Go package, which Go handles arrays much differently but I never tested that package at this scale though...yet. However with my conversion I did.

[warm sleet]

Well

I can tell you that this would be the same in almost every language

Go might be able to optimize compiler output

but interpreted languages like js do not get the same kind of benefit

[upbeat nymph]

Sadly, which is why I decided to run these benchmarks. The next test is to test accuracy over 1k matches...however first...I gotta publish this update and make sure it still passes tests

[warm sleet]

hashtables will always give you O(n) complexity.

Only scales with the size of your key and the algo you use for the hash

[upbeat nymph]

I guess I do it so often I never think of it. Until now

And it's UUIDv4.

[warm sleet]

that's 128 bits

[upbeat nymph]

So the probability of collision is like...slim as fuck

[warm sleet]

impossibru

[upbeat nymph]

Would take trillions of entries IICR to even have a one in a billion chance

[warm sleet]

UUIDs are perfect as key surrogate

@upbeat nymph 2^128 - 1

:D

amount of combinations

[upbeat nymph]

So...lets see how much map increase performance over 1m players and 3 rating periods of 1m matches.

[warm sleet]

map performance should remain O(1)

regardless of set size

access time only depends on the complexity of your hash function

the larger your key, the longer that takes

[upbeat nymph]

Avg of 8.25s down to 7.95s. 3.6% performance increase.

Of object -> Map

[warm sleet]

Only gains you could get after this

is smaller key size

[upbeat nymph]

Which would increase risk of collision

[warm sleet]

not nessesarily

I've worked with hashmaps that can handle colissions

[upbeat nymph]

I mean, sure. But logically you wouldn't want collisions when generating Glicko2 ratings

[warm sleet]

https://upload.wikimedia.org/wikipedia/commons/thumb/7/7d/Hash_table_3_1_1_0_1_0_0_SP.svg/1280px-Hash_table_3_1_1_0_1_0_0_SP.svg.png

You can have multiple keys that resolve the same hash

[upbeat nymph]

Now the major scale risk would be how much information NodeJS can store in memory

[warm sleet]

You'd do a 2nd comparison, inside the bucket specifically

@upbeat nymph

everything is a float

[upbeat nymph]

Which struggles at 8.379 million entries and runs out of memory XD

[warm sleet]

for real though

[upbeat nymph]

At least when I was testing the generation of fake data

[warm sleet]

who keeps 8 mio records in running memory??

[upbeat nymph]

Well, yeah

[warm sleet]

You call upon middleware at this point

[upbeat nymph]

But it has to store the ratings somewhere until you can calculate it.

However, I could offload it to redis

or something

[warm sleet]

some kind of database yeah

redis for high speed access

or sql

you'd be surprised how fast SQL can be if you optimize your key index

[upbeat nymph]

But this package is not for "scale" necessarily.

But I could create a redis variation that scales on this

[warm sleet]

I can filter through couple million rows in under 2 seconds

with a fulltext search

[upbeat nymph]

And yeah, but it wouldn't be permanent records. It's instances of a class

[warm sleet]

object instances to represent data will come haunt you soon

it works fine

until some goof sends you a 50MB json file, and you try to parse that...

[upbeat nymph]

Well this is all the information. I "could" put this into an object. However, I would have to do changes to the main rating period class.

[warm sleet]

what is that font 👀

[upbeat nymph]

Fira Code

iScript variation

[warm sleet]

https://github.com/blackro-se/blackfont/raw/master/blackfont_comparison.png

Blackfont ^ the one I use

with ligatures

so -> will be an arrow

[upbeat nymph]

this one does too

You just don't see in that class

[warm sleet]

@upbeat nymph mh. I normally don't do lot of JS.

let me look at how I did this with ts recently

I had some fun with maps too

import parser from "accept-language-parser"

export const langMap = {
  en: "en-US",
  nl: "nl",
}

export const getLanguageFromHeaders = (acceptLanguages: string) =>
  langMap[parser.pick(Object.keys(langMap), acceptLanguages)]

export const getLanguageFromQuery = (langQuery: string) =>
  langMap[langQuery]

[upbeat nymph]

Do you have this text? I'll show you mine

[warm sleet]

mh. yeah map literal as object

objects are maps

and with Objects.keys() you can iterate them

[upbeat nymph]

This is my ligatures look like

[warm sleet]

neat

[upbeat nymph]

my cat found a spot

[warm sleet]

they always find the best spots

[upbeat nymph]

I had to wake her because I need to write code

[warm sleet]

@upbeat nymph I had courses for like 3 months on advanced algorithms

they taught us how to implement search algorithms and sorting algos

so braindead sometimes

[upbeat nymph]

I've never had a single course on any of this

Never went to college

Self taught

[warm sleet]

the final exam was quite funny

we had to do live coding

implement an RPG style game in a castle

where you could move from one room to another

the castle was represented by a graph, with the rooms as nodes on that graph

and then you implement dijkstra's to find the shortest route through the castle

and such

[upbeat nymph]

lol

that sounds like a course I need tbh

my algorithm knowledge is horrible

[warm sleet]

https://en.wikipedia.org/wiki/Graph_theory

binary trees

also fall into this category

There's algorithms that can walk these trees or graphs

like this one

https://upload.wikimedia.org/wikipedia/commons/5/57/Dijkstra_Animation.gif

Dijkstra's Shortest path algorithm

[upbeat nymph]

This mother fucker slowly worked herself over

[warm sleet]

slow traffic ahead

[upbeat nymph]

this feels so good to post

[warm sleet]

so 7h

@upbeat nymph lol might as well write 'dnf'

[upbeat nymph]

I was at work

and I let it run

LMAO

Didn't expect that

[warm sleet]

when you go to test ackermann benchmarks on a microcontroller

once it goes past 5 mins, I just ignore the result

write down "too long"

[upbeat nymph]

I'm redoing all these benchmarks anyway. Gonna automate it with mongodb and allow it to be expanded to multiple languages

So I need consistent, reliable datasets

[warm sleet]

^ those two words do not mesh with mongodb

Its neither consistent, nor reliable

sorry, hating on mongodb is like a hobby almost

[upbeat nymph]

I'm open to other options.

[warm sleet]

Good 'ol SQL.

[upbeat nymph]

I just have been used to mongodb because I use it a lot recently and it scales with my discord bot pretty well

[warm sleet]

mongodb is more a document storage type database

[upbeat nymph]

But which SQL? MySQL? Maria? Cockroach? PostgreSQL? etc

[warm sleet]

Any, really

[upbeat nymph]

Like PostgreSQL would probably be my second choice

[warm sleet]

Postgres is nice if you like working with json

I mostly use MySQL and MSSQL Server

as much as I hate microsoft. MSSQL is the most ANSI-standard SQL db out there

next to OracleDB

[upbeat nymph]

I just need to think how I want to do the benchmarks. Like if I wanna do each dataset as a different benchmark I can do that

To "simulate" how data would be stored

[warm sleet]

@upbeat nymph well, you are shifting your workload

currently, you do your lookups in code

[upbeat nymph]

I guess I could just use MySQL and use just incrementing values and say fuck it to the "fake" usernames.

[warm sleet]

this is work that is typically done by the db engine

[upbeat nymph]

And generate random matches based on the count of the table.

Which would be a useless query because I know the size

but gotta account for a failed write

[warm sleet]

SELECT COUNT(*) FROM <table>

[upbeat nymph]

yeah

I know that 😛

I've worked with SQL

[warm sleet]

hurrr

[upbeat nymph]

Especially if I scale the benchmark to potentially 40m player dataset

[warm sleet]

@upbeat nymph my university made me implement a whole application, inside SQL

so every single interaction with the database, was declared as 'use cases'

and each use-case had an associated stored procedure

including error handling, safe-points and rollbacks

[upbeat nymph]

But I can't use drivers which "simplify" the query and have to use raw SQL to make it fair and not reliant on a package's optimizations

[warm sleet]

query builders are terrible anyways

[upbeat nymph]

I guess, but still. I think it's only fair

[warm sleet]

if you stick to ANSI sql, you can replicate the setup on various engine types

as long as you do not need CONSTRAINT, TRIGGER or PROCEDURE

mysql does not support CHECK's

triggers in mysql are terrible too

[upbeat nymph]

I'll probably just install MariaDB

Since I'll use that for other projects

[warm sleet]

ye maria fills the shortcomings of mysql

[upbeat nymph]

Actually, is Maria even better than MySQL these days?

Haven't looked in years

[warm sleet]

Different

MySQL uses InnoDB

pretty sure MariaDB has their own fork of InnoDB

InnoDB lacks CHECK constraints

this is something that maria aims to provide

[upbeat nymph]

yeah, writing the tests now.

[upbeat nymph]

not good, not bad. It works.

[upbeat nymph]

@warm sleet since you seem to know you're sql...how would I go about selecting all the matches and also selecting the associated rows from another table for both player1 and player2?

If you don't mind me asking

I know it's a join...but is it a multi join? 🤔

Hmm, I actually may know

[warm sleet]

@upbeat nymph ehh

@upbeat nymph if you have one field you are matching against, but its a one to many relationship

so you expand the result into multiple records

you can use LEFT JOIN

[upbeat nymph]

Hmmm.

I was originally gonna do inner join because that was the first example because I've honestly never written sql past a simple select, count, etc.

[warm sleet]

you can ignore the inner outer stuff

JOIN is RIGHT INNER JOIN

there's lot of semantics once you get into this

but just using LEFT JOIN should help you

say you have a list of Players, and multiple sessions per player

you can do something like

[next igloo]

Anyone have a script that makes a “gun” or other device that spawns a bullet and makes it move

In JavaScript

[warm sleet]

SELECT session.* 
FROM player 
LEFT JOIN session ON player.id = session.playerid

@upbeat nymph this would return only the sessions for that player

but you can still filter on all the fields from all tables

and even do grouping

and then do counting based on those groups

[upbeat nymph]

Well here is my results initially

[warm sleet]

I think you should normalize your database properly first

player1 sounds like a bad idea

[upbeat nymph]

It's not for real data. Just seeding fake data for benchmarking

[warm sleet]

do you have a schema?

[upbeat nymph]

Really this is all I'm doing code wise tbh

[warm sleet]

and your relationships?

[upbeat nymph]

Since I work with so many MVC frameworks, I never really had to learn the SQL. You know...I should add the relationship constraints. Forgot about that. Gimme a second

[warm sleet]

what are you attempting to do lmao. let me ask that first

[upbeat nymph]

I've had to do that many times.

[warm sleet]

you have two tables, players and matches

and you want to get what exactly?

[upbeat nymph]

Literally just seeding randomized data which can range from 1k rows to 10 million rows

Just to feed into the algorithms and benchmark calculation times

That's why "production perfect" isn't needed, but the better optimized...the better the results will be for all benchmarks

[warm sleet]

well, your algorithm speed here really depends on the way you implement this query

you can just do a regular old SELECT

on all the fields

if you have a one to many between player and matches

then just select from matches, and join players

then sort by players

or select from players, and left join matches

[upbeat nymph]

Really what matters is the matches.

because at the end...this is the benchmarking code

gimme a sec

[warm sleet]

yeah but this match you are doing

normally you'd do this in the query

:3

oh wait

derp

[upbeat nymph]

Ahh fuck

I just realized

[warm sleet]

sorry, I am just realizing its a 'match' as a game

xD

[upbeat nymph]

Each library implements it differently. So this joining is completely irrelevant D:

[warm sleet]

yeah

You can only really benchmark this

if you know exactly what you are going to running

[upbeat nymph]

So really foreign key constraint is not relevant

[warm sleet]

really, if you have just a score you want to store in the database, for 2 players in that match

you'd write a procedure

and just do CALL write_score(player1, player2, score)

[upbeat nymph]

yeah

[warm sleet]

you can resolve individual fields though

like

ohh

wait

I see

@upbeat nymph you mean to join the player table twice

for both fields

thats a bad idea like that