programming

[gritty dust]

bro that happens to me everytime I write something for science lol

at least teachers here can't use AI detectors for marking since they do false positives

[olive sable]

here they can

and they also use ai to make their test questions now

ive seent hem do it, they told me they do it

[tender river]

a person i know went to uni before chatgpt but dropped out and then enrolled again and said its now a completely different experience with much less material comprehension

[rough bloom]

using LLMs to help make questions seems fine to me
AI detectors though

[dense cosmos]

My clg experience was basically going

My brain cells are returning to me now

[real sierra]

found you

[noble zodiac]

I'm gonna lose my marbles

[real sierra]

the mystery of the broken ALU is fixed

time to re-run the right shift code

it worked

so all the problems ive been experiencing

were because of this stupid bad trace

[olive sable]

well, at least it works now

[real sierra]

im amazed it didnt cause more problems tbh

given its position tho

you would have to do an operation that enables the swap flag, and have differing values in bits 10 and 11 of the D register

[grave fractal]

finally

[real sierra]

to even have a chance of noticing it

well thats exciting

now i have right shift

i guess i implement that division code now

[grave fractal]

good luck

[real sierra]

is there a reason for returning 32767 in particular if the divisor is 0?

[grave fractal]

he just returning maximum value of signed short

as an error signal

[real sierra]

surrounded by so many experts i feel like an idiot

[grave fractal]

nah you better than me

i'm just a computer engineering student trying my best to help , while you guys are just vibin'

[real sierra]

theres an annoying quirk with this architecture

for loading literals from code, data instructions are used

these are indicated by the highest bit being 0

which means you only get to load 15 bits

if i ever want max short as a literal, i have to left shift 7FFF and add 1

agony

[haughty briar]

chat chat chat, look

[olive sable]

my brother in christ

[real sierra]

tbh for some form of rudimentary error handling

[haughty briar]

:3

[olive sable]

wtf is that?

[real sierra]

ive been considering just making labels to jump to

[real sierra]

wait what the hell

[haughty briar]

:3

[real sierra]

i expect real neuro in 6 months

good luck

[olive sable]

bro is collecting huiman body parts to make robots

[haughty briar]

calm down its the aniamtronics they usedd for harry potter

[olive sable]

so when is five night at neuros opening?

[haughty briar]

but i am making somewhat of a similar thing though

[haughty briar]

lowk

gime time

[real sierra]

i was just gonna halt if a divide-by-zero happens

with a jump to a label that looks something like

@ Exception_DivideByZero
goto Exception_DivideByZero;

not much of a "halt" but given i can follow the code while it runs, it's enough for me to see

[rigid snow]

neuro pog high definition

[olive sable]

[noble zodiac]

Im gonna commit triple fault

[real sierra]

wait what are you cooking with this condition

[olive sable]

3% in 14 hours.
i dont trust that 150 hours left for a second.
at this rate it will take 460 hours

[tender river]

hi t

[real sierra]

surely the last condition is just t < 0

[stray dragon]

hi chayleaf

[real sierra]

by like

...math

[nocturne olive]

How big was the model again?

My PC has a GPU with more compute than yours and it still took 3 days on just 150M on a fairly small dataset

[olive sable]

"model_dim": 1536
"depth": 32
"heads": 16,

[nocturne olive]

I don't know how to convert that to params

[olive sable]

me neither

[real sierra]

@sage crag

# math_div(a, b) returns the quotient a/b.
function math_div 2;
    push_arg 1;
    pop_d;
    if_else_d JEQ math_div_bad_divisor math_div_good_divisor
    @ math_div_bad_divisor
        goto Exception_DivideByZero;
    @ math_div_good_divisor
    push_arg 0;
    push_arg 0;
    pop_d;
    if_else_d JLT math_div_neg_a math_div_pos_a;
    @ math_div_neg_a
        neg;
    @ math_div_pos_a
    push_arg 1;
    push_arg 1;
    pop_d;
    if_else_d JLT math_div_neg_b math_div_pos_b;
    @ math_div_neg_b
        neg;
    @ math_div_pos_b
    pop_d
    push_d;
    push_d;
    push_value 1;
    call math_div_recursive;
    push_retval;
    push_arg 0;
    push_arg 1;
    pop_d;
    if_else_d JLT math_div_sign_flip math_div_sign_flip_end;
    @ math_div_sign_flip
        neg;
    @ math_div_sign_flip_end
    pop_d;
    if_else_d JLT math_div_q_flip math_div_q_flip_end;
    @ math_div_q_flip
        neg;
    @ math_div_q_flip_end
    return;
        
# internal backing call for math_div(a, b)
# math_div_recursive(ua, ub, t, d)
function math_div_recursive 4;
    push_arg 0;
    push_arg 2;
    push_arg 2;
    add;
    sub;
    pop_d;
    if_else_d JGT math_div_recursive_test math_div_recursive_pos;
    @ math_div_recursive_test
        push_arg 0;
        push_arg 2;
        sub;
        pop_d;
        if_else_d JGT math_div_recursive_break math_div_recursive_neg
        @ math_div_recursive_break
            push_value 0;
            return;
        @ math_div_recursive_neg
        push_arg 3;
        push_arg 0;
        sub;
        push_arg 1;
        push_arg 1;
        push_value 1;
        call math_div_recursive;
        push_retval;
        push_arg 3;
        add;
        return;
    @ math_div_recursive_pos
    push_arg 0;
    push_arg 1;
    push_arg 2;
    push_arg 2;
    add;
    push_arg 3;
    push_arg 3;
    add;
    call math_div_recursive;
    push_retval;
    return;

(sorry for wall of text)

[olive sable]

np

[real sierra]

ported your code tho

wonder if it actually works

well i hit start

its doing something for sure

not sure what

failure

i probably made an oops somewhere

[inner pike]

the hidden bitcoin miner:

[real sierra]

trying to do 15 / 3 and the result im getting is uhhhh

0xE004

[rough bloom]

should be something like ~1-2B I think depending on the other hyperparameters like vocab size, embedding size, head size, and ffn size

[olive sable]

"vocab_size": 50257
idk embed size and head size

[nocturne olive]

That's gonna be a lot of compute depending on how big the dataset is and how many epochs it's doing

[olive sable]

10 epochs

[nocturne olive]

Have fun with your way overfitted model with too much compute spent on it

With LLM pretraining usually you want to do at most 1 epoch I believe

[hoary lion]

Yes

We go less than 1

Bc it is too big

Don't pretrain more than 1

[rough bloom]

never repeat any data, it only causes overfitting
usually there's way more data available than you have compute to actually use

[olive sable]

@prime ridge

[tender river]

math_div(a, b) {
    if args[1] == 0 {
        throw;
    }
    a = args[0];
    if args[0] < 0 {
        a = -a;
    }
    b = args[1];
    if args[1] < 0 {
        b = -b;
    }
    c = b;
    d = 1;
    ret = math_div_recursive(a, b, c, d);
    a = args[0];
    if args[1] < 0 {
        a = -a;
    }
    if a < 0 {
        ret = -ret;
    }
    return ret;
}

math_div_recursive(a, b, c, d) {
    if sub(args[0], add(args[2], args[2])) > 0 {
        if args[0] - args[2] > 0 {
            return 0;
        }
        return math_div_recursive(args[3] - args[0], args[1], args[1], a);
    }
    return math_div_recursive(args[0], args[1], args[2] + args[2], args[3] + args[3]);
}

[nocturne olive]

Whoever is telling you to train that model clearly doesn't quite know how training a model should be done

[hoary lion]

I am not so sure why python libraries are trying to escape pythonic way

[tender river]

ported back to pseudocode (my translation might have errors though )

[hoary lion]

Totally not frustrated by flax

[real sierra]

chayleaf

[hoary lion]

I can't even use print() easily 😭

[rough bloom]

you should try Equinox

[hoary lion]

See tbh

Everything is because of jax actually

JIT

[rough bloom]

a JIT is necessary though, both because Python is kinda slow and because that way Tensor operations can be fused and launched more efficiently

[hoary lion]

I think there is this fundamental tradeoff between easily debuggable code and performant code

[rough bloom]

I don't think that has to exist, jax JIT-compiled code and normal eagerly evaluated code look pretty much the same (other than the static shape restriction, but that doesn't affect debugging, it just causes pain getting stuff to work at all)

current ML frameworks are just too shit

[real sierra]

oh i misread konii's code

need to add d

maybe it works this time

or not...

[rough bloom]

[real sierra]

looks like it got 13 as an answer

[hoary lion]

Or maybe it's just python

[real sierra]

to 15/3

[hoary lion]

Python might be a bad language for ml

[real sierra]

it gets 15/15 right

its ok, i believe in your code konii

i probably just ported it wrong

why is it wrong tho

it looks right

[hoary lion]

Maybe I should start a mini educative stuff

Like tensor manipulation 101

[real sierra]

found another bug

its working

https://gyazo.com/ed0383db7a20b3ceb4c51826393b774f.mp4

its actually very impressive

thank you for the help

do you wanna help me devise square root

i was thinking about it earlier and i feel like theres a shortcut somewhere

because

log2 is easy* with binary
-# *precision not guaranteed

so i think you can just

log2(x^(1/2))
= (1/2)log2(x)

= shr(msb(x))

so like

i think if you just do 2^that

you get something resembling a square root?

but im not sure on the accuracy

log2 might be too imprecise

in fact log2 is only one of 16 possible values isnt it

bin search time

ALU supports add, sub, inc, dec, and, or, xor, not

just added code for n-bit shifts to left/right

left, by far

do you want to review it

if not then it's the only algorithm, which makes it optimal

anyone else using this chat

# math_shl(x) returns x left-shifted by one position.
function math_shl 1;
    push_arg 0;
    push_arg 0;
    add;
    return;
    
# math_shln(x, n) returns x left-shifted by n positions.
function math_shln 2;
    push_arg 1;
    pop_d;
    if_else_d JEQ math_shln_done math_shln_go;
    @ math_shln_done
        push_arg 0;
        return;
    @ math_shln_go
    push_arg 0;
    push_value 1;
    push_arg 1;
    sub;
    call math_shln;
    push_retval;
    push_retval;
    add;
    return;

# math_shr(x) returns x right-shifted by one position.
function math_shr 1;
    push_arg 0;
    push_value 1;
    push_value 2;
    call math_shr_recursive;
    push_retval;
    return;
    
# math_shrn(x, n) returns x right-shifted by n positions.
function math_shrn 2;
    push_arg 0;
    push_value 1;
    push_value 1;
    push_arg 1;
    call math_shln;
    push_retval;
    call math_shr_recursive;
    push_retval;
    return;

# internal backing call for math_shr(x)
# math_shr(v, mask_out, mask_in) performs a right shift by copying bits using masks.
function math_shr_recursive 3;
    push_arg 2;
    pop_d;
    if_else_d JGE math_shr_recursive_if math_shr_recursive_else
    @ math_shr_recursive_if
        push_arg 0;
        push_arg 1;
        push_arg 1;
        add;
        push_arg 2;
        push_arg 2;
        add;
        call math_shr_recursive;
        push_retval;
        goto math_shr_recursive_end;
    @ math_shr_recursive_else
        push_value 0;
        goto math_shr_recursive_end;
    @ math_shr_recursive_end
    push_arg 0;
    push_arg 2;
    and;
    pop_d;
    if_else_d JNE math_shr_recursive_shift math_shr_recursive_ret
    @ math_shr_recursive_shift
        push_arg 1;
        or;
    @ math_shr_recursive_ret
    return;

yeah thats what i have

it cant be that easy

surely

ok let me port this

what's the i stand for

i see

like, loading a literal into memory somewhere?

literals can be put into A register in one instruction
it takes 7 instructions to push a literal onto the stack

the stack is all software too unfortunately

still porting

wtf it works

your code was so nice and concise

i feel like my port doesnt do it justice

# math_sqrt(x) returns the square root of x.
function math_sqrt 1;
    push_arg 0;
    push_value 1;
    push_value 0;
    call math_sqrt_recursive;
    push_retval;
    return;
    
# internal backing call for math_sqrt(x)
# math_sqrt_recursive(n, odd, count)
function math_sqrt_recursive 3;
    push_arg 0;
    push_arg 1;
    sub;
    pop_d;
    if_else_d JGT math_sqrt_recursive_break math_sqrt_recursive_go;
    @ math_sqrt_recursive_break
        push_arg 2;
        return;
    @ math_sqrt_recursive_go
    push_arg 1;
    push_arg 0;
    sub;
    push_arg 1;
    pop_d;
    LD INC_D;
    LD INC_D;
    push_d;
    push_arg 2;
    pop_d;
    LD INC_D;
    push_d;
    call math_sqrt_recursive;
    push_retval;
    return;

even looking at this, i cant figure out how its working

oh

i see it now

clever

# math_shr(x) returns x right-shifted by one position.
function math_shr 1;
    push_arg 0;
    push_value 1;
    push_value 2;
    call math_shr_recursive;
    push_retval;
    return;
    
# math_shrn(x, n) returns x right-shifted by n positions.
function math_shrn 2;
    push_arg 0;
    push_value 1;
    push_value 1;
    push_arg 1;
    call math_shln;
    push_retval;
    call math_shr_recursive;
    push_retval;
    return;

# internal backing call for math_shr(x)
# math_shr(v, mask_out, mask_in) performs a right shift by copying bits using masks.
function math_shr_recursive 3;
    push_arg 2;
    pop_d;
    if_else_d JGE math_shr_recursive_if math_shr_recursive_else
    @ math_shr_recursive_if
        push_arg 0;
        push_arg 1;
        push_arg 1;
        add;
        push_arg 2;
        push_arg 2;
        add;
        call math_shr_recursive;
        push_retval;
        goto math_shr_recursive_end;
    @ math_shr_recursive_else
        push_value 0;
        goto math_shr_recursive_end;
    @ math_shr_recursive_end
    push_arg 0;
    push_arg 2;
    and;
    pop_d;
    if_else_d JNE math_shr_recursive_shift math_shr_recursive_ret
    @ math_shr_recursive_shift
        push_arg 1;
        or;
    @ math_shr_recursive_ret
    return;

it just goes thru the high bits of the input one bit at a time

and copies it to the output

crazy how bad shr is via software when its literally just a few wires in hardware

division is definitely the largest function here

its worse than sqrt somehow

im amazed any of these work honestly

i was kinda expecting to be working on math functions for ages

so i didnt plan what i wanted to do with them

doom

[trim valve]

[prime ridge]

That training doesn't have enough data for a model that size

[real sierra]

surely doom fits in my 16 bits of addressable memory

[prime ridge]

I'm still zipping all the new data

idk why it's taking so long

[olive sable]

i was just testing on my side

looks like batch size 16 is fine

ran for 15 hours without going over 24GB vram

[real sierra]

on this instruction set?

[old totem]

chat how do you feel

[real sierra]

how do YOU feel

[olive sable]

i stopped it at step 59125
btw, what about the epochs? they said 10 is too much and it should be 1 or less

[old totem]

hello shiro

long time no see

[prime ridge]

should be 1

[old totem]

brain blast

[prime ridge]

don't train it rn

[olive sable]

how do i feel about what?

[prime ridge]

ur wasting gpu time

[old totem]

type shit fr

[real sierra]

always nice to see you :>

[prime ridge]

that's the old code

[olive sable]

ye i stopped it already

you've changed the code again?

[prime ridge]

The data is still zipping. It's so slow

it's been zipping for almost like 5 days

it's zipping like 5 files a second

[old totem]

i might fall in ths rabbit hole i should NOT get used to

[nocturne olive]

Concatenate the data files and zip it in like an hour instead
You'll save way more time than you'd lose by stopping it now

[prime ridge]

If sam was on linux i'd just do a tar

woulda taken like a single day max

but .zip is so horrible

[nocturne olive]

Then use 7z or something?

[prime ridge]

I don't have 7z

also IO is super slow because it's on an external usb drive

[nocturne olive]

Also the 7zip tool can basically handle almost all packed formats

[real sierra]

mf chief() {
  mf ts be cap rn
  ts fr vibin fr yikes rn
  sussin (ts finna cap) {
    ts deadass ongod rn
  }
  yeet rn
}

[nocturne olive]

Just throw it into a tar if you want, Sam can probably easily unpack it

[prime ridge]

alr bet

[nocturne olive]

But also just don't make your dataset like 30 billion small files
Concatenate them

[olive sable]

7 zip can extract tar no?

[prime ridge]

bit late for tha tlol

[nocturne olive]

It can extract basically anything

[tender river]

ideally you'd implement it like this

function sqrt 1;
push_value 0
push_value 1
push_arg 0
push_value 1
sub
pop_d
if jge @loop @exit
@loop
  push_value 2
  add
  ; ??? somehow increment second last value on stack by 1
  ???????
  push_d
  ; ?????? i need to push the second last value on stack to stack again how do i do it
  ??????
  jmp @loop
@exit
pop_d
return

[real sierra]

but i thought i was the best shr

[nocturne olive]

Why would it be? You can just Python script to read all the contents into a single file no?

[prime ridge]

I already have like 100 million files in a single directory

[olive sable]

100 million files is def gonna take a couple hours to extracts, maybe even a full day

could you remind me how much storage i need?

[real sierra]

new sqrt dropped

[trim valve]

I have made this mistake before and may or may not have output a single 500MB json file

[prime ridge]

I mean I cancled it

[prime ridge]

120gb

most of that is file metadata

[olive sable]

oh, thats way less than i was expecting

[tender river]

its old

[prime ridge]

like 80% of it is metadata lmfao

[real sierra]

; ?????? i need to push the second last value on stack to stack again how do i do it

https://tenor.com/view/thats-the-neat-part-you-dont-invincible-gif-27194608

[tender river]

can a function return multiple values on stack or would that mess up the return code

[real sierra]

the way it's written now, only single returns

[prime ridge]

atp I might just have to retokenize 😭

ughhhhh

[trim valve]

tbf 100m files shouldn't be that bad to process into smaller files

[tender river]

makes sense you use recursion everywhere its the only way to have more than a couple variables

[real sierra]

recursion my beloved

[trim valve]

I mean the number 100 million and python should rarely be in the same sentence

[real sierra]

stack frames? you mean variables?

[trim valve]

but still should be dooable

[prime ridge]

not even python is the issue

the IO is insanely slow

it's on an archive external drive

15 mb/s writes

[nocturne olive]

Easy:

File("out.txt").bufferedWriter().use { writer ->
  for (file in (File("path/to/files").listFiles())!!) {
    writer.write(file.readText())
  }
}

[trim valve]

[tender river]

you wont need it if you implement arbitrary stack offsets or multiple return at least

[real sierra]

is there a quicker way to do squaring than multiplication

[trim valve]

is this like trying hard to max out i/o speeds

[prime ridge]

The kernel is having a heart attack and the drive is on it's death bed

[real sierra]

i think malloc is probably in my future somewhere

then you can return a pointer to a struct

[trim valve]

[tender river]

malloc is useless

[real sierra]

i mean

[nocturne olive]

This is all the code required to read the contents of all files in a directory and write them all into a new file

[real sierra]

im more just looking for the notion of abstracting away ram management

then my code can just say "yeah i need this much memory for my crap"

[tender river]

its called stack

[prime ridge]

Yeah but how would the code deliminate new files?

[real sierra]

the stack

[prime ridge]

it's not text or anything it's a .npz file of integers

[real sierra]

chayleaf please i want malloc

[trim valve]

hm

[real sierra]

it will give you multiple return

albeit indirectly

[nocturne olive]

You can just add a prefix/postfix to the write command

[tender river]

❌ no malloc allowed

malloc bad

[real sierra]

[trim valve]

honestly I mildly want to suggest sqlite but I don't remember if that can efficiently handle binary data

[real sierra]

@sage crag write malloc

[tender river]

i wrote a garbage collector recently

[real sierra]

ill just call part of ram the heap

[prime ridge]

also can't be read text since it's binary

[real sierra]

its fine

[nocturne olive]

Then read binary and add delimiting bytes where you need them

[real sierra]

other than the stack, heap, and a tiny address range for my own stuff, nobody else should be touching ram

so it should be fine

[trim valve]

otherwise diy a binary archive format that's like ```
uint(total length),uint(metadata length),bytearray<metadata>,bytearray(data)

[real sierra]

[prime ridge]

nah but like the code needs to load these files independantly anyways

[nocturne olive]

Why? It seems like your approach completely and utterly sucks

[real sierra]

i cant do garbage collection

[prime ridge]

can't just load 120gb into vram 😭

[trim valve]

??

[prime ridge]

it'll work but it's slow asf

[real sierra]

my performance will never survive

its malloc or nothing

[tender river]

malloc is just a worse version of garbage collection

[nocturne olive]

Most of that is probably filesystem overhead instead of actual data if most files are under 4KB

[prime ridge]

atp shipping the drive internationally seems like the move

[trim valve]

apollo can I just double check my understanding of the situation rq

[prime ridge]

it's 16 billion tokens

[trim valve]

you have:

• 100m tiny files
• stored on a slow drive
  you want:
• some easy way to transfer all of these files to sam

[tender river]

its like GC except you dont know what the user code will do so you have to do all sorts of tricks to properly allocate and deallocate memory

[nocturne olive]

And is each token a single Int32?

[prime ridge]

yeah

so a lotttt of overhead

I think it's actually more like 200 or 300 on the drive

it takes a long time to even check

[tender river]

just have two stacks, one for functions and one for values, that should solve any problems with not being able to use more than one variable, you wont even need an args pointer anymore

[trim valve]

if you want me to try throwing to throw together a terrible rust program with far too much multithreading I'd be happy to give it a shot if you can give me an output format

[prime ridge]

I cancelled the check at like 120 gb

[nocturne olive]

Then you'd have an exceptionally easy time concatenating them all into a single big binary file while keeping separation, just do

collection of chunk size 4-byte Int32s
repeat for every file```

[prime ridge]

yeah

[trim valve]

yeah then unless you just give sam a disk image of the drive you're gonna have to cope with it taking a while to do loads of small reads

is it a flash drive or spinning metal

[prime ridge]

loading 100m files will still take a while but I guess I gotta just bite the bullet atp

[olive sable]

on my side everything should be fine, nvme ssd and stuff.
even if there is not enough space there i also have 2 empty sata ssd's

[prime ridge]

it's an external usb drive

[prime ridge]

nah ur golden dw

it isnt even that much data tbh

im just a bit stupid

mostly lazy actually

[nocturne olive]

Making your dataset 100 million small binary files rather than one big raw text file and including the tokenizer in your trainer code seems stupid

[real sierra]

two stacks

[prime ridge]

I mean it was a lot easier to implement at the time

[trim valve]

everyone does stupid things, you have to learn somehow lol

[prime ridge]

not a lot easier

but slightly

[nocturne olive]

Just send the raw data and tokenize it at the target

[tender river]

you can have one grow from the top of the addressable memory and one from the bottom

[prime ridge]

I will I will

[real sierra]

heap heap heap heap

[trim valve]

this offer still stands

[prime ridge]

I know the int32 values are 0-50000 inclusive

[real sierra]

[trim valve]

oh damn you could use u16s then

[prime ridge]

so ig I just use 4294967296 as the delim?

[real sierra]

pov: stacks fighting over 0x0FFF

[trim valve]

half the filesize

[olive sable]

my wifi is

[prime ridge]

I could indeed

[nocturne olive]

Yeah, that'd work if you know the maximum allocated range of the actual data

[prime ridge]

Can same use gzip?

[hoary lion]

Bwaa

[trim valve]

probably

[olive sable]

probably

[prime ridge]

sam

[trim valve]

7zip handles most stuff

[prime ridge]

alr cool

fuck it gimme 10 min

[nocturne olive]

Yeah, it's a great software

[olive sable]

bwaa

[prime ridge]

oh it's only 8 million files

we chillin

I forgot I changed ctx

[tender river]

if you have two stacks you can implement forth
https://www.forth.com/wp-content/uploads/2018/11/thinking-forth-color.pdf

[real sierra]

oh god

[olive sable]

rip 87.5% of the dataset

[real sierra]

instant headache upon seeing the typeface

[tender river]

you havent seen https://www.forth.com/wp-content/uploads/2018/01/Starting-FORTH.pdf

[prime ridge]

no no no it's fine. It's still 16 billion tokens

that's enough to train comfortably

[olive sable]

ah ok

[trim valve]

the slowest part of this was saving the json somehow, not actually opening the files

[prime ridge]

just a larger ctx

[real sierra]

what the hell

im not reading all that

[prime ridge]

oh hellll no

[real sierra]

malloc()

[prime ridge]

that's not even bad

[real sierra]

free()

realloc()

perfect

[prime ridge]

one of my json files was 50gb

[trim valve]

smh

[prime ridge]

90% was wasted metadata

[rare bramble]

how does a json file get that big

[prime ridge]

it was from the dataset so not even my fault

[hoary lion]

Io time

[olive sable]

as long as it doesnt have to train for more than a month im good.
im going on vacation again on the 28th and i think it would be nice if it finished by then

[tender river]

surely a memory allocator is like a couple hundred lines of assembly code

[trim valve]

people using json because its easy to use

[prime ridge]

realistically a max of 2 weeks

[olive sable]

[trim valve]

without thinking about the problems it brings

[nocturne olive]

Welp, gotta recalculate the batch size again

[tender river]

and not a couple tens of thousands of C code

[prime ridge]

if it takes longer just save the checkpoint and i'll finish it

[real sierra]

the rom and ram are separate 16-bit addressable spaces, so no worries there

[real sierra]

ok maybe minor concern

[prime ridge]

I had to anyway cuz I implemented latent attention and flash attention

[real sierra]

MALLOC

[prime ridge]

oh yeah btw @olive sable u need to compile flash attention v2

[olive sable]

okay

[rare bramble]

people should learn to use compressed data formats

[tender river]

steal memory from the host PC and attach it to the virtual one

[prime ridge]

I'll find how to do that after I start fixing the ds

[trim valve]

nah just build faster pcs with more memory

that solves all issues

[real sierra]

even rocks would have syscalls

[tender river]

anyway forth is literally the simplest high level language and it doesnt require an allocator (trust me allocators are hard)

[nocturne olive]

This whole LLM pretraining venture seems quite pointless
There's like 30 thousand LLMs you can just prompt to get whatever you want with much higher quality than any home-made LLM can, LLMs take way longer to train than even NeuroSynth, and LLMs are honestly way less interesting than NeuroSynth

[prime ridge]

Not pointless

[tender river]

you dont even have an allocator just an arena

[olive sable]

If your machine has less than 96GB of RAM and lots of CPU cores, ninja might run too many parallel compilation jobs that could exhaust the amount of RAM. To limit the number of parallel compilation jobs, you can set the environment variable MAX_JOBS:
hey thats me, i have 16 cores and less than 96GB of ram

[nocturne olive]

Well, what's the point then?

[prime ridge]

Nobody wants a cringy LoRA tuned llm

[tender river]

well it is an allocator

but it doesnt have free

[rare bramble]

you dont need an allocator, just write to random memory addesses and assume that no one else is using them at the time

[real sierra]

it looks really weird

[nocturne olive]

What's wrong with those?

[real sierra]

[prime ridge]

just set the workers to a normal amount

[prime ridge]

Extremely inefficient

[trim valve]

surely you're not going to use more than 6GB of ram per job

[prime ridge]

requires instruction tuning and more parameters

You aren't going to see a 0.5b model outperform larger models without pretraining

[nocturne olive]

Whar
It's wayyyy more efficient than home-pretraining a tiny model

[olive sable]

tbhi haver no clue how to compile flash atttention V2.
cant i just pip install?

[prime ridge]

not per parameter

[prime ridge]

Yes

pip will compile it for you

lemme find the package because I think it's specific

[hoary lion]

I think he is talking about the vram

@nocturne olive

[nocturne olive]

Yeah, of course not
But you can just grab an off-the-shelf pretrained model and finetune that

[prime ridge]

git clone https://github.com/Dao-AILab/flash-attention.git
cd flash-attention
pip install .

[prime ridge]

have fun running that locally 🤡

if you want a good local model. It has to be custom

[nocturne olive]

Llama 1B
Llama 8B
Llama 13B

[prime ridge]

they suck

I already tried it

especially 1b

[nocturne olive]

A home-made model is gonna suck even more

[prime ridge]

Nope

[olive sable]

[hoary lion]

@prime ridge there is a repo to convert any GQA to MLA
Use qwen 3B ish model and finetune it after converting it to MLA

[rare bramble]

[hoary lion]

Classic

[prime ridge]

u need pytorch

[hoary lion]

We hate torch dependency

[olive sable]

i though i had pytorch

[hard shale]

maybe in another py-env

[prime ridge]

yeah

[nocturne olive]

And how would you know that your tiny model trained on amateur collected data can beat a model made by a big company with loads of data scientists cleaning up the dataset?

[prime ridge]

sooooo many parameters are completely wasted

it doesn't need to know how to solve differential equations to chat like a human

[hoary lion]

What

[prime ridge]

literally burning compute

[olive sable]

cuda 11.8 for 3090?

[trim valve]

but I want my ai girlfriend to do my homework whilst rping !!!

[nocturne olive]

CUDA 12 is recommended

CUDA 12.1 is my go-to usually

[olive sable]

12.6 or 12.8?

[prime ridge]

yeah should be cuda 12

[hoary lion]

Yeah ofc it will lost some but it is better cram

[nocturne olive]

Doesn't matter, it's backwards compatible anyway I think

[hoary lion]

Much efficient than starting from scratch imo

[nocturne olive]

Though check what CUDA Torch is available for

[hoary lion]

12.6 is stable

[olive sable]

these are the options

[prime ridge]

Duh. But once it's trained it will have exceptionally low memory usage while having the performance of models orders of magnitude better

[hoary lion]

Okay lol good luck then

[prime ridge]

Don't select linux

[olive sable]

ye

fuck it, 12.6 it is

[nocturne olive]

"while having the performance of models orders of magnitude better"

[hoary lion]

Tbh neither do I know the architecture, so can't really confirm "it would be magnitude better" in both perf and compute/mem

[prime ridge]

Idk how being trained on wikipedia is supposed to improve the accuracy of a model meant to sound human

that's just not true

garbage in garbage out

[olive sable]

i dont mind training it anyways, lets j8ust do it

[prime ridge]

exactly

[nocturne olive]

Sure, it may require less compute and memory, but it'll overall not be better if the data is not significantly well curated

[hoary lion]

in your dreams oof
Can you learn english with discord messages, thats my question here

[prime ridge]

It will learn to imitate the data

that's how ML works

[nocturne olive]

If just I had a 3090 like you so I could just train NeuroSynth models whenever I feel like without it taking 30 hours

[hoary lion]

I think wiki is just for priming basic knowledge, being coherent

[nocturne olive]

Here's my prediction:
the model will output English words, but without any sense behind them, and be unable to properly keep a conversation

[hoary lion]

Imagine a super brainrotted llm spamming non existent emoji

[trim valve]

me :3

[hoary lion]

Us core

[nocturne olive]

-# If I'm right, that'd be very silly

[prime ridge]

who cares. I'd be hillarious anyways

plus I doubt it

a lot of the data comes from "smart people" servers too

[nocturne olive]

Is your data in any way grammatically consistent?

[prime ridge]

prolly 50%

[prime ridge]

somewhat

it isnt an english major. It will sound natural

[hoary lion]

Nahh it's done for
50%

[prime ridge]

it's like 70% brainrot and 30% coherent

[nocturne olive]

By the way, if you're training off of Discord messages, you're breaking the Discord TOS and you should consider not doing that so you don't have the same happen to you as happened to Shapes

[prime ridge]

worst case

[prime ridge]

All I see are numbers 0-50,000

doesn't seem like discord to me

[nocturne olive]

What matters is what the numbers are derived from

[prime ridge]

probably math

[nocturne olive]

If it's numbers generated by putting Discord messages trough math formulas, you're in trouble

[real sierra]

doom seems hard to fit in this computer

[prime ridge]

no ofc not

[real sierra]

but bad apple could be possible...

[trim valve]

apollo you just have a very lucky random number generator right

[nocturne olive]

Surely
-# Well, have fun with the Discord lawyers

[hoary lion]

Return back to slack

[prime ridge]

just rng bud

[olive sable]

Ah yes, it is but it isnt the same

[hard shale]

pseudo rngs are very useful

[prime ridge]

https://tenor.com/view/ahoklollmao-noooo-nooo-noo-no-gif-17139315586294057944

[trim valve]

rngs power my bogosort

still need to optimise that bad boy

[hoary lion]

I hate manually managed rngs

[prime ridge]

Bad apple is a gift from the gods im convinced

[trim valve]

poor thing obliterates my cpu

[hoary lion]

Jax you again

[trim valve]

and still runs awfully for some reason

you would think

but alas

[rough bloom]

jax.random.split

[prime ridge]

no need for llm. Just stream the rng

it's already accurate

[trim valve]

I think I tried that and it was the same speed

it was fast enough

[hoary lion]

Love when Shuni randomly spawns as i mention jax/flax

Who would be spawning if i mention tf

[trim valve]

lemme check the speed rq

[prime ridge]

DAMN it's fast

[trim valve]

it is

[olive sable]

my brother in christ, pytorch ***is *** installed

[prime ridge]

90 min to encode the entire dataset of psudo random numbers

weird

[real sierra]

i heard bogosort

[prime ridge]

worst case it's not required but makes it like 3-4x faster

[hexed grove]

yall i need some advice

[real sierra]

are there any other important math functions im missing

[prime ridge]

check pins

[real sierra]

pow might be nice

any bright ideas on how to do pow efficiently

log_n

surely just

[burnt aurora]

is adding arduino libraries supposed to take 5min and not be done

[real sierra]

log2(x) / log2(n)

precision

[prime ridge]

my build times can be up to 10 min for arduino sometimes

[trim valve]

my beloved

[prime ridge]

depends on the code

[trim valve]

I love how it slowly but surely drops from 9bn/s to 8bn/s as my cpu cooks itself

[prime ridge]

@olive sable it's tokenizing and should be finished in 90-120 min. I gotta finish my paper just ping me if u need anything

[hexed grove]

i need to integrate live2d into my c code, live2d doesnt have C bindings what do i doooo

[olive sable]

[trim valve]

uh wait how hard is a 16 item list again

[nocturne olive]

Make C bindings

[stark needle]

it improves generalization

[trim valve]

ah ok that is nonideal actually

[hexed grove]

how hard should that be (i do not know c++ )

[trim valve]

wait does someone here remember how to do basic stats

I forgot all of it

[real sierra]

nope

[nocturne olive]

I don't know, depends on the complexity of things you're making bindings for

[trim valve]

if I have a 1 / 16! chance of getting my list in the correct order, and I'm doing 8 billion attempts per second what's the average time to have a 90% chance of the list being sorted

[real sierra]

i have mul

but its just repeated addition

i dont think i can get much better

[trim valve]

I swear I used to know how to do this

[stark needle]

this is so real

[trim valve]

I would say you can use the standard normal distribution or something but

[nocturne olive]

Silly

[prime ridge]

Nobody likes yet another derivation of GPT. Enough said

that's super boring

or Llama

[olive sable]

no module named torch bwaaaaa

[nocturne olive]

LLMs in general are pretty boring

[trim valve]

sam how are you installing torch?

[nocturne olive]

Now vocal synthesis, that's where it's at

[prime ridge]

it's fine. You don't need it. It would just make it a bit faster

[trim valve]

I personally prefer eating rocks

[olive sable]

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126

[trim valve]

hm

try pip freeze I'm curious

[prime ridge]

are you in a env btw

[olive sable]

nope

[hoary lion]

See we have a man of culture

[olive sable]

i dont really use envs all that often

[trim valve]

do I leave my desktop attempting to bogosort 16 numbers or do I let it rest

[real sierra]

i dont suppose those bit hacks require shr or shl

[nocturne olive]

This is the peak of silliness

[trim valve]

honestly every time I see ai voice stuff it reminds me I need to train voice irl

get back on the grind

[stark needle]

uh isn't that what ur doing basically

[prime ridge]

not at all?

[noble zodiac]

wish I could sing but my vocal coords are fricked up

[trim valve]

😔

with effort they can get a bit better

[stark needle]

how is it different? From my understanding it's also a transformer

[prime ridge]

transformer != llama

[stark needle]

llama is a transformer

[prime ridge]

Ok surely this is bait 🪤

[olive sable]

i have no clue what is happening nor what im doing

imma watch some anime

[noble zodiac]

a bit aint enough to stop ears from bleeding. Ah well, it do be what it do be

[tender river]

forth is a lovely language
you should learn it

: SQRT
    0 SWAP 1
    BEGIN
        2DUP >= WHILE
        DUP 2 +
        ROT ROT -
        ROT 1 +
        ROT ROT
        SWAP
    REPEAT
    DROP DROP ;

25 SQRT CR .

[prime ridge]

@olive sable yeah ur good we can just wait for the encoding to finish

[stark needle]

llama is literally the name pf The Architecture, which at the end doesn't matter that much

[trim valve]

i mean

you gotta start somewhere

[noble zodiac]

I rather play instruments than fight with biology

[tender river]

otherwise, you will forever be remembered by me as the java programmer

[real sierra]

stacrobatics

[prime ridge]

llama is an LLM developed by Meta not an architecture

[trim valve]

why not both

[hoary lion]

It is???

[noble zodiac]

there are only so many hours in the day

[hoary lion]

Have you not heard of llamafy

[stark needle]

ur referring to the llama weights

[trim valve]

yeah fair

[nocturne olive]

Meanwhile me, I'm quite good at making vocals, but suck at instrumentation

[noble zodiac]

that has nothing to do with singing

[prime ridge]

yeah. LLaMA != Llama

[trim valve]

I can sing really well in my head

[nocturne olive]

Technically it does
I know how to make the computer sing pretty decently

[trim valve]

god bless the discord emote autocomplete

[trim valve]

ok but

like ur throat isn't a computer

[signal trout]

real

[trim valve]

you have to put in effort

[prime ridge]

either way it's not just a fine tuned model

[trim valve]

instead of letting your rock think extra hard

[signal trout]

both require effort, and both are valuable skills

[nocturne olive]

I like when my rock gets to think real hard about singing

[trim valve]

yeah sure but not comparable skills

[signal trout]

I couldn't imagine doing what superbox does; very impressive stuff

[trim valve]

good

[nocturne olive]

We're only just leading Neuro singing voice cloning and exploring completely untapped potential in the field

[olive sable]

https://tenor.com/view/llama-animal-gif-27259607

llama

[stark needle]

i'm open to agreeing but I'm still not quite understanding the exact point of not pretraining and instead working directly with chat data and what makes this exactly better than just fine-tuning

Is there any documentation that ya got?

[signal trout]

"only" looking pretty impressive

[noble zodiac]

agreed, completly different things

[real sierra]

there must be some better way to do powers tho

surely

[hoary lion]

help

[nocturne olive]

Silly
Either way, NeuroSynth is gonna be good
Did you see BETA-3?

[hoary lion]

my pc has less than 8 gb of space

its dying

[trim valve]

😭

[signal trout]

I don't think so

[tender river]

binpow is the fastest you can get with software

[trim valve]

so glad I overspecced my storage

[nocturne olive]

Delete any big games you have installed
And if you have Unity you can free up like 50GB by getting rid of it

[hoary lion]

I already did 😭

today is the day my pc is going to be stacked

[nocturne olive]

Here's some NS-B-3 highly tuned action

[prime ridge]

A model with a single intended purpose that has been pretrained for that purpose will outperform a model trained on irrelevant data of the same size. It's the classic example of "garbage in, garbage out".

Training data is the #1 most important factor of model performance

Just look at how Llama 4 seemed good on llmarena despite being bad elsewhere since it was made for that task

literally the #1 principle in ML

[tender river]

something like this surey i didnt make a mistake

int pow(int a, int b) {
    if (b == 0) return 1;
    int ret = pow(a, b / 2);
    ret *= ret;
    if (b % 2 == 1) ret *= a;
    return ret;
}

[maiden geyser]

what's next, zaporozhian cossack division?

[real sierra]

shr1 recurses 15 times not sure the mul savings are worth

[tender river]

we wrote the same algo so it must be correct

[prime ridge]

[real sierra]

this is strangely very relevant to the code im writing rn

[stark needle]

but it's not directly irrelevant data, when the way grammar works, etc is similar. You may be thinking of the weaknesses of instruction tuned models, those sure yea they got problems but on base models you don't necessarily have that problem

[signal trout]

seems promising

[hoary lion]

wait

who TF installed games at my c drive

[trim valve]

me

duh

[hexed grove]

i sshed in

whoops

[nocturne olive]

Still running on synthetic data by the way, NeuroSynth-1.0 will be using oragnic data

[hoary lion]

😠 istg i definitely set it to other directory than c on my steam

[prime ridge]

I am GRPO tuning the base model afterwards. Also, just to be so serious, grammar does not matter almost at all in my model...

[signal trout]

what do you mean by "synthetic" data?

[nocturne olive]

Data made with Neuro RVC and existing compatible datasets

[hoary lion]

migrating my limbus company to D or E/F

byebye

[real sierra]

this one i do know

think you're missing some parentheses tho

[tender river]

POW :
    DUP 0 =
    IF DROP 1 ELSE
        2DUP
        2 /
        POW
        DUP *
        ROT
        2 % 1 = IF * ELSE DROP THEN
    THEN ;

untested

[stark needle]

But grpo assumes the model output is good and you got a high quality autorater on which the model doesn't hack the rewards

[real sierra]

i would still never write it without parentheses, it scares me

[tender river]

also binpow is trivial to translate to an iterative approach too

[prime ridge]

Why are you so insistant that the model will be bad? I literally already trained it at 30m parameters and it was "eh". Considering the fact that it was only 30m parameters and at least could form sentences is extremely telling don't you think? At a higher scale it will be significantly better

[hoary lion]

bc

grpo is for like

super big big model yk

like frontier

unmatchable

[prime ridge]

GRPO is just RL. It don't need to be big. I'm not using it for test time compute

[hoary lion]

and you are objectively wrong with using GRPO on heavily non-factual stuffs

it does not work

how are you going to measure the perf? ELO or something? then we back to RLHF

[prime ridge]

GRPO is for tool usage. Specifically, actions taken to avoid being detected as an AI

im not using it as a replacement for fine tuning

specific words might indicate doubt of human interaction

[nocturne olive]

Tool use on such a tiny model? That's gonna go horribly

[prime ridge]

im not 100% set on GRPO btw. Just an experiment

holy fuck

let a man experiment

I am curious

[hoary lion]

...

[prime ridge]

I know the base model will be good but idk about fine tuning

[stark needle]

10M model overfit on tinystories has shown it to have barely cohesive sentences even on extremely simplified and perfect data. The model was also clueless with anything remotely outside it's exact distribution

[prime ridge]

where is 10m coming from

[hoary lion]

tinystories

its the model name

ig

[trim valve]

is it really an ai discussion in #programming if people don't try to tell you 1000 reasons why you're wrong and a terrible person

[prime ridge]

ofc 30m was overfitted

[prime ridge]

on god

like damn bro. let a man live

[hoary lion]

we just don't want gpu to melt for things that won't work

give me the 3090!! ill change the world trust

[trim valve]

why do you care

[nocturne olive]

Nah, give me 3090 and I'll make the best Neuro vocal synthesizer ever

[hoary lion]

ooh wait

we have karaoke

ima listen

[prime ridge]

Reguardless im still going to learn how to actually make an LLM instead of just yoinking a pretrained model

[hoary lion]

fk im lagging

[stark needle]

fair enough but only on conversational data is objectively a poor idea that even the Lamda google model was still like 60% web and only 40% conversational which is alr even an insane ratio

[olive sable]

$800

take it or leave it

[olive sable]

$750

[trim valve]

how about $1

[olive sable]

[stark needle]

for 800$ u bet i take it😭😭

[nocturne olive]

I don't have that much money (if I did I would definitely get that)

[olive sable]

for $1 i could give you a singular core

[hoary lion]

its a steal btw

[trim valve]

can I take one of the capacitors from the board

DISCORD

[stark needle]

???

[trim valve]

PLEASE

[olive sable]

damn, are 3090's that exxpensive now?

i paid 675 for this

[stark needle]

1.1k used here

[olive sable]

[stark needle]

It's a fucking scam

[nocturne olive]

Cheapest I've seen in Finland is 650€, but more usually they're in the 700€ range

[trim valve]

powercolour please can you release the 9070xt reaper I'm getting desperate

[olive sable]

my entire pc used was 1.3K

[rigid snow]

i don't understand why one wouldn't rent gpu compute when it isn't that expensive

[hoary lion]

brother what

3100 canadian dollars

[stark needle]

1999$ after scalper pricing

[hoary lion]

2000+ USD

[nocturne olive]

I need it consistently, for me it'd almost certainly be cheaper overall to just own the hardware

[stark needle]

Or something 😭😭

[trim valve]

¯_(ツ)_/¯

[nocturne olive]

-# Also I hate cloud stuff

[stark needle]

Apparently

[trim valve]

doubt it'd be that much

but also I have zero idea how monopoly dollars translate to gbp

[hoary lion]

the lowest bid in EBAY is 840 USD rn

[stark needle]

China got access to western 5090s somehow so now they are extremely stacking themselves with all 5090s on the market

[hoary lion]

like what 😭 😭

[trim valve]

is this for a 3090 or the card I was talking about

[hoary lion]

3090

what card?

[olive sable]

They really arent that expensive here

[hoary lion]

nahh

send me there rn

[trim valve]

ok nvm me then I'm just lost in this mess of a chat

[rigid snow]

how many hours of training per day do you realistically need, even if you iterate super fast

[stark needle]

Cheapest

[trim valve]

but pre orders are not exactly how I want to live my life

[nocturne olive]

Well, I trained NeuroSynth-BETA-3 for 30 hours straight, and I'll have to soon do it again for a special append for BETA-3.1

[stark needle]

I wonder if there's somehow an underground dark web network for china illegal gpu trafficking😭😭

[trim valve]

also neither of those are the card I was talking about

I specifically need the powercolor reaper

nothing else will fit in my case

[stark needle]

Cause how come everywhere GPUs are out of stock

[olive sable]

i dont mind buying and selling 3090's to yall. i just aint paying for shipping

[hoary lion]

i am definitely joining it if I had a chance

[trim valve]

also like surely there are fees involved for shipping it across borders

[nocturne olive]

599€?? Holy cheap

[stark needle]

SAME😭😭

[olive sable]

sometimes ye, like 21% import tax

[nocturne olive]

I wonder how much the shipping from wherever you are to Finland would be

[stark needle]

3192999$ shipping

prob 40-50$

[trim valve]

sounds like a good deal

[hoary lion]

wait I actually did kept a note of

[trim valve]

does that shipping include throwing the package down a flight of stairs

[hoary lion]

chinese 4090 48GB version

[stark needle]

• import tax but idk how inport tax works between EU countries

[opaque sigil]

There's none

[stark needle]

ok that's based

[opaque sigil]

Unless that changed

Used to apply to the UK too but not anymore since they left the EEA

[olive sable]

this is without a possible import tax tho

[opaque sigil]

I miss ordering custom cables from the uk

[nocturne olive]

If this dumb AI summary is correct, it's not a thing
But I don't know if I can trust it

[stark needle]

my dad had to import his DGX GB10 order via italy cause it's not even sold in Switzerland 😭😭

[trim valve]

order some to me and I can take them to the netherlands then ship them

[olive sable]

then it should be fine

[trim valve]

surely that is a valid way to avoid fees

[hoary lion]

totally won't snatch in between

[olive sable]

send me 650 and il buy it

trust

[hoary lion]

shadow

how much is 5090 rn

in your country

[stark needle]

Bitcoin address ahh reseller

[nocturne olive]

Hm, maybe
I'll consider it when I have more money if you can find me a good 3-slot one for which it's possible to find a pair later
I really, really need a 3090 for NeuroSynth training

[olive sable]

i will if you want me to, im just in debt rn so i need the money beforehand

[stark needle]

2.5-2.8k CHF (3000$-3400$) but out of stock everywhere

Apparently companies do the conversion of usd price -> chf price 1:1

[nocturne olive]

Just gotta figure out where I'm gonna get all that money
-# definitely not NeuroSynth commissions, nobody wants those anyway

[olive sable]

3.4K here asswel

for 5090

[hoary lion]

i have no fucking clue how variant can 5090 be

is this canadian dollars or nah i dont even know

probably US since it's bestbuy

[stark needle]

7381748382$ gpu😭😭

[nocturne olive]

Why does money have to be so hard?

[olive sable]

imagine aftger game-jam 3.
i get a bill:

usage of neurosynth:
$800 or 1 RTX 3090 24GB

[trim valve]

[nocturne olive]

That's so silly

[olive sable]

dont actually send me a bill btw

[nocturne olive]

But no, I've always operated on a "pay if you feel like it" model

Though I may sometimes provide stuff for free too easily

[trim valve]

sam can I mail you an invoice for an rtx ada 6000 please

I want one

[stark needle]

meanwhile @viral oasis (hi op) with 1 4090 and 2 3090s😭😭

[trim valve]

[opaque sigil]

that invoice for a gh200 is on its way

[stark needle]

my dad also has like 5 jetson orin nx on backorder😭😭 each of em is like 600$ and worse perf than rtx 2060

[olive sable]

none for sale here so bwaa

[olive sable]

hi operator lol

didnt know she was in this server

[trim valve]

smh

[stark needle]

she's everywhere

even in your walls

[olive sable]

damn

kinda weird

dont do that

[stark needle]

how did i get 4 bwaas in 1 second

[olive sable]

elvyn

the monent 1 bwaa was tpyed a 2nd imediatly appeared

and im just terminaly online

[trim valve]

I am stalking this chat because its more fun than revision

[olive sable]

i dont remember what i was doing tbh

[hoary lion]

i think i found all the 5090 that shadow can't find

[olive sable]

oh ye, i have to read a book and make a collage about it by tuesday

fuck

[trim valve]

i have to revise for an exam on wednesday :3

(I am not looking forward to it)

[opaque sigil]

what's the exam about

[olive sable]

undertstandable

[trim valve]

marketing

[opaque sigil]

ew

[olive sable]

marketing?

[trim valve]

yeah exactly

[olive sable]

what?

[trim valve]

i hate the subject so damn much

[olive sable]

suply and demand type of shit?

[hoary lion]

what the fuck lmao
chinese stores definitely have more gpus, even those that US did not allowed them

[trim valve]

no way more brainrotted than that

[real sierra]

@sage crag wakey wakey i need some convenience functions for using a display

[trim valve]

its basically just "people buy stuff"

[hoary lion]

see that dgx and h100 🤦‍♂️

[real sierra]

i attempted to write some but they arent working

theres a display that treats a region of ram as a video buffer

[olive sable]

yep we do, what about it?

[real sierra]

i can write data into it

but i want a nice way to just pick an (x, y) coordinate

[trim valve]

idk something about how to make it so people buy your stuff instead of other people's

but like I really don't like the subject nor the lecturers

if my math is right I only need to get 5% on this exam to pass though

[olive sable]

if i make it cheaper, and people know its cheaper, and the quality doesnt suck, then people will buy it

[real sierra]

current config is 64x64 pixels, monochrome, each 16-bit word of memory controls 16 pixels, in order from left to right, top to bottom

[olive sable]

ez

[trim valve]

well

you see

[olive sable]

see

im see

[trim valve]

that is one of the many things you are told

[olive sable]

uhuh

[trim valve]

but the subject basically boils down to "idk wing it"

with the exam basically just being a history lesson

[olive sable]

https://tenor.com/view/skeleton-reaction-information-my-reaction-to-that-information-my-honest-reaction-to-that-information-gif-613275461876515673

[real sierra]

for starters shouldnt that be (y * screen_width) + x

[trim valve]

like I'm expected to write an essay answer to one of these questions in like 30 minutes

[real sierra]

unless your y axis is horizontal

but secondly, the problem is that one word in memory is 16 pixels

so if i want to turn an individual pixel on

[tender river]

shiro i'm so sorry

you have to use right shift

[real sierra]

you can avoid right shift

by taking 16 - (x % 16)

and doing left shift

still i dont know why what i have isnt working

[stark needle]

Idk how common it is but ive been told by my dad that in russia the pc stores are extremely premium looking in comparison to the west (at least the ones he visited)

[real sierra]

# fill_pixel(x, y)
function fill_pixel 2;
    push_arg 1;
    push_value 64;
    call math_mult;
    push_retval;
    push_arg 0;
    add;
    call place_pixel;
    push_value 0;
    return;

function place_pixel 1;
    push_arg 0;
    push_value 8;
    call math_div;
    push_retval;
    push_value DISPLAY_BASE;
    add;
    push_value 0b0000000000000001;
    push_arg 0;
    push_value 15;
    and;
    push_value 16;
    sub;
    call math_shln;
    push_retval;
    pop_memory;
    push_value 0;
    return;

[tender river]

shiro can you teach me erlang i dont know it

[real sierra]

i also dont know erlang

[tender river]

you can learn it

[real sierra]

[trim valve]

can we combine our fragments of erlang knowledge into a full understanding

[real sierra]

i know nothing about erlang

literally 0

[tender river]

i know that pleroma is written in erlang (may or may not be true)

[trim valve]

0 is more than a number less than zero

so we're not losing anything

[tender river]

and i know that its based on message passing with in-process mutable state but no shared memory between processes

[rigid snow]

lmao what, did he visit boutique prebuilt stores or what, i can't call any of what i visited "premium"

[tender river]

idk what premium means in this context

[rigid snow]

like not even remotely

just regular ass stores

[tender river]

i mean it will be cleaner than a supermarket

[stark needle]

idk

He was in st petersburg

[tender river]

its better to work in whole words instead of individual pixels anyway

maybe have a function that bitands or bitors a mask to a certain word

that way you can do maths in your head instead of making your computer do it

thats the optimal way

please understand, you are replaceable, your computer is one of a kind

it deserves care and adoration

and precomputed maths

[rigid snow]

i'm in moscow so if anywhere is premium it's here

[real sierra]

this compiler agrees with you

the reason my instruction tokens are so garbage is because +, -, *, /, and any other arithmetic symbol is evaluated at compile time by the precompiler

so i cant use them as tokens in my instructions :(

[tender river]

did you want to make an apl or something

why would you need them in your identifiers

[real sierra]

no i just wanted to write "D+A" instead of "ADD_DA"

looks nicer

[stark needle]

@hoary lion ur not gonna believe it

[tender river]

makes sense, well you can force whitespace between identifiers to recover that

[stark needle]

i'm pretraining a llm with qlora 😭 😭 😭

AND ITS WORKING

[real sierra]

will try to adapt this since my attempt isnt working

[tender river]

i wonder if that means my stack breaker broke

let me see

the holy trinity

it broke as i expected

its fine i can fix it

this is totally a productive use of my time and very important for using the language

wait i think it crashes even without running the shellcode

[real sierra]

are you sure this is right

[tender river]

can you confirm

main := fn(): uint {
    start := "abcdefgh".ptr
    len := 8
    ptr: ^u8 = @syscall(0x9, 0, 4096, 7, 0x22, ~0, 0)
    i := 0
    loop if i == len break else {
        (ptr + i).* = (start + i).*
        i += 1
    }
    return 0
}

[real sierra]

these lines are a bit dubious

[tender river]

oh its because the old version of lily checked out the old commit

[faint sandal]

thanks ida for the descriptive error