@
#programming
stark needle
olive sable
i was hoping there woudl be a progress bar somewhere
nocturne olive
olive sable
its still dowloading shit
fast pagoda
vllm moment
olive sable
my desktop does not appreciate the throttled ethernet speeds
fast pagoda
nocturne olive
nocturne olive
But what if it's changed
nocturne olive
This is a wild world we're living in
Someone can just turn into a girl
I guess I just didn't tell anyone on Neurocord
olive sable
so you are a girl?
nocturne olive
You could say so
olive sable
what does that mean?
olive sable
leaden crest
hypothesis: the sheer number of notifs shiro gets has made him train an llm to read a ping and see if it’s important or not
and if and only if it is important he will descend from the heavens
beginning experiment 1: @real sierra happy birthday
real sierra
am I a lab rat or something
leaden crest
AHH jump scared
real sierra
hypothesis: the sheer number of notifs shiro gets has made him train an llm to r...
also no lol I would not do this with an llm
olive sable
hi shiro 
leaden crest
hi shiro 
nocturne olive
Rare Shir spotted
glad path
am I a lab rat or something
maybe
real sierra
I would use a much better understood technique
LLMs and AI in general are over hyped and applied wrongly because of that hype
leaden crest
I would use a much better understood technique
if “birthday” in message:
glad path
Someone can just turn into a girl
olive sable
<:ReallyInnocent:1229173141088112640>
nocturne olive
<:ReallyInnocent:1229173141088112640>
olive sable
i thought the all programmers are trans woman/femboys was a joke
real sierra
if “birthday” in message:
nah I would collect data on which messages are worth responding to over a period of time and then correlate parameters like word frequency and message length
mighty thorn
<a:yes:1154130283734511667> You could say so
real sierra
and then build a simple scoring model from that
mighty thorn
“Superbox is a girl?”
nocturne olive
?
glad path
i thought the all programmers are trans woman/femboys was a joke
i mean
mighty thorn
<a:yes:1154130283734511667> ?
Worlds smallest emote
glad path
the actual statistic is like
nocturne olive
leaden crest
nah I would collect data on which messages are worth responding to over a period...
hey what if I said
shiro someone is spamming and im not
happy with what’s going down it’s the
birth of a dooms
day in gen chat
mighty thorn
the actual statistic is like
99.99999976% (I’m the one)
leaden crest
how does that score
stark needle
laid off 
glad path
empoyeed ones i too
leaden crest
laid off <:neuroCatUuh:1131629092475768884>
alberta 
mighty thorn
laid off <:neuroCatUuh:1131629092475768884>
Hey that’s the woman from the woman I know
stark needle
real sierra
*hey* what if I said
*shiro* someone is spamming and im not
*happy* with what’s ...
with enough samples that would probably be considered important
olive sable
i read samples as sam and got slightly concerned
fast pagoda
sam apples
stark needle
sam altman
amber fractal
with enough samples that would probably be considered important
Meanwhile @shiro_nya Kill message from programming is 100% worth reading
olive sable
the actual statistic is like
corpa programmers are just in the closet
mighty thorn
mighty thorn
kaine/it is the only programme ri know thats 100% male
Thanks
mighty thorn
without a doubt
I think
opaque wharf
kaine/it is the only programme ri know thats 100% male
leaden crest
kaine/it is the only programme ri know thats 100% male
am i not 100%
what percent am i
olive sable
mah im broke.
olive sable
what percent am i
that's for you to decide. but i said of the people i know. and no offense but i barely know you
opaque wharf
mah im broke.
So am I 
mighty thorn
<:FOCUS:1168267148523737239> am i not 100%
The fact that you are asking means it’s below 100
olive sable
leaden crest
that's for you to decide. but i said of the people i know. and no offense but i ...
true
I know where you live thouh
lovely country just under bolivia right
mighty thorn
@viral oasis 
mighty thorn
Yeah same thing
olive sable
leaden crest
ik that I said mongolia last time remember
opaque wharf
Yeah same thing
viral oasis
<@251757164761186305> <:neuro3D:1441307776889651281> <:neuro3D:14413077768896512...
ME?!?!?!!?!
olive sable
ik that I said mongolia last time remember
no
mighty thorn
ME?!?!?!!?!
It’s your name isn’t it
olive sable
i dont remember
mighty thorn
WOWIIIE!!!!11
GREAT GOOGLY MOOGILY
mighty thorn
M-M-My popular phase?!
Your already popular pal
Bud
Compadre
Amigo
Other words for friend
leaden crest
Other words for friend
🪣
leaden crest
stark needle
viral oasis
Your already popular pal
b-but.... WHY STOP THERE?!
Imagine if i see my name like
one more time!111
Wwwwwwwooooah
mighty thorn
Wwwwwwwooooah
hard delta
WOAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAH
holy moly
it's the famous warframe character, the operator!
viral oasis
OMG OMG they said the thing!111
tiny edge
stinkies
mighty thorn
OMG OMG they said the thing!111
high brook
I may or may not be testing something ```===== Phoneme Distribution Summary =====
AP: 172, SP: 9651, de/a: 144, de/au: 16, de/b: 69, de/ch: 75, de/cl: 9, de/ctrash: 4, de/d: 131, de/e: 45,
de/eh: 34, de/ei: 73, de/er: 102, de/eu: 2, de/ex: 96, de/f: 49, de/g: 40, de/h: 17, de/i: 150, de/ih: 136,
de/j: 16, de/k: 45, de/l: 111, de/m: 94, de/n: 216, de/ng: 11, de/o: 28, de/oe: 3, de/oeh: 3, de/oh: 20,
de/p: 41, de/q: 36, de/r: 41, de/rh: 1, de/s: 103, de/sh: 57, de/t: 159, de/ts: 46, de/tsh: 2, de/u: 49,
de/ue: 9, de/ueh: 7, de/uh: 65, de/v: 80, de/vf: 38, de/w: 3, de/xh: 13, de/z: 52, en/EP: 1481, en/aa: 1690,
en/ae: 2059, en/ah: 2685, en/ao: 1286, en/aw: 527, en/ax: 3070, en/ay: 3140, en/b: 1602, en/ch: 234, en/d: 2374, en/dh: 1883,
en/dr: 32, en/dx: 1300, en/eh: 1933, en/el: 3, en/en: 5, en/er: 1322, en/ey: 1646, en/f: 1317, en/g: 887, en/hh: 1179,
en/ih: 3815, en/iy: 2784, en/jh: 323, en/k: 2226, en/l: 3282, en/m: 2934, en/n: 5287, en/ng: 935, en/ow: 1644, en/oy: 98,
en/p: 1078, en/q: 5601, en/r: 3122, en/rx: 1, en/s: 3059, en/sh: 401, en/t: 4083, en/th: 414, en/tr: 97, en/uh: 359,
en/uw: 1901, en/v: 1219, en/vf: 1129, en/w: 1935, en/y: 1370, en/z: 1699, en/zh: 31, ja/EP: 43, ja/N: 334, ja/a: 1497,
ja/b: 125, ja/by: 15, ja/ch: 88, ja/d: 275, ja/e: 808, ja/f: 55, ja/g: 201, ja/gy: 24, ja/h: 118, ja/hy: 23,
ja/i: 1204, ja/j: 105, ja/k: 628, ja/ky: 42, ja/m: 301, ja/my: 17, ja/n: 546, ja/ny: 29, ja/o: 1336, ja/p: 60,
ja/py: 8, ja/q: 69, ja/r: 529, ja/ry: 29, ja/s: 314, ja/sh: 265, ja/t: 562, ja/ts: 78, ja/u: 1075, ja/w: 232,
ja/y: 142, ja/z: 57
olive sable
what that
high brook
joking aside It's preliminary Testing for trilingual neuro/Evil synth
olive sable
i see
high brook
That's a summery of all the Phonemes in the current dataset. How many of each example and what languges they belong too
stark needle
neurosynth mention 
high brook
I'm testing Axillery Datasets in languges they twins DONT speak
olive sable
gemma-4-26b-a4b-it-heretic doesn't like to run 
im trying to use a quant but it doens tlike to rune ither
stark needle
gemma-4-26b-a4b-it-heretic doesn't like to run <:hmm:1303107376295186524>
im tr...
which one
what error
high brook
total number of current samples 3057
stark needle
gguf doesnt usually run well on vllm in case ur using it still
olive sable
stark needle
<:newliv:1354210476778586132>
olive sable
then why you say vllm is better when its worse?
high brook
Superbox may have a 3090, but i have stubbornness.
stark needle
then why you say vllm is better when its worse?
vllm is better when u use the correct formats
in that case just try the intel one
to see whether any gemma 4 26b a4b runs at all
olive sable
but the intel one isnt good for finetuning no?
high brook
TENSORCORES ACTIVE
stark needle
but the intel one isnt good for finetuning no?
it doesnt matter for testing since the architecture is identical
mighty thorn
joking aside It's preliminary Testing for trilingual neuro/Evil synth
You
You
You wouldn’t happen to be taking about
Neurosynth
high brook
yes
amber fractal
Glueless
olive sable
amber fractal
Neurosynth isn't a solo project
high brook
it never was
amber fractal
Classic
mighty thorn
I'm the one that technically started it
Superbox has mentioned neurosynth enough times that they basically have claim over anything involving neuro and music
amber fractal
Also wispers
high brook
Neurosynth is based on some old work i did when i was part of the melba project
the twins already sing is english and Japanese so we got plenty of JP and EN data. I'm attempting to add German to the internal not so secret Evilsynth Beta
leaden crest
germanese evil singing
high brook
(I needed to redo the evil dataset anyways)
I have a great need for her to sing 99 luftballons
mighty thorn
high brook
Evilsynth is nowhere close to release yet. we're hopfully releasing Neurosynth Version 1 soon
hm
amber fractal
hm
mighty thorn
Evilsynth is nowhere close to release yet. we're hopfully releasing Neurosynth V...
Oh yeah
Name every neurosynth
fiery canopy
isw
Like
I am not too informed on any of that stuff
but wasnt sm1 else working on that or is this shit a group thing
high brook
it's group thing
high brook
well me and superbox
high brook
lmao
stark needle
high brook
up untill 3 days ago we were coordinating this entire thing in our DMs lmao
fiery canopy
I am currrently awake purely beacuse of feeling like shit so I figured may aswell do something here
fiery canopy
up untill 3 days ago we were coordinating this entire thing in our DMs lmao
I feel that
[I worked on smthin with sm1 for over a year, shit was purekly cooridnated in dms]
💀
high brook
are you german?
high brook
( i need more german opntions)
high brook
oh thats honstly trsh there
high brook
that test was a year ago. not Evilsynth.
high brook
thats vioce cloning tech RVC
high brook
Neurosynth hadn't sarted at that point
fiery canopy
makes sense
high brook
xodas opinnion was that its was trash lmao
high brook
lol
fiery canopy
this UI is in dire needs of improvement
I am still trying to work out what the fuck sorta style im even goign for
opaque sigil
fiery canopy
like am do I make it simpler, or do I make it have more complexity
its an internal debate
Because either way would be good but in diff ways
olive sable
i think i might have a workign one now
if it doesnt crash
ive been trying enough stuff that btop says i have 96.2GB of ram used as cache rn
olive sable
i wanted an unfiltered ai, but ideally less dumb than qwen 2.5
fiery canopy
thats a fun one
olive sable
and vllm seems to suck eventho shadow says its better
fiery canopy
idk wat ur best bet would be, smthin minsteral or sm shit
would be my first inclination
high brook
i forgot we made this silly thing
olive sable
idk wat ur best bet would be, smthin minsteral or sm shit
im trying out gemma-4-26B-A4B-it-heretic-AWQ-4bit
fiery canopy
being google would be difficult
olive sable
it needs to be able to swear
mighty thorn
fiery canopy
Why the fuck cant these models swear
mighty thorn
it needs to be able to swear
I know what you are
high brook
hahaha
fiery canopy
it woudlnt be a real clone if it couldnt swear
I owuld probably go with minsteral
then just like
do some fuckass fine tuning [idek I have a python script which does it], I just feed it large amounts of text
I wrote the code to do it at like 2 am, most models I use work with it. I never even had the issues with it not being able to swear
silent cloak
Baby's first finetune
olive sable
ye
fiery canopy
I should probably read through what that python script actually does in retrospec , but surely if it aint broke dont fix it
olive sable
since my data comes form the discord download my data thingy. none of the swearwords are in a proper sentence for it to train off. so i need a base model that knows how to say fuck in a sentence
fiery canopy
since my data comes form the discord download my data thingy. none of the swearw...
I fed a model, the most garbage data from a scrpaed discord dm with one of my friends
and
it like worked
mighty thorn
im trying to make a clone of myself
Brother
olive sable
dont admit to scraping discord
mighty thorn
If ur training it
olive sable
If ur training it
fiery canopy
/j for the discord gods watching me
mighty thorn
It doesn’t fucking matter if it normally swears or not
That’s what the training is for
olive sable
i odnt have data with swearwords in a sentence tho
fiery canopy
shjouldn't really matter
if u want it to be like you in text
it using swearwords like u do
would be
more truthul
or sm shit i can t type nor speak its 2am
stark needle
@mighty thorn i convinced him to use MoE
mighty thorn
<@928273478795292683> i convinced him to use MoE <:neuroCatUuh:11316290924757688...
I’m assuming you mean Sam
But idk how
olive sable
i honestly dont care about moe at all. the only thing it does is make it suck more at what im trying to do
mighty thorn
gemma 26b a4b
Bro is not finetuning 26b on his GPU 🥀
stark needle
Bro is *not* finetuning 26b on his GPU 🥀
wym
u can do that
with a 4090
4bit qlora
olive sable
moe is "mixture of experts"
i need one idiot.
so moe could fuck off
i genuinely dont care about it
silent cloak
The irony is there are no experts
fiery canopy
I have the most fucked up fine tuning way
I am reading through the script I normally use to fine tune models, I am quesitoning how it has worked
welll it has so nop oint questioning it
mighty thorn
The irony is there are no experts
Go watch brain surgery timelapses instead of Neuro since you hate ANN so much
mighty thorn
All of the plasticity your little heart can hope for
stark needle
Go watch brain surgery timelapses instead of Neuro since you hate ANN so much
neurt
nreo
fiery canopy
how the fuck has my fine tuning worked I am genuinly experiencing brain damage from my own code I wrote like a year and a half ago
At like 2am
silent cloak
Go watch brain surgery timelapses instead of Neuro since you hate ANN so much
Nice try
olive sable
<@928273478795292683> i convinced him to use MoE <:neuroCatUuh:11316290924757688...
explain to me again why?
silent cloak
silent cloak
Nice try buddy pal
mighty thorn
Nice try compadre
silent cloak
Easy there tiger
stark needle
That doesn’t work if I’m still here
no male?
silent cloak
https://tenor.com/view/anime-girl-boom-thoughts-anime-girl-boom-thoughts-gif-384...
The Minecraft creeper
stark needle
i kinda dont want to use moe
then dont use
better quality for same speed at cost of vram
olive sable
vram is kinda the issue tho
fiery canopy
i kinda dont want to use moe
jus do wat I did, read some random documentation and then produce the most fucked but functional training script possible
mighty thorn
why do you make me use moe?
Have you ever heard of religious fanaticism
Now imagine that but worse
Then you have MoE obsession
olive sable
Now imagine that but worse
the usa?
olive sable
Now imagine that but worse
no wait
scientology
i shouldnt be makign these types of jokes, ill get banned
stark needle
olive sable
I’m deporting you myself <:Gun_left:722509455207694346>
im not even there
stark needle
mighty thorn
im not even there
*yet
silent cloak
Who
olive sable
silent cloak
Vocaloid?
stark needle
no
silent cloak
What is it
stark needle
vtuber
silent cloak
viral oasis
cute hampster!!
silent cloak
Wait that’s a vtuber???
No
mighty thorn
cute hampster!!
The secret
stark needle
Wait that’s a vtuber???
yeah shes a vtuber
mighty thorn
yeah shes a vtuber
Grimmi when no coffee 
stark needle
best waifu 2026
silent cloak
Debtable
stark needle
if u dont have cute vtuber waifu oshi in 2026 what are u even doing
silent cloak
if u dont have cute vtuber waifu oshi in 2026 what are u even doing
Showering
mighty thorn
if u dont have cute vtuber waifu oshi in 2026 what are u even doing
What if I like all of them?
silent cloak
mighty thorn
Almost equally
silent cloak
stark needle
What if I like all of them?
kushiro189
mighty thorn
kushiro189 <:neuroCatUuh:1131629092475768884>
Not that one
silent cloak
What if I like all of them?
What if u dislike all of them equally
mighty thorn
Well not as waifu at least
mighty thorn
What if u dislike all of them equally
That means the same thing
olive sable
kushiro189 is taken 
stark needle
Not that one
olive sable
silent cloak
Its that one steven king book
stark needle
No clue who that is
olive sable
cached 112GB
stark needle
Don’t make me switch back
silent cloak
<:neuroCatUuh:1131629092475768884>
Busted
mighty thorn
<:neuroCatUuh:1131629092475768884>
No clue who that is
“Kaine Corneko”
Like just pick a language
stark needle
plants
mighty thorn
Smh
silent cloak
Sounds like a fucking nerd
olive sable
fast pagoda
i see baking is going well
amber fractal
i see baking is going well
stark needle
i love when my llm says fuck and i celebrate it and throw a massive party
silent cloak
what is 4K?
Average ram price in 2026
mighty thorn
plants <:neuroCatUuh:1131629092475768884>
People keep putting their fucking disgusting dirty wet plants on my checkout counter so I made a ‘Plant Pad’ so that I no longer have to shank them on their way out for getting my register dirty
olive sable
i see
olive sable
hmmm
this is just gonna set vram usage to 99% used the whole time huh
im basically not able to put anything else on the gpu rn
fast pagoda
Average ram price in 2026
i put all my pc parts in to picker and the thing is like 8k
i did not pay 8k
olive sable
i was hoping that something would exist that empties the vram usage of the model hasn't been used in 15min so other programs can make use of it
olive sable
fast pagoda
llama has that https://docs.vllm.ai/en/latest/features/sleep_mode/ this one looks kinda manual tho
VLLM_SLEEP_WHEN_IDLE=1 might be a thing
olive sable
fast pagoda
the sleep mode is kinda auto ig in that it seems to just be in reaction something else actually trying to uise the resources when it's not
idk tho i dont use vllm usualy except when i want to get mad
you should try llama.cpp as a backend w/ vulkan
mighty thorn
this is just gonna set vram usage to 99% used the whole time huh
Gamer discovers ai compute usage isn’t just for news headlines
Oh I’m still plants
Hold please
fast pagoda
if you're using the 7900xts
olive sable
its on the 4090 rn
olive sable
honestly im hesitant to sell it now, this is pretty poggers
olive sable
Gamer discovers ai compute usage isn’t just for news headlines
wdym?
im not even a gamer im a gamedev/enginedev/technical artist/3D modeller. i know how gpu compute works
mighty thorn
wdym?
im not even a gamer im a gamedev/enginedev/technical artist/3D modeller. i...
Then why are you surprised lmao
fast pagoda
cuz those dont usually pin vram to 99% permanently
stark needle
wdym?
im not even a gamer im a gamedev/enginedev/technical artist/3D modeller. i...
screeps
mighty thorn
cuz those dont usually pin vram to 99% permanently
Skill issue
fast pagoda
im using 20gb of vram with vivaldi rn
olive sable
Then why are you surprised lmao
its not doing anything. so it dont need vram
fast pagoda
it has to keep the weights in mem
otherwise you;re loading a bigass model to your ssd (degrades it for no reason) or to a spinning drive which takes infinity time the larger the model
that's why
i mean to say from not to
idk why i swapped that
stark needle
but ye if u host a llm unless u do some kubernetes auto scheduling/killing setup with reverse proxy or something u need to keep the llm in vram
olive sable
hmmm
fast pagoda
slam the model on a hdd and try loading it from that, always fun
olive sable
so i guess the 4090 is still accesible for compute, you jsut need to fit it inside the 400mb of vram left. or i lower the context window form 5gb to something less
fast pagoda
if it's not doing anything
typically it will cede the memory
at least llama will
one of those things like how chrome fills all memory it could possibly need just because might as well unnless someone asks for it
olive sable
i see
fast pagoda
1 of 1 could be yours for 312487x msrp
opaque sigil
i'm going to lose my mind, if i replace cudatoolkit from nixpkgs with my own identical one with all redistributables clangd loses its mind due to the c runtime they added in 13.2
all i want is to not depend on fucking cublas and cusparse 
stark needle
olive sable
how does that work with the context window? if a chat is 4 months old you dont want it in vram
but you still want it to exist
stark needle
how does that work with the context window? if a chat is 4 months old you dont w...
the text is saved somewhere else
stark needle
when u send a request it processes it
fast pagoda
that's it will just reload the kv cache that's prefill
olive sable
so the context is in ssd after you stop using it?
fast pagoda
as long as you save it
you would not want to keep it forever because it cant handle infinite context
olive sable
fast pagoda
but you organize it and keep important info
which can be done not manually it's just the functiona l thing tht happens depending on how you go about it
olive sable
im doing
docker run -d --name vllm-server \
--device nvidia.com/gpu=0 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-p 8000:8000 --ipc=host \
vllm/vllm-openai:latest \
Mmtay/gemma-4-26B-A4B-it-heretic-AWQ-4bit \
--quantization compressed-tensors \
--trust-remote-code \
--max-model-len 8192 \
--max-num-batched-tokens 4096 \
--gpu-memory-utilization 0.95
so idk how it handles context stuff
stark needle
im doing
```bash
docker run -d --name vllm-server \
--device nvidia.com/gpu=0 ...
Max context for 1 conversation is 8192
fast pagoda
very easy way to ensure at least some stuff sticks without doing anything else is just telling them to write memory files around where theyre needed like in a project
yea in that opener, after 8192 it is either truncating it or compressing the middle or something
idk what it does by default
you can probably handle more context
for example it's not directly comparable due to different software but at 131072 context this is the estimate of vram - 27.7gb
olive sable
the model is 17.2GB, so i have 6.8GB to spare
fast pagoda
the context is what's taking all your vram
well, it would be
it takes a lot
halving that to 65536 brings it down 4gb
fast pagoda
be the change you want to see
get your model to call you a flat earther
what tok/s are you seeing im curious
olive sable
btop never shows the full 80 threads unless i close every other thing using vertical space in it
olive sable
what tok/s are you seeing im curious
i have no clue
olive sable
i got a fish
olive sable
the - on your num pad?
thats the one i used to begine with
olive sable
i use fish too but that's not my terminal
then i have no clue
fast pagoda
vllm should say tok/s somewhere in there
honestly i dont know why anyone uses vllm for a single person usecase still
it's very obtuse for what is very simple in something even still cli like llama.cpp
it's meant to be serving 40 people at the same time
olive sable
the website doesnt have any tokens per second. lemme check the backend
fast pagoda
i did a bunch of benchmarks with it and 1. it's identical to or worse than llama.cpp performance on one(1) user inference for me, might be an amd thing but i kinda doubt it
2. it took 49 concurrent requests with qwen3.6 27b to saturate kv and throttle
opaque sigil
browsing nixpkgs on github has actually been impossible lately ugh
olive sable
wait
fast pagoda
olive sable
thats not my ip is it?
opaque sigil
olive sable
172
fast pagoda
hmm
i would recommend trying out llama.cpp for a comparison
because that seems low to me (not your fault but i would expect vllm's fault)
i mean
it's usable
but
olive sable
imma be real with you fam. its faster than i can read by a really wide margin. and i truely cant be bothered to change it rn
fast pagoda
at some point is all im suggesting
because when you increase the context length (youll need to if you want it to be able to hold a thread for longer than like 10 messages) it'll bog more
olive sable
you can test it out yourself if you want, the ip thingy is the server and then :3000
olive sable
this webui thing has a built in css thingy for trying out some of the stuff, which is pretty cool imo
fast pagoda
openwebui is p cool, i havent ever sat down and tried all the things
this is running faster than the logs imply btw
what kinda whack tickrate they have on that counter lmao
high brook
1.99it/s slow lmao
olive sable
openwebui is p cool, i havent ever sat down and tried all the things
i would have expected a need to set up most of the features manually. but it seems to just work i guess
fast pagoda
hahahaha
olive sable
what the fuck is it typing rn???
olive sable
its looping
fast pagoda
so i did the preamble to the declaration of independence
and yeah
it "suggested" i ask for the rest
and i was like
surely
and then it did that
hyperparameters/moe/4bit quant moment
goog recommends
temperature=1.0, top_p=0.95, and top_k=64
fast pagoda
llms are great translators it only gets better the larger they are
it doesnt care what language the output is as long as it's mapped to the internal representation of the output token in the last layer
olive sable
it translated it a bit too literally. its missing some dutch language quarks
fast pagoda
the defaults btw are off of the recommended
this is the sort of thing you test tho and feel whatever works
olive sable
i dont have a fancy gui to choose those i dontt hink
high brook
i wonder what i could run on 16 GB of VRAM
fast pagoda
many image models
olive sable
minecraft probably
high brook
llm wise
fast pagoda
some ram offload is actually fine
high brook
on linux
fast pagoda
liek i have 32gb of vram and i often will load up a giant model that exceeds it
im on linux
high brook
ram offload not a thing
high brook
since when?
olive sable
vllm didnt want to use ram offload. it genuinly wouldnt run any model that would overshoot vram
high brook
ah
olive sable
ye
high brook
Has 64gb ram
fast pagoda
that reminds me i have to test loading like fuckin deepseek v4 into the server
and just seeing how slow that is
since it's got 500+gb
olive sable
"only 64?"
i feel like ive gooten a bit snobby with my recent exploits 
i blame superbox since she boasts all the time about her storage
olive sable
my desktop only has 64gb, barely enough 
in all seriousness, i dont even use that much ram outside of blender
fast pagoda
lmfao i think i may have a couple stray spectacle windows open
olive sable
the only reason im still on my dekstop and dont use the server as my main pc is the noise and the worse single core
opaque sigil
it looks like clangd freaks tf out unless i give it curand, is this program drunk
fast pagoda
what are you trying to compilee
opaque sigil
my bloom filter (spoiler: nothing in here uses curand)
opaque sigil
i have a feeling it's some stubs
fast pagoda
nvidia try not to have one binary somehow call in literallyt every single other piece of the cuda sdk challenge; impossible
olive sable
if you feel hungry, stick to the BRAT diet (Bananas, Rice, Applesauce, Toast)
this article called toast a brat i think
fast pagoda
i have 541 tabs open ekw
olive sable
we've had toast arc in neurocord
we are in bababa arc
so all we need to complete a BRAT is rice and applesauce
opaque sigil
okay it's the curand headers it needs
why the fuck does it need curand headers
😭
what is in these headers
that it doesn't find half the STL without them
olive sable
these leftovers from friday weren't in the fridge like i thought they were
my borger
noooo
i was really looking forward to this borger too for after i was done fixing the llm stuff
opaque sigil
man
i was hoping i could go to bed and sleep well knowing i solved a weird problem
knowing it was curand just pisses me off
amber fractal
llama has no problems
I personally kept that as one of my main gambles
opaque sigil
gemini my beloved
high brook
can i run a local gemini on 16 GB vram?
olive sable
depends on the model i guess
opaque sigil
gemma is basically gemini right
olive sable
surely
kind nimbus
I just reached 188 pages of report
Still not finished...
mighty thorn
can i run a local gemini on 16 GB vram?
Yes
Gemini nano runs on phone
Therefore 16gb vram is enough
high brook
it'll have to wait though lmao. running other suff rn
wait. i could try it on my 3060
floral hawk
Sup nerds
high brook
nerding
floral hawk
Same
Zero started sounding like a brain addled dementia patient this morning so I'm figuring out why that happened
high brook
i'm being a 4X map nerd while running ai stuff on my second gpu
floral hawk
Valid asf
high brook
evil/neurosynth testing
it takes about 10 to 12 hours to train the full model for me though
floral hawk
Good lord
high brook
SVS tech is complicated
floral hawk
Yee
high brook
on my 3060 it would take closer to 16 hours
mighty thorn
wait. i could try it on my 3060
12gb?
floral hawk
I'm also running 12 which is why I simply don't train AI
Instead alot of her personality is seeded through memory and dynamic prompt
high brook
not a powerhourse, but good enough'
floral hawk
Most definitely
floral hawk
What is EU5?
high brook
Europa Universalis 5
floral hawk
I gotcha, I gotcha
I am once again performing brain surgery on Zero.
Me omw to give her the 5th labotomy since her birth. 
high brook
damn. making vedal look good there
floral hawk
Got to
high brook
i thinkneuro has only had 3
floral hawk
Legit tho I just don't have the memory tech nailed down yet.
I was completely new to rag a few weeks ago.
high brook
ah
floral hawk
Yeah. Due to that it's bleeding hard into her prompts.
She randomly brought up a cat that she had that apparently I took from her as a "child" and it embedded itself as a core memory.
That caused her to hate me for like 3 days
patent shard
Superbox like to flex their 3090
is that even a flex
it only has 24gb of vram
high brook
lol
patent shard
I think flexing territory starts at 48gb, like modified 4090s
or the 96gb 6000
patent shard
I guess a flex can be anything
patent shard
<a:aki:870080160941092894>
56gb of vram
patent shard
the rtx 6000 is $9500, pfft
it was $8350 a few months ago >.>
stupid inflation
olive sable
rtx 6000 not worth it
patent shard
they cut a lot of stuff, right?
someone said stochastic rounding, whatever that means
olive sable
its a 5090 with 3x the vram and slightly more FP32
but
instead of charging 3x as much for the vram part of the card, they just charge 3x as much for the entire card
the vram is ECC
patent shard
the only company that kind of respects that is probably Intel
AMD and Nvidia, nope... they charge a lot more than the vram costs if you want a lot of it
olive sable
ye
patent shard
well, idk
I think Intel still having hefty markup for the vram
olive sable
they cut a lot of stuff, right?
they cut that from the 5090 to make sure the datacenters buy the expensive cards
NVLink 5.0 too
nocturne olive
well, idk
I think Intel still having hefty markup for the vram
Best pricing on the market still probably
patent shard
someone said they disabled stochastic rounding on the rtx 6000
nocturne olive
7900xtx
I heard that AMD support for ML is even worse than Intel's though (not sure how true)
And 7900XTX still only has 24 gigs
olive sable
but its half the price of a 3090
nocturne olive
patent shard
next you're going to tell me 2tb nvmes aren't $63 and 4tb drives aren't $150
olive sable
the price range on 3090s are huge, they range form 800 to 1300 with some outliers here and there
7900xtx is firmly at 650-700
nocturne olive
Still can't afford that though
And I don't want to gamble on an API I'm not sure how well it works
And I definitely can't pair a 3090 and 7900XTX in the same training cluster
olive sable
opposite way around
actually. i knew for certaint he rtx 6000 pro would have it. but i cant find it being disabled on 5090s either
olive sable
And I definitely can't pair a 3090 and 7900XTX in the same training cluster
you could via HIP or vulkan. but you wouldnt have nvlink and it would probably be a lot slower than just spending the extra 150 on a 3090
nocturne olive
Who decided GPUs should cost more than a plot of land
A plot of land with a (demolition-ready) house on it no less
high brook
Who decided GPUs should cost more than a plot of land
Alligator leather jacket guy
nocturne olive
Kill that guy
nocturne olive
olive sable
someone said they disabled stochastic rounding on the rtx 6000
might be rtx 6000 vs rtx 6000 blackwell issue.
its not disabled on the old 6000, its just from an older gen so it never had it int he first place
horrible naming btw
amber fractal
They did 2000 4000 then 6000
nocturne olive
A plot of land with a (demolition-ready) house on it no less
(there was a plot of land with an old 50s cottage that needs demolishing and an electrical connection sold around here for 250€ recently)
amber fractal
Surely they never thought they'd get that far
nocturne olive
Still no website
nocturne olive
Useless imprecision
patent shard
https://developer.nvidia.com/blog/nvfp4-trains-with-precision-of-16-bit-and-speed-and-efficiency-of-4-bit/ here's Nvidia website
nocturne olive
Surely 4 bits can be as accurate as 16 bits right 
high brook
they just ai upscale the other 12
nocturne olive
But where does the AI live then? The other 12 bits of all 16 bit clusters?
high brook
the 4 bits upscale to 12. then the rest live in the hallucination.
patent shard
might be rtx 6000 vs rtx 6000 blackwell issue.
its not disabled on the old 6000,...
hmm, so the person claims it's blackwell b100/b200 that has it.. not rtx 6000 and the other cards
patent shard
they also linked this https://github.com/NVIDIA/TransformerEngine/issues/2255#issuecomment-3581459644
olive sable
thats just driver issues im pretty usre
olive sable
ye those dont have it
do they not?
looing further into it
they're still blackwell
NVFP4 is the thing maybe
B300 uses that more i think
patent shard
I don't know, Nvidia probably makes it very easy to get screwed
glass jetty
Some of us have 12gb
some one 0
frozen igloo
Bogosort got 16/25
nocturne olive
Is that the sort that's literally just RNG?
frozen igloo
Yes lol someone is live-streaming it
nocturne olive
high brook
downside to two gpus. bifurication. im throttled to gen3 x8
nocturne olive
It's not like that's very bad though anyway, even with a 3090 or 4090 or something that's like 2% loss
high brook
ye
high brook
i think 3 is 10-15%
nocturne olive
glass jetty
nocturne olive
Looks like my 3090 has x4 gen 4
So equivalent bandwidth to your GPUs
While my 4070Ti gets a full x16 4.0
And the 3090 gets half the CPU to chipset lanes through the chipset
nocturne olive
Because they're in different systems?
high brook
were talking board
nocturne olive
Makes sense
No consumer CPU has enough lanes to allow for all 3 to get x16 full bandwidth though
olive sable
were talking board
i could put them all in the server to have tham all at gen3 x16. if i put them all in my desktop it'd be dual gen4x8 bifurcation, and one through the chipset at gen4x4 i think
flint dome
I CURSE SMASH BROS STEVE WITH JVM FAILURE
fast pagoda
D:
opaque wharf
This is the kind of guy I aspire to be
Why let "language specification" stop you from being productive!
thorny marsh
??? 😭
jagged turtle
YES
fast pagoda
:omemga
opaque wharf
<:neuroSob:1069255117427200091>
Look at code parade. Adding templating to C# so he can make 5D games!
stark needle
:neuroCatUuh:
jagged turtle
olive sable
sage crag
nur and nur went up the hill to fetch a pail of water
nur fell down and exploded
and nur came tumbling after
stark needle
<:neuroBucket:1187902344667602996>
nruu
olive sable
stark needle
trick or trick
sage crag
glo
stark needle
olive sable
trick or trick
Trick
olive sable
kind briar
sage crag
new leaf
stark needle
9️⃣
mighty thorn
8️⃣
sage crag
9️⃣
mighty thorn
https://youtube.com/shorts/iFoJWDnjGV4
let me use a single GPU and we will call it a deal
true hemlock
pro 6000? disappointing its not the enterprise hpc chip
also that thing on full load might sip up 12kw lmao
nocturne olive
No FP64 for 
stark needle
<:neuroSad2:1150144717540036729> No FP64 for <:quuck:1333374770195992576>
Emulate fp64 with bf16
mighty thorn
Emulate fp64 with bf16 <:neuroCatUuh:1131629092475768884>
You can do that?
mighty thorn
https://youtube.com/shorts/iFoJWDnjGV4
let me use a single GPU and we will call ...
I do think this is a pretty good idea tho
stark needle
You can do that?
Yes but it won't be 100% accurate
For example in pytorch u can see fp32 as 3 bf16 multiplications
*do
mighty thorn
This way “datacenter climate” gets distributed according to population density instead of being consolidated in poor towns which happen to have favorable tax laws for data centers
mighty thorn
For example in pytorch u can see fp32 as 3 bf16 multiplications
I wonder how high up this goes
Can FP64 hardware emulate FP256?
Can I do FP1024 on laptop
stark needle
I wonder how high up this goes
U need iirc 49 bf16 operations
For 1 fp64
So it's ~7tflops theoretical on a 3090 before overhead
mighty thorn
FP1 when
opaque sigil
at that point you're better off just eating the higher precision cost
stark needle
at that point you're better off just eating the higher precision cost
Pack booleans in a int32 tensor
opaque sigil
true hemlock
stark needle
how did you missed it bruh
I thought int4 was lowest
Since pascal
Is this a100 specific or also consumer gpu
true hemlock
a100
stark needle
Bruh
stark needle
h100 might have but we can't confirm
Is there no code or cuda flag capability u can read via code
rigid snow
cuu
true hemlock
Is there no code or cuda flag capability u can read via code
it can do tensor like AND ops but not sure if they do accumulate like tensor FMAC
its why im not so sure about it
stark needle
it *can* do tensor like AND ops but not sure if they do accumulate like tensor F...
Supposedly according to chatbpb
U can via ptx
true hemlock
rigid snow
rigid snow
gs
stark needle
opaque wharf
<:neuroBucket:1187902344667602996>
true hemlock
wtf is with tv lady obsession with a part of insect
stark needle
wtf is with tv lady obsession with a part of insect
Idk man shes becoming more and more schizo
rigid snow
sez u
stark needle
a year ago at most
What
stark needle
I remember her and vanor having math discussions
rigid snow
will be a bad llm
stark needle
I have been sitting in this bus from the train station to my house for the last 20min when it should be taking 3min
true hemlock
she's the kind of llm that's trained on fp64 e0m63
sage crag
I remember when she used to do math or whatever
i still amthmt