programming

[sick owl]

Good point

[opaque wharf]

Would be hard with material in the way. But I think that's just another BSDF

[rigid snow]

near useless tech tbh

[opaque wharf]

Well, for simulation it is the best you can approach it

[rigid snow]

maybe not audio necessarily then but rather emf simulations or something?

[opaque wharf]

Hmm, I feel like EMF simulation will require a different kind of ASIC. But who knows. Spectral rendering is quite new

[rigid snow]

because i've never heard of practical path traced audio applications

[opaque wharf]

Same

[lilac flame]

more immersive audio in video games

[rigid snow]

that's not what i meant by practical and that requires realtime on consumer hardware

[opaque sigil]

just one more asic bro

[rigid snow]

also steam audio exists and is foss and best in class

and cpu only

[opaque wharf]

I'd rather company that trains AI have ASIC to do it than using GPU

[rough bloom]

looks very cool but is definitely not for consumers, at least for this first iteration of it (and likely not for at least the next decade)
it wouldn't need 400 GbE QSFP if it wasn't meant for larger deployments in datacenters

[opaque sigil]

tbf a h100 is already basically an asic

[lilac flame]

oh thats cool i havent heard of that actually

[opaque wharf]

Petition to change H100 classification from a GPU to whatever dedicated ML processor is

[rough bloom]

GTPU

[opaque wharf]

Can it perform the usual graphic operation tho? Like rendering pixel on a screen?

[rough bloom]

N

[lilac flame]

doesnt gpu origiinally mean an asic but for graphics

[opaque wharf]

It stands for Graphic Processing Unit yes

[lilac flame]

now we have gpus without the gpu

[olive sable]

ehhh

kinda

[opaque wharf]

More like TPU

Tensor Processing Unit

[olive sable]

the whole part of the gpu that made it a necesity to add it to a sepereate device are the parallel cores, and we kept those

they just removed the display stuff

[rough bloom]

the H100 doesn't support any of the graphics APIs and doesn't have display outputs, so no graphics unless you use CUDA
it does have a few ROPs for some reason but they shouldn't be used for anything

[opaque wharf]

There's also the term NPU (Neural Processing Unit)

[opaque sigil]

there's a lot of specific texture and shader related hardware

[lilac flame]

yeah i meant more the term “graphics processing unit” in particular

[opaque sigil]

wouldn't surprise me if google patented this

[opaque wharf]

One of you guys from #programming would use that poor few ROPs to run AAA games I swear

[olive sable]

you can procces graphics without showing them

its just kinda weird to do it like that unless you're rendering

[rough bloom]

not the H100
it literally does not support it

[olive sable]

no?

[rough bloom]

no

[olive sable]

thats a shame

[opaque sigil]

there's no support for any graphics apis

[warped narwhal]

aren't rops just the part that does rasterisation? if so you could emulate it in software no problem.

[olive sable]

i feel like removing graphics api support doesnt even save that much money, its just to make people not use it for gaming???

[opaque sigil]

it saves space you can use to put more tensor cores

[olive sable]

ah

ok ok

[rough bloom]

yeah, you'd basically be writing a software renderer in CUDA
I don't think you can use the actual ROPs without a graphics API but I may be wrong on that

[opaque wharf]

The silicone literally doesn't have the needed stuff to do it

[olive sable]

well yes, but you could always add it

[opaque wharf]

[opaque sigil]

there are some nvidia cards where they disabled the graphics api in the driver but the hardware is there but iirc the h100 for example literally does not have the hardware for it

[warped narwhal]

graphics apis have minimum specs for capabilities, so if we can space by not adding all the fixed function stuff and rendering output, then it is either more space for cores, or you can have a smaller die and a higher yield per silicon cookie

[olive sable]

its not that expensive if you're doing the lithography of the rest of the chip anyways

[opaque wharf]

Silicone space is premium. Like VERY premium

[olive sable]

but ye you dont need it

[warped narwhal]

if you could add dx12 to your gpu, but then you can only fit 10 onto a wafer instead of 15, then is it really worth it? esp. when 99% of your customers will not use the api?

[olive sable]

h100's are designed for clusters anyways so you dont need 100 display outs you wont be using

[warped narwhal]

you basically lose 5 $10k sales for an unused feature

[cosmic sphinx]

all matches for today have ended
pretty much all expected results

[rigid snow]

opus 4 not winning a single game is surprising, did they forget to turn on reasoning or what

[cosmic sphinx]

no, they did
claude models just suck at chess ibr

better at coding in some way

[opaque sigil]

what bothers me the most

[cosmic sphinx]

but I honestly dont see even a point in using claude in 2025 with their prices

[opaque sigil]

why first to 4

[cosmic sphinx]

its bo4

[rigid snow]

bo7

[cosmic sphinx]

or was supposed to be

[opaque sigil]

bo4 makes no sense

[rigid snow]

first to 4 is bo7, why are they saying bo4

[cosmic sphinx]

I think they dont understand how best-of and first-to work :LULE:

but the games went first to 4, not 2
wtf fr

[rigid snow]

vibe-organized tourney

this format makes no sense

[desert wave]

bo3 but sometimes bo5

[hoary lion]

tpu

TRC program is also giving off tpuv6

V6!!

just one generation away from their frontier

and probably what served gemini for a quite long time

poggers

[olive sable]

aight i have split the 1 megafile into 3 smaller ones

ill need to reformat some stuff still tho

rn i define the needed persistent variables in the main.cpp class and i pass them as a pointer to the other file, but if a variable is mainly used in that other file wouldnt it make more sense to define it there?

ofcourse that would mean it'd be global but still

[opaque wharf]

What are you trying to achieve exactly?

[olive sable]

triangle

[opaque wharf]

I mean, what variable do you need and where?

[olive sable]

for example, i create VkDebugUtilsMessengerEXT debugMessenger in the main.cpp class, but i only use it in the debug manager file

[opaque sigil]

why tf is the version for the extra/haskell-data-default at version 0.7.1.1-356
what are they doing that they need so many revisions for the pkgbuild

looks like most of the haskell packages are at a couple dozen or 100+ hmm

[olive sable]

thats the day of the year the build was released

on the 356th day of the year they released 0.7.1.1

but ye i have no clue

[tender river]

mm ghc or transitive dependency updates probably

[opaque sigil]

i guess that would make sense hmm

[opaque sigil]

you could factor them out into individual modules that hold the relevant variables and then use that

[tender river]

since a version bump is basically a manual nonenforced indicator "the build output changed please rebuild/redownload"

[dense marsh]

mornibg peeps

[olive sable]

like a struct or something?

[opaque sigil]

that'd work yeah

[stark needle]

Was it already said that the new gpt models dropped

They suck

Trained to benchmaxx

[olive sable]

bwaa

is deepseek up to anything cool recently?

[hoary lion]

[olive sable]

[hoary lion]

no r2 means no cool stuff yet

[olive sable]

ai dev has certainly halted

[hoary lion]

deepseek is not the only team tho

but still

[stark needle]

Chat this model is so bad I can't

I have it running on a h200 with the lilac training mixture

[olive sable]

google shut the fuck up. ive said no to this 6 months ago

[stark needle]

And loss wise

Gemma 3 4b outperforms by a mile

In mean token accuracy

It's so bad at vtuber which means

It was trained only on stem

Aka to answer questions no one asked

[glass flower]

i mean.. they did say they primarily trained it on coding

tho i have no idea how you are suppose to run the 20b version.. its so slow on my 4080 and qwen3 30b outperforms it for me

[stark needle]

It's slow cause

It needs custom megablocks kernel

[hoary lion]

custom kernels ugh

we all hate them

[glass flower]

it also runs like 60% on the cpu for me.

[hoary lion]

vs 20b?

[stark needle]

Ye

The gpt 20b is especially bad

[hoary lion]

i haven't touched any of them yet

[nocturne olive]

I guess I can determine those things to be pointless to even experiment with

[glass flower]

tho the 120b seems to atleast be decent

from the few people that i heard talk about it

stick with qwen3 for now.

[stark needle]

Qwen 3 megabased

[glass flower]

i wish there was a qwen3-coder:8b version

[stark needle]

Maybe i can pull off a good finetune for gpt oss

[rigid snow]

this is the stuff i expect a 1b model do

[tender river]

openai is a small indie company please understand

[glass flower]

that has probably something to do with their "safe" training

[stark needle]

true

[stark needle]

They overfit the model on their rules actually

There were some posts showing

"based on my internal rules"

Type shit

Where it would list them verbatim within the reasoning traces

lmfao

[opaque wharf]

Small indie company indeed

[glass flower]

honestly i don't get it... gpt hasn't been top in a long time and now they are just shooting themself in the foot

[stark needle]

Cause they need to release gpt5

[hoary lion]

this probably

[glass flower]

coding: claude or gemini 2.5 pro.
everything else... one of the chinese models

[stark needle]

Sam altman circlejerk on twitter trying to maximize reaction

It's so bad

[opaque wharf]

"Look, our latest model performs better than the previous by a wide margin!"

[stark needle]

Yea

They first "accidentally leaked" on hf

That's so fake

[hoary lion]

that hype tbh sucked

[rigid snow]

yeah created like 30 repos, very accidenta;

[stark needle]

Lmao

[glass flower]

btw are these 2 the horizon models? probably not right

like the anon models that are available on openrouter

[stark needle]

Hopefully not

That's likely gpt 4.99999

[glass flower]

no it doesn't seem like it. the horizon models are still up

[glass flower]

idk.. i feel like maybe its a new deepseek?

from what i saw its pretty good at coding. so it can't be a gpt model

[hoary lion]

I would forever not forgive altman for not releasing gpt 5 today

[hoary lion]

?? lol

[rigid snow]

people have now started abusing the app name header feature on openrouter to spread anthropic propaganda

[stark needle]

Wanna see how funny this is

[rigid snow]

insane

[hoary lion]

i mean it;s true tho

[dusky jackal]

LMFAO BASED SCHIZO RANT

I was kind of excited for the open weight models, but this is even better than I thought!

[rigid snow]

wait they link to a domain

[stark needle]

Chinese Model vs gpt oss

[rigid snow]

that redirects to some random person's github

lol

[stark needle]

2.4 loss vs <2 loss

[dusky jackal]

Oh, nvm then.

[stark needle]

Even for finetuning it sucks

[rigid snow]

cause moe

[stark needle]

I hope it fixes itself magically

[rigid snow]

right

[stark needle]

SystemL 21b is also a moe

It's 20b vs 21b

[opaque wharf]

Moe moe kyun

[stark needle]

Both 3b active params

[rigid snow]

i mean regading finetuning

[opaque wharf]

Surprised no one named a model this yet

[stark needle]

No the chinese moe trains just fine

Fyi

The stuff in the hf repo

[hoary lion]

shad

[stark needle]

Is for some stuff 1:1 copied from t5

[hoary lion]

i think the activation is too small

[stark needle]

Lmfao

[hoary lion]

3.6B A

too smol

what did we exepct

[stark needle]

No qwen 30b is also 3b

Active

It's just they copied from fucking t5 for some reason

[dusky jackal]

Kind of off the rails, but personally I don’t really believe AI VTubers are really seperate from their base model unless they’re fine-tuned. A prompted mistral 7b is just a mistral 7b to me.

[stark needle]

Yea that's why i have a training corpus for lilac

[dusky jackal]

Based.

[stark needle]

Why tf is mean token accuracy stuck at 55%

Actually

[hoary lion]

lmao

[stark needle]

I'll let it be for the night

And see

How much money I'll waste

The trash architecture was expected tho

They aint google

Google actually drops huge arch improvements

E.g. gemma 3n is based

[hoary lion]

pretty sure

they debuff them before release

cause "GPT 5"

[dusky jackal]

I’m guessing it’s too strict, yeah?

[hoary lion]

suckers

[stark needle]

No it was just trained on math and code so it's having rough time absorbing the vtuber data

[dusky jackal]

Ahh got it.

That’s a shame.

[stark needle]

Yea

Tbh

I'll just release this finetune oss

Lmao

Yall can do whatever with this then

[hoary lion]

sauce for uhh

lilac

i actually don't know who is lilac still

[dusky jackal]

Guessing Lilac won’t be using that then

[stark needle]

lilac

[olive sable]

[hoary lion]

fumo always have that stupid gaze i like

[stark needle]

I love this plushie so much

If i had the money I'd buy 20

[hoary lion]

fumo is life ahh

[stark needle]

3600 bucks tho

😭

For 20

[olive sable]

thats crazy

i can make most parts of a plush myself, except for the stitching on the eyes

[stark needle]

For that u need an embroidery machine

[hoary lion]

a what machine??

that sounds ominous for no reason

[olive sable]

i have an embroidery machine at home, its just form 1920

[opaque wharf]

Do it the olden ways

BY HAND

[olive sable]

its permanently attatched to a table and you need to manually spin it with your feet

its not feasable to make anything useful with it

[opaque wharf]

It is feasible. Just need a lot of practice like most things

[olive sable]

its one of these. probably not the exactr same model but the same brand

[hoary lion]

this is ancient wow

[opaque wharf]

Ahhh, singer. My grandma has one of those.

And still very dexterous too when using the machine

[olive sable]

ive only made an actual plush once, and it was a pain in the ass

[opaque wharf]

To sew stuff

[olive sable]

manual stitching and i stole the stuffing from an old pillow

it was pretty shabby but it was free

[rigid snow]

#sewing

[opaque wharf]

I forgot how easy it is to get sewing and other textile product here since it is one of our country largest export

[rigid snow]

i'm so glad the #screeps arc is gone at least temporarily, no interesting topics were brought up

[hoary lion]

was it necessary to mention that forgotten word

now screepers would arise

[rigid snow]

i'd bonk them back into their holes they'll crawl out of

[tender river]

shiro slander

[olive sable]

so anyways back to sewing

[glass flower]

soo.... any good wall building algorithms you guys developed for your screeps? i currently just letting mine do their thing and upgrade my controller

[rigid snow]

you buy bricks and lay them down

good technique

[olive sable]

i just place the walls manually

[hoary lion]

why tf perplexity brought this up

no I HATE GPT

[stark needle]

Holy shit also have one of those 1:1 at my dads house

[glass flower]

gpt-5

[olive sable]

in the middle ages flanders used to be the best textile makers and the richest county in Europe.

[rigid snow]

openai engineer in this chat

[hoary lion]

[hoary lion]

you broke the nowaying chain

[glass flower]

he is leaking gpt-5

[stark needle]

HES LEAKING AGI

What

No way

[hoary lion]

fun fact is that gpt 5 actually got leaked on perplexity

pretty sure someone got fired

[glass flower]

[stark needle]

Perplexity got the hidden url

[rigid snow]

does perplexity ignore robots.txt or what

if so fuck them

[stark needle]

Yes

Cloudflare blocks them now

Lmao

[rigid snow]

good

[opaque wharf]

Almost everyone nowadays

[olive sable]

gpt5 gpt4 + fusion card + gpt4

[rigid snow]

who's everyone

[stark needle]

They were rotating ips and headers and everything

[glass flower]

[rigid snow]

i can understand that

[opaque wharf]

Any AI company

[rigid snow]

yes but perplexity is a search engine

[stark needle]

Even outside their officially documented ip ranges

[opaque wharf]

With AI

[glass flower]

so google?

[stark needle]

Perplexity is chatgpt wrapper

[rigid snow]

i don't think ai companies roll their own web search while ignoring robots.txt

[opaque wharf]

Yes, but I don't know if they use the same crawler for search index and their AI stuff

[stark needle]

They dont

They have a extended bot

For ai

[rigid snow]

scraping for data and for indexing are a bit different

[opaque wharf]

Yes, and the AI division is what ignoring robots.txt

[stark needle]

Me when ai companies say "any public data is open to take"

[rigid snow]

that doesn't make sense

[rigid snow]

because how would gpt-5 leak then? 3 mentions of it in pretraining wouldn't suddenly make the model recall it

that means that their search ignores robots.txt

[stark needle]

It does

It explicitly does

https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/

Lmfao

Cloudflare blog

[rigid snow]

absolutely insane

[stark needle]

sometimes failing to even fetch — robots.txt files

[opaque wharf]

Now the question is, do they use the same crawler for their AI and their search engine? Because google as you said, uses a different bot

[glass flower]

isn't perplexity just their AI?

[opaque wharf]

Hence my statement, AI companies ignoring robot.txt

[stark needle]

Perplexity is chatgpt wrapper

https://huggingface.co/openai/gpt-oss-20b/discussions/14

[opaque wharf]

In short, they want open source to remain the near exclusive domain of autistic coding nerds.

[hoary lion]

its simple

they dont want 4o competitor

so openai made it clueless on real world info

[cosmic sphinx]

m8 find me a goddamn phone that can run a 20B model

[olive sable]

sam altman's phone, similar to a gaming laptop, uses 5W on batrtery and 1500W when plugged in

[cosmic sphinx]

yh bro has 3 built in H100's with a threadripper CPU in his phone

custom package

[olive sable]

bro has sub-zero cooling, in kelvin

[sick owl]

GPT oss 20B with 128k tokens of context on my GPU

Holy shit

[sick owl]

The new Pixels are getting 16 gigs of ram IIRC

[hoary lion]

where is the remaining 4gb sir

quant?

[tender river]

you too can run gpt-oss on your phone with a ChatGPT™ Plus® plan! buy today at https://chatgpt.com/plus (not sponsored)

[sick owl]

Its 20B parameters not 20GB lmao

[cosmic sphinx]

i aint runnin this on my pc 🥀

tried downloading the fucking qwen-image yesterday and BSOD'd my PC by accident

[sick owl]

Its MoE, you can run it EZ with some offloading

[cosmic sphinx]

on a 4080?

[sick owl]

Easily

You have 16 gigs to play with on a 4080

[cosmic sphinx]

ima try getting lm studio back in and cook smth ig

[sick owl]

Hell you won't have to offload

Just drop context down to like 90k instead of 128

You've only gotta save 1GB of VRAM so your OS can do its thing

Hell drop down to 64k context and you can run it while doing other shit with zero issues

[glass flower]

i can barely run the gpt 20b with 8k context

how are you running it with 128k

[sick owl]

You're using ollama aren't you

[glass flower]

[sick owl]

Lmao, yeah ollamas memory management is fucked

I'm using llama.cpp with flash attention

[fast pagoda]

be me
distrohopping
"man i wish there were a way to just kinda store my exact configuration across installs of my system, that way even if i haev to wipe it for some reason it's just configured exactly the same way"
realize that's what nixos is for
it's over

[glass flower]

i haven't figured out a good alternative that allows me to quickly use models.... i really should get to it and get rid of ollama. i hate where its heading with their GUI that silently installs when you update

i should make my own thin-ollama client

i just need something to spin up models and then shut them down after some time

[fast pagoda]

vLLM

[glass flower]

but doesn't that always run the model?

i thought its more for server setup

[fast pagoda]

huh it serves it like ollama does, i didnt know it did anything differently, you can just do like vllm serve `Qwen/Qwen2.5-1.5B and itstarts serving that on an endpoint

are you saying like you want it to idle and unload the model if idle?

[glass flower]

since i only need the model to load when i use it with vscode

and then it should go away if its not in use

[fast pagoda]

good question

seems like it would be something it would support

[opaque wharf]

For mentioning the possibility of using NixOS alone, you have summoned the NixOS council here

[fast pagoda]

lmao

i have nixOS on my ventoy

havent had the courage yet

[opaque wharf]

LM Studio

[cosmic sphinx]

@sick owl where do I switch my models download directory I forgor

o found it

[fast pagoda]

lmstudio uses llama.cpp

just like ollama

[opaque wharf]

Good luck with that lol. I am content with my Arch setup

[fast pagoda]

im on cachyOS right now

for my main pc

i just keep distrohopping on my laptop

[glass flower]

but does it do memory management better or worse?

[opaque wharf]

You have a GUI at least to control a lot of the parameter

[cosmic sphinx]

only 12 gb of files wauw

this is gonna be ez

[fast pagoda]

my ventoy drive rn lol

[glass flower]

a gui is exactly the reason i want to move away from ollama. so thats not a positive

[fast pagoda]

lm studio's UI is the reason i am moving away from it

it has some stuff but it's frustratingly limited

[cosmic sphinx]

better than running into a risk of bsod'ing my own pc bc I cant handle memory shortages

[opaque wharf]

I see lol. I thought you would be interested in the auto unload model

[fast pagoda]

yeah lm studio do be having that i forgot

that's what i used to use for echo but i havent set him up on this pc yet

[glass flower]

surely... its not diffcult to get that functionallity with vllm

[opaque wharf]

And also this config for inference

[fast pagoda]

https://github.com/mostlygeek/llama-swap

apparently you can use this with any backend including vllm

and it supports auto offload

Automatic unloading of models after timeout by setting a ttl in features

[opaque wharf]

I honestly have no idea lol. I use LLM the same way you do, that is only loading it when needed

That's why I don't need a daemon running all the time

Just open another app as needed

[fast pagoda]

was reading this thread and found a few that support it but it's mostly thru frontends

https://www.reddit.com/r/LocalLLaMA/comments/1k63qy6/any_llm_backends_that_autounload_models_like/

[glass flower]

vllm has /sleep and /wake_up endpoints.. so you can probably just have a slim python script that is proxy

[fast pagoda]

speaking of ai slopshit i found that stable diffusion is like 2x faster on my 3080 on this cachyOS install

than it was on windows

so that's cool

check this out

https://github.com/BoredBrownBear/text-generation-webui-model_ducking

that's the whole point of this thing

but it's for oogabooga text gen webui and sillytavern

[cosmic sphinx]

ok chat this is first time im trying a MoE local model

what do you increase number of experts for

[fast pagoda]

dont

just dont change number of experts usually

it can cause it to be weird and/or shit

it's fine tuned with the built in number of experts

[cosmic sphinx]

[fast pagoda]

more experts you would think is better automatically but there is usually a ssweet spot

the router itself is the problem with that

i mean theoretically more experts = better

but unless you're on base model only and planning on fine tuning further with your chosen number of experts it'll probably get unstable quickly

[olive sable]

the more i use c++, the more i question if i suck at programming or if c++ sucks as a language

and then i relaize the anser is both

[fast pagoda]

i was gonan say

it's both

lol

[olive sable]

it was never a valid question

[fast pagoda]

hmmm claude has opus 4.1 now i didnt even notice that happened

pre-empt gpt5 i guess

agi

[olive sable]

but ye anyways i have now made the seperate files not contain functions, but a class with functions and variables cuz it jsut works better that way idk

chay would not be proud cuz of the OOP

[fast pagoda]

if it's not haskell i don't think chay is proud

ever

[cosmic sphinx]

ok im runnin in a lil bit of trouble here
I think lm studio prioritizes taking up my normal RAM over VRAM for some rsn

like it fills up close to 100% way too fast

[olive sable]

i started using capitals for the first letter of a classname tho, that is huge

[fast pagoda]

choose the gpu offload layers

appropriately

it'll default to like

8

when there's way more than that

you would configure that when you're initializing the model

[cosmic sphinx]

its literally set to 20/24

[fast pagoda]

oh

try 24/24

[opaque wharf]

The brute force way lmao

[cosmic sphinx]

gonna lower the context

[fast pagoda]

it should put whatever doesnt fit in system ram anyways

using anything more than like 8k context locally balloons memory usage pretty fast yeah

[cosmic sphinx]

???????

ok fine system, you win

[olive sable]

i cant read that, is that ram?

[cosmic sphinx]

yes

[olive sable]

ah

[fast pagoda]

your problem is that says память and not memory

[cosmic sphinx]

4k length it is LULW

[olive sable]

i may not have a brain gentleman but i have an idea

[fast pagoda]

an idea and a dream

[olive sable]

remove your ram and then it cant use your ram

[fast pagoda]

the perfect plan

[cosmic sphinx]

am scared

it says 15 gb is taken but FUCKING WHERE

im fairly certain i toggled off the reserve memory checkbox

am really not clocking where is that 15 gb coming out from

[fast pagoda]

waht size model

[cosmic sphinx]

the 20b one

[fast pagoda]

what is your VRAM

[cosmic sphinx]

16 gb

[fast pagoda]

what quant of model

i would expect you to fit like a 20b in q4

[cosmic sphinx]

?? im sure oai didnt release any quants yet

[fast pagoda]

oh yeah

so

it's gonna need like

40gb

[cosmic sphinx]

nah no way fr fr

@sick owl did u scam me

[fast pagoda]

i dunno i'd epxect it to somehow use that much

for a 20b

[sick owl]

?

unsloth/gpt-oss-20b-GGUF

ggml-org/gpt-oss-20b-GGUF

The model is MXFP4 by default

[cosmic sphinx]

ok so i did download the wrong model POG

[fast pagoda]

yeah a 20b if it's full precision

bf16

needs 2 bytes/param

which is like 40-45 gb of vram

[cosmic sphinx]

but hey it works

[sick owl]

Its 20B

[fast pagoda]

err

i meant 20b

[cosmic sphinx]

sort of, but im scared for my RAM so im gonna delete this shit

[sick owl]

Oh gotcha

[cosmic sphinx]

is it released already tho

[sick owl]

Yes

I'm literally running it

[cosmic sphinx]

[fast pagoda]

make sure ur using flash attention and kv offloading stuff

[sick owl]

Repo stats probably haven't been updated yet

But Unsloth are my go-to quant providers

[fast pagoda]

unsloth are based

[sick owl]

And if you go to the ggml-org one they're literally the guys who make llama.cpp

[cosmic sphinx]

f16 is the only available so ima go for this one ig

[fast pagoda]

just a couple real smart mfs mathing it up

f16 is thicc

that's gonna need 40gb

[sick owl]

"F16" here is the mixed precision format the model was trained in

Its actually mostly 4 bit

[fast pagoda]

oh

[sick owl]

Comes in at around 14GB

[fast pagoda]

i was thinking it was a full f16 weight model

i havent looked into it at all yet

[sick owl]

Nah, the embedding and output layers are FP16 but most of the model was trained in FP4

[fast pagoda]

interesting

guess they determined that was enough

[sick owl]

Very, its the first release like this I've seen

They merged in support for the MXFP4 format in llama.cpp too

[fast pagoda]

makes sense with how q4 is like the default i keep running into

everyone keeps measuring what their model will run on with fineprint of **in q4

[cosmic sphinx]

if that one was 12 t/s unoptimized I wonder how fast this one gonna run

[fast pagoda]

12 t/s inc

[sick owl]

Very

[cosmic sphinx]

download on this 1 is holy fucking slow tho

[sick owl]

Its ridiculously fast

[fast pagoda]

wpmder wjat this lm studio linux download is inb4 flatpak or appimage

god dammit it's .AppImage

[sick owl]

Classic lmstudio

[fast pagoda]

time to download a sketchy AUR version

[sick owl]

Very much a closed source program so you get stuff like that but its good enough

[cosmic sphinx]

3 MB/s

[sick owl]

I still prefer to stick with llama.cpp though

[fast pagoda]

i just like using it for very fast tests

cuz i havent set up any llm shit on this install of cachy

[sick owl]

You are downloading in llama.cpp itself right?

Because that's a lot faster, they throttle browser downloads on huggingface iirc

[cosmic sphinx]

im downloading in the fucking lm studio

[sick owl]

o7

In that case you just have a slow connection to their servers

[fast pagoda]

no viruses here

surely not

[cosmic sphinx]

the og model got yeeted in fast but i guess hf is fucked up

[sick owl]

[fast pagoda]

actually they look normal thankfully

[sick owl]

Optimal sampler params here btw

[cosmic sphinx]

Mistral GPT pog

[sick owl]

https://huggingface.co/unsloth/gpt-oss-20b-GGUF/blob/main/params

[fast pagoda]

lol this is already the default

that gets shown

[cosmic sphinx]

Native MXFP4 quantization: The models are trained with native MXFP4 precision for the MoE layer, making gpt-oss-120b run on a single H100 GPU and the gpt-oss-20b model run within 16GB of memory

within 16GB of memory

https://tenor.com/KoqH.gif

only if you use like 2k context

[fast pagoda]

gotta be your connection

[cosmic sphinx]

thats not the

unsloth

u fell down the same slope I did

[fast pagoda]

it was recommended by lm studio surely it'll be fine

[sick owl]

BF16 ono

[fast pagoda]

now i know it's your connection for sure

cuz i get 70MB/s while downloading both

for each

engine revving sound

i need to get a 10g ethernet card

[cosmic sphinx]

[sick owl]

Weird

[fast pagoda]

need new ethernet

[sick owl]

You got a VPN on?

Could be that

[cosmic sphinx]

not really a vpn, just a bypass script

[fast pagoda]

lmstudio should be connecting to hf direct anyways

[cosmic sphinx]

restarted app & download, seems sorta better

[fast pagoda]

it must be something with hf where you are maybe

[cosmic sphinx]

12 mb/s

i'll download it rn and toy around with tmrw

1:46 AM over here

[fast pagoda]

lmstudio is unusable

wayland icon

instead of the app icon

it's over

did you maybe have this toggled @cosmic sphinx

i forgot about this setting

[olive sable]

you know you fucked up when you change a file, and then you get 20 errors from completely different files

[fast pagoda]

nah that just means your 20 other files are wrong

not the one that just changed

[olive sable]

surely

[fast pagoda]

[fast pagoda]

exactly

i feel like i read some anecdote recently about someone who was having inexplicable errors in his code for something and it turned out to be a cpu bug

[olive sable]

[sick owl]

That would actually drive me insane in the truest sense

[fast pagoda]

now i find the reason i didnt want to dive in to running this crap on linux

exciting times ahead

[olive sable]

i can imagine intel saying "do no under any circumstances do these 25 instructions after each other or your cpu will literally kill itself"
with the shit they've been doing lately

[fast pagoda]

i think this just means it's running out of VRAM

[fast pagoda]

they just need a longer instruction pipeline

bigger branch predictor

how aboout 120 instead of 40 y'all

just predict a year ahead

[olive sable]

oh i forgot they've been doing those "core ultra" cpu's

to me i just thought 14900KS is the latest and greatest they have

[sick owl]

After locking the model in memory at 128k tokens

This is while watching twitch mind

[fast pagoda]

yeah they changed the naming again for the lulz so now it's once again impossible to tell what is what unless you already kjnow

[glass flower]

im downloading the unsloth version lets see if ollama can run it

[fast pagoda]

y'all i was so close today too

[olive sable]

apparently 290K or whatever the fuck its called has a built in npu

[olive sable]

i feel like thats anti-marketing, nobody wants that

[sick owl]

Be aware ollama blows up the context cache size

[nocturne olive]

Not like it helps anyone if you can't get good memory bandwidth

[fast pagoda]

should've just done it

[nocturne olive]

In LLM inference bandwidth is always the limiting factor

[opaque sigil]

until it isn't

[fast pagoda]

they wanted over 3k still

so i said fk that

[nocturne olive]

And when is it not? On any modern GPU they run out of bandwidth before compute

[olive sable]

its kinda funny how all these big companies are now boasting features that activly make me not want to get their product

[nocturne olive]

AI slop?

[fast pagoda]

was tempted on the 24gb intel part though

that i should've done

ddint know this model was out at the time tho

that makes sense why claude put out 4.1 opus today suddenly

[olive sable]

kinda. im fine with ai in ms paint to remove backgrounds and shit, but dont use my data please

no recall

absolutly not

[fast pagoda]

hey that's why i moved to linux

[nocturne olive]

Linux is peak

[fast pagoda]

i really dont feel like getting thought policed randomly cuz i opened the wrong shit when it took a screenshot

[nocturne olive]

If I want AI features, I can just host my own model on my own hardware and keep my own data as my own

[cosmic sphinx]

no
the og model is just too large and unoptimized
thats why we have based unsloth

[olive sable]

im moving to linux soon enough, but i do need windows for my college program so ill be dualbooting

[fast pagoda]

run windows in a virtual machine lule

[nocturne olive]

If I ever need Windows again I'm gonna just VM it

[olive sable]

nah

[glass flower]

ollama can't run the gguf version of it

[olive sable]

im doing a game-dev program, i dont want the performance to suck as

[fast pagoda]

for a non-graphical application idk why vm wouldn't be more than fine

be the one to make it work in proton

//wine

//bottles

[nocturne olive]

Then just run on WINE or using the Linux versions of software

[fast pagoda]

why run windows when i could have a system clock THIS verbose

[sick owl]

Okay seems like the unsloth quant is fucked

GGML org it is

[fast pagoda]

tis a bit drunk

[cosmic sphinx]

why thebfucj jm downloadkng this thing

[olive sable]

i will need Visual Studio Enterprise, Microsoft Office, Adobe Creative Cloud, Visual studio community, Unity, Figma and Blender

[opaque sigil]

my condolences

[fast pagoda]

visual studio is the killer there probably

[olive sable]

and a drawing tablet for some reason too

[nocturne olive]

Adobe and Office are the two problems I see

[fast pagoda]

those too

[steel lily]

I wouldn't wish VS upon my worst enemy 🫡

[fast pagoda]

idk i never expect visual studio fat to work properly on linux

[cosmic sphinx]

tell us how that runs, i may consider testing tmrw

[sick owl]

Will do

[opaque sigil]

it's not like it has a reason to exist on linux with how tightly integrated it is with windows development tbf

[fast pagoda]

whhere's the lobotomized and incoherent mradermacher q2 quant

perhaps merged with a random qwen model for no reason too

[nocturne olive]

technically with both GPUs combined I could load the 120B model at Q2

[olive sable]

the lenovo tablet i got a year ago already has a pen digitiser thing built in along with a magnet wireless charger for it on the back, i just need the pen for it still cuz it wasnt included with the tablet itslef for some reason

[fast pagoda]

oh yeah definitely it's just one of the things that i expect to fit the "never, absolutely not, definitely won't be a smooth experience" type thing when trying to use it on *nix

[nocturne olive]

But not like I need it, I have no use for an LLM on my local system

[opaque sigil]

as expensive as it was, i absolutely love my surface pen

[olive sable]

the tablet itself was only 240 bucks of aliexpress, im also getting the pen on there

[fast pagoda]

there are some mega cheap drawing tablets on amazon

i wonder how shit they are

[olive sable]

if i were to get them from a local company the tablet would be 700 and the pen 140

[sick owl]

Okay, even with the fucked up unsloth quant GPT oss outperformed Qwen 3 32B

[fast pagoda]

kinda thought about getting one the other day

[sick owl]

This is the real deal

[opaque sigil]

also i sure love how there are like a dozen different standards for digital pens

can we not

[olive sable]

imo you're better of with aliexpress than amazon in terms of dropshipping

[fast pagoda]

yeah i use amazon to find the item

and then buy it on ali

lol

[olive sable]

[fast pagoda]

might as well get it from the same place

but amazon easier to sift through

[fast pagoda]

relevant xkcd

[cosmic sphinx]

honestly
how exactly is it fucked up

[fast pagoda]

gibberish

[sick owl]

Tokenisation bugs

[olive sable]

yep, even lenovo in their tablet ecosystem has 5+ different pens that don't work together. And Lenovo also likes naming things in the Chinese market differently than the western market.
the xiaoxin pad pro 2023 i got is called the tab P12 here, and its 3x the price

[fast pagoda]

i wonder how the unsloth version ended up larger

guess just from saving it in flat f16

assuming that's what was done

the mxfp4 weights wouldnt be real f16 weights but would still be slightly larger in disk i assume

[cosmic sphinx]

unsloth barely was tested so far it seems

so we were running in blind

[fast pagoda]

that's par for the course

model releases

everyone releases some form of it

fix later

[cosmic sphinx]

I mean, knowing its THE OAI model, I thought more ppl would run it thru by now

[fast pagoda]

it's been like

6 hours

hasnt it

lol

[sick owl]

Unsloth also had the sampler parameters wrong

[cosmic sphinx]

unsloth version released 40 minutes ago

[cosmic sphinx]

temp 1.0

[sick owl]

Its a reasoning model, they get weirdge with this stuff

[fast pagoda]

actually classic

i feel like this particular param thing does happen like

every time

with unsloth lol

love them but they love shipping the wrong inference params

[rigid snow]

ok is the 20b good at whatever nerd slop they trained it on

math coding whatever

[fast pagoda]

probably p good for programming but idk why you'd do a local model for brogramming

[cosmic sphinx]

minimal chances of data leaks to bro corpos

[fast pagoda]

blud is working on top secret code

[cosmic sphinx]

if I worked in the defense industry

[fast pagoda]

i figured out long ago that anthropic has no use for my shitty code

[rigid snow]

don’t give it an email tool

[fast pagoda]

well yeah defense industry would be a little different if u need a clearnace

[cosmic sphinx]

but even then it'd be hard to work from my own pc

[sick owl]

It can do agentic stuff like browser tasks too which is neat

[cosmic sphinx]

honestly yeah, making it work as a local agent

[rigid snow]

i’ve seen schizo gh issues filed by claude

[fast pagoda]

every time i've used something to agentic anything i'm sat there like wow i could've done this myself already

i guess i don't have that much to delegate away

[cosmic sphinx]

opus 4.1 is such a meme upgrade

[fast pagoda]

2% better on swe bench == agi

[cosmic sphinx]

consider it a sneak peek to the future possibilities

[fast pagoda]

yeah i mean i know it's poggers that it's possible i just don't necessarily have anything for an agent to do atm

other than research

i got gpt pro on a whim since i've been testing the $2 billion dollar subscriptions one by one for amonth

and i'm struggling to find things to make use of it

[sick owl]

Okay I'm getting mixed signals, they recommend unsloths samplers on some guides at their website

And then the ones I just screenshot on their github repo

[fast pagoda]

in the same article?

i figure the article would be out of date

i'd trust what's on the repo

[cosmic sphinx]

yeah no1 upgrades articles fast enough

[fast pagoda]

im malding that i tried the $200 gemini subscription right before they put out deep think mode because i got it thinkin deep think was imminent

but i was 1 month ahead

might've done the same thing with gpt5 tbh

because i see this out

but when gpt5

although sama was posting gpt5 screenshots on xitter the other day

[cosmic sphinx]

I love how sam just gave a sneak peek in that video, where bros whole desktop was cluttered with txt files

'can u move all these to trash pls thx'

Blud press Ctrl+A and Del

[fast pagoda]

ngl ive had claude code do that same task

"help, too many files, fix plz"

i guess i do have agentic tasks i just give them all to claude code

and dont consider them that

because they dont need to use a visual thing

they just do it cli

[cosmic sphinx]

on the other hand, if like 70% of the 100 files on desktop were empty, but the other 30% had important content, then if you could trust the agent to look which had the content and save them instead of deleting, that'd be cooler of a demo

altho even that is sorta ez when u just sort by file size

[rigid snow]

reminds me of the promptboard thing i made *presses cmd-space* “textedit pls” *waits 10 seconds for it to type “textedit”*

[fast pagoda]

i did that w/ claude code with a bunch of versions of something i'd written but kept saving in different directories with useless differentiators for a name

so i had it open each one and figure out which was furthest ahead

and then order them in order of like

up-to-date-ness

[cosmic sphinx]

its nice that for this model u can select the reasoning effort, like for o series models

if that acc gives some improvement

[fast pagoda]

well

max reasoning can actually cause some degradation

depending on the task

so that's what i imagine you'd use it for

low effort for low effort stuff

max for hard stuff

if the task at hand is too simple sometimes they overthink on max reasoning mode

[cosmic sphinx]

"What is 2+2"

Thought for 3 minutes 48 seconds...

[fast pagoda]

7

cuz it considered 48 different possibilities of what you mean by 2+2

[sick owl]

Not experienced the same bugs with the ggml org quant

[sick owl]

Lemme guess, high reasoning effort

[fast pagoda]

and came to the conclusion that you're actually tricking it 3 ways

"the user is asking a simple math question, so the answer would be 2+2 = 4

BUT WHAT IF...."

[sick owl]

Classic

[fast pagoda]

get the answer

every time

[olive sable]

simple math question so 2+2=4
but what if the user lives in a non-Euclidean dimension?

[opaque wharf]

What am I looking at here now?

[fast pagoda]

ai slop schizophrenia

[cosmic sphinx]

discussing openai open source slop

[opaque wharf]

Nice

[fast pagoda]

that's what OSS stands for

in gpt-oss

not open source software but open source slop

the article on hf is titled Welcome GPT OSS, the new open-source model family from OpenAI!

family

unless just meaning 20 and 120b

maybe more inc

at some point

[olive sable]

no no no

wait wait wait

[fast pagoda]

sam's in high reasoning mode

[olive sable]

thats me, thats my pfp

interesting

didnt know i had that emote

[fast pagoda]

ur kirito??

[olive sable]

no

cid kagenou

[fast pagoda]

oh

wow he's the same guy

that's crazy

ive never seen eminence in shadow

[olive sable]

you know how these days you have power fantasy isekai slop?

[fast pagoda]

are u aware ur oshi is a kirito clone

[olive sable]

the eminence in shadow took that concept and ran with it to such a high degree its become one of the best recent power fantasy isekais

he's not just a kirito clone, he's the kirito clone

[fast pagoda]

isn't that just overlord

[olive sable]

well

overlord takes itself seriously

this one doesnt

[fast pagoda]

does it

[olive sable]

mostly

[stark needle]

Gpt oss is so bad someone pls kill me the training is going so poorly compared to any other model pls

[sick owl]

Hmm

Disappointed in oss 20b so far

[fast pagoda]

i imagine for fine tuning that something trained in weirdo mxfp4 weights and f16 mixed would be fucked up using finetuning tools not expecting that

[sick owl]

Ima give it time to see whether there's just bugs that need smoothing out

[fast pagoda]

me every time i try a 20b class model

[glass flower]

kinda funny if there are bugs.. they already delayed it

[fast pagoda]

which is why i gave up on using local ai for anything real until i have wayyy thiccer hardware

[olive sable]

the eminence in shadow literally has characters named po tato and skel etal
also the mc has alter egos names john smith and mundane mann

[glass flower]

qwen3-coder 30b is good for local ai

it barely doesn't fit into my gpu vram... but it runs fairly quickly

the normal qwen3 models are also good local ai's

[opaque wharf]

Pete Saman

The guy who delivers pizza shall from now on be called Pete

[olive sable]

[sick owl]

I still consider Mistrals 20b class models stellar

[fast pagoda]

ok this may be worth watching nvm

i will no longer shit on eminence in shadow

[olive sable]

the bad guy is literally named perv ass hat

[fast pagoda]

reminds me of one of my favourite manga of all time

thinking of what it's called i cannot name it here though rip

[olive sable]

[fast pagoda]

tfw rsync'd 2TB to a new storage drive and then thought let's verify the integrity real quick, that's a good idea

and it's a HDD

it's been reading for sooooo long

[stark needle]

It is

Bad

I have it finetuning on a h200 rn

[fast pagoda]

how's the graph looking

loss

[stark needle]

Bad

[fast pagoda]

kek

[stark needle]

Like a 4b model

Higher a bit

[fast pagoda]

high then super suspiciously low forever?

and then it comes out way overcooked

that's been my experience

with wrong params anyways

[stark needle]

Nah

I have robust strategy

But

[fast pagoda]

not robust enough

[stark needle]

Loss is higher like

[fast pagoda]

[opaque wharf]

Why is there so many orange here

[stark needle]

Orange

[fast pagoda]

if not orange

why

[opaque wharf]

We need more green

#programming needs to touch grass more

[olive sable]

ye fair

[opaque wharf]

You're blue and already visit beach many times

[stark needle]

2.3 vs 1.4 loss

1.4 is qwen 3 30b a3b

It's a big diff

55 vs 70 accuracy

Im trying to see if i can fix somehow

[fast pagoda]

2.3 is high

also depends kinda on the model

unless the test inferences are also schizo

wow pop! OS i thought would be running cosmic by default

and it looked like it

but it's gnome

[stark needle]

Both are active 3b so

[fast pagoda]

and just looks exactly like cosmic

well, the other way around

they made cosmic look exactly like their gnome flavour

wow

twitch L

does this if launching on zen browser

cant figure out that it's just ff

[opaque wharf]

Am I missing anything by using vanilla firefox?

[fast pagoda]

not really

i just use it because i like the vertical tabs

and ff can't currently be configured that way without a lot of addon slop it seems

because i tried

[opaque wharf]

???

[fast pagoda]

well yes you can have that but it wont let you jam literally everything in the sidebar like zen

[opaque wharf]

Bro, wtf are you doing to the poor browser now

[fast pagoda]

i dont even have a top bar

cuz i wanted it to be like a portal type thing so there's nothing on the top of the window, it's all on the sidebar which is hidden

[opaque wharf]

Ahh, I see

More docs should be written in middle English

For yon other intricate joint forms, such as the ball joint, we remain in discourse regarding its potential implementation. Yet, a series motor doth rank last upon the list, owing to the gimbal lock that dost constrain the true movements attainable by a mortal.

[fast pagoda]

i wouldn't want king arthur to be unable to comprehend my documentation that's true

[cosmic sphinx]

its the #2 isekai comedy after konosuba for me

[fast pagoda]

well mucho loved konosuba

speaking of which i need to watch the megumin spinoff

see this is why i use zen cuz the entire ui can be blasted onto this left bar

and then it's like this without that hovered

so my space is used for the actual page

ignore my too-thicc border

i need to fix that

[opaque wharf]

I honestly like the minimalist look. But I use a vertical monitor anyway so I have a lot of vertical space

[fast pagoda]

so do i

on the left side

that's actually the main reason

because it was a pain in the ass

if i'm in mega slouch 3am mode

to get the mouse up to the top

lool

so i was like fuck that i want everything in this sidebar
and ff wouldn't let me

and now i'm coping

okay..... maybe twitch is mad that my user agent says linux? this is updated ff wtf

reeee what is the latest windows ff update version

[cosmic sphinx]

funniest thing about this model might be the safety settings that they cooked into the model

[fast pagoda]

wait did they really

i mean

i guess they always were gonna cook safety shit in

but like you can adjust it at inference time?

[sick owl]

Yeah this release is rough

[fast pagoda]

natively

[cosmic sphinx]

yeah im reading all about it

[fast pagoda]

woof

[cosmic sphinx]

its disgusting

[faint ferry]

I'm having a little trouble with my AI, which I coded, and it's having trouble using expressions in VTube Studio. Can someone give me a little help

Im runing it in python

[olive sable]

uhhh

im not an ai dev but i feel like we'd need more info about how your system works

[faint ferry]

It uses open ai

gpt

[opaque wharf]

firefox 141.0-1 on arch

So that seems the latest version

[olive sable]

well ye, but how does the expresion system work?

[fast pagoda]

i won't even do it with ff

i think it's just mad because it's reporting ff for arch or something

but even if i spoof ff

it says no

and if i use ff

spoofing windows ff

no

i guess they dont want me to ever log in

it literally prevents you logging in at all

[opaque wharf]

I don't have twitch so can't exactly reproduce it lol

[fast pagoda]

what the fk

[opaque wharf]

I think toast encountered that issue once

Or was it tsurai?

[fast pagoda]

oh shit suddenly 46 updates on paru

time to break system brb

[faint sandal]

#programming message

[fast pagoda]

maybe it's mad about the html5 player i have installed

never had that issue before with it tho