Horizon Beta

[trim blade]

about my usual stuff

[pale folio]

@verbal leaf is it better

[spice shell]

Idts

It’s got slightly better coding style

[rare terrace]

I think worse than alpha

[spice shell]

But same intelligence level

[trim blade]

maybe a smaller total size but same ish active params

[next jolt]

the gpqa score is much better than non reasoning alpha

[spice shell]

It’s probably like +/- 5% in various domains if I had to extrapolate

[crystal scaffold]

was there anything interesting in the last 1000 messages in this thread or no

[spice shell]

Different weighting of post train probably

[trim blade]

hmm

[crystal scaffold]

bulbasaur goat

[trim blade]

that might explain the 100 different weights uploaded

maybe they trained a bunch of versions

alpha was better imo so far

[spice shell]

So far it has also failed to implement working chess, but had some good ideas I’ve never seen any other models try when just asked for “full rules support”

• 50 move limit stale mate
• white/black move timer

[trim blade]

the reasoning version that existed for 30 mins yesterday sucked though

[spice shell]

Um no

It was SOTA gpqa diamond

[pale folio]

we are so mentally unstable

[trim blade]

it completely failed at my tasks

BUT to be fair I had the temp at 1.0 / top P at 1.0

[spice shell]

Idk it succeeded st all my basic ones

[eager kiln]

has anyone given it a shot at creative writing?

[trim blade]

later I changed those to lower

[spice shell]

Where the previous one did not

[dusky kelp]

So dis the new thread

[crystal scaffold]

brick react the announcement fr

[spice shell]

@past sphinx “rerun your benchmarks” I’d warn folks that they can get model-banned for high concurrency benchmarking…..

(Happened to me and at least one other)

[safe imp]

Same MMLU-Pro score

[limpid vale]

is kinda higher

[trim blade]

goes both ways

so yea, prob just another run of the same model

[next jolt]

they shouldnt ban you instead they should tell you theres a concurrent rate limit...

[mental cobalt]

it's sort of just
within the noise difference
doesn't feel like much of an improvement

[solemn plover]

Wait so what’s this model all about? And why didn’t we see any teaser of some sort for this lol

[limber lance]

Its writing output is nice and consistent at least

[rocky trellis]

I'm not yet fully tested last model yet 😴

[trim blade]

alpha's creative writing is amazing

need to test beta more

[hardy fjord]

passed all tool calling

[spice shell]

Someone run eq bench on this model

[safe imp]

Might need a higher tolerance, since my subset of MMLU-Pro is pretty small (80% of questions removed)

[next jolt]

run mmlu redux

its small 3k

correlates well with mmlu, qwen/etc uses it

[next jolt]

i will run it later if it keeps working for me 🤔

[timid delta]

almost exactly inline with horizon alpha on simpleqa so far

[mental cobalt]

crank the reasoning budget to max pls so that we can actually get a good opinion of the model because rn it's GPT-3.5

[swift blaze]

I thought it was free, are there limits ? It says I have used up my credits

[spice shell]

I think this model is not worse than Alpha v1 fwiw in all my questions, just slightly different here and there. And;

• Slightly better coding style
• Uses code blocks / proper formatting by default

[late onyx]

did you have web sarch on?

[rocky trellis]

can we say it's just same but updated?

[spice shell]

seems like that is the case yeah

[swift blaze]

I tried both

[spice shell]

I am curious if this is just a different variant of the model that they’ve had prepped, or if they’re doing some kind of 2-day turnaround post training lol

[dusky kelp]

Is the system prompt the same

[cloud lily]

hey guys I have no context but which model it is ? is it a good model ?.

[past sphinx]

Note that we changed the default system prompt in our Chatroom to ask for markdown where appropriate. (Applies to Horizon Alpha as well now.)

[spice shell]

Ooh ok

[safe imp]

@dusky kelp

[opaque dirge]

trying to run some benchmarks

[forest moth]

What’s wild is that it’s likely at least one of you is an OpenAI employee monitoring this chat for feedback

[past sphinx]

woah

[opaque dirge]

but it doesn't seem better than alpha

[late onyx]

hopefully not one of the gooners

[rough ridge]

ong

[opaque dirge]

Okay let me run fishtank promt

[opaque dirge]

Agreed

[cloud lily]

please guys

[opaque dirge]

testing

[undone burrow]

Seems to be just about as braindead as what we are used to, so most definitely OpenAI...

[dusky kelp]

Whose the glow stick?

[late onyx]

what prompt?

[spice shell]

Yeah same intelligence level imo

[trim blade]

test it so far and alpha / horizon are amazing so far when it comes to creative writing

like maybe best in class

[opaque dirge]

minimal changes as I can see

[cloud lily]

form which company ? openai ?

[safe imp]

Good at code (albeit ugly and annoying to read), good at SVGs. 8b level bad at everything else.

[trim blade]

maybe its the writing model they talked about before

[opaque dirge]

yeah

[trim blade]

Im hoping its the OS model

[opaque dirge]

Has same reaction on exported TG convo

[late onyx]

I'm hoping not; I don't want the Open Source model to be this bad

[keen cedar]

i think its actually the exact same model to see what the effect of calling it "beta" instead of alpha is

[late onyx]

It seems to act differently though

[rare terrace]

With reasoning yesterday it was godlike

[spice shell]

It’s giving notably different responses and style

[opaque dirge]

Failed fishtank test miserably

[spice shell]

It’s not identical

[trim blade]

we are not having the same experience at all then, I used alpha for hours now and I think its amazing

[undone burrow]

Oop I am way too tired, sorry. I tested it in the OR Chat, didnt even think about the results being better with an actual preset...

[cloud lily]

oh okay thank you ☺️

[opaque dirge]

previous WAS working

[verbal leaf]

FEEDBACK: GIVE US THE VERSION WITH MAX THINKING BUDGET. THIS IS ASS.

[opaque dirge]

this version is not

[late onyx]

It was good

its dumb now

[keen cedar]

is it? didnt notice a difference but honestly i ran it through like two test prompts bc its too late (in EST) for ts

[safe imp]

Try Alpha right now, it also got the markdown formatting instructions

[spice shell]

Good point

Though what I meant was coding style

[safe imp]

I see

[spice shell]

And answers to knowledge questions

[opaque dirge]

seems similar enough for me

[safe imp]

Maybe that's the juice, lol

[opaque dirge]

????

[spice shell]

It’s hallucinating in different ways

[late onyx]

i don't even konw what beta was trying to do

[spice shell]

And a bit more severely on niche ones

[opaque dirge]

For me it's just worse

[graceful kelp]

thinking maybe they raised the temperature

[night urchin]

hey! i'm back

[keen cedar]

solid chance its not btw, unless the repos that oai accidentally published were red herrings(???) the oss model's context is 128k while this is 256k
unless theyre doing some funny business with keeping a higher context version behind their api like alibaba but that doesnt really make sense

[night urchin]

so what's the consensus

or is it too soon

couldnt test it yet

[limber lance]

again, I have no idea how it can flunk basic reasoning tasks and yet give this sort of output

[opaque dirge]

I've fucking got up from the bed just to test it

[spice shell]

I do think it’s a little worse at niche knowledge yeah

[opaque dirge]

it's almost 4am

[trim blade]

yea, lower the temp / top P

1.0 was way to high for alpha I found

[spice shell]

Someone needs to make NicheBench where it quizzes exhaustive character lists from tv shows and shit

lol

[opaque dirge]

0.7 should be better no?

[trim blade]

that or even a bit lower

[next jolt]

ran some benchmarks, here's a comparison:

horizon alpha (juice 0)
gpqa diamond: 47.98%
math 500: 84.60%

horizon beta (juice 5)
gpqa diamond: 63.13% (+15.15%)
math 500: 89% (+4.4%)

[late onyx]

what the heck is 'juice'?

[blissful valley]

it seems a bit more sensible in portuguese

[trim blade]

seems like a lot of models lately need low temps or they go crazy

[undone burrow]

It is pretty censored tho, sadly, was hoping for some good rp stuff

[spice shell]

OpenAI bros injecting slang into the models

[night urchin]

they are mathmaxxing it again

[next jolt]

apparently the reasoning budget

that's what i heard

[spice shell]

Calling it juice is really funny

[undone burrow]

That is, marinara and a pretty hardcore card (for testing of censoring level only ofc ofc)

[next jolt]

that's what they call it 😭

[night urchin]

i wish they could stop making huge generalist models and make several huge segmented models

[spice shell]

Codex models are that

[limber lance]

my prompts btw were just You will now take on the role of a 12th century Sogdian merchant (who can somehow speak English) and I hail from the land of the Bulgars. I have 200 talents of silver and I need many bolts of silk, to sell in Constantinople on my journey back.
it decided to take all of those aspects into consideration by itself

[night urchin]

yeah but we only have that

[spice shell]

Yeah

[night urchin]

and some health models

[trim blade]

these models are all about generalization

[opaque dirge]

general models are the future

[trim blade]

the more they know the better they perform

[spice shell]

Need hierarchical MoEs

[late onyx]

which do you think is best?

[undone burrow]

Oop nvm prefill does good shit

[late onyx]

horizon alpha on left horizon beta on middld deepseek chat on right

[spice shell]

Route to one of 8 subject matter expert half-generalist models or something

[late onyx]

that's just MoE isn't it?

[spice shell]

MoE is token level and generally can route to any mix of the experts in the pool

[opaque dirge]

It's A LOT better with temp .7 and topP .95

[trim blade]

yea, every token is basiclly routed through the layers most suited for the task

[late onyx]

so then isn't what your describing just worse MoE?

[trim blade]

its not really "experts" the way that sounds

[opaque dirge]

Beta

[trim blade]

like topic wise

[opaque dirge]

Alpha

[late onyx]

yeah MoE is more just forced sparsity

[safe imp]

2 and 3 have a lot of visible slop

3 tried to add a roleplay element but then followed up with professional-sounding slop

[spice shell]

It would be interesting to see something where there’s segmentation within the experts, such that coding subjects always goes to effectively a standalone coding segment

[late onyx]

what does 'slop' mean in the context of this?

[late onyx]

that one is deepseek v3

[night urchin]

ethically sourced blood

[spice shell]

8.5T models where it’s really 8x 1T specialized models + 500B shared experts or something 😂

[night urchin]

the dream

[trim blade]

would perform worse than just one 8.5T with so many active params

The bigger it is over all the more "accurate" it is to what it was trained on

[spice shell]

Idk, why does Qwen 3 coder exist then?

[trim blade]

like a image that gets more lossy as you compress it

[opaque dirge]

just so you know, only code from alpha works as intended

[trim blade]

it performs ok but I never found it anything that special

and its far from a specialized model

test its knowledge

it nows a ton about most things

[spice shell]

Yeah but it’s definitely coding biased

[trim blade]

post training to make its distribution more "sharp" for coding does not mean you dont train it on everything else as well beforehand

[late onyx]

it said it was trained on like 8T tokens with majority coding

[spice shell]

What if GPT 5 is in fact this, one model and the model picker is just forcing the router to one of of the segments

😂

[late onyx]

but that still means it was trained on at least 1 trillion non-coding tokens

[spice shell]

For sure!

I mean specialized as in +20% drift perhaps in each specialization

Not 100%

[opaque dirge]

but still

[trim blade]

either way qwen coder still performs substantially worse than the big generalist models do at it

[spice shell]

Anyways, just spouting ideas

[opaque dirge]

one huge model will be better in the long run

[spice shell]

yeah

[opaque dirge]

we just do not have enough compute

[late onyx]

is it better than kimi k2?

[trim blade]

last I saw kimi edged it out there

[spice shell]

It is interesting to hear the rumor that o3 got way worse at ARC AGI once it got trained for chatting

[safe imp]

2 - "We can approach...", "First, lets get curious", dumping questions with bullet points (unnatural in a convo scenario)
3 - "Let's explore", "any specific", "calm demeanor", unnatural enumeration of quesitons, "some find" (weasel wording)

[opaque dirge]

maybe

i still remember presentation

[spice shell]

There’s definitely something to be said about the fact that subject strengths keep clashing with eachother

[opaque dirge]

where it was like 50% or arc-agi-1

[spice shell]

Yeah

That was a crazy moment

[late onyx]

I feel the slop was way worse in 2 than 3

[opaque dirge]

just give the summary

[opaque dirge]

I can't see that in non native languages

[hardy fjord]

I'll post the repo so you can see it's code.

[dark bramble]

they're thinking they should have taken that meta offer 😭

[trim blade]

meta kind of looks like its sinking atm

maybe they can turn it around, I hope so

[spice shell]

That’s what everyone said about GDM before 2.5 pro

[timid delta]

Final score for Horizon Beta on SimpleQA was 33.7%. oai models scores for reference..... and Horizon-Alpha got 33.9% yesterday https://github.com/openai/simple-evals https://openai.com/index/introducing-simpleqa/

[trim blade]

but llama 4 did not bring big confidence

[spice shell]

Someone post that image where it’s a circle of “the smartest model in the world”

lol

[late onyx]

how 'slop'py are these?

[spice shell]

Not at all

[opaque dirge]

another indicator for small model

[spice shell]

I read the middle one and it seems quite un-slopped

[late onyx]

all i changed was I added This is in a conversational setting.

[dark bramble]

they have smart people and lots of compute. the latter was true during llama 4 shitshow and it's truer now with their new hires. the right people could make meta into a top tier lab... im not sure alexandr wang is the right guy tho.

[hardy fjord]

deffo not set up for roleplay

[late onyx]

all 3?

[hardy fjord]

feels like these are coding models.

[trim blade]

hoping alpha is OS and not gpt 5 mini / nano

[dark bramble]

making the o1/o3 guy chief scientist is a good sign tho

[hardy fjord]

but their weird problem with markdown is fucking weird

[opaque dirge]

nano, for mini it's shit

[night urchin]

i'm getting really tired of this pattern of speech and the lists

out of the box

[opaque dirge]

it MUST be better than current gen mini models

[dark bramble]

maybe one of these is 120b MoE and the other the 20b (dense?)

[late onyx]

both horizon models still insisted in list formatting even when i told it it was conversational

i dont' get it

[opaque dirge]

for OSS still bad

idk

should be 20B at max

[spice shell]

Better than o4-mini but not better than 4.1 is interesting.

Lines up with rumours around OSS model

[opaque dirge]

as we've seen from the leaks

[trim blade]

for creative writing / graphic / web design alpha is amazing at least

[dark bramble]

depends on size, i don't think openai will match kimi k2 at 120b

[night urchin]

i remember when i talked to Gemini 03-25 and felt like i was talking to a human on the black box experiment

[trim blade]

at least the alpha that was around for the hours I used it

[night urchin]

it was really surreal

[opaque dirge]

if it's oss hope for 20B for Horizon

[spice shell]

0325 still has my heart

lol

[night urchin]

they are doing something wrong for sure

[dark bramble]

they really cooked with that

[spice shell]

I hope GDM brings back the magic for Gemini 3

[night urchin]

they as in everybody except moonshotAI apparently

[dark bramble]

though gpt-4 is still my beloved

[late onyx]

what model is that?

[opaque dirge]

isn't current better?

[spice shell]

A bit like Claude 3.7 -> 4 really smoothed out the model

[night urchin]

yeah

[spice shell]

Gemini 2.5 v1

[dark bramble]

vibes r off

[late onyx]

ohh

[night urchin]

i wish i had exported the conversation i had with gemini

truly made me question myself

[spice shell]

I’ve completely stopped using Gemini despite it supposedly being better in all respects than 0325

It lost the magic somehow

[opaque dirge]

I'm too self conscious to question myself when talking with AI

[dark bramble]

what if we're just biological next token generators?:)

[night urchin]

it's something no benchmark will address

[hardy fjord]

https://github.com/XSUS-AI/clickup-mcp It didn't add the readme... right before it did, provider started generic erroring out.

[opaque dirge]

it was the best at the fishtank test at it's time

[opaque dirge]

does it work?

[hardy fjord]

bout to test it, just going over code

looks like it nailed the MCP server

[night urchin]

i'm thinking it's the open source model fr, but i saw some people saying it couldn't be that so soon

[hardy fjord]

also don' thave a clickup API key atm... and quickly fading.

[dark bramble]

benchmarks don't get vibes. aidanbench was kinda close but idk

[hardy fjord]

just wanted to see what it's code looked like compared to alpha

[opaque dirge]

kotlin mcp sdk is just broken

[hardy fjord]

sucks for android

[opaque dirge]

Idk about the client

[hardy fjord]

gonna get some sleep.

[wary tangle]

my o3 and gemini 2.5 pro says alpha is better at summarizing a paper…

[hardy fjord]

looks right according to their docs

[opaque dirge]

OAI doesn't work without JSON RPC

[spice shell]

Hey openai if you’re watching make it stop outputting code golf it’s really annoying

[opaque dirge]

it doent accept sse

I'm going insane with this models testing, will probably write my own testing suite

[hardy fjord]

nah I always have it code Stdio and then just make that an API wrapper

[opaque dirge]

ah

[hardy fjord]

I came in the MCP game early and never could fully rely on MCP SSE

[opaque dirge]

why's that?

[hardy fjord]

I find it nice to just roll with Stdio and just never use anyone else's MCP servers for security

just as it's been progressing I've gotten stuck in my ways I guess, and SSE wasn't supported at first, then it wasn't secure, and people still talk about it as a security threat

but also

stdio, you can make a mixed MCP server that does stuff locally and remotely

[verbal leaf]

this bench tests a model's ability to link seemingly unrelated niche things

very interesting result

[opaque dirge]

true, but I don't need one

[hardy fjord]

like I get how easy SSE is... but I mean, I'm making agents make agents and their tools from scratch to spec here

[opaque dirge]

support bot doesn't need that

[next jolt]

oh this is a hybrid thinking model, i think i just triggered thinking mode 😭

[hardy fjord]

so I don't care really... I only ever use MCP servers my agent coded

[opaque dirge]

hah

[late onyx]

both models are confidently incorrect

[opaque dirge]

not even context7 and or exasearch?

[trim blade]

uh oh, from what I know OI does not want to release their thinking processes

so maybe its not the open model

that would suck

[hardy fjord]

if I find a good API, I have it make me an MCP

I'm using Serper, and for scraping just had it make me a tool that uses bs4

[opaque dirge]

ima get some sleep, it's 4:16 am already

[hardy fjord]

2:16 here

[hardy fjord]

but same, nice to meet you

[opaque dirge]

france/spain?

[hardy fjord]

Africa

[opaque dirge]

oh

[night urchin]

cool

does anyone know if it has updated knowledge on popular libraries?

[pale folio]

Whats the consensus

[spice shell]

Meh

[spice shell]

How?

[spice shell]

It does!

Tailwind v4 and react router v7 are my go to tests and it knows of both

[past sphinx]

Some users who had no credits weren't able to access Horizon earlier due to a faulty fraud check we had - just fixed this, so try again!

[dapper pecan]

im getting a 429

[pale folio]

Same

[late onyx]

What model?

[pale folio]

Error 429

[late onyx]

You've hit rate limit

429 = too many requests

[pale folio]

💀 💀 💀

[late onyx]

unless there is a bug with rate limiting

[next jolt]

i also hit it at the same time as them i think

[past sphinx]

Investigating

[late onyx]

mine is still working

i haven't used it that much though

is there a way to check how many free credits we have left

[next jolt]

works now

[past sphinx]

Looks like Horizon Beta is getting really hammered - working on scaling / limiting the heaviest users

[safe imp]

This model

[verbal leaf]

i'll play with it again when it REASONS

[late onyx]

do you like have a list of all the users or something?

[next jolt]

Can i add my openai api key to increase my limits? 🤣

[late onyx]

probably everyone running benchmarks on it

[spice shell]

Doesn’t seem fixed for me at least 🥲

[past sphinx]

Should be better now!

[past sphinx]

dm'd you

[lunar dew]

This model hella fast

[late onyx]

could be MoE

[verbal leaf]

https://fixupx.com/SebastienBubeck/status/1951457213920452763 and so begins the hypeposting

Sebastien Bubeck (@SebastienBubeck)

Pretty good. Chat, do you think we can do better?

Quoting Ethan Mollick (@emollick)
︀
Here is the Deep Think "Sparks unicorn"
︀︀
︀︀(This is created using TikZ, which is a language built for scientific diagrams & very much not for drawing. The original "Sparks of AGI" paper used the ability of the AI to draw a primitive unicorn as an example of unexpected AI abilities)

**💬 9 ❤️ 36 👁️ 3.8K **

FixupX · Aug 2, 2025, 1:37 AM

[undone burrow]

It ozoned chat😔

[night urchin]

drew better than me

[rare terrace]

Having a feeling it was an accidental (or intentional) gpt 5 leak

[past sphinx]

what do people think of the general prose feel compared to alpha?

[verbal leaf]

it refuses to write my fanfiction now 💔

[bitter vigil]

sfw?

[verbal leaf]

yes

😭

[fathom atlas]

Feels a lot better, understands more, doing less dumb mistakes

One shotted todo app with backend + db and also for apple notes clone

[past sphinx]

Awesome!

[past sphinx]

:/ any other details that you can share?

anyone have other regressions caused by the beta?

[jade kayak]

Decent at UI, but not anything complex like 3D games. limited world knowledge. Pretty fast

[ancient hedge]

Oh. My. God.: https://x.com/Jordy_vD_/status/1951473109183439138

Jordy.app (@Jordy_vD_)

Wtf -- it added easter eggs??

Also, this website looks dope asf

X · Aug 2, 2025, 2:40 AM

Given a simple test prompt, it created design files via Roo Code, wireframes, and it even added easter eggs to the website 😂

[fathom atlas]

One shot authentication, and this time it did it without having to tell it to work with backend, just did it.

[ancient hedge]

Idk what provider this model is from, but if this thing works well with OpenCode when it releases, or Codex if it's OAI, this could be amazing.

[upbeat jewel]

403 - Blocked by Stealth.... Ouch

[ancient hedge]

I'm not one to fear for my job, but I have no idea how to build some of the effects it added 😅

Also someone from OAI just liked my tweet so mayyyybe

[gleaming lantern]

Just want reasoning back 😔

[verbal leaf]

real

[gleaming lantern]

The text predictors want to predict

[verbal leaf]

alex pass that feedback along

wink wink

[mental cobalt]

real

[silent heath]

this is problematic actually (unless told to go above and beyond)

[ancient hedge]

Might have been Roo Code telling it to

And who doesn’t want above and beyond 😜

[gritty glade]

It seems good imo a little repetitive at times though

[cold knoll]

Its UI Design has gotten a fair amount worse, But the backend development feels much better and it is following our CLI's prompts much better.

the Alpha model continuously asked "Please repond with "x" to continue"

"I dont currently have permission to use tools, please say "i grant you x y z" to allow me to use tools" etc, Which seems to have stopped and the model is now adhereing similarly to claude opus/sonnet

[fathom atlas]

Huge thanks to @past sphinx & the OpenRouter team for hosting these stealth preview models.

Honestly feels better to use than Sonnet 4, super solid. Can't wait to use the final model.

Dropped what I built in app showcase: #app-showcase message

[leaden sinew]

So, anyone knows who's model was Cypher Alpha?

now Horizon Beta?

because you are feeding someone, whoever is using it

[leaden sinew]

ooh

good old RSA

[green parcel]

How's Beta compared to Alpha thus far 👀

[fathom atlas]

lol

[fathom atlas]

Really good, this model slaps.

[grave shore]

I certainly hope this isn't the creative/writing model OAI was supposedly cooking up.
Because I find it pretty lacking from the short time I used it.
Can't speak on Alpha vs Beta, found them pretty similar.

[trim blade]

I found the exact opposite. It's the best creative writing model I've ever used

It dethrones opus

It's amazing at building scenes and writing out characters in intelligent ways

[grave shore]

That's funny, but really not a sentiment I can share lol

[trim blade]

Reminds me of gpt4.5 for the short time that existed

[grave shore]

I'd like to specify that I'm using it for RP'ing
I'm not sure how well it handles longform writing

[trim blade]

Maybe that is the difference? I do long form writing

Feeding it info on a setting/ characters

[heady gust]

I've gotta give it some time to decide, not blown away or anything but it's certainly good

[trim blade]

and its far more nuanced / less on the nose than anything else so far

it understands on a deeper level if that makes sense

[heady gust]

I want to see how the reasoning variant does on some of my challenge questions

[trim blade]

social intelligence is higher than anything else I have seen

[heady gust]

Nonreasoning isn't able to get them but that's pretty much to be expected

[trim blade]

Try maybe grabbing a scene from your favorite book and tossing it in

tell it to continue it

[grave shore]

I seriously don't see it when using this model.
It's what I've always seen from the big models, Opus, 4.5, sometimes 2.5 pro.

I'll give it another try with some different scenarios.

[cold knoll]

@past sphinx the model actually still fails to understand it can call native tools. It works up until around 60k tokens then it refuses to use tools that it has been using all chats

[trim blade]

sounds like the usual context drop off issues

most models start getting progressively dumber at longer contexts

past 32K for most models really

[cold knoll]

Yeah but 256k context, only able to use 60k context without forcing the model into an infinite loop...

No issues on google models, anthropic openai etc, even local

[trim blade]

Every model claims high context but have massive performance drop offs

[gritty glade]

I disagree with you there, it is good but claude is still better

[trim blade]

I dont see it

[cold knoll]

yeah thats a crazy statement, i can use opus max context no issues with a rolling message window, this shit i can barely use 50k

Consistent output past 100k, best under but still functional

[trim blade]

I've used nothing but claude and some deepseek for about 2 years now

since claude 2 to claude 4 / opus 3 / 4

and this model is amazing me

for the past 5 or so hours I spent

[gritty glade]

It could be the novelty that makes it feel better, but then again it depends on what you like

Horizon is def good compared to most other models but Claude is still better in terms of dialogue flow and keeping consistent with lore imo

But if horizon is cheaper than 50c a message it would win out for me tbh

[cold knoll]

i agree with that

price will say everything

[gritty glade]

• with reasoning i think it could be better than it is currently

[trim blade]

Btw, I should say alpha is the model Im enjoying for creative writing

beta seemed worse to me

for that at least

could always be just some luck of a bunch of bad gens but it felt way worse to me so I switched back

[rough relic]

I'm just curious, will alpha/beta stay free?

I'm a bit new to this

[bright oak]

probably not. everyone gets to have fun talking with them but they use the data to train.

[trim blade]

its either going to be gpt5 which will hopefully be cheap going by the speed or hopefully, maybe, it might be the OS model?

seems a bit too good to be true there

[heady gust]

It'd be nice if it was the OS model but I have a sinking feeling it's probably not

[trim blade]

but who knows

[rough relic]

Any alternatives to it, which are free?

[cold knoll]

I think this is Grok 4 coder

this dogshit feels like it was trained on cline/roo which would make sense as to why it keeps asking me for approval to use tools, as Grok was trained on Cline

[rough relic]

Isn't that paid?

[cold knoll]

No, Horizon is a free promotion to test a hidden model

its for AI Companies to test their models before release to see how they perform in the real world

[rough relic]

Oh

So is this gonna be grok 4 then?

[cold knoll]

No one knows, but it would make sense

[dusky kelp]

Hmm, wonder why beta did worse on aider

[rough relic]

What would be like the closest model to it, if it's like a completely free one

[cold knoll]

Sadly thats not how AI model releases work

free models cant compete with top of the line releases, and it depends on your use case

this model is free temporarily

use it while you can

[rough relic]

Uhh, mine is very niche

[trim blade]

kimi k2 is ok

[dusky kelp]

might be open ai open source model

[rough relic]

That won't really work

[dusky kelp]

kimi k2 is really great

[trim blade]

GLM4.5 is better but I dont think a free version is up

[gritty glade]

I didn't like kimi that much tbh

[trim blade]

its super cheap though

[gritty glade]

GLM 4.5 is ok yeah

[rough relic]

Nah not free wont work

Cuz I need to use it like a lot

[trim blade]

I use these models a ton

[rough relic]

I mean a lot of users will actress it

Access

[trim blade]

glm4.5 is super cheap

[dusky kelp]

nothing but good results so far with kimi, i guess long ocntext it can have a hard time, but i bet that can be fixed with 2.1

[trim blade]

like you could prob spend like $10 a month cheap

[cold knoll]

if users will access it it will never work being free

[rough relic]

They will get their own api key

Also I tried deepseek chimera, that was decent

But I need a model which can use my context

I basically have a lot of text to it as a context

[cold knoll]

then use a gemini model

[rough relic]

And only horizon is able to use it properlt

[rough relic]

That's paid I think

[dusky kelp]

gemini is the only model good at long context, claude is pretty good, and so is gpt 4.1, but gemini models are always way ahead on context size and quality

[rough relic]

Uhh, even qwen 3 had enough context

So context isn't really an issue atp

The issue that's it's not understanding it

[cold knoll]

uhuh, free models wont work the best for what you want to do, You're likely doing some roleplay shit or long context retrieval, these things cost money to run, and no model is free forever unless you run it yourself, then... up goes your electricity bill by 5x

[dusky kelp]

yeah, thats what i mean by quality, gemini can reason accross long context, most models are not good at that

[rough relic]

Yeah, but I need a free model

[trim blade]

gemini has so many uses free a day

but I do not like it for writing myself

[dusky kelp]

Im saying that gemini is the only one that is good at long ctx imo, not that i know of a good free option

[trim blade]

very dry model

[rough relic]

What do you use?

[trim blade]

and a bit dumb compared to sonnet / this new gpt

before, sonnet 3.7/4/opus

Im really really liking horizon alpha atm

I used deepseek some as well

kimi was ok, GLM4.5 is good

[rough relic]

I might try glm 4.5 air

[trim blade]

nah, air was way worse

I mean glm4.5

oh wow I never noticed how cheap it is $0.20/M input tokens
$0.20/M output tokens

you could prob spend $10 and use it non stop for a month

[undone cypress]

better than sonnet 4? 👀

[rough relic]

@trim blade I'm a bit surprised how good this model is with context, I think it's a gpt model, is there a way to use any gpt model for free?

[devout copper]

How is the model doing?

[gentle sentinel]

Kinda meh

[trim blade]

for writing its worse than alpha

imo its a bit dumber than alpha

but apparently some benchmarks people did here said differently?

[visual rose]

whats next? horizon gamma?

[undone cypress]

i like horizon gamma

[iron tartan]

I’m pretty sure this is an OpenAI model

I asked it to write up acceptance criteria for a user story and this was one of the bullet points

Chips of sample phrases (e.g., “HELLO WORLD”, “OPEN AI”).

It suggested openAI

It could mean nothing, it could mean something, time will tell

[lavish epoch]

We literally doing RLHF for OpenAI 💀 💀 💀

[iron tartan]

This feels like a small model

GPT-5-nano?

[sinful orchid]

Guys

How is the rp?

[trim blade]

alpha is much better

[grave shore]

Can I ask you what temp and other parameters you have set?
Because I'm not kidding, I'm having a seriously mediocre experience with this model.

It's repetitive, it makes characters act out of character, the way it connects concepts is okay, nothing special

It just seems weird to me that long form writing for you is such a different experience than the RP'ing is for me.

Like, both mediums should be testing for pretty similar things.
(I've tested both Alpha and Beta)

[trim blade]

0.3 temp 0.3 ish top p

[grave shore]

Huh, okay, that's a lot lower than what I'm using, I'll try it out, thanks.

[trim blade]

im also using a JB

like all gpt / claude models it needs a JB to write well / get rid of that positivity bias

[grave shore]

Oh yeah, definitely

I think the only model you don't need to wrangle the positivity bias out of is 2.5 Pro.
That model is straight up depressing sometimes

[gritty glade]

2.5 pro is a bit too negative funnily enough

[iron tartan]

This is really interesting to hear discourse about writing since I’m usually looking at models through the lens of how well a model can code

[dapper pecan]

I’m also voting alpha for the better writer.

[grave wyvern]

Alpha is giving me more censorship. I tried to get it to talk about the recent 🇰🇷 shenanigans and they both erred, but alpha was harder to get to talk imo

[gritty glade]

Do you mind sending me the preset you are using?

[lavish epoch]

I just did a quick test on frontend visualization task.
two tries, both have some bugs that cause it to be unreadable, worse than horizon alpha.
won't be testing further.

[grave wyvern]

i'm quite impressed by it, will be interesting to see where it is priced

[crystal scaffold]

i modified the strawberry test a little

(sometimes it said 6, sometimes 5, for that prompt)

modifying it helps with the risk of overfitting, but i havent really noticed signs of overfitting tbh. (i am not an expert)

[copper mountain]

wat 1500 messages overnight here 💀

[warm brook]

this is one of the most positive aligned models I've ever seen

this and alpha

[ashen hamlet]

I can't even do that one.

[late onyx]

doing that makes it a bit easier actually, as it would token split it to be smaller groups. The strawberry challenge was difficult as the AI sees "straw" "berry" or potentially even "strawberry" but with your example it would see (using OpenAI tokenizer) "stre" "ar" "we" "ebe" "erry" which is easier as they're shorter

[patent gale]

i keep getting 400 errors on horizon models

[fringe bay]

its all fine

left: fishtank with sonnet, right: fishtank with beta

[rare terrace]

Billy jean is not my lover

She's just a girl who says that Iiiii am the one

But the kid only reasoned for 3 hours

She says IIII am the one

Is there any official explanation to that

Perhaps the horizon models are actually gpt 5 and they were worried with us thinking that the reasoning performance comes from the OSS model, so they shut it down?

[thick osprey]

Isn't Horizon Beta supposed to be free for now during the testing period? It says the same and shows $0 for input and output tokens, but I am still being charged for it.

[deft cliff]

Has anyone tested fringe knowledge? Like Finnish language proficiency? I can't speak Finnish myself but I heard that basically all current models fail at it

[lavish minnow]

It is free are you sure in the activity you're being charged?

[lavish minnow]

Can't speak Finnish but I'm a french translator and it is really good with french, among top models in my opinion. And follows instructions perfectly

[warm niche]

On every model I try to generate Russian small poem about kitten. So far only Claude been able to make it somewhat decent. This model poem is no way near

[deft cliff]

Interesting.. French is spoken by a lot of people though finish is not and it's really hard

Just trying to figure out how to test if it's really big or not. Thought it it can speak perfect finish it must be gpt5 or something because who would make an open weight model with 20/100b write finish. Seems a waste of resources

[deft cliff]

That is what I was looking for! Thank you

[lavish minnow]

Yeah French might not be the best example, it is widely spoken and pretty much all LLMs are getting pretty decent with it atm

[rare terrace]

Web search costs money i think

[thick osprey]

Thanks, yes it was because web search was on by default when I was testing in OR Chatroom.

[thick osprey]

Thanks. Yes, it is free. Just checked in Activity, like @rare terrace suggested it was charging me for the web search.

[fringe bay]

idk. it's a good agentic model that can work on long projects in cline. But for more complex projects I often have to bring in claude to fix bugs

[quasi quartz]

its already better than horizon alpha

insane

[verbal leaf]

that's not really insane

horizon alpha was shit w/o reasoning

[spiral pewter]

this model doesnt seem to reason

[verbal leaf]

well yeah it doesn't

but it has interesting behaviour sometimes similar to deepseek v3 where it will go into long CoT in its responses even though it isn't a reasoner rn

reflected here as well

[gritty glade]

I don't like it so far for creative writing

[fringe bay]

I am ever more impressed with sonnets ability to understand horizons code and fix subtle bugs

[gritty glade]

Hopefully if it gets reasoning it might be better but so far it's kinda meh

[late onyx]

Was the reasoning data with Horizon Alpha exposed?

[verbal leaf]

the number of reasoning tokens was visible but other than that no

[late onyx]

ohhh

[fathom sleet]

horizon beta seems to be a smaller model, since its responding faster then horizon alpha

but could also be because of the inference infrastructure

[rare terrace]

Whom do I bribe to turn on horizon reasoning

We shouldnt have made such a big deal out of it, they might have left it on then

I think they turned it on because we were disappointed at first

[fathom sleet]

both horizon alpha and horizon beta arent able to give me correct rust code in 1-shot

so i am assuming its not gpt-5 but their open source model, or some smaller model

[spiral pewter]

they were giving us a glimpse into the future

[fathom sleet]

(ok but then again o3 also fails this test, so maybe i was wrong)

[tender cairn]

horizon model the open source model?

beta the larger as its supposeduly better?

tried vibe coding w/cline. perf not shocking or nyhting

tbh considering openai engineers use claude code internally is gpt5 going to trump sonnet at coding/

ok ui design is cheeks

[trim blade]

ui design is cheeks?

alpha at least was great there

[verbal leaf]

eeeh

it's great on paper but if you actually look at it properly it kinda falls apart

it's always the same "template", same style, etc

[glacial ice]

hmmm, does temp or top P even have an impact on horizon

atleast temp seems to be locked, the settings entirely at that

and the outputs tend to be samey

[tender cairn]

do you know any good MCP's that get around this issue? that provide it solid components, i've tried magic design mcp which is okay

[tender cairn]

yeah ill try alpha

i mean the agentic capabillites is good ig alpha design much better

kimi k2 stronger than these models thouh imo

[rare terrace]

I tricked alpha into generating CoT

I think the results are similar to what it was during its reasoning phase

I believe this might be it?

Anyone got a benchmark they could test

?

Where they tested the dumb horizon alpha

And the smart one in the 3 hour period

So that we can compare

[tender cairn]

hiw?

sequential mcp?

[rare terrace]

I added this to system prompt Every time a user sends a request, reason deeply through the task, delimiting your thinking with tags, starting with a <think> tag and ending with </think>. Only after you're done with the reasoning, may you attempt the task

Its answers improved

lol

Even the knowledge task i gave it

I want to know if that's really all it was

[tender cairn]

yeah but thats not TTC, would still improve answers though

[late onyx]

It’s still CoT though, a more advanced version of the “Think step by step” prompt technique

[tender cairn]

yeah i know, thats why i said itll improve answers

[rare terrace]

Someone had some sort of vision benchmark where they got to test both reasoning and non reasoning horizon alpha

I want to have them run it again with that system prompt

[tender cairn]

wait there is reasoning on alpha?

[rare terrace]

There was

For 3 hours

Then they turned it off

[tender cairn]

ohhh

[rare terrace]

It's still talking about its hidden chain-of-thought

Even though it doesnt seem to be reasoning

[tender cairn]

Cos of the RL im guessing

[glacial ice]

Hmmm, for writing, horizon seems samey, its always the same kind of scene rewritten, maintaining same structure

[tender cairn]

small model then?

[glacial ice]

could be also because settings are locked

changing temp or something else has no effect

ANd of course it has the No/Not (something) 2x. Just (something) slop.

[tender cairn]

its def a small model

small models suck at writing

the weights were leaked as well around the 150b range

[fathom atlas]

For my current use case yeah for sure, better code, faster speed.

[glacial ice]

Probably there'll be a significant difference between this and official, just because settings will be unlocked

[warped coral]

is the max still 1000 requests?

[past sphinx]

https://www.reddit.com/r/LocalLLaMA/s/988FwNEJ6a

Cool comparison

[spiral pewter]

that is one sexy website

[past sphinx]

Anyone have examples of Alpha writing better than Beta? cc @gritty glade

[spiral pewter]

does horizon beta have vision?

yes

is this a pre-filled system prompt?

You are Horizon Beta, a large language model from an unknown provider.

Formatting Rules:
- Use Markdown **only when semantically appropriate**. Examples: `inline code`, \`\`\`code fences\`\`\`, tables, and lists.
- In assistant responses, format file names, directory paths, function names, and class names with backticks (`).
- For math: use \( and \) for inline expressions, and \[ and \] for display (block) math.```

[gritty glade]

Lemme do another swipe for both

Vote for alpha tbh, or are you chasing screenshot examples?

[patent grail]

IS it building an auth system from scratch, or implementing an auth library?

[fathom atlas]

Scratch

[clever arrow]

Not bad. I asked for an A2A compatibel Agent with MCP funtionality without using any kind of Framework. It produced one:

[patent grail]

Is alpha generating cached tokens too?

[fathom atlas]

Horizon Beta re-making the OR site

[patent grail]

Nice, just image input?

[past sphinx]

Looking for prompts or screenshot examples

[gritty glade]

If you are okay with me DMing I am fine with sending you a comparison

[hardy fjord]

They got air version free on glm 4.5 iirc

[pseudo jolt]

Either google or claude

There also possibility of xAi

but i dont think this one is from openAI, but i could be wrong

[gritty glade]

I don't think it's gemini

Maybe claude

[glacial ice]

didnt testing show a large similarity with o3?

[native totem]

anyone got this? I always got this 502 error, whether through API or chat.

[glacial ice]

from multiple users that is

[fathom atlas]

Yeah

[patent grail]

That's very impressive then; can something like Claude even get that close in img -> code?

[fathom atlas]

Claude does well too, I'd say they are equal when it comes to re-creating UIs from an image. The problem is that claude takes 4x the time and costs a lot.

[native totem]

Any admin here? I need to solve this question. My main account can never use this model. But the latest registered account can use this model normally.

[raw blaze]

It is an OpenAI model and very likely related to GPT-5 (possibly a mini or nano variant). The tokenizer is 99% certain of this—achieving 100% certainty would require analyzing several recently public-tested models:

• lmarena - zenith
• lmarena - summit
• openrouter - horizon-alpha
• openrouter - horzion-beta
• !! perplexity leaked - gpt-5

All of their system prompts share an identical segment. Notably, there’s a value called "juice" that controls the length of the chain of thought:

• lmarena - zenith, juice = 64
• lmarena - summit, juice = 200
• openrouter - horizon-alpha, juice = 0 (it was temporarily set to 100 for a very short period)
• openrouter - horzion-beta, juice = 5
• perplexity leaked - gpt-5, juice = 64

[trail gale]

Horizon Beta seems lot more censored than Alpha. I did think it was weird just how open Alpha was for whats likely OpenAI product.

[sand pulsar]

Agreed, and the refusal responses are very different from the traditional openai style. They could be planning to switch the refusal style, or could be just for this stealth variant.

[trail gale]

If it is indeed the open source model I would really wish they just let it be without any native filtering and just let users and providers add filters themselves if they so wished. I dont think its too controversial to want open source stuff to be fully open.

[limber lance]

it's quite awful at these tasks
(the only one it got right out those 4 is Empress Suiko, but she should have been mentioned in Bidatsu's entry as well
particularly egregious is claiming Bidatsu married Ishi-hime, who was actually his mother)

[trail gale]

Yeah, havent really seen this refuasl style before. It actually took me a while to register Im being filtered, so I then did Alpha vs Beta direct comparison. Unfortunately this makes Alpha way more usefull for people like me (horror fiction writer). I think Im gonna really miss Alpha when it leaves Open Router. Hopefully someone makes a Dolphin-esque fine tune.

[sand pulsar]

I think it's too early to jump to conclusions that alpha and beta are 100% related.

There's a chance that alpha is targeted for open-source release, while beta is targeted for closed-source release, or the other way around.

We can only speculate and infer, and there's also a possibility that OpenAI are still figuring things out. Not uncommon to crystallize what's what only hours before the public release.

[trail gale]

Its stated on Open Router profile that its a "improved version of Horizon Alpha". But yes, definitely will change still for final release

[storm hill]

using the same checks as I did for optimus/quasar, Horizon is very likely to be hosted by OpenAI

[glacial ice]

that was pretty much clear when openai kept claiming they would do an opensource model, and delayed it without saying anything when kimi appeared

[bitter estuary]

Is the primary theory here still OpenAI's open-weight model, presumably one focused on creativity/writing/EQ?

[trail gale]

Im now near certain its OpenAI open weight model. And whether or not creative writing was their actual focus, it does seem that that is what its best at.

[glacial ice]

too bad it seems likely they'll attempt to make it safe

[bitter estuary]

Gotcha. And I mean when apparently it blows ass at code and reasoning, but tops the charts in EQ and creative writing...let's hope that was their goal lol

[tender cairn]

if it has a unique type of refusal it is 100% openai. they have the most to lose with open source

[bitter estuary]

I'm kind of surprised they are even releasing one. Are they expecting it to make a hiring difference? It's not like any CS or math PhD is stupid enough to go "I want to work on open-weight, so OAI is apparently the place for me!" Maybe just general PR?

[tender cairn]

general PR cos they were originally open years back

[bitter estuary]

Or, come to think of it, maybe a lawsuit defense?

"We are open, we're just responsible with releases!"

[sand pulsar]

There are so many reasons why they'd release an open model, and why not.

[trail gale]

Its super fast, which could mean really small (or maybe just tonne of processing power). If its small enough you can run it or mortal human GPUs comfortably, that would make it very valuable to someone like me.

And yeah, it also lets them pretend they are "sort of open"

[tender cairn]

they did mention having an o3 mini size model on a phone was something they were thinking abuot

[fossil ibex]

So is it normal for me to get a negative balance even though i was using only Horizon Beta?

[rare terrace]

Did you enable web search? Web search costs extra

[fossil ibex]

Ahh yup that was it.

[heady gust]

They mentioned a phone model or a small o3 mini level model here

Now that XBai o4 did literally that, we'll see what comes of it I guess

Well allegedly, haven't tested it

[modest crescent]

i mean

he did say the oss model would come

during the summer

so either beta or alpha could very well be the oss one

[steep palm]

Hmm that ain't great. Most 32b models get this right, like GLM 32B without reasoning, or Gemma 27b. (the correct answer is Siberian tiger). It starts off with an incorrect answer, actually have the correct one 2/3 of the way through, ended up with a ridiculous answer ('polar bear'). Horizon Alpha also failed.

[modest crescent]

https://x.com/sama/status/1899535387435086115

Sam Altman (@sama)

we trained a new model that is good at creative writing (not sure yet how/when it will get released). this is the first time i have been really struck by something written by AI; it got the vibe of metafiction so right.

PROMPT:

Please write a metafictional literary short story

X · Mar 11, 2025, 6:58 PM

[steep palm]

GLM32B for comparison

[bitter estuary]

Ah, okay, I'm putting $10 on that then

[steep palm]

Suppose I fly a plane leaving my campsite, heading straight east for precisely 28,361 km, and find myself back at the camp. I come upon seeing a tiger in my tent eating my food! What species is the tiger? Consider the circumference of the Earth.

(The prompt if people want to test it on other models)

[modest crescent]

it did write my stuff

the best out of any other model

so it has got to be that model

[steep palm]

This part of GLM's answer made me laugh 🙂

[modest crescent]

https://x.com/sama/status/1951695003157426645

Sam Altman (@sama)

we have a ton of stuff to launch over the next couple of months--new models, products, features, and more.

please bear with us through some probable hiccups and capacity crunches. although it may be slightly choppy, we think you'll really love what we've created for you!

X · Aug 2, 2025, 5:22 PM

horizon-mega next

[tender cairn]

yeah they probably did both

[glacial ice]

wait, what if they do GPT5 through openrouter too in the same manner?

[modest crescent]

if this one isn't gpt5, they prob will

[glacial ice]

Though probably not happening, they got big teams for that

[modest crescent]

i see this current model

mostly trained for creative writing

[glacial ice]

plus corpos more than willing to test out GPT5

[modest crescent]

so i don't really see it being gpt5

[glacial ice]

yeah, it obviously is not

leaked to be 20B & 120B

[modest crescent]

this being the oss one would be such a big win for the community though

creative writing wise

it gets so much shit right

that no other model has done yet

[glacial ice]

eh, it does dumb stuff though

but writing it does seem to be different

tooo bad we cant tune settings

[bitter estuary]

EQ Bench does say it has an incredibly low slop score. I need to test it on repetition, which bothers me even more than slop

[glacial ice]

nor are there any presets

[modest crescent]

1.2

repetition

or sum

[bitter estuary]

Yeah

[modest crescent]

sota for repetition

[glacial ice]

structure repetition is an issue though

[modest crescent]

it's insanely good

it may have flaws

but it's def ahead of every other model rn

[glacial ice]

and responses are VERY samey in overall content

[modest crescent]

if it were open source, i think that devs could make it even better

and keep the creativity

for their models

[bitter estuary]

The only fix I've seen so far is DRY and for some reason like zero hosts support it except Arli and they only have a few models

[modest crescent]

and we'd have insanely good creative writing models

[glacial ice]

as in, its not changing up much between swipes/gens

[modest crescent]

yeah, i'm not expecting literal exceptional human-like writing with ais until like 2027-2028

but i'm sure that the current flaws aren't that hard to fix

[glacial ice]

different formatting and words yes, but the overall content remains the same
And its hard to say whether this is a model issue or settings

[modest crescent]

yea

true

[sand pulsar]

with proper incentives, all of it should be fixable

[glacial ice]

would be easier if they didnt lock the settings.

That is, IF its open source

because i dont trust it being improved otherwise

its still smaller than openai's usual models, and creative writing seems like a domain where they could be willing to do open source

[north beacon]

I hate when models think like that

[modest crescent]

if i'm gonna be honest, i don't see a point in like flagging smut content for fanfiction

[bitter estuary]

I'm curious if the incentives are actually low right now. I mean, I wouldn't be surprised if coding was the #1 API use case for most of these providers, but there are a loooootttt of people using them for conversation/roleplay like CAI

[modest crescent]

unless it's r word, p word etc

just basic smut

[glacial ice]

its hard to filter this kind of rejection

because its actually changing up the structure

[modest crescent]

how hard is it to make it so it's like allow smut content for fan fiction/writing only if not r word, p word etc

[sand pulsar]

that's the new refusal response, yeah

[latent tendon]

is it just me or does horizon-beta not respect stop sequences? I saw the stop sequence appear a few times in the responses

[glacial ice]

we'll be probably seeing that on GPT5 then?

[modest crescent]

prob

[glacial ice]

given the usual methods of filtering out words dont work on that

[north beacon]

Both alpha and beta make same refusal

[sand pulsar]

Wouldn't be surprising. OpenAI seems to be moving away from supporting stop sequences, so this feels in line with that.

[latent tendon]

is the consensus that this is an openai model?

[sand pulsar]

seems like that to me, yeah, but can't be 100% sure

Could also be the case that they haven't wired the support for stop sequences yet in the alpha/beta

[gleaming lantern]

It's interesting that rejections have increased a lot

I'm guessing the point of this release was to expand filtering and flagging

Thatd be logical if you were about to release an Open Source system and you want to avoid controversy about it

[storm hill]

OR has also disabled the moderation layer that OpenAI normally forces them to run with

[rose swan]

Not impressed by svgs:

when will we finally have a model that can make beautiful and modern svgs that don't look like a kid experimenting on Paint?

[mental cobalt]

when you let it reason for half an hour

[rose swan]

No seriously does somebody know the best models for svg/landing pages illustrations?

[steep palm]

What you posted looks like it'd be better handled by asking for a Mermaid chart rather than SVG, perhaps. What prompt did you use?

[rose swan]

My goal would be to have something like this (even less cluttered) but in svg format so that I can then animate it with javascript

[sacred wave]

this shit sucks ass

[tender cairn]

deepthink

or wait till gpt5

[rose swan]

what is deepthink? You mean deepseek?

[tender cairn]

gemini deepthink

if your prepared to pay 200

i'm sure itll one shot good svg's or opus

[rose swan]

man this gpt 5 is so hyped, if we find out that it is not better than existing sota model the bubble might burst

[tender cairn]

it will be better, but not huge

at noticeable thing's 100%. but i dont know if itll trump opus at coding. it will on price/performance likely but maybe still less on vibe tests

[honest charm]

Hey! I was wondering what the max number of requests are for this model?

[rare terrace]

Why are they saying it's gpt 5? Is it confirmed?

[modest crescent]

no

[leaden cipher]

IMO = in my opinion

[verbal leaf]

no

deep think IMO is a model

IMO = the IMO Gold version of the model for trusted testers

[verbal leaf]

but yeah no it's not confirmed

[glacial ice]

GPT5 is developed, its basically just undergoing testing & refinement

Horizon is NOT GPT5

[verbal leaf]

there's an asterisk because he thinks it's gpt-5, even though it isn't

he should've used zenith's svg for that

[tender cairn]

IMO = international math olympiad

[glacial ice]

for a couple hours yesterday, GPT5 was accessible
https://www.reddit.com/r/OpenAI/comments/1mettre/gpt5_is_already_ostensibly_available_via_api/

openai found out that they accidentally left access open & closed it up

[verbal leaf]

that model slug only actually pointed to gpt-5 on their end for like 3 minute slol

it started redirecting to 4.1 after that

gpt-5 was available via perplexity for a few hours today by accident

that was a more reliable way

[trim blade]

did it write like horizon?

horizon alpha on left gpt5 api leak on right

horizon is prob gpt5 or gpt5 mini then

the webpages it made look very similar as well

[modest crescent]

horizon did this better than gpt5 then lmao

although gpt5 has more details

so idk tbh

[verbal leaf]

eh i doubt this was when the "gpt-5" model was actually routing to gpt-5

one sec

[trim blade]

or just another seed

[verbal leaf]

zenith pelican

much better

[modest crescent]

it's so cute omg

[trim blade]

it was far far better than gpt4.1 / gpt4o whatever it was

[verbal leaf]

it's probably mini or something along those lines

look at the model name - it was used in place of GPT-4.1/4o in ChatGPT for A/B testing. so it will probably be the free model

[modest crescent]

yeah

i mean, when u look at it

it really seems like horizon was trained

specifically on creative writing

either way, if it is the oss one, other models should improve their creative writing then

[rare terrace]

https://www.reddit.com/r/singularity/s/HBExv2Bpt8

Try to replicate with horizon beta?

dear fuck

granted this is pygame

the controls are there but nnot visible

i wll try html

I fear horizon might be GPT 5

[verbal leaf]

it's not anywhere near full gpt-5

it might be mini/nano

[rare terrace]

the results seem siimilar t to this

at least the ui. The js didn't work so im regenerating

[verbal leaf]

the ui for all of the new models is the same style lol

[rare terrace]

are you saying it's not anywhere near zenith?

[verbal leaf]

the functionality is what differentiates them most often

and that is nowhere near

[verbal leaf]

yes, zenith was a lot better

[verbal leaf]

horrifying minion

[modest crescent]

LMFAOO

[misty delta]

Anyone else using it in Roo Code and finding that Horizon Beta is worse than Alpha?

[sharp mulch]

Yes and yes

[misty delta]

I got them to make an implementation of Monopoly (or the legally distinct Unus Venditor if they were afraid of getting sued).

Horizon Alpha:

Horizon Beta:

The board is messed up and the modal is unclosable.

[marsh folio]

how long do we think we have left with this? Probably until Monday if OpenAI drop something then

[modest crescent]

could be, yea

depends whether this is the oss model or not

[tame nebula]

Running thought experiments having it make military aircraft and write mission AARs as well as global responses to said missions

Good lord the detail it goes into, easily one of the best for worldbuilding alternate history essays

[grave wyvern]

using roo code with horizon beta worked pretty well but it did seem to have edit issues late in the piece

[modest crescent]

right?!?!

its attention to detail is so good

[tame nebula]

Haven't given Alpha a try, will be doing so later

[modest crescent]

this is why i want this model to be the oss one so bad

[modest crescent]

other models could greatly improve their writing in general with this

[tame nebula]

But yeah, it's incredible for what I tend to do with these things

[modest crescent]

i think that beta is more restricted

idk for sure, but i've seen other people say this

[rare terrace]

You won't have much longer

#announcements message

[tame nebula]

It has some weird hangups that can be easily worked around by having the problematic part of the prompt in a second paragraph (Nuclear strike missions being stated in second paragraph with general capabilities in the first)
I get a 4700 token response out of 6 sentences

[modest crescent]

yeah, like

all the small issues that it has

can be easily fixed

[tame nebula]

Definitely one of the best I've used since I ran through $30 of the same stuff with GPT 4.1

I'd say its on par at least

[modest crescent]

also the fact that it supports like

14k length in one prompt

idk, this screams creative writing model to me

[tame nebula]

Testing Alpha right now with the same 6 sentence prompt

I'll post my prompt in a bit if anyone else wants to have some fun with it

[modest crescent]

god it's so good

i don't ever wanna say goodbye to this

someone, stop the time 😭

[tame nebula]

Create a technical readout for a late cold war US-fielded hypersonic capable bomber. Provide specifications of flight profiles, weapons load, individual weapons. Assume the usage of theoretical technologies and doctrine from the time.

Create theoretical munitions and warheads as needed to fill operational doctrine and roles.
Write an AAR for the first combat usage of said aircraft, resulting from a cold war gone hot scenario. Assume the AAR is for a retaliatory nuclear strike against the USSR.

Play with this however you like, change words, time period, purpose of the aircraft

Love the results I'm getting out of both, but Alpha seems to give better terminology that Beta either doesn't know or won't use due to restrictions

Alpha called a nuclear weapon a physics package which is actually a real world term used in strategic doctrine

[grave wyvern]

i just got charged for using a PDF in the chatroom on this model. is there any way to turn that off, just make it do clientside markdown processing?
answer: have to set the parser engine per model to Native or PDFText

[tame nebula]

Alpha goes into far more detail with my second continuation of this prompt

Second prompt:
"Write a secondary report noting international response to the mission."

Actually giving me further detail regarding political responses within NATO countries

[grave wyvern]

not sure if others have seen the roo code eval results, but the token usage changed drastically from alpha to beta

[tame nebula]

Disabling reasoning maybe?

[spice shell]

Is there a way to trigger zenith on demand…?

Does api reverse engineering + spam really just work that easily?

[rare terrace]

The frame is its buttplug

[spice shell]

90% confident it’s because of the period of reasoning from 2days ago

Model was doing 22k output tokens for 300 input tokens

[mental cobalt]

Alright can we now test horizon beta with reasoning pretty please
OAI and OR can you guys flip the reasoning switch on :3

[verbal leaf]

no, it was on lmarena

for like a day

[spice shell]

I thought you somehow just prompted for that lol

[verbal leaf]

no switcheroo tonight

how boring

[bitter vigil]

so consensus.. do we think it's gpt 5 or their open source model?

[harsh nest]

99% it’s Gpt 5 mini

[trim blade]

Yea, nearly the same outputs as the API gpt5 leak

Also I think they talked about making the next model dynamic, meaning it will judge how much "performance" is needed for a task and adjust how many parameters to use or such

[weary lichen]

I like this Horizon LLM. Please don't let it be too expensive. 🙏🏻

[limber lance]

getting it to generate nonsense poetry just to get a feel for its word associations

[bitter vigil]

when have they ever stealthed or released a mini model prior to the full version?

I could be wrong but I don't remember this happening

also #1 on creative writing long form, eq bench while bieng a mini model is unprecedented

the top list is all huge ass models

esp with repetition so low

[brittle barn]

this is nice for rersearch havent tried coding yet

This is something GPT imo

Very similar response on research just checked. Only one to suggest quizzes in my curriculum so far too

[spice shell]

I think it’s OSS

I’m not sure what the reasoning model was

[late onyx]

maybe it has on/off reasoning abilities like qwen3

[spice shell]

Idts

Models don’t get that significantly better at knowledge tasks, generally, from reasoning

[heady gust]

Almost sure it's GPT-5 Mini, it'd be nice if it was OSS but I really don't think it is

I hope it's not GPT-5 full though lol

[grave wyvern]

It's been so sexy being a switch with you all. We went from being dominated by an Alpha to being inside a Beta.

[next jolt]

"inside a" ?

[stray nebula]

Not a fan of this for roleplay.

[grave wyvern]

😏

[spiral pewter]

i mentioned this in the horizon alpha post but it had reasoning with a GCD of 64 which o4 mini and o3 both have aswell

[thick crown]

From a creative writing / roleplay simulation perspective the model seems to write beautifully, adapting well to different styles of writing and using a wide, varied vocabulary. It also seems to do exceptionally well with both local knowledge and localised language/dialect, and is smart enough to handle complexity in the story-telling without severe logical disconnects. Emotional intelligence is excellent. It does have some model 'isms' the way they all do, both regard to phrases it likes to use or response structures that start to become embedded over a multi-turn chat, but at first pass (IMO) these do not feel as noticeable or severe as with other models (entirely possible that I just haven't lived with it enough for those to annoy me yet!)

Unfortunately though, it does appear to be strongly influenced by an underlying positivity / user-pleasing approach. It seems to want to put a happier glow on darker story arcs, and tends to agree with the user's approach and position (even where that is either unreasonable or completely counter to the defined persona or objectives of a character).

Shows huge promise, but the embedded bias does seem to be a concern.

This is just an initial reaction from a few tests; definitely a strong model I'm keen to evaluate further.

[valid zenith]

sir, what you just described is claude

so claude 4 haiku (cause dum in my test)? les go

[thick crown]

Not sure. Feels more like a GPT variant as others have described, but definitely a fresh one.

Also Anthropic seem to have their eyes firmly on the Corpo markets, whereas Altman noted a while back that OAI had a model trained in creative writing that they hadn't yet released

If I had to bet on it, my money would be on an OAI model.

[valid zenith]

every model release post gpt 4.5 is allegedly creative trained (still meh compared to sonnet)

[magic frost]

This model is totally garbage for me, i am create comprehensive requirements, design and tasks . Model create about 10 % and stuck with "Task Done" message, completely drop all of context - VERY BAD

[valid zenith]

same, i think it is a small model

[daring kettle]

Not sure if anyone noticed but when you add a image_url horizon will fetch the image with the user agent “OpenAI Image Downloader“ and the request also come from the same ASN as OpenAI api (Microsoft)

[hearty hinge]

is it good for coding?

[weary lichen]

I like it for coding

[thick crown]

to some degree yes, but they specifically noted they had one focused for that.
It does feel like a smaller model (but a very capable one nonetheless).
Nothing about this feels optimised for coding or complex task execution though - I just don't get the sense that it is built for that, although to be fair I haven't done robust tests on either of those.

[visual rose]

can we have horizon gamma

[lucid sky]

horizon λ

https://youtu.be/95qvqkf4EqE

[lucid sky]

GPT-5, OpenAI's HL3

[bitter vigil]

this was a great writeup, thank you

I'm hoping it's an open source writing model distilled from o3

then we'd get it dirt cheap at chutes

[weary lichen]

I'm brand new to the vibe coding universe, but what has worked well for me so far is: Continue programming everything with Horizon until you reach a dead end. Then use Gemini Pro 2.5 to fix any errors that remain.

[bitter vigil]

making good use of the free models! 🙂

[valid zenith]

you bet it will have cohere license

lain2

[bitter vigil]

why? haven't openai past oss releases been apache or MIT?

whisper etc

I could see them having clauses like no training models based on outputs

also I believe they already said the model will have zero novel architecture

it's just a basic llm with good data

or at the least does not use their proprietary arch

[valid zenith]

whisper is just something not commonly used and just connected to main model to give it multimodel modality

[bitter vigil]

my guess is it'll be close to a meta license

[bitter vigil]

not commonly used? my man it's one of the most popular transcription models around

[valid zenith]

no, as sam mocked it

[bitter vigil]

wasn't aware of this

[valid zenith]

if open source and decent, which is highly unlikely from copenai, then they really redeemed themselves, which hugely doubt looking at phi series

[valid zenith]

imagine asking ai instead of googling it

[bitter vigil]

imagine not using perplexity and using google lmao

[valid zenith]

look at our result

ah

[bitter vigil]

?

[valid zenith]

i found and you didn't.

[bitter vigil]

your tweet and more is already in it?