general

[torn mantle]

Actually gemini has a different reasoning approach

So it will be interesting

[keen beacon]

this seems different compared to flash thinking

[gentle plinth]

yeah openai hiding the reasoning chain for which you are paying is just ridiculous

[torn mantle]

Probably modified or smth like oai one

[keen beacon]

in my personal experience flash thinking had several limitations on ood

[brittle tiger]

deepseek reasoing outputs > gemini > openai

for me at least

[sick mountain]

flash thinking:

[torn mantle]

Yea for now

[torn mantle]

I always return to deepseek just to read the reasoning process tbh

Everytime you find interesting stuff

I think it may be time to sub to gemini advanced

Its not a bad plan

[sick mountain]

if 2.5 is SOTA then it is for sure worth it

[torn mantle]

And also they improved the deep research

[keen beacon]

the gemini models on gemini the product kinda suck tho

aistudio is better

[sick mountain]

i feel like they've likely toned down the censorship recently no testing tho just vibes

[torn mantle]

That's the issue with google, they have great models, imagen 3 & veo 2, but the marketing and implementation is just not it

[keen beacon]

holy hell google moves fast now lol

[rigid widget]

I'm genuinely surprised that even people within this community are believing the "safety" garbage coming from these artificial intelligence corporations.

They can mark any requests they want as dangerous, using the excuse of security. Weapons, porn, drugs, workarounds, modifications... you can mark all of these as potentially dangerous. You can even consider giving information about yt-dlp and ffmpeg as "potentially dangerous," and even "content blockers, VPNs, userscripts" as "potentially dangerous."

[calm sequoia]

It was anomaly from the start that Google is not the No. 1 at everything LLM and AI

[alpine coral]

im not saying i'm agreeing with it

just that safety is not the same as propaganda

[north vale]

Fwiw i think the 2.5 thing is prolly real now

[alpine coral]

yeah me too

[keen beacon]

ya it is

its just an unbelievable pace

they did gemini 2 sooo fast

then pivot to this that quickoly

they didnt even release 2.0 pro stable

😉

[rigid widget]

please just look at elon musk

[silk haven]

I thought that the 2.5 Flash and 2.5 Pro would be released near Google I/O in May, like last year

[brittle tiger]

would make sense for this to be today. bc yea 2.5 before GA 2.0 Pro would be funny

[alpine coral]

tbf this is a good point lol

[keen beacon]

i dont think so tbh. i think theyre skipping it

[silk haven]

Google's Sergey Brin: Google’s AI products “are overrun with filters and punts of various kinds.” -> Google’s co-founder tells AI staff to stop ‘building nanny products’

Google’s AI products “are overrun with filters and punts of various kinds.” According to Brin, Google needs to “trust our users” and “can’t keep building nanny products.”

[silk haven]

Last month

[keen beacon]

that certainly didnt stop them from developing stuff (other than safety filters) this fast lol

[north vale]

Very Bullish

[leaden palm]

top 5 google naming fails of all time

[north vale]

Yeah this doesnt make any sense

Surely they add thinking to the name

[keen beacon]

cot models will probably be the default going forward

[north vale]

Nah

[keen beacon]

this is what openai says

they wont release a non cot model anymore

[leaden palm]

well gpt-5 won't use chain of thought if unneeded as i understand

[keen beacon]

yea but its gonna be a hybrid like sonnet 3.7 presumably

[north vale]

Idk if it’s hybrid and mostly doesnt use reasoning for most completions it doesnt count as cot model to me

So ig depends how u look at it

Maybe models with cot capability will be defaukt

But asking it how are u and it reasoning about it before answering will not be default

Bc that’s useless

[silk haven]

Sergey Brin full note:

“It has been 2 years of the Gemini program and GDM. We have come a long way in that time with many efforts we should feel very proud of. At the same time competition has accelerated immensely and the final race to AGI is afoot. I think we have all the ingredients to win this race but we are going to have to turbocharge our efforts.

Code matters most — AGI will happen with takeoff, when the Al improves itself. Probably initially it will be with a lot of human help so the most important is our code performance. Furthermore this needs to work on our own 1p code. We have to be the most efficient coder and Al scientists in the world by using our own Al.

Productivity — In my experience about 60 hours a week is the sweet spot of productivity. Some folks put in a lot more but can burn out or lose creativity. A number of folks work less than 60 hours and a small number put in the bare minimum to get by. This last group is not only unproductive but also can be highly demoralizing to everyone else.

Location — It is important to work in the office because physically being together is far more effective for communication than gve etc. And, therefore you need to be physically colocated with others working on the same thing. We need to minimize reporting lines across countries, cities, and buildings. I recommend being in the office at least every week day.

Organization — We need to have clear responsibility and organization with high functioning groups with shared management and technology leadership.

Simplicity — Lets use simple solutions where we can. Eg if prompting works, just do that, don’t posttrain a separate model. No unnecessary technical complexities (such as lora). Ideally we will truly have one recipe and one model which can simply be prompted for different uses.

Excellence — whether it’s an eval or a data source or a dashboard or a message in an internal Ul, please make sure they all work and all are good.

[rigid widget]

because thanks to AI Studio, they want to create models that are constantly being tested with new data and are always getting better. They don't want to offer something as "stable" without doing something really big.

[silk haven]

Speed — we need our products, models, internal tools to be fast. Can’t wait 20 minutes to run a bit of python on borg.

Iterate at small scale — we need lots of ideas that we can test quickly. The best way to do this is small scale experiments until you can ramp up and hopefully see increasing advantage at scale. This is an excellent validation. Working too much at just large scale has a habit of minor tweaking and overfitting to evals, checkpoint sniping, etc. We need real wins that scale.

No punting — we can’t keep building nanny products. Our products are overrun with filters and punts of various kinds. We need capable products and [to] trust our users.“

[north vale]

https://x.com/testingcatalog/status/1904539290899533838?s=46 lol is this real chat

TestingCatalog News 🗞 (@testingcatalog) on X

Is OpenAI ready with GPT-5? 👀👀👀

X · Mar 25, 2025, 2:22 PM

[silk haven]

https://www.theverge.com/command-line-newsletter/622045/google-ai-nanny-products

[rigid widget]

Folks, please resist the hype and be patient. Real-world tests are consistently the most crucial.

[keen beacon]

well people have been testing nebula here for a while and its been good

^

although it could be possible phantom is 2.5 pro exp and nebula is something else, or vice versa

[pure nova]

I cant believe they release nebula already

[sick mountain]

how long has it been in lmarena?

[keen beacon]

GDM employees have been hinting it nonstop for the last day or two

[pure nova]

Its in aistudio

[keen beacon]

~5 days

[keen beacon]

specter/phantom/nebula i think are the same

maybe different temperatures?

[brittle tiger]

looks very fake

[sick mountain]

it is fake

[keen beacon]

dont think it would be noteworthy enough to split into different names i think

just different revisions

[brittle tiger]

screencap?

[keen beacon]

wrong

[pure nova]

Im in the uk and i have it

[keen beacon]

send a screenshot

[keen beacon]

lol no

[north vale]

Agreed

[sick mountain]

lmao

[keen beacon]

it's not actually called nevila

nebula

[pure nova]

thats what it says on my studio

[keen beacon]

right

🙄

u changed pro to nebula lol

with inspect element

its supposed to be 2.5 anyway

[pure nova]

03-25

yeah thats what it says

hmm thats odd 🤔

[brittle tiger]

I'll feel bad for my laugh emoji if you aren't BSing but that would be pretty strange

[rigid widget]

bruh nebula it's anon name

[pure nova]

im not sure

thats what its saying on my studio

oh its out on gemini now

[brittle tiger]

lfg

[pure nova]

so any announcement or any news / changes for this ?

when can we expect official benchmarks

[sick mountain]

probably at 11 am or 12 pm est

[north vale]

ok but fr why is polymarket not summing up to 100%

ik they all resolve no from a tie but a tie seems very unlikely?

it seems priced at 10%+ rn

[pure nova]

holy f

2.5 is so good wtf

every other ai model i asked, it just made complete ass

but 2.5 nailed it

[sick mountain]

prompt?

[pure nova]

it was legit the simplest html website

and every other model couldnt do it for its life

[barren prairie]

I don t have it on my google ai studio 🥲🥲🥲🥲🥲😥😥😥

[scarlet flint]

have you tried deepseek?

i saw on internet today

that they released new model and its very good

[scarlet flint]

Can you share picture in higher resolution?

[scarlet flint]

me too

[rigid widget]

of course and too much soon i release my comparison for too many tasks

[pure nova]

its my own website?

[scarlet flint]

can you share the image? i wanted to try it on other models?

[scarlet flint]

thanks

yeah newest deepseek model can't replicate that

not even close

[elder rapids]

they must be super confident

2.5 is crazy

[scarlet flint]

yeah

[pure nova]

Gemini did it perfectly pretty much

try to plug it in gemini

compare it with me

[scarlet flint]

i don't have the pro model

[elder rapids]

saying it because I read the previous messages

so

[pure nova]

show me what deepseek gave to ou

this is what i got fm gemini

[north vale]

if anyone wants to run a prompt and doesn't have access, i got access to 2.5 pro

[elder rapids]

I have access too

it's pretty good, but it's nerfed

[pure nova]

how

[elder rapids]

they're not allowing it to think long enough

lmao

it's cutting short its own CoT

[pure nova]

yeah wtf

compared to gemini

its dogshit

[elder rapids]

alright what's going on tho

2.5 is insane

[pure nova]

wow how the hell did deepseek mess it up THAT bad

[elder rapids]

for a lot of tasks

but they're cutting off the cot

[loud leaf]

nebula = 2.5 pro?

[elder rapids]

I think so yeah

[north vale]

ye

prolly

[pure nova]

2.5 can also do geogussr

its cool

i asked it and it got it right first attempt

[scarlet flint]

i think its bad to replicate images

i've tested it on my JFrame design in java

and it upgraded it heavly

[elder rapids]

2.5 has to be SOTA

I'm asking it to think longer

and it's actually getting these puzzles right lmfao

[rigid widget]

joke bro

[elder rapids]

flash doesn't think longer when I ask it to either

[pure nova]

ask it to think for a certain amt of time eg 30sec

[brittle tiger]

I want it out in AI studio. have access in gemini but ai studio better for testing when not hooked up to memories and apps like in main app

[elder rapids]

are they gonna add 2.5 to AI studio?

[north vale]

likely

[rigid widget]

left to right: deepseekv3, gemini2.0pro, claude3.7sonnet, deepseekv3-0324

[keen beacon]

system prompt related, this has always been an issue with models on gemini

wait for it to launch on ai studio

Gemini version is weaker

[elder rapids]

ye

has to be affecting the thinking length

[scarlet flint]

i aksed it to upgrade the design

[rigid widget]

By the way, Gemini didn't do that thing we talked about earlier, they've clearly restricted it.

[pure nova]

yes but its not about that its about copying the image

[keen beacon]

4o image gen coming

[pure nova]

gemini could dothat too

[rigid widget]

My English is not very good. Which poem is better?

[rigid widget]

Dall-e 3 is totally garbage let's see

[scarlet flint]

yeah i know, can you show me version of your page upgraded by the gemini 2.5

?

i don't have access to it since i don't have the subscription i guess

[keen beacon]

my theory is that it's upgraded compared to the initial preview of 4o image gen

otherwise it just looks embarrassing

[elder rapids]

what if 2.5 pro is an entirely different and new model

it doesn't say it's thinking

[keen beacon]

yeah we know

[elder rapids]

nah what I mean is

[keen beacon]

qwen 3 on thursday 👀 apparently

[scarlet flint]

there was this new thinking method for models

more efficient

[elder rapids]

what if it's better for context

that all the other models

[scarlet flint]

draft something

[scarlet flint]

and its like internal thinking

[elder rapids]

ye

[pure nova]

its not as good as deepseek but all i said was upgrade the page

[scarlet flint]

yeah its nice

it kept the style of original page

[pure nova]

yeah

[elder rapids]

I'm predicting it becomes the best long context reasoner

by a large margin

[scarlet flint]

chain-of-draft

[keen beacon]

great week for acceleration

hopefully we get o3 soon given it's been threatened

[pure nova]

i wonder if 2.5 pro is better than 3.7 thinking at its max both

[rigid widget]

As long as there is that "security" nonsense I mentioned before, they can never compete with others in image generation.

[silk haven]

https://x.com/officiallogank/status/1904559860378915127?s=46&t=P8-tRi_JAVcI6l5U6nOT4A

Logan Kilpatrick (@OfficialLoganK) on X

🌌🥎👍

X · Mar 25, 2025, 3:43 PM

[keen beacon]

https://x.com/OfficialLoganK/status/1904561688134967357?t=zVeue3sku3MQJM3XIRKcyA&s=19 LOL

Logan Kilpatrick (@OfficialLoganK) on X

@OpenAI : )

X · Mar 25, 2025, 3:51 PM

[rigid widget]

can anyone help?

[keen beacon]

they potentially have a year lead on native image gen though

[pure nova]

whats the prompt for 2.5 to do this

oh wait that is 2.5 right

[elder rapids]

from my experiments with nebula

i think it's better

but we'll have to wait and see for the un nerfed version

[pure nova]

how are u so sure that this is nerfed at all

[willow grail]

finally gpt4o releases today

[rigid widget]

Can someone who speaks English help me?

[pure nova]

lol no way
i just asked gemini 2.5
Simulate a gravity-affected ball bouncing inside a rotating square using Python, with realistic velocity, collision, and rotation-aware physics.
and it gave me a syntax error

insane

[scarlet flint]

i see that google has this tendency to release very good models from time to time like that experimental model that got almost to the top of leaderboard, i think it was sitting on second place, from my tests it was better than 2.0 flash they released into gemini website

[elder rapids]

because it's worse than nebula by a ton and doesn't think long enough

either it's not nebula at all

or it's nerfed

only possible cases

[keen beacon]

just wait for the release on aistudio 🙈

[elder rapids]

https://twitter.com/Oimachi6020_En/status/1904555690246685046 is this true

Oimachi@𝕏 (@Oimachi6020_En) on X

🚨BREAKING🚨
Google is preparing to release up to four models in addition to the Gemini 2.5 Pro

The Gemini website has already prepared it🔥

X · Mar 25, 2025, 3:27 PM

[pure nova]

braindead?

[elder rapids]

ong

[rigid widget]

bro this is the unnerfed version, we will see the nerfed version in the future.

[elder rapids]

dawg

if it's saying things differently, not reasoning as long as nebula

it's nerfed

this isn't skepticism or anything

😭

[pure nova]

holy sht every question i ask it im getting syntax error

[rigid widget]

did i shadow banned? Why doesn't anyone see what I post?

[pure nova]

write a Python program that shows a ball bouncing inside a spinning hexagon. The ball should be affected by gravity and friction, and it must bounce off the rotating walls realistically
and
Simulate a gravity-affected ball bouncing inside a rotating square using Python, with realistic velocity, collision, and rotation-aware physics.

[keen beacon]

the gemini product models suck

[elder rapids]

apparently it's unstable idk

[keen beacon]

the aistudio release will be good

[scarlet flint]

i used your prompt in deepseek

[scarlet flint]

this is what he gave me

it loooks like

animation

or something XD

always the same

[sick mountain]

might be a tokenization issue ive seen other gemini models miss empty brackets a lot, also see https://www.reddit.com/r/Bard/comments/1jjmta6/gemini_25_cannot_write/

[pure nova]

ok yeah wtf

deepseek one works instanly

[scarlet flint]

not really

[pure nova]

ok nvm

[scarlet flint]

keep watching

[pure nova]

the ball fell through the square

[scarlet flint]

its glitching and also play it again

its 1:1

the same simulation

the same path

[pure nova]

but at least it didnt get syntax errors like gemini

[scarlet flint]

the same colission hit

[loud leaf]

how long's the typical lag to add to aistudio? longer thinking time will help

[keen beacon]

in a few hours

[scarlet flint]

like the animation was hard coded but it wasn't

[elder rapids]

hours

[scarlet flint]

yeah

[sick mountain]

@pure nova

[scarlet flint]

i wonder if deepseek r2 will be one of the best if not the best thinking model or if it will be total trash and will fail every expectation

[pure nova]

yeah this is probably why , cause it was giving a syntax error of an empty variable

it was literally vertices =#comment

which doesnt make sense

[sick mountain]

ive gotten that same thing with 2.0 ft before

[rigid widget]

Look at Deepseek V3 0324 and you will understand

[elder rapids]

from my testing it's alright

seems to have an aptitude for coding

2.0 pro does seem better still tbh

3.7 sonnet and 2.0 pro are visibly better than the other non thinking models

no one talks about it for some reason

[keen beacon]

yeah i just found the sysprompt

it's literally an entire book

when will they understand how badly this impacts performance

[keen beacon]

https://www.reddit.com/r/singularity/comments/1jjm9s9/gemini_25_pro_internal_instructions/

[keen beacon]

wtf

it seems theres a lot more to that 🤣

[lime coral]

Legit or not we will see

[keen beacon]

lol no he's not a Google employee

the sentence doesn't even really make sense

that guy is a weirdo

[wintry tinsel]

The information he has is reliable but he stole it from somewhere else

I have a question about the new deep seek V3 is the api for V3 updated or do I need a new checkpoint

As in a new API

[scarlet flint]

i mean can't they like train model to behave in some way

instead of feeding it with system prompt that breaks performance by like 60%

[scarlet flint]

i think its just "deepseek-chat"

[torn mantle]

@keen beacon what am i reading here? Did they mess with the model again? Does it feel different than what we got on lmarena?

[keen beacon]

it's out!!

[keen beacon]

nah it's just the Gemini system prompt sucking

omg

[silk haven]

I https://x.com/sundarpichai/status/1904575384466710607?s=46&t=P8-tRi_JAVcI6l5U6nOT4A

Sundar Pichai (@sundarpichai) on X

Nebula

X · Mar 25, 2025, 4:45 PM

🚀🚀🚀

[torn mantle]

finally

[keen beacon]

this version is NOT a thinking model, that appears to be still on its way on studio

[brittle tiger]

LIve in AI STUDIO

[pure nova]

why is ai studio somehow better than normal site

[keen beacon]

???

[pure nova]

i dont understand what they do to it

[keen beacon]

its thinking for me

oh it's a hybrid

interesting

[keen beacon]

its thinking for everything for me

[pure nova]

wtf the google ai studio got the hexagon wrong

insane

it does it fine for like 10 seconds then it falls through

[thorny drum]

wow sundar tweeting nebula

discord hype goes far

[brittle tiger]

glad it launched with 1m context

def better than gemini app version for some reason. arc-agi test i used that failed gemini app over and over one shotted it in ai studio like it did in arena as nebula

[keen beacon]

yeah it was just bugged

[north vale]

wild

[barren prairie]

Is pro2.5 nebula confirmed ? Or maybe specter ? And nebula is still didn t come

[sick mountain]

wow

[north vale]

that's nuts

[keen beacon]

specter nebula phantom are the same just different revisions afaik

[lime coral]

No confirmation

[keen beacon]

theres never gonna be official confirmation

[lime coral]

Can as well be flash 2.5 or flash in the app or whatever

[sick mountain]

sometimes there is

[keen beacon]

on unreleased variants that never get released?

[sick mountain]

oh i mean if it is released

[keen beacon]

ok but theres no evidence for that

while there is some for them being the same

[silk haven]

https://x.com/sundarpichai/status/1904579419496386736?s=46&t=P8-tRi_JAVcI6l5U6nOT4A

Sundar Pichai (@sundarpichai) on X

1/ Gemini 2.5 is here, and it’s our most intelligent AI model ever.

Our first 2.5 model, Gemini 2.5 Pro Experimental is a state-of-the-art thinking model, leading in a wide range of benchmarks – with impressive improvements in enhanced reasoning and coding and now #1 on

X · Mar 25, 2025, 5:01 PM

[lime coral]

None in both case. All I see is « their way to answer looks the same ». Not enough

[keen beacon]

what? did u account for when they arrived in the arena? anyway i feel like the community here is extremely good at this

[silk haven]

https://x.com/jeffdean/status/1904580112248693039?s=46&t=P8-tRi_JAVcI6l5U6nOT4A

Jeff Dean (@JeffDean) on X

🥁Introducing Gemini 2.5, our most intelligent model with impressive capabilities in advanced reasoning and coding.

Now integrating thinking capabilities, 2.5 Pro Experimental is our most performant Gemini model yet. It’s #1 on @lmarena_ai leaderboard. 🥇

X · Mar 25, 2025, 5:04 PM

[keen beacon]

whats ur track record btw?

[brittle tiger]

1443 dam

[pure nova]

can we see the questions that it gets asked though

it would be nice to see what it got wrong/right etc

[keen beacon]

https://x.com/OfficialLoganK/status/1904580368432586975?t=fKVOERgBUn3dfxTBvbtOgA&s=19

Logan Kilpatrick (@OfficialLoganK) on X

Introducing Gemini 2.5 Pro, the world's most powerful model, with unified reasoning capabilities + all the things you love about Gemini (long context, tools, etc)

Available as experimental and for free right now in Google AI Studio + API, with pricing coming very soon!

X · Mar 25, 2025, 5:05 PM

[pure nova]

why cant we see the claude 3.7 one for code stuff lol

he hid it?

[keen beacon]

GOOGLE IS SO BACK

[pure nova]

dog whistle

so back

[keen beacon]

huh

[pure nova]

those benchmarks are insane

gemini is like 5x better

apart frm coding somehow

[silk haven]

2.5 free, OpenAI is cooked

[pure nova]

idk if its forever tho

how is o3 mini high still better at code

insane

[keen beacon]

o3 mini is based on 4o mini too lol

[lime coral]

Are you blind haha

[red sluice]

Nebula (Gemini 2.0 Pro Thinking) is really inconsistent. It can produce never seen before results for an LLM, but sometimes it totally cracks up & produces language that is too familiar for a professional result... I'm not so sure, it's certainly a very good model, and no wonder it destroys chatgpt already. It also struggles a lot with formatting on elaborated prompts, o3-mini is (unfortunately) better for my usage.

[pure nova]

https://i.imgur.com/MGVl272.png

[gentle plinth]

maybe try a lower temperature?

[red sluice]

At this point I'm not even sure Google will dare to release it in this version

[lime coral]

lol it’s always strange with new reveal. Wait 1 week. Also why they label « experimental » and serve it free on the api

[red sluice]

The issue is that temperature is already something normies don't use.

[gentle plinth]

thats why its called experimental

[keen beacon]

google's experimental models are free its not a new thing

[sick mountain]

tested under code name nebula: https://x.com/lmarena_ai/status/1904581128746656099

lmarena.ai (formerly lmsys.org) (@lmarena_ai) on X

BREAKING: Gemini 2.5 Pro is now #1 on the Arena leaderboard - the largest score jump ever (+40 pts vs Grok-3/GPT-4.5)! 🏆

Tested under codename "nebula"🌌, Gemini 2.5 Pro ranked #1🥇 across ALL categories and UNIQUELY #1 in Math, Creative Writing, Instruction Following, Longer

X · Mar 25, 2025, 5:08 PM

[pure nova]

lol , this week is ai week, 3 new models will come out and beat gemini 2.5 mark my words

[keen beacon]

openai should drop o3

they just got smoked

[cloud meadow]

woah

[lime coral]

Like the instability

[willow grail]

gemini 2.5 pro destroy chatgpt o3-mini-high

hmmmmmmm i hope its just a benchmark..... thingy....
if its gooder than o3 high then... omg
dont give me hopes

[pure nova]

c 3.7 still remains undefeated for web

[keen beacon]

lol that jump is insane tho

[lime coral]

The confidence is too large right now

[keen beacon]

^

[cloud meadow]

For now

[pure nova]

meaning?

[lime coral]

Would not validate anything on lmsys until way more people try the model

Confidence interval +-15 pts for gemini and +-10 pts for Claude.

[cloud meadow]

Claude didn't do anything too crazy with 3.7. Soon enough R2 will drop. I can't wait.

[keen beacon]

yall are sleeping on qwen

[gentle plinth]

• with 95% certainty

[lime coral]

Anyway 2k vote is not enough. Same think for the global ranking. Every new model is #1 because of this

[north vale]

2k is enough

every new model is not #1

[lime coral]

Most of the big labs model are #1

Where is 4.5 now?

[north vale]

doubtful

[silk haven]

They don't have enough GPUs for that

[north vale]

would you predict 2.5 pro gets off of #1?

[red sluice]

Weird that on multi turn, it's not being that dominant so far.

[keen beacon]

rl is usually done only in single turn

[cloud meadow]

Where is Samuel Altman's moat?

Evaporated

Meta needs to release soon

[keen beacon]

o3 mini which is based on 4o mini being competitive with a much larger model 🤔

[lime coral]

If they are making it somehow part of gpt5 it’s maybe better for them like that. Imagine they release o3 and then gpt5 has like 0.1% better reasoning stats

No more hype for hypeman

[red sluice]

OpenAI is cooked if they don't have an unreleased model to quickly drop asap lol

[keen beacon]

they have o3 they're just stalling on it

[red sluice]

Doesn't need to be o3 or 5, just needs to be something that can keep up

[hardy pecan]

They have a stream in 40 mins... to show off image editing most likely lol

[lime coral]

Full stats

[hardy pecan]

oh dear...

[keen beacon]

oh wow

that simpleqa score

is bonkers

gemini has always been great at simpleqa

but it appears with 2.5

they literally

leapt for almost every benchmark

they have been cooking

[north vale]

wish they'd shown their prev top model in the benchmark table

[red sluice]

Is o3 that good? Is there any decent benchmark available somewhere? Couldn't test it, too expensive 💀

[north vale]

to compare the improvement

[cloud meadow]

Imagine Llama 4 drops and it overtakes 2.5 lmfao

Unlikely to happen but it would be crazy

[north vale]

seems likely that it's pretty good at using more test-time compute

[keen beacon]

have u gone on the arena?

[north vale]

so maybe like not that good in single completions

[keen beacon]

llama 4 seems to be disappointing

[barren prairie]

Impossible

[keen beacon]

based on the meta model spam

[cloud meadow]

If it's anything like those new anonymous models, yeah true.

Still holding out hope

Trust the Zucc

[keen beacon]

i doubt they will be able to beat qwen 3 even when releasing one month later

[cloud meadow]

What sizes do you think they will release?

[keen beacon]

confirmed sizes are 8b and 15b moe for now. i would expect a successor to the 32b model, maybe moe

[silk haven]

I’m testing 2.5 Pro on the Gemini app and the experience is better than in AI Studio, the integration with Google Search and YouTube is insane

[lime coral]

With the new DeepSeek v3 llama got postponed for at least six more months

[keen beacon]

no way they dont announce llama 4 at their first llamacon

[lime coral]

I know. It was a joke lol

[keen beacon]

depends on when oai launch o3

[keen beacon]

but yeah even with it being delayed 6 months i dont think meta will be able to beat deepseek 🤣

[lime coral]

Depends on when Google drop the experimental

Or will they ship gemini ultra 3 instead

[keen beacon]

i think the qwen team are the dark horse in this, but i dont think they will outright be sota

[red sluice]

Is there a simple way to see against which models an LLM loses the most on average?

[silk haven]

The 🐐
https://x.com/noamshazeer/status/1904581813215125787?s=46&t=P8-tRi_JAVcI6l5U6nOT4A

Noam Shazeer (@NoamShazeer) on X

Introducing Gemini 2.5 Pro Experimental.

The 2.5 series marks a significant evolution: Gemini models are now fundamentally thinking models.

This means the model reasons before responding, to maximize accuracy -- and it’s our best Gemini model yet.

Blog -

X · Mar 25, 2025, 5:11 PM

[elder rapids]

predicted the long context leap

[elder rapids]

no way I predicted this too

lmaooo

[elder rapids]

^

[elder rapids]

^

[keen beacon]

i said that here first

[elder rapids]

quiet buddy

I'm taking it

[keen beacon]

#general message

shhhh

[rigid widget]

yes it's updated both api and app

[elder rapids]

ig good predictions tho

I was gonna analogize 1.0 → 1.5 distinction tho

and then the evolution of 1.5 → 2.0

since they probably started completely from scratch on each one

so if it doesn't signify thinking, it's probably inherently a pure thinking model

that was my thought process

[red sluice]

🤔 What's that

[silk haven]

https://x.com/googleaidevs/status/1904586624333471975?s=46&t=P8-tRi_JAVcI6l5U6nOT4A

Google AI Developers (@googleaidevs) on X

Join the team behind Gemini 2.5 as they dive into the model’s thinking and coding advancements.

🎙️Space starts at 12:20pm PT. Drop your questions below.
https://t.co/wBOHiC0n9k

X · Mar 25, 2025, 5:30 PM

[north vale]

do we know api cost for 2.5 pro

[keen beacon]

lmao what

[lime coral]

Asking about native image. Native audio is a myth now

[lime coral]

0 for now. Still experimental

[keen beacon]

4o avm is native audio tho if ur talking bout that

[lime coral]

Speaking about gemini. They teased it for gemini 2.0 and then ~

[keen beacon]

oh

[lime coral]

https://youtu.be/qE673AY-WEI?si=XsJ1AQqyriRzlv-Y

Crazy demo

[elder rapids]

also just tried 2.5 pro on AI studio

and it's clearly different from the product

ngl I don't even know where to start, I was like an hour late on discovering 2.5 pro

[torn mantle]

still havent tried it

but i said they tend to nerf their model

[keen beacon]

That simpleqa score is crazy

[torn mantle]

maybe its not nebula

specter was google model right?

[keen beacon]

Gpt 4.5 is much much larger and 2.5 pro is somewhat competitive

[keen beacon]

They said it was nebula

[torn mantle]

mm

did you notice any difference?

[lime coral]

https://x.com/alexandr_wang/status/1904589984591695874?s=46

Alexandr Wang (@alexandr_wang) on X

🚨 Gemini 2.5 Pro Exp dropped and it's now #1 across SEAL leaderboards:

🥇 Humanity’s Last Exam
🥇 VISTA (multimodal)
🥇 (tie) Tool Use
🥇 (tie) MultiChallenge (multi-turn)
🥉 (tie) Enigma (puzzles)

Congrats to @demishassabis @sundarpichai & team!

🔗 https://t.co/pVIgk6rIcL

X · Mar 25, 2025, 5:43 PM

[olive mesa]

just saw this wow

i dont know its context length

im guessing 2 million or 3-4 million since its 2.5

[north vale]

not 3-4 million

[olive mesa]

oh only 1m

well i mean thats still huge but i guess since it's experimental rn

[lime coral]

It will increased with time

[olive mesa]

yeah

[lime coral]

OG 1.5 was first released with 128k even though they teased the 1-2M

[elder rapids]

it seems to brute force the 30 hares 20 wolfs thing

and it gets it correctly

[keen beacon]

um tf it has a cut off of january 2025?? wtf the turn around is insane

[elder rapids]

did Google find the secret sauce 😭

[keen beacon]

the gemini 2-2.5 timeline is absolutely insane

[keen ferry]

opinion's on gemini 2.5 pro?

[keen beacon]

im not sure if its correct. ill have to see if it knows events after june 2024

[keen ferry]

was it worth the wait

[keen beacon]

meanwhile gpt-4.5:

what a joke

[keen beacon]

yes

[keen beacon]

bro they continue pretrained the model/did all this stuff in like a month/two 🤣 if thats actually true

[elder rapids]

didn't even have to wait lmao

[keen beacon]

i thought the gemini 2 timelines were short but this is CRAZY

[elder rapids]

this really is crazy

vibes + insane reasoning

[torn mantle]

are we sure this is the same model?

[brittle tiger]

gemini image editing looks better than openai feature coming out today

https://x.com/wintermoat/status/1904593298008006924

Alphabetting (@wintermoat) on X

@testingcatalog Seems like it changed his face. Gemini w native multimodal doesn't do that.

X · Mar 25, 2025, 5:56 PM

[elder rapids]

told you guys you werent glazing it enough lmao

[keen beacon]

yes, this is nebula

[lime coral]

Gpt4o also failed the hands

[keen beacon]

embarassing

[olive mesa]

the wait wasnt even long lmao

[rigid widget]

wow 😲

[olive mesa]

it gives me a different vibe

[elder rapids]

ong

[torn mantle]

its different

[olive mesa]

yeah

i like how it writes code now

it just overall looks better

[elder rapids]

Claude and Google's vibe switched

talking about the models

[silk haven]

2.5 is a Breakthrough

[elder rapids]

3.7 became more robotic, 2.5 pro is so creative

[elder rapids]

has to be

[oblique flint]

Damn I was wrong about 2.5 pro lol, it's actually better at coding than I initially anticipated. Would be great if it's cheaper than claude too

[elder rapids]

look at that long context lmao

[torn mantle]

the model is so good

[olive mesa]

fr

[pure nova]

coding though?

[torn mantle]

good so far

[pure nova]

anyone tried c/c++ ?

[torn mantle]

yea

[pure nova]

i dont think any ai is fit for c/c++ right now tbh to build an actual decent project

its getting somewhat closer but its still lacking a lot

[elder rapids]

2.5 pro brute forcing webdev is crazy

[silk haven]

2.5 series… not only 2.5 pro
When 2.5 flash? Maybe phantom?

[olive mesa]

#1 on lmarena jeez

anybody know what it gets on arc-agi-2?

[elder rapids]

probably similar to sonnet 3.7 thinking

[olive mesa]

yeah its close to 64k

[torn mantle]

not yet

[olive mesa]

so far ive only seen it compared to 16k and 32k and it's a lot better

[lime coral]

Arc agi is useless

I prefer humanity last exam

At least you solve practical things

[oblique flint]

Crazy that 3.7 sonnet is still 90 points ahead of 2.5 pro in webdev arena

[elder rapids]

p sure Claude is made for these kinds of tasks specifically

it's worse in other things compared to 2.5 pro

[lime coral]

What is crazy is Google jump

[elder rapids]

ye

[lime coral]

Claude was like the ultimate King there

3.5 only dethroned by 3.7

[elder rapids]

ong

[cloud meadow]

This new Google Gemini 2.5 model is insane

No other model has continously followed my instructions this well

It's also picked up things better than any other model

Might be my new favorite

[silk haven]

2.0 pro was removed from Gemini app

[silk haven]

Rip

[cloud meadow]

The reasoning is interesting too

[eager crater]

today is crazy

we got dalle 4 and the best overall llm

[cloud meadow]

I remember just a few months ago (I think before deepseek r1) some dude talking about how everything has been boring since the finetune days and then r1 and distills dropped

It was pretty funny lol

[rigid widget]

They've taken user data in aistudio seriously. I tried to make them do this dozens of times, but they couldn't.

[rigid widget]

Even though most of you aren't aware of it yet, the best non-reasoning model also

[cloud meadow]

You are definitely right. I tried feeding a list of names of java obfuscators into the model since it had no idea past something like proguard and now it lists the top 5 I've continously asked questions about lmfao

Idk if I am the sole reason but I think I made it into the dataset

[keen beacon]

theres a huge jump in world knowledge

maybe not

[cloud meadow]

I should hope so considering the fact they own a search engine

[torn mantle]

yea i noticed that

i was trying like some niche prompts on aistudio

and seems like they improved on them a lot

way way better than grok 3 + reasoning

blows it out of the water

[oblique flint]

From reddit:

Just a couple of days ago I wrote this:

This is my exact experience. Long context windows are barely any use. They are vaguely helpful for "needle in a haystack" problems, not much more.

I have a "test" which consists in sending it a collection of almost 1000 poems, which currently sit at around ~230k tokens, and then asking a bunch of stuff which requires reasoning over them. Sometimes, it's something as simple as "identify key writing periods and their differences" (the poems are ordered chronologically). More often than not, it doesn't even "see" the final poems, and it has this exact feeling of "seeing the first ones", then "skipping the middle ones", "seeing some a bit ahead" and "completely ignoring everything else".

I see very few companies tackling the issue of large context windows, and I fully believe that they are key for some significant breakthroughs with LLMs. RAG is not a good solution for many problems. Alas, we will have to keep waiting...

Having just tried this model, I can say that this is a breakthrough moment. A leap. This is the first model that can consistently comb through these poems (200k+ tokens) and analyse them as a whole, without significant issues or problems. I have no idea how they did it, but they did it.

Finally they're starting to utilize that context window

[torn mantle]

so consistent too

[eager crater]

deepseek v3?

[thorny drum]

that's crazy

imagine being in middle school rn

you can literally copy paste your whole book in for your book report

[rigid widget]

v3 0325 You can really move and place blocks but it laaaags a lot.

[keen beacon]

lmao

[red sluice]

Totally a bug, not openai's core prompt 🤔

[rigid widget]

meanwhile 2.5 pro

left deepseek v3 0324 (non-reasoning model) right gemini 2.5 pro (reasoning model)

[cloud meadow]

oneshot minecraft?

[rigid widget]

it create a good Hacker News clone, but does Hacker News have anything to do with hackers at all?

[keen beacon]

it has hackers in the name

this thing is great at web design

(w/ custom cursor)

[cloud meadow]

It made that?

[keen beacon]

it made the entire landing

that's just a part

it can be pretty bold.. personally i think this is cool

[rigid widget]

Hahaha Security

[brittle tiger]

"Gemini 2.5 Pro just zero-shotted a task o3-mini-high made no progress on after burning through millions of credits via Aider"

https://x.com/_clashluke/status/1904612478199173346

Lucas Nestler (@_clashluke) on X

tbc, this is 100% real

X · Mar 25, 2025, 7:12 PM

[keen beacon]

holy moly. there are some bugs here but i'm confident it could solve them in 1 prompt

fully built by gemini 2.5 pro

(the svg is naturally pretty bad, llms can't do word svgs just yet)

[fleet lintel]

what was your prompt?

[keen beacon]

"Write full HTML, CSS and JavaScript for a very beautiful, bold, creative, sleek, polished landing page for Cosine, an AI lab", then "Make it much more beautiful, bold, creative, sleek, and polished. Do not use comments." x2

[olive mesa]

google is progressing faster than openai

[rigid widget]

what x2 mean?

[olive mesa]

well from the public view

i know they have a lot better stuff they're working on rn

[keen beacon]

sent it twice

iteration

[olive mesa]

woah

[keen beacon]

lol

[elder rapids]

glaze me I predicted this exactly

[keen beacon]

bro wants that validation

[elder rapids]

glaze me dawg

I need it

[rigid widget]

Aistudio crashed and I can't even access my other prompts.

[elder rapids]

I said ts verbatim

[keen beacon]

if you insist?

[elder rapids]

yo tell me why I'm a genius

[keen beacon]

yeah it did for a sec for me but it's working again now (?)

[keen beacon]

u just joined bro

[elder rapids]

pre 2.5 pro

I came here to talk about my observations with nebula

enuff speaking chat, lift me onto my pedestal

[elder rapids]

0 shot?

[keen beacon]

i didn't give it any examples but i did ask it to iterate on itself

[elder rapids]

damn

this is crazy

alright so wait

this implies it can reason through granularity now through 1m context

this is nuts

[keen beacon]

did somebody say nuts 🗣️

but fr

[rigid widget]

Hey, you're Google, snap out of it!

[elder rapids]

granularity has been the no. 1 problem for ages lmao

there was definitely a breakthrough

also, I think sometimes it still breaks, in certain CoT processes it stops putting in the numbers for a calculation, but keeps the surround formatting

[torn mantle]

looks great

[olive mesa]

0 shot?

[ocean vortex]

what's your prompt? Curious how this compares with gpt4.5 and grok3

[pure nova]

no way someone has already jailbrken gemini

[olive mesa]

that's 2.5 pro exp??

[pure nova]

ye

[olive mesa]

damn

[eager crater]

Code SVG of a detailed crab

[eager crater]

yes

[elder rapids]

just gave 2.5 pro 800k tokens worth of material and it processed it faster than flash and pro, and gave extraordinary summary results, and didn't miss a single granular thing, and also gave interpretive results rather than just data points

Google did something

[wintry tinsel]

Dm me the jailbreak?

[elder rapids]

and then I said I was surprised and that its crazy it's able to do things like that over long context and it pinpointed exactly why it was different, just from the quality of its own output

[pure nova]

cant he told me to not send it to anyone

he's good with SE and stuff

so he done it many times before with claude too and everything

[elder rapids]

this model is literally 1 of 1

[wintry tinsel]

K can you give me his contact than, I know Pliny will release a jailbreak but his stuff is annoying in how it’s formatted

[pure nova]

such as?

[elder rapids]

such as what

[pure nova]

like whatd u ask

[elder rapids]

the type of information?

[pure nova]

he wont give it to no one lol

[elder rapids]

it was just a book lol

[wintry tinsel]

Finee

[pure nova]

if u wanna try his dc is

access44

[rigid widget]

What is gpt4o-lmsys-0315a-ev3-text

0315? is this mistake

[keen beacon]

march 15th?

[rigid widget]

first time out

[keen beacon]

weird ass name

[rigid widget]

it can't be march 15

but it not good at translation

[keen beacon]

doesnt have to be trained just today

[rigid widget]

Shouldn't it be according to their release date?

[keen beacon]

could be but i think thats an internal name lol

[barren prairie]

It looks alike sarcoptes scabei 🤔

[lime coral]

https://x.com/paulgauthier/status/1904637913411031410?s=46

Paul Gauthier (@paulgauthier) on X

Gemini 2.5 Pro sets SOTA on the aider polyglot leaderboard with a score of 73%.

This is well ahead of thinking/reasoning models. A huge jump from prior Gemini models. The first Gemini model to effectively use efficient diff-like editing formats.

https://t.co/mBVaUPGHPl

X · Mar 25, 2025, 8:53 PM

[rigid widget]

who is rhea from?

it's very good

[cedar tide]

Big model from meta

[brittle tiger]

Damn that's wild

[north vale]

prolly llama 4 checkpoint

[torn mantle]

its bad

[torn mantle]

impressive

[elder rapids]

only for coding, but it's not better than the SOTA models in that either

not that good

[north vale]

https://x.com/petarv_93/status/1904643818030317579?s=46

Petar Veličković (@PetarV_93) on X

Gemini models are now capable enough to assist with fundamental AI research!

Several theorems featured in our recent ICML submissions were co-proved with Gemini's help.

2.5 Pro is a really good model; give it a try if you haven't already :)

X · Mar 25, 2025, 9:17 PM

[elder rapids]

crazy

[rocky wing]

Hi all

[mint relic]

Hello, anyone tried R2 ? Is there any place to use it ?

[elder rapids]

it isn't out

[sour spindle]

What do you actually get with paid gemini vs. just using the models in ai studio

[rigid widget]

Deepseek V3 vs Deepseek V3 0325 real outputs. (Claude, almost there!)
here is https://rentry.org/deepseekv3-vs-v3-0325

[rigid widget]

Unfortunately no

[elder rapids]

@keen beacon do any other models get this right?

besides 2.5 pro

[keen beacon]

not as far as i know

[elder rapids]

do you have access to o1

ive been trying a ton of these puzzles and it seems like 2.5 pro is way ahead in this aspect

[keen beacon]

yeah

[elder rapids]

even when made into text form

including o3 mini and deepseek

they just can't get them right

[willow grail]

eu didnt get the new 4o image?

[rigid widget]

gemini modals really good at multimodal

[elder rapids]

yeah they tend to understand things more, but Im making them into text form

as I said

and the gap isn't THAT large

[brittle tiger]

a worse experience until they leave experimental mode. no reason to use them in gemini app until they are no longer experimental. when that happens tho you get integration with search, drive, gmail, youtube, image generation, etc

[keen beacon]

dont they still suck even if theyre not experimental?

[rigid widget]

I also do not like using artificial intelligence directly in a multimodal manner because I always get worse results. So I OCR it into text first.

[keen beacon]

in the gemini product

[elder rapids]

ye that tends to be a good option

[elder rapids]

not anymore tbh

once they leave experimental

[rigid widget]

I think it's just speed and less error Otherwise, models are trained with data whether you pay for or not.

[keen beacon]

added to eqbench

also

im not keeping track but i feel like i should've hit the ai studio RPD rate limit by now

it is nowhere to be seen

[keen beacon]

i think the models are unlimited on the site but limited on the api

cuz i use aistudio all the time

random ocr? random tests conversations etc

[elder rapids]

damn that's really crazy

wonder if they did the same thing deepseek did, training for specifically eq

[keen beacon]

pretty big jump

[sour spindle]

I feel like this too. I sometimes get this message: "failed to list tuned models user has exceeded quota" but it says I am still using the model.

Maybe they are simply bypassing the rates for the time being?

[elder rapids]

you can definitely see it too, the way it speaks is pretty great

after some hours with it

it has moments where it resembles Claude

[keen beacon]

ive been getting that since yesterday i think

its just bugged

[elder rapids]

or at least, a very large, intelligent model

makes me wonder how Big pro is

if this model is below 100b that would be really crazy

[keen beacon]

it is not that would be absurd tbh

[silk haven]

Apricot-exp-v1?? Amazon model?

[hazy quest]

Finaly midnight in Europe. What a day this has been lmao

[elder rapids]

wym?

isn't sonnet and 4o at least 100b

[keen beacon]

4o is estimated to be 200b, sonnet is 400b

pro is definitely within that range

[sour spindle]

"There are decades where nothing happens; and there are weeks where decades happen"

Could change this to days and weeks with AI development lol

[elder rapids]

damn fr?

[keen beacon]

total params. theyre all moe i think

https://epoch.ai/gradient-updates/frontier-language-models-have-become-much-smaller

[torn mantle]

yea this new model is on next level

[elder rapids]

8b flash to 200b would be wild tbh

I expect pro to be 120~

[keen beacon]

flash is not 8b

[elder rapids]

or 150

[keen beacon]

8b flash is a different model

[elder rapids]

flash 8b dawg

[keen beacon]

its a different model

theres flash and flash 8b

[elder rapids]

yeah duh

😭

I said that

but anyways

[brittle tiger]

Think ppl discount how much stuff like this will drive progress as models inprove

[keen beacon]

1.5 line:
flash (larger)
flash 8b
pro

2.0
flash lite (direct technical successor of flash)
flash (larger than flash lite/1.5 flash)
pro

[elder rapids]

🙏

anyways

8b flash to 200b would be wild tbh

[north vale]

yeah i think they're calling them flash and pro based on the speed and cost more than the size being comparable to 1.5 's flash and pro

basically flash could be 200B with 40B active params

and pro could be 1.3T with 150B active params

really uncertain but that would make sense to me

[keen beacon]

lol this is wild

[elder rapids]

yeah but we know that's not true so it's kinda trivial

[north vale]

oh ok, i don't know it to not be true

how do you know

[keen beacon]

economics lol they are not increasing model size to that level anymore

they didnt even release 1.0 ultra access and a google employee confirmed it wasnt even close to og gpt 4 afaik

[elder rapids]

there are traits of models + that would be heavy and unnecessary + ton of money for no reason

[sick mountain]

why not? hardware is getting better too

[elder rapids]

if models are 27b with similar performance

yeah that's not true lmao

you still need the total params to run it

[north vale]

i really don't think 27B models have similar perf

[sick mountain]

ultra did not use MoE iirc

[elder rapids]

wym?

-10b are visibly worse

[north vale]

27B models have much lower perf than gemini pro

[elder rapids]

but 30~ is just fine

[north vale]

is what i'm saying

[elder rapids]

yeah but I'm not talking about Gemini pro

[north vale]

oh ok

[atomic locust]

I want to give out my MacBook Air 2020 &** for free, it's in perfect health and good as, alongside a charger so it's perfect, I want to give it out because I just got a new model and I thought of giving out the old one to someone who can't afford one and is in need of it... Strictly First come first serve !
DM IF YOU ARE INTERESTED

[keen beacon]

no they arent btw. they directly said flash lite is based on 1.5 flash size/architecture/whatever i dont recall the exact quote

[elder rapids]

but anyways back to the point

I do think Google has always had special models, and the speed both perform at is crazy

so they can't be that big

[north vale]

I just don't see gemini 2.5 pro being within 30% of the size of gemini 1.5 pro

elo 140 pts apart

[keen beacon]

one was pretrained way before and wasnt even a thinking model + modern stuff

[elder rapids]

ye

[north vale]

yeah good point

[elder rapids]

I think the pros are maximum 10b params deviation

[north vale]

that'd be completely wild google dominance

[elder rapids]

but I have no idea how large they are

ye

[keen beacon]

the pros are still around the same size. but it is quite plausible they increased the size a little but its not a trillion parameter model

[elder rapids]

and I don't think 1.5 pro is above 200b

it's both faster and seemed like it had less "raw" intelligence than Claude, which was similar in time, and 4o

seemed to know less stuff without search as well

completely up to you whether you agree

but I do think 200b+ models tend to just feel heavier

so I'm inclined to believe it's at most 150b

[keen beacon]

no it isnt faster lol

its the same

[elder rapids]

wym?

it's way faster

especially now

[keen beacon]

2.0 pro and 1.5 pro are the same speed

[elder rapids]

dawg

[keen beacon]

i cant tell right now for 2.5 pro because there are no measurements for 2.5 pro

[elder rapids]

I'm not talking about 2.0 pro vs 1.5 pro

I'm talking about 1.5 pro vs 4o

and then equating to 2.0 pro

[keen beacon]

well its relevant because 2.5 pro is highly likely to be continued pretrained from 2.0 pro

the timeline seems absurd if it isnt

[elder rapids]

I don't think it's absurd at all tbh

[sick mountain]

google does have the most compute

[elder rapids]

working on both 2.0 and 2.5 at the same time is super reasonable

if they're going for completely different architectures

[keen beacon]

you just pretrained gemini 2.0 pro spending millions and ur gonna throw it away and rush a model from scratch in a month or two??:?

[elder rapids]

as they explained that it's inherently a reasoning model

[keen beacon]

ok, thats still on top of a base model. not relevant

[elder rapids]

ye they did that with 1.5 pro 002 to 2.0

[keen beacon]

yes but that was a sizable amount of time

[elder rapids]

that was also like 2 months

bruh

[keen beacon]

????

[elder rapids]

002 came out in like October

2.0 pro was in experimental in November

so ig one month

preceding 1206

oh yeah it was 2 months too I'm tripping

002 came in September

2.0 pro came in November

[keen beacon]

they were working on gemini 2 in parallel

[elder rapids]

it was on lmsys too, everyone was talking about it

[elder rapids]

yeah I know

that's what I mean here

it's not like they're throwing away progress

since the "progress" is research itself

so they could completely ditch 2.5 tomorrow if they find another breakthrough

[keen beacon]

002 wasnt a new pretrained model it was just another tune afaik

[elder rapids]

yeah I know

but 2.0 is completely different

and then jumping from 2.0 to 2.5 within a couple months seems reasonable

that's how they managed going from "bard" to "ultra 1.0" and then a month later, into 1.5 pro

and then ditched ultra

[keen beacon]

thats like 5 months, they started work after june 2024. the supposed cut off of 2.5 pro is january 2025

[elder rapids]

that's not crazy tho

they've done this more than once

[keen beacon]

so ur saying they pretrained a new 2.5 pro model from scratch, did reasoning rl, safety, etc. in 2 months??

[elder rapids]

saying the best AI compute in the world can't do ts is wild

safety aligning is the hardest part of that process

and I'm pretty sure past models would be insanely informative of that process

they probably wanted to get 2.0 over with, with a breakthrough

[meager sun]

🧱 🤣 👍

[elder rapids]

and then follow up with 2.5 to use it

the fact that 2.5 isn't actually that affected by the transformer context drop off is insane

it has to be different, there's no other way tbh

what if it's TITANS

that'd be crazy

we'll literally never know, they could have something that actually performs with titans techniques

etc

2.5 is simply different from 2.0

[verbal nimbus]

Found this interesting benchmark

[keen beacon]

yeah he posted it here earlier lol

[verbal nimbus]

Gemini is indeed the best at 120K

[verbal nimbus]

Ah

It struggles a bit at 16K (typical transformer behavior)

[elder rapids]

that might be a testing issue rather than it's actual performance

[verbal nimbus]

Didn't notice V3 0324 is on there too

[elder rapids]

damn

eclipsed in context

[verbal nimbus]

There's a reason why transformers struggle in the middle: https://www.youtube.com/watch?v=FAspMnu4Rt0

[elder rapids]

ah yeah I know, but I mean

I'm not sure it would be so sudden

and THAT great

the other models don't seem to be affected

[verbal nimbus]

Hmm yeah that's a bit odd

[elder rapids]

that's really crazy tho tbh, 90 vs 65.6

2.5 pro vs 4o

[keen beacon]

2.5 pro can do 2m-10m context+, 4o total context is 128k-200k

[verbal nimbus]

Sonnet is surprisingly even worse

[elder rapids]

ye, but 1.5 and 2.0 pro still struggled in granularity

it would be more like need in a haystack

rather than actual reasoning

but with 2.5 pro that kinda just stopped existing as a problem

I tested it too

I don't think you guys realize how crazy this is tbh

[verbal nimbus]

That would be, but I don't think they'd start with such a big model for Titan?

[elder rapids]

I'm really not sure

[keen beacon]

its highly likely its just 2.0 pro with continued pretraining

[elder rapids]

2.5 pro just kinda shook me, especially testing it on lmsys with nebula

[keen beacon]

at least the base model

[verbal nimbus]

It's a thinking model too, probably RL trained?

[elder rapids]

ye ofc

[keen beacon]

yes they updated the base model then tuned it for reasoning/rl on it

[verbal nimbus]

I wish there was more news on Titan/Mamba-variants

[elder rapids]

it has a unique cot too tho

[keen beacon]

i dont think mamba is good

[elder rapids]

read it's reasoning process

it uses weird code words

symbols

etc

[verbal nimbus]

Google made two variants based on Mamba that performed better, but I haven't heard anything since.

[keen beacon]

this is a good article from what i recall https://magic.dev/blog/100m-token-context-windows

why mamba/etc dont actually work

[elder rapids]

wonder how this is gonna go in notebook llm

I've had problems with it

nobody seems to care since it's trivial

but I think the products could be so much better

[keen beacon]

its been a while since i read this tho 🤣

[verbal nimbus]

I'll check it out

These were the Mamba variants Google made: https://www.reddit.com/r/MachineLearning/comments/1b3leks/deepmind_introduces_hawk_and_griffin_r/

Haven't heard anything since though 🤷. Same with Titans.

[keen beacon]

transformers keep being improved and improved tbh i dont see anything replacing it lol

[verbal nimbus]

Probably not anytime soon. Diffusion LLMs seemed interesting though.

[elder rapids]

ye but I guess still not technically the same architecture

as current thinking models

but probably gonna remain the base

I think the problems we currently have now will eventually be fixed, like better reasoning by creating a CoT

and then more attachments

[ocean vortex]

What. Thinking models are literally your standard transformer architecture with some fine-tuning. Nothing under the hood is changed

[keen beacon]

yeah this guy is wild man

[elder rapids]

what's with the lack of reading comprehension here

I don't want to be rude, this has happened more than once too

but goddamn

[elder rapids]

^ literally said this

🙏😔

[leaden palm]

??

[rigid widget]

good luck

[elder rapids]

because it isn't technically the same architecture lmao you guys are confusing transformer with what we have now, which has been established as a change for a while now, as with gpt or native multimodality

now I'm wondering if you guys are trolling lmao

this is getting ridiculous

[leaden palm]

what we have now, which has been established as a change for a while now

are you saying "all modern models (even llama) have tweaks and improvements over the original gpt, and gpt is a large improvement over transformers" (pedantic) or "thinking models have an architecturally different way of generating text" (incorrect, see r1)

[willow grail]

what the rate limit for gemini 2.5?

[keen beacon]

I've been using it a lot and haven't encountered it yet

If there is one it's very high. I don't think aistudio is limited like the free api offering

[willow grail]

but ai studio is also free same as openrouter

[keen beacon]

Ya but u have low rpd

[willow grail]

have u connted a ide with 2.5?

[keen beacon]

Requests per day

Idk about openrouter tho

[keen beacon]

I mean on the aistudio website there aren't limits

[willow grail]

oke so u copy paste everything into your ide

[willow grail]

top rpm vs bottom rpm vs req day?

[keen beacon]

I don't use ai to code yet they suck at rust

[leaden palm]

openrouter actually gives you more limits lol

[leaden palm]

top one is if you have a payment method

(which is weird because it's free either way)

[willow grail]

doesnt openrouter share the one 2.5 one with all users?

so everybody has less prompts

[leaden palm]

they contacted google for higher limits

[elder rapids]

this would make sense if this premise weren't my own claim lol, they suggested fundemental architectural change but I said it isn't technically the same but it doesn't matter since with or without inherent limitations (transformer, or not), we can optimize for other specific tasks like we did with CoT, and what were already doing (for agentic use)

[willow grail]

thanks honey

[elder rapids]

since it's architectural identity wasn't a primary claim, and what I said operates on its lack of relevance already, this is just a comprehension issue

[leaden palm]

comprehension??

you're the one saying that thinking models use different architectures

and don't get that r1 is just v3 RLd on thinking

[elder rapids]

I explicitly said "remain the base" dawg 😭

and even clarified "not technically the same" so I consider what I'm saying pedantic posturing, but for rhetorical purposes

since the discussion is operating primarily on the CORE architecture, ie titans vs transformers and I'm explicitly stepping away from that dialectic, what do you think I'm saying

not only that, I even clarified why they can technically be distinguished between the base transformer architecture (Ie gpt, multimodality) and since yes comprehension is an issue, you dismissed it with "pedantry" knowing that's the premise, not my rebuttal towards what they're saying

[leaden palm]

nobody asked

[keen beacon]

can someone with access to o1 pro give it this

the answer is permanent

but gemini 2.5 pro, grok 3 thinking and claude 3.7 sonnet thinking all fail

[leaden palm]

Question 5

A particle P, of mass m, is attached to one end of a light elastic string of natural length 0.5 m and modulus of elasticity 2mg. The other end of the string is attached to a fixed point A on a rough horizontal surface.

P is held at a point B, where |AB|=0.5 m and given a speed of 1.4 ms⁻¹ in the direction AB.

P comes at rest at the point C.

Determine whether this position of rest is instantaneous or permanent.

heres the transcription

[keen beacon]

looks like 2.5 pro gets it with code execution