SAM ALTMAN DEPARTS AS OPENAI CEO!!

[haughty oak]

what I judge to be their goal was to cull Altman's power and stop his scheming from getting unilateral control of the company. They may have very well succeded in that

removing Altman altogether might have been a better outcome for them, but this was a good enough compromise

gosh I'm making so much typos

sleep deprivation

[lavish pecan]

might, but that's where your model gets extremely sparse so by your own reckoning you should probably not weight that conclusion very highly

don't misunderstand I have no desire to blame the old board or to take away their merits, I have no allegiances here either way, I appreciate they must have been in some really impossible situations over the past few days and I wouldn't have made half the good decisions they made

but

[haughty oak]

look

[lavish pecan]

we don't know whether their actions will have made things better or doomed us harder yet, not even close

[haughty oak]

the situation before is that Sam was CEO, was on board, and Greg was on board, AND was scheming to remove other member, and might well have succeeded

the situation now is that he and greg are out of the board

[lavish pecan]

the last part is assumptions based on very little evidence though

[haughty oak]

the scheming?

[lavish pecan]

yes

[haughty oak]

this was reported by NYT

he tried to kick out Helen Toner for publishing an article critical of OpenAI

[barren lynx]

if sam and greg are off the board, that would seem like quite a positive outcome

probably the best outcome given the employees' rebellion

[lavish pecan]

was this an official statement from someone or more information from "sources"?

[barren lynx]

I'm confused about the choice of larry summers for the board though, doesn't seem strong on safety

[lavish pecan]

because we've had a LOT of information from "sources" over the past 4-5 days and we have zero idea which is valid and which is made up

[haughty oak]

the article mentions they had access to the email where Sam was feuding with her

[lavish pecan]

anyway, I think we're running away and it's a pretty pointless argument to have, we see things a bit differently and that's fine I don't think there's a very hard right or wrong here

you should go and grab some sleep if there's still time wherever you are 🙂

[haughty oak]

yeah I think the whole thing is very uncertain

[lavish pecan]

@barren lynx I thought it was supposed to be a temporary board or something? no idea, I haven't fully read the stuff yet

[haughty oak]

and this is far from over, there will be ramifications for a long time

[lavish pecan]

my biggest hope is that it'll spur some tough regulation from govts, that would be potentially the biggest win from this but I'm not at all confident that will actually happen

a fool's hope 🤷

the big drawback however is, regardless of how the rest plays out - is any board going to have the guts to go against the CEO ever again after this?

on the plus side now we can get back to the usual business of getting governments to do something

[mint oyster]

Hey everyone. Anyone has a short write-up of all that happened this week end from the point of view of AI Safety? Or can point me to a good resource so that I write it? ~300 words

[lavish pecan]

just the facts or safety opinionated?

I'm betting Zvi is going to come out with something good, he already did a summary up to yesterday I believe

might be hard to condense in 300 words 😄

[mint oyster]

facts followed by what it means for ai safety

[lavish pecan]

https://thezvi.substack.com/p/openai-facts-from-a-weekend that's a good start probably, as to what it will mean for AI Safety? likely too early to say for anyone but you can take your own stabs in the dark 🤷

[rotund zenith]

Of course

EA was never going to be enough, as the past few decades have shown.

More than that- we need to enact laws which let the public enforce their ideas

I'm wondering how many OpenAI employees are sick of this drama and would prefer to work somewhere else

[rotund zenith]

Maybe that's the point? A neutral, competent third party?

[rotund zenith]

I'll have a go at this today

[slate storm]

I count 2 EAs on the board and one neutral guy(?). I'm calling that a partial win for safety.

[barren lynx]

the upside is that Altman is not on the board, that's definitely going to be the biggest win from this

the downside is that he's going to be still scheming to get back like the weasel he is

so it's not over, we'll have to see the next season

[slate storm]

I hope the board notes down everything the CEO says to them from now on. If he lies again, there needs to be a contemporaneous record of it next time.

[barren lynx]

it appear that the board people didn't have the Machiavellan skills to outplay Altman

not surprising

[fresh shore]

No, the board won https://x.com/daniel_271828/status/1727267216985624980?s=20

Trust me, everyone

[signal nimbus]

I think this is a losing language battle given everyone seems to be doing it!

But equating EA with "want to mitigate AI x-risk" is misrepresenting a correlation as an equivalence, and presumably really confusing to some when one of the other relevant positions (e/acc) has the same acronym.

Just saying.

[fresh shore]

The most important is that Sam Greg and Ilya lost their board seats

while the nonprofit ones remain

and Adam is staying on

[lavish pecan]

we'll see if the board won when we see the final board of 9

lots can change still

[fresh shore]

yea

More important win is that OpenAI was a clownshow and a natsec threat

[lavish pecan]

one hopes that people in govt noticed that part

[fresh shore]

I'm pretty sure they did

[lavish pecan]

https://openai.com/our-structure this page needs an update 😄

[spiral mortar]

https://youtu.be/UdBMkj2WViY?si=r6FGMAkfLcGLC_6m

[tawny linden]

Almost none, apparently

[rotund zenith]

Hmm, interesting

Did you see the Blind thing?

[tawny linden]

I have no idea what this means. All we know is Helen and Tasha are off the board, Microsoft will have a seat, and Sam isn’t on the board for now

[tawny linden]

Blind thing?

[rotund zenith]

https://www.teamblind.com/post/To-all-600-OpenAI-employees-who-are-ready-to-resign-ZHPCiBdj

[tawny linden]

We’re not— Helen and Tasha are EAs and were put on the board by Open Philanthropy. When I say EA players I mean EA players.

[tawny linden]

What about it?

[hidden socket]

The Board had immense leverage... And Sam still got back to OpenAI's CEO seat.

[haughty oak]

how well did y'all predict what was going on?

take that into account when judging your current predictions

[hidden socket]

It does NOT matter how many EAs there are! We previously had MORE than just two, and they got out-played. 'EAs on the Board!' is meaningless.

[fresh shore]

Wait, Microsoft will have a seat?

[haughty oak]

Microsoft will supposedly have an OBSERVER seat, not voting

[delicate bane]

I don't think this is quite the right way to think about it. You can have immense leverage but if you fumble getting rid of someone this badly, you're going to lose no matter what.

[hidden socket]

That's 100% what I'm saying.

The Board had immense leverage... AND this is a win for Altman.

[delicate bane]

Ah fair. I see what you mean now. I think some more competent board members would have had a chance of kicking him out though. Much less chance now.

[fresh shore]

So they didn't get the goods

[delicate bane]

This shifts the locus of control from the board to the employees significantly, which is something to consider.

[hidden socket]

Employees who, according to at least one account, have been cowed by group-think.

[fresh shore]

What would have been a better outcome?

[hidden socket]

Sam Altman not returning to the CEO position. Emmett retaining the CEO position.

Basically, status on Sunday night.

[fresh shore]

:/

The news articles are saying Microsoft won

[hidden socket]

Well... Yeah.

[delicate bane]

And the employees?

[hidden socket]

Who cares?

'oh no they all quit and openai stopped existing'

Based.

[delicate bane]

I somewhat disagree with this. If all of the employees went to Microsoft under Sam, then Microsoft Ignite would be a worse company than OpenAI currently is.

As in X-risk from Ignite would be higher than it currently is from OpenAI

Which is still higher than it would have been from OpenAI before this debacle

[lavish pecan]

it very much depends on what sort of final board we get. if it's going to be 95% business people + Microsoft rep, it's pretty much the same thing

[fresh shore]

Yes, also more risky

[lavish pecan]

do we know if Ilya is still with OpenAI as staff? presumably yes?

[hidden socket]

I think what makes my calculus on this different is that I don't think X-Risk is a serious problem.

I'm not worried about human extinction at all. I'm worried about AI-driven social control.

And that's gonna happen, no matter which company builds ASI.

The only solution is not to have ASI controlled by techno-capital.

And this is actually a battle we've already lost.

[lavish pecan]

not sure I have too much faith in ASI being controlled by anyone. what alternatives are there that would work? I can't think of many

ah

[hidden socket]

No ASI. Not now. Not ever.

[haughty oak]

human cognitive enhancement

high-bandwidth brain-to-brain interface

tbh my probability that we survive is tiny, my hope is mostly placed in exotic scenarios

[buoyant helm]

a lot of eacc used to be anti openAI because they weren't open source. I feel like most are superficial vibe tweeting now. the intense fight to control openai seems to suggest that all involved expect that AI will be a massively centralized power. Not the eacc dream of liberation

[sick night]

I think this is probably worse than the status quo ante inasmuch as Altman might have good reasons to feel like he has consolidated power and is less constrained by the board going forward (even though he's not on it). Not at all certain about that though. And I'm not sure we know enough to say how this compares to other outcomes that were possible once the whole saga was set in motion

[fresh shore]

some other outcomes include:
OpenAI defecting to Microsoft

[delicate bane]

OpenAI continues basically as normal with new CEO
OpenAI/Microsoft Ignite schism
OpenAI shifts to 100% safety with minimal personnel losses

Then also consider
Sam leaves respectfully
Sam vows revenge
GPT-4 weights get leaked
Novel tech gets leaked
Schism but along safety/capability team lines

[rotund zenith]

I was a mix of very uncertain and wrong. I was right that it wasn't about safety and more about communication- though I thought it was to do with not seeking board approval before doing fundraising for gpt5

[dreamy meadow]

"According to one of the sources, long-time executive Mira Murati told employees on Wednesday that a letter about the AI breakthrough called Q* (pronounced Q-Star), precipitated the board's actions."

https://www.reuters.com/technology/sam-altmans-ouster-openai-was-precipitated-by-letter-board-about-ai-breakthrough-2023-11-22/

[barren lynx]

https://www.cnbc.com/amp/2023/11/22/sam-altmans-ouster-at-openai-precipitated-by-letter-to-board-about-ai-breakthrough-sources-tell-reuters.html

The maker of ChatGPT had made progress on Q*, which some internally believe could be a breakthrough in the startup's search for superintelligence, also known as artificial general intelligence (AGI), one of the people told Reuters. OpenAI defines AGI as AI systems that are smarter than humans.

Given vast computing resources, the new model was able to solve certain mathematical problems, the person said on condition of anonymity because they were not authorized to speak on behalf of the company. Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*'s future success, the source said.

so, AGI internally achieved?

[slate storm]

So the idea that there were no real grievances and it was all fabricated, was just another lie that was fed to the media. Another lie that fell apart very quickly.

[barren lynx]

well, first we had the report that Altman tried to get Helen Toner off the board because she published a paper criticizing OpenAI for moving too fast

It seems pretty clear to me there were valid grievances

looks to me also like they couldn't be open what happened for legal reasons, which caused employees to revolt, giving Sam leverage

[slate storm]

Right. I always suspected it could be something like "achieved AGI internally, but couldn't tell the world because of NDA", although that particular possibility seemed to me to be a bit far-fetched. This sounds more plausible - not AGI yet, I think, but showing promising signs of becoming it.

[hidden socket]

Wow, I can't believe AGI was achieved internally.

Honestly, if this is true... Haven't we already lost?

[fresh shore]

https://www.metaculus.com/questions/5121/date-of-artificial-general-intelligence/

https://www.metaculus.com/questions/6509/control-problem-solution-before-agi/

[fresh shore]

What about a 3-8 year plan to regulate AI?

[hidden socket]

I think that would take too long at this point. I genuinely believe we'll see AGI by 2025.

[fresh shore]

Then all hope is on the 2026 election

[hidden socket]

Wouldn't the AGI just do some cheeky election fraud?

I think our hopes need to be on 2024. We need to do whatever it takes to make AI a major election issue.

[fresh shore]

lmk if you have great ideas here!

[fresh shore]

I think this take may make sense: https://www.vox.com/future-perfect/2023/11/21/23971765/openai-sam-altman-microsoft/

It's important to credibly demonstrate the OpenAI nonprofit board's ability to fire the CEO (and even to destroy the for-profit company if it's necessary for safety), but also firing Sam Altman at this time would have risked a mass defection of top talent to Microsoft which would be quite bad. So, overall outcome may actually be decent contrary to the public narrative

[fresh shore]

The final board composition will be the key thing to watch: https://x.com/TheZvi/status/1727379995742384509?s=20

Key will be how many people with conflicts of interest (Altman, OpenAI equity holders, AI industry equity holders) will get on the board.
https://x.com/liron/status/1727218709016854863?s=20

[lavish pecan]

if that's the operating standard though, when do you know something is actually worse than mass defection? that could be used on a perpetual basis to dissuade any board from firing Altman now far as I can see it. isn't that an equally bad precedent to set? also, if the first part is true and supposing that the Q* stuff is true, it's hard to see much difference between that getting developed under Altman at Microsoft or under Altman at OpenAI. some difference maybe but not a lot.

but yes, I agree that the final board composition is still key

[fresh shore]

Yeah, so we should aim to prevent the employees from signing mass letters in the future

Or to defect from the mass letter once it's signed

There's a very clear argument: If you mass defect, your OpenAI equity will become worthless.

[lavish pecan]

doubt any board/CEO will have the metaphorical cojones to enforce a rule like that

[fresh shore]

No, if OpenAI dies, then OpenAI equity becomes worth $0 normally

Board doesn't need to have courage, the three board members don't have OpenAI equity

The message is: "It is in your interest to preserve OpenAI the company, because if you cause it to die, you will lose out"

[lavish pecan]

well no, but if we get investors/business types on the board, their thinking may be wholly different because they have direct and non-direct interests via other things than equity

so that part yes, depends on what we end up with for a board

[fresh shore]

Not talking about the board. Let's travel back in time. Adam, Helen, and Tasha should, after firing Sam, immediately have reached out to employees and told them the succession plan

[lavish pecan]

though tbf the implication in this case was just that, wasn't it? that the board would destroy OpenAI and all value and employees still threatened to defect en-masse and it still worked so?

yeah that I agree with, lack of communication was their biggest failure imo

[fresh shore]

It was an emotional decision that didn't make financial sense.

[lavish pecan]

both with employees and with the outside

still unsure how they thought that was a good idea - if it was about protecting Q* that clearly also failed because now it got leaked anyway

[fresh shore]

Microsoft was not in a position to pay the OpenAI employees the same amount they would have gotten honestly

It was all fake news

The Q* might also be fake news too

[lavish pecan]

maybe, but I'm not convinced. there's some other pieces that fit that narrative. circumstantial to be sure, but they are there

I'm at least 60% that the Q* stuff is true. don't think it's AGI but it's a significant advancement of some nature

it may all be some clever marketing ploy but that seems less likely

[delicate bane]

Or someone just trolling the media. "Two anonymous sources who (said they) work for OpenAI trust me bro"

[fresh shore]

Even though the board's decision to fire Sam was suboptimal (no PR), I think it still is a net win.

1. Sam, Greg, and Ilya lost their nonprofit board seats. They have a financial conflict of interest, so they never should have had those board seats to begin with.
2. Sam agreed to be put in place under an internal investigation.
3. Helen and Tasha were replaced by Larry Summers and Bret Taylor. It's not that bad because Larry Summers is EA-adjacent and no-nonsense. Also, it's probably good if famous and well-connected academics like Summers really gets up-to-speed about the AGI race, which would get Washington DC up-to-speed as well. Bret Taylor I don't know too much about but as long as he has no financial conflicts of interest it's a win.
4. Adam is staying. This is a great win for the original board of Adam, Helen, and Tasha, like a symbol that their move was justified.

The fact that Sam threatened to lead a bunch of OpenAI employees to Microsoft is both evidence of bad character and a weakness that should be patched in the future when the board fires (or attempts to fire lol) the CEO.

[lavish pecan]

am I correct that Adam isn't a huge X-risk person? I know he's concerned about safety but it seems like it stopped short of X-risk?

not a criticism just a factual question

[fresh shore]

I have no idea

[lavish pecan]

that was my read of what I could quickly skim in terms of public statements and such

[fresh shore]

But if you think OpenAI's goal is AGI, then...?

[lavish pecan]

but I didn't spend a huge amount of time on it

[fresh shore]

Like, I don't understand how you could be an OpenAI board member and not think about AGI x-risks? Usually people who don't believe in x-risks don't believe in AGI

At the very least, you'll have Sam Altman brief you his plan to create AGI

[lavish pecan]

not really true imo, I've seen plenty that believe in AGI but don't believe it would be dangerous in an X-risk way

I assume he would still be in favor of slowing down because of non-x-risk dangers from AGI

[fresh shore]

I see

[zealous jetty]

Sure, they're not equivalent, but in practice EAs that are doing something AI related are overwhelmingly pro-AI safety. e/acc as a name is a play on EA, but it's a different demographic

[zealous jetty]

Strong disagree on "not ever". Never building ASI means dying from other xrisks eventually. And it's not even certain it would reduce AI risk over the long run. You could argue a permanent ban all but guarantees an irresponsible actor eventually builds it first. I also know some people that are currently not in the pause camp out of fears of a permanent ban that would do more harm than good, so also makes it harder to build coalitions..

[haughty oak]

I've seen Adam engaging in long x-risk discussions on Twitter. I'm not sure about his stance but he has engaged with the arguments seriously

[faint pumice]

Well the name "Q*" makes it kinda obvious what they're doing.

Does that make it the shortest infohazard ever at 2 characters?

[haughty oak]

my understanding is he thinks the problem is serious but solvable

[haughty oak]

you have to think of how the future would actually play out in the case of a "permanent" ban. I think the risk of progress stagnating because of it is ~0

[lavish pecan]

depends on the nature of the ban. if it's a complete moratorium on AI reserach including training of any size and capability, then that would pretty much stop it dead. if something less serious like ban training runs over X flops or somesuch, then I expect research would continue merrily albeit with some limitations.

[surreal rover]

Can someone please give a summary of the last 48 hours of developments?

[lavish pecan]

Altman back at OpenAI as CEO but not on board, same for Greg, Adam still on original board, rest gone, new board in place interim which will decide a final 9-member board. Also rumors that "AGI achieved internally" may have only been 1/2 a joke and there was some significant development at OpenAI in the last few weeks that some people wrote a letter to the board about triggering concerns (based on "sources").

I think that's about it condensed

[buoyant helm]

yep: ⬇️ ⬅️ ↪️ 🔄 🔄 🔄 ⤴️ ▶️

[haughty oak]

no it wouldn't, people would figure out ways to circumvent it

like distributed training

you'd have to ban all computers, even that wouldn't work long time as people would just develop alternative forms of computation, the ban would fall, etc

progress is unstoppable

so we have to BE the progress

[surreal rover]

This sounds like an e/acc argument. I'm not saying it is one, just that it sounds similar

[surreal rover]

Ok. I'm presuming this is as bad as it sounds from a safety point of view

[lavish pecan]

I don't think anyone really knows. It's worse than if the board had succeeded removing only Altman whilst replacing him with a safety-conscious CEO and not losing any staff. whether it's the preferable outcome to Altman and a large portion of staff going to Microsoft is debateable imo, but I can accept there's a good chance it might be. it will depend a lot on the composition of the final board I think.

[lavish pecan]

I'm not entirely sold on that. people could figure out ways to circumvent human cloning, but that moratorium seems to have been fairly successful. it's not necessarily a like for like comparison but it holds enough to prove a point I think

[surreal rover]

There are suggestions that this whole event was a power play by Altman to gain more control over the board. Does this seem supported by what we have seen?

[lavish pecan]

distributed training would require A LOT of computers and sounds a lot easier than it would actually be in practice imo

[lavish pecan]

stricly imo, I think all of the 4d chess hypotheses are pretty unlikely. so far he has less control over the board if anything with him and Greg both off the board. once again this will clarify more once we get the composition of the final 9 member board that the current one will nominate

[surreal rover]

How concerned do we need to be of this: https://arxiv.org/abs/2311.08105

[silent sierra]

There's also something to be said for having the public aware that AI companies aren't as virtuous or competent as they look

[lavish pecan]

yeah absolutely that I think was so far the only clear win from this, even when the overall result is still in doubt

[lavish pecan]

could be a problem, it's hard to predict how the tech might evolve, but as long as it requires some form of communication it's both detectable and restrictable. sure, this would end up in an arms race of offence vs defense, same as we have with current day malware (granted we're not doing massively great on that front) and if people were really determined and organized enough to do it, ultimately it could be done I guess. it would all depend a lot on the international participation in a moratorium, currently most malware actors are either state-sponsored or operate out of states with little legal enforcement. if you managed to prevent that, the odds would be far better.

[haughty oak]

and? what is the relevance of saying that? noncentral fallacying?

saying technological progress is inevitable (which I deeply believe in) is not the same as saying it's good

[surreal rover]

I would like you to think about what 'being the progress" means

[haughty oak]

sure but the incentives and barriers are wildly different

[haughty oak]

what I meant by it is human cognitive enhancement in some form

[surreal rover]

Yes, but the traps are still the same. I'm advocating being wise enough to not fall into them

[haughty oak]

also, I don't think the cloning ban will stand long term

[lavish pecan]

agreed. tbh I do also agree it would be extremely unlikely for something to be banned indefinitely. however I think likelihood of success does depend on what you consider to be "long term". I'd be happy if a ban gave us another 50 years for example. primarily because I'd get to live out the rest of my own life at least if not actually get some better ideas at how to crack alignment 😄 selfish I know but there we are.

to be clear though, I'm not arguing for a ban

[haughty oak]

I am

lol

[surreal rover]

A 50 year AGI-development pause looks very helpful. It would likely give enough time to solve the triple problem of alignment-coordination- surveillance

[haughty oak]

I see no way how we'd be able to realistically achieve a 50 year pause

only if all GPUs were banned

[surreal rover]

Would you rather ban all GPUs or be dead?

Also, I think we are heading off topic, perhaps a thread?

[haughty oak]

ban all GPUs, I'm talking about realistic cenarios, not the desirable ones

[surreal rover]

Ok

[lavish pecan]

you wouldn't even necessarily have to ban GPUs as such, just tightly control their sale

same as we do for restricted materials used for making explosives for example, but.. better 😄

the biggest challenge over all this is the international cooperation bit

you'd need an overwhelming majority in agreement with a treaty enforceable by force

nothing else would do imo

[haughty oak]

but there already GPUs out there

algorithmic improvements are probably enough with the current level of compute

[lavish pecan]

so just ban their usage and export 🤷

can't prevent distributed use of end-user GPUs I guess but you can't have everything 😄

[hidden socket]

Humankind will eventually go extinct. This is an overwhelming likelihood.

But getting AGI wrong could potentially result in things worse than extinction. There is perhaps only ONE thing which is potentially as bad as AGI in this regard (hostile extraterrestrial contact). And it's only as bad, not worse.

I truly, genuinely believe there is nothing which justifies the risk of building AGI.

[slate storm]

https://manifold.markets/RobinGreen/will-the-openai-board-contractually

[tawny linden]

Observer “at first” is what I read

[haughty oak]

I think the details are hard to predict

the most important thing to answer is who are Bret Taylor and Larry Summers

[haughty oak]

Microsoft and Sam will defintely try to get more power, but there are some forces against them

[fresh shore]

Very easy to put public pressure on the new board to not let Sam and Microsoft on the board. Remember our huge win:

1. Sam Altman threatened to lead a bunch of OpenAI employees to mass-quit, which is really bad (endangers national security and humanity if AI capability employees mass-leave). OpenAI is beholden to a nonprofit board with an overwhelming fiduciary duty to AI safety, whereas e.g., Microsoft is beholden to shareholders. (And who knows, the employees might split up. Some to Microsoft, others to foreign adversaries potentially.) The negotiation successfully prevented this outcome.
2. Sam, Greg, and Ilya lost their nonprofit board seats. They have a financial conflict of interest, so they never should have had those board seats to begin with.
3. Sam agreed to be put in place under an internal investigation.
4. Helen and Tasha were replaced by Larry Summers and Bret Taylor. It's not that bad because Larry Summers is EA-adjacent and no-nonsense. Also, it's probably good if extremely famous academics like Summers really gets up-to-speed about the AGI race, which would get Washington DC up-to-speed as well. Bret Taylor I don't know too much about but as long as he has no financial conflicts of interest it's a win.
5. The public narrative trajectory for OpenAI looks really bad (Sam is unaccountable, AI industry is a natsec&humanity risk due to employees' conflicts of interest, AI safety inside the companies was always BS), and we can super easily help this effort out. (Important for pressuring new board to be very careful and highly prioritizing safety when they choose additional board members.)

6. Adam is staying. This is a great win for the original board of Adam, Helen, and Tasha, like a symbol that their move was justified. Also, strategically important: Adam can give relevant information from the old board's context (including why they fired Sam) to the new board, as a member.

Note that the only outcome that's even better than the current one (Adam+two new board members & Sam internal investigation) is total capitulation by Sam (old board still has voting majority & Sam internal investigation). This is actually arguably the second-best possible outcome of the negotiation, where the first-best outcome is Sam's total capitulation. Sam leaving for Microsoft would have caused extreme chaos given the OpenAI employees' mob mentality, so from what we know today, it would have been risky for natsec and humanity.
Important to emphasize that Sam, not the board, jeopardized natsec&humanity in this way. Firing the CEO is very normal. Brainwashing employees into threatening a mass defection (to Microsoft, but who knows where they would have gone to, maybe not even US companies) shows extremely bad character, and is a HUMSEC weakness that needs to be patched going forward: not just for OpenAI, but for the entire AI industry.
I think what remains to do is we need to put a lot of public pressure so that the new board, when they select additions to their board, are very careful and highly prioritizing of safety. This shouldn't be that hard, as OpenAI board has an overwhelming fiduciary duty to AI safety. (https://x.com/GavinSBaker/status/1726634033164034097?s=20) We need to strategically put a lot of public pressure so that Sam, Sam allies, Microsoft, and other parties with conflicts of interest will never get on the new board.

[fresh shore]

Clarification on why I keep mentioning natsec: Larry Summers is very natsec-sympathetic, so important to highlight that getting the OpenAI employees to sign a threat to mass-quit is both a threat to national security and humanity's safety.

Big picture, we need to put a lot of public pressure and spread arguments that can convince either Larry Summers or Bret Taylor (ideally both) to choose highly pro-safety people. Adam has my full trust, given that the old board as has made this courageous attempt to remove conflicts of interest from the board, and I think Adam will speak for the old board (Adam+Helen+Tasha) in his decisions and actions.

[haughty oak]

Yup I agree this seems to have been mostly positive, and there will be continuing repercussions and increased scrutiny

we should continue to raise awareness and move the public opinion

[fresh shore]

^Key is to get journalists to look into the Altman-led OpenAI, thereby publicly pressuring the nonprofit board

[fresh shore]

https://pastebin.com/kL72w0n7

[barren lynx]

now that I think about it, it's clear to me that the board acter retarded

I support their cause, but they clearly could have pulled this off much better

1. have a compliant CEO candidate ready *before* you fire Altman

2. have a list of candidates ready for key maintenance positions so that the employees can't just threaten to shut down the company

3. wait to see which employees really mean it about moving to Microshit with Altman

that way they probably could have won this, if not by completely getting rid of Altman, they could have ensured that he's truly subservient to them, as he should be

but maybe this kind of foresight and strategic planning is something only Altman is capable of

[rotund zenith]

this would have been very useful- not sure if they could have done it quickly or discreetly enough to not have Altman catching wind and countering though

esp considering he was moving to have Toner removed for publishing criticisms of the company

[fresh shore]

Yeah, that's the problem. I think the board did the optimal play (risky, but ended up working)

It needs to be fast

[rotund zenith]

I've read your posts but I don't think I understand

how was this good?

what did we gain?

[fresh shore]

The removal of Sam, Greg, and Ilya

100% of the new board is outsiders with no/minimal conflict of interest

[rotund zenith]

Why was the removal of Ilya good?

[fresh shore]

He had equity and loved his colleagues

That's why Greg's wife could flip him by crying

[rotund zenith]

I don't think that was fake

[fresh shore]

Ilya's betrayal of the board allowed the board to avoid the hard decision (remove Ilya later?)

[fresh shore]

It definitely was not fake. Very real

It's a HUMSEC weakness (human emotion)

[rotund zenith]

what about the whole thing where openai's employees are now even more strongly loyal to sam? and maybe greg?

[fresh shore]

Who cares, we avoided a sam-controlled board catastrophe

The nonprofit (which in theory has control over the for-profit, i.e., can fire the CEO) has an overwhelming fiduciary duty to safety, is not beholden to shareholders https://twitter.com/GavinSBaker/status/1726634033164034097

[rotund zenith]

what would sam have done before that he can't do now?

[fresh shore]

Take control of the board and make himself an unfirable CEO

Claim OpenAI is a nonprofit with an overwhelming fiduciary duty to safety, even when he controls the nonprofit board

He can't do that now, depending on who the six new board members are

Who will be chosen by Adam, Larry Summers, and Bret Taylor

[rotund zenith]

is he firable now?

[fresh shore]

Yes

As long as the new nine-person board wants to fire him

(majority)

[lavish pecan]

I think a larger board of non-employees would definitely be a significant win. it does matter who will be on the board though as if there's a majority that bears a low likelihood of sanctioning Altman or can be easily swayed, then wether he can be fired or not is moot.

[fresh shore]

Yes. That's why we need to keep fighting the good fight. We can't let Microsoft allies, Altman allies on the board. Even among outsiders, some choices are better than others for AI safety

Need to put public pressure!

[haughty oak]

doing this would leak the info they wanted to fire Sam

[fresh shore]

Yeah exactly, They had a limited time window, and needed to act fast

[zealous jetty]

Can't think of any plausible AI S-risks... are there any sources on this you go share that you find persuasive?

[hollow coral]

It's plausible. Here's the LessWrong tag, which has links to more articles.

I also think Complexity of Value is a closely related topic, which helps explain why s-risk outcomes are plausible.

[mint oyster]

https://openaiboard.wtf

[lavish pecan]

some great points from Zvi's newsletter

[spiral mortar]

The GrAIvy TrAIn is unstoppable

[buoyant helm]

not regarding the issue at hand, that site makes me wonder about possibilities for 'twitter visualization'
that giant mood board of posts at the bottom is interesting

[fresh shore]

Wow

[mint oyster]

Hmm I guess it's an interesting visualisation if your aim is to persuade people. My own reaction when seeing this is to disregard it because I know I could cherrypick any 100 messages from twitter to pick any point I want to prove

It does carry more weight when the names are big

[fresh shore]

My prediction was correct; we won https://twitter.com/pitdesi/status/1729620915149750743

[lavish pecan]

we haven't won anything until we know the composition of the final board imo

[fresh shore]

True! Need to put up the public pressure

[slate storm]

https://www.threads.net/@ezraklein/post/C0MpLJNuN7U Ezra Klein seems to think that the board were telling the truth, is my reading of this

[barren lynx]

https://www.threads.net/@markmcan/post/C0MqlvKM8Gg

[slate storm]

Board member Helen Toner's resignation statement. Doesn't say much. https://twitter.com/hlntnr/status/1730034017435586920?t=iQ6JuXOcEoT350BXskKupA&s=19

[slate storm]

Finally some information on what happened, from the New Yorker https://twitter.com/rickasaurus/status/1730661545456116102?t=RBoUTZJtjm9R0but-SqpDg&s=19

[wheat ferry]

I am so sick of hearing about this story. I made a deep dive at the beginning of the whole affair and learned some interesting things. It soon became apparent that this is a tellenovella and not an existential risk story.

[fresh shore]

It's relevant for x-risk because it's about OpenAI

[wheat ferry]

It's more x-risk adjacent by now, but many things are.

[barren lynx]

the drama never stops https://fxtwitter.com/ilyasut/status/1732442281066832130

[faint pumice]

It's deleted now

[barren lynx]

I should have archived it probably, it said:

I learned many lessons this past month. One such lesson is that the phrase “the beatings will continue until morale improves” applies more often than it has any right to.

https://www.reddit.com/r/singularity/comments/18c8ces/ilya_i_learned_many_lessons_this_past_month_one/

[barren lynx]

it would be nice if he left OpenAI, in my opinion

just let Altman's cultists stay

Ilya could leave to xAI or Anthropic or whatever

[wheat ferry]

Our Boi @AISafetyMemes summarized the Business Insider article best. It is as we thought: no real unanimity between OpenAI employees on Sam. It was all a show for investors.
https://fxtwitter.com/AISafetyMemes/status/1732809562141405265

My personal impression is that the whole event deepened rifts already existing within openAI. Ilyas tweeted and deleted post is telling as well. Financial self interest and working on the cutting edge will prevent further escapades probably. But accelerated progress might be hampered in the future by bad work climate resulting from tensions between camps.

[spiral mortar]

roon posted a rebuttal. https://twitter.com/tszzl/status/1732927157897449856

[fresh shore]

but it's roon

[wheat ferry]

Fair enough. Might be damage control again.

[spiral mortar]

@fresh shore @wheat ferry Yes agreed! It's roon and doesn't at all contradict the OpenAI faction rift theory

[barren lynx]

I hope Ilya leaves. That would be a great blow to the company. https://www.businessinsider.com/openai-cofounder-ilya-sutskever-invisible-future-uncertain-2023-12?IR=T

[rotund zenith]

unlikely, imo. he's very closely emotionally connected to the people there

[rotund zenith]

ok, this seems like important new info

[signal nimbus]

Thanks. (And this pet thread is now officially for life not just for Christmas.)

[fresh shore]

Hypothesis: I think this is a signal that Adam is going to try to negotiate with Sam from a position of strength
https://twitter.com/adamdangelo/status/1739411681825247638

[dreamy meadow]

Not sure how you mean? Adam has been tweeting about AI news from a curious and tech businessy point to view for quite a while. This one feels fairly normal?

[fresh shore]

This is his first post since the Battle of the Board, has to mean sth