#Strawberry jam lobbies crash on newest core release

265 messages · Page 1 of 1 (latest)

slim goblet
oak harbor
#

are all your mods up to date?

slim goblet
#

yup

#

on a related note, ive also had stuttering after switching to core, not quite sure why

#

fixes itself after a bit

oak harbor
#

@rich zenith

weary karma
#

seems to be a known issue #modding_dev message

#

Just Core Things™️

slim goblet
#

wonderful

#

and the stuttering?

oak harbor
#

do you use vulkan?

slim goblet
#

im not sure

#

i have no graphics option set in everest-launch

#

should i deliberately set it to d3d11 or opengl

oak harbor
#

no normally it should be fine on the defaults thinkeline

oak harbor
#

@slim goblet can you check if the crash also happens on core version 4111?

slim goblet
#

it doesn't

oak harbor
#

@rich zenith looks like it's an issue with the new monomod version then

rich zenith
#

That's great, because that should not have any influence

#

Do we have a dump of all detours which apply to that particular method?

oak harbor
#

you can get a full dump with the logdetours command

#

and then ctrl+f that laugheline

rich zenith
#

Ideally we would need one of both versions

#

I am busy rn, so I can't get a log myself rn (don't have easy access to my installs rn)

#

But I might be able to take a look early if someone else manages to dump them in the mean time

slim goblet
#

(do you just want all of them?)

oak harbor
#

yeah probably

slim goblet
slim goblet
#

just happened again when trying to enter kevintechspam

delicate wharf
#

I have a feeling the ponytail mod might conflict with gravity helper

#

Plz try without one or the other

slim goblet
#

i dont have the ponytail mod

#

...do i?

rich zenith
#

@slim goblet Compared both logs - none of them mention any IL hooks on Celeste.Player::Update (the method which Helping Hand's hook crashes on)
I assume this means that either something is wrong in the dump, or it's not conflicting with an external mod and the hook itself is just broken (@weary oyster in that case)

#

Wait it's orig_Update, not Update

#

Yeah there are a lot of orig_Update hooks maddyplant

oak harbor
rich zenith
#

Considering old MonoMod had a wrong hook order, I disagree pausefrogeline

oak harbor
#

what I'm saying is that anything that works on stable should work on latest core without people having to change their mods

rich zenith
#

Yeah ik

#

I'm gonna try a bunch of stuff tomorrow

oak harbor
rich zenith
#

Assuming my brain will be able to fight the new time blackhole tomorrow peaceline (see #modding_offtopic)

slim goblet
slim goblet
#

i see a detour from helping hand but no ILhook lol

oak harbor
#

are you dumping while in a level?

#

I think some helping hand hooks are applied at runtime

slim goblet
rich zenith
#

It's hook ordering

#

I just temporarily reverted the hook ordering fix I got into MonoMod, and now it works again

#

Which is weird because it now behaves like legacy

oak harbor
#

wait but the previous reorg hook ordering was different from legacy too, wasn't it?

#

I'm confused

rich zenith
#

Yep

#

Fixing it to be like legacy broke it

#

I have a feeling it might be related to mod loading order

#

Where core has a different order from legacy, and the sheer coincidence that the ordering logic changed in the same way "fixed" it

oak harbor
#

why does core have a different order from legacy pausefrogeline

rich zenith
#

It used to have

#

Oh wait you are talking about mod loading order

#

Well, that's just a suspicion

#

Truth is that Everest has no defined loading order at all

#

See inode loading BS

#

It just so happens to be the order the FS API returns stuff in

oak harbor
#

have you figured out which hooks are being applied in a different order?

rich zenith
#

Hooks with the same priority (hooks with no explicitly defined priority have priority 0)

#

So TLDR all hooks which are not part of a context (and I think that pretty much includes all of them) are reversed

oak harbor
#

oh boy

rich zenith
#

Note that "reversed" has the connotation that now it's broken

#

Their were reversed compared to legacy before this

#

Now they aren't, and the fixed order breaks the hook

oak harbor
#

so what's different to legacy now

rich zenith
#

In theory nothing

oak harbor
rich zenith
#

Which only leaves mod loading order

#

I don't have a dev env for legacy (yay gladeline)

#

So I can't compare internals between the two

oak harbor
rich zenith
#

No, it's not that I am too lazy to set one up

#

I just can't get one working on Linux

#

I already tried a few times

#

It always ends up compiling in a few calls to newer APIs it shouldn't have access to youknowicattodoittoem

oak harbor
stable zodiac
#

Reading this gave me a heart attack I was sure hook ordering was fix

rich zenith
#

It should be

#

The issue is that the correct hook order is now causing issues cluelesseline

stable zodiac
oak harbor
#

do we have minimal reproduction instructions for this

#

testing shit on a full SJ install is so painful

rich zenith
#

Nope

#

There are like 18 mods which mess with the method at fault

#

And any of them might conflict

#

Unless we can safely disable some SJ helpers we can't narrow this down

oak harbor
#

could we dump the IL of the method

#

oh wait it's Player.orig_Update isn't it Catstare

rich zenith
#

Yep

#

The worst method by far to have a conflict in

oak harbor
#

actually wait, it's not on first level load, it happens in more specific situations

rich zenith
#

And for some reason MonoMod screws over my intelisense now

oak harbor
#

so it has to be an issue with how hooks that are added/removed at runtime are handled

rich zenith
#

Because helping hand adds some hook during runtime for some reason

rich zenith
#

Wait

#

Unless Before/After/Priority/etc are switched on legacy for ILHooks pausefrogeline

#

I will cry in that case

oak harbor
#

wasn't that also the case for on hooks?

rich zenith
#

No like

#

I mean that the meaning of priority flips for IL hooks compared to On hooks

stable zodiac
#

No way

#

That would make literally 0 sense

oak harbor
#

that would be very jank

stable zodiac
rich zenith
#

Only issue:

#

How would we convince the MonoMod team to mutilate reorg with that pausefrogeline

#

(in case it's true)

oak harbor
#

isn't that something we could do as part of the compat layer?

#

we already switch the meaning of before and after for legacy mods

#

so we could just selectively do that for only one hook type, right?

rich zenith
#

We couldn't

#

Because we can't switch the way hooks with the same priority are ordered

#

That code is in reorg, and it's shared between On. and IL. hooks

oak harbor
#

first we should check whether this is actually a thing before we try to figure out a fix

rich zenith
#

I'm currently working on logging all hooks

oak harbor
#

like logging when hooks are added/removed?

rich zenith
#

No like logging all hooks to orig_Update in chronological order with the exact prio / before / after they use

rich zenith
#

All the affected hooks

#
  • CommunalHelper: prio=0 before=[] after=[] | Void Player_orig_Update(MonoMod.Cil.ILContext)
  • CommunalHelper: prio=0 before=[] after=[] | Void Player_orig_Update(MonoMod.Cil.ILContext)
  • EmHelper: prio=0 before=[] after=[] | Void Walkeline_triggerupdate(MonoMod.Cil.ILContext)
  • MaxHelpingHand: prio=0 before=[] after=[*] | Void modPlayerOrigUpdate(MonoMod.Cil.ILContext)
  • MaxHelpingHand: prio=0 before=[] after=[] | Void ilHookCameraTarget(MonoMod.Cil.ILContext)
  • XaphanHelper: prio=0 before=[] after=[*] | Void modPlayerOrigUpdate(MonoMod.Cil.ILContext)
  • XaphanHelper: prio=0 before=[] after=[] | Void ilHookCameraTarget(MonoMod.Cil.ILContext)
  • XaphanHelper: prio=0 before=[] after=[] | Void patchOutStamina(MonoMod.Cil.ILContext)
  • ExtendedVariantMode: prio=0 before=[] after=[] | Void patchOutStamina(MonoMod.Cil.ILContext)
  • ExtendedVariantMode: prio=0 before=[] after=[] | Void modCanDash(MonoMod.Cil.ILContext)
  • ExtendedVariantMode: prio=0 before=[] after=[] | Void patchNoRefills(MonoMod.Cil.ILContext)
  • ExtendedVariantMode: prio=0 before=[] after=[] | Void modCoyoteTime(MonoMod.Cil.ILContext)
  • FrostHelper: prio=0 before=[] after=[] | Void modFeatherState(MonoMod.Cil.ILContext)
  • VivHelper: prio=0 before=[] after=[] | Void Player_origUpdate(MonoMod.Cil.ILContext)
  • CrystallineHelper: prio=0 before=[] after=[] | Void Player_orig_Update(MonoMod.Cil.ILContext)
  • CrystallineHelper: prio=0 before=[] after=[] | Void Player_IL_Update(MonoMod.Cil.ILContext)
  • StrawberryJam2021: prio=0 before=[] after=[] | Void playerOrigUpdateHook(MonoMod.Cil.ILContext)
  • MaxHelpingHand: prio=0 before=[] after=[] | Void modCollideChecks(MonoMod.Cil.ILContext)
  • GravityHelper: prio=0 before=[MaxHelpingHand, SpringCollab2020] after=[] | Void Player_orig_Update(MonoMod.Cil.ILContext)
  • MaxHelpingHand: prio=0 before=[] after=[] | Void filterOutJumpThrusFromCollideChecks(MonoMod.Cil.ILContext)
#

As I suspected all of them are prio=0

#

GH has a before on Helping Hand, but I doubt that's the issue considering Before/After handling is unaffected thinkeline

#

Ok, legacy invokes ILHook modifiers in the order of low priority -> high priority (for whatever reason)

#

Looking into reorg now pausefrogeline

#

Aaaaaaaaaaaaaaaaaaaand I think reorg invokes them in the opposite order (if I read the code correctly) maddyplant

#

:yippeee:

#

Yep it does

#

@oak harbor gladeline

oak harbor
rich zenith
#

Ok, so we're sorta screwed frogeline

oak harbor
#

is this a conscious decision with a good reason, or just an accidental result of the rewrite?

rich zenith
#

We can ask

oak harbor
#

if it's the latter, maybe we could suggest to change it for backwards compat reasons

rich zenith
#

But I assume it's intentional

#

Considering it makes sense that hooks with high priority are applied first

#

Especially because in reorg, "Before" means a hook has a higher relative priority

oak harbor
#

you're gonna hate this, but one option would be forking monomod to the everest org gladeline

rich zenith
#

Maybe this is why reorg's ordering of Before/After is screwed

#

*legacy

#

Because for ILHooks, legacy actually made sense

#

Before means an ILHook is applied before another one

#

I'm gonna bring this up in the MonoMod discord

#

If we could introduce some hidden "switch order" flag somewhere this would already solve it

#

But not sure if they would be up to that

#

If that fails, I'll go for the nuclear option

#

Use hooks to hook MonoMod itself youknowicattodoittoem

#

May god have mercy on our souls in that case

oak harbor
rich zenith
#

I will not regularly bump an entire repo landeline

#

Honestly the most realistic outcome is that MonoMods add a switch (yes they have an entire mechanism for that) which switches the order

#

Then Everest just turns that on and everything is fine again

oak harbor
rich zenith
#

It's like one whole more button press MADeline

oak harbor
#

also lets be honest we only bump monomod versions if we need a new monomod feature

rich zenith
#

I do it whenever I merge from dev as well 3celine

#

Just as a sort of "Everest houskeeping routine"

#

But yeah

oak harbor
#

it's a tradeoff between getting new features and potentially breaking stuff

#

the average rate of commits that break stuff on core is much higher than we would like in normal development

rich zenith
#

Hopefully this will be over after this commit laugheline

#

Because then we are finally in a state of "everything works" again

#

Honestly after this is over I have nothing left to work on for Core

#

FYI We also got the go-ahead from the SRC team, so the next push will reintroduce that patch

#

Also we should probably document the true meanings of Priority / Before / After somewhere

#

This entire debacle is a mess

#

Maybe as a #code_modding pin?

oak harbor
#

wiki

rich zenith
#

Not sure where we would put that

#

And I don't wanna touch the Everest wiki again after the incident distracteline

oak harbor
#

what incident, I don't remember this

rich zenith
#

(GH web editor leaked my private email with full name; I force pushed the wiki repo because of that btw, don't think anyone noticed till now)

#

Doubt anyone has a clone of the Everest wiki repo sitting around somewhere, so I figured it was fine

oak harbor
#

you should probably tick this

rich zenith
#

I did

#

But for some reason, it didn't work

oak harbor
#

or you can clone the wiki and use normal git methods to edit it

rich zenith
#

Yeah ig

#

I would say we put it in a #code_modding pin in addition to the wiki though

oak harbor
#

anyway whenever someone suggests discord as a method for storing documentation, an archivist dies

rich zenith
#

Fair

#

That's why I would say we put it into both

#

Because a fair amount of modders are also active here

#

Just checked btw: flipping IL hook order in MonoMod fixed the issue frogeline

#

I can now play paint again without any jank peaceline

rich zenith
#

Not looking good 3paul

#

I'll sleep now, hopefully we can get something into the codebase

weary oyster
#

Is there something I can fix on my end ohnoshiro

#

Is there a way to make it spit out the IL of all IL-hooked methods or something?

#

Like, the IL it's trying and failing to "compile"

stable zodiac
rich zenith
#

Nope, because we would flip it in the LegacyILHook code

stable zodiac
#

good news I think I found a way to reproduce this bug bad news is that its present in normal On hooks as well sadstare

#

Ill share it soon, making sure I didn't mess up anything

rich zenith
#

The bug can't affect On. hooks

#

Because the bug is literally in the IL hook code

stable zodiac
#

so its a new one then

rich zenith
#

Please no

stable zodiac
#

well its super jank but this is my setup, you only need the 3 mods in the zip file.
if you run this on stable you get something like this:

(08/09/2023 20:02:58) [Everest] [Info] [MayMay] About to patch IL in second with no priority 
(08/09/2023 20:02:58) [Everest] [Info] [MayMay] stfld
(08/09/2023 20:02:58) [Everest] [Info] [MayMay] About to patch IL in first with priority 1
(08/09/2023 20:02:58) [Everest] [Info] [MayMay] ldfld
(08/09/2023 20:02:58) [Everest] [Error] [MayMay] !!!!!!!!!!FIRST RAN AFTER SECOND!!!!!!!!!!
(08/09/2023 20:02:58) [Everest] [Info] [MayMay] About to patch IL in third with priority 100
(08/09/2023 20:02:58) [Everest] [Info] [core] Module Third 0.1.0 registered.

and on core you get this:

(08/09/2023 20:03:31) [Everest] [Info] [MayMay] About to patch IL in first with priority 1
(08/09/2023 20:03:31) [Everest] [Info] [MayMay] stfld
(08/09/2023 20:03:31) [Everest] [Info] [MayMay] About to patch IL in second with no priority 
(08/09/2023 20:03:31) [Everest] [Info] [MayMay] ldfld
(08/09/2023 20:03:31) [Everest] [Info] [MayMay] About to patch IL in third with priority 100
(08/09/2023 20:03:31) [Everest] [Info] [core] Module Third 0.1.0 registered.

For the normal hooks inconsistency If you dash in stable you are fine but on core you crash, its due to the fact that both On hooks have no default value so there load order is swapped not sure if we will ever see this in the wild.

#

if you don't like the fact that its a random zip file from a random stranger lmk ill upload it to git

rich zenith
#

The swapped priority for IL hooks is known and the underlying issue here (stable goes lowest -> highest, core does the opposite)

stable zodiac
#

its just the minimal amount of mods needed to reproduce this

#

for the IL hooks

rich zenith
#

I'm a bit confused why About to patch IL in third with priority 100 comes after priority 0 thonkeline on core

stable zodiac
#

its supper funky, but from what I have seen it seems that new IL hooks always run last the first time they get applied ( not tested at all )

rich zenith
stable zodiac
#

for the On hooks

rich zenith
#

Where are On. hooks involved here? thonkeline

stable zodiac
#

if you dash

#

you crash on core

rich zenith
#

Honestly I have no clue what you are testing or what is breaking here

stable zodiac
#

stable is fine

#

I just tested the order of hooks with no config. On stable it's Second then First and on core its First then Second.
The issue is that both mods have the same default prio and the reorg changed the loading order of hooks with the same prio

weary oyster
#

I think going from lobby to paint makes the jumpthru hooks on orig_Update apply in the sideways > upside-down order (just because the lobby has no upside down jumpthru I guess), while a level that has both usually does the opposite

#

I could just make it consistent I guess 🤷‍♀️

rich zenith
#

Because that exact issue is fixed on latest core

stable zodiac
#

Righttt I need to sync fork on Github to update my fork

#

no it wasn't on the latest version my bad

#

ill test it again after I am done with dinner

stable zodiac
#

Okay good news, The only thing that is broken is my credibilitycatfacepalm IL hooks, the On hooks work as expected on the latest core version so that's great, and for the hook IL hook ordering I get:

(08/09/2023 22:50:56) [Everest] [Info] [MayMay] About to patch IL in third with priority 100
(08/09/2023 22:50:56) [Everest] [Info] [MayMay] About to patch IL in second with no priority 
(08/09/2023 22:50:56) [Everest] [Info] [MayMay] ldc.i4.0
(08/09/2023 22:50:56) [Everest] [Info] [MayMay] About to patch IL in first with priority 1
(08/09/2023 22:50:56) [Everest] [Info] [MayMay] ldfld
(08/09/2023 22:50:56) [Everest] [Error] [MayMay] !!!!!!!!!!FIRST RAN AFTER SECOND!!!!!!!!!!

its still kinda weird that the patch order is priority 100 -> no config -> 1 but w/e. sorry for the false alarm widegladeline2 .

rich zenith
#

No config is priority 0

#

Reorg treats no config hooks differently, but the compat layer force-assigns all hooks a config

rich zenith
#

Wait I just realized what you actually said 3paul

#

100 -> 0 -> 1

#

@stable zodiac 1. is that only for IL. or for On. hooks as well 2. can you send the code snippet you use for this

#

I have a suspicion that it's not applying priority at all

#

Because it goes third -> second -> first, which is the expected order when everything has the same prio

delicate wharf
#

it inserts an OpCode.Ret at the start of the MHH hook code baddyhueh

#

if MHH is allowed to add its jumpthru collision code it breaks gravity helper

#

same with the SC2020 variants

#

the catch with the SC2020 version is that (unless it's changed?) it adds the hook on startup rather than map load

stable zodiac
#

create a basic mod using dotnet new celestemod --Core and use the Load function from here:

stable zodiac
#

Okay I think I found the issue

#

Yay my fix works

(08/10/2023 12:55:21) [Everest] [Info] [MayMay] About to patch IL in 5th with priority 150
(08/10/2023 12:55:21) [Everest] [Info] [MayMay] About to patch IL in 3rd with priority 100
(08/10/2023 12:55:21) [Everest] [Info] [MayMay] About to patch IL in 4th with priority 50
(08/10/2023 12:55:21) [Everest] [Info] [MayMay] About to patch IL in 1st with priority 1
(08/10/2023 12:55:21) [Everest] [Info] [MayMay] nop
(08/10/2023 12:55:21) [Everest] [Error] [MayMay] !!!!!!!!!!FIRST RAN AFTER SECOND!!!!!!!!!!
(08/10/2023 12:55:21) [Everest] [Info] [MayMay] About to patch IL in 6th with no config
(08/10/2023 12:55:21) [Everest] [Info] [MayMay] About to patch IL in 2nd with no config
(08/10/2023 12:55:21) [Everest] [Info] [MayMay] nop
(08/10/2023 12:55:21) [Everest] [Error] [MayMay] !!!!!!!!!!SECOND DIDN'T FIND OPERAND!!!!!!!!!!
rich zenith
stable zodiac
#

I think so, by upstream you mean Everest/Core right?

rich zenith
#

yep

stable zodiac
#

so it is

#

but I am almost certain its a different issue then what's going on with SJ, currently playing paint to test it

rich zenith
#

I see the issue

stable zodiac
#

well

#

good news SJ didn't crash

#

not sure why tho

#

I was sure the current SJ crash was due to hooks with the same prio running in a different order then in stable

rich zenith
#

The SJ lobby crash is already solved btw