Hotpatching Rust code 🔥

[mellow flint]

it has an entity() method

[elder heart]

The only thing now though is that we replace RegisteredSystem's system field with the local function by initializing it via System trait with that old system's local data. However, this isn't a public struct. but that system will get access to the local state

I feel like I'm just saying words now, system has lost all meaning temporarily. this seems possible

[mellow flint]

I'm completely fine with leaking a bit of memory on hot-patching and fixing that in a follow-up

[mellow flint]

Yeah same. Quite a few functions are now the "system"

[elder heart]

how do you feel about flaky macros

[mellow flint]

what do you mean?

[elder heart]

introducing bevy_kill_my_system_params. Inspired by bevy_kill_my_cache, this macro will add a random amount of unused system parameters to ensure your system's typeid never remains constant

[mellow flint]

ship it

Seriously, it's a prototype crate 😄

If that's what it takes, that's fine by me

Most important is the user experience imo

I have to go to a thingy now. I'll still be on my laptop, but I don't have much in the way of a coding setup on it. But I can still review stuff 🙂

[idle stone]

you can do a #[derive(SystemParam)] struct Foo<const N: usize>; with random (or strictly increasing) N maybe

[mellow flint]

Could we also use register_system and cache the system id ourselves, then register the system again on a hot patch?

Per the docs, registering a system multiple times is allowed

[elder heart]

would like to do this as long as the world is tricked into thinking the changed function is the original function for those locals

but I think that without doing this, that function pointer's not going to change since the typeid doesn't change. it's still in memory somewhere...but idk

maybe we need a third function

you have
hotpatch which yknow, for caching
outer for being registered in world so those local vars keep state
inner - the user-facing fn, which outer for passes its local state and other query params into
so now the registered system doesn't change either

[mellow flint]

Yeah the naive version would just clear locals whenever the system is hot-patched

There’s also the possibility of making #[hot] not support changing signatures by default

Since the code generated by that variant is supremely trivial, it would support everything we can throw at a system
Trouble only starts when you change signatures or want to rerun systems

[viral dust]

Ey nice, dx serve --hot-patch works in PowerShell too, so my normal VSCode editing experience "just works"

[mellow flint]

Oh nice! 😄

[viral dust]

I have noticed I need to run cargo clean and usually stop RA too, but otherwise it's much more seamless now

[mellow flint]

I found the getrandom "drama" very entertaining to read btw 😄 certainly made my commute more interesting haha

[mellow flint]

Why do you need to do these two?

[mellow flint]

People in this chat: thoughts?

[viral dust]

Sometimes dx serve fails to link if the project is already built. I think Jon mentioned it was because there weren't any linker flags to copy if no linking happens

[mellow flint]

Ooh I see

[viral dust]

RA I suspect for the same reason I need to clean; it starts building so it might prevent linker environment variables being set for dx to see

Or just otherwise "mess" with things

[mellow flint]

You probably know this, but you can do cargo clean -p my_crate to only clean the tip 🙂

[viral dust]

I actually didn't know that, I'll try that next time!

[viral dust]

I think we're reaching a point where we need a debug UI with buttons to control reloading tbh

Ideally a checkbox to control whether the reload should happen immediately, and a button that's active when a reload is pending

That way things like struct migrations, signature changes, etc. can be controlled by the user

[mellow flint]

Fair enough, I see the merit in that. But I primarily want a prototype in which slapping #[hot] on a function will work for most usecases

Having a side by side view of the game and the code at the same time and seeing my changes without even moving my mouse at all was kinda electrifying, ngl

[viral dust]

Oh yeah definitely pick good defaults. I think signature changes should probably not be default-hot-reloaded, since they have things like locals which could be lost and other possible hitches

[mellow flint]

Though I wouldn’t be opposed to also adding a visual / hotkey mode

[mellow flint]

Yeah I'm gravitating to the same conclusion

"Here’s the default with very few caveats"
"Here’s the spicy features with lots of caveats"

[elder heart]

definitely want a jesus-take-the-wheel feature

[raven valley]

This is where the argument for the editor being in-process starts to take hold again

[elder heart]

I think we can fix this bug. I've failed twice and sleepy but will try at it in the am

[mellow flint]

Though I will say, warts and all, writing and adding new systems at run-time on @raven valley's PR felt like wielding some kind of forbidden dark power

[elder heart]

yeah that really shewn the light

[viral dust]

I think that power feels good right now because we're missing other tools that are better suited for that niche. Like, editing UI elements spawned at startup. That should be something handled in an inspector or BSN hot reloading rather than patching. Bevy needs both for a wholistic "fast" experience IMO

Basically code vs data

[mellow flint]

You’re definitely right in that regard! For UI, this is more of a stopgap than anything, agreed. Though it’s a really really good stop gap compared to what we had before.

I will say though that I had quite the "aha!" moment when my roommate was standing next to me and telling me how to change the n-body simulation at runtime, e.g make the black holes bigger the longer you click on it

Being able to just hack in feature after feature without recompiling is quite mindblowing.

And doing fun things like adding a query to despawn certain objects for testing, then spawning some others in.
Again something that is the job of an editor, if we had one

[quartz whale]

I'm glad it worked! And I'm glad you figured out how to do, sorry for dipping out, I forgot to check discord 😂

[mellow flint]

Hehe all good 🙂

[quartz whale]

I think the first time I got that aha feeling of hot reloading was editing shaders and having then automatically update the visuals, but this is basically the same thing, but for logic, which is marvelous

Is there a video showcasing the hot reloading yet? Would be really nice to have it in the readme so I can send it to my friends 😄

By the way, how does the hotpatching run so quickly? Do you just have super fast computers, or is hot patching faster than a normal recompile somehow?

[mellow flint]

a few, sec 🙂

[raven valley]

Its extra fast to hotpatch compared to normal recompile

In fact with the dynamic linking bevy version he was recompiling in 150ms

[quartz whale]

I'm definitely gonna have to mess around with this over the weekend 😄

[raven valley]

Hell yeah

[mellow flint]

Very little is actually done during the hot patch, most is just changing the jump tables 🙂

[raven valley]

Has anyone tried doing the hot patch without debug info + crainlift?

• dynamic linking?

How fast is that on Linux with mold

[mellow flint]

• https://discordapp.com/channels/691052431525675048/730525730601041940/1374181112758276147
• https://discordapp.com/channels/691052431525675048/1374187654425481266/1374389245728395344
• https://discordapp.com/channels/691052431525675048/1374187654425481266/1375291982687109151

[mellow flint]

0.3 s on a toy project, 3 s on Foxtrot

That is without dynamic linking, as that is broken on Linux

[raven valley]

Did u remove debug info?

[mellow flint]

note: wild is even faster, but does not work with subsecond right now

[mellow flint]

hmm nope

should try that later

[raven valley]

Wahhhh

Yes

We gotta just maximize the speed

I wanna get sub 100ms recompile hehe

[idle stone]

it's 4-8s for me :')

[quartz whale]

Thanks!

[median vale]

maybe crainlift would help with speed?

there's an experimental backend for rust which is pretty quick

[quartz whale]

Looks the dynamic linking is fixed by https://github.com/DioxusLabs/dioxus/pull/4174

[quartz whale]

That's not as good, but how are your recompiles normally? Especially with dynamic_linking

[idle stone]

like 5-10s i think

the real timesave is not losing my in-game context though

[quartz whale]

Yeah, it could even be several seconds slower and it would still be better than restarting 😂

[idle stone]

i'm already happy but i would be even happier with actual subsecond hotpatching (on my machine / project)

[mellow flint]

Here's my config: https://gist.github.com/janhohenheim/5731c11e91736bab5e9ef58c2a982c36

Happy to get suggestions 😉

[raven valley]

https://www.reddit.com/r/rust/s/7VijXteq9l

[quartz whale]

It took me until this moment to understand the name, I feel a fool 😭

[idle stone]

interesting. bevy_wasm_bench didn't pick up any improvements from strip = debuginfo or panic = abort, but i guess debug = none has an impact

[mellow flint]

Nice link inside: https://nnethercote.github.io/perf-book/build-configuration.html

[idle stone]

debug = none reducing compile times is the only thing new to me here (after the many days of workflow optimization 😩)

but i think release builds already default to debug = none and my dev builds in CI already have debug = line-tables-only for cache size savings

[mellow flint]

same haha

[idle stone]

this link is also interesting: cargo-wizard

basically .config_fast_builds.toml as a cli tool

[idle stone]

oh also -Zthreads=8 being recommended over -Zthreads=0

[mellow flint]

Oh I didn't catch that

[late stratus]

wheres your target dir stored? Rust reads/writes a lot during compiles, and having target dir on a spinny disk is a fast way to kill compile times

might even be worth detecting and warning about that at some point

[celest magnet]

Has it already been reported that events are triggered twice when a system is annotaded with #[hot]?

[elder heart]

getting pulled into a work thing for a few hours but will try again on the Local, Changed/Added query, and EventReader types later

hopefully someone else will figure it out 😅

the thing that's irking me the most is observers honestly

I feel like there’s a way to caress (this is the bevy discord) two birds with one stone

[late stratus]

have yall considered implementing System on a custom type? like, in a way which allows you to exchange the function being called internally but allowing you to keep the SystemState around as a separate field?

from what i can tell, currently theres only nested functions and systems are stored in a resource, but maybe you could store most state in the "system" itself if that system was a struct and not a function?

i can imagine that making it easier to deal with SystemState, because dealing with it seems to be part of the System trait implementation. like, thats also how bevy works internally, a system gets stored as a struct with its state. see: https://docs.rs/bevy_ecs/0.16.0/src/bevy_ecs/system/function_system.rs.html#597

[late stratus]

it might even be possible to allow them to not be exclusive systems, if and only if ScheduleGraph::build_schedule is enough to refresh this. after digging in, i dont think it is, but yeah

[elder heart]

really like this idea

only question is how to handle local state

If we were to handle local query data outside of World, something feels off about that. I'm gonna try out this direction though

[late stratus]

well, local state handling was why i had this idea originally lol

[elder heart]

oh, please elaborate!

I don't know all the ins and outs of local state storage

[late stratus]

pub struct FunctionSystem<Marker, F>
where
    F: SystemParamFunction<Marker>,
{
    func: F,
    state: Option<FunctionSystemState<F::Param>>,
    system_meta: SystemMeta,
    archetype_generation: ArchetypeGeneration,
    // NOTE: PhantomData<fn()-> T> gives this safe Send/Sync impls
    marker: PhantomData<fn() -> Marker>,
}
struct FunctionSystemState<P: SystemParam> {
    /// The cached state of the system's [`SystemParam`]s.
    param: P::State,
    /// The id of the [`World`] this system was initialized with. If the world
    /// passed to [`System::update_archetype_component_access`] does not match
    /// this id, a panic will occur.
    world_id: WorldId,
}

this is one way bevy stores systems

so, all you'd need to do is make sure that FunctionSystemState persists after hotpatch

how? recreate this and its trait impls and go from there

[elder heart]

I love where this is going. This sounds fantastic. I'm gonna sandbox this

[late stratus]

(this is one of the things which implements System)

you also then have to implement System::run_unsafe which is this body:

#[cfg(feature = "trace")]
let _span_guard = self.system_meta.system_span.enter();

let change_tick = world.increment_change_tick();

let param_state = &mut self.state.as_mut().expect(Self::ERROR_UNINITIALIZED).param;
// SAFETY:
// - The caller has invoked `update_archetype_component_access`, which will panic
//   if the world does not match.
// - All world accesses used by `F::Param` have been registered, so the caller
//   will ensure that there are no data access conflicts.
let params =
    unsafe { F::Param::get_param(param_state, &self.system_meta, world, change_tick) };
let out = self.func.run(input, params);
self.system_meta.last_run = change_tick;
out

as you can see, this is is how the param state gets used: self.func.run(input, params) where params was given the state

so basically, you can just... store the state and only replace self.func with the new one

weeeelllllllll

its not gonna be that easy because if you want to support changing system params you'd need to somehow match/transfer from the previous params state to the new one

[elder heart]

that would be the cherry on top. Discarding all local state and starting with fresh state due to new typeid could be the first step

[late stratus]

well

you already have that no?

[elder heart]

Yes, but without any param that requires world cache

[late stratus]

?

ah

[elder heart]

so like any Local or Changed will not run correctly right now

[late stratus]

exactly

OH WAIT ofc the typeid would only change if the signature changes?

[elder heart]

right

[late stratus]

i get it, so you reduce the loss of state from "every time" to "only when signature changes" with this, thats what you meant

[elder heart]

yes!

[late stratus]

gotcha gotcha

anyways feel free to keep me up-to-date with what you're doing (or maybe we could hop in VC at some point?) - i've read through a lot of the systems/state/schedule code to come up with this idea so i might have some ideas

[elder heart]

however, @mellow flint's macro will actually produce an exclusive system world as you mentioned, getting half-way there. So this approach is kinda like, orthogonal to that.

yeah that sounds great!

I'm gonna work on this for a few hours and ping when I do something dumb

[late stratus]

oh sure, yeah this will not solve that, just the state

the System implementation would have to act like its a exclusive system, for now. at least for testing thats easiest

i did try to check and sadly, it seems bevy just doesnt expose the stuff needed to recompute schedule for a existing system after it changes, so no luck on getting rid of the exclusiveness requirement

[elder heart]

also feel free to try it yourself 😅 I'm doing this to learn, not necessarily to execute. there will be bumps

[late stratus]

like, its almost there, i feel it'd be a 5 line function in the Schedule to make it work

[elder heart]

yeah...seems ilke there's some really valuable stuff I found that isn't public

[late stratus]

maybe, but i'm assuming you know more about subsecond at this point than me

hell, i haven't actually tested the current state, i just saw the discussion, read the code, and thought of this

has someone kept track of a list of changes to bevy itself which would be good to have or well, needed to integrate this fully?

[elder heart]

you mean while hotpatching?

[late stratus]

yea

[elder heart]

yes, there's a resource called __HotPatchedSystems in bevy_simple_subsecond. Looks like:

    #[derive(Resource, Default)]
    pub struct __HotPatchedSystems(pub HashMap<TypeId, __HotPatchedSystem>);

    #[doc(hidden)]
    pub struct __HotPatchedSystem {
        pub system_ptr_update_id: SystemId,
        pub current_ptr: u64,
        pub last_ptr: u64,
    }
}

[late stratus]

like, if it was upstreamed what currently non-pub stuff in bevy should be changed, that kinda thing

[elder heart]

ohhh

great point.

[late stratus]

i would bet a lot that when considering upstreaming sth like this, its gonna look completelydifferent in the end, but it might be worth writing down the ideas everyone has had "if i could edit bevy source" to not loose that collective brainstorming

[raven valley]

Track location doesnt work with spawning on the normal commands api? It only works when spawning from world?

This seems like an oversight

[elder heart]

oh, hmm

I didn't know that...explains a ton

[elder heart]

So I've started out fresh...wondering if it would make sense to entry on some type that implements Condition

reason being...

pub fn plugin(app: &mut App) {
    app.add_systems(
        Update,
        (|| subsecond::call(|| tick())).run_if(common_conditions::on_timer(Duration::from_secs(1))),
    );
}

fn tick() {
    info!("owo");
}

as long as you connect the dioxus cli notification to subsecond::apply_patch, you can get away with a ton. I really really like the idea of an extension trait, somewhat similar to how Malek added the ability to add and remove systems

the idea is that calling app.hotpatch() would return some type that wraps App and essentially allows for hot reloading of literally everything. Interception of adding/removing plugins, observers, and systems. This might be a bike shed moment

but I've written code about 5 different ways now 😆 gotta start from the basics

oh and this actually sucks. This would break all plugins. maybe it needs to be called last.

[raven valley]

Yeah I think there's tons of potential designs here and we all should probably iterate through several

[mellow flint]

I have a PR almost ready that makes the default an ultra thin nonexclusive wrapper (that does not support signature changes)

[elder heart]

you don't sleep! that's cool!

[mellow flint]

I just woke up 😄

[elder heart]

hahaha lovely

[raven valley]

The sad part is I would have liked ~a few weeks to iterate on a buncha designs and come out with an awesome 3rd party crate but we only have 7 days left

For game jam

[mellow flint]

Well you still have 3 months for 0.17 :p

[raven valley]

Haha true

[mellow flint]

Francois PR already makes these a field on the systems themselves, FYI 🙂

[elder heart]

what

omg

it's okay. I'm so far out in another direction right now, totally lost it around 11am

[mellow flint]

Maybe you need sleep :p

[elder heart]

not until I can implement System

just wait (I feel like there's a lot of me saying this, but this time I mean it)

this is my current very small problem

like do I really need to go <T as System>::In

ok yeah

I need to sleep

use an =

[median vale]

yeah, because rust dosn't know what trait the associated type In comes from otherwise

[raven valley]

@median vale u did the track location work right?

Is there a reason it doesnt exist for commands.spawn?

@mellow flint guess what I have working

Auto despawn startup schedule

I did a gigabrained move

It only works if you use spawn directly on a &mut World tho, because track location doesnt exist on commands

[idle stone]

on my ssd. i'm using sccache currently

[median vale]

i mostly sniped aevery and specific protanist into doing the work for me

and i think spawning is implemented on main?

location tracking does exist for commands, it's just cursed

and behind it's own flag, i think?

[raven valley]

Hnm?

OH?

What flag!

Gimme gimme gimme

Wait or maybe im just stupid and it does have it

And I just didn't see

One sec

Nope

Okay si

I think it exists for components on commands

But yall just forgot to add it to spawn_empty

On commands

@fathom spruce @quick bloom

If this does exist as I think it does it would be goodly for a bug fix bevy patch before the jam 🥺

[quick bloom]

Good catch ❤️ I got World::spawn_empty but forgot about Commands::spawn_empty; that's the trouble with API duplication. Gonna make a PR in a minute.

[raven valley]

Yayy @languid sable could this make it into a 0.16.1?

[median vale]

mira, knower of locations

keeper of the source map

[opaque mulch]

now wishing for a literal map of the bevy source code lol

[quick bloom]

bevy_mikktspace getting labeled Hic Sunt Dracones?

[late stratus]

did you solve the SystemState problem? i might have sniped dsgallups into trying to solve that using a struct per system which implements the System trait which should allow solving that, but honestly, i hope that idea might be overcomplicating it and theres a simpler fix

dammnit, the more i think about this the more i want to try it myself

[raven valley]

though if we end up doing my thing of allowing you to write new systems during hot reload then caching local state for reload doesnt make much sense ( we may not end up doing write new systems but I hope we do!)

[late stratus]

well, my idea would still only work for the same TypeId at least by default. I'm basically thinking of overwriting how bevy runs systems, instead of overwiting the systems themselves

well

overwriting how bevy runs each of the #[hot] systems

[elder heart]

I'm going way beyond out into left field, do your thang

[late stratus]

fair fair

it did kind a sound like you were gonna go for something quite different

[elder heart]

I'm tearing into the fabric of reality searching for answers. can't even see my screen

[raven valley]

Im also overwriting how bevy runs systems ( you dont need the hot macro with my reloadable closure lol )

!!

Lmao

Though u can add it

It doesnt hurt

[late stratus]

do you implement System?

[raven valley]

Nope

I do schedule stuff

U can look at my old pr on Jan's repo

[late stratus]

aight so my option is still not explored

dont wanna duplicate work

[raven valley]

Yeyeey

Yeah go for your option im interested to see what lies yonder

[celest magnet]

Why is the #[hot] annotation making the EventReader have duplicate events?

[late stratus]

because it doesnt store local state as of yet

see the limitations documented at the bottom of readme

[idle stone]

it effectively makes your system an exclusive system called by run_system_once

[celest magnet]

I think I glanced over that part 😅

[raven valley]

OHHH

[celest magnet]

Its interesting that even on a normal run (without hot reloading) it still triggers twice when annotated.

[late stratus]

did something just click?

[raven valley]

Yeah

I was trying to debug this with occurus

[late stratus]

LMAO

sorry

[raven valley]

Yeee

Okay wait so I think my reloadable chunk works

Lemme test

[idle stone]

yeah the code generated by the macro already makes your system act like a run_system_once exclusive system

in order to make potential hot patching work

[raven valley]

Yep!

@celest magnet with my pr it works lol, not hot reload annotation

Just put em in here instead lol

And u can hot reload the write event and stuff

[late stratus]

nice!

[raven valley]

Hmmm im wondering if I should break this out into its own crate given its basically a different approach 🤔

[late stratus]

maybe?

[raven valley]

bevy_simpler_subsecond

Heheheh

[late stratus]

inb4 tomorrow i have to make bevy_evensimpler_subsecond

[idle stone]

it could be added to the same crate, no?

[raven valley]

Yeah!

I mean ig might as well collate all the strategies together in the same crate

No reason not to

[idle stone]

yeah so i can experiment with all of them easily :)

[raven valley]

:)

Wait so I am confused @grave bane are you trying to persist system state between reloads or just have it work during runs?

[late stratus]

between reruns which don't change the function signature

[raven valley]

Okie

[late stratus]

ALSO

wrong ping

[raven valley]

OWO?

Oh shit

[late stratus]

lmao

[raven valley]

My pseudo-dislexia strikes again

[late stratus]

i can.. see

-# *dyslexia

[raven valley]

Wahhhhh

Also my method means systems aren't exclusive

The only downside of my method is you can't schedule systems inside the hot reload relative to systems outside the hot reload ( because its actually a secret different schedule )

@late stratus we might be able to combine our approaches depending on what you do here 🤔

[elder heart]

pub trait AppExt {
    fn add_hotpatched_systems<S, M1, F, M2>(
        &mut self,
        schedule: impl ScheduleLabel,
        systems: S,
    ) -> &mut Self
    where
        S: IntoSystem<(), (), M1, System = FunctionSystem<M2, F>>,
        F: SystemParamFunction<M2>;
}

slowly but surely...

[raven valley]

Actually I might be able to simplify my crate even more? Okay the question is can I remove a system from a schedule

Does anyone know?

[late stratus]

https://docs.rs/bevy/latest/bevy/prelude/struct.World.html#method.unregister_system

maybe

[raven valley]

Hmmm

Its not quite what I wanted because u could have systems on multiple schedules

But good enough let's see what I can cook

I might be able to make reloadable systems act exactly identical and work every way like normal systems

Except they are relodable

And then whatever system state preservation thing u guys do when the signature doesnt change, I can also add

Or wait

I might be able to just do it?

OwO?

Hmmm

Lets see

[mellow flint]

Yesn't.
I just made the default macro expansion use the original system signature directly when registering the newly generated system

That way, every signature that Bevy accepts should work exactly the same for #[hot]

The "change your signature at runtime" feature is then opt-it by using #[hot(hot_patch_signature = true)]

Which comes with the previous caveats

[mellow flint]

Same crate plz 😄

[late stratus]

gaahhh why is everything i need private!

bevy is teasing me with the ability to implement the System trait easily but making me duplicate half of bevy_ecs!

[median vale]

You generally don’t want to implement system

[late stratus]

but i do

[median vale]

Implement IntoSystem

If you can

[late stratus]

i want to keep param_state but replace func

that was the whole idea

[median vale]

If you are implementing system, that probably deserves to live in the ECS crate

[late stratus]

that does not let me: keep param_state alive or alternatively overwrite run_unsafe

[median vale]

I don’t think it’s really built for public use

[late stratus]

issue: i'm doing this specifically for subsecond

[elder heart]

for a second I thought this was like the answer, but you'd still have to have some type that implement System...I guess recursively this could be a function?

but not FunctionSystem. only way to get this is to call .into_system on an inner type. My type in question can't do this properly

it would be some F that implements IntoSystem that ends up being some system I'm guessing

saying because I'm looking at the options...https://docs.rs/bevy_ecs/0.16.0/bevy_ecs/system/trait.System.html#implementors

[median vale]

still maybe expose something in the ecs that is designed to be consumed by your crate?

[elder heart]

ig just calling into_system on F will do?

[late stratus]

i am writing a wishlist for things which i would like to be pub

but honestly, if i'm allowing myself to edit bevy_ecs, i'm just gonna use a far simpler and nicer approach of making the ability to replace the underlying func an inherent part of the the System trait

or maybe just a method on App/World/Schedule or sth

[median vale]

that would probably be a better way to go about things.

modify the ecs and take the nice path

no reason to hack around it, it's not like it's closed source

[late stratus]

yes but i had a hunch this may be possible without that, being usable as a normal plugin

still might, i just might need a &mut World to create such a system. no nice extension trait to App for add_hotpatch_systems()

[elder heart]

is what I'm saying super wrong? my strat is to have a wrapper that will, on the call to .run_unsafe. The struct wraps some function that calls into dioxus's hotpatching function, and in turn, calls the inner types' .run_unsafe with the world args

I feel like a copy of FunctionSystem is kind of necessary. edit: I think at this point, I've failed. Gonna let the pros handle this, but yknow, it was fun and maybe productive? Just wish I'd figured out how to embed hotpatching into the fabric of SubApp

[languid sable]

This is fine for 0.16.1

[raven valley]

Hmmm. Schedules are not manipulatable enough

For what I want

Its totally doable I just cant do it with how scheudles work rn

Sadge

[mellow flint]

ah heck

BTW here's a demo from @amber grotto's component migration PR. It works! Only issue left is that the health is reset at the end 🙂

[vivid ember]

[raven valley]

Okay actually, so the more I think about it the more I think that what might end up being ideal is the following: we have a hot macro you can apply to systems, this allows you to change the insides but not their signature, and this works for any system anywhere. It's quick, and easy development debugging and modification.

Then, I think that local state issue is too much of a handicap so that's as far as the macro goes.

Then for everything else we have fully manipulatable hot reloadable systems you can write and edit at runtime non exclusively using my hotpatch closure.

We have a second macro called like "startup_respawn". This is for the StartupRerun schedule ( its needed to do the auto despawn ).

This is my vision rn

[mellow flint]

I don't understand what startup_respawn is, but other than that, I agree

Also note that the #[hot] macro you're describing is what systems will be doing by default when enabling the correct bevy feature on Francois PR

So no macro needed for those 🙂

[raven valley]

Ooo yeah

Hmm I hope what francois does doesnt somehow conflict with what I do

I would guess not but I honestly dont know 😂

[mellow flint]

Pretty sure it does not 🙂 @lone token

[raven valley]

But yes that would be an even better future!

Does francois systems allow u to modify signatures without losing local state every run ?

Like the event reader problem

If not its fine but if so then it really makes things much much better, then u just write inside the closure when u are actively developing ( and u can leave it there if u want )

Otherwise there is extra reason to leave stuff in the closure ( the ability to modify signatures)

[raven valley]

So startup respawn basically uses line macros + counting newlines to figure out the span of the function

So it auto despawns entities

I got it working here

With hot

[raven valley]

@mellow flint

[raven valley]

I wanna be able to remove systems from schedules based on system type id

Then I can remove all the new systems added during the last hot update, and then re-add them for this hot update

So that way you can still schedule relative to other things in normal Update

Systems as entities would help this lol

I dont think the changes I want will get added until systems as entities

[vivid ember]

That should be doable if we swap to slotmaps to be able to remove systems, and add a way to go from TypeId -> SystemTypeSet -> query the hierarchy graph to find the corresponding system NodeId to remove specific ones

[raven valley]

[vivid ember]

hmm actually I don't think we can do TypeId -> SystemTypeSet, unless we swap SystemTypeSet to storing a TypeId rather than be generic

which is probably doable

I will take the nerdsnipe and do a minimal PR to swap to using slotmaps this weekend

unless someone else wants to

[late stratus]

hmm

do you think re-computing a existing systm as if it was just added is reasonable?

in the schedule

[languid sable]

This sort of thing would be lovely. I really don't think we're more than a handful of PRs away from being able to dynamically add and remove systems

[vivid ember]

yea slotmaps + some graph querying functions should be all we need

[raven valley]

[vivid ember]

just uhhh, don't know it too often because big schedules are expensive to rebuild afaik

[mellow flint]

Gonna merge https://github.com/TheBevyFlock/bevy_simple_subsecond_system/pull/16 when CI passes and do a release so that people using Local or EventReader are unblocked 🙂

[raven valley]

Yee

Only on hot reloada

Reloads*

[mellow flint]

There we go, published 0.1.10
Pinging @celest magnet and @idle stone so you two can update 🙂

@idle stone is this enough to allow to use #[hot] on all functions?

[celest magnet]

This makes writing UI and particle effects finally fun in bevy… (amongst so many other things of course)

[mellow flint]

Yeah, completely agreed 😄

[idle stone]

i still have a few edge cases:

• function that returns a system
• exclusive system with a Local
• system returning Progress

[mellow flint]

I'm gonna be AFK for 45 minutes or so, then I'll remove the ugly __HotPatchedSystem::system_ptr_update_id. Pretty sure none of you depend on that

[mellow flint]

Weird, I added compilations tests for those

[idle stone]

oh well i haven't checked if they work now

[raven valley]

Lmeow

[idle stone]

i assumed the latest release only added support for saving system state so that Local etc. work?

[mellow flint]

😄

[mellow flint]

Nope, I just change the default strategy on #[hot] to be minimal

[idle stone]

ah ok, i'll try it out

[mellow flint]

No changing signatures, no reruns unless you opt in

That allows us to pass the system nearly unaltered to Bevy

I think the only thing that wouldn't work is destructuring stuff in the params, i.e.

fn foo(In<age>: In<f32>) { }

But who would do that????
Just kidding, Yarn Spinner heavily relies on that 😄

[raven valley]

As do i!

[mellow flint]

But Yarn Spinner needs its own integration layer anyways

[mellow flint]

orly?

[raven valley]

I love destructuring

[idle stone]

is there a technical limitation there, or just more macro fu needed

[mellow flint]

macro fu

[raven valley]

Leave it for now tho it is rare enough

Unless ur really itching

[mellow flint]

just need to make sure that the destructuring is only happening on the inner function and not the outer one generated by us

I have an idea of how to do that, but I'm too lazy right now 😄

(we could just give out variable names automatically for the outer function: a1 for the first param, a2 for the second, etc.)

(dunno if rust-analyzer will like that)

or error messages for that matter

[idle stone]

i can confirm my systems returning Progress piped via .track_progress() are now hot-patchable

[raven valley]

Hopefully this new pr unfucks rustrover

It was unhappy when I pulled last

And when I tried to manuallly expand the macros they got upset and said they were unwrapping a none, lol

[idle stone]

can confirm my Locals are working

can confirm system with simultaneous &mut World and Local params is hot-patchable

[raven valley]

Lessgoo

[elder heart]

or is this because these are system inputs

can't wait to see hahaha

[idle stone]

function returning a system is not exactly hot-patchable currently?

it compiles, and it says "hot-patching..." when i change it, but the system it already returned to app.add_systems doesn't change

is it possible to like.. annotate the inner function?

[late stratus]

damn the new PR arrived at basically the same conclusion i did

well

new update now, not a PR anymore after its merged is it

SystemState::new() is the ticket to it working

[elder heart]

what pr :o

[late stratus]

uhh

not a PR anymore, as i said

the 0.1.10 version

[elder heart]

ok

[late stratus]

Q: can rust-analyzer expand non-recursively? i just want to expand 1 layer of macro

[raven valley]

Rust rover can if u have that!

[late stratus]

nope

mostly i'm getting annoyed at all the tracing macros

i might just remove those

[mellow flint]

Same

[late stratus]

oh its just 2 lines

commented

[mellow flint]

Remove them for debugging and its much cleaner 🙂

[late stratus]

wait

i did and its still there :(

probably rust-analyzer didnt refresh the macro

[mellow flint]

Yep, totally

Try cargo expand

[late stratus]

i'm now looking into whether this can be done macro-less

[elder heart]

no fking way SystemState has a constructor to build a FunctionSystem. like if someone had said that that would've been gold.

[late stratus]

i feel it can be

[elder heart]

this can be done macroless

[late stratus]

generics hell, but it can be

[late stratus]

yup, what i just said thats what i just realised as well

huh? i apologize, i didn't mean to imply anything by that beyond "we just arrived at the same conclusion" @elder heart

[late stratus]

found a way: rust-analyzer has "rebuild proc macros and scripts"

expanding after that accepts the modifications

[late stratus]

I GOT IT AHAHAHAHAHAH

[raven valley]

[late stratus]

need to test stuff first but maaaayyyybee a PR incoming

i may or may not have gotten rid of all the events and HotPatchedSystems handling

also did not implement/keep rerun-on-hotpatch functionality

[raven valley]

OwO?

OWO?

[late stratus]

false alarm: i can only change strings

false alarm again

it only works properly when i'm not changing the signature

sadge

yeah i think i'm arriving at a inferior solution to https://github.com/TheBevyFlock/bevy_simple_subsecond_system/pull/12

[raven valley]

OwO

Speaking of!

.... drumroll

🥁

[mellow flint]

-# (it's compiling tets)

merged!

[raven valley]

WAHOOO

[mellow flint]

I'll quickly yeet system_ptr_update_id and then do a new release

[raven valley]

You should put together a demo video showing off all the features now that there are so many, and post that in the crate update!

( you don't have to, just an idea 😄 )

[late stratus]

in case anyone is curious, heres what i did. it doesn't work with signature updates, and probably has some other issues, so i'm gonna just drop this here and be done with it. not worth making a branch IMO, i didn't actually manage any of the "custom System impl" idea i had because turns out thats not neccessary and hindered by tons of private types

[mellow flint]

Thanks for trying, though 😄

I see what you were trying to do

[mellow flint]

First need to feed my rats, I think they're getting hungry 😛

[late stratus]

yea

[raven valley]

Haha

[elder heart]

I guess my main source of confusion, at this point, is understanding the persistence of SystemState...and what happens when someone adds a Local to their params in some hot patchable function. iiuc, the data within systemstate is 1:1 with the systemparams of the fn

so how to get that to change on hot reloading...a dyn system param, perhaps? but even that wouldn't work.

if someone's already tackled the problem of local data queries, then hell yeah, ill set it all aside

but if not

[raven valley]

I have!

[elder heart]

I think this is something that will need figuring

you have? :o

please put my rabid brain to rest

[raven valley]

Yeah local data works fine in the reloadable section

[elder heart]

was that in ur pr? I'm gonna look for it

[raven valley]

Look at the repo at the add_systems example

[elder heart]

definitely want to see the implementation. that part's killing me.

[raven valley]

Its not what you would expect!

[elder heart]

has me pacing

[raven valley]

Haha

The implementation is in the hotpatch_app file

[mellow flint]

Well we have a PR with a nearly-complete struct migration per reflection

I think we could do something similar for Local

@amber grotto

[raven valley]

Ah more in terms of, local works at all, we dont migrate from one local to another

When u modify a system it still resets the locals

[elder heart]

wait, so essentially a sidestep of SystemState...something that recreates SystemState on every frame before running?

[raven valley]

Not recreated every frame! Only when a hot reload event comes through!

If i understand what ur asking

[elder heart]

You do, that makes sense

simple. genius. so the whole closure is reloaded?

[mellow flint]

Done, version 0.1.11 contains @raven valley's new feature 😄

In the meantime, I'm being scammed by anime girls

Ah yes, the future is now

[vivid ember]

Hey, you just did a great job in Zombies, team up?

[mellow flint]

oh.sorry, i seem to have recognized the wrong person

sorry to bother u.

[vivid ember]

unfortunately I am not an anime girl

[mellow flint]

ty
but it's nice to meet you, I'm 25 years old, you?

Hey, can I invite you to play TWD with me?

[vivid ember]

swapping to slotmaps cleans up so much stuff

[mellow flint]

How should I be fooled by this, they don't even use kawaii emojis

[elder heart]

Hey, can I invite you to play TWD with me?

[mellow flint]

oh my god you even changed your profile pic hahahahahahha

[raven valley]

AHAHAHHAHHA

[mellow flint]

@raven valley when you have time, could you make with_hot_patch desugar to something slim on release and Wasm? 🙂

[raven valley]

Ooo yep

Sure

[mellow flint]

Also, something that can rerun OnEnter would be nice

I think systems added in the closure are allowed to change their signature, right?

If so, I think your feature could be used instead of the #[hot] macro

I think keeping the base macro would still be good for usability

[raven valley]

Hard agree

[mellow flint]

But we could consider removing the optional "extensions"

[vivid ember]

it appears that running the ci test suite locally crashes my desktop env, so we publishing the pr without testing 😎

[raven valley]

[idle stone]

probably needs to rerun OnExit first

[raven valley]

Ngl I don't actually know how states work in bevy at all

I've been meaning to learn

[idle stone]

StateTransition schedule has systems with appropriate system ordering that call the OnExit, OnTransition, and OnEnter schedules if there was a transition

runs once per frame*

pretty simple conceptually

[raven valley]

Hmmm

Ill have to play with it

Maybe I'll do that this jam

[mellow flint]

You should take a look at bevy_new_2d then, set it up with bevy new cool_project -t 2d. I also didn't know much about how to use states until Pyri and Mini implemented those parts there 🙂

Now, I have nearly nothing set up in Startup at all; most setup lives in some OnEnter and is StateScoped

[elder heart]

just gotta shout out pyrious' awesome work. pyri state is good once it clicks, but it needs to click. it will be my cup of tea sooner than later

[raven valley]

Oooo

[warm thistle]

this was also a blocker for being able to use systems for the render graph

[amber grotto]

I think I found a - if you have a component and one of its fields is another struct, and then you update that inner struct, the component still has the same type id

#[derive(Component)]
struct C(Inner); // <- doesnt change type id

struct Inner {} // <- modify this one

@mellow flint so that example you sent is not really an issue with migration

When you modify that enum it seems to work because it just updates the string in the Debug impl and both versions of the enum have the same underlying representation

[amber grotto]

This is a problem in general because type id doesnt change -> component id doesnt change -> systems continue operating on stale data with (potentially) different layout

[worldly ether]

debugging TLS issues pushed me to use lld instead of ld and now subsecond properly respects your linker setup - leading to a consistent 130-140ms patch time

[quartz whale]

Sick! What was the patch time with ld?

[slender relic]

@mellow flint #[hot] makes lsp-based code highlight red(like it has error) and autocompletion broke :Inspect also confirms that rust-analyzer thinks the code has "type.unresolvedReference"

it latest from crates.io also from git (editor is neovim)

[worldly ether]

1.3 seconds

[slender relic]

just changed rust-analyzer settings to ignore it and now it normal

    lspconfig.rust_analyzer.setup({
      capabilities = capabilities,
      settings = {
        ["rust-analyzer"] = {
          procMacro = {
            ignored = {
              bevy_simple_subsecond_system_macros = { "hot" },
            },
          },
          diagnostics = {
            disabled = { "proc-macro-disabled" },
          },
        },
      },
    })

[worldly ether]

if you want to provide proper autocomplete you need to do partial expansion.

https://github.com/TheBevyFlock/bevy_simple_subsecond_system/blob/main/macros/src/lib.rs#L56

to properly get autocomplete when doing function parsing in macros you need to fork the stream, output the original token stream regardless if it's actually a function, and then do the code-gen using the forked stream

it's a pain, but we do it here for rsx

https://github.com/DioxusLabs/dioxus/blob/5db8b95c54d1c481d4e4c0525260cc2c827dbc98/packages/rsx/src/raw_expr.rs#L16-L58

https://github.com/DioxusLabs/dioxus/blob/5db8b95c54d1c481d4e4c0525260cc2c827dbc98/packages/rsx/src/partial_closure.rs

[quartz whale]

Holy, that's an insane boost in speed 😮

[quartz whale]

Mhm, my suggestion before was to output the original when parsing fails, but always emiting the original is pretty smart

[raven valley]

I have an Idea

[amber grotto]

You could recursively get a hash of the whole type hierarchy in the derive macro to detect changes

But the problem is that you need to update TypeId -> ComponentID map in bevy which isn't public

[raven valley]

U can do a custom CustomId -> ComponentId map

Make the custom id be a compile time macro based which bases it off of the system time

Plus type id hash

[mellow flint]

Thanks for the link!

[mellow flint]

Hehe, try mold and wild next 😉
Did you have time to look into my issue on main?
I would love to help you test the new fixes, but the CLIs main branch was broken for me

[worldly ether]

My branch fixes plain Linux, tested at least on aarch64 though. Apparently android and plain Linux are different and fixing one breaks the other

[mellow flint]

Oof that’s unfortunate.
I'll try it in a few minutes then 🙂 which branch?

For context, that was my issue on main ^

[quartz whale]

I haven't heard of wild before, but apparently they're trying to add incremental linking, which would be kinda crazy for the hotpatching usecase. It's not yet implemented though, so who knows whether it'll actually save a significant amount of time or not

[amber grotto]

Yeah but then you need to trick systems into using ComponentIDs not matching the TypeIDs in system params. Which might be possible to do with some clever intercepting of system params...

[raven valley]

Just remap right after reload!

All u have to do is make sure u migrate all the types

So everything stays valid

[amber grotto]

Not sure I understand, the systems will still be looking for the old component id/type id (which will be actually the only type id)

[quartz whale]

Has anyone gotten dx on NixOS yet? cargo binstall falls back to building from source (and actually I'm not sure it would even work if it had pulled down the binary), and I can't manage to build dx from source due to not being able to find openssl via pkg-config. I've looked at the flake.nix in the repo, and the nix file in nixpkgs, but regardless of what I add on my system it doesn't work. Strangely the actual error claims to be unable to find the pkg-config executable, but that's nonsense, because Bevy already requires pkg-config, and Bevy builds just fine

[raven valley]

😭 nix

[celest magnet]

small suggested for the example (for setup systems), ideally it would show an ergonomic way to also deal with resources (when using world)

something akin to this.

(its basically the bevy sample, but hot-reloaded)

[mellow flint]

Yes, @glacial lagoon and @slender relic

[mellow flint]

@raven valley does it work with Resources yet?

[late stratus]

feedback from @karmic current in lifestream: install instructions, especially one command ones, shouldn't be hidden behind a details collapse

[raven valley]

Yes just we need to wait for 0.16.1 to not have to use world to spawn entities

[mellow flint]

Fair

[quartz whale]

Hm, I just pulled down the source and built it with the flake they provide, which obviously works. I have no idea why my own one didn't work, but whatever 😂

[late stratus]

feedback 2: just show a full working example for the greet system and setup, don't split it into multi codeblocks and hide things like the add_systems

[mellow flint]

Also fair

@karmic current I updated the readme accordingly, refresh the page 😉

[amber grotto]

Every couple of hours dx start throwing the windows paths are too long error, so I'm progressively moving the project folder into shorter and shorter paths XD

[mellow flint]

Yeah @elder heart ran into the same thing

[elder heart]

yeap

throw that bad boy in root, right next to Program Files

it's annoying

but like, it's worth if you're getting good stuff out of hot patching

[mellow flint]

@worldly ether I think your PR doesn't address this yet, right?

[elder heart]

wait...did it work?

I must've forgotten to check, I'm gonna see if this fixes the paths problem

[mellow flint]

@amber grotto I'm fine just leaving that quirk in for now 🙂
https://github.com/TheBevyFlock/bevy_simple_subsecond_system/pull/11#issuecomment-2906891146

[elder heart]

oh yeah, cuz last I remember it wouldn't compile on windows...checking now to see if the paths problem is fixed on windows on the jk/fix-dyn-link-subsecond branch

[mellow flint]

@amber grotto if you fix the merge conflicts, we can merge it 🎉

Just add your observations to the readme

[worldly ether]

I’m not sure what’s going on here

[mellow flint]

some other people ran into path length issues with subsecond on Windows as well, so I assume its a consequence of that

[elder heart]

yeah, rip. per the original issue

https://github.com/DioxusLabs/dioxus/issues/4159

funny enough, can't find CARGO_TARGET_DIR in this env

I'll see if I can squash this for the issue, but yeah..no dice

[quartz whale]

Has anyone run into issues like these? They only show up when switching to Mold (though of course the default linker fails for other reasons):

mold: error: undefined symbol: bevy_asset_loader::loading_state::dynamic_asset_systems::resume_to_loading_asset_collections::__CALLSITE::h4d3a7e634fee5fe9
>>> referenced by 17uoh8fy54ggkrbt22dg9ndib
>>>               /home/ole/workspace/rust/sinhala_training/target/x86_64-unknown-linux-gnu/debug/deps/sinhala_training-26c1043305835bbd.17uoh8fy54ggkrbt22dg9ndib.rcgu.o:(bevy_asset_loader::loading_state::dynamic_asset_systems::resume_to_loading_asset_collections::h8b2737a1acf2d4c6)
>>> referenced by 17uoh8fy54ggkrbt22dg9ndib
>>>               /home/ole/workspace/rust/sinhala_training/target/x86_64-unknown-linux-gnu/debug/deps/sinhala_training-26c1043305835bbd.17uoh8fy54ggkrbt22dg9ndib.rcgu.o:(bevy_asset_loader::loading_state::dynamic_asset_systems::resume_to_loading_asset_collections::h8b2737a1acf2d4c6)
mold: error: undefined symbol: bevy_asset_loader::loading_state::systems::resume_to_finalize::__CALLSITE::h24db4c0b788caf68
>>> referenced by dnnu86e1dngosybtwoodipj59
>>>               /home/ole/workspace/rust/sinhala_training/target/x86_64-unknown-linux-gnu/debug/deps/sinhala_training-26c1043305835bbd.dnnu86e1dngosybtwoodipj59.rcgu.o:(bevy_asset_loader::loading_state::systems::resume_to_finalize::hb58d54b7bacae0e0)
>>> referenced by dnnu86e1dngosybtwoodipj59
>>>               /home/ole/workspace/rust/sinhala_training/target/x86_64-unknown-linux-gnu/debug/deps/sinhala_training-26c1043305835bbd.dnnu86e1dngosybtwoodipj59.rcgu.o:(bevy_asset_loader::loading_state::systems::resume_to_finalize::hb58d54b7bacae0e0)
collect2: error: ld returned 1 exit status

[celest magnet]

It was more to get around the mutability restrictions of resources in the enclosure.

Not sure if the resources would be properly reset 🤔

[quartz whale]

(It's not all bevy_asset_loader btw, it seems to be thousands of symbols missing)

[mellow flint]

Could you share your config.toml? I'm using the following, but it's still giving me Build failed: I/O Error: No such file or directory (os error 2)

[target.x86_64-unknown-linux-gnu]
linker = "gcc"
rustflags = [
  "-C",
  "link-arg=-fuse-ld=lld",
  # Nightly
  "-Zshare-generics=y",
  "-Zthreads=0",
]
rustdocflags = [
  "-C",
  "link-arg=-fuse-ld=lld",
  # Nightly
  "-Zshare-generics=y",
  "-Zthreads=0",
]

I believe it's still the same issue as on main

[mellow flint]

Ping @raven valley

[raven valley]

Confused

What is the question

Oh you mean adding resources?

Doesn't handle it yet

I have some ideas im gonna do this weekend

I wanna only add when it doesnt already exist

[mellow flint]

New Chris feedback incoming?

[late stratus]

thing i noticed: i think my non-working version from yesterday had it roll back to the previous function on panic yesterday, but for chris it just fully panicked. now, it was like 3am at that point, i might be misremembering, but i wonder if the way the macro works breaks that unwind to last working function

[mellow flint]

Hmm, what do you mean?

What exactly is panicking in this scenario?

e.g. if you request a Resource that does not exist, that will panic, nothing we can do about it

[late stratus]

tbh i'm less and less sure about what i observed, but i definitely got previous output at some point yesterday

[late stratus]

right but a unwrap or panic on the system

[mellow flint]

Oh yeah that is supposed to panic as usual

[mellow flint]

Some ideas:

[raven valley]

Can't wait till he sees the beauty of .with_hotpatch 🤞

[mellow flint]

• hot patching can break down if you change signatures when using #[hot]
• if you revert your changes to the original code when the app started, subsecond does not pick up on that

[late stratus]

ah but i was using a extension trait and .add_hotpatch_sysyem() yesterday

no macro

could be a syntax error causes it to roll back instead?

[mellow flint]

That would be the same, #[hot] now does the bare minimum boilerplate

[raven valley]

Yes

Syntax error prevents hotpatch

[mellow flint]

AFAIK there is no mechanism for rolling back. Syntax errors will just not hot patch at all

[late stratus]

that's what i'd think, but i distinctly remember it going back to a older version for some reason. i probably made a mistake

[raven valley]

Someone should add the .with_hotpatch to the readme otherwise people are gonna miss it

If no one does it I'll do it when I get home, space is too cramped on this flight to use my laptop

[quartz whale]

I can't quite get it working unfortunately 😭 If any of the other NixOS people have a link to their configs I'd love to see it 😌

[mellow flint]

I assume you checked the part of the readme about mold?

[quartz whale]

Yeah, and it's working with cargo build, but not via dx for some reason, I tried copying the stuff you have in your .cargo/config and stuff as well (and the nixos mold install), but while it's definitely using mold it just gives weird errors. Now that I think about it I still have dynamic enabled, maybe that's the issue actually, oopsie

No it still doesn't work 😂

[mellow flint]

yeah dynamic definitely does not work

[mellow flint]

Can you run the examples on the repo?

[quartz whale]

I can't serve even the default thing generated by cargo new 😅

Also @worldly ether it seems like the UI crashes if you don't have any dependencies (reproduce with dx serve right after a cargo new) it looks like it's due to the progress bar being too small for some maths to make sense

The closest I can get is this error Failed to generate fat binary: mold: fatal: /home/ole/workspace/rust/hotpatching_test/target/x86_64-unknown-linux-gnu/debug/libfatdependencies-8dbb7bc8-bda9-5670-a51e-7eb48041797c.a: unknown file type

That file is just an empty file for the record

[mellow flint]

@amber grotto I merged your PR 🎉

Are these changes also okay for you? https://github.com/TheBevyFlock/bevy_simple_subsecond_system/pull/18

If so, I'll do a release after

[quartz whale]

Ah the file was just empty because I forgot to add stuff, when I actually use things I get the missing symbols again

[amber grotto]

Yeah, looks good, thanks!

[worldly ether]

I might not be able to get -clinker= override working, but for people who use -fuse-ld= I do have it working

getting RUSTFLAGS="-clinker=... might not be possible without dx overriding the rustflags

[mellow flint]

let me know when to test 🙂

[mellow flint]

@worldly ether I see you merged your PR, so I checked out main. Unfortunately, it is broken for me on Linux

[worldly ether]

what's the issue? works for me on aarch64 linux with and without -fuse-ld

[mellow flint]

Let just really quickly check that the problem is not in my code before I go into details

[worldly ether]

you might need to update both the CLI and dependency

that changes the symbol we use for offset detection so now the implicit API is no longer compatible with alpha.0

the new api relies on main being present so that shouldn't be an issue again

[mellow flint]

Oh, I did not update the dep

Let me do that quick 🙂

[mellow flint]

can confirm it works without fuse-ld!
I think this means https://github.com/DioxusLabs/dioxus/issues/4144 is also resolved 🙂

Now let me try changing my linker

[idle stone]

same

[mellow flint]

try patching the dependency on deoxus-devtools

"link-arg=-fuse-ld=lld" also works!

Now let's try mold

@worldly ether you mentioned that linker is not respected. I suppose I need to set DX_LINKER?

[worldly ether]

the issue is that while we respect -clinker when passed during rustc we don't respect it when passed via RUSTFLAGS

that requires rewriting the rustflags

[mellow flint]

hmm, so how would I set gcc to my linker?

[target.x86_64-unknown-linux-gnu]
linker = "gcc"
rustflags = [
  # Use mold
  "-C",
  "link-arg=-fuse-ld=mold",
]

^ this is what I need

[worldly ether]

-fuse-ld should work, and modern compilers can typically respect -ld-path

in this case gcc is a driver. we use the cc environment variable (rust also relies on cc in env
cc=gcc dx serve --hotpatch

[mellow flint]

00:15:35 [dev] Build failed: I/O Error: No such file or directory (os error 2)

[worldly ether]

i tested with clang - does cc=clang work?

[mellow flint]

Oh wait, do I need to remove the linker param from my config?

then it does stuff

[mellow flint]

^ talking about this one

[worldly ether]

Oh hmmm, I've been setting linkers with -clinker but we technically do handle the linker field, but I think that has similar issues to the RUSTFLAGS issue

[mellow flint]

Hmm, when removing the linker key, it runs.
But then on the first hotpatch:

[worldly ether]

the issue is that unless we rewrite the flags, we don't even see that a different linker was requested

[mellow flint]

Note that it says /usr/bin/ld for some reason?

[worldly ether]

this is likely mixing linkers on accident

[mellow flint]

Anything I should try?

wait, why am I even using gcc 😄

[worldly ether]

gcc isn't a linker technically though it can be used as one. even in normal operating mode if all you want is to use mold you don't need gcc as a flag

[mellow flint]

Mold expects clang

sorry, my bad maybe

let me to clang

[worldly ether]

in practice I think you symlink clang to /usr/bin/cc or you place clang in your path symlinked as cc. But I think cc=clang worked for me with the mold setup. FWIW mold and lld were the same performance

[idle stone]

doesn't seem to help

tried this with and without rev: dioxus-devtools = { git = "https://github.com/DioxusLabs/dioxus.git", rev = "929cab9ecfdc991f478e91b09f9e8a7549421e1f" }

[mellow flint]

Could you show me your config.toml?

[mellow flint]

I tried branch="main"

[idle stone]

well that made cargo update actually do something, but then dx serve still fails

15:21:05 [dev] Build failed: I/O Error: No such file or directory (os error 2)

[worldly ether]

Just

[target.aarch64-unknown-linux-gnu]
rustflags = [ "-C", "link-arg=-fuse-ld=mold" ]

I believe, and then symlinking clang to cc or setting cc=clang

[mellow flint]

FYI dx seems to crash every time on "file not found" when linker is set there

[idle stone]

this was my issue it seems

[mellow flint]

Still results in

00:23:05 [dev] Failed to generate patch: /usr/bin/ld: /home/hhh/git/bevy_simple_subsecond_system/target/dx/minimal/debug/linux/app/stub.o: .symtab local symbol at index 20 (>= sh_info of 1)
/usr/bin/ld: /home/hhh/git/bevy_simple_subsecond_system/target/dx/minimal/debug/linux/app/stub.o: error adding symbols: bad value
collect2: error: ld returned 1 exit status
00:23:05 [dev] Build failed: No such file or directory (os error 2)

I'll try symlinking now

[mellow flint]

same result when symlinking clang to cc

Now let me symlink mold to ld again

[worldly ether]

when I ln -s /usr/bin/clang /usr/bin/cc ... if I already had cc there then it doesn't do anything

if you're on x86 then there might be some weird interaction with path since rust-lld is stable

[mellow flint]

this works

[worldly ether]

My path is very clean, but if you have an interesting enough path then some tools might be taking precedence. Can check with which cc , etc

Is mold in your path?

[mellow flint]

amd64, so yep

[mellow flint]

yes

[worldly ether]

one quirk is that rust prepares args for cc using -Wl,-arg but when using linkers directly, it prepares them with -arg, and sometimes it's hard to detect that

[mellow flint]

cc was indeed still pointing to gcc

Let me try again 🙂

[worldly ether]

You generally want to use -fuse-ld and then muck with cc

well, at least for now until we get all the quirks shaken out

[mellow flint]

Interesting. All readmes of linkers always set the linker key in the config, so I expect lots of users having that set

Can you reproduce the crash when the linker is set?

[idle stone]

cc doesn't work with mold, no?

[mellow flint]

@worldly ether same error when I really make sure cc is clang.
It looks like it's still using ld for some reason:

The only solution seems to symlink mold to ld

[mellow flint]

🤷‍♂️

[worldly ether]

default cc doesn't have the -fuse-ld thing I believe

but modern clang/gcc do

[idle stone]

ah yeah: cc: error: unrecognized command-line option '-fuse-ld=/usr/bin/mold

[mellow flint]

(note that that also worked with cc pointing to gcc)

[idle stone]

so symlink mold to ld and symlink gcc to cc?

[mellow flint]

either clang or gcc

(I prefer clang)

[worldly ether]

all you need to do is override cc to a modern linker and then use -fuse-ld unless you're using a weirder linker

or set cc=clang via env

[elder heart]

I think this issue needs to be reopened fwiw https://github.com/DioxusLabs/dioxus/issues/4159#event-17801646683 just want to capture its attention 😄

[worldly ether]

for now, need to properly handle the linker= thing. that's the same issue as RUSTFLAGS

[mellow flint]

yeah that works for lld for me, can confirm

I'll now test the global dir thing 🙂

[idle stone]

oh, can you do cc=clang ld=mold dx serve ... instead of a global symlink? that would be much nicer

[mellow flint]

that would be MUCH nicer. does that work?

[elder heart]

or maybe this needs to be a new issue as CARGO_TARGET_DIR doesn't seem to be the culprit

[mellow flint]

Yesss the global dir + mold setup works now

[idle stone]

idk i'll test it. i was reacting to this msg

[worldly ether]

This works for me:

CARGO_TARGET_DIR="target/bargt" dx serve --hotpatch --verbose --platform desktop

[elder heart]

on windows? I'll set the env accordingly

with this value unset it fails at least...which is kinda normal behavior I guess?

[worldly ether]

dx does not disambiguate itself in the target folder, so packages with the same name will collide

[elder heart]

well... 18:33:55 [dev] Build failed: I/O Error: The filename or extension is too long. (os error 206) idk if that's the issue

[worldly ether]

We don't have an issue tracking the long paths thing but the recent merge to main shrunk the name of the archive file we were generating to 18 characters

[idle stone]

well i got hotpatching working now with bevy/dynamic_linking enabled, so i can confirm that's working

[elder heart]

agreed. I think it's something else looking at the full log output. I'll investigate this if I have time tomorrow. honestly the output doesn't give much about what specifically is too long, but I think I'll see if I can update this somehow

[idle stone]

i can also confirm that hotpatching is triggered by reverting a single edit and saving again

[mellow flint]

lovely!

[mellow flint]

@worldly ether is DX_LINKER or DX_HOST_LINKER supposed to be used for this?

[idle stone]

well this runs.. but with this and bevy/dynamic_linking enabled, my hotpatches still take 4.5s, so it might not be doing anything

[worldly ether]

we pick up the linker you use from linker= but the "missing files" error means the linker field took precedence over us and we need to force dx to be the linker manually

[mellow flint]

oh this is without linker=

[mellow flint]

talking about this here 😄

[worldly ether]

I added some more verbose timing logs - one for rust building and one for linking, you can enable with --verbose
https://github.com/DioxusLabs/dioxus/blob/b2bd1f48d434becfff9f0ea390ad8ef46996300a/packages/cli/src/build/request.rs#L928-L951
Curious if it's rust or the linking process

Oh, we don't have timings for thin linking

[mellow flint]

I'll open two bug reports later (crash on linker and required symlinks for mold)
I'm just really happy that dynamic linking works now, good job 😄

Let me also check wild real quick

[idle stone]

yea i don't see that log

[mellow flint]

Could you open a new issue for it?

[mellow flint]

Nope, no wild support yet

[elder heart]

In a sec..I might've found something else

but I need to make this issue concise

seems also that incremental will fail after first build on wandows

[idle stone]

did you see a speedup from dynamic linking enabled?

[mellow flint]

Need to test

I mean, in general, sure

For hotpatching, I don't know yet

[idle stone]

btw, wild linker lists cargo-binstall installation option first 😄 (https://github.com/davidlattimore/wild?tab=readme-ov-file#installation)

[mellow flint]

hehe

nextest too, I believe

@idle stone does cargo binstall foo --git bla work?

I remember there was an issue with the Bevy CLI alpha

[idle stone]

confirmed that you cannot: when passing -fuse-ld with cc=clang, i still got the error from cc not recognizing it

[idle stone]

yes. the issue was in relation to custom release artifact path specified in Cargo.toml while the version in Cargo.toml does not match

[elder heart]

ugh I'm out of time. Having a party in about 15 minutes

[idle stone]

fixed by hard-coding the correct version in the path instead of having cargo-binstall expand it automatically

[elder heart]

I found the windows command that's causing the problem though

for whoever's reference, if someone stumbles upon this conversation: packages/cli/src/build/request.rs line 1839

will work on tom

[mellow flint]

hmm binstall does not support --rev it seems

[idle stone]

DX_LINKER=clang DX_HOST_LINKER=clang cc=clang dx serve ... still errors if i have -fuse-ld in my RUSTFLAGS :')

[mellow flint]

I'd like to change the readme to point to what we know to work right now

[mellow flint]

meh

[idle stone]

yeah since there won't be a released binary for every given git rev

[mellow flint]

fair

well, main it is

[idle stone]

although it could use the rev when installing from source as a fallback hypothetically

funnily enough, in my system, cc is already a symlink to gcc

but i assume it runs in cc compatibillity mode or something if arg0 is cc

maybe clang does the same thing ^\_(``/)_/^

ok interesting

$ /usr/bin/cc --version
cc (GCC) 14.2.1 20250207
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ /usr/bin/gcc --version
gcc (GCC) 14.2.1 20250207
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ file /usr/bin/cc
/usr/bin/cc: symbolic link to gcc

and cc=alksjf cc --version still runs cc, so the shell gives priority to PATH ig

ok, my new setup on Linux is:

• linker = "clang" commented out
• "-Clink-arg=-fuse-ld=/usr/bin/mold" in rustflags no longer commented out
• cc symlinked to clang
• ld symlinked to mold (this was already the case)
• bevy/dynamic_linking no longer disabled
• dioxus-devtools patched to main

hotpatching works and takes ~3.5s, which is ~1s faster than before EDIT: and now it takes 4-8s again 😔

[elder heart]

it's somewhere here

I'm wondering if the error is a red herring, and it's actually the command that's too long, not a file

the character limit seems to be 32767 for commands

yep

that's it

[worldly ether]

We were creating command files but then linkers that aren’t link.exe don’t accept it 🤷‍♂️

I guess you could turn down your codegen unit number in the interim

[elder heart]

oo ok!

[mellow flint]

Can you send me your codegen settings, if that fixes it?

I want to include it in the readme

[quartz whale]

Wohooo, with main it works now! I don't know what caused, as it now works both with ld and mold, but at least it works 😄

[mellow flint]

that's great 😄

[quartz whale]

I'm getting hot-patching times of very consistent 550ms, which is not quite as good as some, but that's solidly in the "feels instant"

[idle stone]

still 4-8s for me

[mellow flint]

@idle stone I get /home/hhh/git/bevy_simple_subsecond_system/target/dx/minimal/debug/linux/app/minimal-d6dab9ba: error while loading shared libraries: libbevy_dylib-6209f69d2edbbded.so: cannot open shared object file: No such file or directory

[idle stone]

i don't 💃

clean compile?

[mellow flint]

done

[idle stone]

patched dioxus-devtools to main?

[mellow flint]

yep

[idle stone]

ive definitely seen that error before in some context

what's your rust toolchain version?

[mellow flint]

hmm questioning whether cargo binstall installed the right binary

[idle stone]

i'm using nightly-2025-05-14

also i didn't use binstall for dioxus-cli

[mellow flint]

(because I has hopping between versions a few minutes ago)

[mellow flint]

well that might be it

[idle stone]

yeah that'll take from GitHub releases page

which is the latest actual release

[mellow flint]

oh right

even when doing --branch main, right

[idle stone]

have to install from source unfortunately 😄

[mellow flint]

Alright, I'll change the readme

[quartz whale]

I built it from source directly, and I had issues with mismatched versions between dx and the dependency (it ran, but didn't hotpatch)

[mellow flint]

that's better anyways, since I want to point to a specific rev

[mellow flint]

will update the crate shortly 🙂

[elder heart]

I don’t think anything can fix this except less dependencies and being closer to the root of your hard drive on wandows

OR

wait

[mellow flint]

[profile.dev]
codegen-units = 1

?

[elder heart]

This is nonetheless a problem for anything pulling in a lot of crates let me demonstrate the next time I get to my computer

[mellow flint]

do you think I shouldn't bother adding it to the readme?

[elder heart]

I’m cohosting this party, a few drinks had so not exactly in a sober mindset to figure this out atm, sorry :(

[mellow flint]

haha enjoy 😄

[elder heart]

I think it would be good to have

Something definitely

[mellow flint]

Could I get a second pair of eyes on this? https://github.com/TheBevyFlock/bevy_simple_subsecond_system/blob/1c0be1d23226e6b6567f228e1d09a724d74693ae/readme.md

@idle stone for example 🙂

(ignore the missing video)

[idle stone]

seeing -> see

a -> an

[mellow flint]

https://github.com/TheBevyFlock/bevy_simple_subsecond_system/pull/19

[idle stone]

perf

[mellow flint]

review plz 😄

I just linked the rendered version for convenience

[idle stone]

ah those two typos are pre-pr

[mellow flint]

I'll fix them while on it

@worldly ether here are the two reports I promised:

• https://github.com/DioxusLabs/dioxus/issues/4193
• https://github.com/DioxusLabs/dioxus/issues/4194

[idle stone]

left a couple comments

[mellow flint]

thx 🙂

[mellow flint]

oh nice!

[idle stone]

i haven't actually tested that but it had better work lol

i'll test it rq

seems fine

[mellow flint]

this is not quite true

[idle stone]

is it not? i'm basing it on the cargo book

or am i missing something else

[mellow flint]

incremental is a separate setting

Also, see https://doc.rust-lang.org/cargo/reference/profiles.html#codegen-units

The incremental links to the incremental setting in config.toml

(or -C incremental)

[idle stone]

hmm ok

dev build sets incremental=true and release sets incremental=false, so i guess the codegen-units defaults for the respective profiles come from there

from this section: https://doc.rust-lang.org/cargo/reference/profiles.html#default-profiles

[mellow flint]

yeah that makes sense

hehehehe fair

Alright, I've applied all your other suggestions

Thanks again 🙂

[elder heart]

I lied

it's good

however may need to fix the recompilation issue next

rerunning with dx serve --hot-patch will fail

running after cargo clean will succeed

will tackle this later

and will do diagnostic on char count for codegen-units

[mellow flint]

Sure, but enjoy your party first 😄

Tell them I said hi lol

[idle stone]

i made an issue for this :)

you just have to ctrl+C and run dx serve --hot-patch again, no need to cargo clean

[mellow flint]

thx!

[elder heart]

Oh, just did that. Failed

You mean run again, control c and then run again?

[idle stone]

like restart the cli itself

[mellow flint]

Yeah @idle stone I think the issue is that the already built artifacts are too long

So I think for this specific case, cargo clean is necessary

[idle stone]

ah i probably misread

[elder heart]

Oh, yeah was doing this originally. Wonder what our diff is

[idle stone]

different issue :)

[mellow flint]

It's Windows

[elder heart]

wandows

[idle stone]

i read "rerunning with ..." and thought you meant pressing o or r in the CLI to re-open the app

[elder heart]

No, straight quitting

[mellow flint]

Alright, gonna do a release supporting the newest version of dx 🙂
Will bump this to version 0.2.0 because it requires installing a new version of the CLI

FYI @idle stone to make moving subsecond issues easier once it's split off from dx, I added the string "subsecond" to every issue I made

Maybe you want to do that too? https://github.com/DioxusLabs/dioxus/issues/4192

[idle stone]

understandable

[mellow flint]

I think after this release I should be done for a moment with hot-patching, it works very well now for a prototype 🙂
Will update things again when there's a new dx version or a bug needs fixing or someone needs a review.
Or if there's feedback for the readme.

But I won't try to add new features unless someone absolutely needs them

(so I can finally enjoy @idle stone pause menu PRs hehe)

I think crates io is drunk

[idle stone]

published the macros crate first?

[mellow flint]

check the first lines

[mellow flint]

Yep

[idle stone]

ah

[mellow flint]

Then confirmed that that worked by publishing it again

[idle stone]

sneaky unicode lookalike characters surely

[mellow flint]

https://crates.io/crates/bevy_simple_subsecond_system_macros/versions

it's also on here

[idle stone]

outdated readme snippet

[mellow flint]

well good thing I didn't publish yet 😄

Fixed

time for

while true; cargo publish; sleep 60; end

Oh, also not showing up on lib.rs: https://lib.rs/crates/bevy_simple_subsecond_system_macros/versions

I guess crates.io is just having issues?

[idle stone]

while ! cargo publish; do sleep 60; done

stop when it succeeds 😄

strange issue

[mellow flint]

fish user here 😉

Oh it worked!

but oh heck

I can't publish something with a git dependency

💀

Time to update the readme

[elder heart]

Run this except increment the patch every time

[mellow flint]

[idle stone]

ah yeah, and you can't publish something with a [patch], even if it's for a [dev-dep] 😡

[elder heart]

That’s what we call CI

[mellow flint]

We fully depend on dioxus-dev-tools' main branch 😅

[elder heart]

ctrl+s -> save + commit + publish workflow

[mellow flint]

is that "bleeding edge"? 😄

[idle stone]

yes lmao you solved it

[mellow flint]

Correction: we depend on a rev we know to be working

[mellow flint]

• publish to itch.io + make reddit announcement post

now we thinking like a 10x dev

[idle stone]

continuous deployment

gotta make it actually continuous, that's still discrete

[mellow flint]

Oh heck you're right. Should open a websocket to crates.io, itch.io, and reddit to continuously shuffle my data in

[idle stone]

deployment streaming

[mellow flint]

Oh hey I just found out #[hot] works for observers

neat

[mellow flint]

Updated all of Foxtrot to use #[hot] and it's not playing well with mold: https://github.com/DioxusLabs/dioxus/issues/4194#issuecomment-2907581448

So I switched to lld, which costs me around 100 ms

I think that's fine

[elder heart]

continuous integration only feasible on analog computers

that's the conclusion reached

we need a crate that works like crater. it depends on every available crate in rust

[mellow flint]

[dependencies]
"*" = "*"

[elder heart]

no way

[mellow flint]

it was a joke 😄

[elder heart]

trolled

[mellow flint]

@idle stone dynamic builds reduce patch time from 3.5 s to 1.8 s on Foxtrot for me

[idle stone]

and you didn't do anything special besides enabling bevy/dynamic_linking?

[mellow flint]

I had the same optimizations on last time

So I don't think so

But I can check if you want

actually, it's a bit misleading since I used the old CLI and dep last time

let me check again with the new stuff in place

[mellow flint]

I take it back

enabling dynamic linking still gives me a patch time of 1.6 - 1.8s

so the speedups are coming from somewhere else

🤷‍♂️

[idle stone]

disabling* ?

[mellow flint]

no, I just enabled it to see if it would be slower

The 3.5 s were last time that I tried it with Foxtrot

[quartz whale]

Ah, I suppose my 550ms patch times are actually pretty good then 😂

I had a feeling that dynamic wouldn't make much difference, because I saw something about dx doing "ThinLink" which seemed to me like it was equivalent to dynamic_linking, but instead of being part of the crate configuration it's part of the build tool

Still worth having it enabled so that you don't get crazy compile times if you happen to use cargo r instead of dx though