#bevy_seedling

1 messages · Page 2 of 1

stoic igloo
#

It’s hope

static quest
#

Well actually, there is a surprising amount of "generic" UI stuff in DAWs like toolbars, menubars, dropdown menus, tileable panels, a settings window, tooltips, scrollbars, and (in my case) a properties panel and a sample browser.

stoic igloo
#

Indeed

static quest
#

Things like knobs, the timeline, and the piano roll need to be custom of course, but all of the other generic stuff is really hard to get right if the UI library doesn't have them built in already.

obsidian tusk
#

No for sure, I don’t think it’s a bad choice at all to go with something that does all that stuff for you, me saying that I would just use Bevy and reimplement everything is more a reflection of my (very possibly outdated) trust in the state of Rust's UI ecosystem than it is a statement on using UI libs in general

static quest
#

(And don't get me started on accessibility)

obsidian tusk
stoic igloo
#

Don’t worry. I just mentioned bevy_egui and got it

static quest
#

Like one thing that's interesting to me about the KDE framework is there is a toolbar widget that automatically puts actions that don't fit on the screen into a dropdown menu. Definitely useful for small screens.

#

They also have a tooltip widget that can show a much longer explanation if the user hits the "Shift" key.

obsidian tusk
#

Oh that’s fantastic

static quest
#

(Though if I do got with it, I will probably make my own tweaks to reduce some of the padding.)

obsidian tusk
#

You might be convincing me to dig into KDE instead of using flutter, although I’m going to try not to nerd-snipe myself with it too hard until I actually start work on the project that needs it

static quest
#

And plus I like the style of their icon theme.

static quest
stoic igloo
#

I like Magix DAW so 80 bit processing might be on the bus is my guess.

#

Meadowlark

static quest
#

Ah, lol. Autocorrect.

stoic igloo
#

I like it.

static quest
#

Though I am actually just aiming for 32 bit processing now. I may add 64 bit processing later, but it's not a priority.

obsidian tusk
static quest
stoic igloo
#

I liked FL Studio 9 32 bit great bass

#

They changed languages going to 64 bit

#

And I stopped

obsidian tusk
#

That’s part of why flutter is appealing to me, it’s old enough (and it’s been pushed enough by google) to be battle-tested but new enough to learn from the mistakes of frameworks like Qt

#

In particular, it was made after react/redux became big, so it learned a lot from that paradigm

static quest
#

I think the main pain with Flutter is the Rust/Dart bindings.

obsidian tusk
#

Absolutely, it’s a nightmare

#

I wrote a system to automatically generate protobuf files from rust types because it’s the only communication method supported by both apart from stuff like json but protobuf is such a garbage format that generating the rust types from the protobuf schema results in an unwieldy nightmare

static quest
#

And a dealbreaker for me was the lack of support for "pointer locking", which is important if you don't want a knob/slider to stop registering drags if the cursor hits the edge of the screen.

#

In fact, hardly any GUI framework supports that. Even Qt doesn't support it directly, but it does seem possible to do it in a "hacky" way.

obsidian tusk
#

Yeah, that’s fair. That’s to be expected from a framework that was designed for mobile first. In my case I specifically want touch devices to be first-class though (even though desktop is the intended primary platform), I don’t necessarily need to support mobile devices specifically but if I can then that’s a nice bonus

obsidian tusk
static quest
#

Ah yeah, that definitely would be important for mobile.

static quest
obsidian tusk
#

JUCE is an absolute nightmare and if I never have to use it again it’ll be too soon 😅 It’s necessary and I’m glad it exists in the abstract but it’s probably my least favourite thing about working in audio

#

Just yesterday we had an issue with our product that meant we had to set a major new feature to disabled-by-default because of an issue that ultimately appears to be caused by a bug in JUCE

static quest
#

The Visage library made by the guy who made the Vital synthesizer looks interesting (you can do some crazy custom shader effects with it), but it's still a fully retained-mode object-oriented C++ library.

obsidian tusk
#

Ooh I’ve never heard of that before, it looks really interesting

#

Love the idea of a library that combines processing-style rendering with standard UI elements

static quest
#

And I've seen more developers using the clap_wrapper library for cross-plugin-API development.

obsidian tusk
#

Really? I don’t what all the major manufacturers are using but all the ones that I know of are using JUCE

static quest
#

Yeah it's really neat. Although I imagine the lack of a declarative API would get pretty hard to maintain for a large application.

obsidian tusk
#

Yeah I feel like it’s more useful for something like a plugin UI than for a DAW

static quest
obsidian tusk
#

Ah fair, yeah I don’t know much about what’s going on with any smaller plugin devs, we make a plugin host at work so we mostly care about the most-popular plugins and for my own personal stuff I have a small stable of major plugins that I use for everything. I don’t know what the cutting edge of plugin dev looks like really

stoic igloo
#

A lot are emulations

static quest
stoic igloo
#

It makes people comfortable

#

I would like to throw my hat in the compressor lot.

static quest
stoic igloo
#

It is.

obsidian tusk
static quest
#

My only complaint is the lack of a dark theme. But I think the developer is considering making a paid version of the plugin with a nicer UI and more features.

stoic igloo
#

I used it as a reference for building my compressor

obsidian tusk
static quest
#

Yep!

#

I like the tagline "everything can become pink noise if you try hard enough".

#

(Though that might be for the similar plugin in that repo called "Loudness War Winner")

obsidian tusk
#

Lmao

stoic igloo
#

Ah

obsidian tusk
#

Do you know the video that’s referencing? It’s one of my fav videos about audio of all time

#

I guess it could be a coincidence since they’re both referencing the loudness war but any excuse to post this vid https://youtu.be/s_ANEQu5Lto?si=j0MN1C1ahPE1oy9q

YouTube
Dan Worrall
I Won The Loudness War

I did it. I won. Good luck getting any louder than this. You can all stop this sillyness now. Loudness doesn't matter, just make it sound good.

Lots of you requested I release this mix as it apparently "slaps". That means it's good, right? So you should now or soon be able to find it on all the usual platforms, or here on YouTube:
https://youtu...

▶ Play video
#

Seeing a + before LUFS is so cursed

static quest
#

Yeah, I've seen that. Great video.

stoic igloo
#

Gentleman EQ edition sound

#

Tokyo Dawn

static quest
#

Yeah, the TDR plugins are great.

#

I wish there was a way to bundle them with my DAW project. But alas, they are not open source.

#

It's easy enough to download them though.

stoic igloo
#

Write a list of free plugins suggestions in readme

static quest
#

Yeah, that's a good idea.

#

The Molot is more of an "aggressive" compressor than a mixing compressor though.

stoic igloo
#

I tried learning the algorithm with video and couldn’t

#

It is aggressive.

static quest
#

But it is a high latency, heavy CPU limiter.

#

As well as ZLEqualizer.

#

Though I learned through porting the Vital reverb that porting C++ to Rust is quite time consuming. I'm probably just going to set up a build system to compile the C++ code into a static library.

#

But I should focus on actually finishing the core DAW first. 😅

stoic igloo
#

Probably

static quest
#

A lot of the backend is done. It's mostly the frontend that's been the blocker.

stoic igloo
#

Gotcha

#

Wow

static quest
#

I'm in a lucky situation where my parents can support me financially for years to let more work on it full time. I of course want to find a way to make a living out of it someday, but that would be much easier to figure out once I have an actual product.

#

It also helps that I'm single and can live comfortably at my parent's place. I'm aromantic, so I'm not interested in "traditional" things like getting married and raising a family.

stoic igloo
#

More the norm

static quest
#

Also the cost of living in the country in Kansas is quite low compared to the rest of the country.

viscid plank
static quest
#

(Of course medical insurance is still much higher than it should be.)

static quest
stoic igloo
#

We need an Alice podcast like pronto about Bevy. She’s so hip.

#

She has same insights as you.

static quest
#

And also maybe try a yearly fundraiser like what Krita does.

stoic igloo
#

Maybe just wait for Bevy GUI

static quest
#

And personally I could get by with just $20-30k a year.

#

More would be nice of course, but I'm in open source for the passion, not the money.

stoic igloo
#

right

static quest
#

I feel like because I don't want to get married or start a family, my life would just feel unfulfilled if I just sold my soul to a corporation.

#

Though that being said, if I can't raise money myself then I might be able to get a part time job or something.

#

What's also lucky is that my parent's property actually has two houses on it. So once my sister finishes her degree and moves somewhere else, I'll probably move there. Granted it's a tiny house, but I don't need anything more than that.

#

Though it would be kind of nice to live closer to a city. But that's not absolutely necessary.

stoic igloo
#

Well …
It’s
Kansas

static quest
#

Kansas has some cities 😎

#

It's a 30 minute drive to the nearest town with a grocery store, and a 50 minute drive to the nearest city.

#

Though the countryside is very pretty. That's where I got the inspiration to name my projects after native flora and fauna in Kansas (Meadowlark, Firewheel, etc).

stoic igloo
#

That’s like suburb without suburbs

static quest
#

Oh yeah and I'm calling the audio engine in Meadowlark "Dropseed", which is a type of wild grass, and it's the seed to make SICK DROPS 🤘

stoic igloo
#

Bevy without GUI… ok
DAW without GUI… not ok

#

It’s all about the drops

static quest
#

Yeah, GUI sucks

stoic igloo
#

dang

static quest
#

But it's necessary, and I'm really passionate about getting both the UI and the UX right.

#

And plus there are some fun parts of GUI, like designing the themes, and doing custom shader stuff to render waveforms and visualizers.

#

I also want to have a "command palette" like in VSCode. That's something I've never seen a DAW do before.

#

But that's a later goal, if I even get around to it.

stoic igloo
#

So wha is MAX channels?

static quest
#

What do you mean?

stoic igloo
#

Since
It’s
32 bit

#

Aw app is 64

#

Sorry

static quest
#

Well initially I'm supporting mono/stereo channels, but the audio graph engine can support any arbitrary number of channels.

#

As for bit depth, I'm planning on 32 bit initially, and then work on 64 bit later.

#

And in reality, there's actually very little quality difference between 32 bit and and 64 bit audio processing, so little that it is hardly audible if at all. The much bigger factor to audio quality is the quality of the antialiasing algorithms.

#

*and the quality of the filtering algorithm. For example SVF filters produce a higher quality output on 32 bit than biquad filters.

stoic igloo
#

Hopefully, GUI will get sorted.

#

For Bevy
For Meadowlark
For Rust

static quest
#

Plus 32 bit has a performance advantage. You can fit twice as many 32 bit floats into a SIMD vector than 64 bit floats, and you can fit twice as many 32 bit floats into the CPU cache.

stoic igloo
#

It’s 64

#

Right

static quest
#

64 what? Are you talking about f32 vs f64?

stoic igloo
#

To fit in CPU

#

You use 1/2 so you have twice

static quest
#

Well, a SIMD vector on SSE2/ARM neon can fit 4 f32s or 2 f64s. On AVX you can fit 8 f32s or 4 f64s.

stoic igloo
#

Any rust SIMD and multi threading learn tips?

static quest
stoic igloo
#

I’ll check them out. Thanks.

static quest
stoic igloo
#

No Arc<Mutex> is also like how?

#

Just saying.

static quest
stoic igloo
#

right

#

Hopefully, you can add that also.

#

with 64 bit

ionic sedge
#

I’m pretty sure we can largely hide the scheduling from the user when they don’t care about it, or if they apply animations with some animation API we write.

That is, they’d still just mutate a simple set of parameters. Everything else would happen in the background. Anyone using bevy_seedling now could probably migrate with zero code changes.

However, they’d now have a big new set of capabilities due to the arbitrary scheduling, if they wanted it. And we would build the animation API around it, which would allow very easy performance tuning at runtime (if you’re applying a slow animation, maybe you only need events every 16ms).

#

I’ll want to double check my assumptions here, and I think we’d need to make those changes I suggested to Diff maybe, but it I’m pretty sure it would just be a straight win for users on the game side.

idk if it would change how easy it is to write nodes, but hopefully not by too much since it seems like a really powerful approach

obsidian tusk
ionic sedge
obsidian tusk
#

Which also lets us have some helpers around receiving patch events, like there could be a type Node: Diff + Patch; assoc type instead of manually doing events.for_each_patch::<SomeNode>

ionic sedge
#

Oh but also I don't think you could just apply patches in bulk like that if they're time sensitive. But maybe I need to check your PR.

obsidian tusk
#

As a helper in firewheel-nodes not in core

obsidian tusk
# ionic sedge Oh but also I don't think you could just apply patches in bulk like that if they...

So the way that param updates are done in the PR is to process up until the next event and then apply all the param updates that occur at the same time in one batch. Since events for different fields aren’t necessarily correlated, the solution I had in mind to prevent accidentally processing small ranges is some kind of opt-in quantisation for param update times, I don't think it should be enabled by default but either Firewheel or seedling (probably Firewheel, as then asserts can be added as close as possible to the process call in order to allow LLVM to optimise better) should be configurable to quantise events in order to minimise how often we process a very short range

ionic sedge
obsidian tusk
#

Yeah it’s something that would very rarely change and it’s an optimisation anyway. Seedling can ensure that it always produces events quantised to time, it doesn’t need to be a feature of firewheel except for the sake of paranoia and giving extra info to LLVM

#

I’m not even sure that LLVM would use the info unless it’s a constant

#

Long story short, I don’t think it’s worth putting in Firewheel unless we find out later that it’s an issue

ionic sedge
#

If the performance is potentially better, and it's not too difficult, it seems a lot easier to me to manage quantization in Firewheel itself. That would guarantee that even if someone was doing manual scheduling (which we should allow) and they didn't quantize correctly, it would still work great.

#

In other words, managing that on the processor side guarantees correctness not just for Bevy, but all Firewheel users.

obsidian tusk
#

For sure, I get that. It’s possible to add in a backwards-compatible way though, at first it’d probably be a global setting hidden away in the FirewheelProcessor and not directly interacted with by nodes (although it could be added to AudioNodeInfo down the line)

#

Btw, is there any plan to introduce latency compensation? If you ever want that then it might be useful to put it on AudioNodeInfo now, even if it goes unused, so that you can add it later without needing to communicate that developers need to retroactively add it to their nodes

ionic sedge
#

I don't think BillyDM was interesting in doing that. Too complicated I think.

#

If you happen to need that in the meantime, it could be done with a LatencyCompensation node or something.

#

I figure it'll only show up in either niche scenarios where you're doing lots of varied processing, or via something like a limiter on the output chain, where that latency doesn't matter.

#

And in the former case, it's relatively easy to look at your graph and figure out where you'd need compensation, right?

ionic sedge
#

Side note -- now that we have a limiter, would it make sense to automatically insert it before the output in the default configuration?

I have some changes in progress that provide a more opinionated default configuration, with a few additional options of increasingly minimal configurations. You can see that work here. Would a limiter with a small (1-3ms) lookahead be a good default for most games?

obsidian tusk
#

but yeah I’d be very happy putting it on the master by default or at least giving a method to add it when instantiating the plugin, I think almost any non-pro software that produces audio should have a limiter on the master by default

ionic sedge
#

Keep in mind that it's added to all other sources of latency, which right now is generally no better than 50ms.

#

(round-trip anyway, idk how much of that is input latency)

mystic epoch
#

Just want to interject real quick that for skilled rhythm game players 10ms is definitely noticeable, but I don't know enough about what you're talking about to judge if it can be fixed with lag compensation which rhythm games normally also have

ionic sedge
obsidian tusk
ionic sedge
# mystic epoch Just want to interject real quick that for skilled rhythm game players 10ms is d...

I think this is very valid! Although it would be so easy to change the default configuration (and probably wouldn't be used by bigger projects anyway) that I wouldn't worry too much about it. In other words, a rhythm game really should not use the default configuration, and it's so easy to change and customize that this (hopefully) shouldn't cause anyone problems where that latency (among other things) is critical.

mystic epoch
#

Yeah, pretty much from when you press a button until the game registers it. However the way lag compensation usually works is that it allows you to desync visual and audio, usually in 1ms steps. So if I put the lag compensation to +5 it'll delay the audio by 5ms I believe

#

I don't know how rhythm games handle it technically and there's probably different ways of handling lag, but the end goal is always the same. To allow users to delay either the visual or audio aspect because their monitor is slow or something else is causing lag

ionic sedge
#

This user's rough calculations put osu at like 50ms round-trip I think. Now maybe they don't have an optimized setup (and I don't know if anyone's replicated it), but it seems like it can be quite high even for a timing-critical rhythm game?

mystic epoch
#

Yeah, if just delaying the audio so that it aligns with the visuals is good enough then the limit is how high (or low) the lag compensation slider will let you go I guess

ionic sedge
#

Oh but you're saying games will also facilitate the opposite? i.e. delay the visuals to line up with the audio?

mystic epoch
#

Well that's the same as offsetting the audio to play earlier isn't it? But yeah, rhythm games will usually let you go from some negative value to some positive value range

#

Could be that some games just delay the visuals, I have no idea how different rhythm games handle it on a technical level

ionic sedge
#

Well, we can't trigger audio to happen before the user presses buttons! So if there's negative compensation, then it seems like the only thing that could be done is delaying the visuals.

#

But it might be more complicated than that.

mystic epoch
#

I guess what's more important to me when making a rhythm game is getting an accurate timestamp for use in scoring calculation

obsidian tusk
#

but sounds that occur in response to user input aren’t latency-compensated

obsidian tusk
ionic sedge
ionic sedge
obsidian tusk
mystic epoch
obsidian tusk
#

Actually, something that I thought might be useful in Firewheel would be multiple coexisting custom_states, using something like type_set::TypeSet. So that nodes can have some shared metadata types that can be read generically. That would be useful for latency compensation because a third-party library like seedling could provide a LatencyInfo type that nodes can set as a custom state without affecting any other custom state they have

#

Although for Seedling specifically it can just define that as a component and nodes can configure it using the required component system

#

That doesn’t work for non-Seedling users but if it's necessary later then afaik it'd be a backwards-compatible change to go from a single custom state type to many custom state types so we can safely kick that can down the road

#

Btw corvus, I made a dumb one-line change in the last commit to my limiter PR that broke it and I’ve submitted a PR to fix it

stoic igloo
#

Delay compensation is a number z = x + y that has its own x delay and group delay y to keep everything in sync in audio where heavy CPU algorithms will add delay y to everything.

#

Just talking audio

ionic sedge
obsidian tusk
#

I don’t think using a TypeSet couples it to any frontend, part of the appeal to me is that, as far as I understand it, it’s frontend-agnostic

#

Not going to annoy BillyDM with an @, but if you see this I think the scheduled events PR is ready

ionic sedge
#

(The component-based appraoch.)

obsidian tusk
obsidian tusk
#

Once you’ve figured out those points, calculating how much latency to add to them is pretty easy

static quest
static quest
static quest
static quest
#

On the first iteration of the audio graph engine for my DAW, we actually messed up on the algorithm and ended up doubling the amount of latency instead of correcting it. 😅

static quest
#

@obsidian tusk The PR looks good. My one nitpick is I'm not sure SimpleAudioProcessor is the best name for that trait. Maybe something like AutomatableAudioProcessor or AutomatedAudioProcessor?

#

Also, I guess it wouldn't hurt to add a latency field to NodeInfo in case we do decide down the line to add that feature.

ionic sedge
static quest
#

Ah, no it wouldn't.

ionic sedge
#

Aside from that, having access to the latency like this would also give frontends a standard way to access and account for it.

#

Even if Firewheel itself isn't using it.

#

(So it would be super useful to have anyway.)

static quest
#

In the CLAP spec (and in my DAW engine), plugins are required to deactivate first in order to change their latency dynamically. That would definitely be complicated to do in the Firewheel engine without adding a lot of complexity.

#

Much of the complexity in a DAW audio engine is just doing bookkeeping on the state of CLAP/VST3/LV2 plugins. While I am planning on adding CLAP support to Firewheel, it will just be basic support.

#

And even then, CLAP plugin hosting might be best handled by a separate (or even a 3rd party) crate.

static quest
ionic sedge
#

But if by convention, node authors write their nodes such that the latency won't change after construction, then having access to a latency value in the node info would be perfect.

obsidian tusk
ionic sedge
#

Simple is kinda not that great though. Simple compared to what? Without deeper knowledge of the crate, Simple doesn't really mean anything. (I'm also not the biggest fan of SpatialBasicNode. I think it should just be SpatialNode, and then if we have a more advanced one, we can give it a fancy name like HrtfSpatialNode.)

obsidian tusk
#

ChunkedAudioProcessor?

ionic sedge
#

haha in one way that's better, but it's also kinda funny

#

ChunkyAudioProcessor

obsidian tusk
static quest
#

Yeah, I'm not sure what the best name is either.

#

Maybe even something like EventSchedulerWrapper?

exotic ingot
#

Batched?

static quest
#

Oh yeah, batched might be the best word to describe it.

obsidian tusk
#

I kinda like Simple even if it’s not as meaningful because it shows the intent as a convenience trait but Batched def explains what it does better

static quest
#

What about SimpleBatchedProcessor?

obsidian tusk
#

You’re ok with the overall design of BufferRange though BillyDM? I wasn’t sure about having impl Trait fields but it does make it nice to use

obsidian tusk
static quest
obsidian tusk
#

Tbh if we’re going to say batched I’d just call it BatchedAudioProcessor

ionic sedge
#

I kinda like that ngl

static quest
#

Yeah, that could work too.

ionic sedge
#

oh the message i replied to is unrelated

#

It looks like itertools 0.14 is used in a number of Bevy crates, so it won't generally constitute an additional dependency for Bevy users, but it might for non-Bevy Firewheel users. But it seems like it's actually not being used at all.

static quest
#

I'm fine with itertools. Handy crate.

static quest
ionic sedge
#

I can also check the real-world performance delta compared to Firewheel main if you want. I'd like to do a quick integration to see how the scheduling API might look in the ECS.

static quest
#

And actually, there is one more thing I thought of. It would be nice if there was a way to pause and discard future events.

#

Because currently there is no way to do that once an event is scheduled.

ionic sedge
#

Actually yeah that would be kinda important if we buffer events on the game engine side.

#

i.e. if you're scheduling events 2-3 frames ahead of time for a pitch animation, but you suddenly want to pause or cancel the animation

#

you'd be kinda toast

static quest
#

The pausing can be solved by counting how many frames have passed since the event was paused for each event in the buffer, and then adding that number of frames to the event's instant when resumed.

#

And discarding of course would be easier. Though I think there should be an option to either discard all events, or just future events.

static quest
#

That being said, we should probably also add a method to the Diff trait that forces every parameter to send an event. That way we can ensure that the ECS state and the node processor's state stay in sync even if future events have been discarded.

#

Hmm, and I think it would also require a special event type for pausing/discarding to ensure that events are always in the correct order. If we just used atomics then the order of pause/discarding commands and parameter events could get out of wack.

ionic sedge
#

oh like a refresh

static quest
#

Yes

ionic sedge
#

If we can inspect what events are removed, we can handle re-syncing any state that is outdated.

#

Potentially mutating the representation in the game loop.

static quest
#

Still, it would be nice if there was an easy way to refresh.

ionic sedge
static quest
#

Well, the events are sent to the audio thread.

#

While I guess in theory we could send the events back to the main thread, there would be a delay.

ionic sedge
#

Well either way, I think a refresh method would be easy? We can derive it in essentially the same way we derive the normal diff behavior.

static quest
#

Yeah, that's what I was thinking.

ionic sedge
#

And yeah it would be a simpler design if we don't have to send back cancelled events.

static quest
#

Keep in mind that all events are batched and flushed to the audio thread at once, which ensures that all nodes receive those events in the same process block.

ionic sedge
#

Hm, the exact details there do seem tricky

static quest
#

Yeah, it is a bit tricky.

ionic sedge
#

What's the simplest approach? Just a Cancel event? Is it impossible to managing pausing from the game-logic side in the general case?

#

I suppose the problem is you wouldn't necessarily know which events have been consumed by the audio processor.

static quest
#

But I suppose we could get away with just a Cancel event. The game engine could re-schedule the events if needed.

#

Oh yeah, and by the way there is no need to pause InstantMusical events because the transport itself can be paused.

#

But having the ability to pause/resume events would be nice. But if it ends up being too complicated to implement, we could just use the cancel approach.

ionic sedge
#

I think we might just get away with it. In practice, we'd typically only evict maybe 4-8 events with a Cancel, and since the whole animation would be paused, the slight jump you'd observe when unpausing probably wouldn't be noticeable.

static quest
#

Yeah, and the only events that really require exact timing are the musical ones, but those can be paused anyway.

obsidian tusk
# static quest I'm fine with itertools. Handy crate.

Yeah, I should’ve explicitly mentioned it in the PR description but it might as well be part of std at this point. It might actually not be used in the PR any more though, let me check because I think I might have gone with a different implementation of the thing that originally required itertools

obsidian tusk
static quest
#

There are a lot of things that should be in the standard library (especially rand and log).

#

And smallvec and arrayvec.

obsidian tusk
#

Which might be a reasonable implementation but I don’t really know

ionic sedge
static quest
obsidian tusk
static quest
#

By that I mean the node's event buffer gets cleared at the end of a processing block.

obsidian tusk
#

I think for seedling's usecase that’s ok, right? Seedling can schedule more events than are absolutely necessary for the current buffer period but it doesn’t need to persist then between frames

static quest
#

Hmm, it might be possible to create an API where a node can request the Firewheel processor to hold on to an event.

#

What I'm saying is that the Firewheel processor itself clears the event buffer at the end of the process block. So the event won't exist in the next processing block.

obsidian tusk
#

IMO it’s a reasonable restriction that scheduled events are only valid for the next update call and you need to reschedule them

ionic sedge
#

But again, that is not a robust way to schedule animations.

#

That will result in gaps with unstable framerates. (If we can't schedule a little in the future.)

static quest
#

I'm not talking about rescheduling. I'm talking about that the way your PR is set up, you won't even get any future events. They will be discarded before you can use them.

obsidian tusk
#

Keeping track of which events have been consumed and which haven’t seems like a huge source of gotchas

static quest
#

True. The only other way though is to require the node to create its own event buffer.

obsidian tusk
obsidian tusk
static quest
obsidian tusk
#

Hm, you’re not wrong there. I wonder if just documentation would be enough to solve the confusion

static quest
#

Especially since on some platforms (like Windows) the size of the process buffer can actually vary from block to block.

#

(Annoying I know, but Windows is Windows)

ionic sedge
#

But it's a fundamental API limitation that, again, cannot be solved by the user code. Maybe I should create a diagram to illustrate the problem.

obsidian tusk
#

If the external code owned the event buffer and passed it into FirewheelCtx::update that would make the usage clearer

#

Then firewheel doesn’t need to decide the event buffer clearing strategy

stoic igloo
#

I started Windows compiling only last week

static quest
#

However, say if you had a video frame that lasted 15ms, but the audio processing block only lasted 5ms. You would lose 10ms of scheduled events.

obsidian tusk
#

Yeah, I still maintain that it’s fine to just not handle that. It’s better than the status quo where you lose 15ms of scheduled events because you can’t schedule them to begin with 😅

ionic sedge
#

Also, because there's no synchronization between the update rate of the audio and game, you'd actually lose events all the time, even with stable frame rates. (Unless I'm misunderstanding the mechansim.)

static quest
#

Though I suppose the typical frame time for a video game is 16ms and a typical audio block time is 23ms (1024/44100).

obsidian tusk
static quest
#

Hmm, yeah. This is turning out to be quite complicated.

ionic sedge
#

But I strongly believe in the direction

#

I think it's worth seeing it through.

static quest
#

Though like I said, I think it would be possible to create an API where nodes can ask the Firewheel processor to hold on to events for the next processing block.

obsidian tusk
#

IMO the ideal is that the frontend only ever schedules events for exactly the buffer period and the processor always processes all the events. Like, that would just be an invariant of the API. If that’s a reasonable invariant is another question

static quest
#

But I don't think it's possible for the frontend to know exactly what the buffer period is.

obsidian tusk
obsidian tusk
static quest
#

And even if you knew the buffer period was constant, you don't know exactly where you are in that buffer period (you could be in the middle of one, or you could be right before a new one is about to start).

obsidian tusk
#

Yep I def don’t think the buffer period should ever be assumed to be constant

static quest
ionic sedge
#

Either that or, if we only clear events when the node pulls them off the event list, the node could just "peek" until it gets to the end of the queue or the next event is beyond the current processing period.

#

I'm ignorant of the implementation, so I don't know how easy that is.

static quest
#

Well, the node will still pull events if it gets a "process immediately" one.

ionic sedge
#

Would it be terrible to have two event queues? One with timestamps, and one without?

obsidian tusk
static quest
#

I think even just returning a list of events indices you wish to be kept the next block is good enough.

ionic sedge
#

Could we fit a "keepalive" flag next to the events in the list, and clear any where it's false?

stoic igloo
#

stopped = true

obsidian tusk
#

I’m still having trouble understanding why firewheel can’t just remove any events that are within the just-processed period

stoic igloo
#

del

obsidian tusk
#

Like, why the nodes need to think about that

stoic igloo
#

last sample played then delete

static quest
#

Like this:

pub struct ProcInfo<'a> {
    // ...
    
    // This is a pre-allocated Vec that will be empty. The user pushes
    // the indices of events it wishes to be retained in the next
    // process block.
    pub keep_event_indices: &mut Vec<u32>,
}
static quest
obsidian tusk
static quest
#

You want a scheduled event to be scheduled even if it hasn't elapsed this processing block.

obsidian tusk
#

Like, it’s one thing if it’s always hidden inside the helper trait but any processor that directly implements AudioNodeProcessor and wants to handle scheduled events now has extra boilerplate

obsidian tusk
ionic sedge
static quest
#

No, I don't think it needs extra boilerplate. The flag will be set to false by default.

static quest
#

Though I suppose we could add a DiscardEventsForThisNode event that gets sent to the Firewheel processor.

ionic sedge
#

Hm, I see. Do you have some objects or scenarios in mind where the node would want to discard a bunch of events?

stoic igloo
#

Node deletes sample

static quest
#

Well, if you have an animation scheduled, but you want to pause or stop that animation.

ionic sedge
#

Oh yeah I guess I just assumed that would also be handled by the Firewheel processor. Is that unreasonable?

static quest
#

But you're right, I think it might be possible to have the Firewheel processor automatically do this. In fact, we might not even need the fancy BatchedAudioProcessor trait, the Firewheel processor could just call the node's process method multiple times as necessary.

ionic sedge
#

Oh one thing I was thinking -- Cancel should only remove events with timestamps, right? Or maybe just the ParamData variant? Like, I don't think we'd want to drop a sampler sequence event.

static quest
#

We could even have a CancelAllEvents event for convenience.

ionic sedge
#

It would kinda just look the same, woudln't it?

static quest
#

Though one thing the user would need to make sure of is that they call FirewheelCtx::update before they schedule any new events.

ionic sedge
#

i.e. exactly how it looks now, but the audio buffers may just be smaller

static quest
static quest
stoic igloo
#

Oh my goodness!

ionic sedge
#

hmmmmmm
that would be kinda cool

static quest
#

Of course the question is if it would work with the ECS. Obviously the Timeline type is handy in a data-driven environment.

#

(This is actually what I was talking about when I said that Firewheel could automatically schedule events for nodes if it used an event-driven API).

ionic sedge
ionic sedge
#

And for the standalone APIs, the Memo type should also keep it simple.

static quest
#

In theory we could require node authors to create a Bevy wrapper struct for their node type that uses the Timeline type. (Unless you have a better solution in mind.)

ionic sedge
#

Oh yeah I don't think we need that at all

static quest
#

*If the node author wants to support Bevy

#

Oh, cool

stoic igloo
#

Oh right

obsidian tusk
#

Is it a reasonable implementation to, when pushing a new event, delete any events that happen later than the just-queued event? Or is scheduling events out-of-order something we want to support?

static quest
#

Awesome, I'm getting excited about this! This would make it so much nicer to create custom nodes.

ionic sedge
#

If it's not complicated to allow, out-of-order would actually be nice imo.

obsidian tusk
#

Ok fair enough

static quest
#

Yeah, the Firewheel processor can automatically sort events.

ionic sedge
#

Imagine you're animating one param while applying simple changes to another.

#

Depending on the order, one might be scheduled "back in time" relative to the other.

#

Even though, once sorted, they'd both be reasonable.

static quest
#

Hmm, although sort_unstable might not work here, as you probably want to retain the order of events that happen at the same time.

obsidian tusk
#

Yeah, you want to handle it param-by-param so you don’t want to care about order

obsidian tusk
#

I only used sort_unstable because that’s what I default to

static quest
#

Does stable_sort allocate?

ionic sedge
#

No I think it's just slower sometimes

obsidian tusk
#

Oh great question, I’m pretty sure it doesn’t

static quest
#

Oh ok, cool.

obsidian tusk
#

Worth investigating but I don’t think it does

#

I think it just uses a different algorithm

static quest
#

Oh wait, no it does allocate. When applicable, unstable sorting is preferred because it is generally faster than stable sorting and it doesn’t allocate auxiliary memory. See sort_unstable. The exception are partially sorted slices, which may be better served with slice::sort.

obsidian tusk
#

Ugh ☠️ Ok

#

Well you can use an incrementing ID to work around that but it’s annoying

#

Although I think it’d be possible to make an API that allows you to not guarantee maintaining insertion order for now, but add it as a guarantee later if that’s desired

#

If a frontend is only scheduling events using the patch mechanism then the order that patches are generated is already not guaranteed

static quest
obsidian tusk
#

So you could just say that you don’t guarantee maintaining insertion order for simultaneous events but if that becomes an issue you can add that guarantee later

static quest
#

Hmm, the stable sorting wouldn't be a problem if there was a way to give the algorithm a pre-allocated buffer. I wonder if there are any crates that let you do that?

obsidian tusk
#

Ooh, interesting idea

static quest
#

If anything we might be able to implement our own stable sort algorithm that takes a pre-allocated buffer.

obsidian tusk
static quest
#

Apparently there are two algorithms that can stably sort a list without allocations (at the cost of being slower) called GrailSort and WikiSort, but there doesn't appear to be a Rust crate for those.

obsidian tusk
#

The algorithm’s speed probably isn’t too much of an issue considering the number of elements is small

#

but I think this is premature optimisation, I think until it becomes a problem then there just shouldn’t be a guarantee of preserving insertion order

ionic sedge
# ionic sedge Oh yeah I don't think we need that at all

To elaborate on this -- we already store a mini event queue per node in the ECS. It's on the same entity, essentially like:

(VolumeNode, Basline<VolumeNode>, NodeID, Vec<NodeEventType>)

We use the baseline for diffing, pushing the patches to the adjacent Vec<NodeEventType>. This does involve allocation for every node, but the flipside is that all nodes can be Diffed in parallel at once, without accessing the !Send audio context.

Typically, you'll modify a parameter in a system like this:

fn mute_volume(mut nodes: Query<&mut VolumeNode>) {
    for mut node in &mut nodes {
        node.volume = Volume::SILENT;
    }
}

Notice how we don't touch the baseline, node ID, or vector of events. For users who don't care about scheduling, this would be the same regardless of whether events have timestamps.

For users who do care, we could change the type of the local event buffer. Instead of a Vec<NodeEventType>, it could be some EventBuffer with a method like diff_scheduled. Then, if you want to manually schedule events at a particular time, you'd do something like:

fn mute_volume_scheduled(
    mut nodes: Query<(&mut VolumeNode, &mut Baseline<VolumeNode>, &mut EventBuffer)>, 
    time: Res<Time<Audio>>
) {
    // Let's schedule in the future by one frame
    let instant = time + Duration::from_millis(16);
    for (mut node, mut baseline, mut events) in &mut nodes {
        node.volume = Volume::SILENT;
        events.diff_scheduled(&node, &mut baseline, instant);
    }
}

(If you're not used to systems, the query might look scary, but it's really not that bad.)

Now, that said, users probably wouldn't have to interact with the scheduling 99% of the time anyway. We'd have a higher-level animations API that handles breaking up Diffs and scheduling them.

#

(I'm sure the example could also be way better, this isn't a refined idea yet.)

obsidian tusk
#

Where TimedEventQueue wraps an &mut T where T: EventQueue, so you tag the queue with a time just before passing it to diff

#

It’s only implemented for ContextEventQueue (again, can’t remember exact name) for now though, I think to make it fully generic you’d need to modify the EventQueue trait

ionic sedge
#

Oh yes that's actually what I was imagining for this, more or less.

obsidian tusk
#

I guess the details aren’t super important

#

If seedling already has a buffer of events, could the event queue be taken out of the firewheel context entirely, with the frontend being expected to handle all that stuff and pass a reference in?

#

or is that too onerous of a burden on non-seedling frontends?

ionic sedge
#

I think it's probably still good to have the event queue on the context.

obsidian tusk
#

Fair enough

turbid vigil
#

Does this support spatial audio?

#

👍

obsidian tusk
#

Yeah it actually has really good support for spatialisation

turbid vigil
#

awesome

obsidian tusk
ionic sedge
#

Oh that reminds me, I want to add some delay for the speed of sound and simulate doppler shift.

#

Those can both be handled purely by the ECS

obsidian tusk
#

Sick, that’d be useful for racing games

#

and I’d love to integrate that into my quake engine because it’d make getting up to crazy bhopping speeds more satisfying 😅

ionic sedge
#

One of my favorite gaming experiences was the impact of the speed of sound on combat in PUBG back when it came out.

static quest
#

Though to be fair, it is a pretty basic spatialization algorithm (just volume, panning, and filtering). I don't have the DSP chops to do any fancy HRTF or anything. (Though that being said, the Fyrox engine has an HRTF implementation we could borrow).

ionic sedge
#

If someone is interested, we could also integrate steam audio into some nodes. It's a C dependency, but it's very high quality, and someone made nice Rust bindings.

obsidian tusk
#

I wanted to integrate steam-audio for that bc it’s got a v efficient hrtf implementation but it’s a lot of work. I think I might have mentioned that before

ionic sedge
#

oh

obsidian tusk
#

Ahaha

ionic sedge
#

XD

obsidian tusk
#

Great minds

#

Yeah valve understandably put a hell of a lot of work into it for HL: Alyx

#

The occlusion stuff is particularly interesting to me with my quake engine, it’s the kind of audio effect that sounds great but doesn’t feel out of place in an old game even if it’s anachronistic, which fits the goals of the project very well

ionic sedge
#

kinda like raytracing on old games

obsidian tusk
#

Basically I want it to be "how you remember quake being"

static quest
#

By "occlusion" do you mean dampening a sound to make it sound like it's being played from behind a wall and such?

obsidian tusk
obsidian tusk
ionic sedge
static quest
#

Oh, neat.

#

In my simple spatialization node, there is a parameter to dampen the sound (literally just a lowpass filter).

obsidian tusk
static quest
#

And I of course would like to see an ecosystem of Reverb nodes for better realism.

#

A convolutional reverb would be especially useful, since you can replicate almost any space with it (at the cost of some CPU usage).

obsidian tusk
#

Yep a general convolver node would be great

ionic sedge
obsidian tusk
#

Yeah I hope one day I’ll be able to justify getting a VR setup

stoic igloo
#

rfd, ring buffer, and the math for convolution reverb, I think

obsidian tusk
#

but it’s not worth it for like 3 games that I’m interested in, all of which are linear games that I’ll play once

ionic sedge
#

VR Chat is kinda cool if you know people who use it. They have like... art exhibits, cute worlds, literal raves where you need to know someone to get in. Kinda wild, actually.

obsidian tusk
#

I live in berlin, we have all those things irl 😅

stoic igloo
#

That’s true.

obsidian tusk
#

I still think the culture around VR chat is fascinating though

ionic sedge
#

yeah but it could be ✨ virtual ✨

obsidian tusk
#

I barely know anyone who plays games, let alone VR chat

ionic sedge
#

Anyway, hopefully VR picks up again. Seems to be in another big trough right now.

obsidian tusk
#

Yeah I want it to become worth investing in because it's amazing when it works. Every time I get a chance to use it I love it and on a technical level I think it's fascinating

ionic sedge
#

Okay so.... the baseline performance with the new changes is essentially identical. Is that expected?

Here's the new plot -- you can see the old one here. Looks like any differences are just within the noise threshold of my system.

If my interpretation is correct, this means that having the ability to schedule finer animations does not impact the baseline performance in any meaningful way. I also ran a quick test with volume fades on a few samples, scheduling them for just the current time. I saw no difference in performance between the scheduled events and Firewheel main (where these events can't be scheduled).

In summary, it looks like this general approach will not distrupt the overall performance of Firewheel. We may observe lower performance on a per-node basis when we schedule animations, but that's essentially eating our cake and having it too; the user has the ability to finely tune how performant their animations are versus how smooth they are, all at runtime.

firewheel-new.png
ionic sedge
#

(Oh and to be clear, I doubt sending a dozen or so events per frame to achieve a fairly smooth animation would really harm the overall performance that much.)

obsidian tusk
#

Wow! Thanks so much for checking this!

#

Would it be possible for you to the performance if you schedule, say, 16 updates per frame? My intuition would be that it'll still be fairly efficient

trim belfry
#

let me just say I am not savvy with audio programming, but I enjoy learning from your discussions 😄

obsidian tusk
#

I think we’re all learning as we go too 😅

#

I’ve been doing audio professionally for years but there’s still far more stuff I don’t know than stuff I do

ionic sedge
#

Okay so here's the setup:

  • 48 total samplers in the graph.
  • 4 samples playing, all fading in over three seconds.

Results:

  • 1 audio event per frame: 108us per block (512 frames)
  • 16 audio events per frame: 112us per block (512 frames)

This is recorded over just about three seconds. The time represents the total audio evaluation time, so it's only a coarse indicator of performance for individual nodes. Note that the 16 events are typically truncated to 8 or so per buffer, given the lack of synchronization and current inability to schedule into the future.

#

Oh I should probably mention my game frame time is 8.4 ms.

#

Once we're able to schedule a little into the future, I realized it actually gives us some really nice properties.

For example, we can reliably ensure that audio events are processed at a fixed rate, regardless of frame rate or frame variability. For example, if we settle on a default rate of 16ms or so, then animations driven by the ECS can effectively guarantee that the events will arrive every 16ms whether you're running at 120fps or 30fps. Note that a rate of 16ms could actually provide better performance than Firewheel main on my computer, since my display typically pushes 120fps.

This would ensure the audio experience remains exactly the same regardless of these external factors, excluding some extreme cases like freezes that last several frames.

obsidian tusk
#

Oh wow that’s even better than I expected

ionic sedge
#

Yeah again it's not exactly the clearest side-by-side comparison, but it seems... totally fine? I think even if you animated every sampler with fairly small time steps, you wouldn't see anything crazy like an order of magnitude worse performance.

In fact, I think you'd still see significantly better performance than rodio 😅

obsidian tusk
#

Oh wow that’s even better than I expected

obsidian tusk
#

It makes sense though, the worst thing about decreasing buffer size isn’t the higher param update rate, it’s that you have to pay the cost of traversing the graph every period. So long as each chunk is larger than the vector width then you're only paying for a few more branches every period to handle the param updates, and with volume then you might not even be paying for that bc iirc there’s only one param. Tbh after seeing these results I wouldn’t be surprised if the cost to schedule an update for every sample isn’t so terrible, Timeline is almost certainly better for that ofc but at least you’re only paying for the graph traversal once. Should be discouraged but maybe quantising event times isn’t necessary

#

We have a strategy at work to massively reduce graph travel cost bc we need to have good support for low buffer sizes but I don't know if it’s proprietary so I’ll hold my tongue

#

It’ll still likely be slower than this tho, albeit with the benefit of lower latency

#

(To be clear, the scheduled events PR is a novel design and not inspired by anything with potential to cause legal issues)

ionic sedge
#

Oh btw there were some errors in the implementation that I had to fix to get scheduled events going. Basically, the proc info needs to be updated to reflect the smaller number of frames and the time range subset, but the PR currently just passes the outer proc info.

I might make a note on the PR about that. It's currently kind of annoying to re-create the proc info struct, even though the entire thing probably should be Clone (and also Copy). We might add a method on it to nudge the number of frames and time ranges to the correct values.

obsidian tusk
#

Yeah I put frames in BufferRange for that reason but it’s def not the best way to do it, it should be more transparent

ionic sedge
#

I noticed it with the spatial node in particular because it uses the proc_info frames to index directly into the buffers instead of iterating over them.

obsidian tusk
#

I’m wondering if there should be a way to specify that an input/output should be interlaced/deinterlaced, with some helpers to when the processor doesn’t care. Like, a volume node would prefer interlaced input/output for vectorisation/cache coherency reasons (and the existing implementation by BillyDM does interlacing for mono and stereo). It’d just be a hint though and all processors should handle input in any format

#

Sorry that’s not really relevant here except that it could potentially improve cases like spatial audio where you need to index directly into multiple channels

#

Since directly indexing into every channel with the same index will be faster with interlaced input

#

handling buffers of multiple formats might be slower overall tho 🤷‍♀️

ionic sedge
#

ya maybe not worth it

ionic sedge
#

Okay so I have some diagrams to illustrate why I've made the assertions I have.

This first one is a simple illustration of frame times vs audio block times at normal rates. At the start of each audio processing block, the audio processor advances its clock to the next block for external reference:

    let mut clock_samples = self.clock_samples;

    // The sample clock is ultimately used as the "source of truth".
    let mut clock_seconds = clock_samples.to_seconds(self.sample_rate, self.sample_rate_recip);

    self.clock_samples += DurationSamples(frames as i64);

    self.sync_shared_clock(Some(process_timestamp));

It then runs the audio processing with the clock_samples variable (not self.clock_samples) so that by the end of the block, everything is caught up.

60-44.png
#

In this scenario, if we only scheduled animations for one game frame's worth of time, notice how we already fail to achieve any kind of smoothness.

Typically, the audio events are flushed from the game to the audio thread once near the end of the frame. So in frame 0, we'd successfully send events for most of audio block 1. The problems really start by frame 1.

By the time we actually flush frame 1's events (again, at the end of the frame), the audio processor will already be done processing block 1! So anything that was supposed to happen near the end of block 1 will be quantized to the start of the next block.

The same thing happens to frame 2, but it's even worse this time! Over half of the block will be missing.

#

Similar things happen when the block rate is a bit faster, although we'd start getting bitten hard if we can't schedule into future audio blocks.

60-80.png
#

Finally, look at what would happen if a frame takes twice as long to render as normal. If frame 1 doesn't schedule a few frames into the future, then two audio blocks (4 and 5) will be processed before frame 2 flushes any of its events.

doubled.png
ionic sedge
obsidian tusk
ionic sedge
#

The status quo being one event per frame (no scheduling)? I'd argue that's also not great in the event of a long frame.

But a long frame isn't really the core issue. If we have the tools to solve the other issues, which are frankly critical, then this gets resolved "for free."

obsidian tusk
#

No totally, I’m more saying if for some reason scheduling far into the future is a huge problem then I’d be ok with choppier params when there’s a slow frame. I think scheduling into the future should be ok though, I’m still of the opinion that firewheel should use the timing info to only clear the events that were inside the just-processed buffer period (and, during processing, to only pass events to processors that are within the current period)

ionic sedge
#

Mm, I see. Although do note that we'd also have choppy params even without long frames. Any time the block rate is significantly faster than the framerate, you'd skip the delta between them on average. That is, if audio processing is twice as fast, you'd lose half of your sceduled events (assuming those event are evenly distributed over time).

obsidian tusk
#

Very true

ionic sedge
#

Alright, so at this point, how are we looking to move forward? It seems like there are a few things we have a rough consensus on. I think the things we want to achieve are:

  1. Introduce optional scheduling for events.
  2. Introduce an event along the lines of ClearScheduledEvents.
  3. Adjust the Firewheel processor so that it breaks up processing for each node into sub-blocks, where events with a particular time are grouped.
  • This requires event sorting, ideally without allocation.
  • Also, the Firewheel processor should only clear the events that it allows the nodes to observe.

Please correct me if any of this is wrong.

Unless I'm missing something, it seems that achieving all this is surprisingly straightforward. As we discussed, this would allow crates like bevy_seedling to start scheduling fine animations without any changes to the audio node code.

These points are partially, but not fully achieved by your PR @obsidian tusk. Do you think we can build the rest of these features on top of it?

@static quest You mentioned you'll likely want to take a break from Firewheel. Do you mind if we iterate on this in the meantime, or would you prefer to take the lead at a later date?

obsidian tusk
#

1 and 3.1 are implemented but not 2 or 3.2, I could implement those two pretty easily I think

ionic sedge
#

For 3.0, I think we can just remove the SimpleAudioProcessor and basically use that code for all processors, right?

obsidian tusk
#

Oh! I didn’t realise that’s what you were saying

#

Hm, I kinda feel like the flexibility of the regular AudioNodeProcessor is desirable

#

and I’m not sure what the downside is of having both

ionic sedge
#

mm, API fragmentation

obsidian tusk
#

but yeah, it’s definitely possible

ionic sedge
#

My understanding was that BillyDM was ready to forge ahead and break up the blocks like this.

#

but i might be wrong

obsidian tusk
#

I mean, after seeing the benchmark results I’m not totally against that

ionic sedge
#

Yeah, again the baseline performance (when no subblocks were created for the SimpleAudioNodes, i.e. just one event per frame) was exactly the same as far as I could tell.

obsidian tusk
#

I don’t think it’s fragmentation really though, or if it is it’s not that terrible. It’s a fairly common pattern to have a "simple" and "advanced" API with the former being implemented in terms of the latter

#

but I guess AudioNodeProcessor could be kept but made private and we can consider our options later

static quest
#

I've been feeling sick the past couple days, so I'll get back to this later

ionic sedge
#

Here's a couple node ideas I had. I'll write them down here so I don't forget 🤣

  1. LUFs node

There's a crate that looks really nice for perceptual loudness analysis: https://docs.rs/ebur128/0.1.10/ebur128/index.html. It would be super easy to wrap it up in a node. It might actually be craaaazy for mixing productivity, especially when we can give it a GUI.

  1. ITD (interaural time difference) node

One of the ways we localize sound is through the time gap between a sound arriving at each ear. The gap is pretty small (a maximum of ~650μs on average, or ~30 samples at 44.1k), so we could easily create a pretty performant node whose sole purpose is to produce this gap. It's still not full-blown HRTF or anything, but you could compose it with Firewheel's existing spatial node to get a slightly more convincing effect.

ionic sedge
#

(It's actually kind of awesome that we could so easily compose more sophisticated spatialization with combinations of nodes. You could mix and match which effects you want depending on performance requirements or artistic goals.)

ionic sedge
#

Oh yeah almost forgot (reminded by some fireworks) -- I'd also love a component that properly delays a sample's start time based on its distance. That would be super easy.

obsidian tusk
#

Oh yeah, fantastic idea, that’d be super useful

viscid plank
#

🌩️

static quest
#

I'm feeling better now, so I can work on this again.

static quest
#

Also yeah, I love the idea of a LUFs node.

obsidian tusk
#

If it can wait a couple more days I can make the final few changes, my mother in law’s here right now so I don’t have much time after work and what I have I’ve been spending on my quake engine

static quest
#

We could also add an "RMS" node. It's not as accurate as LUFs for measuring loudness, but it is a lot faster to compute.

ionic sedge
# ionic sedge Okay so.... the baseline performance with the new changes is essentially identic...

Yes, at least with some foreknowledge of the parameters. That's where I was gathering these performance metrics from that I'm replying to.

In other words, I knew the shape of the volume object, so I just manually interpolated its volume like this:

    for i in 0..steps {
        let elapsed = (previous_elapsed + time_step * i as f32) / total_duration;

        // we'll break up lerp into a bunch of steps
        let baseline = chain.fx_chain.spatial_basic;
        chain.fx_chain.spatial_basic.volume =
            Volume::Linear(fade.event.start.lerp(fade.event.end, elapsed));

        let instant = now + DurationSeconds(time_step as f64 * i as f64);

        chain.fx_chain.spatial_basic.diff(
            &baseline,
            Default::default(),
            &mut queue.with_time(firewheel::clock::EventInstant::Seconds(instant)),
        );
    }

You can see that this isn't the most efficient way to generate these events -- we're diffing the whole struct every time even though we know we're only changing one parameter. Obviously for volume, that doesn't matter (only one parameter), but we could make it nicer for larger sets of parameters.

I think that would look like some small addition to the Diff trait that allows us to break up diffs into chunks. However, I'm fairly certain that can be almost entirely separate from whatever changes need to happen elsewhere in firewheel-core.

#

(Oh I actually already implemented the LUFs node -- it's on an in-progress branch in bevy_seedling.)

#

Hm, actually maybe we don't even need to change the trait?

The events are a big enum. And it's pretty obvious which events can be interpolated and which can't (excluding type-erased ones, which I think it's probably fine to leave out).

Anyway, point is -- there are many ways we can manage the animation aspect, and I don't see any blockers with regards to how it fits in the ECS. For users who don't care about scheduled events, literally nothing would change!

static quest
#

I can fork your PR so that you'll still get credit for the changes you contributed.

ionic sedge
#

(that seems like a good idea tbh!)

static quest
#

Actually working on this, I'm thinking that double-boxing ParamData::Any might not be as necessary anymore. The addition of timing information already increases the size of NodeEvent by 16 bytes, so reducing the final size of 60 to 52 by double-boxing doesn't seem as drastic.

#

Although actually, if it's not too much trouble, it would be nice if we could instead use ArcGc<Box<T>> or ArcGc<T>. Now that the event management is more complicated on the Firewheel processor's side, it would make it a lot easier if these types could be automatically garbage collected.

ionic sedge
#

Another thing we could do is

pub enum ParamPath {
    Single(u32),
    Multi(ArcGc<SmallVec<[u32; 4]>>),
}

->

pub enum ParamPath {
    Single(u32),
    Multi(ArcGc<[u32]>),
}
#

idk if we could do both simultaneously

obsidian tusk
ionic sedge
#

Actually I think this doesn't change the total size

static quest
ionic sedge
static quest
#

Yeah it doesn't appear to change the size.

ionic sedge
#

so that's convenient

static quest
#

Hmm, it turns out that ParamData::Any(ArcGc<dyn Any + Send + Sync>) makes the generics in the diffing and patching quite complicated. I might just make it ParamData::Any(ArcGc<Box<dyn Any + Send + Sync>>).

ionic sedge
#

Oh does it?

static quest
#

Yeah, I can't figure out how to go from, say, an Option<T> to an ArcGc<dyn Any + Send + Sync>.

#

Hmm, maybe if there was a way to convert a Box to and Arc.

ionic sedge
static quest
#

This seems to work, but it's a bit complicated:

ArcGc::new_unsized(|| {
    let a: Arc<dyn Any + Send + Sync + 'static> = Arc::new(value.clone());
    a
})
#

In leaf.rs, I'm getting the error the trait bound ArcGc<(dyn Any + Send + Sync + 'static)>: From<Option<T>> is not satisfied

#

Adding this seems to fix the error:

impl<T: Clone + Send + Sync + 'static> From<Option<T>> for ArcGc<dyn Any + Send + Sync + 'static> {
    fn from(value: Option<T>) -> Self {
        ArcGc::new_unsized(|| {
            let a: Arc<dyn Any + Send + Sync + 'static> = Arc::new(value.clone());
            a
        })
    }
}
ionic sedge
#

Hm, that's interesting. I didn't even know we has that impl

#

I don't think you need to clone though, do you?

#

since it takes it by value

static quest
#

It's because I think From<Option<T>> is automatically implemented from Box<dyn Any>, so it worked before.

ionic sedge
#

(But yeah you shouldn't need the Clone bound unless I'm mistaken.)

static quest
#

Although we only use it in one place, so I could probably just construct the ArcGc there instead of implementing From<Option<T>>.

#

Actually, I can add this method for ArcGc:

impl ArcGc<dyn Any + Send + Sync + 'static> {
    pub fn new_any<T: Sized + Any + Send + Sync + Clone + 'static>(value: T) -> Self {
        ArcGc::new_unsized(|| {
            let a: Arc<dyn Any + Send + Sync + 'static> = Arc::new(value.clone());
            a
        })
    }
}
ionic sedge
#

(Yes, although no need for the clone XD)

#

((also Sized is one of them auto trait bounds so you don't need to specify it))

static quest
#

Oh you're right, I can remove the clone.

static quest
#

I also realized we probably need to add a way to do realtime-safe logging. Though we can do that in a future version.

ionic sedge
#

ya that would be nice at some point

ionic sedge
#

Okay so without filtering, turns out ITD is pretty subtle 😅
I can definitely localize sounds with merely ITD (which is pretty cool -- I've never heard it demonstrated in isolation before), but the effect is dwarfed by volume adjustment.

#

but idk maybe it works better in context with more sounds blobshrug

ionic sedge
#

Oh also, as I was working through this, I realized it's a little unconventional that SpatialBasicNode doesn't downmix the input to mono, then spatialized the mono signal.

Conceptually, it doesn't really make sense to merely attenuate the left and right channels of a stereo signal to spatialize it. That's essentially mixing two very different paradigms. It becomes particularly strange when using ITD -- you'd be introducing phase relationships that don't necessarily respect how things actually sound.

Now, there are more ways to manage this disconnect than simply downmixing. They tend to be a fair bit more complicated though, so I wouldn't want to mess with them right now.

rodio, for references, downmixes inputs to its spatial node before spatializing.

Of course, Firewheel is very flexible, so we could leave it as-is and allow users to decide how to manage inputs to the SpatialBasicNode. The new ItdNode in bevy_seedling does this downmixing, so if it precedes the SpatialBasicNode in a processing chain, then it all works out.

#

Another solution would be treating the left and right channels as individual emitters, and allowing them to have some amount of separation.

#

(Where a lot of sounds could just have a separation of zero.)

static quest
#

Ah yeah, you're right. I didn't think about that when making the SpatialBasicNode.

#

It would be pretty simple to add an option to the config of SpatialBasicNode.

ionic sedge
#

ya that probably makes sense

static quest
#

It could even be a parameter, if for example you wanted to downmix one track but keep the stereo separation for a different one.

#

Oh yeah, I also need to add a bitfield similar to SilenceMask to let the plugin know which channels are actually connected.

ionic sedge
#

which for certain domains is very useful/flexible

static quest
#

I also think I found a way to avoid having to allocate an event buffer for each node. Taking advantage of the fact that events for a node are often clumped together, I can just store the starting index for each clump in an ArrayVec (and then just do a linear search if that fills up).

obsidian tusk
#

Hey BillyDM, I just read your two DAW GUI articles, really interesting stuff and I’m happy I dug into it! Funny that when I got to the end of your second one, you mentioned a third-party dev using Flutter for the UI, did you decide against that in the end? I’m interested to know if you had specific problems since (in case it wasn’t obvious lmao) that’s my current preference for my own project and I don’t want to go too far down that rabbit hole if it’s a bad idea

trim belfry
ionic sedge
#

yeah eventually we'll have animations to make this easy

#

but replicating that functionality should be no problem -- is it the fade_towards that you're missing?

trim belfry
#

I guess so. Maybe I can just copy it to a local trait or something for the time being 😄
Or even add a PR to seedling?

ionic sedge
#

(Assuming a fade_towards method, it would look something like this:)

fn fade_in(
    mut commands: Commands,
    players: Query<(Entity, &SampleEffects), With<FadeIn>>,
    mut volume: Query<&mut VolumeNode>,
    time: Res<Time>,
) -> Result {
    for (entity, effects) in players.iter_mut() {
        let mut volume = volume.get_effect_mut(effects)?;

        volume.volume = volume
          .volume
          .fade_towards(Volume::Linear(1.0), time.delta_secs() / FADE_TIME);

        if volume.volume.linear() >= 1.0 {
            volume.volume = Volume::Linear(1.0);
            commands.entity(entity).remove::<FadeIn>();
        }
    }

    Ok(())
}
#

Volume is actually from Firewheel, so I think you'd want a PR there. @static quest might have some thoughts about it

trim belfry
#

Ok, that's enough to get me going, thanks!

obsidian tusk
#

Hey Corvus, are you planning to release a new version of seedling soon? I’d like to use the limiter from my PR and I’m wondering if I should wait until a new release or just switch to using a git dependency

ionic sedge
#

yeah i was planning on a release relatively soon, although I think at this point I'll wait on @static quest's changes

#

so it might be a week or two before that actually happens

obsidian tusk
#

Makes sense to me! If I get back to working on my quake engine before then I’ll just switch to the git release then, I’m doing a big cleanup

static quest
# obsidian tusk Hey BillyDM, I just read your two DAW GUI articles, really interesting stuff and...

To be honest, I'm still not completely decided on the frontend framework.

At the time I discarded Flutter as an option because it didn't support "pointer locking", but since then there has been a 3rd party plugin made that does this https://github.com/helgoboss/pointer_lock. But the main thing is that creating a bridge between Rust and Dart is quite a pain. That and I've heard some mixed feedback that Flutter's performance might not scale well to large applications. I can't say for sure, but it is a concern of mine.

Since writing that article, Vizia (https://github.com/vizia/vizia) has gotten quite a bit better, and I did make some progress on it. But the main problems are the bus factor of one, less features than the big frameworks, and it still has some quirks that haven't been ironed out yet.

I have also been looking into Qt, and more specifically Kirigami (https://develop.kde.org/frameworks/kirigami/). Apparently they actually have pretty decent Rust bindings, and I'm also a big fan of their human interface guidelines https://develop.kde.org/hig/. Though one of the drawbacks is that even though QtQuick (what Kirigami is based on) is declarative, it is not data-driven.

#

Plus I am a KDE user myself.

ionic sedge
#

hear me out...
what if you used Bevy? ferrisExplode

...in reality it's not ready yet, but it could become viable in a year or so

static quest
#

Well, I need a lot of features that are specific to desktop applications:

  • multi-window support
  • well-integrated accessibility and localization features
  • good support for dockable panels
  • robust drop-down menus (this is a lot harder than you might think if you want proper drop menu nesting and scrolling for large drop downs)
  • efficient UI updates (the UI engine can't just rebuild and redraw everything every frame)
  • declarative data-driven API that gives clear separation between frontend and backend logic
#

Unless I'm mistaken, bevy's UI is "immediate mode", meaning it rebuilds the whole widget tree and redraws the whole UI every frame where there is an update.

ionic sedge
#

nah it's already retained actually

static quest
#

Oh interesting.

#

Still, I don't want to have to implement all of the complex widgets myself.

ionic sedge
#

yeah that's a work in progress right now, with some core widgets being landed for 0.17

#

but we'll still be waiting on a few until proper reactivity happens

#

Technically Bevy hits a lot of those bullet points already. It's just that the sum total isn't quite there yet, as well as some critical features like reactivity and stuff. But that's why I think it might just be great for this sort of thing in a year or so.

obsidian tusk
# ionic sedge hear me out... what if you used Bevy? <a:ferrisExplode:1311504831243882618> .....

I was saying this before, I think BillyDM was dead on when we talked about this before saying that there are too many UI-specific features that Bevy's missing but I totally agree that the core of Bevy fits a high-performance custom UI really well. I think I could see a future UI library for Bevy being best-in-class among all game engines, the core (esp the way the ECS and asset system work) feels like it fits it well

obsidian tusk
static quest
#

Yeah, I'll keep it in mind. And because I'm separating the frontend and the backend logic, it shouldn't bee too hard in theory to switch frameworks if I find the need to.

#

Oh, right, people are in the process of making an editor for Bevy using Bevy's UI. That does make it more promising in terms of making desktop applications.

obsidian tusk
#

That’ll be amazing dogfooding

obsidian tusk
# static quest To be honest, I'm still not completely decided on the frontend framework. At th...

Fair enough, I remember those two being an issue before. I wrote a library that used protobuf under the hood but ultimately looked very seamless in practice to solve that but Flutter's been putting a few more things in for the automotive industry related to directly embedding it in other programs. I think that was always just the rendering and stuff though, I don’t think I ever saw something about directly manipulating values in the runtime like you’d do when embedding e.g lua or whatever

obsidian tusk
#

Anyway thanks for the update, even with what I’m saying about Rust making porting between frameworks easier you still haven’t scared me off using Flutter for mine just yet (whenever I get around to it)

static quest
#

@obsidian tusk Looking into Flutter more, one dealbreaker I found for me is the lack of good multi-window support. There are ways around it, but they are unofficial and kind of hacky.

cedar bison
#

I think there is something wrong on linux, using: bevy_seedling = { version ="0.4", features = ["mp3", "flac"] } OS: Linux (latest) Bevy: 0.16.1 Using dyn lib for Bevy, development/debug build by the dependencies are optimized.

Issue: application ~freezes (very very slow until it gets killed by the OS)

Output
INFO firewheel_cpal: Attempting to start CPAL audio stream...
INFO firewheel_cpal: Starting output audio stream with device "default" with configuration StreamConfig { channels: 2, sample_rate: SampleRate(44100), buffer_size: Fixed(1024) }

Reproduce: start the application, load bevy_seedling plugin, do not load/play any sound, leave the application open/idle (e.g. on a second monitor). After ~13 minutes output gets spammed after which the application (and whole pc) becomes very slow (<1 fps) rather quickly. Only solution is to close/kill it. This line is printed over and over again: ALSA lib pcm.c:8772:(snd_pcm_recover) underrun occurred (in the Visual Studio Code terminal)

Known issue or configuration issue on my end? Some more information i can provide/test?

obsidian tusk
obsidian tusk
ionic sedge
cedar bison
# obsidian tusk What audio driver do you use? e.g some combination of pulse, jack, pipewire and/...

Seems to be pipewire 1.4.6

In case it helps: this is the audio device (headphones connected)
0e:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Radeon High Definition Audio Controller [Rembrandt/Strix] Subsystem: ASUSTeK Computer Inc. Device 8877 Flags: bus master, fast devsel, latency 0, IRQ 105, IOMMU group 27 Memory at f6a80000 (32-bit, non-prefetchable) [size=16K] Capabilities: [48] Vendor Specific Information: Len=08 <?> Capabilities: [50] Power Management version 3 Capabilities: [64] Express Legacy Endpoint, IntMsgNum 0 Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+ Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [2a0] Access Control Services Kernel driver in use: snd_hda_intel

(i do not have any audio/performance problems on other applications: browser, music player, ...)

cedar bison
ionic sedge
#

hm, yeah that should do it
but it sounds like there may be an unexpected leak or something

ionic sedge
#

I suppose for now, feel free to make an issue on the bevy_seedling repo until we figure out a little more about it.

obsidian tusk
cedar bison
#

I tried to reproduce it with the examples of bevy_seedling (on master), and could not. Tried with one_shot example and removed the system. Tried also with adding the bevy default plugins, in both cases it went +15 min without any issues. So most like it is my bevy application itself, will be fun to debug. Will report back if i find something interesting (and time to debug).

obsidian tusk
#

Huh, that’s fascinating. Are you maybe still using the default Bevy audio system in addition to seedling?

cedar bison
#

No, disabled all bevy default features and bevy_audio is commented out (except if some other dependency enables it).

obsidian tusk
#

Disabling the features isn’t the most-reliable way to do it because other dependencies can re-enable the features (actually, I think Corvus should probably change that recommendation in the readme). Here’s what I do in my project https://github.com/eira-fransham/seismon/blob/0ba6b2cbb02931bc65d9fd2dcd03062f35f8ec26/src/bin/quake-client/main.rs#L238

GitHub
seismon/src/bin/quake-client/main.rs at 0ba6b2cbb02931bc65d9fd2dcd0...

A total rewrite of the now-abandoned Richter Quake engine, splitting it up into Bevy components that can be independently used. Work-in-progress but is confirmed to be able to load and play demos f...

cedar bison
#

Oh, did not know that; TIL (bit ironic i have to enable bevy_audio to be able to access the plugin type to disable it + it got me prelude ambiguity for PlaybackSettings).

Well i tried it, went afk, and came back to a frozen pc. I have confirmed it is a memory leak, ~50mb/s. So i ran again with bevy_audio and bevy_seedling disabled (not including the package and disabling the bevy audio feature) and it still leaks, so yea nothing to do with bevy_seedling not sure why my initial test (before i posted the initial message) ran fine (+15 min) when i disabled bevy_seedling in my app. This also explains why i could not reproduce it with the bevy_seedling example.

Summary: not related to bevy_seedling, apologies for the noise, and thanks for the suggestions!

obsidian tusk
#

Maybe try valgrind? It’s good for diagnosing memory leaks

#

(If you’re on linux/macOS)

ionic sedge
obsidian tusk
#

Ah, yeah that’s a good point. Might be worth mentioning both options with their upsides and downsides though just in case

static quest
#

@ionic sedge Is the new event timing system working well for bevy_seedling? If so, should I go ahead and push another beta release?

static quest
#

Actually, I've thought of one more thing for the sampler node. I think it would make more sense and also be more deterministic to have the playhead be part of PlaybackState::Play instead of its own parameter.

ionic sedge
# static quest <@164224139316428800> Is the new event timing system working well for `bevy_seed...

Well I was hoping to get something quick and dirty going for animations, but the more I got into it, I just couldn't resist making something a bit more substantial. So that's taken some of my time away.

My primary reservation with the new approach is that patches are taken by value now rather than by reference. This means that we can't efficiently update the baseline structs after diffing them on the main thread -- we instead have to rely on cloning the latest value into the baseline. (Just for clarity on what I mean, this is how it's down now in bevy_seedling.)

Is that a problem? I dunno, maybe. For small objects, cloning will probably be faster, but it wouldn't scale as well as patching for large objects.

#

I've had no issues with actually scheduling things -- I haven't written a fancy API for that just yet, but it'll be no problem.

static quest
#

I could change it back to patching by reference if that will help.

ionic sedge
# static quest Oh, you're using the patching stuff directly in `bevy_seedling`? I thought we we...

Yeah so to clarify on this usage:

In the ECS, we store the paramters and their baseline on an entity. That might look like (pseudocode):

(VolumeNode, Baseline<VolumeNode>)

Users will query for the VolumeNode and make all sorts of changes to it. Then, once per frame, we run through all the VolumeNodes that have changed and diff them against the value in Baseline<VolumeNode> (Baseline is just a newtype wrapper).

After diffing, we need to update the baseline so it matches the normal value. We can just clone the normal one into it, but we did just make fancy fine-grained per-field patches. Why not apply those to the baseline instead?

#

So that's one way they're currently used in the ECS.

I actually do use them elsewhere -- we sometimes want to keep multiple instances of parameters in sync. This allows us to co-locate the parameters so it's easier to query for them. However, for this use case, we just consume the patches anyway, so I didn't have to change it.

static quest
#

Ok, I can change it back to patching by reference. The user can still retrieve owned data from events, they will just have to use core::mem::swap to do it in a realtime-safe way.

ionic sedge
#

Personally, I do like the by-reference approach! Since we have to align the baselines every time we diff, I'd love if we can do it as efficiently as possible.

static quest
#

@ionic sedge Alright, I switched it back to patching by reference!

ionic sedge
#

oh sick lemme give it a whirl real quick

ionic sedge
#

Yeah, definitely checks out from my perspective!

Let me also take a quick look at throwing lots of events at the processor.

ionic sedge
#

Okay so I think it scales fine. I played almost a thousands samples at a time over the course of 30 secons or so, all with animations generating one scheduled event per frame (120hz I think) and... idk, it didn't underflow or anything. Max was ~10ms with a 23ms budget, average of 4.5ms.

The animations would be all bunched up in time (i.e. for each frame, all 1k would be scheduled at the same time), so maybe not the best test for the sorting.

#

But it's probably a somewhat realistic animation scenario -- I don't think most animations will schedule at a resolution finer than 8ms.

ionic sedge
#

I think another thing I'd like to get in Firewheel this round is bevy_reflect. I don't know if you're familiar, but despite the name, it's written as an independent reflection library. We'd probably want *another* feature flag for it, if you can bear it! But it'll be increasingly important for a first-class Bevy experience as we start integrating inspectors and editors into Bevy.

#

Hopefully I can get to that today. The only thing that hindered my progress last time was implementing it for the sample resource trait object.

#

(And to be clear, a few folks have requested the feature already, so there is existing demand.)

ionic sedge
# ionic sedge Hm, wouldn't this make it a lot more annoying to move the playhead around?

To expand on this -- the default position for the playhead in bevy_seedling will always be the 0 when you queue up a sample. This is because, if you don't explicitly provide a playhead position, it will insert the default one. And due to how the sampler pools work, queuing up a new sample for an existing sampler will automatically and immediately align the sampler's parameters with the incoming values.

So my thinking is that putting the playhead parameter inside the playback state enum will be kinda cumbersome. But I could definitely be wrong!

static quest
# ionic sedge Hm, wouldn't this make it a lot more annoying to move the playhead around?

No, you would just send another PlaybackState::Play event to change the playhead.

The problem I've been running in to is that in the sampler engine itself, it doesn't make sense to have a concept of a "playhead" parameter when the sample is paused/stopped. It makes state management more complicated, and it can lead to less deterministic timing (essentially if an "update playhead" event is sent before a "play" event or vice versa, correctly accounting for the delay between those events is very complicated and hard to make deterministic. It would be way easier if the state just had the playhead as part of the play event.

ionic sedge
#

Yeah I suppose that's fine. It's not as if reading it is particularly valuable anyway -- the actual playhead value is stored in the shared state.

static quest
#

Either it's "you're playing a sample from a given position, or nothing is happening". There is no inbetween.

static quest
#

@ionic sedge Alright, I made the change to the sampler node. Now the way you "play a sample in the past" is either by setting the playhead in PlaybackState::Play or by scheduling the PlaybackState::Play event in the past (and if you do both, the delay will correctly be accounted for).

ionic sedge
#

I think that should be easy to enforce with bevy_seedling

static quest
ionic sedge
#

yeah it correlates things nicely

#

btw do you think there's anything wrong with just... scheduling all events by default?

static quest
ionic sedge
#

Well, this would be in Firewheel's userspace. As in, in bevy_seedling, we could attach a timestamp to all changes by default.

#

I think you'd end up with slightly more regular event timings, but... it probably doesn't really matter much.

static quest
#

Hmm, well unscheduled events have less overhead.

#

Oh yeah, and in fact I was planning on adding a feature gate to disable scheduling and/or musical transports if you don't need it.

#

But I'll do that in a later PR.

ionic sedge
#

yeah i wonder if it's enough to matter for most games

#

maybe I'll profile that at some point

static quest
#

I meant more like if you were using the engine for other, non-game applications.

ionic sedge
#

oh yeah for sure

#

I'm just thinking for bevy stuff

obsidian tusk
# static quest Hmm, well unscheduled events have less overhead.

I’m not sure that the difference would be measurable without checking, the difference in performance seems like it would be maybe a branch or two at most since they’re still all going through the same sorting and timestamp-checking mechanisms. I do think unscheduled events should still be an option for simplicity though, most of the time if you’re interacting with firewheel directly you just want an event to be handled asap and don’t really care about scheduling events. That’s more of an advanced feature

ionic sedge
#

Okay I'm most of the way through reflecting types in Firewheel and bevy_seedling. I'll integrate the latest playhead changes later today just to make sure it all feels good.

ionic sedge
#

Okay yeah I really didn't have to change much for the playhead stuff. Constructing the variant with a value is definitely a touch verbose

    commands.spawn((
        SamplePlayer::new(server.load("selfless_courage.ogg")).looping(),
        PlaybackSettings {
            playback: Notify::new(PlaybackState::Play {
                playhead: Some(Playhead::Seconds(6.0)),
            }),
            ..Default::default()
        },
    ));

but I could easily add some shorthand for PlaybackSettings.

#

I think I'm pretty satisfied with where I'm at adding reflect derives too. I had to manually implement everything for Notify to preserve its invariants, which is a little unfortunate, but it definitely works!

#

@static quest feel free to merge your scheduling PR -- I feel like it's in a good place, personally

#

I can either PR the branch or wait until you've merged and PR main.

static quest
#

Ok, I'll merge it!

#

I'll add a feature flag to disable scheduling and musical transports, and then I can publish a new beta release!

ionic sedge
#

On a slightly unrelated note, I had a nice idea for automatically re-initializing nodes when you want to change the configuration.

Basically, we can very easily detect when you've mutated a configuration struct in the ECS. On its own, that doesn't do anything since the configuration is used only when the audio processor is constructed. However, we can just re-create the processor and reinsert it into the graph whenever you make changes to the configuration.

This would be easier for me if we had something like a insert_at method where you could replace an object of a given ID. If it also allows you to avoid re-compiling the graph, it could be decently performant maybe??

I don't know the details of the graph enough to know whether the above makes sense. If not, then I can just remove, insert, and re-route connections, which would be fine.

static quest
ionic sedge
#

Yeah I thought that might be the case

#

I'll just do it in user-space then

static quest
#

Hmm, should I have those features enabled by default?

ionic sedge
static quest
ionic sedge
#

Hm, hard to say exactly. It's pretty easy to add features, and people often don't disable a crate's default features.

#

(And then wonder why they have so many dependencies (ask me how I know).)

ionic sedge
#

If they're adding it, then leaving them disabled by default makes sense to me. If they remove it, then it might be a bit more contentious.

# 
[features]
default = []
# I'd have no problem excluding these by default
transport = []
scheduled-events = []

[features]
# whereas this might be more annoying
default = ["disable-transport", "disable-scheduled-events"]
disable-transport = []
disable-scheduled-events = []
viscid plank
static quest
#

@ionic sedge Merged your PR!

ionic sedge
#

Okay I think it should be ready for a beta release from my end. It'll take me a few more days to polish the features I've added to bevy_seedling, and at that point I'll be able to confidently determine whether there are any last minute features I need.

static quest
#

Cool, I'm working on adding those features, and then I'll publish a beta release!

obsidian tusk
#

Been super cool following this, nice work on the latest version both of you!

static quest
#

Thanks!

static quest
#

@ionic sedge Version 0.6.0-beta.0 is published!

ionic sedge
#

Awesome! I'll try to get things wrapped up here as soon as possible check

static quest
#

Oh yeah, I also need to add an option to use a mono signal in the spatial basic node.

ionic sedge
#

ya it would be easiest if there was an option to basically just downmix all the inputs to one channel

#

i mean we could insert a node before it to do that

#

but that's a little more annoying

#

I figure people would only want to do that if they have specific mixing needs

static quest
#

Well the problem is that currently it only works on stereo signals. It should be easy to fix though, I'll do that after lunch.

static quest
#

@ionic sedge Alright, I've added a "downmix" parameter to the SpatialBasicNode and published a new beta release!

#

I also ended up adding in_connected_mask and out_connected_mask fields to ProcInfo that tells nodes which channels are connected and which ones are disconnected. This way I can have the SpatialBasicNode detect if it should treat the input signal as mono or stereo.

ionic sedge
#

oh nice!

static quest
#

@ionic sedge Hmm, I have thought of one more thing. I'm not sure how I feel about having glam as a dependency. I might make it so that glam is an optional dependency.

ionic sedge
#

I do like that the glam types can just be in the events as-is.

obsidian tusk
ionic sedge
#

well the types in bevy_math are glam types

static quest
obsidian tusk
ionic sedge
#

blobshrug that's probably fine

obsidian tusk
#

On second thoughts maybe it does make sense to have two features, so you don’t need to import all of the bevy stuff

static quest
#

And currently the only node that uses it is the SpatialBasicNode, which I can also add a feature flag to switch between using glam types or just [f32; 3] as the parameter type.

ionic sedge
#

I suppose, but is glam isn't heavy I think. Without default features, it has no other dependencies.

obsidian tusk
ionic sedge
#

in other words, it would be better to simply add a new variant with the feature flag, or otherwise force people to use the custom or bytes one

static quest
#

Hmm, fair point.

obsidian tusk
#

Is it so bad to just enforce using an array, casting to/from it? The spatial node can use the glam types internally if it’s more efficient and you can use into() to convert to an array

#

on the user side

#

and since it's just an implementation detail at that point it’s fine to put the usage of glam behind a feature flag

ionic sedge
#

it is nice to be able to provide Diff and Patch implementations for glam types, but the implementations could be optional and they could write to a [f32; 3]

obsidian tusk
ionic sedge
#

If we don't plan to extensively use glam, then not exposing glam types publically would make long term maintenance potentially easier

static quest
ionic sedge
#

oh yeah that was the original contention

#

We could just use a newtype or something

#

that would be more convenient anyway i think

static quest
ionic sedge
#

struct Position3d([f32; 3]);

static quest
#

Ah, ok.

obsidian tusk
#

You could maybe implement it for any field types that implement Into<[f32;3]> + From<[f32;3]> since that covers p much all Vec3-like types across the ecosystem

#

but I don’t know if the current derive mechanisms support that kind of generic code

static quest
ionic sedge
#

right

obsidian tusk
ionic sedge
#

that's a big reason we initially did it, but a newtype or even a custom Vec3 would also serve a similar role without any of the downsides of depending on glam publicly

obsidian tusk
#

Would users still be allowed to define Diff structs with bevy vec3 fields?

#

So long as the right feature flags are enabled

ionic sedge
#

ya imo we should still have glam be optional and provide implementations for Vec2 and Vec3

obsidian tusk
#

Makes sense to me

ionic sedge
#

and then bevy_seedling would just enable those

obsidian tusk
#

Yep, seems like a good plan

static quest
ionic sedge
#

might not matter in practice

#

but yeah since we can toggle on the glam Diff/Patch implementations, I don't think there will be any regressions for Bevy projects 👍

slim pulsar
#

hey do y'all know what synths typically use if not soundfonts?

#

is it just like raw samples

ionic sedge
#

depends on the synth!

slim pulsar
#

okay I guess, let's say you move a project from one DAW to the next. I imagine...MIDI and a generated soundfont would work

#

I don't know what the standard is in that situation, but I'm not positive DAWs can generate sf2s on the fly

#

I'm not building a DAW, but I am building a networking system to transfer someone's voice with MIDI, so i just didn't know if there's a better system for this, or even a better networking protocol

#

these files are typically tiny though so like the latter is a spitball

ionic sedge
#

Well, sound fonts are just collections of samples, more or less. Quite primitive ones at that, usually.

#

Modern sample-based synthesizers typically use more sophisticated techniques to sound more life-like.

#

Often, for each note in an instrument's range, samplers will have multiple samples for each velocity bin.

#

MPE synths may even do a lot of fancy processing and blending to get good sounds in-between notes.

slim pulsar
#

oooo good to know! gotta look into that

static quest
ionic sedge
#

i can only imagine kontakt is a bear to work with 😅 it just gives me those vibes

static quest
#

I heard that their scripting language is actually quite powerful. It's just a shame that it's proprietary.

obsidian tusk
static quest
#

Ah yeah, there is a distinction between "synthesizers" and "samplers".

ionic sedge
#

well it does depend on your definition of "synth"

#

but yeah I think most people use it to refer to like
non-sampled stuff these days

#

I had some older professors who called the big old keyboards that have a million samples in them synths though

#

so blobshrug

obsidian tusk
slim pulsar
#

there must be some medium of exchange there right? even if it's not guaranteed to be deterministic

static quest
#

It's not open source, but someone in the Rust Audio Discord recently released a really nice soundfont player plugin. https://estrobiologist.gumroad.com/l/sflt

It could be worth reaching out to them to see if they would be open to releasing a headless version, that way it can be loaded into Firewheel as a CLAP node (once we add that feature).

ionic sedge
slim pulsar
#

I mean I have a soundfont player

obsidian tusk
slim pulsar
#

no but I mean the voice itself

#

ah I see

static quest
#

Oh wait, it is open source under GPLv3!

obsidian tusk
ionic sedge
#

midi :3

slim pulsar
#

I think the difference we're talking about is I'm assuming that a synth is separable from the actual instructions that can be used to make a voice, and you're saying they are one and the same

obsidian tusk
#

VST and CLAP have an endpoint to get the current param state as a byte array and then you can load it back into the synth later, people use it for presets. It’s specific to each synth though, you can’t transfer those presets between synths

slim pulsar
#

I understand

static quest
#

CLAP/VST plugins are just a list of f32 parameters with audio and MIDI input/output buffers.

ionic sedge
# ionic sedge midi :3

but to clarify, depending on the complexity of the data you need to share, you can just control the synth parameters via midi too

obsidian tusk
static quest
#

Oh wait, plugins can have their own custom state stored as raw bytes. That's how you can, say, store a location to a sound file for a preset.

slim pulsar
#

I thought that a synthesizer could take some waveform, or some common language to produce a waveform, and to provide modifiers onto those, like additive work with other voices, etc. kinda like how firewheel works

#

but I didn't realize I'm a step ahead

static quest
ionic sedge
#

my understanding is that a big reason for midi 2.0 is to allow synthesizers / processors to enumerate their parameters in a reflection-type approach

#

so at least it would be slightly more accessible

slim pulsar
#

I stole this code

static quest
#

Oh actually, you would probably interested in the OSS (open sound system) spec instead of MIDI since it was designed with networks in mind.

obsidian tusk
#

That’s fine too, the good thing about a soundfont player is that you can express a really wide range of sounds with a single implementation code-wise

static quest
#

I've went ahead and added a PiecewiseTransport type since that was fairly simple to do.

And so now, I think I'm pretty happy with the API of Firewheel! There shouldn't be much if any breaking changes now going forward (unless we notice something major).

#

(Although once negative implementations have been stabilized for a while, we might make use of that for the patching and diffing stuff).

#

Also just released 0.6.2-beta.0

#

Ah shoot, I just realized it would be better to make the musical transport a dynamic trait. I'll just yank it to avoid publishing yet another release.

ionic sedge
#

oh ya that seems like a good idea

static quest
#

Alright, the musical transport is now a dynamic trait!

static quest
#

@ionic sedge Though actually, how do you feel about having a type-erased field in TransportState?

#

I suppose I could have an enum with a Static(StaticTransport) variant and a Custom<ArcGc<dyn MusicalTransport>> variants.

#

Yeah, thinking about this some more, I'm not sure how necessary it would actually be to let the user define custom MusicalTransport types. Once we have the AutomatedTransport type, that would cover all possible use cases.

ionic sedge
#

It may still be nice. If we expect the cost of the dispatching to be low (is it especially hot?), then unless the trait object makes it harder to use, I don't mind it personally.

static quest
#

Well, the way I have it set up, it's only possible to have linear interpolation for tempo. The math is already tricky for linear, I can't imagine what it would be for non-linear.

#

Even the CLAP spec itself only supports linearly automated tempo.

#

I could imagine users wanting to dynamically change the bpm of a static tempo in response to gameplay. Although allocating what is essentially an Arc<f64> probably isn't that expensive.

ionic sedge
#

What are you thinking for the AutomatedTransport?

ionic sedge
static quest
ionic sedge
#

I'd imagine people would want finer animation than the ECS can provide anyway, to be fair.

#

Almost certainly better to handle it more precisely in the MusicalTransport implementation.

static quest
#

Ok, then I guess there is still a bit more work to do. 😅

ionic sedge
#

At the very least, if you keep it as a trait object, it won't be breaking or anything to add another implementor.

#

But I'm sure that can be tackled later either way

#

as folks ask for it

static quest
#

Oh, I just got a great idea. Instead of having the user automate the bpm directly, they can automate a "speed" multiplier that would work for all MusicalTranports.

static quest
#

Hmm, and maybe we don't even need a MusicalTransport type. The user could just schedule a sequence of transport events.

ionic sedge
#

Hm, but would that be able to capture non-linear tempo changes?

static quest
ionic sedge
static quest
#

I already have. There's a delta_bpm field in ProcInfo.

#

The reason why nonlinear tempo automation is difficult is that tempo is the derivative of the playhead position in seconds, and thus automated tempo is a second derivative.

#

For linear automation you can use the kinematic equation pos = speed(t) + 1/2acceleration(t)^2. I suppose you could in theory derive equations for more complicated curves, but that seems like a pain.

#

Add on top of that the fact that keyframes make it a piece-wise function.

#

Essentially it's like you're trying to animate the speed of an animation.

#

Though I suppose we could omit the bpm information in ProcInfo.

static quest
#

@ionic sedge Ok, here's what I've come up with. Instead of making things complicated with interpolated tempo, I decided it would probably be good enough to just support "piecewise" automation for tempo (I doubt anyone could even tell the difference between true linearly interpolated tempo and a piecewise animated tempo updated every beat or so.) I also added a TransportSpeed field to TransportState that lets you set when to update the speed multiplier, or even have a bunch of speed multiplier keyframes. I feel this will give plenty control and precision for games. https://github.com/BillyDM/Firewheel/pull/58

ionic sedge
#

I haven't made a fancy ECS API or anything for the transport, although maybe I should blobthink but either way this seems all good to me from the bevy_seedling perspective.

static quest
#

I just realized it would probably be better to have "instants" for each of the keyframes instead of "durations". I'll fix that after I eat.

#

I also need to figure out how to do a binary search on floats. (Right now it just uses linear search which can be quite inneficient for transports with a lot of keyframes). Any ideas?

ionic sedge
#

Hm, is the issue in sorting them?

static quest
#

No, the issue is in figuring out which keyframe the current time falls in.

ionic sedge
#

Wouldn't it essentially be the same as a normal binary search, but with slightly more care in splitting the set?

static quest
#

Probably. I've just never implemented a binary search manually before. It probably isn't too hard though. I just wondered if there was an easier solution first.

#

The binary search built into Rust's standard library only returns Some for exact matches (though I might be wrong on that).

ionic sedge
#

actually I think you can use the error variant

#

If the value is not found then Result::Err is returned, containing the index where a matching element could be inserted while maintaining sorted order.

static quest
#

Oh, sweet

ionic sedge
#

you'll also have to use binary_search_by to you can manually call total_cmp

static quest
#

I can also just assert that all values are valid finite number with no duplicates in the constructor.

static quest
#

Ok, I've made those changes.

#

Waiting for the CI tests to complete and then I'll publish a new beta release.

#

Oh, I've thought of one more little thing. I want to future-proof adding realtime logging. It won't actually be realtime yet, but it will be there once we do set that up.

ionic sedge
#

ya that'd be super neat

static quest
#

Actually while doing this I realized it was actually pretty simple to implement a realtime-safe logger, so I just went ahead and did it!

static quest
#

And one more thing I've thought of. I thought it would be handy to have a midi message type (using the wmidi crate), so I also went ahead an added a feature flag for that.

#

And now 0.6.3-beta.0 is published!

static quest
ionic sedge
#

Oh nice! Hm, I'm pretty sure bevy_reflect has been made no_std compatible.

#

Let me see if I can find what they do in no_std contexts for that

static quest
#

Yes, bevy_reflect itself is no_std compatible, it's just that that line in firewheel-core uses Cow.

ionic sedge
#

Yeah, that's nearly what the macro produces, but they may export a Cow type for no_std.

#

I just cleaned up the macro output for that part.

#

mm, looks like they have a doc_hidden internal type for it. It's a little ugly, but you should be able to do

bevy_reflect::__macro_exports::alloc_utils::Cow::Borrowed("T")
static quest
#

Oh interesting, thanks for looking into it!

ionic sedge
#

Hopefully we'll be able to clean up that whole implementation eventually. We only need to change a couple lines from the derive macro to preserve Notify's invariants.

static quest
#

Ok, that worked!

#

And with that, I think the API is actually more or less finalized! At least with firewheel-core.

#

I'll go ahead and publish one more beta version.

static quest
#

Oh, apparently I need to use libm to get float functions in no_std.

ionic sedge
#

Yeah a fair amount are std only. If you don't want it to be annoying, you can use num_traits, which provides a float trait that basically provides the exact same methods that std does.

#

(via libm)

static quest
#

Ok, I'll look into that.

static quest
#

@ionic sedge Ok, got it fixed and now 0.6.4-beta.0 is published!

static quest
#

I've also went ahead and made firewheel-nodes and firewheel-graph no_std compatible. I'll wait to publish a release for that though since it doesn't affect much.

obsidian tusk
#

Moving everything to no_std is exciting, it’s making me want to build a little synth with a teensy or something. Does that mean you don’t have a hard requirement to allocate anywhere?

static quest
#

No, alloc is required.

obsidian tusk
#

Ah fair, nvm then 😅 Still nice to support no_std

static quest
#

Yeah, I make extensive use of Arc, Box, Vec, and HeapRingBuf.

#

And also String for the realtime logger.

obsidian tusk
#

Yeah I thought boxed trait objects were a pretty fundamental part of the system and getting rid of allocation would’ve been a big change

ionic sedge
#

a teensy can totally do allocation though

obsidian tusk
#

I mean, it’s probably possible to make allocation optional but I don’t really know why you would

static quest
#

Yeah, implementing an audio graph engine without alloc is a tall order.

ionic sedge
#

If you’re a touch careful with it, you can probably do very reliable processing on microcontrollers.

obsidian tusk
ionic sedge
#

But anyway, this may end up being super helpful for future wasm (which may forgo std) and console support

#

consoles generally don’t provide targets with standard library implementations

static quest
#

True

#

Though one hurdle is that Symphonia isn't no_std compatible (and neither is rubato, at least I think). Though you can replace it with other audio sample loading libraries.

ionic sedge
#

ya we could definitely work around that

#

and anyway if we do start getting serious about console support, we could probably help symphonia work towards no_std

static quest
#

I also just remembered that the sampler pool currently depends on the context provided by firewheel-cpal. I definitely need to fix that.

obsidian tusk
# obsidian tusk I made one ages ago but it’s a complete labyrinth of trait magic

Had a check back and it does allocate in a few places to work around the really early state of generic associated types and type alias impl trait at the time, here’s the repo if you’re interested but it’s an unreadable nightmare and I feel like I wrote it in a fugue state https://github.com/eira-fransham/octahack-rs

GitHub
GitHub - eira-fransham/octahack-rs: A fast, efficient, modular musi...

A fast, efficient, modular music creation toolkit, designed for live performance - eira-fransham/octahack-rs

#

It does work at least

obsidian tusk
ionic sedge
#

Oh also, as it turns out, the Fyrox developer made a pure Rust HRTF implementation a long time ago. Pretty impressive that Fyrox has had it since day one! Anyway, it uses the ircam database, which provides the impulse responses in a very simple format. Ideally we'd support SOFA like Steam Audio and most other modern audio software, but Fyrox's approach would certainly work in the meantime.

I asked the dev if he'd be okay with his hrtf crate ending up in Bevy, and he gave me an approval. I've actually already written an integration for bevy_seedling, so I can include HRTF support in the next release!

#

Neither Fyrox nor sofar handle fast-moving sources all that well, so we'll probably need to invest a little time fixing that up in the future.

static quest
ionic sedge
#

Oh nice! I should update the audio demo with this

ionic sedge
#

Oh one thing I didn't think about until now @static quest is the logger might be problematic for Wasm. At least, by convention we might want to be able to completely cfg out logging. If you so much as sneeze at a string in an audio worklet, it'll throw a JS exception.

ionic sedge
# ionic sedge Oh one thing I didn't think about until now <@714940838781911051> is the logger ...

Okay maybe this is hyperbolic -- I'm not sure exactly where the error comes from. But wasm-bindgen tries to access a JS TextDecoder when handling strings, which isn't available in audio worklets. This happens when the worklet panics, for example, making debugging very hard.

If we're unlucky, it might even be merely constructing a String or manipulating it in normal ways that causes the exception. I haven't found the root cause of it myself. I just send a closure from the audio worklet that formats a string on the main thread if I need info from a worklet.

static quest
ionic sedge
#

i can do some testing with the current API

#

should be able to do that tomorrow

ionic sedge
#

Oh something else that would be really helpful for artists and just people in general is a quick and easy way to profile the audio processing. In fact, it would be amazing if there were a mode breaking it down by how long each processor took. I don't think it could be supported on all platforms (read: Wasm), but yeah I think people would love it. I see people regularly praising the profiling capabilities of middleware.

I don't think it needs to happen right now since it should be possible to introduce without any processor API changes. Maybe I'll make an issue for it.

#

This'll be especially useful as we start to introduce more involved processing like HRTFs, convolution reverb, and so on.

static quest
#

Have you figured out the problem with logging in wasm? Or do I need to disable that feature in wasm?

ionic sedge
#

no hopefully i can get to it here in an hour or two

ionic sedge
#

oh btw @static quest where does the logging go? do I need to activate a feature for it to print somewhere or something?

#

oh nvm i was missing a bevy feature

#

okay apparently it works even with string formatting

#

i wonder if it's the string allocating that causes panicking to fail

#

anyway this is great

#

personally, I would love if the internals of the debug functions are feature gated rather than the whole thing

#

that way, you don't have to also apply #[cfg(debug_assertions)] at the callsite

#

the migration is also a little annoying when a new argument is added to process
(nothing crazy of course, it only took a few seconds to move all my nodes over)

#

would it be crazy to just have one big "context" struct for this sort of thing?

#

technically adding fields to that would also be breaking, but I think it might tend to be less so?
not a big deal either way

static quest
static quest
static quest
ionic sedge
static quest
#

Oh ok, I get what you mean.

ionic sedge
#

there’s a small possibility that the code at the callsite won’t get optimized out occasionally if you do that, but I don’t think that’s worth worrying about personally

static quest
#

I think it's likely it will get optimized out.

ionic sedge
#

ya in almost all cases id expect

static quest
ionic sedge
static quest
#

Possibly. I also don't want to have double-indirection for the common stuff, so I don't think I'll put the input buffers, output buffers, and the event list in the context.

ionic sedge
#

ya id say it makes sense to separate those anyway tbh

#

ah but I see — I guess I was just assuming the buffers would be “flattened” inside the context, and in that case the lifetimes aren’t too crazy (if verbose)

static quest
ionic sedge
#

oh that looks nice!

#

hm, do the processors need a mutable reference to it? (i didn't look at the internals or anything)

#

if you want to get fancy with it, you can also add a #[non_exhaustive] annotations so adding new fields isn't a breaking change

static quest
ionic sedge
#

since it has mutable references inside it too

static quest
#

Yeah, it has to be a mutable reference, otherwise Rust won't let you borrow any of the fields as mutable.

ionic sedge
#

I mean as opposed to passing it by value

#

idk if that makes sense (i.e. if it's stored in a way that makes that annoying, then disregard!)

ionic sedge
static quest
#

Ah, I see. Passing it by value would add an extra 16 bytes to the function arguments. (The size of the function arguments is already 56 bytes).

#

It's probably not that big of a deal though.

ionic sedge
#

ah i see

#

for objects that do use the extras, I'd expect passing by value to be faster (I'm sure it's much more expensive to chase a couple pointers), but for objects that don't, the reference should naturally be faster

#

it would be cool if we had a good end-to-end benchmarking suite

#

that would make it super easy to validate this sort of thing

#

how many registers can the x86 abi use before it has to spill arguments onto the stack?

#

(and aarch64 ig since that’s what i’m on)

#

oh i think it’s 48 bytes

static quest
#

Ah, so either you're an apple user or you're a bleeding edge linux enthusiast. 😁

ionic sedge
#

sadly im away from my linux machine for a while ferris_sob

static quest
#

Ah ok, then making the function arguments smaller probably won't make much of a difference then.

ionic sedge
#

oh i think it’s only 32 bytes on windows

static quest
#

I probably don't event need to make ProcEvents borrowed either.

static quest
#

Oh wait, no there is a way around it.

static quest
#

Or actually, wait a minute. I'm not sure the problem is just a lack of registers. The larger the function arguments, the more memory that has to be copied into the function stack. So it still might be better to try and keep it smaller.

#

I think it's probably fine the way it is then.

#

Oh actually, I can just store the extra stuff directly in the ProcExtra struct. No need to have a struct of references.

ionic sedge
static quest
#

But the majority of nodes probably are not going to use it.

#

Still, we can have the best of both worlds by storing the extra stuff directly in ProcExtra.

ionic sedge
#

ya i think that makes sense

obsidian tusk
#

At least, I believe that Rust uses fastcall by default. I might be misremembering though, it’s been quite a while since I’ve needed to know that info

#

I think that was just a case of misremembered wishful thinking 😅

static quest
#

Alright, I've stored the extra stuff directly in ProcExtra.

#

Oh, I can actually store the fields in ProcEvents directly as well.

#

I'm realizing now that having a struct of references is probably code smell.

obsidian tusk
#

If you want to make the fields generic over either by-ref or by-value (for example, if you don’t want to force the user to spill something to the stack and enforce its lifetime) you can have fields of type T: Borrow<Foo> instead of &'a Foo. No idea if that is useful info here and it can make annotation burden worse (because now you need to annotate explicitly in fn args not just struct fields) but it’s a pattern that’s come in handy a fair bit

static quest
obsidian tusk
#

How do you mean?

static quest
#

I mean that the processor struct already owns all of the fields in the struct of references, so there is no need to create that temporary struct to send them to nodes.

obsidian tusk
#

Ah, can you send a reference to the struct instead? Is that what you mean? Sorry if these are tedious questions, I’m a bit out of the loop

static quest
#

Yeah, I can store the relevant fields in a smaller struct and just pass a reference to that to the nodes.

obsidian tusk
#

Ahhh I see, yeah that makes sense

static quest
#

Oh wait, the ProcEvents struct is defined in a separate crate from firewheel-graph. I suppose it could still be possible without exposing the internals to nodes by doing some std::mem::move shenanigans. However, I also realized that the event buffers would have the same level of indirection whether it was a reference to a slice or a reference to an owned Vec. While I can save one level of indirection to theindices field, the user only interacts with it via drain, so it probably would hardly make a difference.

#

I'm going to call it good as it is right now.

#

@ionic sedge Oh yeah, do I need to disable logging in wasm, or is that feature working?

ionic sedge
static quest
#

Cool.

#

Say, do you know the semver rules for prerelease versions? If I had a version 0.7.0-beta.0 and I created a breaking change in 0.7.0-beta.1, is that breaking semver?

ionic sedge
#

I'm not totally sure to be honest

#

im tempted to say it's not as important for a beta release like that

slim pulsar
#

Just gotta say, y'all are killing it