#bevy_seedling

1 messages · Page 3 of 1

slim pulsar
#

hell yeah, thank you!!

#

@ionic sedge lemme sponsor you

ionic sedge
#

but i still recommend supporting Firewheel for now ;)

Buy Me a Coffee
Billy Messenger

I am an open source developer who is passionate about audio software!My three key projects are Meadowlark (a digital audio workstation), Firewheel (an audio engine for games), and Yarrow (a GUI librar

slim pulsar
#

hehe I am

#

im his #1 fan atm

image.png
#

more people should tbh

#

this is that xkcd

ionic sedge
#

idek what’ll happen post upstreaming if we get there — I guess I’ll start becoming a frequent Bevy contributor since there’s still so much we could add

#

but conveniently Firewheel won’t need to change at all

#

I do think we’d see a huge uptick in contributions though post upstreaming. The modularity of the whole thing makes it so much easier to add functionality.

slim pulsar
static quest
# slim pulsar im his #1 fan atm

Hah, definitely appreciate it! This career of mine has definitely been slow to kick off, but I expected that coming in to this. Once my projects are actually in a usable state, I'm hoping it will be much easier to find avenues to support myself financially. Firewheel isn't even my main project (although I did end up spending a lot more time on it than I was expecting. 😅 )

lapis stone
#

Is it known if seedling support iOS/are there any extra steps for that? Getting everything working across my devices except mobile

ionic sedge
#

I haven't tracked it down yet, but we use the same backend (cpal) as kira and rodio by default, so it shouldn't be anything fundamental.

ionic sedge
#

I think it's just an issue with Firewheel's default cpal config.

lapis stone
#

got it, thank you!

ionic sedge
#

aw man, the ios target triple issue is making it hard to test 😅

lapis stone
#

its so awful lol

#

assuming you have a physical device but if not happy to test anything on my own.

ionic sedge
#

I do actually, although I've never gone through the process of Bevy with mobile devices. It seems like a fix might be coming soon in 0.16.2. But if it's pretty easy to test with a device, I might just give that a go instead of waiting

lapis stone
#

it's somewhat easy, but you need to update to ios dev, which might not be a route you want with a personal device. outside of getting that set up its pretty simple. i was able to get the POC running right after configuring signing

ionic sedge
#

oh boy 😅 well maybe i'll cross my fingers for that patch in a couple days

#

But I'd like to get a patch for this in before the next release either way.

lapis stone
#

I didn't realize you'd hop on so fast, so before that I asked about this in the rust audio discord - sorry to split the conversation! a little bit of extra context, though you probably know this already - but in case it is useful mikedorf replied with this

So it looks like what's happening is that you can't use BufferSize::Fixed on iOS (this is hardcoded in ios.rs in cpal, I'm not sure if it's correct or not, the PR is 5 years old and doesn't have a comment about why that check was added). At the layer above, firewheel-cpal is setting BufferSize:Fixed if the default device returns a range. I think the most obvious fix is updating that check in firewheel-cpal to include one for iOS.

ionic sedge
#

well they got right down to it didn't they haha

#

that makes sense

#

(I hadn't actually diagnosed the issue -- thanks for the context!)

lapis stone
#

@ionic sedge @static quest just patched firewheel with a potential fix

heady robin
#

Just saw the version update, congrats 🎉
I looked at the two new options for better spatical sounds, and my entities sometimes move a big distance on a short amount of time so HrtfNode might not be the best choice
Looking at ItdNode, I see that it has a direction field? Does that mean I have to update it all the time based on listener and emitter world positions?

ionic sedge
static quest
#

Oh yeah, another thing I would like some feedback on is a good default value for parameter smoothing in the Volume, VolumePan, and SpatialBasic nodes. Right now I have it set to 15ms, but I'd like to get some feedback from someone using it in an actual game.

#

Doing some quick math, with the default block size of 1024 samples at a sample rate of 44100, that would be 23.2 ms for each block. It could make sense to bump up the default value to something like 20ms. But of course the downside is that it adds extra perceived latency to changes in volume, so we need to find a sweet spot.

ionic sedge
#

i imagine we might want to tune the default distance attenuation too

lapis stone
#

20ms seems detectable but still rather short. I can't imagine a use case that would need more precision 🤔 ducking audio?

static quest
#

Yeah, I just pulled some numbers out of thin air and thought "this sounds ok". It definitely needs some refinement.

#

(And also keep in mind that the parameter smoothing algorithm is exponential, not linear. So we might need to increase the smoothing time even more to completely remove the stair-stepping effect.)

#

Although I could add a linear smoothing option to the parameter smoothers.

#

Exponential smoothing is just easier to implement.

viscid plank
static quest
#

What were the problems when you tried it with bevy_seedling specifically?

#

Note bevy seedling did recently get an update.

viscid plank
#

Ping @jaunty bloom

static quest
#

Also, right now the built-in filter node in Firewheel/bevy_seedling is very basic. We do need to get that SVF filter node completed.

ionic sedge
#

ya

static quest
#

I think the previous holdup with the filter was trying to design a "generic DSP API" for it. But now that we have decided that it's best to not make Firewheel a generic DSP library, I can just whip up an SVF filter node from the EQ DSP I have lying around.

ionic sedge
#

i will note that unreal actually does have a legitimate, fully-featured node-graph DSP feature called MetaSounds, which i meant to bring up earlier

#

in other words, sound designers for games can benefit from the much more general approach I advocated for way back

static quest
#

Oh, that's cool!

ionic sedge
#

Not suggesting we do that at all, but I was surprised I hadn't heard of it until somewhat recently.

#

or i mean, there's no reason Firewheel itself needs to do all that

#

but i suppose we certainly could in bevy_seedling, or some third-party crate

#

With the new arbitrary scheduling, there's basically no limits ferrisOwO

static quest
#

An SVF filter node would go quite a ways to make that happen regardless.

ionic sedge
#

ya we could definitely use that

static quest
#

I imagine a full on EQ could be useful too, although I'm not sure if that's something that should be part of Firewheel or be in its own crate.

#

Oh yeah, and someone said they were working on a limiter/compressor for Firewheel/Bevy.

ionic sedge
#

not an issue for bevy_seedling either way -- i spun out HRTF processing into its own crate

#

i mean, mostly because i wanted to embed a few IR subjects in the binary so it got kinda big 😅 but it's easy to integrate arbitrary nodes intobevy_seedling

static quest
#

Ok, I'll work on the SVF/EQ nodes once I'm done with this other thing I'm working on. It'll be nice to jump back into pure DSP instead of architecture plumbing stuff.

slow elk
#

If I'm working on my own dynamic music engine, would it be best for overall ecosystem integration that it implements/targets bevy_ssedling?

ionic sedge
ionic sedge
#

The capability for a music engine certainly exists in Firewheel, and I’d expect that integrating it with bevy_seedling would make it super easy to use.

ionic sedge
# viscid plank

I'll note that "sounding good" can be a tricky quality! I'm not sure if it needs to be said, but just in case: there is nothing preventing Firewheel and bevy_seedling from sounding exactly like Audacity, barring a lack of effects of course. But Audacity happens to be open source :3

Look, here's their pink noise algorithm! You could trivially port this to a custom node if you don't like Firewheel's built-in pink noise. Not that this is ideal or anything -- I'd love if everyone can get the sounds they want out-of-the-box.

I am curious what exactly you feel doesn't sound good though @jaunty bloom, and what effects you'd like to have.

#

I decided it would be fun to give it a try myself, but I was shocked that even fundsp doesn't have a compressor! While fundsp and bevy_seedling have a limiter, you really need a compressor to sell the effect.

ionic sedge
#

As an aside, while I'm not the biggest fan of fundsp, we could easily provide an integration for bevy_seedling. It's actually rather elegant ferrisOwO

fn radio() -> impl Bundle {
    let graph = highpass_hz(400.0, 2.0)
            >> bell_hz(1200.0, 4.0, 1.5)
            >> shape(Tanh(16.0))
            >> lowpass_hz(2800.0, 4.0)
            // poor crow's compressor
            >> limiter(0.005, 0.250) * 0.25;
    
    // more processing...

    FundspConfig::new(graph)
}

fn play_sound(mut commands: Commands, server: Res<AssetServer>) {
    commands.spawn((
        SamplePlayer::new(server.load("divine_comedy.ogg")),
        sample_effects![radio()],
    ));
}
static quest
#

Oh, I'm surprised fundsp doesn't have a compressor either.

#

It's probably worth opening an issue for one if one hasn't already been opened.

#

Interesting. Audacity's pink noise algorithm looks more SIMD friendly. I'll have to do some benchmarking to see which is faster.

jaunty bloom
ionic sedge
#

I think we're definitely at the stage where we can heavily prioritize authoring the core set of nodes we'll want in Firewheel. For my part, I'll probably start by placing them in bevy_seedling, since I may not write fully optimized DSP off the bat. A compressor is definitely a high priority for me, personally.

static quest
#

I realized I haven't actually made the FastLowpass, FastHighpass and FastBandpass filter nodes either. I'll whip those up too.

static quest
#

Ok, the fast filter nodes have been added!

#

I also realized that the one pole filter model used for the fast filter nodes and the distance attenuator dsp could be optimized using a lookup table. But I'll save that for a later time.

lapis stone
#

fast filter done fast

static quest
#

Also, what are the use cases for an equalizer in a game engine? The main use of an EQ is of course to balance the frequencies of samples, but I imagine that could also be done by just rendering the samples with the EQ (which would be a lot more performant too.)

#

That could even be done by the asset loader.

ionic sedge
#

For some use cases it's kinda required, i.e. anything involving live synthesis, but I'd argue it's a trade off even for pre-rendered samples. It's vastly easier to dial in a mix if you can just twiddle knobs while it's running, after all.

In fact, we could easily build an asset pre-processor that does offline rendering using a simple Firewheel backend. That way, you could bake your live mix into your processed assets using exactly the same effects chain (if you want extra performance).

static quest
#

While debugging SVF node I discovered that the parameter smoothing algorithm is actually wrong (it doesn't always properly detect when smoothing is finished). I need to fix that too.

static quest
#

Alright, I fixed the issue with parameter smoothing and also finished the SVF node!

#

And I got to say, it's pretty fun playing around with the bandpass mode with pink noise!

#

Although I did add a somewhat breaking change. I realized I could make the rate at which filter coefficients are updated a user-configurable parameter.

lapis stone
#

I think I have stumbled across something strange... I was attempting to take the pink noise gen node from firewheel and add it to seedling. I noticed when I add more than one of the node, I get strange distortion. I went to record my system audio as an example and it is showing it as playing completely fine...? (image above)

Here's what the actual audio sounds like [LOUD] (a la iphone recording).

Any thoughts on what is happening?

(Related code bits: game and node). Sorry if I'm just doing things wrong!

image.png
ionic sedge
static quest
lapis stone
#

i imagine since i am decreasing the amplitude over time that it wouldnt do that? and weird that the actual system audio seems fine

ionic sedge
#

ya i could be wrong, but it doesn't sound like clipped noise to me
although i'd prefer a random initial seed by default -- I've seen fixed seeds cause lots of confusing problems for users in the past, even if the seed is right there and easily modifiable

ionic sedge
#

is there like... feedback going on or something? well, feedback would still show up in a desktop recording

lapis stone
#

yeah, like that audacity recording is what the blown up audio sounds like. it sounds completely fine when recording system audio

ionic sedge
#

well maybe it is a certain level of amplitude that some part of the system or hardware doesn't like

#

have you tried with fully randomized initial seeds?

lapis stone
#

yeah, same issue with random seeds.

static quest
#

And you gave each pink noise generator a different seed, right?

lapis stone
#

I did:

 fn construct_processor(
        &self,
        config: &Self::Configuration,
        cx: ConstructProcessorContext,
    ) -> impl AudioNodeProcessor {
        let seed = rand::rng().random();

        Processor {
            gain: SmoothedParam::new(
                self.volume.amp_clamped(DEFAULT_AMP_EPSILON),
                SmootherConfig {
                    smooth_seconds: self.smooth_seconds,
                    ..Default::default()
                },
                cx.stream_info.sample_rate,
            ),
            params: *self,
            fpd: seed,
            contrib: [0; 5],
            accum: 0,
        }
    }
#

gonna attempt to build on linux and see if I have the same issue

static quest
#

And what volume do you have the pink noise generators at? Noise is loud, and you need to set the volume quite low.

ionic sedge
lapis stone
#

it still seems very strange the way it is erroring (it sounds like the same few samples are being looped over?)

The amplitude is set to 0.4 linearly, but I also decrease the volume over time, so I would expect there to be no interference

obsidian tusk
#

This is dumb but could you try replacing your rng with just rand::thread_rng().random()? It's not technically guaranteed to be RT-safe but I think in practice it should be and it'd rule one thing out

ionic sedge
#

didn't they rename thread_rng to rng?

lapis stone
#

yeah, theyre the same thing

obsidian tusk
#

Oh yeah, sorry. We use an ancient version at work 😅 I had to look up what they renamed gen to because we're not even on the latest Rust edition

lapis stone
#

i would be very surprised if it was constructive rng, it doesn't sound like it to me

lapis stone
obsidian tusk
ionic sedge
#

(it sounds like the same few samples are being looped over?)

One of the new additions in seedling 0.5 is automatic reinsertion when the config struct changes. I could see that reinserting every frame, causing the noise to restart. It shouldn't be able to malfunction like this (I haven't seen it personally), but I suppose it might be possible.

Anyway, your code is fully available, right @lapis stone ? I should be able to take a look later to track it down in case it's some seedling internals.

obsidian tusk
lapis stone
#

yeah, you can pull and run cargo run --example poc and press space to add a player

static quest
#

Hmm, it also kind of sounds like a buffer not correctly being cleared. I'll add an option to force the Firewheel engine to clear all buffers before processing and see if that fixes the issue. If it does, then there is a bug in the audio graph algorithm somewhere.

lapis stone
#

yeah i was thinking it sounds like a buffer thing. like when hl2.exe stops responding

obsidian tusk
#

Like, I agree that it sounds like stale buffers

static quest
#

Also, I'll revert the breaking change I did to the "Spatial Basic" node and save that for a future release.

obsidian tusk
ionic sedge
#

the build logs are complaining about bevy stuff and reflection -- i think we may just need to add a bevy_reflect flag to the optional bevy_ecs dep (sorry for missing that!)

#

or something -- it's kinda weird

lapis stone
#

Just was able to replicate on Linux, so rules out my Mac having any weird setup haha

obsidian tusk
ionic sedge
#

oh i see

obsidian tusk
#

It works when you just build firewheel because it adds the features to everything and features are applied to a crate no matter where in the tree they appear

ionic sedge
#

right

obsidian tusk
#

It'll be the same for all the features that should percolate down the dependency graph

#

I'm actually waiting on something before I can do the task that I actually opened my laptop to do so I'll just put a quick PR in now I guess

static quest
#

@ionic sedge Ok, I've added an option to FirewheelConfig to force clear the buffers. If this fixes the issue, then we know that it is a stale buffer bug.

obsidian tusk
#

I feel like if the issue was stale buffers then it would just make the output silence instead, right?

static quest
#

Well the way the audio graph engine works is that it uses a flag to keep track of which buffers contain silence and which doesn't, so that it knows when it can skip clearing a buffer and when it can skip summing it to the output.

obsidian tusk
#

Ahhhh ok cool

static quest
#

So if a buffer is being incorrectly marked as being silent when it is not, then the buffer won't get cleared.

obsidian tusk
#

Yeah that makes sense

static quest
#

This means node authors also need to be careful when manually setting the silence mask themselves. Which is why I encourage the use of ProcessStatus::ClearAllOutputs and ProcessStatus::outputs_not_silent().

#

I should probably also make that clearer in the docs.

obsidian tusk
#

You need to do:

for output in buffers.outputs {
    for s in output.iter_mut() {
        ...
    }
}
#

Checked and it fixes the issue

static quest
#

Oh whoops, that's my bad then. They just copy-pasted the noise generator nodes from Firewheel.

#

I must of not catched it because I only tested the noise gen nodes with a single channel.

#

Oh wait, no, I have it set to where the noise gen nodes are mono output only.

lapis stone
#

oh yeah that is my issue then, whoops! it gave me a runtime error with Mono since it didnt match outputs so i naively switched it to stereo

static quest
#

I'm not certain the pink noise generator algorithm even works in stereo. You might need to have two separate rngs running in parallel. In which case you could just use two noise gen nodes to get stereo noise.

#

I should probably also make it clearer in the docs that node authors must fill all output buffers completely.

lapis stone
#

that solves the issue then 😅 sorry for to rabbit hole!

static quest
#

It's fine! It led to us figuring out where we need to clarify things in the docs.

static quest
#

Also, it shouldn't be necessary to copy-paste nodes into Bevy. Bevy should just expose the node with a feature flag.

static quest
#

Like this

Screenshot_20250827_124217.png
#

Ok, i published 0.7.2 with the new filter nodes!

#

Oh wait, I need to fix the doc building issue too.

obsidian tusk
#

but I think you’d have to see the issue happen more often before deciding that was necessary

static quest
#

That would make the types significantly more complex though. The borrow checker already makes &mut[&mut[f32]] difficult.

#

And even then, it's not guaranteed that a user will completely fill a buffer even if they borrow it mutably.

#

Ok, I fixed the doc building issue (hopefully).

ionic sedge
ionic sedge
# lapis stone oh yeah that is my issue then, whoops! it gave me a runtime error with Mono sinc...

Ah, that was one of the things I meant to improve for 0.5, but it looks like I left it out. We can easily get at the arity of a node before the point where we connect it. If you don't provide an explicit connection map (which is what happens when you just call connect or even let it automatically connect to the main bus), then we should (imo) automatically downmix/upmix connections if the channels don't match. Possibly emitting a warning at most.

#

(This would all be in bevy_seedling to be clear, no need for any Firewheel changes.)

lapis stone
#

one more q on best practices (though I don't think this is a very common use case): i'm making a little karplus-strong generator, which will need some filtering with the SVF or a LP. Should I be doing something to use the existing nodes already within firewheel, or copying and pasting everything into my processor code?

e.g., I can think of a few ways to model this relationship, but there may be more or less:

  1. Separate nodes on the graph compose the generator, i.e., there is no specific generator, but a nodegraph in firewheel consisting of its components (e.g., pink noise -> envelope -> delay -> feedback attenuation). This seems the most composable, but maybe not user friendly (and I am assuming it's not exactly meant to do this sort of stuff - does it play nice with cyclical graphs?).

  2. Copy and paste the relevant code into my own processor. This seems much more straightforward both from the development and end user POV but loses out on flexibility of the previous solution.

3... something else?

obsidian tusk
static quest
#

Depends on what you want to achieve. If you're looking to create a custom-tailored effect with lots of custom DSP, then doing everything in one node makes more sense (as well as having less processing overhead). But if you want to create reusable components that other game developers can use, then separating them out would make sense.

#

And also keep in mind that nodes can't send events to eachother, so an "envelope" node would only be able to automate the volume of the audio coming in and out.

ionic sedge
#

well you can send the envelope value through an audio channel if you want

#

and then have an "envelope input" in the receiving node

static quest
#

Oh yeah, I guess you could. Typically those are called "CV" (control voltage) ports.

obsidian tusk
#

and an envelope follower wouldn’t need CV input anyway I don’t think, just a sidechain input

static quest
#

Yeah, it depends on what they meant by "envelope".

ionic sedge
lapis stone
#

Cool, I'll check that out! For user friendliness, I was thinking about the end user specifically - for example, lets say I have a graph that works quite well for generating impact sounds. It may be unexpected that instead of some ImpactGenNode, they instead must reconstruct a graph of discrete components. However, I imagine this can be mostly resolved with required components or builders if super necessary.

ionic sedge
#

i'd definitely love to make it super easy to work with a chain or subgraph -- it's quite common in node graph systems after all

#

like ideally we'd be able to "pretend" some entity that actually contains a chain or subgraph is just one node

#

so you can connect to its inputs and outputs like normal

lapis stone
#

that would definitely be ideal! i can also see some usefulness in exposing more abstract parameters (e.g. "brightness") that control "lower-level" subgraph components.

#

though i have to remember we aren't building a modular synthesizer, so that may have limited use haha

ionic sedge
lapis stone
#

mm, as in ensuring the parameters change with sample accuracy across all components?

ionic sedge
#

mm more specifically that they change at exact times specified by the user
for now, I decided to go with automatic scheduling by default for all nodes according to the frame timings, so everything will change in sync

#

That is, when you mutate a node (like VolumeNode), any events it generates are given a timestamp from that frame.

#

But if the user wants something to happen at an exact time (say at exactly halfway though a sample), then they'd want to schedule it manually. But you can only schedule changes to registered nodes, so some proxy parameter like Brightness would inhibit that scheduling.

#

(Not to say that you shouldn't do it! That's just a limitation of meta params like that.)

ionic sedge
ionic sedge
#

but cart envisions a future where you could do something like

fn my_sub_graph() -> impl Scene {
    bsn_list! {
        (
            #Head
            LowPassNode
            Connections [ #BranchOne, #BranchTwo ]
        ),
        (
            #BranchOne
            Distortion
            Connections [ #Tail ]
        ),
        (
            #BranchTwo
            Delay
            Connections [ #Tail ]
        ),
        (
            #Tail
            VolumeNode { volume: Volume::Linear(0.5) }
        ),
    }
}

bsn! {
    SamplePlayer { sample: @"sample.wav" }
    :my_sub_graph
}
static quest
#

Oooo, that's fancy!

ionic sedge
#

what's cool about it too is that when BSN as an asset lands (like foo.bsn files) then you could edit your audio graphs and easily hot reload them

#

and ideally we could have a node graph editor GUI that just works with these BSN assets

lapis stone
#

that would be sick

static quest
#

Sick!

lapis stone
#

would you either accept a PR enabling the core firewheel noise nodes as components? I noticed it doesnt appear to be derived with the bevy feature on pink/white noise at the moment in firewheel but wasn't sure if that was for a reason

tepid vigil
# static quest Ok, i published `0.7.2` with the new filter nodes!

I haven't been able to keep up with all of the discussion (although I try to). Do I understand correctly, that the corrent implementation does not cascade any filters and always uses const generics for channel count? If so, is that just a solution for now or have you guys agreed on keeping it like that?

ionic sedge
ionic sedge
#

a pr would be great!

tepid vigil
static quest
#

And btw the current SVF implementation already supports lowpass and highpass of order 4. Going any higher is probably overkill for a game engine.

#

@ionic sedge I noticed there were a few types that are missing the bevy derives. When exactly is it appropriate to use the component derive and the reflect derive? If you don't mind, you could comb through the codebase and add/remove those derives as needed and then send a PR.

ionic sedge
static quest
#

Well someone sent a PR to enable the Noise nodes as components. And that got me thinking that I might have missed some things somewhere and applied the derives to things that don't really need them.

ionic sedge
#

ya i should be able to do that in the next couple days

#

i'll need to update bevy_seedling soon to fix at least one problematic bug

static quest
#

Thanks!

#

(To be honest I haven't actually delved that deeply into Bevy. I still get some of the concepts confused. 😅 )

#

I'd like to learn someday, but I'm more focused on audio software development than game development.

tepid vigil
ionic sedge
static quest
#

I mean it is possible, we would just need to switch between different channel counts at the top of the processing loop.

#

Does making the channel count a const generic really make the API more difficult to use? If so I can switch it to do that. Though that would introduce a limitation of only having a set maximum number of channels (as well as leading to a lot more dead code being generated in the binary).

tepid vigil
#

I don't mean to bring up the whole discussion again if something was decided already

static quest
#

It's fine. I just personally prefer the const generic approach.

#

Most games will be stereo-only anyway.

tepid vigil
#

I thought the only way to accommodate both approaches would be implementing stuff 2 times. Which sucks, of course

ionic sedge
# static quest Does making the channel count a const generic really make the API more difficult...

It has two main downsides.

  1. For code-driven workflows, we'd have to potentially dynamically register the generic components as they're added. This is possible, but more fiddly, and it would be easy to accidentally query for the wrong channel count.
  2. For asset-driven workflows, it is not possible to register all channel counts ahead of time (or at least, it would be unreasonable to do so).

I would much prefer pushing the annoying-ness onto the processing loop if possible, assuming we can achieve effectively identical performance.

#

wait point 2 isn't quite right

#

it's not possible to register these nodes dynamically with assets, so you must register ahead of time

#

and by assets, I'm really talking about BSN or node graph editors

static quest
#

Ok, so if we did change these nodes to be dynamic, what would be a good maximum channel count? Ideally the lower the better, since each additional channel will lead to more memory usage and more dead code being generated.

tepid vigil
static quest
#

Same goes for the peak meter node which also uses a const generic.

ionic sedge
#

Just allocate the required amount once at node creation

static quest
#

In order for the optimizations to work, the compiler will essentially need to create a separate loop for each channel count.

#

A maximum of 4 channels probably makes sense since 4 channels nicely fits into the ubiquitous 128 bit SIMD register.

ionic sedge
#

that wouldn't play nice with most multichannel formats, no?

#

5.1 or 7.1 would miss out

static quest
#

Yeah, it wouldn't. Which is why I personally prefer the const generic approach.

#

Maybe the only solution is to create two different nodes types. One that is a const generic, and one that is mono or stereo.

ionic sedge
#

Why not just make bespoke loops for the most common channel counts and a catch all for the rest like some of your other nodes?

static quest
#

Though I guess it could be done. It just wouldn't be as nice as clean as const generics.

ionic sedge
#

Yeah that makes sense. It's very much up to your discretion, of course! I would really prefer no const generics in commonly-used nodes -- the combinatorial nature concerns me if we do this for lots of nodes. And since bevy_seedling is tightly coupled with the node representations, I wouldn't be able to build any reasonable abstractions over it to compensate.

static quest
#

And not having auto-vectorization for multi-channels formats would be a significant performance loss for filters compared to the other plugins.

#

It works for volume and panning because those are what is called "embarrassingly parallel". But a filter is serial, meaning the only way to parallelize it is to parallelize it across channels.

#

(Technically there is a really complicated way to parallelize filters by converting it to something called "state space", but that is over my head.)

ionic sedge
#

It's possible we could abstract over a set of pre-registered fixed filters in a dynamic way. i.e. for arbitrary arities, we'd assemble as many fixed nodes as we need to reach the dynamic channel count.

┌────────────────────┐               
│5.1 Sampler         │               
└┬──────────────────┬┘               
┌▽────────────────┐┌▽───────────────┐
│FourChannelFilter││TwoChannelFilter│
└┬────────────────┘└┬───────────────┘
┌▽──────────────────▽┐               
│5.1 Output          │               
└────────────────────┘

I'll note that this is not guaranteed to work out in bevy_seedling. It could turn out that the abstraction is too fiddly to be reasonable. But this idea of subgraphs (which this composition of four channel + two channel filters represents) is very promising in general, so I'll likely give it a fair shot either way.

#

and in bevy it would just look like

commands
    .spawn(five_one_sampler())
    .chain_node((
        Svf::default(),    // dynamic proxy node
        SvfConfig::new(6), // with six total channels
    ));
tepid vigil
tepid vigil
ionic sedge
tepid vigil
#

Definitely! That's why I asked in the first place, wanted to know what the final decision was, because I know you care very much about all use cases

ionic sedge
#

The main thing that gets more difficult is scheduling. I could easily hack it in (I've already done it actually 😅 ) such that a proxy type produces patches that the real nodes can use. But maybe I can generalize even that.

There's also the indirection inolved in the subgraph, but that's kinda inevitable, so I'm not too worried about that.

tepid vigil
#

Why does scheduling get more difficult?

ionic sedge
# tepid vigil Why does scheduling get more difficult?

Specifically user-defined scheduling. Arbitrary, user-defined scheduling looks something like this:

fn scheduling(
    lpf: Single<(&LowPassNode, &mut AudioEvents)>, 
    time: Res<Time<Audio>>,
) {
    let (filter, mut events) = lpf.into_inner();

    // In exactly 2.5 seconds from now, set the filter's cutoff frequency
    // to 250 Hz.
    events.schedule(time.delay(DurationSeconds(2.5)), filter, |filter| {
        filter.frequency = 250.0;
    });
}

The main takeaway here is that the type we're scheduling must implement Firewheel's Diff and Patch traits. So if we wanted to have a proxy type that otherwise appears to be a node itself, it would have to implement Diff and Patch in a perfectly compatible way with the real nodes it proxies.

static quest
#

Oh yeah, a talented dev (who was actually the one who created the audio graph algorithm that I based Firewheel on) has been tinkering around with a version of the SVF filter that is more SIMD-friendly. https://github.com/m-hilgendorf/simd-svf/tree/main

ionic sedge
# ionic sedge Specifically user-defined scheduling. Arbitrary, user-defined scheduling looks s...

Now, it's pretty easy to write helper functions for scheduling / animations in general, even for non-proxied nodes. Here's a simple fade out with VolumeNode:

fn fade_to(main: Single<(&VolumeNode, &mut AudioEvents), With<MainBus>>) {
    let (volume, mut events) = main.into_inner();

    // Fade the main bus to zero, silencing all sound.
    volume.fade_to(Volume::SILENT, DurationSeconds(2.5), &mut events);
}

So there are ways to bridge the gap.

tepid vigil
#

I'm not really familiar with the Diff and Patch traits, but wouldn't you just need to clone them and pass them on to the children? E.g. the FourChannelFilter and the TwoChannelFilter? Or do they have different types?

ionic sedge
tepid vigil
#

Oh I see. Thanks for the explanation 🙂

static quest
#

Ok, let me know if you figure out a good way to use the const generics. If there is not a good way, then we could do the dynamic approach.

#

Or make a separate mono and stereo version of the plugin.

ionic sedge
#

oh in case it's not clear -- the const generics are totally okay if the alternative is just a bespoke mono and stereo version

ionic sedge
slow elk
# ionic sedge The capability for a music engine certainly exists in Firewheel, and I’d expect ...

I mean if nothing else, my music engine would rely on a signal graph (sources and sinks), operating on potentially generic signal types (16-bit? 32-bit? float vs int?), sample rates and buffer sizes, with some processing nodes performing arbitrary operations on signals between source and sink.

I'm still thinking about whether the signal processing should happen at the same rate as the visual framerate (likewise making there be a minimum buffer size), or in a separate loop maybe separate thread with channels using a Resource.

ionic sedge
slow elk
ionic sedge
#

Firewheel operates purely on floats, so that might not satisfy your goal of arbitrary signal types. I'd argue that floats are best suited to the task, to be clear, but you don't have to stick to them of course.

lapis stone
# lapis stone one more q on best practices (though I don't think this is a very common use cas...

hmm i was lightly investigating this a little more and i dont think a fully realized option 1 is possible since firewheel is acyclic (and i think it makes a lot of sense for most purposes). so essentially anything that requires feedback needs to be self contained in a node, which isn't a bad tradeoff

tbh this all kind of just should be a CLAP plugin at the end of the day (and I'll probably shift that direction once that is integrated) but still gonna try in pure firewheel nodes just for fun

slow elk
ionic sedge
#

i might make those nodes for bevy_seedling, actually blobthink

ionic sedge
#

It might seem unnecessary at first, but I think feedback nodes are actually pretty important! It would be a real bummer if you had to throw out your composed effects chain and write everything manually in a single node if you happen to need feedback.

Sure, a single node will be more performant and will have no latency, but I'd prefer that people reach for it only when they have real issues with the built-in feedback.

ionic sedge
#

Well, no I suppose you'd always have delay in the signal, so it couldn't match a real single-sample feedback system.

#

I suppose we could have a little subgraph that's actually just a separate firewheel processer inside of a node, where the buffer size is one frame blobthink obviously leaving a lot of performance on the table, but it would be easy to use.

lapis stone
ionic sedge
#

The downside being that, because the node graph processes in blocks, there's a minimum delay amount proportional to the size of the blocks.

#

Here, FeedbackIn writes to the delay line, and FeedbackOut reads from it.

lapis stone
#

oh, interesting. that diagram does help, thanks! so re: that limitation, e.g. if blocks are 128 samples, you could not have a delay line shorter than 128 samples, right?

ionic sedge
#

ya

lapis stone
#

i think thats probably more than enough for most applications!

ionic sedge
lapis stone
#

I don't think so, but waveguide modeling is a rather minority usecase imo. At 44.1k sampling rate and a 128 sample buffer, the highest tone I could replicate would be around 400hz

#

honestly probably the more useful thing i could do here is look at just implementing a comb filter node. it would be nice to be able to tap into other node processors outside of the node graph though

static quest
#

I will say there is a big difference between making a general purpose audio graph that links plugins together and an audio graph for a modular synthesize or a DSP environment like Reaktor or Bitwig's The Grid. When you get to that level, the approach is generally to JIT compile all the nodes together into a single processing loop. Otherwise you're going to have a ton of overhead.

#

Feedback is hard to do with an audio graph in general.

static quest
ionic sedge
static quest
#

Well the main thing is just plumbing overhead. A composed effect made from a small handful of nodes is probably fine, but if your node gets into the dozens or even hundreds of nodes, that's probably not going to run well even with the silence optimizations.

ionic sedge
static quest
#

Huh, is that true? Well, maybe I am wrong on that.

ionic sedge
#

Reaktor does do aot as far as I understand

#

Faust (a popular music DSL) has a JIT if you can't do AOT as well

#

Max does have its gen~ environment, which is an AOT, sample-by-sample environment.

static quest
#

I guess it wouldn't hurt to do some benchmarking with Firewheel to see how many nodes it can handle.

#

One of my main concerns is cache misses due to nodes being type-erased objects on the heap.

#

(Although if you allocate all your nodes at once it might not be as big of a deal.)

ionic sedge
#

i wonder if a bevy ecs style storage (just dump the bytes into a table, basically) would improve that
you could arrange the nodes according to their access pattern in the compiled graph blobthink

#

but i figure that's amortized over the amount of samples they process per block, anyway

#

to the point where it might be almost negligible

static quest
#

True. We won't know until we benchmark. And I could totally be wildly overestimating the plumbing overhead of Firewheel.

ionic sedge
#

i should probably set up a benchmarking backend (for real this time)
i think i started on that a long time ago, but never wrapped it up

#

should be simple enough anyway

#

i mean in theory we could have a build step that creates a static subgraph
construct_processor returns an impl AudioNodeProcessor, not a trait object

static quest
#

Oh, and the way you do feedback in an acyclic graph is to have a "sender" node and a "receiver" node that internally share an Rc<RefCell<Vec<f32>>> buffer.

ionic sedge
mystic epoch
#

Out of curiosity, if there are no nodes that use rng, is the final result deterministic? If so, are there any caching opportunities at graph or node level? Especially if the user is willing to accept some loss in precision? Is precision less important for quieter sounds than loud ones?

#

I can imagine that it depends a lot on what the nodes do. Is it common for very small changes in input to cause significant output changes? For example an input value of 0.001 vs 0.002, or more broadly any input difference within a threshold (0.001 in this case), perhaps inversely proportional to the loudness, so quiet sounds give a larger threshold

static quest
#

Well, only if you knew up front the timing of every single parameter update (and you had no microphone input).

#

Which is pretty much never.

#

At least for a game. In a DAW it might technically be possible, but it would be pretty difficult to implement seamlessly. It's much easier to just have the user freeze tracks to save CPU.

#

It's like asking if you could cache the pixels that are rendered in a 3D video game. While you technically could, as soon as you move the camera the cached data is useless.

static quest
#

Going further with the GPU analogy, samples are like the textures, the timing of events are like the geometry that the "textures" are rendered on to, and effects are like the shaders.

mystic epoch
#

That helps put things into perspective, thanks! I guess I had two things in mind. Caching and going with the gpu analogy something similar to LOD

#

I can see now why it doesn't make any sense

static quest
#

Oh yeah, I also need to create a delay compensation node. That's really easy to do, so I'll just do that real quick.

ionic sedge
#

oh ya we could have an option to automatically insert those in bevy_seedling maybe

#

according to a node's reported latency

static quest
ionic sedge
#

it can't be too bad with a dag, can it?

static quest
#

What's a dag?

#

Oh, directed acyclic graph

ionic sedge
#

ya

#

might be a neat research project

#

I'd probably at least make an issue in bevy_seedling in case anyone wants to tackle it

static quest
#

Yeah, it's definitely possible. I suppose we could also do this at the audio graph compiler level.

#

I'll just make the node so people have the option to do it manually if they need it.

ionic sedge
#

true
but luckily, it's definitely possible in userspace already in case you didn't want to deal with it

static quest
#

Keep in mind you also need to persist delay compensation nodes across compiles. Or else you will get small dropouts every time you compile the graph.

static quest
#

I also just realized it would probably make sense to have a "reset" command to tell all nodes to clear their buffers. That'll be really simple to add.

static quest
#

Ok, the delay compensation node has been added! It was a little trickier than I thought it would be to get it to work nicely with the silence mask stuff, but I got it figured out.

#

Oh, and on the topic of silence mask stuff, if we are going to support the use of CV (control voltage) ports, then it would make sense to change the silence mask to a "constant mask". That would allow you to quickly check if a buffer has all of the same value for any number, not just zero.

ionic sedge
#

ya i do think that would be useful

static quest
#

It would be a breaking change of course, but I'm fine with doing that for a 0.8 release.

ionic sedge
#

silence = 0 is really just a subset of silence anyway

#

since an audio signal could be silent at any value if it doesn't change

static quest
#

The CLAP plugin spec actually uses the constant mask (and is where I got the idea). I originally didn't plan on Firewheel supporting CV ports, so I thought I would simplify it by making it a silence mask.

ionic sedge
#

oh interesting

static quest
#

(Please excuse the C code in this Holy Rust server 😛 )

lapis stone
slow elk
#

Sure, a lot of the more advanced effects can be turned off automatically (if we time the audio thread programmatically) in lower end computers.

#

Just some food for thought.

ionic sedge
#

I find it exceedingly unlikely that modern computers would perform better doing general audio DSP on 16 or 32 bit integers.

static quest
#

The only reason to do integer or fixed point DSP nowadays is if you are targeting low-end embedded systems.

slow elk
static quest
#

So are you not targeting modern CPUs?

slow elk
#

But if we have an audio thread and a music thread, and they do need to get execution with high frequency (correlated to desired latency, though SFX tends to have lower/stricter latency desireability than music), that means the game thread gets less CPU time, no? I'm not entirely sure how this works.

#

I'm targeting consumer budget PCs that might have been built in 2010.

#

My laptop was built in 2022 by Lenovo and it has soldered 4 GB RAM.

static quest
slow elk
ionic sedge
#

You'd have to be pretty unlucky to end up with the audio thread starving the game threads.

slow elk
static quest
slow elk
ionic sedge
#

depends on the architecture

#

back in the day it would be a math coprocessor

static quest
slow elk
#

Yeah probably

#

What counts as low end hardware?

slow elk
ionic sedge
static quest
#

(And RiscV too)

ionic sedge
#

even microcontrollers typically come with a decent FPU though

slow elk
static quest
#

Yeah true

slow elk
#

What do you feel like a reasonable point?

ionic sedge
slow elk
#

Like at what point should I just come out and say, "sorry son, even I can't support your machine"?

ionic sedge
#

things like convolution reverbs or HRTFs tend to be rather heavy

static quest
#

How could integer math be faster? If all you were doing was adding, subtracting, and multiplying, sure. But what if you want trig functions, square roots, powers, and logarithms?

slow elk
#

HRTF isn't heavy at all on my machine but my machine is uh, middle end (if you don't count the RAM)

ionic sedge
#

i guess it depends on what you mean by "heavy"

slow elk
ionic sedge
slow elk
#

Or use fixed point math

static quest
lapis stone
slow elk
slow elk
#

With the exception of fixed point multiplication and division which actually require two operations.

static quest
#

All I'll say is, do some benchmarking to see if using fixed point math gives a significant improvement over floating point on old machines. Unless it does, then you're just making the audio graph system unnecessarily complicated.

slow elk
#

true

#

I think CPUs could provide fixed point instructions where ECX is the radix/exponent or something

#

but it'd be no different from hardware FP

#

so

static quest
#

I mean, they don't because floating point is better in almost all cases.

#

Floating point is especially good at maximally representing the "normalized" range of -1 to 1 with full accuracy, which is where most DSP lives.

lapis stone
#

side thread: would someone be able to point me in the right direction for getting the peak meter data from within the UI loop? I wanted to play around with some visualization. I am not sure how to get it from around here 🤔. Is there a way to query the node graph? (I might be missing something obvious)

static quest
static quest
# static quest I mean, they don't because floating point is better in almost all cases.

There are a few rare cases where if you can gaurantee none of your intermediary steps will overflow your fixed point number, then fixed point can give you better accuracy. But if you really do need better accuracy, you can just use double precision floats. And even then, typical DSP rarely benefits from the extra precision. Things like proper smoothing, better filters, and antialiasing have a far larger impact on the quality of sound than the tiny amount of noise introduced by precision errors.

#

(And sometimes low precision artifacts are desirable, like in bitcrushing distortion.)

ionic sedge
static quest
ionic sedge
static quest
#

Yeah

ionic sedge
#

well i did a rather ugly job of integrating Fyrox's HRTF crate -- their API isn't the most clean fit with Firewheel's
i'm sure it could be improved a bit by just vendoring the hrtf crate basically

#

i doubt that would speed it up all that much, but it would be easy enough

lapis stone
#

cool I have Very Simple Physical modeling. btw idk if it's useful or not but let me know if any of these nodes would be useful to upstream. right now its just in my experimenting area on codeberg.

ionic sedge
lapis stone
#

yeah the snarl graph is super nice. thankfully BillyDM did all the hard work setting that up with egui so all i had to do was add nodes and fill in wherever enum matches complained haha

ionic sedge
#

And just to dissuade any performance concerns -- this would be so nice even if just to mock up effects. For performance sensitive or frequently used effects, more technical members of the team could trivially translate little subgraphs into bespoke nodes.

I know for a fact sound designers / artists would kill to have a node graph editor that can work in the game, even if it's not recommended for the final product. At least one professor back when I was in school showed us a sound system mockup written in Max MSP, which was then translated by the technical sound designers into the game's final build.

#

oh right it was for End War

ionic sedge
#

which could be part of a third party lib

#

maybe we should make a firewheel-dsp crate or something blobthink

#

or maybe firewheel-synth, which wouldn't be strictly for synthesizers, but probably gives some indication of the more general nature of it

lapis stone
#

i think that makes sense to me! though, on second thought, i bet the firewheel api will still change a lot, so could also be best to keep things separate/very unofficial so there isnt as much of a maintenance burden.

ionic sedge
#

hm, do you think you'd be interested in contirbuting to a seedling_synth crate? We could enable it in bevy_seedling with a synth feature.

lapis stone
#

definitely! i might need a bit of review on my dsp code since I'm still pretty new but that would also be a big help to learning.

ionic sedge
#

to be clear, I think we'd make sure it wouldn't require bevy at all, just like a normal firewheel crate
but we could avoid putting pressure on BillyDM / Firewheel with a difference namespace

lapis stone
#

yep, that makes sense

#

I'm good with either approach, so can defer to you and BillyDM

static quest
#

You might even be able to create a VST3/CLAP plugin with this, given that nih-plug supports egui. Although we would need to add a way for nodes to receive parameter/MIDI events from the audio thread.

#

I also like the purple grid background you added.

static quest
#

The nodes included in the main Firewheel repo are ones that are "essential" for typical game audio.

ionic sedge
#

ya that seems like a good split

heady robin
#

If I insert sample_effects! twice, does it override the previous one?

heady robin
#

Seems like it does

ionic sedge
#

yep! it’s just a relationship like children!

heady robin
#

How would you spawn effects dynamically?
Meaning
I want to use a defaultly spawned sound and then add effects to it conditionally

heady robin
#

Is there a way to umm
Not kill all previous effect nodes?

#

I assume, by adding each new one manually

ionic sedge
#

ya like spawn with EffectOf(target_entity), although that won’t add the effect if it’s already playing

heady robin
#

In my case it's actually for spawning the effect, so it's fine

#

Thanks!

ionic sedge
#

ya that’ll work if you add them in the same frame it’s spawned in

heady robin
#

Should be fine I think

#

Once I add the others manually

#

It works
Thanks for the help

ionic sedge
#

this should improve with BSN

#

i think the plan will be to automatically merge relationship targets

heady robin
ionic sedge
#

where you could do something like

fn basic_sound(asset: &str) -> impl Scene {
    bsn! {
        SamplePlayer { sample: {asset} }
        SampleEffects [
            SpatialBasicNode,
            ItdNode,
            VolumeNode {
                volume: Volume::Linear(volume)
            },
        ]
    }
}

fn fancy_sound(asset: &str) -> impl Scene {
    bsn! {
        :basic_sound(asset)
        SampleEffects [
            FreeverbNode
        ]
    }
}
#

and the SampleEffects targets would merge.

#

idk if this is the exact syntax, but this should illustrate the overall idea

heady robin
#

That's cool actually

lapis stone
heady robin
lapis stone
#

all it really does is exponentially "interpolate" values with a smoothing filter provided by firewheel including release (just interpolating to 0) based on a little internal state that dictates gate on/off behavior

solid vine
lapis stone
vague crow
#

seedling keeps giving me this in the terminal. I think it's because my sounds are .ogg? but I can't find anything about it or how to turn it off.

image.png
ionic sedge
#

it has been addressed, but who knows when they’ll actually publish a new version

solid vine
heady robin
#

I'm considering ways to implement "sound blocking", meaning- adding lowpass and making sounds softer the more walls there are between their sources and the player.
The problem is the volume (as it may change both because the wall count between the entitiy and the player changes and because the of the volume the entity emits).
I'm currently considering two approaches-

  1. Add another component which stores the volume emitted if there are no walls EmittedVolume(f32)
  2. Making the audio source more distant relative to the player the more blocks there are between its source and the player

Which approach, you think, is preferable?

ionic sedge
heady robin
ionic sedge
#

If you have a volume node dedicated to occlusion, the normal one doesn't need any kind of input management. You can just set it freely.

#

But you can, of course, manage that however you like. There's certainly no hard requirement to have another node.

heady robin
#

(nods) alright
I'll see when I get there, hopefully tomorrow
Thanks!

static quest
heady robin
# ionic sedge If you have a volume node dedicated to occlusion, the normal one doesn't need an...

Can you please explain again?
If I understand correctly, for each node I should attach:

  • A lowpass node
  • A volume node that's the actual volume in which it emits sound
  • A volume node that represents the volume of the sound the sound source emits without taking other parameters into account. This one should be (I assume) tagged with a specific component for ease of access, and routed (I'm not sure how? you mentioned occlusions?) to the other volume node
heady robin
ionic sedge
#

But I was just thinking you'd do something like

commands.spawn((
    SamplerPool(OcclusionPool),
    sample_effects![
        VolumeNode::default(), // < primary volume node
        LowPassNode::new(20_000.0),
        VolumeNode::default(), // < very much optional
    ],
));
heady robin
heady robin
ionic sedge
#

ya and then presumably you'd just have a system that reads the blocking factor and writes to the volume and low pass when it changes

heady robin
#

(thinking)

#

Would that be preferable to having a custom node that has the blocking factor as a field and applies volume and lowpass changes by itself?

ionic sedge
#

It's not any less efficient, except for that fact that you'd have two nodes instead of a single custom one. Although you wouldn't be able to just get_effect() on a query for the VolumeNode (if you use the EffectsQuery trait) since there's two, you have to iter_effects().next().unwrap() to get the first one.

heady robin
#

I think I don't use it tbh

#

Yea it won't be a problem, I use volumes: Query<(Entity, &VolumeNode, &EffectOf)>,

ionic sedge
#

hm, although you'd catch both of the volume nodes with that, right?

heady robin
#

unless I tag one

#

I still need to consider which approach to take

#

I think I'll try making the custom node and if it ends up being too complicated I'll try the tagged volume nodes approach

ionic sedge
#

As an aside, this might be another good use case for subgraphs. If you could define a "composite node" that appears to be a single node, but is actually a LowPassNode -> VolumeNode, then you wouldn't have the querying issue.

heady robin
#

Where can I read about subgraphs?

ionic sedge
heady robin
#

Interesting

#

Is there a stable-enough branch I could route my project to to try these out?

ionic sedge
#

No sadly it's just an idea 😅

heady robin
#

haha alright

ionic sedge
#

In a way, sample_effects is just a subset of a true subgraph. It allows you to define a template for an effects chain, which is then cloned around and instantiated in various places. So we might be able to replace it with subgraphs when they come around.

#

A chain can be limiting in certain circumstances, so being able to define a full graph would be nice.

heady robin
#

I have a question about the LowPass filter
What is the "Smoothing" thing?
It seems to act differently based on smoothing, which I assume has something to do with transitions not being too abrupt, but I don't understand what that is exactly

#

Like, what does it do?
Why should you act different based on smoothing?

ionic sedge
#

That's just parameter smoothing -- i.e. how quickly the frequency changes according to events from the ECS. If it just immediately transitioned to the new frequency in the audio processor, you'd likely hear pops or artifacts.

heady robin
#

I see

#

So I should use it for just about any node I guess

#

I already have an interpolator for volume nodes, but I should use it for the new custom one

ionic sedge
#

ya

heady robin
#

I just noticed that the LowPass node sets the frequency of the channels. What that means is that if I have two of them for the same sound source, only the one later in the chain would matter, right?

ionic sedge
#

A "one-pole" filter (like the LowPassNode) has a slope of 6dB per octave. If you have two one-pole filters in series, they become a two-pole filter.

heady robin
#

So when they change the channel frequency it's like
Just the node's step in the processing of the sound
So if there are two in a row, it would happen twice

#

(right?)

ionic sedge
#

ya, in that node the channel frequency is just over its own collection of filters -- there's one stateful filter per audio channel, in other words

heady robin
#

This is really interesting for me to deal with on the programming level
I've been a composer for ten years before I started programming seriously so it's interesting to see whats going on inside the engines

heady robin
#

(sending it here for other people to use if you wish, but I'd also appreciate feedback)

ionic sedge
#

Looks good except that you'll want a low pass filter for every channel -- you can't share the state between the channels.

#

So like

struct BlockedSoundProcessor {
    blocked_sound_factor: SmoothedParam,
    node: BlockedSoundNode,
    low_pass_filter: Vec<LowPassFilter>,
}
heady robin
#

But now I remember it's probably for like
the stereo channels

heady robin
#

It doesn't work 🥹

#

I logged the value I set to the node and the value changes and it's not the same

#

I'll get back to that tomorrow or in a few hours

heady robin
#

It won't let my mind rest haha

#

I set the value this way

image.png
#

I should expect the value setting to appear as a patch in events.drain_patches::<BlockedSoundNode>() I assume

ionic sedge
#

what's the surrounding context?

#

how do you query for it again?

heady robin
#

Hold on I'll send the entire fn

# 
    sound_sample_effects: &SampleEffects,
    blocked_sound_nodes: &mut Query<&mut BlockedSoundNode>,
    sound_transform: &GlobalTransform,
    player_ears_origin: Vec2,
    sound_blockers: &Query<
        (
            Option<&RayCastBlockingBox>,
            Option<&RayCastBlockingCircle>,
            &SoundBlocker,
        ),
        (Or<(With<RayCastBlockingBox>, With<RayCastBlockingCircle>)>,),
    >,
) {
    for sample_effect_entity in sound_sample_effects.iter() {
        if let Ok(mut blocked_sound_node) = blocked_sound_nodes.get_mut(sample_effect_entity) {
            let sound_origin = sound_transform.translation().xy();
            let new_factor =
                sound_blocking_factor_between(sound_origin, player_ears_origin, sound_blockers);
            blocked_sound_node.blocked_sound_factor = new_factor;
            
            //DEBUG
            info!("setting blocked sound factor to {}", new_factor);
        }
    }
}
ionic sedge
#

Hm, ya this should work as expected.

#

This won't solve your problem, but I made the EffectsQuery trait to remove some of this annoying boilerplate!

# 
fn update_blocked_sound_factor(
    sound_sample_effects: &SampleEffects,
    blocked_sound_nodes: &mut Query<&mut BlockedSoundNode>,
    sound_transform: &GlobalTransform,
    player_ears_origin: Vec2,
    sound_blockers: &Query<
        (
            Option<&RayCastBlockingBox>,
            Option<&RayCastBlockingCircle>,
            &SoundBlocker,
        ),
        (Or<(With<RayCastBlockingBox>, With<RayCastBlockingCircle>)>,),
    >,
) {
    if let Ok(mut blocked_sound_node) = blocked_sound_nodes.get_effect_mut(sound_sample_effects) {
        let sound_origin = sound_transform.translation().xy();
        let new_factor =
            sound_blocking_factor_between(sound_origin, player_ears_origin, sound_blockers);
        blocked_sound_node.blocked_sound_factor = new_factor;
        
        //DEBUG
        info!("setting blocked sound factor to {}", new_factor);
    }
}
ionic sedge
heady robin
#

Thanks

#

one sec

heady robin
ionic sedge
#

is it actually changing, as well? it won't send anything if it's set to the same value

heady robin
#

I'm logging these parts of the process fn and will update

image.png
heady robin
#

here's what it logs (which is the correct factor since it's behind one wall)

image.png
#

but if I log the actual value it sets to with the smoothing, it's far from 0.3

#

one sec

ionic sedge
#

Oh, sorry I didn't look very closely. next_smoothed is meant to be called at the sample rate.

heady robin
#

What do you mean?

ionic sedge
#

You're calling next_smoothed once for every buffer of samples, which is anywhere from 500-1000x slower than it's expecting.

heady robin
#

Ohhhhhhh

ionic sedge
#

When the smoothing is active, it should be called once per "frame"

heady robin
#

Ywa it should be within the samples loop

#

I missed that

#

Thanks

ionic sedge
#

You don't have to get fancy with it also -- you can just always call next_smoothed. It's slightly less performant, but it'll still be quite fast overall.

heady robin
#

It will be called for every single spatial node for most of the frames in the game so I do want to be performant haha

ionic sedge
#

it's probably still like orders of magnitude within the acceptable range, but it's still good to be vigilant!

heady robin
#

It works

#

I'm so happy

#

Sending the fixed version here in a bit for those who come after

ionic sedge
#

i think this is pretty much exactly what peak does

#

for player voices

ionic sedge
#

just like, shoot a ray between players and see how many things it hits

heady robin
heady robin
#

Oh you mean like the game?

ionic sedge
#

bideo game

heady robin
#

I thought there was a bevy_peak or a peak_node lmao

ionic sedge
#

we do have a LUFs monitor which can track amplitude peaks actually

heady robin
#

I'm not sure about my calculation though, since it's linear and that's not really how sounds work

ionic sedge
#

that's okay it's just an approximation anyway -- as long as it sounds mostly right, you should be good

#

more physically accurate approximations require a lot more work -- some physical modeling (absorption / reflection properties), ray tracing, etc

lapis stone
#

is it possible to patch both firewheel & seedling to a specific git version? im thinking no since iirc they need to be the same versions listed

ionic sedge
#

i think it depends on what changes are in the new version maybe, but patching just Firewheel will probably work

lapis stone
#

i need to try again but i think last time it gave me a complaint when i tried to register the node, since the version mismatch made the trait type technically not the same and non register-able

#

not really a big deal though in any case

solid vine
lapis stone
#

this is super cool! need to check out the game

#

i've been playing no man's sky a bit recently and its neat guessing where they are using procedural audio. i think theres a talk or interview somewhere. can definitely hear it in the alien voices.

solid vine
lapis stone
#

hmm, am I doing this right? trying to rig up what I had earlier in bevy and idk if im connecting nodes correctly.

Try to accomplish this setup.


PinkNoiseGen ───► Multiply ► Comb ───► SystemOut (Stereo)
                   ▲
                   │
              AdsrEnvelope

...

  let length = diff.length();

  cmd.spawn((
      ...
      AdsrEnvelopeNode::default(),
  ))
  .with_children(|cmd| {
      let adsr = cmd.target_entity();
      let noise = cmd.spawn(PinkNoiseGenNode::default()).id();
      let multiplier = cmd
          .spawn(MathNode::<1> {
              operation: Operation::Multiply,
          })
          .id();

      let freq = 1.0 / (length * AUDIO_SCALE);
      let period = sample_rate.get().get() as f32 / freq;
      let comb = cmd
          .spawn(CombNode::<1> {
              delay: period as u8,
              ..default()
          })
          .id();

      // Attach nodes
      cmd.commands()
          .entity(adsr)
          .connect_with(multiplier, &[(0, 0)]);

      cmd.commands()
          .entity(noise)
          .connect_with(multiplier, &[(0, 1)]);

      cmd.commands()
          .entity(multiplier)
          .connect_with(comb, &[(0, 0)]);

      cmd.commands()
          .entity(comb)
          .connect_with(MainBus, &[(0, 0), (0, 1)]);
  })
  // Trigger the gate in the ADSR
  .observe(on_hit)
  .observe(on_hit_end);
}
lapis stone
#

oh yeah got it working, appears to be that!

lapis stone
lapis stone
ionic sedge
#

I love it!

ionic sedge
#

it would allow you to express it as

let adsr = cmd.target_entity();
let noise = cmd.spawn(PinkNoiseGenNode::default()).id();

let freq = 1.0 / (length * AUDIO_SCALE);
let period = sample_rate.get().get() as f32 / freq;

let multiplier = cmd
    .spawn(MathNode::<1> {
        operation: Operation::Multiply,
    })
    .chain_node(CombNode::<1> {
        delay: period as u8,
        ..default()
    })
    .head();

// Attach nodes
cmd.commands()
    .entity(adsr)
    .connect_with(multiplier, &[(0, 0)]);

cmd.commands()
    .entity(noise)
    .connect_with(multiplier, &[(0, 1)]);
obsidian tusk
obsidian tusk
mystic epoch
ionic sedge
#

(bevy_seedling actually has a limiter on the master by default now!)

obsidian tusk
lapis stone
#

well, karplus strong, if you count that as physical modeling

ionic sedge
obsidian tusk
lapis stone
#

yea it seemed like there was a limiter on it cause it was ducking instead of deafening me

ionic sedge
#

oh ya we could insert the LUFs meter on debug builds

obsidian tusk
lapis stone
#

did the folks at editor dev work through if there would be an audio portion or mixer available? could be cool

#

kind of want to mock that up myself if not

ionic sedge
#

I figure we'll make sure it happens :)

obsidian tusk
#

IMO regular amplitude would probably be more useful than LUFS, I don’t think LUFS is that important for games it’s more about if you’re over-compressing the master by pushing the limiter too hard

ionic sedge
obsidian tusk
ionic sedge
#

I'd argue LUFs are actually quite useful for games -- the various measurements can help you smooth out your levels. But yeah we could work with a simpler meter by default.

obsidian tusk
#

In my experience people just do not understand LUFS 😅 It should 100% be an option though

#

I wish I had some more time for side projects rn because I’d love to make some audio debugging gizmos and I’m itching to do some DSP programming but sadly not

#

Hopefully I’ll still have that itch in a month or two when my other commitments ease up a lil

obsidian tusk
ionic sedge
#

But maybe that's not what most people would want

lapis stone
#

i would love the graph tbh. i think it would be cool to have a more linear arrangement for less tehcnical users too, like just a regular old mixer

#

its all just ui really tho

obsidian tusk
#

I reckon an ideal system would have a linear effect sequence interface like ableton or other DAWs with an advanced mode that lets you edit it as an arbitrary graph

ionic sedge
#

node graphs aren't too scary though tbf, artists use them all the time!

obsidian tusk
#

Bc it needs to be optimised first for people who don’t know jack about audio

ionic sedge
#

and providing an interface that essentially matches the API should really help people understand how the API works

lapis stone
#

oops wrong reply lol

obsidian tusk
ionic sedge
#

i wonder if it would be easy to create a daw-like abstraction over the graph blobthink

#

Creating linear effects chains, mixers, and sends would be very easy. Although displaying non-trivial effects graphs might be really hard 😅

obsidian tusk
#

In my mind it’d be like blender, where it makes a best effort to map the node graph to the simple interface but if it’s too complicated then it'll just link you to the graph view and you have to edit it there

ionic sedge
#

ya if you have some chain or signal flow that's non-linear, we'd probably just give up

lapis stone
obsidian tusk
#

and the simple view would have at minimum simple chains, parallel fx chains (like ableton's fx rack when you have multiple chains, so split-process-merge), simple fx sends and master fx

#

It’d probably be way easier to build that as an MVP interface anyway and at first maybe there isn’t even a node editor, if you want more complex routing you need to do it in code

ionic sedge
#

oh that's funny, I think a node graph UI would be way easier since it doesn't abstract anything

lapis stone
#

i think thats true though it depends on the ui tech i suppose

ionic sedge
#

basically just 1-1 with the entities

obsidian tusk
#

Yeah actually if we’re talking about an mvp without qol features maybe a graph editor would be easier at first

lapis stone
#

since i dont think anyone has built a node graph editor with bevy ui yet

ionic sedge
#

i guess the one thing you'd have to figure out is node positioning when you're displaying something created by code
maybe you could use like a force simulation to get initial placements

obsidian tusk
obsidian tusk
#

It as in a graph editor I mean

#

I deleted a sentence and it made the wording confusing ahaha

ionic sedge
#

There’s also an argument that it teaches people how to use the programmatic API bc the workflow is the same
right, this seems to have some value to me (assuming people don't immediately bounce off)

#

I've built a node graph editor several times at this point, so I'm not too worried. bevy_ui with BSN and reactivity should be a great workflow.

#

i mean, at the very least, we could always display just a mixer that has strips for all the buses

#

that's trivial and still useful for people just starting out

#

We could very easily dynamically insert vu meters on all the buses whenever you pull up the UI.

#

Or maybe we'll just persist them when you have the editor open so you can catch any peaks.

lapis stone
ionic sedge
obsidian tusk
#

Oh you said buses not nodes

#

Still agree 😅

ionic sedge
#

well we can get fancy with it in the node graph editor
we could dynamically insert meters / scopes when you hover over edges, for example, displaying the data in a little pop up window

#

plugdata (a Pd UI wrapper) does this

obsidian tusk
#

If it was up to me I’d make a suuuper basic mvp node editor to get something out there, then build a more DAW-style simple interface on top of that and put more of the qol feature work in the simple interface

#

A nice thing about a simple interface is that you can put more guardrails in place since you can make degenerate states like loops without a feedback node either unexpressable or harder to create

ionic sedge
#

In the eventual upstreamed + bevy editor future, they could even be worked on in parallel! I happen to be more interested in a node graph editor, so I'd probably focus on that. But anyone could contribute to a daw-style interface at the same time.

obsidian tusk
#

V true! Actually another reason to focus in on a graph editor is that the underlying UI is useful in other places too. Materials for example, even potentially for basic logic (not thinking of something like blueprint here, more like how triggers etc work in the source engine)

lapis stone
ionic sedge
slim pulsar
#

do you think this could be useful?

obsidian tusk
slim pulsar
#

main thing was I was looking at the NodeEvent enum, and while downcast reffing a custom event is nice, there's kinda a caveat which is "variant lock in"

obsidian tusk
slim pulsar
#

the generic pollution is absolutely nuts, howeer

ionic sedge
slim pulsar
#

but everything should work like before when I readd the default generics back (which is currently OwnedGc<Box<dyn Any + Send + 'static>>).

oh and another edit: I didn't get rid of NodeEventType, because that seemed to be too much. However, there's probably a good trait to also handle that

obsidian tusk
slim pulsar
#

that's how it works now, and it's fine

ionic sedge
slim pulsar
#

for the most part

obsidian tusk
#

Oh I’m sorry 🙈 I misunderstood the implementation

slim pulsar
#

removing EventType would mean that E must implement a node trait though

ionic sedge
slim pulsar
#

there's nothing wrong with downcasting as I am already doing fwiw

obsidian tusk
slim pulsar
obsidian tusk
#

Yeah for sure, the motivation for the change is good

#

I’m a little suspicious of requiring nodes to annotate the events they’re generic over, that seems like a pretty significant annotation burden when most nodes don’t care about custom events at all

slim pulsar
#

gotta just finish off these last nodes

ionic sedge
#

I can definitely see the motivation! And in a number of ways, this would be very nice for people who extensively use custom or more complicated events.

But my initial impression is that I would be very hesitant to go down this route. The introduction of event generics makes the whole system less composable. All of bevy_seedling's event-related systems would have to be generic over the custom event. And, if I had to guess, I'd also say it probably isn't all that helpful for performance in general. Usually (but not always), events arrive relatively infrequently, maybe just a few per processing block. A few downcasts like this shouldn't be too bad, especially since they're extremely predictable (they'll essentially never fail in practice).

As for ergonomics, I'd love to explore ways to make custom events nicer to use if you feel they're a little annoying right now. I feel like there's a lot of opportunity here as-is.

Again, this is just my initial impression. I haven't considered the design deeply.

obsidian tusk
#

Plus downcasts are incredibly cheap in Rust, it’s just a couple of unconditional jumps and a cmp. At worst it’s a cache miss

slim pulsar
#

it's mainly because I will only really be sending one event, many times through the graph. one sec

#

I agree

#

I'll prolly just keep my fork synced

#

I'll probably have to fork seedling too, but that's really not a problem

#

it's just me adding a generic variant to the enum is all, because that downcast ref is really important in addition to my direct variant

obsidian tusk
#

Hmm, on second thoughts, if we were changing the custom event system anyway it would probably be worthwhile to keep in mind the potential future usecase of nodes sending events to one another and/or sending events to nodes without necessarily needing to know what the node’s concrete type is. I wonder if there’s a way firewheel could expose an interface that lets seedling integrate with Bevy’s event system to send events from the host or between nodes

ionic sedge
#

To expand on the composability, what happens if two separate crates want different custom variants? You'd simply be out of luck, I think. Also, a dependency that wants a generic variant would force users to ensure they propagate that generic event everywhere, potentially changing many of their systems.

Basically, I think it would have a negative effect on Firewheel's overall ecosystem. Kinda like how Rust enforces the orphan rule, even though it would be really useful for individual projects / binaries.

slim pulsar
#

well i.e. if they create like an AudioNode<E>, they'd be generic over E if that makes sense

obsidian tusk
obsidian tusk
slim pulsar
# obsidian tusk but in that case, either you need to choose a monomorphic event type for the top...

this is what my change is

/// An event type associated with an [`AudioNodeProcessor`][crate::node::AudioNodeProcessor].
#[non_exhaustive]
pub enum NodeEventType<E = ()> {
    Param {
        /// Data for a specific parameter.
        data: ParamData,
        /// The path to the parameter.
        path: ParamPath,
    },
    /// Custom event type stored on the heap.
    Custom(OwnedGc<Box<dyn Any + Send + 'static>>),
    /// A user defined event type
    DirectVariant(E),        <---- I added this
    /// Custom event type stored on the stack as raw bytes.
    CustomBytes([u8; 36]),
    #[cfg(feature = "midi_events")]
    MIDI(MidiMessage<'static>),
}
obsidian tusk
#

I’m not sure I understand what the benefit of that is, doesn’t it just give two ways of doing the same thing? Is it just to avoid the Box?

obsidian tusk
slim pulsar
obsidian tusk
#

Ja it's a neat trick and I don’t see people use it enough

slim pulsar
#

but the benefit isn't seen here haha, like you're right. what's the point? and yeah, downstream libraries could choose to define E

#

but that's me! I'm not building for the ecosystem, if I were, I'd have E implement some trait

#

I just don't want to bite off the whole apple, but I would actually discard NodeEventType for a trait object

ionic sedge
# slim pulsar well i.e. if they create like an `AudioNode<E>`, they'd be generic over E if tha...

Hm, maybe I'm missing something. Let me try to illustrate.

Say crate a decides it needs a specific direct variant, A. Another crate, b, needs B. These crates don't know about each other, they just implement Firewheel nodes.

If I want to include nodes from both a and b, it's simply impossible, isn't it? The same graph, which must be generic over a single direct parameter can't support both A and B simultaneously. In other words, it would be impossible to insert a and b nodes into the same graph.

Does that seem right?

slim pulsar
#

but then there's like

#

"wow I'm going to prevent anything I don't care about from working" which obviously is a bad idea

#

and you can't say "hey, trait A is actually my supertrait" from crate B

#

but you can default impl

slim pulsar
#

that scenario is correct

#

but ig that's not what I had in mind haha

obsidian tusk
slim pulsar
#

the biggest issue probably has to be approachability. like there's gotta be a basis of reality somewhere 😆 and I've done a generic sled before that's been a real problem. But I think this for me would be really nice at this point.

obsidian tusk
# slim pulsar but you can default impl

Yeah requiring default impl for basic functionality is usually a sign that you need to rethink the design in my experience, IMO it should only be used for optimisation purposes

slim pulsar
#

so actually looking into this code, there is a bit of use of the DummyNode which makes sense. and I think in the scenario I'm describing, at least the "blessed" traits would probably have automatic impls so things like a VolumeNode, or uhh idk, some oscillator effect would be noop

obsidian tusk
slim pulsar
#

it's like totally fine

#

I've been getting along with firewheel really nicely and bevy seedling for that matter :)

ionic sedge
#

ya it sounds like you want an optimized / more convenient event for your workflow, which makes sense to me

obsidian tusk
#

Oh 😅 I said before that I understood the motivation because I thought it was about making the event types expressed in the AudioNode impl (so you can know what events to send to a node from rustdoc alone)

obsidian tusk
ionic sedge
# obsidian tusk I’ve taught a few workshops about max/msp and vcvrack and it’s a pain point, it’...

Hm, btw did you find that people had a hard time grasping the whole thing in general? Or was the more "programmy" side of it?

I tutored max in college, and the big hurdle that people had in my experience was just general programmatic thinking. And that was only really required for the control-rate scripting nodes, which Firewheel won't have. The pure signal flow stuff didn't seem as bad.

Not to belabor the point, just curious if it might be a little better with DSP only basically.

obsidian tusk
#

and with VCVRack I’m not certain but with hindsight I think it could’ve been hard to understand less because of the node-based thinking and more because of the fact that all the nodes use unfamiliar terminology and interfaces since they're supposed to be 1-to-1 with real hardware

#

It’d be super interesting to implement a node interface and then see how amateur game devs without audio experience interact with it and what they do and don’t find intuitive

ionic sedge
#

ya or just devs of any experience

obsidian tusk
#

I would wager that "amateur game dev without audio experience" describes something like 95+% of Bevy users but that’s also an assumption that should be tested if possible

ionic sedge
#

It could be very natural. Orrrrr it could be a huge stumbling block, in which case we'd definitely want a traditional option 😅

obsidian tusk
slim pulsar
#

Wait I know

#

Ok one sec

#

So volume right

#

I want my midi commands to multiply all velocity commands by a volume node’s value right

#

I can’t do that now

#

I can do that if the event was generic

ionic sedge
ionic sedge
# slim pulsar I can’t do that now

would it be too onerous to make a custom volume node for your purposes? i mean a volume node happens to be simple, but it might be annoying if you needed to do this for all nodes

obsidian tusk
slim pulsar
#

It would be nice to reap the benefits of other node processors in addition to that though

#

Which of course, you can like

#

Just make bespoke ones

#

And that’s fine, but it means that I need to have many more edges

ionic sedge
#

yeah i mean my ideal is that people rarely find the need to remake nodes

slim pulsar
#

Over a single pipeline of data

#

Definitely. For essentially every game except the one I’m working on like you’d never need to

ionic sedge
#

To those of us comfortable with making them, it seems easy. To those who've never done it before, it's still a tall order regardless of how nice we've made it.

slim pulsar
#

Everyone’s just going to use samples and regenerated audio formats

static quest
#

What are you finding inadequate about the built in MIDI event types?

slim pulsar
#

Nothing! Haha I’ve used it before and could totally do it again if I was made

#

But that crate has the capability of invariance

#

And the special bit aligned types just were difficult for me to agree with. But it’s a fine crate

static quest
#

So is the PR you sent obsolete now?

slim pulsar
#

It’s not. I haven’t sent it yet, but I haven’t had the chance to discover what seedling would look like on my fork

#

So I’m debating if it’s worth the squeeze

obsidian tusk
#

Hey billydm, I was talking about this earlier and I’m interested to hear your take: do you have any thoughts about using an interface that is compatible with Bevy's Events<T> for node events? Like, basically externally-managed event storage. You’d need to have some kind of trait system to avoid relying on Bevy directly and I’m not 100% sure what the best way is to tie events to a specific node, but my thoughts are that it would make custom events a bit more ergonomic and efficient, since instead of a collection of boxes you'd have a bunch of monomorphic collections (which would obsolete the awkward bytes variant). Plus, and for me this is the main motivation, it gives a better framework for allowing nodes to send messages both to the host and to one another.

static quest
#

Hmm, maybe.

#

Though my intention is that "custom" events should rarely need to be used, and so the overhead of boxing doesn't matter much.

#

Keep in mind that I also want people to be able to use Firewheel with a C API. So we can't rely too much on Rust's type system.

obsidian tusk
#

Ooh that’s a really good point, although you could have the C API use a version monomorphised to use simple flat buffers (or some other C-compatible type) with other implementations of the event-related traits just being used to reduce copying when used from Rust

#

Reduce copying + ensure that state (e.g the number of consumed events) are synced with the host

static quest
#

Hmm, though I'm not how well we could make it work with the event scheduler system (and by proxy the event cleanup system).

#

And again, my intention is that custom events should be rare. Is there a use case that requires a lot of custom events to warrant a better API?

obsidian tusk
#

Yeah for sure, I don’t have a specific design in mind or anything, but me and corvus talking about gizmos made me think about it bc it’d make stuff like a spectrum display and PPL meter pretty simple to implement

static quest
#

Oh, are you talking about being able to send events from the processor to the GUI?

obsidian tusk
#

That’s what got me thinking about it just now (and it also applies to subtitles, which I was thinking about before) but it also would be useful for sending events to processors for e.g sending trigger events to envelopes, sample playback control, maybe sending syllables to a speech synthesis node. I don’t think external events are necessary for either (and the latter is already expressible) but it feels like it could be an elegant solution for both. Plus if you need to send the same events to a bunch of nodes you only need to store the events once and the nodes just need to store a cursor I guess, but I don’t know how useful that is in practice

#

Anyway I’m not really suggesting that you implement this right now, I have no idea if it’s even a good idea 😅 I was just interested to hear your take

obsidian tusk
static quest
#

Yeah, I understand the concern. We need to strike the right balance between "generic" and "custom" in the API.

slim pulsar
#

But I’m really stretching it to its limits. Downcasting is perfectly fine

#

For now at least, like even if I’m sending tons of different custom events every second, I’ve not felt anything slow yet.

#

Essentially what I’m doing is probably never going to be found in other video games, and maybe what even I’m doing now is wrong. That I shouldn’t be relying on firewheel in the way I am. Just making it be an audio sink might work better, and coordinating everything else outside of firewheel.

#

I just haven’t realized the right layers of abstraction yet I think

ionic sedge
slim pulsar
#

it's a bit tricky with crossbeam though

#

since events aren't cloned across consumers

slim pulsar
#

need that shit to be so snappy

#

which I was able to do really easily with tinyaudio. not so easy with firewheel/seedling

#

but that's a skill issue I think. the idea is play now, broadcast later

#

but doing that without making the code a total mess, is tricky. it would require that I have only one component that can consume midi_io events, which doesn't really defeat any purpose I'm searching for with multiple nodes. And it's a pull/push problem. firewheel audio processors are pushy, not pully I believe.

#

just hairy

slim pulsar
#

@ionic sedge is there any way to run event polling faster without making a subapp? :o

ionic sedge
#

Hm, actually it would be a bit tricky blobthink you'd need access to the context, but that's locked to a particular thread.

#

what was easier with tinyaudio?

slim pulsar
# slim pulsar <@164224139316428800> is there any way to run event polling faster without makin...

to follow up, this has been my take:


struct MidiSynthProcessor {
   channel: crossbeam_sdiofjsoi::Receiver<ChannelVoiceMessage>,
   ..
}
impl AudioNodeProcessor for MidiSynthProcessor {
    fn process(
        &mut self,
        info: &ProcInfo,
        ProcBuffers { outputs, .. }: ProcBuffers,
        _events: &mut ProcEvents,
        _extra: &mut ProcExtra,
    ) -> ProcessStatus {
        while let Ok(message) = self.channel.0.try_recv() {
            self.process_message(message);
        }

        let frames = info.frames;

        // guaranteed to be 2 due to our node's STEREO value.
        let (left, right) = outputs.split_at_mut(1);
        // Render audio from the synthesizer
        self.synthesizer
            .render(&mut left[0][..frames], &mut right[0][..frames]);
        ProcessStatus::outputs_not_silent()
    }
}
#

so I kinda sidestep everything with crossbeam. idk if this is the only way to go HomerHide

slim pulsar
# ionic sedge what was easier with `tinyaudio`?

haha I think it was probably because tinyaudio was pulling from my channel for events, and so like sending events through ecs is pushy. So kinda the way I've gone about this since there's no such thing as like lazy poll (i.e. some closure to represent the event queue for events), I kinda had to sidestep the ProcEvents passed into this processor. which means ill lose out a bit on prior edges but I think it's fine

ionic sedge
#

Although I do wonder if there's all that much value in a channel like this. Often, the audio processing only runs at a frequency close the the frame rate anyway. In other words, there's a certain minimum latency that is all but unavoidable.

slim pulsar
#

I see. dang some type of buffer would be so nice I think

ionic sedge
#

As long as you're okay with that latency, typically on the order of tens of milliseconds, then simply handling everything in bevy_seedling or Firewheel events should work decently.

#

The relative timing between events can be perfectly preserved.

slim pulsar
#

this channel sidesteps ecs straight from the input device and rebroadcasts to the world after playing.

#

okay

#

I can really hear it unfortunately, it might just be me though

#

was considering running a subapp at a much higher frequency

#

the preplanned timing stuff is freaking cool

ionic sedge
#

The overall latency from Bevy does seem to be a bit higher than it needs to be. Maybe it's partly related to that?

#

I'd love to test it a bit myself. Maybe I'll make a little keyboard app too.

slim pulsar
#

I got this in my dev tools with ui picking + i/o if you want a head start

image.png
#

though it is really fun to build as well 😆

ionic sedge
#

now that's a keyboard ferrisOwO

slim pulsar
#

I pulled this from blackphl0x's bevy_midi crate in the 3d_piano.rs example. Took a little patchwork to get it going again :)

slim pulsar
#

wait..<AudioEvents as firewheel::diff::EventQueue>::push doesn't work if your node is simple? :o

ionic sedge
#

In other words, bevy_seedling won't automatically send events

#

however, you can easily do it yourself!

slim pulsar
ionic sedge
#

actually, sorry, that's not true -- it should still send them

#

It won't automatically insert AudioEvents though.

slim pulsar
#

in the past

#

I think there was some write event on uhhh

#

on Events::push

#

I thought AudioEvents was the spiritual successor. Okay gotcha

#

ill def have to check it out in the am

ionic sedge
#

Oh, actually this is probably related to #49. I've already fixed it in some in-progress work, but I haven't merged it. I can publish a patch for this in the morning.

slim pulsar
#

nw! I'll see if I beat you to it 😉

#

thanks!

ionic sedge
#

Well #49 does need to be fixed, but it's actually not that. It's just

It won't automatically insert AudioEvents though.

#

Manually inserting it here resolves the issue:

    // Add the node and its configuration to the entity
    // bevy_seedling will automatically handle node creation and connection
    commands.entity(entity).insert((node, AudioEvents::new(&time)));
#

Should we be inserting AudioEvents on simple nodes? I suppose so. It's not like there's any real downside.

slim pulsar
#

you're amazing

#

im in bed but if i was standing id be jumping for joy

ionic sedge
#

i think i'll go back to automatically inserting it in the next patch

slim pulsar
#

haha I think it was just because I've used it in the past that I didn't look to check, makes sense

slim pulsar
#

not joking, somehow it's like

#

way faster now, not sure what happened

#

so I don't need to do the weird ECS bypass :D like i can't hear much latency

ionic sedge
#

was it something silly like bluetooth headphones

slim pulsar
#

I compared the previous version to this one and anecdotally

#

it's gotta be >200% latency improvement

#

could be bevy or something I did sped up the frame rate, but I highly doubt it. it's just way faster now, almost instantaneous

slim pulsar
#

hallo, is there an easy way to reset the plugin's config after startup? i.e. I want to update the desired_sample_rate in game...would need to somehow restart everything I think

#

I was looking for this but idk if that's a thing

#

(didn't look too hard sorry 😆 )

ionic sedge
#

just write to it :3

slim pulsar
#

:0

ionic sedge
#

bevy_seedling will automatically restart the stream with the new configuration

slim pulsar
#

dang it was too simple! perfect. thanks haha

#

I couldn't find the message that the sample rate was updated, it's probably buried in logs

ionic sedge
slim pulsar
#

right

#

i have trouble reading

ionic sedge
#

If it doesn't actually change, there could be a few things going on. Let me know if it seems like reinitialization isn't occuring!

slim pulsar
#

okay sure! thanks :)

ionic sedge
slim pulsar
#

ahh

#

yeah something's awry

#

it's all g, though! I'm just not triggering a message for this for some reason. nbd at all. I'll look into it when it comes time. gotta finish a few things

#

just planning for the future...gotta see how I can graph like hardware => block frame + sample rate such that latency ~0-20ms and sounds good

#

96k is crisp

#

but im on a ridiculous cpu

ionic sedge
#

really the core consideration is block size as far as latency is concerned

#

i wouldn't recommend higher sample rates for realtime audio

slim pulsar
#

why :o

ionic sedge
#

it's basically twice as much work for zero audible difference 😅 the main use would be in studio work where you want to preserve the quality of sounds after resampling

slim pulsar
#

idk. like 96k over 48 is sooo good to my ears

#

I can tell

ionic sedge
#

I think that might be placebo 😅

slim pulsar
#

well ok one sec

#

this will take a moment so I can ensure encoding matches up

ionic sedge
#

The critical factor here is the nyquist frequency -- the highest frequency at which a sound can be represented without aliasing. This is half the sampling rate, so at 48kHz, that's 24kHz. 24kHz is well above what we can perceive, so it's not possible, barring other factors, that we could tell any difference between 48k and 96k.

#

Both will losslessly represent any sound that we can hear.

slim pulsar
#

not like this totally but it's way closer to this

ionic sedge
#

If you happen to be producing sounds that do alias at 48k, then you may be able to tell the difference.

ionic sedge
slim pulsar
#

it's like this almost white noise that happens at the default. I match up my synthesizer sample production with the config

#

across a few soundfonts. I'd really want to show you hahaha

ionic sedge
#

you might be running into aliasing in that case blobthink

slim pulsar
#

it's cool, but it could be placebo

#

gotcha

ionic sedge
#

which is definitely perceptible

#

Although 96k is still a bit of a sledgehammer. You might try throwing a low-pass filter on the synthesizer way up at 16-20kHz

#

but also 96k might be fine depending on the load haha, Firewheel is quite optimized

slim pulsar
#

definitely

#

grr gotta go spend money on a worse computer

heady robin
ionic sedge
#

oh exciting

#

Is there any context for this failure?

#

oh it's the itd node

#

looks like the delay line indexing could use some new tests

heady robin
heady robin
ionic sedge
#

Yeah no worries, likely something just within the margin of error for some float calculation.

potent zenith
#

hello, is it normal that PlaybackSettings.pause() and then .play() some time after restart the audio sample from the start ?

#

oh yeah looking at the code:

pub fn play(&mut self) {
    *self.playback = PlaybackState::Play {
        playhead: Some(Playhead::Seconds(0.0)),
    };
}
ionic sedge
#

Ah, yeah this was updated in response to a change in Firewheel itself. I see how that's problematic! To get it playing again from where it left off, you can

*settings.playback = PlaybackState::Play {
    playhead: None,
};
#

But clearly that method should change.

#

I'll make an issue for that too!

potent zenith
#

thanks !

slim pulsar
#

seems like 0.17-rc1 won't be too bad

viscid plank
slim pulsar
#

yeah just a version bump on Firewheel and an update to a proc macro

image.png
slim pulsar
ionic sedge
#

I think we'll also have to update the entity-targeted event (PlaybackCompletionEvent)

slim pulsar
#

@ionic sedge why is this impl in a const block? :o curious to learn more about it

image.png
#

this is a macro expansion

#

all i needed to do was move this to the new DynEq trait, but I saw this const block impl and I've never seen that before

#

is it because you import alloc?

#

I've branched firewheel for the rc and seedling, lmk if you'd like me to share some slack! also could spend tomorrow morning combing through a few issues. I've personally had no issues with seedling or firewheel even when I was sandboxing my lil saw-wave (and I've been looking 😆)

#

so lmk if you have some issue pointers and id be happy to jump on that

ionic sedge
#

im sure dtolnay knows exactly why it's necessary

ionic sedge
#

It could be an issue with requesting devices before opening a stream, maybe? That's a notable difference from previous versions.

slim pulsar
ionic sedge
slim pulsar
#

just for the bevy rc

#

nothing more than a version bump

ionic sedge
#

oh right it does depend on bevy

#

well we will want to potentially add a new feature flag for glam 0.30

#

we have some Diff and Patch implementations for a couple glam types

slim pulsar
slim pulsar
#

found it. ircam-hrtf. have no clue what that means but I sure can increment a number

ionic sedge
#

haha thats the HRTF crate, it uses the IRCAM database

#

fancy spatialization

slim pulsar
ionic sedge
#

that's okay! it's probably something weird tbh

slim pulsar
#

that's the really interesting part which has me excited

#

like wtf

#

hahhaah

slim pulsar
#

@ionic sedge if you go to src/pool/mod.rs:744, and you add

let foo: Undef = 3;

does your editor get mad at you?

#

mine thinks that's totally cool

ionic sedge
#

no it no likey
but i'm just on master atm so blobshrug

slim pulsar
#

oh, hmm. same

#

yeah something's off. idk why my ra is so chill

slim pulsar
#

would you mind sharing your rust_analyzer settings?

ionic sedge
#

hm, i don't think i've really configured it at all tbh

#

are you sure it's not an error somewhere in a higher dependency like firewheel? I know we talked about it earlier -- just double checking

slim pulsar
#

it's editor specific at this point. Just tested with v🤢sual studio code and i'm getting errors

#

so I think it's a zed bug

slim pulsar
#

alright should be working since my 0.17 migration was absolutely flawless and no bugs

#

(I haven't tested)

slim pulsar
ionic sedge
#

oh nice! thanks for going though the process <3 it's quite a few changes haha, I thought we'd get away with less

slim pulsar
#

the only things that I think are like behavioral changes/not absolutely 1:1 are

  1. EntityCloner::build -> EntityCloner::build_opt_out (there's an opt in which might be cool for sanity purposes)

2, EffectsQuery lifetimes. I elided the new lifetimes on this, but I you can actually extend 's now for get_effect and get_effect_mut as the second param. might go back and add that. edit: added

slim pulsar
#

oh btw I havea a bunch of one-liners in my branch. might be worthwhile to "squash and merge" so i dont bloat the commit history

#

i can also squash the commits inline if you'd like

#

@ionic sedge I can investigate wasm if you're not on the topic atm. just saw the pr CI

ionic sedge
#

what if i want this in my history though

Screenshot_2025-09-16_at_6.54.51_PM.png
slim pulsar
#

hahaha whatever you wanna do. Also weird that the semver failed ferrisHmm

ionic sedge
#

for getrandom

slim pulsar
#

ill see what's up with semver then

ionic sedge
#

might just be that it can't handle patches

slim pulsar
#

yeah, strange. ill see if there's an issue for that

ionic sedge
#

btw @slim pulsar I'd recommend using a global git config for ignoring files like .DS_Store

slim pulsar
#

great idea. also the semver thing I just discovered

#

it's real

#

so [patch.crates-io] really only works for local packages. the value of firewheel itself in the toml should be updated

#

sorry!

#

My mistake came from using foxtrot which I realize...yeah I straight up cloned that, it's not a library

ionic sedge
#

no worries haha, ya crates io only lets you publish crates where all the dependencies are also published

slim pulsar
#

I'm going to update stuff on another branch at least so I can finish my migration

#

actually not the same branch, don't wanna touch work in progress if there is any

ionic sedge
#

oh are you working on it? I published ircam and am working on wrapping up seedling now

slim pulsar
#

at least for this I am because you can't actually target those crates

#

you can ofc build them locally

#

but for downstream, they can't actually build their packages on these branches... 🫢

#

it's a learning lesson at least

#

firewheel is okay, but yours are not

#

1 sec

#

will pr a fix

ionic sedge
#

ya im not quite done with it yet

slim pulsar
ionic sedge
#

should only be a few minutes, assuming i can wrap up a couple issues that have been lingering a while

slim pulsar
#

fixed by adding the lockfile on my other branch

heady robin
#

@ionic sedge is there a branch I could connect to to get the index fix without moving to 0.17 rc?

#

I'm still on 0.16 and my game keeps crashing because of it

#

not urgent, in the meantime I'll just

image.png
ionic sedge
#

Oh yes, let me backport the fixes

slim pulsar
#

the alsa one

#

I saw the issue you posted in better audio

#

I'm wondering what changed think_blob

ionic sedge
#

yeah that's to be expected i think, in 0.5 we started fetching devices eagerly

slim pulsar
#

ahhh

ionic sedge
#

we spawn them in the ECS

slim pulsar
#

would it be worth delay-fetching? I'm still grokking this explanation but thinking

ionic sedge
#

well the idea is that the ECS should enumerate all the I/O devices at startup in case you want to select one before initialization

slim pulsar
#

if I'm reading this correctly, then if you wait to iterate on devices, then no error would be thrown. if I'm not reading this correctly, then you don't have a choice

ionic sedge
#

hm, wait on what though blobthink

slim pulsar
#

should seedling have a one-shot preupdate pass?

#

iterate on audio devices, these new inputs

#

but idk if CpalBackend::available_input returns a colelcted result

#

or if it's lazy

#

seems like it's collected so that idea's no good

ionic sedge
#

but it's a bit awkward to wait for the event to be handled -- you'd have to trigger it in one system and then query the devices in another

slim pulsar
#

agreed

ionic sedge
#

Although the whole API isn't completely fleshed out. It's only a partial abstraction over the backend. So maybe it's fine to just not automatically fetch devices for now.

slim pulsar
#

yeah, I was doing this for my MidiInput plugin, and it felt icky to do to be honest

#

so I do what you did in the past, which is the user has to query. it isn't eager since I don't know when a good time is to refresh

#

not sure how you handle refreshing...a signal? :o

ionic sedge
#

Basically, for a good API, I think there's no way to avoid implementing a hefty trait on each backend. I don't like that because it means you can't just pull in any third-party firewheel backend. But I don't see any way around it.

#

We could have a cute ECS API like using relationships for input and output.

slim pulsar
#

tbh eager totally makes more sense. PurpleG sent this, and it's often the right route for ecs plugins

We need less decisions before we know what we want
so at least from a user perspective, this seems fine. Might be fitting in a "sub-plugin"

#

because the biggest API problem with ECS is

  1. good documentation. things are really difficult for plugins without this due to the overly-flexible nature of the paradigm
  2. knowing what the usecase is
    strong opinion weakly held ofc
slim pulsar