Better Audio

[cerulean shoal]

Maybe I missed something, but I thought the point is to have other systems also use resources so having rendering running would help increase load on the system and put more pressure on the audio system to perform well.

[rapid musk]

Having Rodio and Firewheel join hands will surely lead to a lot of discussion that is out of scope for this working group, so it should be done somewhere else I think. Perhaps a discussion on Rodio's github repo or somewhere else that is more suitable and easier to discover for other users of Rodio who will be impacted and would also like to join the discussion. While that is being evaluated I'd personally like to see Firewheel integrated into Bevy so that we can start to take advantage of the work put in by @dusky mirage and @slate scarab

[dusky mirage]

I actually have a Discord server for Firewheel where we could talk. https://discord.gg/rKzZpjGCGs

Though I do need to something IRL for a bit. TTYL!

[rapid musk]

That could work and Rodio could open an issue on their Github linking to it. Though I am somewhat averse to having it in Discord, as this greatly reduces discoverability since it cannot be indexed by Google or other search engines. Forcing any user of Rodio who wants to participate in or just view the discussion to register a Discord account isn't ideal I think.

And should something happen to the server, the discussion will be lost. If Discord is preferable, then for example scheduling meetings for discussion and producing written summaries to be hosted on Github or somewhere else is a possibility. That is what the C# language team does I believe, but that may just add unnecessary overhead to the discussion. Do whatever works best for Firewheel, Rodio and their users.

[lean cloak]

I was planning to make a summary of the outcome of this discussion and save that in the rodio repo. I would like some time to talk to the two most active rodio maintainers before making this public.

For now I'm okay moving this active discussion to Firewheel's discord, to not clutter up the audio working group too much 🙂

[celest whale]

Good call 🙂

[dusky mirage]

@slate scarab I've merged the PR, thanks!

I'm seeing if I can fix the CI by disabling parallel tests.

But man, it takes a long time for it to compile those egui dependencies.

[slate scarab]

I'm used to almost 20 min CI times by now for some of my projects, so four minutes seems great XD

[dusky mirage]

Shoot, still fails. (Also it took 12 minutes).

[slate scarab]

I thought about a generic garbage collector implementation a little more, and I think it would actually be very easy. It might be worth adding that just for this.

i.e. something along the lines of

trait Collector {
    fn register(&self, data: Box<dyn StrongCount>);
    fn remove(&self);
}

struct ArcGc<T: ?Sized, C: Collector = GlobalCollector> {
    data: Arc<T>,
    collector: C,
}

Where in most cases C will just be a ZST.

lemme see if I can whip that up real quick -- probably worth it if it doesn't add too much complexity

we shouldn't have to change anything besides that one file at least

[dusky mirage]

Another thing we could do is instead of checking for 0 in the test, we check it against what it starts at when the test is run.

[slate scarab]

I did think about that a little, although I think it could actually still fail

like if several values are dropped between when we add one in the test and then check how many exist after it's dropped

[dusky mirage]

True, if multiple tests are run in parallel that could happen.

[slate scarab]

Okay, let me know what you think!

The trait bounds had to be a made a little more verbose due to the Drop impl, but aside from that the changes aren't too bad.

Oh, hold on. I didn't realize I already pulled in the CI changes.

[slate scarab]

okay fixed

[slate scarab]

Okay I'm pretty happy with my overall API and I've finally got ergonomic spatial audio going. I think all I need is another documentation pass and additional testing before bevy_seedling is ready to publish.

@dusky mirage Do you think Firewheel is ready for another publishing? One thing that's been on my mind for a while is the just-newly-merged volume additions in bevy audio. In short, volume is expressed as an enum with linear and decibel (log) variants. Here's the main bits of the implementation.

This could be maintained as just a Bevy abstraction, but it could also be integrated directly into Firewheel (which I think would make working with volume nodes a little more ergonomic and flexible in general).

Aside from this, I don't have any major notes.

[dusky mirage]

Yeah, I think it makes sense to expose volume like this. It should be easy, I'll do that real quick.

[slate scarab]

I wouldn't complain if VolumeNode just became a tuple struct btw -- I'd love the brevity:

commands.spawn(VolumeNode(Volume::Linear(1.0)));

obviously not critical or anything but

[dusky mirage]

Yeah, that makes sense!

[vale quarry]

Is firewheel open to contributions?

Would a gain node be something useful? I know it's similar to volume but it seems that the volume node can't boost the gain of a signal

[slate scarab]

Hm, I think the volume node supports this. This code here only does a little clamping around zero and one so it can do bypass and silence operations, but it should allow any values from zero to f32::MAX.

[vale quarry]

ah fair enough, my bad, I misread the code

[slate scarab]

no worries!

there's definitely plenty of room for contribution, though -- I think there's quite a few issues on the repo marked as "good first issue"

[dusky mirage]

@slate scarab I've added the new Volume type, I'm just waiting for the CI to finish to merge it.

But we do have another problem, it appears that the new ArcGc implementation is broken somehow. Updating an atomic's value doesn't update to the other instances for some reason.

[slate scarab]

Hm, okay I'll look into this asap

Things seemed to function fine in my larger tests, but maybe I missed something.

Do you have a minimal repro?

[dusky mirage]

I do not, I just noticed that the sampler pool example wasn't detecting that the sample has finished playing. I've debugged it and the processor is indeed setting the atomic correctly, it just doesn't appear on the other end.

[slate scarab]

Is it possible that the audio-side set of parameters is not using the same ArcGc instance as the ui side?

This could happen if you do something like

context.add_node(SamplerNode::default(), None);

// assuming we use this one in the UI
let sampler = SamplerNode::default();

as opposed to

let sampler = SamplerNode::default();

context.add_node(sampler.clone());

In the first example, we're not actually sharing the atomic -- we're creating two separate instances.

If the code is like the second example, I'd be pretty surprised. I don't see how they could become decoupled.

I suppose this could indeed be a footgun, and maybe for things like this it would be better to put all shared data in the context. Of course, that has its own problems since the context will generally be !Sync and often !Send.

[slate scarab]

This problem actually comes up a little in the ECS API as I've been wrapping it up -- that is, you can sometimes end up with decoupled atomics if you're unaware of certain semantics.

[dusky mirage]

Aha! Found the problem: worker.sampler_node = sampler_node.clone();

[slate scarab]

One thing that might come to mind as a solution is rolling back the merging of Handle and Node types, but I'm not sure that would completely solve this problem (at least in the general case).

I think it would still be fairly easy to accidentally create separate instances.

[dusky mirage]

It should be worker.sampler_node.sequence = sampler_node.sequence.clone();

Hmm, yeah. I imagine it would be easy to accidentally create separate instances.

Would it break ECS to remove Clone from SamplerNode?

Although wait, that wouldn't entirely solve the problem either.

[slate scarab]

To clarify on where it's problematic for me, here a crash course on the API I've settled on for sample pools:

// Here we spawn a pool with a custom label and
// insert a spatial audio node as an effect.
Pool::new(MyPool, 4)
    .effect(SpatialBasicNode::default())
    .spawn(&mut commands);

// To play a sound in this pool, we can simply spawn a sample
// player with the pool label.
commands
    .spawn((
        MyPool,
        SamplePlayer::new(server.load("snd_wobbler.wav")),
        Transform::default(),
        SpatialBasicNode {
            panning_theshold: 0.9,
            ..Default::default()
        },
    ))
    .log_components();

So what happens here is that we create a sampler pool where each sampler node is followed by a SpatialBasicNode. Then, when we play a sample, the sample entity (which is totally separate) has another instance of SpatialBasicNode which the actual node tracks.

In other words, we've got a "free-standing" SpatialBasicNode on an entity that we use to set the parameters on the actual SpatialBasicNode that's connected to the audio graph.

This is very convenient and works beautifully.... for nodes that don't have shared data.

It would not be easy to make sure this works for nodes that do have shared data. Possible I suppose, but not elegant.

And to clarify, the SamplePlayer type isn't an audio node. It just holds a sample, and is queued up and assigned to an actual SamplerNode when it's spawned.

[dusky mirage]

Hmm, if only Rust had an Assign trait.

[dusky mirage]

Although if you are using the sampler pool internally, the fix I just pushed should also fix that.

[slate scarab]

In my head, requiring shared data to be stored in the audio context (much like !Send data) would fix this problem completely with only a minor impact to ergonomics.

Most users won't directly interact with a SamplerNode (in Bevy), so they won't have to access the context directly to get any of that information. And that's also easy to do in the ECS API if they really do need it, like if people are reading back tempo information or peak metering or whatever.

It's totally possible this will make the standalone API worse though, so that's obviously an important judgement call.

[dusky mirage]

Hmm, that's not a bad idea actually. I'll think on that.

[slate scarab]

The Firewheel sampler pool is a great mechanism, but it's not quite suited to the more ad-hoc approach I've set up here. You can see what you think once I've got my stuff published. I might be able to integrate them properly, but I haven't put much effort in yet!

[dusky mirage]

@slate scarab Shoot, I'm running into another issue. I've added a custom Diff implementation for the new Volume enum type, but it seems that the Patch derive macro ignores it.

[slate scarab]

Oh yeah that's probably just down to a data model difference.

When dealing with enums, it's probably better to just write a diff and patch impl if you start writing one.

[dusky mirage]

How would I write a custom patch impl?

The issue is that doing something like this no longer works

// The parameter struct holds all of the parameters of the node as plain values.
#[derive(Diff, Patch, Debug, Clone, Copy, PartialEq)]
pub struct NoiseGenNode {
    /// The overall volume.
    ///
    /// Note, white noise is really loud, so prefer to use a value like
    /// `Volume::Linear(0.4)` or `Volume::Decibels(-18.0)`.
    pub volume: Volume,
    /// Whether or not this node is enabled.
    pub enabled: bool,
}

Nevermind, I figured it out!

[slate scarab]

Well it depends on how you did the diffing!

But in general it's fairly simple. The docs actually go over it in decent detail (specifically on the Patch trait). Enums are a little more tricky though since you have to match on them during diffing (and patching if the type is involved enough).

In this case, since it's an enum with two variants that both hold f32, you can actually simplify things a bit:

impl Diff for Volume {
    fn diff<E: EventQueue>(&self, baseline: &Self, path: PathBuilder, event_queue: &mut E) {
        match (self, baseline) {
            (Volume::Linear(a), Volume::Linear(b)) => a.diff(b, path.with(0), event_queue),
            (Volume::Linear(a), _) => event_queue.push_param(a.into(), path.with(0)),
            (Volume::Decibel(a), Volume::Decibel(b)) => a.diff(b, path.with(1), event_queue),
            (Volume::Decibel(a), _) => event_queue.push_param(a.into(), path.with(1)),
        }
    }
}

impl Patch for Volume {
    fn patch(&mut self, data: &ParamData, path: &[u32]) -> Result<(), PatchError> {
        match path {
            [0] => {
                *self = Self::Linear(data.try_into()?);
                Ok(())
            }
            [1] => {
                *self = Self::Decibel(data.try_into()?);
                Ok(())
            }
            _ => Err(PatchError::InvalidPath),
        }
    }
}

This is more or less how I'd do it, just for reference!

[dusky mirage]

I got it working now.

[slate scarab]

oop sorry

should be for Volume

[dusky mirage]

I needed to add

impl Patch for Volume {
    fn patch(&mut self, data: &ParamData, _path: &[u32]) -> Result<(), crate::diff::PatchError> {
        *self = data.try_into()?;
        Ok(())
    }
}

[slate scarab]

(Just a note -- we could certainly refine the derive macros for enums to produce this maximally optimized implementation in this case, avoiding all unnecessary allocations. Right now it's simpler, so it will do a little allocation in the diffing impl.)

[dusky mirage]

@slate scarab I've added the ability for a node to store custom state in the Firewheel context and for the user to get that state with the node ID. I'm fine with making the API a little more complicated (Firewheel is meant to be low-level after all). Plus this completely solves the problem with replacing the ArcGc. Let me know what you think! https://github.com/BillyDM/Firewheel/pull/33

[slate scarab]

Oh awesome! I'll integrate this on a branch of bevy_seedling to see how it feels in practice. I should be able to do that this evening.

[vale quarry]

What does diffing and patching do?

[slate scarab]

It allows us to synchronize data between the ECS and the audio processor.

Basically, we compare a piece of data to some baseline (diffing), send the differences to the audio thread, and apply those differences in audio code (patching).

[vale quarry]

So this is specific for its use in bevy?

[slate scarab]

We do this instead of, say, sharing an Arc<Mutex<T>> with the audio thread because acquiring a lock can block the audio thread or run for an indeterminate amount of time (due to syscalls).

[slate scarab]

At least right now, it's the general mechanism in firewheel for synchronizing audio code.

[vale quarry]

Comparing to a baseline rather than the current graph state? What's the baseline?

[slate scarab]

Usually just a copy of the data.

I made a Memo<T> type for standalone uses that's defined roughly as

struct Memo<T> {
    data: T,
    baseline: T,
}

With a DerefMut implementation that provides a mutable reference to data only.

Whenever you want to synchronize the audio code, usually once a frame, you can compare data with baseline to compute the differences, and then send those along.

In the ECS, we can hide this a bit more by just adding a Baseline<T> component for any component T that implements Diff.

[vale quarry]

So what does this data look like on the ECS side? Are you holding some kind of handle to a node in the graph that's on the audio side?

Presumably you don't have a graph on both sides?

[slate scarab]

(Actually it would be convenient to duplicate some information if only for ergonomics, such as connections between nodes.)

But yes, each entity that represents an audio node has a NodeID component, where the NodeID refers uniquely to the actual node in the audio graph.

This is automatically acquired from the graph when inserting a registered audio node into an entity.

So for example, when you spawn a VolumeNode, the final representation is something like

// this
commands.spawn(VolumeNode(Volume::Linear(1.0)));

// turns into
// (VolumeNode, Baseline<VolumeNode>, Node(NodeID), Events(Vec<NodeEventType>))

[vale quarry]

hmm I think I need to see this in context, have you got a link to your implementation?

[slate scarab]

Yes, it's available here. (I'd specifically check out the diffing branch I linked to -- that's the one I'm very close to publishing.)

Take a look at the examples to see things in practice.

Although most of them deal with sample playing, which doesn't involve too much parameter accessing on its own. I'll make some more examples to illustrate this part of the crate better.

[vale quarry]

ty!

[slate scarab]

Well as an aside, it turned out to be very easy to get Wasm support going. All firewheel needs is a feature flag to set cpal's "wasm-bindgen" feature.

This is, of course, the terribly jank single-threaded solution, but it's a good start!

[slate scarab]

bevy_audio exposes a GlobalVolume resource. I'm curious if people think it's valuable to preserve this resource in bevy_seedling. Currently, to set the global volume in bevy_seedling, you query for the terminal volume node:

fn mute(mut q: Single<&mut VolumeNode, With<MainBus>>) {
    let mut params = q.into_inner();
    params.volume = Volume::Linear(0.0);
}

Does this seem too cumbersome? Personally I like treating all nodes the same (since GlobalVolume would represent an exception), but I can see the value in a simple resource.

[oak walrus]

There's a global volume because Rodio has a global volume, I don't think there's anything more than that, I think it makes sense to not make an exception, as long as it's documented

[celest whale]

Seconding this. bevy_audio, for the most part, isn't particularly "designed"

[dusky mirage]

@slate scarab At some point we should figure out an ergonomic way to have "send tracks". For example, sending the output of an instance to a reverb node with some volume. You don't want to have an individual reverb node for each instance because reverb is computationally expensive (especially convolutional reverb).

[slate scarab]

ya from the bevy_seedling side of things, that should be super easy, even with the existing API -- but we could and probably should make it even easier

like for illustration purposes you could do like

#[derive(BusLabel, Debug, PartialEq, Eq, Hash)]
struct Send;

commands.spawn((
    Send,
    MyFancyReverb,
));

#[derive(PoolLabel, Debug, PartialEq, Eq, Hash)]
struct MyPool;

Pool::new(MyPool, 4)
    .spawn(&mut commands)
    .connect(MainBus) // You can think of this like a 50/50 dry/wet
    .connect(Send);

// Play a sound with effects
commands.spawn((
    MyPool, 
    SamplePlayer::new(server.load("my_sample.wav")),
));

This spawns a new pool, but you could also query for the default pool and connect that to the send.

If you want to route 100% of audio through the send though, that's a touch more tricky right now. bevy_seedling doesn't expose disconnect / reconnect / splicing functionality in a very elegant way yet. That's definitely on the todo list though.

[sacred osprey]

I dunno about the 50/50 dry wet thing. My expectation as a user would be that connecting to two destinations would create an identical copy for each destination.

[slate scarab]

That's what it does, yes. Sorry if that wasn't clear!

[sacred osprey]

Ah ok cool that sounds good then

[slate scarab]

it would be slightly annoying to make an actual dry/wet split with parameters, so maybe that's something we should add special support for

[slate scarab]

btw @dusky mirage have you had a chance to wrap up the state PR? I made some comments, but there's definitely no obligation to follow them!

[dusky mirage]

Oh right, I got distracted with other projects. I'll look into it here soon.

[slate scarab]

Once Firewheel gets another release (I don't think it matters if it's a beta or minor version bump?) I'll be able to publish bevy_seedling. There are some more things to add over time, but the core functionality is all there.

[vocal dock]

Correct me if I misunderstand, but 0.16 is not going to have the new audio engine upstreamed, but users can/will be able to use the bevy_seedling crate as a dependency?

[celest whale]

Precisely

[vocal dock]

Great! Looking forwards to it!

[slate scarab]

Yes this is more or less the case. Right now I just depend on 0.15, but I don't think there's anything that needs any serious migration for 0.16.

Also, the design and implementation will need to be proved out in practice by more than just me before upstreaming.

[vocal dock]

Makes total sense, I was just hoping it's going to be available to users soon, which it is, so I'm happy about that.

[dusky mirage]

@slate scarab I've added some changes to the PR!

[slate scarab]

Sick! I'll re-integrate this in a couple hours -- I think it won't need any changes besides those component derives to compile as well, so it's probably good.

Oh before that release though I'll make a super small PR to add the wasm bindgen feature.

[slate scarab]

Alright @dusky mirage , I'm pretty much ready to push go on bevy_seedling. I took a little extra time to improve the overall design and docs after some feedback.

If you're not feeling like tackling a minor release right now, feel free to just make another beta release! That's all I'd need for publishing as far as I know.

[dusky mirage]

Alright, I got the new version published!

[slate scarab]

Awesome, thanks!

[slate scarab]

Okay, it's up!

https://crates.io/crates/bevy_seedling

To get everyone up to speed -- bevy_seedling is my integration of @dusky mirage's Firewheel audio engine for Bevy.

bevy_seedling's docs should get you up to speed on how to use it! Note that it's currently written for Bevy 0.15 only, and I'll probably wait until 0.16 is stable (or very near stable) to update.

After giving bevy_seedling a final test-run in some of my game projects, I'm really excited! I think Firewheel, in combination with my crate, more or less completely satisfies the features proposed in this working group's initial design doc.

Please give it a try! Here's a sneak-peek:

fn play_sounds(mut commands: Commands, server: Res<AssetServer>) {
    // Play a sound!
    commands.spawn(SamplePlayer::new(server.load("my_sample.wav")));

    // Play a sound... with effects :O
    commands
        .spawn((
            SamplePlayer::new(server.load("my_ambience.wav")),
            PlaybackSettings::LOOP,
        ))
        .effect(LowPassNode::new(500.0));
}

Now, there's still some work to be done.

1. Sample playback needs speed settings and a pause/play API
2. I'd like to build a param automation API on top of bevy's animation code (using it directly is a little clunky, most automation doesn't need all the features of bevy's animation)
3. bevy_seedling (or Firewheel) could use a bunch more effects

(1) is obviously pretty important to have before bevy_seedling could outright replace bevy_audio, but it won't be too difficult to get in there.

[dusky mirage]

By "speed settings" do you mean doppler stretching (as in making a sound higher/lower pitched by changing the playback speed)?

[slate scarab]

Oh yeah just playback speed. Sorry, did you already add that?

(doppler shift would be cool though eventually)

[dusky mirage]

But what do you mean by playback speed if not doppler shifting?

[slate scarab]

Well I suppose playback speed would be the mechanism to enable doppler shift effects, but doppler shift implies calculating relative velocities and so on.

Playback speed control it useful for many things though, such as slightly randomizing the pitch of a common sound like a footstep.

(bevy_audio (via rodio) allows you to control the pitch of a sample and modulate it while it's playing. This example demonstrates that on some music, for reference.)

In fact, even with merely sample speed control, we could build simulated doppler shift in the ECS without any special Firewheel support very easily.

[dusky mirage]

I mean, that's what doppler shifting literally is. Just playing a sound faster or slower.

It shouldn't be too hard to add rudimentary speed control using simple linear resampling. (There are a lot of more advances methods that have better sound quality, but having something simple for now is probably good enough. My guess is that rodio was using linear resampling anyway since I don't see any reference to an advanced resampler like rubato in its source code.)

[slate scarab]

ya I imagine people looking for better resampling can just implement their own player -- most users will probably appreciate the minimal performance impact of linear anyway

[dusky mirage]

And in theory it should be possible to add a custom "Sampler Effect" API that lets users plugin in their own DSP into the built-in sampler node.

[slate scarab]

that could get pretty fancy

[dusky mirage]

Yeah

We could also have the resampling quality as an enum where we can add more options in the future if needed.

[slate scarab]

that would be more user-friendly for sure

[dusky mirage]

For my DAW engine that I'm working on, I've been thinking of reproducing the resampling algorithm as described in this paper. It seems to provide a great middle ground between low quality linear resampling and the high quality but CPU-heavy sinc resampling. https://github.com/BillyDM/awesome-audio-dsp/blob/main/content/deip.pdf

Though I haven't gotten around to it yet.

[dusky mirage]

@slate scarab Actually, after trying to add a "speed" parameter to the sampler node, it might be worth reworking the sampler node to use the new diffing API. It probably also makes sense to have the playback state and the playhead be parameters.

(I probably should have done it before I released a version on crates.io, but oh well).

[dusky mirage]

Alright, I think the new API for the sampler node is turning out much better than it was before! It's still going to take some time though. I'll have it finished sometime tomorrow probably.

(I think I figured out a way to not need the SamplerState at all when queuing events.)

[rapid musk]

Cool! Is it possible to seek / set position in an audio file? I had a look at the docs but I couldn't find an API for it. I think I still have the code for the beatmap editor I stopped working on since this is missing in bevy_audio.

[slate scarab]

I believe @dusky mirage is actually working on that right now 👀

[rapid musk]

Neat! Then I might be able to provide some feedback once that's done.

[slate scarab]

To be clear, I believe it's possible with Firewheel's existing API, but it would be tricky to access directly through bevy_seedling's pool abstraction. Luckily, as far as I can tell, the changes BillyDM's making right now should make seek behavior super easy for me to work into bevy_seedling.

[dusky mirage]

Yes, I am!

[rapid musk]

Great! I'll look into how bevy's changed over the last couple of versions while I wait then 😄

[slate scarab]

Oh by the way, I just had this thought:

The overall Diff and Patch API can be nice, but it can also be inconvenient. If you want fine-grained control over what happens when individual fields receive updates, it's not really possible to recover that information without manually matching on paths and types in the audio processor.

I'm not sure if you're working on that right now with the sample player, but it might be relevant.

I'd like to clarify that bevy_seedling actually does rely on Diff and Patch to update two instances of an implementor so that they become equal, or the ECS will endlessly generate events.

Well, I just thought of something that might be really useful. We can preserve the Diff and Patch behavior so that data can be properly synchronized in the ECS while simultaneously giving node authors fine-grained, field-wise access to updates.

In short, we can just generate an enum when deriving Diff that encompasses all updates a type could receive.

To illustrate:

#[derive(Diff, Patch)]
struct Params {
    a: f32,
    b: f32,
}

// the generated enum would look like
struct ParamsUpdate {
    A(f32),
    B(f32),
}
// and it would be able to construct itself from node events

// so instead of doing something like this in a processor
if self.params.patch_list(event_list) {
    self.expensive_calculation_a = self.params.a;
    self.expensive_calculation_b = self.params.b;
}

// you could match on the underlying updates
event_list.for_each(|event| {
    match ParamsUpdate::try_from(event) {
        Ok(ParamsUpdate::A(a)) => self.expensive_calculation_a = a,
        Ok(ParamsUpdate::B(b)) => self.expensive_calculation_b = b,
        _ => {}
    }
});

For some nodes, updating the entire set of params is totally fine and desireable. For others, especially those where a change to a field (like a playhead) is really more of an event that should trigger special behavior on change, merely updating the whole struct isn't ideal.

Just sketching this out -- I think it's not only feasible but a beautiful solution.

trait Diff {
    // diff would need an associated type to represent the update
    type Update; 
    // ....
}

impl Diff for f32 {
    // for leaf types like f32, the update is just itself
    type Update = f32;
}

enum ParamsUpdate {
    A(<f32 as Diff>::Update),
    B(<f32 as Diff>::Update),
}

impl Diff for Params {
    // And for structs and so on, it would be an enum
    type Update = ParamsUpdate;
}

The downside being that it's a little more work to implement Diff manually.

[dusky mirage]

Ah yeah. That seems like a good idea. I'll finish up what I'm doing first, then we could focus on improving the Diff and Patch API in a future PR.

In the sampler node I've ended up defining Diff manually and not using Patch at all.

The enum thing would make that cleaner.

[slate scarab]

Ah yeah I thought that was probably the case. Luckily, that's not a problem! bevy_seedling treats sampler nodes as special anyway.

[dusky mirage]

@slate scarab Alright, I've finished the new API for the sampler! https://github.com/BillyDM/Firewheel/pull/36

[slate scarab]

I'll take a look as soon as I'm available!

[slate scarab]

Okay, I think it all checks out! It only took a couple minutes to bring in the changes, although I still need to make sure everything works. Basically, I have no objections!

Although if you're thinking of creating a new release right away, it might be nice to wait juuust a touch. My ideas for an improved diffing API (which I'm partway though implementing) are definitely breaking.

Also, I noticed you built up a type like this:

#[derive(Default, Debug, Clone, Copy, PartialEq)]
pub struct PlayheadState {
    pub playhead: Playhead,
    pub id: u64,
}

The idea being we want to force an event to be sent, event if Playhead is actually the same value as the baseline.

Well, I think we can perfectly generalize this behavior with a type like:

pub struct Notify<T> {
    value: T,
    rng: XorRng,
}

impl<T> core::ops::DerefMut for Notify<T> {
    fn deref_mut(&mut self) -> &mut Self::Target {
        self.rng.generate();

        &mut self.value
    }
}

That is, we have some wrapper that forces an update on every mutable access to the wrapped value, regardless of whether it's actually changed. In combination with the new diffing API I'm working on, this should perfectly achieve what you want for the sampler node while still allowing types in the ECS to update each other through diffing.

This is super ergonomic and very cheap, with the main downside being that we'd need to bring in the getrandom crate so the XorRng can be constructed with sufficiently random state.

getrandom is all but guaranteed to be in the tree for Bevy projects, but it could be a new dependency for non-Bevy projects.

Although I suppose it could be feature-gated.

Another approach I thought about, but ultimately has problems, is to just increment a counter on a mutable dereference. i.e.

#[derive(Debug, Clone, Patch)]
pub struct Notify<T> {
    value: T,
    counter: u32,
}

impl<T> core::ops::DerefMut for Notify<T> {
    fn deref_mut(&mut self) -> &mut Self::Target {
        self.counter += 1;

        &mut self.value
    }
}

[dusky mirage]

Yeah, incrementing a counter makes way more sense. If we use a u64, we don't have to even worry about wrapping around.

[slate scarab]

I mentioned that it's not perfect though -- you can run into situations where events are eaten.

That's why the random value is a more robust solution

[dusky mirage]

Even if you add 1 to a u64 every nanosecond, it would take thousands of years to overflow it (I can't remember the exact math, but it's something like that.)

[dusky mirage]

What do you mean by "events are eaten"?

IMO an RNG value is even less robust because an XOrng pattern eventually repeats.

[wary bridge]

~5800 years, I happened to do this math yesterday

[slate scarab]

The period should be long enough that it doesn't matter.

On the other hand, a counter presents a common scenario that is problematic, mainly arising from the fact that all new constructions start at zero (or whatever initial value you want to provide).

The issue arises when you "swap out" what baseline you're using for diffing. bevy_seedling does this a fair amount for sample players. Essentially, when a sample is spawned and assigned to a pool, the sampler node (and any effects) will track the node components on that sample player entity for diffing. This is super convenient in general, and works very well with the POD convention.

However, imagine you have a Notify object based on counting. If you clone it and then diff those two values, all will be well. However, what if we watch one instance of Notify for a moment, then switch to another?

let mut baseline = Notify {
    value: 0.5,
    counter: 0,
};

let mut instance_a = baseline.clone();
*instance_a = 0.6; // this increments the counter

// Let's say we do some diffing here. This will generate an event
// because `instance_a`'s counter is 1, while the baseline is zero.
// We then set `baseline` to `instance_a`
*baseline = *instance_a;

// So now, both the baseline and `instance_a` have their counter set to 1

// Let's create another instance without cloning
let mut instance_b = Notify {
    value: 0.5, 
    counter: 0,
};
// and finally, before diffing with the baseline, we set it once
*instance_b = 0.6;
// now, if we diff `instance_b` against `baseline` _no event will be generated_!
// both their counters are at 1, and even if you _also_ check the value itself,
// in this case they happen to be the same.

This would be a nasty bug, and because all counters would start with the same value, it would be unacceptably likely in my opinion. This would be (almost) completely avoided with a simple RNG seeded with entropy.

If this weren't an issue, I don't think there'd be any real problem with using a u32. It's okay if it wraps -- there would only be a problem if it wraps at the exact moment you start to diff with a new baseline that's also at zero. That seems exceedingly unlikely!

[slate scarab]

With all this in mind, you could probably also just use a counter if it starts at a random value. The chance of collisions would be so low that I don't think it matters.

(i.e. the chance of two counters initialized with a flat, random distribution colliding is just 1 / 2^32)

[rapid musk]

Could the baseline be based on a static that gets incremented or increased by some amount for each new instance?

Without atomics it might be iffy I guess

[slate scarab]

Actually yes, a static atomic that uses a basic RNG like the XOR rng, updated once per construction, should be sufficient.

[slate scarab]

Merely incrementing could trigger this issue I think

Atomics are totally okay for this, and in fact would be more performant than OS entropy lookup.

like way more

[rapid musk]

Might be worth considering that over bringing in another dependency then perhaps

[slate scarab]

Due to the imperfectness of a simple RNG, it could increase the likelihood of collision, but... it's probably still something like one in a million

[rapid musk]

How bad is it if one event is swallowed?

[slate scarab]

So uh.... watch out for a single dropped event every one million samples you play

Well in this case it would be used for a sample playhead, so you'd go set the playhead to some value only for it to not actually happen.

[rapid musk]

Which could be improved with better randomness. But only having to generate a new random number per new instance seems more performant than every mutable access, though I know nothing about the getrandom crate or cost of atomics

Ah, I jumped a bit too far back in the convo. It'd still be one new random per instance even without static.

[dusky mirage]

The simplest an RNG can be is 3 lines of code. I'm using this to generate white noise. I'm not sure what the period is though. https://github.com/BillyDM/Fast-DSP-Approximations/blob/main/rng_and_noise.md

[slate scarab]

I'm actually checking now XD

wow okay

[rapid musk]

Neat. It's 1am here and I can feel my brain turning to mush, good luck!

[slate scarab]

it uh

doesn't loop very often

[dusky mirage]

Though actually I think the simplest solution is to have all Notify types share a global static u64 counter.

[rapid musk]

Actually, even if there's a collision for every 1 mill generated number. You'd have to be diffing the two events that has the collision, and they would both need to have had the same amount of changes to them... That seems very very unlikely, or is it likely to happen once a collision has occurred?

changes / mutable derefs

[slate scarab]

Oh yes this would be true.

So it didn't loop before I ran out of memory, something like 1 billion values (in fact, it didn't even visit a previous value once). It's possible, in fact, that certain seeds won't loop until very late in the cycle, and it seems to be the case here unless I made a mistake.

[rapid musk]

You could also end up randomly generating a baseline that matches another events existing counter, though that as well seems exceedingly unlikely.

[slate scarab]

Right yes, which makes it a little tricky to fully characterize the probability. But my intuition is that it's so unlikely that we should not expect to see it occurring in practice.

[rapid musk]

I agree

[dusky mirage]

I mean, I see no downside to using a global static u64 counter. Let's just do that.

[slate scarab]

Hm, how would that work?

I'm not sure I follow 100%

[dusky mirage]

I'll get my computer, hold on

[rapid musk]

Without randomness you still have the issue Corvus described before I believe. Though it would be less likely to happen, but not as safe as a random number being generated per new instance. But if the global static u64 is good enough then that sounds great

[slate scarab]

Luckily, either way (whether you merely increment a global value or do randomization), this would be a cost incurred only on construction, and it would even be real-time safe.

[dusky mirage]

Like this ```rust
use std::sync::atomic::{AtomicU64, Ordering};

static COUNT: AtomicU64 = AtomicU64::new(0);

pub struct Notify<T> {
value: T,
count: u64,
}

impl<T> core::ops::DerefMut for Notify<T> {
fn deref_mut(&mut self) -> &mut Self::Target {
self.count = COUNT.fetch_add(1, Ordering::Relaxed);

    &mut self.value
}

}

[slate scarab]

Ah yes, that would have a lower likelihood of collisions. Although you would have to do an atomic operation for every mutable access

I think you'd only ever mutably access this type off the audio thread, and even if you did so in audio code I don't think the characteristics would be too bad.

[dusky mirage]

atomic operations are pretty cheap (likely cheaper than an RNG). There is also zero chance of collisions because there is only a single counter in the entire application.

[slate scarab]

until it wraps like you mentioned haha

[dusky mirage]

5000 years 🙂

[slate scarab]

not that that's frequent

[slate scarab]

I think even relaxed access enforces some constraints that could be worse, but it's not significant enough that I care right now.

[dusky mirage]

Also, you are only mutably borrowing this value off of the audio thread to set it. The node's processor just reads this value.

[dusky mirage]

Yes there are some constraints. But like I said the overhead is probably lower than a good RNG.

[rapid musk]

I guess this value would be so frequently updated that its practically guaranteed to always be available in L1 cache as well.

You wont have that benefit with a random number

[dusky mirage]

One thing I thought of though is that AtomicU64 isn't completely portable. A better solution is to use the AtomicCell<u64> type from the crossbeam_utils crate (which I think I'm already using in Firewheel).

[slate scarab]

Oh I think the simple xor rng would be totally sufficient (no repeats in a billion values is pretty good) and that's less than a dozen instructions which will probably always be faster than any kind of memory access. But again I'm not worried about the performance of your approach.

[dusky mirage]

I'd rather have zero chance of collisions. A one in a million chance could still happen if you have a million players playing your game after all. 🙂

[rapid musk]

I do like the simplicity of it as well

[slate scarab]

let me calculate real quick

[dusky mirage]

I mean, since this will only every be borrowed mutably when the user updates the parameter, I don't think we should worry about the overhead of atomics.

[slate scarab]

For sure, I'm just thinking of u32 vs u64 atm.

[dusky mirage]

Well, if it's a globally shared value across all Notify types in the application, I imagine there is still a small chance of collisions.

And like I said, the user updates these kind of parameters relatively infrequently, so even if it's wrapped in a Mutex I think it would be fine.

[slate scarab]

Hm, yes I suppose so.

[dusky mirage]

(And that's what AtomicCell from crossbeam does. It wraps the value in a Mutex if the system doesn't natively support an atomic large enough to fit it.)

[slate scarab]

Basically, if you increment the counter 100 times per frame, it'll wrap in 200 hours. Of course, after I calculated that, I wasn't sure how to even characterize the likeliehood of a collision once it has wrapped.

Oh, actually it seems almost impossible.

You'd have to have a value last for 200 hours without being updated.

(this is for a u32)

[dusky mirage]

Yeah, I guess that is pretty unlikely then. Whatever you think is best.

[slate scarab]

We will have to think about no_std eventually, but.... I'm not sure I'm too worried about supporting platforms with only 32 bit atomics

Maybe that's too harsh, I think we could probably get Firewheel going on my company's microcontroller platform XD

[dusky mirage]

Hmm, no standard would be kind of hard anyway considering that the Symphonia and Rodio dependencies don't support it.

[rapid musk]

But a different backend could, so it might be relevant as a feature perhaps

[slate scarab]

Yeah symphonia will be tricky.

@limpid river is the resident champion of no_std, and I imagine their preferred solution would be to cfg out the parts of the library that can't work with no_std.

[dusky mirage]

And that being said, the Firewheel engine itself uses AtomicU64. That would need to be changed if we want to support platforms with only 32 bit atomics.

[slate scarab]

haha yeah maybe we don't care about 32-bit

[rapid musk]

haha

[celest whale]

That should be an easy feature flag. Do type AtomicInteger = AtomicU64

rapier has the same dilemma with f32 vs f64

[limpid river]

I would add to this that the way Bevy uses portable-atomic means you can use AtomicU64 on any target

[dusky mirage]

Well, the thing is it's being used for things like the sample counter (counting how many samples have been processed). That would overflow a u32 pretty quickly.

[limpid river]

So the atomic types in bevy_platform_support::sync::atomic are fully cross-platform, including on std targets

[dusky mirage]

I believe portable-atomic also uses mutexes for platforms that don't support it. (Correct me if I'm wrong.)

[celest whale]

We're gonna have to spin out bevy_platform_support sooner or later 😂

i-cant-believe-its-no-std

Or maybe nonstandard, if I'm being less memey

[limpid river]

I'm not super familiar with the internals, but I know that it'll use architecture specific instructions if it can, bypass atomics entirely if it's a single-threaded target, or rely on critical-section for effectively mutex-like control

[limpid river]

Technically it already is, since it's independent of the rest of the Bevy crates, so you can just use it if you want haha

[dusky mirage]

Yeah, the docs say that it will try native atomics first, then some assembly trickery, then mutexes. https://docs.rs/portable-atomic/latest/portable_atomic/struct.AtomicU64.html

[limpid river]

And bevy_platform_support will also only use the types from portable atomic if not(target_has_atomic = foo) for whatever size foo

[slate scarab]

But to be clear, I think no_std will be nice to support with Firewheel (even if we avoid 32-bit targets), probably sooner rather than later. As long as there aren't any major ergonomic downsides for maintenance, it should prevent severe pain down the line, say if the Wasm target really does move to no_std.

[dusky mirage]

I suppose no_std support could be possible if we disable resampling and only support loading WAV sound files using the hound crate.

[limpid river]

So if a platform has native AtomicU32 it'll use core::sync::atomic::AtomicU32, even if it needs to use portable-atomic::AtomicU64 for the 64 bit one

[slate scarab]

Yes, or even no direct wav support at all tbh

[celest whale]

Yeah but people don't like pulling in Bevy-branded crates 🥲

[limpid river]

Yeah if features need to be disabled that's perfectly fine IMO

[rapid musk]

I'll see if I can pull in some bevy crates at work 👍

[dusky mirage]

It would be kind of hard to make a game without samples though. Unless you want to rely entirely on chitptune synthesizers or something.

[limpid river]

In my experience, the no_std people will hit that wall, and then provide a solution if it matters

[limpid river]

agb provides WAV support with the GameBoy Advance, so this might be a case of shape the API in a way where a 3rd party crate can provide the playback mechanism?

[slate scarab]

Oh yeah I just mean that people can load samples however they please -- the SampleResource isn't sealed, so... people could acquire their PCM buffers however.

[dusky mirage]

Oh wait, Firewheel relies really heavily on the ringbuf crate. I'm not sure it supports no_std. https://lib.rs/crates/ringbuf

[limpid river]

Yeah it does

[celest whale]

https://lib.rs/crates/ringbuf/features#feature-std

[dusky mirage]

Oh yeah, it does. Nevermind.

[slate scarab]

Is it one of those no_std libraries that just removes all the functionality though? XD

[limpid river]

The way to check is first open the Cargo.toml and see if there's a std feature, and if you can't see one, just check the lib.rs file, since it must have a #![no_std] attribute to have no_std support

[limpid river]

Thankfully it looks like a real no_std crate. They even use portable-atomic too lol

Gold Star ⭐

[dusky mirage]

Yeah, it looks like all of Firewheel's other dependencies support no_std as well.

[slate scarab]

So I think I'll go with the u64 counter. 200 hours is a lot, but... if people use Firewheel for art installations, I wouldn't want a glitch to happen after 200 hours because of theoretical performance problems with 32-bit platforms.

[limpid river]

Oh that's your crate! Happy to help no_std support if you need it, just @ me or message me directly

[dusky mirage]

Yeah, and honestly it's probably a good idea to replace all the atomics in Firewheel with the portable_atomics as well.

[limpid river]

Could always use a feature flag to control that counter size if it's contentious. big_space uses something like that for controlling the coordinate size

Clearly we need an ep feature lol

[dusky mirage]

Bevy on my toaster 😎

[celest whale]

@faint wigeon they're thinking of you!

[dusky mirage]

Man, if only the entire world could use 64bit RISC-V CPUs.

[limpid river]

If only we had a time machine

no_std support would be so much easier

[faint wigeon]

yes!! thank you for thinking about this, seriously, even if just to document it 🙂

[lament briar]

would make it easy to swap with a u8 to quickly test wrapping behavior

[limpid river]

That's also a very good point!

[lament briar]

e.g.: #1171171694526869554 message

that's how I discovered how big_space wrapping actually works

[slate scarab]

So there's another potential issue I wanted to get some feedback on. I've been thinking about it for a while, but it's pretty tricky.

In short, diffing and patching Rust enums has a bit of a failure case. Let's work with a (slightly contrived) example.

// Let's say we want to be able to express filter parameters
// as either frequencies in Hertz or in MIDI notes.
#[derive(Diff, Patch)]
enum FilterParams {
    Hertz { cutoff: f32, quality: f32 },
    Midi { cutoff: u8, quality: u8 },
}

The way Firewheel's Diff and Patch macros are set up, this will allow fine-grained updates for every inner field. For example, if the FilterParams enum is currently on the Hertz variant and the user changes just cutoff, the audio object will receive a single f32 patch. I think that's pretty cool! And it's perfectly composable, meaning you could have far more complex types in your variant fields that all receive this perfectly fine-grained diffing.

For this to work, however, the main thread and audio thread enum instances must remain in sync. That is, their variants must match -- if the main thread FilterParams is on Hertz, the audio thread FilterParams must also be on Hertz. Under normal circumstances, this is handled by the derive macros. However, if too many events are sent at once, it's possible some of them will be dropped. If you're particularly unlucky, a variant-setting event could be dropped, meaning the main thread and audio thread enums will fall out of sync. Any patches sent from the main thread can no longer be applied in the audio thread until the variant is changed again.

@dusky mirage, do you think occasionally dropping events is likely to occur? If so, this will be quite confusing to debug. The straightforward solution would just be to send the entire enum any time it's different, but you'll lose out on the fine-grained patches.

[dusky mirage]

IMO you should just send the entire enum if it changes. Using enums like this in audio DSP is pretty rare anyway IMO.

Parameters are almost always just a single value or a pointer to some data.

[slate scarab]

For simpler nodes certainly, although I could imagine scenarios where fancy custom nodes might want a more Rusty approach to state management.

A good example would actually be Option! It would suffer from this same issue.

[oak walrus]

Okay so since I can't sleep, and landed on the message, I've had a quick think and came up with potential solutions:

• adding priority to patches, such that dropping an event drops the lower-priority ones first
• detecting we're about to dropout a patch and switch to sending the full value instead
• introduce the concept of merging events such that you can do that instead of dropping events

Practically though I think events being dropped is not going to happen a lot, and the solution can always be "increase the event buffer size", so maybe the solution is that 😆

[slate scarab]

Hm, maybe the safest approach is to just send the whole thing at first. If we find that people do this often, especially with more deeply nested structures, we could pivot and support it again.

These each have some downsides that maybe just aren't worth the cost.

[slate scarab]

I think this is probably true, but it does make me a little uneasy.

If you generate too many events, there will necessarily be some error. Maybe we just and call enum desync an acceptable problem.

At the same time, it can't as readily self-correct as structs, which could be pretty problematic (i.e. it'll be difficult to diagnose when it happens).

[oak walrus]

The only reason it would fail is if we're at capacity on the ring buffer. We can detect that.

From my experience with audio plugins, it's similar to overflowing a note event buffer, and the solution I've seen implemented are either drip silently, or be a little smarter and prioritize dropping Note On and expression events first to not create stuck notes. But most often the solution is "make the event buffer bigger"

[slate scarab]

I suppose one of the ParamData variants could be something like Variant(Box<dyn Any>). Swapping from one variant to another means that, in the general case, the data must always be boxed (because all of it has to be present at once).

That way, we'd get a cheap way to infer event priority.

[celest whale]

https://github.com/RustAudio/audio-ecosystem/pull/10 @limpid river I'm putting out the rat signal on this one

[slate scarab]

Okay, I'll be honest -- I just took the easy way out 😅 I just have the derive macros send the whole enum every time.

There's a few side benefits. For one, this makes the macros simpler. More importantly, it'll make working with enum patches easier after I land my Patch changes.

I'll illustrate why here:

#[derive(Diff, Patch, PartialEq, Clone)]
enum MyParams {
    Pair(f32, f32),
    Single(bool),
}

// With fine-grained diffing, you need a Patch enum like:
enum MyParamsPatch {
    // We send this if the variant changes.
    Variant(MyParams),
    // Otherwise, we send individual fields.
    PairField0(f32),
    PairField1(f32),
    SingleField0(bool),
}

MyParamsPatch allows the diffing and patching to capture single-field changes within each variant, but it's kinda of a lot. The new API doesn't require you to ever interact with this type directly, but you can.

The alternative, where we just send the whole dang enum every time, makes it dead easy. Patches are just the entire enum.

Sorry this is all a little vague -- it's hard to convey exactly what's going on concisely. But in any case, we could always add a #[diff(fine)] container attribute that enables fine-grained enum diffing in the future.

[tender fiber]

A global volume usually has more than one input so adding is usually done. The clamp 0..1 is done so speakers don’t blow up. Adding 16 inputs raises the input theoretically to 16 so clamp output is done to maintain the 0..1 relationship

If something else is used and global volume not required, then that should be clamped

And like that, 16 32 64 channels work more or less the same

and Billy is great

Clamp enforces a 0..1 relationship. Sorry.

[oak walrus]

This is an audio mastering problem, not a technical one (float samples can go over 1, and most DACs gracefully handle out of range samples by doing the clamping themselves). I'm not for adding hidden audio processing, this has to be done consciously either automatically on Bevy's side (by automatically instantiating a peak limiter/clipper) or left to the user to properly master their audio output. I understand not everybody is an audio mastering engineer, but we don't expect them to be CG experts either yet we provide an extensive configuration space for the graphics engine.

[tender fiber]

He was curious

[slate scarab]

I uh... I don't follow 😅 I was merely talking about the API -- global volume as a resource. bevy_seedling provides a non-special-cased global volume by just exposing a terminal volume node.

[tender fiber]

If not necessary, even better

Use is use and working 80% when working 100% is possible with clamp is just a suggestion. Digital mix staging is key to headroom

It’s a better product when 6dB of headroom isn’t eaten

A 24-bit converter clamping a truncated 32-bit file will lose headroom

and 32-bit won’t

Converter or code

and 90+% converters are 24-bit

Hence, my clamp suggestion. Just a suggestion

[slate scarab]

Firewheel processes audio as 32-bit floats. Samples are converted from their native format to 32-bit floats immediately at the source. The only error you'd accumulate between nodes would come from imprecision. I think it's safe to say that running your audio through even a hundred nodes will produce exactly zero audible difference!

[tender fiber]

right
code is code
However, it will be truncated with a 24 bit converter into a 24 bit file and 24 bits of that 32 bit file will now get converted

and clamped losing the headroom

However, if it’s done in code in 32 bit no info will be lost

awesome work btw

[tender fiber]

You are not clamping because you are worried about 100 nodes. You are clamping because the output is (in most cases) not going to a 32-bit converter. A 16 bit converter will immediately distort
and a 24 bit one can distort in the most action packed scene as well

Clamp is on the output

Only on the output

Of course, you don’t want to clamp in 32 bit audio engine

Just a suggestion

[dusky mirage]

Aww, thank you!

[tender fiber]

Your nih posts were great

[dense estuary]

btw bevy_seedling has issues with web builds:

• required import in context/web.rs
• mismatched fn types in the same file
• extra argument
• unexistent generic values
• one other

Do I do a PR to master branch when/if I fix this?

[slate scarab]

Ah, sorry. I must have made changes since my last web tests. I'll add Wasm builds to the CI.

[dusky mirage]

Firewheel actually already has an option to hard clip outputs https://github.com/BillyDM/Firewheel/blob/main/crates/firewheel-graph/src/context.rs#L46.

[tender fiber]

That’s awesome.

[slate scarab]

If you want to tackle it, a PR against master would be great! I can fix that up in a little bit here though, if you're not working on it right now.

[dense estuary]

I'll see if I can fix the problem 😁

[tender fiber]

nice

[dusky mirage]

There is a concept called "dithering" to reduce the noise introduced from converting a 32 bit signal to a 24bit/16bit. Though the noise is barely even audible if at all, but I suppose it wouldn't hurt to add that as an option.

[tender fiber]

Clamp as default is sufficient

[dusky mirage]

The mathematics of dithering is kind of interesting. It works by adding noise to the signal at an extremely low volume. Then when the signal is converted to a lower bitrate, this added noise sort of phase-cancels with the noise introduced by bitrate reduction.

[tender fiber]

That’s better than a delta something something explanation

[dense estuary]

It's working 🙂

[dense estuary]

criterion has some troubles with web builds, so I'd like to hear your thought on the point of removing the rayon feature. I have tested with integrated to workspace examples, so examples are compiled with dev-dependencies.

Here is the scope of the issue (dev-dependencies section):

[dev-dependencies]
bevy = { version = "0.15", default-features = false, features = [
  "bevy_debug_stepping",
  "bevy_asset",
  "bevy_color",
  "bevy_state",
  "multi_threaded",
  "sysinfo_plugin",
] }

# Current criterion features: plotters cargo_bench_support rayon
criterion = "0.5" # Causing issues: its default version uses rayon, which is not supported by wasi32

Trace:

error: Rayon cannot be used when targeting wasi32. Try disabling default features.
  --> ...\.cargo\registry\src\index.crates.io\criterion-0.5.1\src\lib.rs:31:1
   |
31 | compile_error!("Rayon cannot be used when targeting wasi32. Try disabling default features.");
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

To fix the compatibility issue for examples, I've disabled default features and imported all default features except rayon:

criterion = { version = "0.5", default-features = false, features = ["plotters", "cargo_bench_support"]}

Tests are still compiling, examples are working fine on web.

I'd like to know if this fix is good for you

[slate scarab]

Hm, I'm curious how you're triggering this. Are you building the examples for Wasm directly?

Like this isn't happening for projects that depend on bevy_seedling, right? The dev dependencies shouldn't be pulled in there.

[dense estuary]

Yes, cargo build --target=wasm32-unknown-unknown --example sample_lifecycle --features=ogg,stream,bevy/bevy_asset --release (there might be a few features that are already imported and/or useless)

[dense estuary]

Projects that depend on bevy_seedling aren't impacted

Because dev-dependencies only applies to the workspace, and examples are part of the workspace

[slate scarab]

Oh I see, right -- this would be an easy way to test the builds in CI.

Yeah, I don't care to keep the rayon feature.

[dense estuary]

I've sent a PR, up to you to check, then approve it!

[slate scarab]

Thanks again! I've published a new version with the fix.

[slate scarab]

Okay, @dusky mirage, I've more or less wrapped up the new Patch API. Let me know what you think!

I went ahead and simplified enums, which made the macros quite a bit simpler! Feel free to merge your sampler node PR first -- I can resolve any conflicts in my PR after you've merged it if you'd like.

[dusky mirage]

Looks good to me! I merged the sampler node PR, and there are some merge conflicts.

[dense estuary]

You're welcome! 🔥

[dense estuary]

I have just fixed another platform-specific error, but now on Android 😁
Well, the issue is a missing feature, but the amount of pain, debug and research to fix it is quite horrifying 😅 .
The solution is to enable an android feature for android builds, as anyway bevy_seedling always need it, even without default features.
I'll submit a PR soon!

[faint wigeon]

is audio input supported yet? is there an example of an input node somewhere?

[dense estuary]

By stating "audio input", do you mean talking and having the audio you put in recording?

[faint wigeon]

i mean being able to read from an input stream of an audio device

for my use case, back into the ecs somehow, not just as part of the dsp graph

audio reactivity basically

[slate scarab]

Ya there's a few ways you could do that. I believe that's exactly what the stream reader node does.

It's probably worth making an example for it though, since you need to access the audio context directly at least once to get the stream handle.

[faint wigeon]

okay sweet, and i'm totally happy to help/contribute here as i realize this is more niche

let me play around a bit and get more familiar

i'm really excited about this work

[dense estuary]

@dusky mirage bevy_seedling currently requires the cpal/oboe-shared-stdcxx feature to run on Android.

I’d love to hear your thoughts on how you'd prefer to expose this kind of feature.

My initial idea is to introduce a feature flag called android_shared_stdcxx in Firewheel, which would:

• Enable the same feature in firewheel-cpal,
• Which in turn would enable cpal/oboe-shared-stdcxx.

Do you think that fits with your vision for Firewheel’s feature structure? Or would you structure it differently?

Totally understand if you're focused on other priorities—I’m happy to make a PR myself if exposing this kind of feature meets Firewheel design!

[slate scarab]

This is how we exposed the wasm-bindgen feature, so at the very least there's some precedent!

[dense estuary]

Nice!

I'll make a PR tomorrow then 🔥

[dusky mirage]

I'll check it out when I'm back at my computer tomorrow.

[dense estuary]

One big issue is that CPAL will move from oboe to ndk (commit of this change)

[dense estuary]

An idea will be to integrate to bevy_seedling a build script, that will do, on Android, this: println!("cargo:rustc-link-lib=dylib=c++_shared");. If the issue concerns Firewheel itself too, add this line to its build script and remove one from bevy_seedling.
So there are three options:

• this line in build.rs isn't working for all cases on Android: we'll have to search another solution
• only bevy_seedling needs a C++ shared STL: line added to bevy_seedling build script
• Firewheel needs a C++ shared STL: add the line only to Firewheel build script and not to bevy_seedling

note: the "line" refers to "println!("cargo:rustc-link-lib=dylib=c++_shared");" in a build script, only for android targets, so final implementation would look like:

fn main() {
    let target_os = std::env::var("CARGO_CFG_TARGET_OS").unwrap();
    if target_os == "android" {
        println!("cargo:rustc-link-lib=dylib=c++_shared");
    }
}

second note: I've just tested this, and this works fine, even without android-shared-stdcxx

[dense estuary]

@slate scarab Finally, C++ shared STL probably won't be needed anymore, as CPAL changes from oboe to ndk::audio and I've just understood that a "pure-Rust" solution will close this need of a C++ runtime. Meanwhile, for a temporal solution we still can use this build script, until cpal won't fix all errors or merge my PR and publish a new version.

fn main() {
    let target_os = std::env::var("CARGO_CFG_TARGET_OS").unwrap();
    if target_os == "android" {
        // This includes 'lib/<arch>/libc++_shared.so'
        // Firewheel on Android will always need C++ shared STL
        // It may be fixed with following CPAL release!
        println!("cargo:rustc-link-lib=dylib=c++_shared");
    }
}

[slate scarab]

I think the responsibility of the build script would lie with Firewheel.

However, keep in mind that cpal has not published a version with this new addition, so we can't use it yet. Also, it looks like they might keep oboe around for a while for legacy android users.

So with that said, I think the original plan is still ideal: add a feature flag to Firewheel for oboe that we can enable in bevy_seedling.

[dense estuary]

For oboe, or for a build script? A build script might be lighter and more independent

[slate scarab]

Oh yeah I think a build script is a better long-term solution, but we can't use it at all yet!

[dense estuary]

Why can't we use a build script in firewheel-cpal?

[slate scarab]

You're saying this is independent of cpal's changes away from oboe? In other words, this build script works now with 0.15.3?

[dense estuary]

Yes

[slate scarab]

Is this still true?

this line in build.rs isn't working for all cases on Android: we'll have to search another solution

Or are you saying it's a possibility?

[dense estuary]

Oh I have forgotten an "if"

100% my fault, sorry

Yes, it's working for all 4 Android targets 👍

[slate scarab]

No worries!
I just think we probably don't need to put too much bespoke effort in at the moment. Are you absolutely sure that your approach with a build script is a robust solution? Are you willing to put further maintenance effort in if it breaks?

The alternative is simple and proven. We can work with cpal as it is now (use the oboe flag) and then move to whatever cpal needs when they drop oboe in the future. cpal has a pretty sluggish maintenance pace, so I doubt we'll see a new version published for a while anyway.

[dense estuary]

I'll maintain the build.rs and potentially replace it by another thing, but for now I think it's robust, see my PR.

https://github.com/BillyDM/Firewheel/pull/38

Anyways, I'm developing a cross-platform game, that aims to exist on Android, iOS, MacOS, Linux, Windows, VR, Web and exotic platforms. I kinda have to watch on all of this stuff and do a lot of testing and research. So hopefully, I'll catch bugs if they will occur.

[dense estuary]

I wasn't really explicit: yeah, I wish to put further maintenance effort in it if it breaks and I'm sure at 85% that it's robust enough.

[slate scarab]

So part of why I don't like a build script this far up in the stack is that it doesn't replicate the behavior of what the flag does in oboe-sys. See here.

This seems to give users the option to statically link. Not sure who that's for, but I'm sure they put it there for a reason.

[dense estuary]

Oh yeah that's the issue

[slate scarab]

If we drill the flag through Firewheel, we can make it a default feature in bevy_seedling, which should make it work for most users out of the box while still allowing static linking.

[dense estuary]

By default, it's linking statically, but we can define oboe/libstdcxx (I don't really remember the feature) to use a shared C++ runtime. And the static one is optimized away so some symbols can't be located at runtime and Android app crashes. To avoid it, the script imports the shared C++ runtime (which is recommended by google as it even reduces the binary size and is consistent)

[dense estuary]

But firewheel itself doesn't work without this on Android

[slate scarab]

Is that necessarily the case? Is it possible for users to build android apps successfully with static linking?

This is somewhat rhetorical -- if it weren't possible, why does oboe expose the feature at all?

[dense estuary]

Yes, to have it working without Firewheel or bevy_seedling implications they will need to import the shared C++ runtime themselves or use an oboe feature or have their cc linker actually knowing that some symbols must not be erazed (which is kinda difficult)

Please tell me if I'm not clear about it. C++ static is basically erazed/optimized by the linker, but because of some internal/hidden libraries, FFI and other stuff, the linker erazes essential and needed symbols. That's why, it's generally better to import a shared C++ runtime, because it's always safer and more consistent, as it's not erazed, but imported as it is, from the google ndk (not sure about it), so there can't be an error "can't locate symbol XXX"

It seems that C++ shared runtime is indeed from the ndk

[slate scarab]

Sure, I think that makes sense. I still don't see why we should bypass the mechanisms oboe's authors put in place to handle this.

Using feature flags is simple, easy, and perfectly flexible. The long-term maintenance costs are very low. If we make the feature default in either Firewheel or bevy_seedling, most users will never run into errors.

If you enable the feature, you're not running into build errors, are you?

[dense estuary]

feature of type android-shared-stdcxx which will enable the build script, or oboe feature?

[dense estuary]

the feature, you mean oboe/shared-stdcxx?

[slate scarab]

The oboe feature itself. The one cpal exposes here https://github.com/RustAudio/cpal/blob/eb3d44542c5ea8ab382a2eae74de7e91415f7ac6/Cargo.toml#L14

[dense estuary]

oboe's code is the same as mine, it allows just to import static android shared C++ runtime too

[slate scarab]

Right, so my question is why duplicate it when we can use a feature flag that's more robust?

[dense estuary]

Because oboe won't be used by CPAL at a moment, and it adds a dependency. It's a question of preference, if you wish to use oboe, there's no problems with that

[slate scarab]

If I understand correctly, we'd have to remove the build script once cpal drops oboe anyway

Pure rust implementation, so libc++_shared.so is not required anymore.

https://github.com/RustAudio/cpal/pull/961

If the build script were in firewheel-cpal, you're correct that this would be less work to remove since no users would need to remove the feature.

[dense estuary]

Yeah

[slate scarab]

But it could be a year before that happens! And in the meantime, I (personally) wouldn't like the build script solution. But it's not my decision anyway (not my crate 😅)

[dense estuary]

I'm kinda surprised, as oboe seems to do println!("cargo:rustc-link-lib=dylib=c++_shared");, but in a modular way (feature-gated way)

[slate scarab]

Okay @dusky mirage , I've marked my patch API change PR as ready to review. Feel free to take a look whenever you have the time.

As I mentioned in the PR, I ran into some issues reliably detecting playback completion, so I added another flag to help with this. If you have better ideas here, let me know!

There's a lot of breaking changes in here, so it might be good to implement sample speed modulation before creating a new release. Once that's all done, though, bevy_seedling will have the full set of sample playback APIs!

[dusky mirage]

Yeah, I think it would be better to have the sampler stuff be in a separate PR.

[drowsy dome]

Hallo :) I'm wondering if there's a design document/overview about goals for better bevy audio in progress. I noticed the tag, but unsure. One of the questions I have is, is there a consensus on building towards firewheel and bevy_seedling? idk if it's still wait-and-see, or if there is still the intention to adopt kira.

Reason I'm asking is because I've been looking to refactor my work on bevy_midix. It'd be so cool to be able to

1. stream real-time MIDI commands -> SoundFont -> bevy_seedling/bevy_audio/bevy_kira_audio
   However, I'm unsure about what better audio entails. Whether I should abstract on top of bevy now or directly interface with flywheel.

Additionally, my use-case probably moves past the idea of playing back audio into generating it on the fly. Would the idea of audio generation via soundfonts be beyond this scope/is this orthogonal to the current consensus about better audio?

[celest whale]

I'm leaning heavily towards firewheel personally, but I think we need to wait for the demo to make a final call

[faint wigeon]

i'm also quite interested in midi for creative tech purposes, but i wouldn't expect midi to be integrated directly with the audio system (or really to be supported directly in bevy at all)

i am not 100% up to speed with firewheel but my understanding is that it should be possible to do this kind of integration yourself, however

[slate scarab]

Luckily those changes are pretty self-contained, so I removed them and I'll follow up later with it.

[slate scarab]

It's definitely "wait and see" for bevy_seedling.

As for generating audio with soundfonts, that seems like a great use-case for a third-party Firewheel crate!

Firewheel is designed to be engine-agnostic, so you wouldn't need to be tied specifically to Bevy. But, at the same time, bevy_seedling makes it really easy to add Firewheel-based processors to your Bevy games.

So while there isn't a guarantee bevy_seedling will be adopted, feel free to check it out! Feedback at this stage is very valuable, and it should help Firewheel's development as well.

[drowsy dome]

I like it! Now just need to determine if bevy_midix interfaces with bevy_seedling or Firewheel...I think that was really my question after digging into it more. I think the question of "what is bevy_audio supposed to do" is still unresolved, and if bevy_seedling is trying to fit that mold, then I guess bevy_audio's design is being driven by this crate, not the other way arround. So then I guess it's a bit of a free-for-all to vibe out what will be exposed in the end.

I was messing with it for a few hours this AM, it's got a great interface. Way better than my fork of rustysynth

Will probably chop off the back half of this fork and shove Firewheel in front of it to make this work, and then see where we're at as design for bevy_seedling is fleshed out over the next few months.

[dusky mirage]

merged!

[slate scarab]

Awesome! Sorry for the huge diff 😅 I think the diffing API is pretty near optimal at this point, so it shouldn't need any more big changes.

[dusky mirage]

Hmm, my idea for reliably detecting when a sequence has completed using atomics didn't work. Solving this is going to be tricky.

[slate scarab]

Here's my best idea.

We could wrap the sequence in a Notify. We add an id method to notify that returns the value of its counter. When the sequence completes, the sampler node writes the sequence's ID to a special finished flag.

The game logic can then look at the current sequence ID, compare it to the finished value, and determine whether the sequence has completed.

We can also reserve 0 in the Notify implementation so that the game-engine side can clear the finished atomic when it begins playback. That way, you could play the same sequence multiple times while still reliably detecting completion.

[lament briar]

do you have a task queue or something?

[slate scarab]

In bevy_seedling yes -- it manages sample pool allocation in the ECS.

The trick is per-sampler completion detection without requiring allocation on the sampler side.

[lament briar]

Idk if this is helpful, but this immediately reminded me of work I just did for parallel transform propagation. I used an atomic counter to figure out when a task pool had completed all work. If that sounds relevant, you could steal the code on bevy main.

[lament briar]

Sorry if OT, this comment made me think it might be a related problem.

[steep dove]

(worth mentioning that forte supports something very similar with task scopes).

[dusky mirage]

Hmm, there might also be a way to use two counters. One that counts how many times the processor has received a "play" event, and one for how many times the processor finishes a sequence. The handle simply sees if the "finish" counter has increased to know if a sequence has stopped. But if it sees that the "finish" counter has increased but the "play" counter has not, then the sampler pool knows it wasn't the latest sequence that finished.

[steep dove]

why not just increment and decrement the same counter?

atomics don't have to be monotonic

[dusky mirage]

Hmm, that's not a bad idea either.

[slate scarab]

ya i see no problems with that

[dusky mirage]

Yeah, that would be much simpler if it did work.

[slate scarab]

Okay I lied -- what happens when a sequence never finishes?

[steep dove]

for forte, at one point I had to determine when there are any remaining tasks before a pool could shut down and I did it this way. You just need to make it a hard requirement that there is a 1-to-1 match. for every call to inc there must be one call to dec.

[steep dove]

nothing ever happened forever - everything eventually needs a graceful shutdown path.

[dusky mirage]

I imagine we could also decrement the counter when the processor recieves a "stop" event.

[slate scarab]

Just to clarify here -- a sample could be replaced before it actually finishes. It would be interrupted.

[steep dove]

then you decrement the counter when it's interrupted

wait, is this for allocation/de-allocation?

[dusky mirage]

Essentially, we are keeping track of how many times the processor recieves a "play" event and how many times it either recieves a "stop" event or it finishes a sequence.

[steep dove]

so that you can do what?

[slate scarab]

Right now, bevy audio gives users a few options for what happens when a sample finishes playback. i.e. despawn the entity, remove the audio components, or leave it alone. I actually like this, and I brought it to bevy_seedling. In order for this to work, we need to be able to know when a sample completes playback (or is stopped).

[steep dove]

it sounds like you want is pretty much is Arc you just want to control the semantics of allocation/deallocation then.

[slate scarab]

This is kinda separate from sample pool selection. We already have heuristics for choosing the best sampler.

[dusky mirage]

Essentially, the problem is that there is a delay between sending an event and the processor recieving that event. There could be a situation where you send a "play" event, and the processor finishes an old sequence before it recieves the play event.

If you are polling a flag to see if the sequence has finished, it could appear as though the new sequence has finished even though it hasn't started.

[slate scarab]

What's nice about this approach is that users don't need to do any extra bookkeeping. You don't have to have the sampler state handy. To me it' a nice balance between ease of use and correctness.

In the scenario I described here

A stopped flag is not enough information to determine whether a sequence has completed. There is some delay between starting playback in the ECS and the stopped flag being cleared, so we might accidentally remove a sample before it's even started playing. To mitigate this, we could poll the flag in the ECS, waiting for it to be false and then storing the fact that playback has started, but this mechanism cannot handle samples that last for less time than the polling frequency.

We'd easily handle both issues.

[dusky mirage]

I guess whatever you think is best. I've been occupied with other projects lately, so I haven't had much time to focus on Firewheel.

[slate scarab]

I'll give it a shot and see how it feels. In terms of allocation costs, it's luckily no different from how it is currently (The option forces the whole sequence into a box either way.)

[drowsy dome]

#general message Y'all might like this demo

definitely would love some recommendations on a clean macro for midi track if you wanna write it instead of including a file. midix can handle both

[drowsy dome]

so

I've come back with this

https://github.com/dsgallups/midix/blob/1aef0c414efa32de5f544f4af7dcc5b3c7cf82f6/bevy_midix/examples/programmatic_song.rs#L50-L54

And from here, hopefully can integrate with Firewheel with an audiofont processor (rustysynth has been giving me problems)

[tender fiber]

Adding a .mid file is so much easier than more code work so adding .mid features would be a better path imo.

Like midi 2

Rather than coding another way to get midi in
…
redundant imho

and more work

Let’s just say midi
vs sf2
I pick midi never sf2
Add a player ok…sound output
Now wha is cool?
Adding midi protocol to get cheesy sustain release etc …

And mockup is done

And all that was added was midi protocol
That’s wha
Music composers do

Sure they automate everything.
However, it is a great tool to shape sound without the automation even if it is cheesy.

And it works

You just add midi protocol to be able to change the midi file

Attack Sustain Decay Release

Et cetera

The other alternative
Well, it just plays the midi and that sounds like a machine

With 127 volume every note

Manipulating volume is main midi protocol use and your manual input of code is great for that if it included ASDR volume et cetera like midi protocol, a way to change the notes

[drowsy dome]

what

I kinda understand what you're saying. I'm gonna permute that idea and do both. imo, you need my former idea for your latter idea. Funnel both .mid and programmatic events into a MidiSink. This should work by allowing commands to be queued ahead of time. The user should be able to add your aforementioned modifiers that will directly affect the command when the sink pushes an event to the synth.

Note the example is intentionally simple hehe, like you can't issue messages on an instant midi tick intentionally. It's just explorative for now

edit: also, realize that my project is growing away from bevy audio, not towards it like I'd initially thought. So I'm going to keep this discussion out of this channel in the future unless I have something to contribute directly to bevy_seedling. Sorry!

[tender fiber]

Firewheel will be able to handle virtual CLAP instruments and effects so midi node could read midi notes from midi file UI as an event list to tweak values (volume, ASDR) should be possible

[drowsy dome]

whelp, actually I'm interested in building the CLAP plugin node for firewheel. I commented in the issue. Any good places to start?

AFAIK, if I have a synth/clap plugin that processes samples, I'd pass its buf into StreamReaderState. Unsure the steps in between for CLAP -> CLAP instruments/reasonable implementation. A good foothold/reference/overview of ideas of things I would need to use to get there would be super awesome!

@slate scarab would it be smart to open with a ClapHostNode (and starting small, emulate the stream_nodes example)? Is the idea that, due to the graph, you can forego implementation on an Output stream by connecting that node's vertex with the OutputStreamNode?

asking before I start making my mess...

[slate scarab]

Hm, I’m not totally sure I follow. What’s the alternative here?

[drowsy dome]

I'm unsure. From what I've read, seems like a single node could host all the plugin functionality in theory, but in practice, I don't know. Just was asking essentially for validation

but if I do that, then essentially there may need to be a higher layer of nodes for this node. i.e. voices, tracks, tuning, etc.

those probably don't fit anywhere in firewheel's graph directly because they're not audio sources

ah I might still be being vague. here is a list of the "fundamental" extensions for the plugin, which, except for audio ports, may not apply to Firewheel.

the implication here is that, since clack only has wrappers, someone would probably need to make a higher level API for this to work.

[slate scarab]

Right, in other words things like note ports couldn’t send information easily between Firewheel nodes. You’re thinking of an API where a single Firewheel node holds onto a chain of CLAP plugins.

[drowsy dome]

indeed

[slate scarab]

Yeah idk that seems reasonable

[drowsy dome]

like...I'm down. This essentially is analagous to my MIDI stack.

Cool, I'll get cracking

[faint wigeon]

have you looked at https://github.com/robbert-vdh/nih-plug ?

[oak walrus]

nih-plug focuses on making plugins, not hosts

https://github.com/prokopyl/clack is lower-level (and what nih-plug uses) and you can do hosts with it

This would be what I'd use to make the CLAP node

[faint wigeon]

ah gotcha! makes sense

[dusky mirage]

@slate scarab I've implemented the "sampler speed" parameter to change the speed/pitch of a sample https://github.com/BillyDM/Firewheel/tree/sampler_speed. Though it's not completely working yet, there is a weird buzzing noise. I'll investigate the bug sometime later (there might be an off-by-one error somewhere).

Oh, and apparently looping isn't working either.

[slate scarab]

Oh awesome! I'm very much anticipating it for a few of my own projects already

[dusky mirage]

Alright, I fixed the buzzing, and I also adding smoothing to the speed parameter.

[dusky mirage]

@slate scarab Hmm, we might have a little problem. Apparently the looping wasn't working because destructuring an enum like this doesn't trigger the Notify for the sequence.

let sampler = &mut self.samplers[sampler_i];

let Some(SequenceType::SingleSample {
    volume: old_volume,
    repeat_mode: old_repeat_mode,
    ..
}) = sampler.params.sequence.as_mut()
else {
    todo!();
};

*old_volume = Volume::Linear(linear_volume);
*old_repeat_mode = repeat_mode;

sampler.params.start_or_restart(None);

sampler
    .params
    .update_memo(&mut self.cx.event_queue(sampler.node_id));

I had to add a method to Notify and call it manually to get it to work

let sampler = &mut self.samplers[sampler_i];

let Some(SequenceType::SingleSample {
    volume: old_volume,
    repeat_mode: old_repeat_mode,
    ..
}) = sampler.params.sequence.as_mut()
else {
    todo!();
};

*old_volume = Volume::Linear(linear_volume);
*old_repeat_mode = repeat_mode;
    
sampler.params.sequence.notify();

sampler.params.start_or_restart(None);

sampler
    .params
    .update_memo(&mut self.cx.event_queue(sampler.node_id));

In any case, the PR is ready. Just let me know if you are happy with the API or we need to tweak it some more. https://github.com/BillyDM/Firewheel/pull/42

[slate scarab]

Ah, sorry about that! We might even just set as_mut to increment the counter directly -- that's probably the most robust long-term solution (unless you have any objections).

[dusky mirage]

Yeah, that's probably the best solution.

I'll just go ahead and add that change then.

[wary bridge]

I've been casually converting my work's audio pipeline to rust+firewheel, and I think it would be useful at some later point to split off ClockSeconds and ClockSamples into a standalone audo units crate. I'm also making use of these two wrappers that might be worth including in said crate:

pub struct SampleRate(NonZeroU32);
pub struct ClockNanos(u64);

[dense estuary]

Oh I wanted to update my project to bevy 0.16 but it has so much dependencies...

2, actually 😂
edit: 2 for a basic desktop build, without debug and abstracted input, that's the first step I'll do before updating other modules

[slate scarab]

Yeah that tends to be annoying right after release 😅

I think once Firewheel has sample playback speed merged in, I'll take some time to update bevy_seedling to 0.16.

Unforunately, this may involve some moderate breakage. I'm almost certain I can leverage the new spawn APIs to strip out some of the existing bespoke APIs around pools and effects.

[dusky mirage]

@slate scarab I've managed to remove the max_speed limitation for the sampler by using the scratch buffers. I also went ahead and merged that PR.

Implementing a custom resampler that can dynamically change speed was a good exercise. I feel much more equipped to create a sampling engine for my DAW engine now.

[hasty cradle]

@slate scarab is bevy_seedling about to be merged into bevy?
Asking because:

1. You recommended it on the other channel
2. If I started migrating out of kira today it'll take me between two and three days to replace all the logic
3. The bigger my project gets, the more difficult it becomes to take new directions that require u-turns

I would love to use the bevy_seedling features if I knew that similar ones (or, ones with similar approach) would be embedded into the bevy engine eventually

[slate scarab]

No, there’s no guarantee it’ll be merged at all! If you’re worried about the amount of time it’ll take to refactor, it’s probably best to wait on bevy_seedling for now.

I’m not sure what the merging timelines will be, but I have a feeling things will be more clear in a couple months.

[oak walrus]

Though, my 2cc is, I feel we're at a stage where we would like to have more user feedback on the design and usage (and performance as well)

so while it's obviously not production ready and should not use bevy_seedling in actual production projects, it would be nice for people to spend a bit of time tring the new library

[hasty cradle]

Thanks

[hasty cradle]

I might try it for a jam (not sure I'll join the next one but I might)

Btw I didn't know you were looking for testers, maybe you should post it somewhere?

Maybe under #crates ?

[slate scarab]

Yeah, I've thought about something like that. I might do that once it's migrated to 0.16.

[celest whale]

Yes please to jam testing. As a maintainer, I want to see some user feedback, and the competitive demo blog post from Firewheel

[hasty cradle]

@slate scarab does bevy_seedling offer any enhancement regarding audio play speed controls?
I began migrating back to raw bevy audio (from kira) and one of the main problems I encountered was that when I tweak audio instances it doesn't work very well.
My game features many at-will time speed tweaks and I'm trying to make the audio work with that, so far with no real success

[slate scarab]

Hm, I'm curious what you mean there.

What problems are you running into?

[hasty cradle]

Wow you're quick

Want to come for a voice call or something so I'd be able to share my screen?

[slate scarab]

I'm on Discord for work and other things all day 😅

[slate scarab]

mm not atm cause work

[hasty cradle]

haha understandable

ok so I'll try to clarify

In my game you can slow down time at will. There are many systems built around it, including a TimeScalerSubscriber component. I make sure that every AudioSink and SpatialAudioSink barrer plays at the correct speed when the time scalers change:

        for time_scaler in &changed_time_scalers {
            if time_scaler.id() == subscriber.scaler {
                audio_sink.set_speed(time_scaler.value());
            }
        }
    }```

On kira I did it by tweaking the channel's speed
Here the audio makes these weird jumps (I guess it doesn't set it's speed frequently enough?) and then the sound is also not synced very well
For example, if I trigger a sound when the time is slowed down and then bring time back to it's regular speed the sample itself would finish long after when it should

(Was it clearer now?)

[tender fiber]

You can use a slowed time samples and normal time samples. Easy answer but you increase sound files x2.

[hasty cradle]

It wouldn't work because the transition won't be smooth.
Right now it basically sounds like a few "samples" are replaced, the problem is that they're not being replaced quickly enough, resulting in audio lags

[slate scarab]

This should work well with Firewheel.

Modulating the speed will push updates to the audio context once every frame. If frames are faster than the audio processing rate (a buffer of 1000 samples will update at around 44 times a second), then updates will be processed a bit slower than the framerate.

Given the architecture, these updates cannot arrive out of sync (assuming you write the speed to all sample players within a single frame), so you shouldn't see any desynchronization.

The speed is smoothed a bit in the audio processor to avoid clicks.

[hasty cradle]

Thanks
Then
I might give your crate a go in my main project after all

btw I saw that there's no support channel for it?

At least I couldn't find any

[slate scarab]

I can get that going as soon as its ready for it. I'm still partway through my 0.16 migration, and the current published version doesn't actually have the sample speed parameter!

[hasty cradle]

I mean, if you put it on a public github branch I could just take the version from that branch

That way I'll have the solution faster and you'll have a "playtester" faster haha

[slate scarab]

Are you planning to (or have you already) migrated to 0.16 with your project?

[hasty cradle]

The project is far from ready for 0.16 though, most of the crates I use haven't been updated yet (including my own crate)

[slate scarab]

Hm, that's a touch tricky. Well, honestly I could just backport the controls and speed code to the 0.15 version. It's very little code.

[hasty cradle]

If you can do that and put that on a separate branch- I'd gladly connect to it and try it out

These are my dependencies, I think some of them would take at least another week to migrate to 0.16 and then I'd still have to migrate the project
I'll start using your crate right after though, if all goes well

[slate scarab]

I think there's a working PR for bevy_tween up -- not sure about the others though.

[hasty cradle]

I know, I opened it haha
But we're waiting on that one for it's own dependencies

Anyway feel free to ping me here or write to me directly when the branch with the speed setting is ready

[thick orbit]

Hey! I'm trying to use bevy_seedling with 0.16, but even though I don't have bevy_audio feature, somehow I get the conflict in code between the types(e.g. Volume).
I checked bevy Cargo.toml, and I don't see any other feature bringing in bevy_audio. I'm confused, maybe something changed in 0.16?

Edit: nevermind, just checked cargo tree and apparently bevy_audio is brought to you by bevy_animation(which is weird)
Edit2: I'm stupid, it's not brought in by animation, just brough in by bevy_internal, when the bevy_audio is certainly disabled
Edit3: nevermind I had vorbis feature enabled 🤦‍♂️

[slate scarab]

I'm actually surprised that's the error! I'm mostly through porting to 0.16. It's taking a bit because I'm adjusting some of the APIs and improving things along the way.

[thick orbit]

I somehow forgot that seedling also needs to be ported. Will be waiting to try then!
Or having spent some time porting my template, if there are any modules I can help porting, I'll be more than happy to do it!

[slate scarab]

I'm actually more or less done now! I just need to finish up some amount of documentation and do some more testing.

[rapid musk]

I've been working on a puzzle game and I want to incorporate sound once the porting is done. Is it possible to restrict the playback of a sample to only a portion of the sample? Similar to how you can pass a Rect to a Sprite to only render a portion of that sprite.

[slate scarab]

Firewheel currently has the interface for doing something like that, but it’s not yet implemented. It might be a little bit before it makes it in!

[dusky mirage]

Oh yeah, that should be simple to implement. I'll work on that tomorrow.

[dusky mirage]

TIL that Rust doesn't auto-vectorize float comparisons optimally. So I went ahead and added a peak detection algorithm with manual intrinsics on x86. I haven't made one for ARM yet. https://github.com/BillyDM/Firewheel/blob/main/crates/firewheel-nodes/src/peak_meter.rs#L299

[dusky mirage]

Do you think it would be best to have this range be in units of seconds, in units of samples, or in the normalized range [0.0, 1.0] where 0.0 is the start of the sample and 1.0 is the end?

[slate scarab]

personally I wouldn't mind an enum over samples and seconds

[rapid musk]

Yeah an enum over samples and seconds sounds good, if I need to normalize it to a range I can easily do that myself. It's not often I need/want a normalized range anyway, usually only when I have a control of some sort like a position marker in a beatmap editor.

[dusky mirage]

Also, I was thinking that the range could be part of SequenceType, though it would mean you would have to restart the sample if you wanted to change the range. In theory it could be possible to have it as its own parameter, but that would make things more complicated.

Though if you think it is worthwhile to have it as an independent parameter, then we could do that. Although the user would have to also make sure that they reset the range if they wanted to reuse the sampler for a different sample.

[rapid musk]

Out of curiosity, can I have multiple ranges for the same sample? For example, say I had some sample and I wanted to play 3 different portions of that sample in any order, or perhaps randomly pick one of the three, would that work?

I think my use case might be very niche though, as I was playing around with making an audio version of a sliding block puzzle, to see if that would work out somehow

It started as a simple project to see how easy it would be for me to take any image and cut it up into a 3x3, or 4x4 grid and jumble them around to create a sliding block puzzle, once I had that working I started wondering if I could do the same for audio which is why I asked about replaying only specific parts of a sample

[dusky mirage]

Could you just split the audio file into three audio files?

[slate scarab]

seamless sequencing might be a little tricky with that setup

I think it would still be possible though

[rapid musk]

I could yeah, but it'd be nice if I could do it in code and apply it to any audio file without having to split it up. I was thinking about simply doing sample amount / n and then playing around with that to see if it's possible to create a sliding block style puzzle, but for audio by having the listener try to figure out which should play after other.

But at the same time, I understand if this is a very niche thing so I wouldn't want you to spend a bunch of time on it as I don't expect many people would use it

[slate scarab]

Depending on how much control you need, that could also be a great candidate for a custom node! Although managing clicks at loop points is always annoying

[rapid musk]

Yeah, I'd have to figure out a way to blend the start / ends together a little

But being limited to one range is still useful for rhythm games though, there it is very common to start playing the song somewhere in the middle and then replay it after 10-15s

Being able to specify that you want a song to start playing from 30s to 50s then start over again from 30s is pretty much something all rhythm games would use in the level select screen

[dusky mirage]

Ah, someone in the Rust discord server discovered that you can get the Rust compiler to properly autovectorize the loop by processing in chunks like this. Much nicer. https://github.com/BillyDM/Firewheel/blob/main/crates/firewheel-nodes/src/peak_meter.rs#L273

I moved the function to firewheel_core::dsp https://github.com/BillyDM/Firewheel/blob/main/crates/firewheel-core/src/dsp/algo.rs#L4

[dusky mirage]

Hmm, it turns out adding loop ranges to the sampler is going to be a lot trickier than I thought if we want to have proper loop crossfading. I may need to just end up rewriting the whole sampler engine using a state machine instead. This is probably something I will do eventually since I need a similar sampler engine for my DAW anyway, but I'll do that later when i find the time and motivation.

It may even be worth splitting out the sampler engine into a separate crate. I'll think on it.

[slate scarab]

Hm, I might round out the simple music-specific sampling node I was working on a while ago in the meantime. It's definitely not nearly as performant and wouldn't be as flexible (no resampling) but it might be a nice stopgap.

[rapid hedge]

How is the new audio stuff integrating with Web at the moment? Current Bevy audio is supremely choppy there for a while when starting (run e.g. https://janhohenheim.itch.io/foxtrot to see this effect when starting the game).

[slate scarab]

Right now it’s no different.

However, one of my top priorities is providing a proper web audio API backend. This will require a nightly compiler (for wasm atomics) and special response headers to allow shared array buffers.

The headers may be difficult to provide on itch.io.

I’ve built such a backend before, so I know how to do it. It’s not too bad!

[rapid hedge]

That would be neat, great! itch has this checkbox, FYI

[slate scarab]

Oh, looks like we’re in business

[rapid hedge]

Also nightly sounds fine since I believe many people are already using that exclusively for Bevy due to compile time improvements

[rapid hedge]

https://itch.io/t/2025776/experimental-sharedarraybuffer-support

[hasty cradle]

looks like I can't secretly try out the 0.16 in-the-works version

[slate scarab]

Hm, I think it should work! Just put the branch as a separate value (i.e. branch = “bevy-0.16”)

[hasty cradle]

oh my bad

Would you say it's... kinda stable as of now?

I do want to give it a try, I just don't want the project to have surprise crashes

[slate scarab]

Yes. I’m in the process of moving, so my timelines have been a bit off. However, I don’t plan to make any more significant API changes.

The examples should give some pointers on the changes — I’ll have a full migration guide when it’s published.

[hasty cradle]

So that's the main thing on my list now, though I'll probably only get back to the game on Wednesday. Will let you know how it goes

[thick orbit]

sorry to bother, but currently seem to be broken?..

error[E0599]: no function or associated item named `from_os_rng` found for struct `SmallRng` in the current scope
   --> /home/pickle/.cargo/git/checkouts/bevy_seedling-854d7bc153e83485/87bb42f/src/sample/mod.rs:423:52
    |
423 |             app.insert_resource(PitchRng(SmallRng::from_os_rng()));
    |                                                    ^^^^^^^^^^^ function or associated item not found in `SmallRng`
    |
help: there is an associated function `from_rng` with a similar name
   --> /home/pickle/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/rand_core-0.9.3/src/lib.rs:521:5
    |
521 |     fn from_rng(rng: &mut impl RngCore) -> Self {
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

# Cargo.toml
bevy_seedling = "0.3"
[patch.crates-io]
bevy_seedling = { git = "https://github.com/CorvusPrudens/bevy_seedling",  branch = "bevy-0.16" }

[celest whale]

Ran into a member of this group at RustWeek 🙂 Anyone else here?

[hasty cradle]

(On the 0.16 branch) I'm having problems with spatial audio. I tried changing the damping factor and the scaling but it still dampens by pixels.

[slate scarab]

I wish if it were held just a couple weeks later, I could have hopped on a train to get there!

[slate scarab]

The damping factor will only affect the fall off curve, but it won't change the scale. Have you tried placing a SpatialScale component on the effect? Alternatively, you could change the DefaultSpatialScale.

[slate scarab]

Sorry about that -- the os_rng feature must have been transitively enabled in my projects. I pushed a commit to fix that. Let me know if that resolves the issue!

[hasty cradle]

I did add a SpatialScale component but I'll try setting the default one

[slate scarab]

Was it to the effect itself, or the outer component? i.e. it'll want it like

commands.spawn((
    SamplePlayer::new(server.load("sample.wav")),
    sample_effects![(
        SpatialBasicNode::default(),
        SpatialScale(Vec3::splat(0.5)),
    )]
));

[hasty cradle]

ohhhhh

Thanks

[hasty cradle]

I decided that a more correct approach for me would be using the default one

but1

Doing .insert_resource(DefaultSpatialScale(SpatialScale(Vec3::splat( DEFAULT_SPATIAL_SCALE, ))))
fails because the SpatialScale field is private

I couldn't find any impl block

Unless!
It's configurable from the main plugin and I forgot to check

Nope, couldn't find anything like that

[slate scarab]

Sorry, the docs definitely need to be a bit clearer (or maybe I'll update the API).

It already exists. You just need to mutate it. It implements Deref to a SpatialScale.

[hasty cradle]

Oh I see
Thanks

It worksss

[hasty cradle]

What's the correct way to set a playback's volume after it's been spawned? I tried changing both PlaybackSettings and adding a VolumeNode and changing it

[celest whale]

@wide epoch over here!

There's a Discord server link pinned here too

[drowsy dome]

this crate is wonderful

[drowsy dome]

I wonder...is there any way to spawn a SamplePlayer paused? I don't see a config for that in PlaybackSettings

edit: seems like there's something with SamplerState, just don't see that it can be set to Paused from the get-go...I'd be down to implement this if there's no aversion

[wide epoch]

very cool! will take a closer look once less busy with conference things tomorrow

[slate scarab]

I’ll explain this properly when I finish the docs, but there are two sample player structs: PlaybackSettings and PlaybackParams. The former is only applied once at the beginning of playback, while the latter can generate events during playback.

The sample volume param exists inside the PlaybackSettings. In other words, it can only be set once! So indeed, the only way to dynamically update the volume is with a volume node.

How did you add a volume node, and what was the system you used to change it?

[slate scarab]

ya!

commands.spawn((
    SamplePlayer::new(server.load("sample.wav")),
    PlaybackParams {
        playback: Notify::new(PlaybackState::Pause),
        ..Default::default()
    },
));

or if you prefer a more imperative approach

commands.spawn((
    SamplePlayer::new(server.load("sample.wav")),
    {
        let mut params = PlaybackParams::default();
        params.pause();
        params
    },
));

[drowsy dome]

cannot believe I missed that

thank you!

this might be an even tougher question...is there any method to get an instantaneous offset from the start of the song without creating a custom node and thread? I feel like that's somewhere, and would have expected it in the sample_lifecycle.rs example (so that in 3 seconds from the song actually processing samples, the loop quits)...

Alternatively, maybe I assume that when I call .play on a loaded sample, that it will play instantaneously and keep track of time within the primary world's ECS

[slate scarab]

an instantaneous offset from the start of the song

Hm, I'm not totally sure what you mean by this. Are you, for example, looking for a way to play another sample at exactly some point in time relative to another sample's playback?

Like play a sound effect at exactly 2.5 seconds into a music sample?

[hasty cradle]

Understandable
Will explain in a sec

[hasty cradle]

I add them on spawn using sample_effects!(VolumeNode { volume: Volume::SILENT }),

Then, I use them in systems like
"here's the entity of the sound I want to make louder, query over the VolumeNodes and if you find one that's an EffectOf this entity, set its volume"

[slate scarab]

Hm, well that should work.

Alternatively, you could make use of the query trait extension I recently added. I hoped to make this process a little easier. As an example:

fn update(
    // In this case there's only one sample player, but you can query for the
    // target however.
    player: Single<&SampleEffects, With<SamplePlayer>>,
    mut effects: Query<&mut VolumeNode>,
) -> Result {
    let mut volume = effects.get_effect_mut(&player)?;

    // ...
    
    Ok(())
}

Either way you should be getting access to the correct volume node.

[hasty cradle]

That's a good extension
I assume it basically does what I do but like
With less boilerplate

What, you think, could be the problem with the way I set the volume?

I mean, I can see that it's putting the volume in the volume node

[slate scarab]

Hm, no that seems fine.

Do you have a little more context for how the sample is spawned? I'll also double check and make sure there's nothing wrong with volume in particular later.

[slate scarab]

Side note -- this approach is still a little awkward. In the future, we'll have queries that can handle relationship information, so you'd be able to get at effects with one query instead of two. In the meantime, I hope this suffices.

[hasty cradle]

I could send the spawn command, one sec

                (SoundBundle {
                    audio_player: SamplePlayer::new(layer.audio_source.clone()),
                    //TODO: make it start from the time specified in the file name once that's possible
                    audio_settings: PlaybackSettings {
                        repeat_mode: RepeatMode::RepeatEndlessly,
                        volume: Volume::SILENT,
                        ..default()
                    },
                    transform: Transform::default(),
                }),
                sample_effects!(VolumeNode {
                    volume: Volume::SILENT
                }),
                TimeScalerSubscriber {
                    scaler: TimeScalerId::GameTimeScaler,
                    identifier: format!("Music instance for level music, layer: {:?}", layer_index),
                },
            ));```

oh that doesn't look well on discord

hold on

https://gist.github.com/Rabbival/7ba42930383c2b4ba1f91e8840240a5a

gtg, will come back to read your insights in like
an hour and a half

[drowsy dome]

sort of! I just want to expand on your sound effect example to make sure that I've confined this scenario. At exactly 2.5 seconds into a music sample, I will also set the meshmaterial of some object to red, OR, I will turn the meshmaterial of the object red on the next run of the Update schedule. Intuitively, it seems like the former would require some triggered event, and therefore, access to World. Unsure about the latter. In the past, I used a thread channel (reader was a Resource to effect this)

[slate scarab]

Oh, so you're looking to coordinate things in the ECS with sample playback if I understand correctly.

Well, there's nothing specific set up to handle this yet. You could do polling in the ECS and check when 2.5 seconds worth of samples have elapsed. Something like

fn poll_playback(players: Query<&SamplePlayer>, mut context: ResMut<AudioContext>) -> Result {
    // I'll probably add a nicer way to get this.
    let sample_rate = context
        .with(|c| c.stream_info().map(|i| i.sample_rate))
        .ok_or("stream info unavailable")?
        .get();

    for player in &players {
        let Some(frames) = player.playhead_frames() else {
            continue;
        };
        let seconds = frames as f64 / sample_rate as f64;
        
        if seconds >= 2.5 {
            // do the thing
        }
    }
}

[drowsy dome]

you're amazing. this is exactly what I'm looking for

[slate scarab]

Keep in mind this approach has flaws

[drowsy dome]

Wait, I was about to say that it must be referencing sometime in the past...but seeing an atomic is so comforting hahaha

[slate scarab]

It's inexact in two ways: for one, the playhead only advances within the audio processing block. In-between, it just kinda sits at the same value. Since we read it directly (it's a shared atomic), we're observing a clock that starts and stops sporadically. That means we can only expect to catch the 2.5 second mark after it's definitely been processed in the audio thread. Depending on how large the audio processing buffer is, that might lead to more latency than the actual game framerate. (A buffer of 1k frames will update the playhead at about 44hz at a sample rate of 44.1kHz).

[drowsy dome]

because I'll just make some epsilon in case it's really close to 2.5. Most likely some combination between some cached FPS and the sample rate of the player.

ah yeah

[slate scarab]

It's probably close enough for most purposes tbh

[drowsy dome]

this is a really good starting point though. no locks on the audio thread which is the constraint, so while there's no way of knowing how far into the sample it is, yeah

self: audio timing construction

makes sense, thanks

[slate scarab]

We could definitely make a first-class solution for this, though. We could come up with an API that'll trigger events on the sample entity when it reaches certain points in its playback.

Part of the reason it's tricky is that a sample is not guaranteed to begin playback on the frame a sample player is spawned. If the sample takes too long to load, it'll have to wait a few frames. Also, if the pool is completely saturated, it might have to wait a few frames (or it could even be dropped). This makes it essentially impossible to get a timestamp of when the sample started at the moment you spawn it.

However, if we're doing polling, we could wait for the sample to start playing, cross-reference the playhead with the current audio context clock, and get a better idea of when the requested amount of time will have elapsed in terms of ECS time.

[slate scarab]

    audio_settings: PlaybackSettings {
        repeat_mode: RepeatMode::RepeatEndlessly,
        volume: Volume::SILENT, // < O.O
        ..default()
    },

Since the volume here is only set once at the beginning of playback, it will always be silent, regardless of what the volume node is set to.

[thick orbit]

hmm, so there is no helper like in bevy_audio to just mutate GlobalVolume?

[slate scarab]

Nope! But I don't think adjusting it is too bad:

fn mute(mut global_volume: Single<&mut VolumeNode, With<MainBus>>) {
    global_volume.volume = Volume::Linear(0.0);
}

[thick orbit]

also I am kind of confused on whether is changing the volume of something that is already playing is possible

[slate scarab]

Yes! If you want per-sample control, add a volume node:

commands.spawn((
    SamplePlayer::new(server.load("sample.wav")),
    sample_effects![
        VolumeNode::default(),
    ],
));

fn update_volume(
    players: Query<&SampleEffects, With<SamplePlayer>>, 
    mut effects: Query<&mut VolumeNode>
) -> Result {
    for player in &players {
        let mut volume = effects.get_effect_mut(player)?;
        volume.volume = Volume::Linear(0.5);
    }

    Ok(())
}

[thick orbit]

for example I have a system to change volume of music alone or general. Or an action to mute/unmute

[slate scarab]

Oh by the way, sorry, this assumes you're on the bevy-0.16 branch.

[thick orbit]

hmm, ok, seems doable.

[slate scarab]

If you want to update the volume of groups of samples, like all music samples, you could create pools for them and play them inside.

#[derive(PoolLabel, PartialEq, Eq, Debug, Hash, Clone)]
struct MusicPool;

// spawn a pool
commands.spawn(Pool(MusicPool));

// play music specifically within the music pool
commands.spawn((
    MusicPool,
    SamplePlayer::new(server.load("my_music.wav")),
));

// adjust the volume of the entire pool
fn update_music(mut music_volume: Single<&mut VolumeNode, With<Pool<MusicPool>>>) {
    music_volume.volume = Volume::Linear(0.5);
}

[thick orbit]

So pools are like channels? its it appropriate to create let's say music and sfx pools

[slate scarab]

They're similar, yes

[hasty cradle]

Wait what

Ohhhh the playback settings are permanent no matter what the nodes say?

How should I initialize it then?

[slate scarab]

Well they're different.

The sample player node has its own volume setting that you can set once when you load a sample. This would be multiplicative with any subsequent volume nodes.

I'd set the volume to whatever level seems good! If you don't need to adjust it, then just leave it at unity (the default value).

[hasty cradle]

Oh so it multiplies

Like, I should provide the settings with what should be referred to as 100%

(right?)

[slate scarab]

When you add the volume node, the graph looks like

SamplerNode -> VolumeNode

The playback settings only affect the SamplerNode.

[hasty cradle]

ohhhh
understandable
Thanks for the help
(want me to make a pr to add it to the q&a?)

[slate scarab]

I suppose we should probably have an entry on "How do I change a sample's volume" or something

Ideally the API itself would make this obvious but... it's a little tricky 😅

[thick orbit]

Yeah, would be nice, I think it's a very common use case for at least general sound level in settings(I am trying to implement just that)

[hasty cradle]

I'll make a sketch and send you in DM

[slate scarab]

Awesome! I should be able to get back on that after work.

[slate scarab]

Yeah at the very least I want to add an example for the typical "Master/Music/Sfx" setup.

[hasty cradle]

The transition is complete

So far I'm really happy, the only thing missing for me is the ability to define a loop region, but it's not as urgent (I made looping versions of my tracks until it'll be available)

[oak walrus]

Thanks a lot @hasty cradle ! Are there any runtime improvements you've noticed? On the web specifically, if you target that platform?

[dusky mirage]

So is the new audio backend now officially part of Bevy? Or did it get merged into some other repo?

[slate scarab]

No that's just for their project!

[dusky mirage]

Ah, ok.

sadly puts back party poppers and champagne bottle 😛

[drowsy dome]

soon™

[celest whale]

We're waiting on the demo to do a final assessment 🙂

But I am personally feeling very keen on this

[hasty cradle]

I never managed to make a bevy build for the web by myself haha

[celest whale]

Have you tried the CLI?

[hasty cradle]

Sorry for being misleading 😅
No, it's just that this is the fourth time I've reimplemented the audio for my game and I wanted to share that I finished transitioning it to seedling and that this is the best solution so far

[hasty cradle]

I did
Someone also helped me by making one for a project we worked on for the spooky jam

For some reason when I try doing cli myself it keeps failing?
But it's ok, I'm not really exporting for web usually other than in jams

[rapid hedge]

You should just be able to do bevy run web, no setup required
If you happen to try again, could you post the error you’re getting in #1278871953721262090 ?

[hasty cradle]

oh I didn't know you could just do bevy run web
will try it on the next jam I join (hopefully the next one but idk because university)
thanks

[drowsy dome]

2025-05-22T16:44:31.068738Z ERROR firewheel_cpal: CPAL and/or the system audio API returned invalid timestamp. Please notify the Firewheel developers of this bug.

[drowsy dome]

https://github.com/BillyDM/Firewheel/issues/43

[dusky mirage]

Hmm, I'm at a bit of a loss on why that bug is happening. The problem is that sometimes CPAL will send a timestamp that has a value less than the previously sent timestamp. So either there is a bug with the APIs, a bug with CPAL, or I'm just misunderstanding how the timestamps are supposed to be used.

For now, I've just disabled using the OS's timestamp and instead switched it to use std::time::Instant::now() as a temporary workaround. https://github.com/BillyDM/Firewheel/commit/a27bea1592594613f09a229583bb2193581cd715

Hmm, a similar issue was opened in 2020. 😖 https://github.com/RustAudio/cpal/issues/440

I also noticed that the bevy_ecs crate has an updated 0.16.0 version. Should I update Firewheel to use that version, or should I wait on that?

[celest whale]

Update please 🙂

[dusky mirage]

Alright, updated!

[tender fiber]

Nice!

[hasty cradle]

Are there any entities that bevy_seedling spawns on which all the audio depends?

Asking because when my game restarts I despawn all entities that weren't tagged with DontDespawnOnRestart and when that happens audio is no more

I could only find this spawning function:
https://github.com/CorvusPrudens/bevy_seedling/blob/2665bd4b1e1d0cf53a261e4bebeb6b26911f70d3/src/pool/mod.rs#L229

oh but it's not public

I don't know what to do then 🏳️

[keen vigil]

Unrelated to audio but I had this issue in my game with 3rd party crates that need long living private entities (bevy_vello) so I caved and just rewrote all my spawn commands with a marker component so I could despawn only things I had spawned. Starting in bevy 0.15, you have to remember to not despawn windows, monitors and observers and I imagine it’s only going to become harder with assets as entities, etc. Good question though - I’m interested in what other bevy users think for the best way to despawn everything without breaking anything

[hasty cradle]

I simply mark anything that spawned before prestartup as something that shouldn't despawn on restart

Which is why it surprised me

[keen vigil]

To be annoying, if you plug in a monitor while the game is running then bevy will create a Monitor entity which won’t be created in prestartup but possibly is an issue?

[hasty cradle]

True

I don't know what to do then

[hasty cradle]

I could go for the reversed logic

[slate scarab]

Hm, we could make more components public, although that does increase the exposed API surface (and thus means more breakage with updates).

[hasty cradle]

It's ok, Josh convinced me to try another approach (with his screen argument)

[keen vigil]

Just in case its useful, I also made this extension trait for commands and then made sure I only use spawn_scoped instead of spawn in my project to avoid accidentally forgetting entities #1349113880944574666 message

[hasty cradle]

Interesting, sending you a dm because it feels beyond the scope of the channel

[deft jasper]

will bevy_seedling get a 0.16 release for the jam? or should I tell people to use the git branch? (or should I not tell people 😅 )

[slate scarab]

yep!

[slate scarab]

I’m very committed to making that happen (I wanna join with my brother too)!

I’ll be flying around tomorrow, but that should give me enough time to wrap up the documentation and any final changes.

[charred pond]

does bevy_seedling support changing the volume of an already-playing sample?

mutating PlaybackSettings doesn't work (same as bevy_audio), so do i need like.. a VolumeNode?

[slate scarab]

Yeah those should probably be immutable to avoid that confusion!

But yeah, for dynamic updates, you’ll want a volume node as an effect. If enough people run into this, we may just special case it.

[charred pond]

my use case is adjusting background music volume from a settings menu

so should be a common use case, held back by bevy UI discouraging people from building settings menus lol

but i can try the VolumeNode approach

[charred pond]

bevy_reflect feature would be much appreciated for entity inspection 🥺

[slate scarab]

Actually I will say — given the routing capabilities, you can simplify things a bit for cases like that.

I’d just make a music pool and play all your music in that. Something like

#[derive(Component, PoolLabel, Debug, Clone, PartialEq, Eq, Hash)]
struct MusicPool;

commands.spawn(Pool(MusicPool));

commands.spawn((
    MusicPool,
    SamplePlayer::new(server.load(“music.wav”)),
    PlaybackSettings::LOOP,
));

[charred pond]

i'm trying this right now actually. for some reason my master volume is working via MainBus but music and ui volume not working via MusicPool and UiPool

[slate scarab]

The marker type would be Pool<MusicPool>

in case you were just looking for MusicPool

[charred pond]

i'm probably just missing something. relevant code:

fn apply_audio_settings(
    audio_settings: Res<AudioSettings>,
    music_query: Query<Entity, (With<MusicPool>, With<VolumeNode>)>,
    mut volume_query: Query<&mut VolumeNode>,
) {
    // Update music volume.
    for entity in &music_query {
        c!(volume_query.get_mut(entity)).volume = Volume::Linear(audio_settings.master_volume);
    }
}

#[derive(PoolLabel, Eq, PartialEq, Hash, Clone, Debug)]
struct MusicPool;

fn spawn_music_pool(mut commands: Commands) {
    commands.spawn(SamplerPool(MusicPool));
}

pub fn music_sample(handle: Handle<Sample>) -> impl Bundle {
    (SamplePlayer::new(handle), PlaybackSettings::LOOP, MusicPool)
}

[charred pond]

yeah this is probably it

can't find the symbol Pool

[slate scarab]

oh sorry 😅 yes that’s correct

I decided Pool was far too common a name to be exported in the prelude.

So instead of just MusicPool in that query, it would be SamplerPool<MusicPool>

[charred pond]

ahh ok

hm i'm seeing the same behavior with With<SamplerPool<MusicPool>>

even though logging suggests that there is 1 entity found in the query

[slate scarab]

Ah sorry, it’s hard to see on my phone.

The query can be super simple. Something like

fn pool(mut q: Single<&mut VolumeNode, With<SamplerPool<MusicPool>>>) {
    q.volume = /* */;
}

That is, the entity you spawn to create a pool has a volume node inserted into it

[charred pond]

so i cut out some context, i can't do that because i have multiple queries, one for each pool

but it should be equivalent i think?

if i get the Entity instead and route that to another query for &mut VolumeNode

[slate scarab]

right ya that seems like it oughta work

[charred pond]

oh, it seems i've made a classic copy-paste error

works now

i was setting the volume to master volume instead of music volume

also am i crazy or is the Volume::Linear range from 0 to 100% much wider than in bevy_audio?

yeah i think i'm crazy, tested my bevy_audio build and it seems comparable

i just never tried setting master and music volume to 100% at the same time i guess 😄

[charred pond]

i think a simple example showing off basically what i'm doing would be great :)

aka controlling the volume of a pool, probably via key press instead of UI

there's the pool / bus example, but that's more generic showing off the API than an actual usage example

[charred pond]

even better if there are at least 2 separate pools, plus master volume control via MainBus

i opened a couple issues

[charred pond]

i think bevy_seedling breaks my web build with error: The "wasm_js" backend requires the wasm_js feature for getrandom

i noticed bevy_seedling (in the bevy-0.16 branch) depends on rand 0.9, while bevy 0.16 is still on rand 0.8

[slate scarab]

This is just kinda… a thing 😅

[hasty cradle]

I guess it's time to find the developers of firewheel

[drowsy dome]

hehe https://github.com/BillyDM/Firewheel/issues/43

[hasty cradle]

interesting

[charred pond]

yeah i'm not sure if it was bevy_seedling related at all, it just started happening on the commit where i migrated to bevy_seedling

possibly some transitive dependency thing. and i've applied the workaround properly now

[slate scarab]

I didn’t realize Bevy still hasn’t moved over — probably due to this issue. getrandom has taken a rather… cavalier approach to the issue.

Since rand isn’t in the public API, I think I’ll just roll it back to 0.8 until Bevy as a whole moves over.

[celest whale]

Much appreciated!

[tender fiber]

That's why dB is used and not linear (so it doesn't feel like someone is using the volume knob)

Like industry (like hardware)

I don't have to crank that 50% up to 100%. It's just +6 dB. In hardware, it's less movement

[viral grove]

Hi guys! I'm interested in contributing to the audio working group. Is this the right place? I'm confused as to where I need to start reading to get up to speed with the current state and what crates/repos are relevant

[oak walrus]

This is the right place! You can work on Firewheel which is the backend of the audio engine, or bevy_seedling which is the Bevy side where the user-facing API resides

[slate scarab]

There's lots of effects we need in Firewheel if you're confident in doing a little DSP!

[oak walrus]

You can talk to @dusky mirage for the former and @slate scarab for both

[slate scarab]

Here's Firewheel, which is the underlying audio engine driving my integration bevy_seedling.

My integration isn't authoritative, but it's the only (nearly) complete one I'm aware of.

I'm looking to complete the Bevy 0.16 version of bevy_seedling a couple days before the jam, at which point it should become very compelling.

[viral grove]

Awesome, thanks for the link, I'll check it out. I am very comfortable implementing DSP algs, will take a look what I can add

[tender fiber]

CLAP is new open source VST plugin standard which would provide cool effects in a yet to be done Firewheel Node

[viral grove]

You mean like a "CLAP plugin" node in Firewheel?

[tender fiber]

Plug-n-play plugin support yes

Also there is Freeverb good 1st issue in Firewheel github

[pure dove]

probably good to implement some of the simpler open issues (e.g. filter nodes and reverb node)

ye

[tender fiber]

It's DSP inside a Node

[pure dove]

generalized plugin support opens up a lot, but most plugins are just not designed or optimized to run alongside a game, or handle any firewheel-specific optimizations

i also don't think you could implement it as a totally standalone node, you'd want to share the plugin hosting setup between all plugin nodes

[viral grove]

i'm currently prototyping filter nodes

[pure dove]

gl

[celest whale]

If you get this ready please ping me and I'll add it to the alpha testing section

[thick orbit]

can anyone shove me into right direction on how to deal with volume of existing playbacks? I am a bit confused about all these pools and nodes

[slate scarab]

It depends a bit on how you want to do it.

Are you looking for control over individual sounds? Or do you want to control groups of music and sound effects?

[thick orbit]

the latter, music and sfx

[slate scarab]

In that case, you'll probably want to define two pools and control their overall volumes.

#[derive(PoolLabel, Debug, Clone, PartialEq, Eq, Hash)]
struct MusicPool;

#[derive(PoolLabel, Debug, Clone, PartialEq, Eq, Hash)]
struct SfxPool;

// spawn the pools
commands.spawn(SamplerPool(MusicPool));
commands.spawn(SamplerPool(SfxPool));

// play sounds in the pools
commands.spawn((
    MusicPool,
    server.load("my_music.wav"),
));
commands.spawn((
    SfxPool,
    server.load("my_sfx.wav"),
));

// control the volume of each pool
fn update_music_volume(
    mut music: Single<&mut VolumeNode, With<SamplerPool<MusicPool>>>,
) {
    music.volume = Volume::Decibels(-6.0);
}

fn update_sfx_volume(
    mut sfx: Single<&mut VolumeNode, With<SamplerPool<SfxPool>>>,
) {
    sfx.volume = Volume::Decibels(-6.0);
}

[drowsy dome]

oh btw @slate scarab feel free to close any of my outstanding PRs or issues :)

it's been nothing but a pleasurable experience with this crate, honestly I think it's what's keeping me from sending it on Godot (their audio plugin's missing a lot, and can't find suitable alternative)

I can close them if you like

oh wait!

LOL didn't even see that I don't have any

[slate scarab]

I think the remaining issue is very valid! I've just been fairly limited on time lately.

Also it's kinda tricky 😅

[drowsy dome]

no worries

[drowsy dome]

ohhh this one. Yeah, not a big deal AT ALL lol

[thick orbit]

Ok, I've successfully replaced bevy_audio, so far I love it.
Mind if I make a PR with an easy example to bevy-0.16 for setting up two pools?

[slate scarab]

Sure! It would be really nice if we could have three overall inputs: master volume, music volume, and sfx volume, since that's the typical split. The examples in the repo all run without any windows, so we'd probably want some keys to raise/lower the volume of each one.

[thick orbit]

np, that's exactly my setup, I'll just copy over what I already have for settings page.

[tender fiber]

Filter example using dsp crate https://github.com/FyroxEngine/Fyrox/blob/master/fyrox-sound/src/effects/filter.rs

[viral grove]

Thanks! I'm working on a Butterworth implementation for now but the RBJ cookbook is also on my list

[thick orbit]

I have a question for a master volume pool: can I somehow nest pools behaving like a channel in channel (general <- sfx, general <- music)?
or do I just spawn entitites with two labels(general, sfx) and work it out from there?

[slate scarab]

Hm, I’m not quite sure I follow. But probably!

What are you looking to achieve in general?

[dusky mirage]

You should use the Simper SVF filters. They are a drop-in replacement for the RBJ cookbook filters and they are better than biquad filters in practically every way. https://github.com/neodsp/simper-filter

(better precision, better stability, and better behavior when sharply modulated)

[viral grove]

Thanks, will look into it

Wanted to learn about SVFs, didn't know they could also do peak, notch etc

[oak walrus]

With a technique called pole-mixing, yes

You can make any biquad filter with SVFs + pole-mixing, they're strictly equivalent filters

But as Billy said, SVF behaves better under modulation, so it's strictly superior to a traditional biquad filter

[tender fiber]

Cytomic The Glue type filters hmm

[thick orbit]

I guess for general to affect both sfx and music, but changing music and sfx not affecting the level of general pool

Also as I was copying code to an example I stumbled on weird inconsistency.
Did the volume field dissapear from PlaybackSettings?..

So now if I want to spawn SamplePlayer I need to spawn VolumeNode, right?

[slate scarab]

It will imminently change again 😅 forgive me for the rapid pace of changes.

The PR for 0.4 is up now, you can check out the major changes there and the rationale behind them.

I'm just doing some final checks before publishing right now.

[sacred osprey]

I think Cytomic plugins are much more advanced and use analog modeling for their DSP

[tender fiber]

SSL Yes … ok … range, sidechain with hp filter, and mix

[thick orbit]

alright, I will just rebase to the bevy-0.16.2 instead of bevy-0.16

[slate scarab]

Unfortunately, it looks like Firewheel brings in getrandom 0.3 via ahash, so anyone looking to make wasm builds will have to deal with that for the moment.

[thick orbit]

yeah, everyone will just use @charred pond fork anyway

[target.wasm32-unknown-unknown.dependencies]
getrandom = { version = "0.3", features = ["wasm_js"] }

[patch.crates-io]
getrandom = { git = "https://github.com/benfrankel/getrandom" }

[slate scarab]

If you run into any issues or you have a strong distaste for the changes, please let me know!

[thick orbit]

Oh, quite on the contrary - I love new with_volume/looping methods way more

[slate scarab]

Hm, something like this?

┌─────┐┌───┐┌───────┐
│Music││Sfx││Default│
└┬────┘└┬──┘└┬──────┘
┌▽──────▽┐   │       
│Bus 1   │   │       
└┬───────┘   │       
┌▽───────────▽┐      
│MainBus      │      
└─────────────┘      

Where the Bus 1 could just be another volume node maybe?

[dusky mirage]

Correct. Andrew Simper decided to release the research he did on the state variable filters to the public for free since he knew it would be useful to the audio industry as a whole. Cool dude.

[dusky mirage]

Oh, I forgot what I'm even using ahash for. Hold on a second.

[slate scarab]

Oh while you're at it, would it actually be possible to yank 0.3.1 and publish a 0.4.0? I think upgrading the bevy version is incompatible with a patch change -- my CI can't build the previous version of bevy_seedling for my own semver checks.

Not really that important, but it might be a little confusing or annoying if anyone sticks around on the older version of bevy_seedling.

[dusky mirage]

Would publishing a 4.0 be enough to fix that?

[slate scarab]

hm I think the only way to fix the old builds would be with a yank, since it'll try to update any version = "0.3" to "0.3.1" (since the old bevy_seedling version doesn't pin it or anything)

[dusky mirage]

Oh ok, I see what you mean.

Is rustc_hash fine to use?

[slate scarab]

Hm, does ahash maybe have a slightly older version that doesn't pull in the latest getrandom?

I believe rustc_hash is actually a fair bit faster than ahash and uses a slightly lower-quality algorithm. But I doubt high quality hashing is all that important for the audio graph.

Hm, or maybe I'm thinking of Firefox's hashing (fxhash).

[dusky mirage]

fxhash was renamed to rustc_hash

ahash is a bit faster that rustc_hash for larger keys. (In Firewheel the EdgeHash type has a size of 24 bytes.) https://github.com/tkaitchuck/aHash/blob/master/compare/readme.md

But yeah, the performance of the hasher in the audio graph compilation algorithm probably doesn't matter that much. Even using the standard library's hash map probably wouldn't make a noticeable difference.

[slate scarab]

It looks like if you pin ahash to 0.8.11, you that'll bring getrandom back down to 0.2 for what it's worth. i.e. ahash = "=0.8.11"

[dusky mirage]

Oh wait, it looks like I could just disable the std feature in ahash.

[slate scarab]

Hm, that might require a bit more manual work https://docs.rs/ahash/0.8.12/ahash/index.html#randomness

Although I guess a fixed-per-process random state is probably fine?

[dusky mirage]

Hmm actually, for the sake of minimizing the amount of dependencies in bevy, what hash map does bevy use internally (if it uses any at all)?

[slate scarab]

It may actually be a good idea to use anyway since Bevy's solution is no_std compatible

I believe it's in bevy_platform

https://docs.rs/bevy_platform/0.16.0/bevy_platform/collections/hash_map/index.html

[dusky mirage]

Cool, I'll just use that then.

I'm wondering if I should add a bevy feature to the graph crate to avoid making other potential game engines from relying on it? Though we are already using bevy_macro_utils in the macro crate.

[pure dove]

i'm working on a non-game program that involves firewheel and i'd prefer to avoid bringing in bevy things, but if it's used internally and doesn't affect compile times much i wouldn't mind

[dusky mirage]

I'll just make it a feature.

[slate scarab]

bevy_platform is self-contained -- it has no other bevy-related dependencies. A few bevy crates are like that, though they may give the impression of being heavy or intrusive with the bevy name.

[wary bridge]

I have a non-bevy future usecase for firewheel, but don't have a strict opinion on it

[slate scarab]

bevy_macro_utils could be yeeted with not too much effort, but I'm fairly confident using bevy_platform unconditionally is a good idea. While it is written for Bevy, it has a lot of stuff in there that would make moving Firewheel towards no_std compatibility much easier!

[dusky mirage]

Yeah, that's a fair point. Also taking a further look at the hash map in bevy_platform, it would be nice to have in general.

Yeah, the list of dependencies doesn't seem bad. https://crates.io/crates/bevy_platform/0.16.1/dependencies

And I was already planning on using portable-atomic anyway.

Also, it looks like bevy 0.16.1 was just released.

[slate scarab]

ya i think that's some jam prep

[dusky mirage]

Alright, I replaced ahash with bevy_platform and published version 0.4.0!

And yanked version 0.3.1

I also wonder if I should also go ahead and replace all atomics with the portable_atomic ones in bevy_platform? It shouldn't cause any breaking changes.

[slate scarab]

Hm, yeah possibly. @limpid river should know whether that's the right idea

[limpid river]

Yeah not a breaking change. It'll only give you a different type of it doesn't exist on the target you're compiling for. But if it didn't exist then it's be a compilation error anyway, this no breaking change

That's also ignoring they're API compatible too lol

[dusky mirage]

Hmm, actually I would have to replace the atomic_float crate with a custom implementation, but it's pretty trivial to implement.

[limpid river]

Oh if you need that portable-atomic itself provides atomic floats

[dusky mirage]

Oh right, that's even easier.

Hmm, it appears bevy_platform doesn't expose the atomic float types. I guess I'll just import portable-atomic directly for now.

[limpid river]

Yeah we don't have a use for it (yet)

[dusky mirage]

It also appears bevy_platform doesn't have thread::panicking().

[slate scarab]

Okay, the 0.16-compatible version of bevy_seedling is up! The GitHub release explains what's new and provides some migration guides. It should now have complete feature parity with bevy_audio, along with all the additional power and flexibility afforded by Firewheel.

With those last changes from Firewheel (thanks @dusky mirage), bevy_seedling is also free of getrandom 0.3 for the time being.

@celest whale, when you get the chance, could we get an #1034543904478998539 thread going?

[celest whale]

#1378170094206718065

[dusky mirage]

I also just published a "0.4.1" version that replaces all of the atomics with portable-atomics, as well as replacing as much as I could with the types from bevy_platform.

[celest whale]

Happy to see it getting use elsewhere in the ecosystem 🙂

[dusky mirage]

Though it's still not fully no-std compatible yet. It still relies on that std::thread::panicking() method I mentioned before. And also both cpal and symphonia require std.

[dusky mirage]

Picked some real life firewheels!

Although where I live we actually call them "Indian Blankets", but I figured that name wouldn't go over as well. 😅

[limpid river]

Yeah unwinding a panic is something only supported with std, so (IIUC) in no_std a panic is essentially aborting (not quite, a double panic is an abort by definition, whereas a single goes to the panic handler which isn't allowed to return)

[thick orbit]

Ok, how do I spawn this kind of structure? and do I no longer need a pool if I use bus? I'm a bit confused
to make this tree do I just spawn them like so:

    cmds.spawn((MainBus, SamplerNode(General)));
    cmds.spawn((MainBus, Bus1));
    cmds.spawn((Bus1, SamplerNode(Music)));↴
    cmds.spawn((Bus1, SamplerPool(Sfx)));↴

[drowsy dome]

hey @slate scarab, currently working on experimenting with using worker threads for using assets. Is this relevant to a discussion on audio for web?

ehh, actually will follow into this crate specifically

[slate scarab]

Right now, a "bus" is really just a node that we've given a label, usually a VolumeNode.

So to create the structure I illustrated, you could do:

// You'll probably want a node label for querying
#[derive(NodeLabel, Debug, Clone, PartialEq, Eq, Hash)]
struct Bus;

commands.spawn((Bus, VolumeNode::default()));

commands
    .spawn(SamplerPool(Music))
    .connect(Bus);

commands
    .spawn(SamplerPool(Sfx))
    .connect(Bus);

The default pool is already connected to the MainBus, and the Bus node will be automatically connected as well since we didn't specify any connections for it.

A sampler pool is basically a collective sound source, so it doesn't really make any sense to route audio "through" it.

We don't use relationships right now to represent connections because Bevy's implementation doesn't support M:N-style relationships. So for now, we have to stick to the imperative connect methods.

[thick orbit]

thank you! will put this in the example as well

[viral grove]

Awesome! So can they also be used to improve the biquad stages of a Butterworth filter or not? Will read through the paper linked on the GitHub repo but am intrigued

[dusky mirage]

Actually yes. Here you can see an implementation of an EQ I've been working on for my DAW project. I'm cascading multiple SVFs to create higher order lowpass and highpass filters. https://github.com/MeadowlarkDAW/meadow-dsp/blob/main/meadow-dsp-mit/src/filter/svf/f64.rs#L55

[viral grove]

Nice! I'll take a look

[dusky mirage]

Here is the paper I referenced for cascading butterworth filters. https://www.earlevel.com/main/2016/09/29/cascading-filters/

[thick orbit]

alright, I got the basic example working.
I will create a PR but I have a practical question - UI features add significantly more dev deps, is it ok to just leave it as a examples/music_sfx_pools.rs or should I wrap it in crate? On the plus side - you don't get to compile 400+ crates for all examples, I think current seedling dev deps baseline around 300, the drawback is that you cannot run cargo run --example music_sfx_pools then.

[viral grove]

Is this anything special? At first glance the coeffs are just what you get from an analog prototype + bilinear transform

[dusky mirage]

What makes a butterworth a butterworth is that it is "flat" around the cutoff point, so the only variable is the Q factor. The rest is just standard biquad/SVF filter stuff.

Although you can still get resonance from a butterworth filter. You just start with the flat response, and then you scale each of the Q values equally by some amount.

[dusky mirage]

Oh wait, sorry, you don't scale each of the Q values, you just add some amount to each of the Q values.

The ORD4_Q_SCALE, ORD6_Q_SCALE, and ORD8_Q_SCALE constants I just derived through trial and error. They are just used to make resonant peaks have roughly the same volume across all the different orders. It's not perfect, but it's good enough.

[viral grove]

Ah, I see. Thanks! Will have to look through it slowly when I get the time

[thick orbit]

hmm, I see is_playing is on the Sampler now, but I cannot query it for some reason?..

[slate scarab]

Oh really?

Like it's not showing up in your queries?

[thick orbit]

oh, apologies, false alarm, I was trying to use Sampler from bevy for some reason. Is it not in the prelude because they clash?

[slate scarab]

Actually it was just because I expected it to be a slightly-less-common component. I'll probably need to make the name a bit more specific regardless.

[thick orbit]

yeah, it clashes with bevy::render::render_resource::Sampler but it's not in bevy prelude either, it's just the only one recommended by lsp for some reason
nvm, it's not, i'm just blind and tired

[thick orbit]

here is the code for anyone interested

[slate scarab]

Okay I've set up an experimental multi-threaded Web Audio backend for Firewheel: https://github.com/CorvusPrudens/firewheel-web-audio.

Feel free to give it a try in bevy_seedling now! You can do so fairly easily like so:

# Cargo.toml
[dependencies]
firewheel-web-audio = "0.1"
# ...

# .cargo/config.toml
[target.wasm32-unknown-unknown]
rustflags = ["-C", "target-feature=+atomics,+bulk-memory,+mutable-globals"]

# This means you'll need a nightly version of the compiler
[unstable]
build-std = ["std", "core", "alloc", "panic_abort"]

fn main() {
    // ...
    #[cfg(target_arch = "wasm32")]
    app.add_plugins(
        bevy_seedling::SeedlingPlugin::<firewheel_web_audio::WebAudioBackend> {
            config: Default::default(),
            stream_config: Default::default(),
            spawn_default_pool: true,
            pool_size: 4..=32,
        },
    );

    #[cfg(not(target_arch = "wasm32"))]
    app.add_plugins(bevy_seedling::SeedlingPlugin::default());
    // ...
}

I'd recommend the Bevy CLI for testing this out locally. It allows you to provide the necessary security headers:

bevy run web --headers="Cross-Origin-Opener-Policy:same-origin" --headers="Cross-Origin-Embedder-Policy:require-corp"

On itch.io, there should be a checkbox for SharedArrayBuffers, which will supply those headers.

[slate scarab]

With this, stutters and glitching in your game audio should be a thing of the past, even for unoptimized or demanding games! This runs the audio code in the browser's dedicated audio thread, rather than on the main browser thread.

So far in my testing, it seems to be working great! If you run into any issues, please let me know.

[dusky mirage]

Oh wow, great work!

[oak walrus]

Looking promising!

[slate scarab]

It's not totally complete (there's some minor realtime safety violations, no input yet, only stereo out), but that should all come in due time.

[dusky mirage]

Is this something that could be added to Firewheel itself? Or is this bevy-specific?

[oak walrus]

It looks like an alternative audio backend for Firewheel, so nothing Bevy specific?

[slate scarab]

Ya it's completely separate, once it's more or less complete we could move it in as another backend.

We might have to adjust the backend API a bit if we want to be able to get input devices -- that's an async operation on the web

[oak walrus]

If it works as advertised I would love to eventually integrate this into my audio backend crate interflow if you don't mind @slate scarab

[slate scarab]

The core behavior is pretty simple and self-contained -- it just uses a couple tricks to make it work without any hassle.

[thick orbit]

this is huge

[tender fiber]

@celest whale

[celest whale]

Try and polish this and promote it in #jam please 🙂

[viral grove]

@dusky mirage since your SVF implementation seems to be complete already, is it preferred to copy the relevant code to keep it flexible in case Firewheel needs to modify it down the line or add it as a dependency? Sorry for the basic question, I have no experience contributing to a public repo :/

[dusky mirage]

I think it makes sense to just copy it into firewheel_core::dsp.

The SIMD stuff in there does require nightly Rust. So you could either just not include the SIMD stuff, or hide it behind a feature flag.

[viral grove]

Sounds good, thanks!

[viral grove]

Sorry, made a stupid mistake (deleted stupid messages)

[glad sable]

I'm having issues trying to get this to work; webgpu seems to be getting confused?

/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wgpu-24.0.5/src/backend/webgpu.rs:3246:18:
called `Result::unwrap()` on an `Err` value: JsValue(TypeError: GPURenderPassEncoder.setBindGroup: Argument 3 can't be a SharedArrayBuffer or an ArrayBufferView backed by a SharedArrayBuffer

I'm hoping it's just a configuration thing, but I am using nightly, and running with this command:

bevy run web --headers="Cross-Origin-Opener-Policy:same-origin" --headers="Cross-Origin-Embedder-Policy:require-corp"

my config.toml has:

[target.wasm32-unknown-unknown]
rustflags = ["-C", "target-feature=+atomics,+bulk-memory,+mutable-globals"]

[unstable]
build-std = ["std", "core", "alloc", "panic_abort"]

I am trying to render some things that use storage buffers; I was able to get the web build working with the storage buffer shaders before I tried adding the Audio backend/ switching to nightly

[slate scarab]

Boy the web sure is a mess, isn't it?

Argument 3 can't be a SharedArrayBuffer or an ArrayBufferView backed by a SharedArrayBuffer

Well this seems a touch problematic -- the SharedArrayBuffer is what allows the multithreading we need for audio! My understanding is that all of the normal memory -- the stack and the heap of the Rust Wasm code -- becomes a SharedArrayBuffer when you apply the nightly settings and target features.

This could be a known issue in wgpu. I wouldn't know. There might even be something you can do in userspace to resolve it.

If it's you constructing and passing around storage buffers, you should be able to construct them without using SharedArrayBuffers. i.e. copy data from the Wasm module over to a normal ArrayBuffer and pass that around.

[glad sable]

I'm using the ShaderStorageBuffer type built into bevy
Maybe it's doing something under the hood to cause that, hm...

[slate scarab]

It looks like wgpu is just calling the JS API with a Rust slice of u32: https://github.com/gfx-rs/wgpu/blob/99e6524b2b03cc35733f199bb5cb28787e8d42de/wgpu/src/backend/webgpu.rs#L3239-L3246

I think this could only be solved upstream.

@coarse sun I hope you don't mind the ping, but have you seen things like this before? That is, when someone sets up their Wasm module for multithreading, do some WebGPU APIs just fail like this?

I couldn't seem to find an existing issue for this, so it might be a good idea to file one, @glad sable.

[coarse sun]

oh that's facinating, I haven't used WebGPU in a multithreaded bundle in a while - it seems we're doing the thing where you sub-array your own memory and pass that to the function, which for some reason it's getting mad

I'm really surprised that's an error though

you're not sending webgpu objects across threads

This looks like a chrome bug? https://gpuweb.github.io/gpuweb/#programmable-passes the array buffer is [AllowShared]

alright, this is a spec bug that was just fixed

https://github.com/gpuweb/gpuweb/pull/5202

so it will take a second for it to trickle down to chrome, I'd imagine

[slate scarab]

Oh interesting, thanks for investigating!

[glad sable]

Ah it was a bug in the spec; I got the error when testing on firefox nightly, so hopefully it trickles down to there soon as well. Glad to know it's already wip.
Thanks for taking a look; I guess they call it bleeding edge tech for a reason ha

[tepid sun]

Hi there!
I recently came across the Firewheel project and Bevy Seedling—awesome work! I was wondering if there's a straightforward way to play back and output multichannel audio using these tools? or is it more focused on stereo playback ?

[slate scarab]

Firewheel is absolutely capable of handling arbitrary channels! bevy_seedling is mostly there, but sample pools could use a touch more work to make that easy.

The reason being that sample pools handle parts of the routing automatically, so it would be a little awkward to get in there and modify it currently. However, I think this would be nicely solved with an additional component on a pool entity to specify the number of channels.

I'll make an issue for this -- I should be able to include this functionality in the next release.

And for clarity, if you're doing manual routing outside of a pool, it's fairly straightforward to manage this now:

// say you want a multi-channel effects chain
commands
  .spawn((
      LowPassNode::default(),
      // Here we can provide the node's startup configuration,
      // setting the number of channels.
      LowPassConfig {
          channels: NonZeroChannelCount::new(5).unwrap(),
          ..Default::default()
      },
  ))
  .chain_node_with(
      (
          BandPassNode::default(),
          // same thing here
          LowPassConfig {
              channels: NonZeroChannelCount::new(5).unwrap()
          },
      ),
      // The default connection is just a stereo one
      // (the first two tuples), so for multi-channel setups,
      // we need to specify the whole thing.
      &[(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)],
  )
  // Finally, to get all channels to the output, we'll
  // want to explicitly provide the main connection.
  .connect_with(
      MainBus,
      &[(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)],
  );

I'll add this and the new sample pool setup as an example once the new component lands.

[thick orbit]

got to check the multithreaded sound workaround in jam. I think it forces firefox to open new window? And until you click on not interactable stuff - audio is absent - I was playing, with buttons and all and only after a while realized that there is no audio.
I did not see many games with this behavior, most are embedded just fine.

[slate scarab]

Unfortunately some of this is completely out of our control. If a browser decides it needs to open a new window for SharedArrayBuffer (i.e. multi-threaded) Wasm modules, we can't stop that.

As for a lack of sound, I'm surprised you ran into that. The backend is listening for any events that the browser deems acceptable for resuming audio. For context, browsers don't let web pages start or resume an audio context unless immediately preceded by certain user inputs. So any event, regardless of whether it's inside some other interactable element or not, should initiate audio. We might need to listen to the capturing phase in case applications stop_propagation.

Put another way, there's nothing fundamentally different with how the WebAudio backend initiates audio compared to normal CPAL. We may just need to make the event listening a bit more robust to start audio as soon as possible.

[thick orbit]

I was thinking there were no sound because if you only click on things bevy classifying as interactable it simple intersepts window inputs, and audio backend just never started because browser thinks you never moved focus to a separate window. does it make sense?

[slate scarab]

Yeah that could be possible

[tender fiber]

Running one_shot.rs made DAC shoot up to 384k. Maybe default to 48k. Just a suggestion. Custom stuff here so you can also just blame me.

[dusky mirage]

Oh really? It is set to use the recommended sample rate from CPAL, so CPAL must be recommending that high sample rate for some reason. I'll add a fix for that tomorrow.

[tender fiber]

Awesome! Thank you.

[dusky mirage]

Alright, fixed it! (See the post in #1378170094206718065)

[dusky mirage]

@viral grove Sorry for not getting to your PR sooner. I've made a review.

[viral grove]

All good, thanks for letting me contribute!

[dusky mirage]

Also, having a max order of 16 is plenty. Even an order of 8 is overkill for most situations.

[viral grove]

Yeah, we can cut down if we want. I wanted to add orders in between anyway, so I set it at an arbitrary 16

[dusky mirage]

I think it's fine as is. Might as well give the user the option.

[slate scarab]

Hm, I have some small API thoughts -- @dusky mirage I hope you don't mind if I leave some review comments of my own!

[dusky mirage]

That is fine!

[dusky mirage]

And thinking of DSP stuff, at some point it would be nice to have built-in utilities for oversampling.

I understand how oversampling works in theory, though I haven't actually implemented one myself yet (Essentially it goes like: zero stuff samples -> bandstop filter -> do processing stuff -> bandstop filter -> discard extra samples). The main challenge will be to find a suitable bandstop filter algorithm. There are plenty of GPLv3-licensed implementations out there, but I need to do some research to find good MIT-compatible ones.

I also know you can use interpolation for oversampling.

[viral grove]

I have implemented some oversampling, not sure how good it was but I can give it a try if you want (IIRC it was a windowed sinc but afaik there are some alternatives). Since that will introduce latency, how is that handled?

[dusky mirage]

Yeah, latency is a tricky one. The audio graph algorithm does not account for latency (since that would add a lot of complexity). We can however add a DelayCompensation node.

Though Firewheel is not meant to be a modular synth engine anyway, so I don't expect phasing issues to come up that often.

[viral grove]

I have never seen minimum phase oversampling btw and have always wondered if it would be that bad

[viral grove]

Yeah, especially if it is used for bevy I think it would be preferable to keep latency to a minimum

[dusky mirage]

I did find this https://github.com/SolarLiner/valib/blob/main/crates/valib-oversample/src/lib.rs

[viral grove]

@slate scarab thanks for the pr comment, hope it's okay to discuss some of it here. I mainly made the channel count and max order generic because I didn't know where dynamic memory allocs were allowed and none of the other nodes needed it. @dusky mirage Can you give me some pointers there?

[dusky mirage]

Having them constant is definitely better since it would allow the compiler to optimize it a lot better.

[pure dove]

do const generics not get monomorphized anyways?

[slate scarab]

ya they do

[dusky mirage]

As for bevy, we could get around the usability issue by renaming the filter nodes to GenericLowpassFilterNode and then defining a type like this:

pub type StereoLowpassFilterNode = GenericLowpassFilterNode<2, 4>;

[pure dove]

yeah so the runtime overhead will be the same (or less than a flexible impl), just more annoying at compile time

[slate scarab]

I think that's a little backwards -- without const generics, the compiler will have less opportunity to optimize.

Personally I see this as a not particularly significant optimization at the cost of notably reduced flexibility, especially in dynamic contexts where it's actually a huge blocker.

I'll generally argue in favor of flexibility 😅.

[dusky mirage]

Actually, instead of having separate lowpass, highpass, bandpass, etc, filter nodes, wouldn't it make more sense to just have a single filter node type with the filter type as an enum parameter?

[slate scarab]

(I haven't benchmarked though.)

[viral grove]

Good idea, I'll fix that!

Actually, it would make sense to also consolidate highshelf, lowshelf, bell and maybe notch, since they also belong to a certain "category"

[viral grove]

I like this approach. I think that would make it usable and would abstract away "channel count" for users, which is a plus

[slate scarab]

Unfortunately, it doesn't solve the dynamic case.

For example, if someone's working on an art installation and they need a bespoke number of channels, they could not avoid writing Rust code. That is, we could not provide a dynamic, asset-based interface where programmers and artists could fully configure channels.

Naturally, many audio people are more artist than programmer, and I'd hope that we consider them heavily in our API decisions!

[viral grove]

Good point, I hadn't considered that

[faint wigeon]

thank you for thinking about this!

[slate scarab]

This isn't so niche that it would only come up here, though -- if someone wants, for example, 5.1 with a maximum order of 8, they'd also need to start writing Rust (we couldn't possibly pre-register / switch on enough types to handle all these cases!).

[pure dove]

it comes up pretty much as soon as you have a UI that allows you to place a filter after a sample, where the sample just-so-happens to have 6 channels

[viral grove]

This looks good! They seem to be based on fairly old techniques but I don't know if there has been any development. Resampling has been around for forever after all

[pure dove]

(for reference, in the FMOD API, every DSP is expected to be able to handle an arbitrary input channel count, and only gets to choose its output channel count)

(you notice if it's implemented correctly because the builtin spatializer outputs multichannel audio)

[slate scarab]

Now, to be clear, we do have a lot of flexibility in bevy_seedling.

If I so desire, I can add a required marker component on audio nodes with generics that will dynamically register an audio node when it's spawned. There are some downsides to this approach (which is why I don't just do it for everything), but it could be workable for code-driven workflows.

But again, this couldn't work for BSN or more dynamic UI editors, which is my main concern here. I think the future of asset-driven audio configuration is super bright in Bevy!

[slate scarab]

(Oh actually I couldn't do this in user space 😬)

I imagine we could also largely satisfy both desires.

With a little bit of working, we could probably have both dynamic and fully static nodes. We could make some static nodes for the most common configurations (stereo, etc) and then have a catch-all dynamic one that can take channels and order at runtime.

[slate scarab]

I'd still prefer a fully-runtime approach since that'll provide a clearer API for programmers and artists alike, but this oughta be workable.