bevy_seedling

[slim pulsar]

and if they want to avoid this error, there should prolly be a mechanism to avoid ti

[ionic sedge]

fn select_output(
    context: Single<Entity, With<AudioContext>>, 
    devices: Query<Entity, With<AudioOutputDevice>>,
    mut commands: Commands
) {
    if let Some(device) = devices.iter().next() {
        commands.entity(device).insert(AudioOutputOf(*context));
    }
}

ya know something like that

[slim pulsar]

this is exactly what I do

for (i, port) in input.ports().iter().enumerate() {
        commands.spawn((
            PortIndex(i),
            ChildOf(port_list),
        ));
    }

just in a much worse way 😆

[ionic sedge]

right i mean in practice... probably no one has made a custom backend so far 😅

[slim pulsar]

just wait until I can do a reverse FFT to convert samples into midi commands

actually I could really use a critique of my crate design. I built midix to be user-friendly, to abstract away the bytes. so for example

if let Some(note) = event.voice_event().is_note_on() {
    //note.key() -> Note::C for example
    // note.octave() -> bounded value -1/9
    pressed_keys.insert(note);
} else if let Some(note) = event.voice_event().is_note_off() {
    pressed_keys.remove(&note);
}

There's like 20 midi parser crates, but getting more visiblity so users don't actually have to investigate the midi spec has been a top priority for me

because I did

and it was really rough

anywho, yeah. what do you wanna do about this error in this case? Not fix? I wouldn't mind seeing the status of alsa-rs

fwiw, the handle to the I/O isn't Send which can be annoying

[ionic sedge]

well it's quite problematic actually -- if we make the backend abstraction better and fully integrated into the ecs.... i mean we have to try to fetch devices

[slim pulsar]

I'll definitely ask if they'd be willing to include tracing feature for users that wouldn't like alsa errors to be printed to stdout

unless there's already something built into tracing to somehow magically capture stderr

[ionic sedge]

hm, well i guess it depends on who's printing -- isn't alsa a c library?

[slim pulsar]

yes, but if taking the explanation in the issue at face value, then the handling of errors from alsa are possible in the bindings

I'm not actually sure if the result is simple or not. looking at the time this ticket has been open, I'm afraid. I'm looking into it atm

https://github.com/RustAudio/cpal/issues/605#issuecomment-1465065005 this explanation of that issue btw

catching stderr deosn't sound easy

[heady robin]

Thanks

(Please ping me when done)

[slim pulsar]

this explanation literally has a solution in code

[ionic sedge]

it's also possible just no one went through the process
audio crates in rust are generally very light on contributors

[slim pulsar]

makes sense. I'll see if I can prevent this error from firing with this solution

[slim pulsar]

I've found it

so annoying I can't make a thread, would love to walk back my steps. might move this elsewhere

[ionic sedge]

ikr
discord doesn't have the technology

[heady robin]

I mainly wish each crate had its own channel

I guess there's no way to do it really without making a "bevy ecosystem-crate" server

[ionic sedge]

okay @heady robin 0.5.3 is published with a fix for the itd node among others

I wasn't able to reproduce the panic, so I'm not like 100% percent sure it's perfect. But I simplified the indexing logic to where it really shouldn't be possible.

[heady robin]

Great, thanks

[slim pulsar]

with regard to the alsa fix

we can probably catch it in seedling if you'd like to try it

it does require some ffi

and unsafe

[ionic sedge]

hmmm
is that worth it? like it's definitely annoying but... idk 😅

[slim pulsar]

up to you :)

it's the same fix I have in the alsa pr, and we'd 5configure it specifically for linux

[potent zenith]

hello ! is it possible to suppress these alsa logs ? i get a lot of them on every startup

ALSA lib pcm.c:2722:(snd_pcm_open_noupdate) Unknown PCM pulse
ALSA lib pcm.c:2722:(snd_pcm_open_noupdate) Unknown PCM jack
ALSA lib pcm.c:2722:(snd_pcm_open_noupdate) Unknown PCM oss

[ionic sedge]

oh man

[slim pulsar]

hahaha this is what I'm talking about

[ionic sedge]

not more

[slim pulsar]

it's the exact same function from the log

LOL

[ionic sedge]

So are these actual errors? Like are no devices fetched?

Or do they still show up in the ECS.

[slim pulsar]

they show up in stderr

if you run this with &2> /dev/null, you won't get it

but who's gonna do that

it's because that error is thrown in the C lib

but my fix temporarily points the error to our own pipe

and then restores stderr afterwards

so we catch all these errors

we could even print them out to tracing

[ionic sedge]

by this I mean are there any OutputDeviceInfo components spawned

If it prints those errors and no devices are found... well, that's pretty annoying. If it does find devices and still prints those errors, that's even more annoying!

[slim pulsar]

to answer your question, it is possible. this is actually printed out by alsa lib in C, and the rust bindings don't catch stderr at the moment. so I believe if you append &2 > /dev/null, you will not get this value. I made a fix this AM, so just trying to figure out what to do in the meantime/if it gets rejected

[slim pulsar]

yes there are

the errors actually mean nothing

because alsa is compla- one second

[slim pulsar]

essentially when you do this, alsa is going to complain on devices that aren't actually configurable devices right

and EVEN if they're not output or input devices. the cpal implementation filters the results

but when you're iterating, there's an interior loop that will keep going and calling alsa's fn until the next handle is caught. when it's caught, it'll try to open the device. but in the process, there will be devices that do not open, and alsa will print out to stderr. it also returns "yeah can't open it". it's handled gracefully, just that the err log isn't caught

but you're essentially enumerating through every possible device no matter what. calling count will trigger this

[potent zenith]

haha it makes sense, thanks

[ionic sedge]

aw man 😅

[trim belfry]

@ionic sedge what is you general stance on convenience methods that help avoid boilerplate but obviously add maintenance burden.

For example I would add Volume::from_volume to avoid writing this:

                VolumeNode {
                    volume: Volume::Linear(0.0),
                    ..default()
                },

oh, right, volume is part of firewheel. @static quest would you accept this kind of PR?

[static quest]

Yeah, that's a good idea!

[trim belfry]

done

[static quest]

It would also make sense to add a from_decibels method too.

[slim pulsar]

whoops misread. agreed!

[static quest]

Are there any more nodes where more convenient constructors would make sense?

[ionic sedge]

mm probably the filters?

[static quest]

Do you want to add the from_decibels method yourself, or do you want me to do it?

[trim belfry]

added

[slim pulsar]

@ionic sedge https://github.com/diwic/alsa-rs/issues/138

IT GOT MER- well it got fixed

hehehe

ill pr cpal

[ionic sedge]

well that was an abrupt close 😅

[slim pulsar]

naw naw that's how i like it

if I hadn't done that work then I probably wouldnt be smiling rn, also alsa is sick. but yeah, think this can be fixed pretty quickly :) glad

[slim pulsar]

fuck

no...I'm still getting errors. rip.

[slim pulsar]

so apparently the fix for the other errors is...to send patches to alsa

uhh

ive never written a line of C in my life

guess it's a good time to learn!

[ionic sedge]

oh really

[slim pulsar]

yeah, maintainer said the other errors are not within the scope of alsa-rs

[ionic sedge]

watch as the alsa people say it's not within the scope of alsa
it's errors all the way down

[slim pulsar]

hahaha if I can't reach the bottom, then I'll travel in the other direction

[ionic sedge]

bevy_seedling -> Firewheel -> cpal -> alsa-rs -> alsa -> ???

[static quest]

@trim belfry Added a new review!

[slim pulsar]

@ionic sedge could you publish ur rc?

trynna get off my goofy branch

hehe

[ionic sedge]

ya ill do that today

[slim pulsar]

how do i restart paused

I feel like it's a -> play at 0, then pause, but I'm not quite there yet

*playback.playback = PlaybackState::Play {
   playhead: Some(Playhead::ZERO),
};
*playback.playback = PlaybackState::Pause;

rip really thought I was oneofdemones

[ionic sedge]

hm, im not quite sure what this means!

like you want to set the playhead to zero, but in a paused state?

[slim pulsar]

yes, after the sample has started

[ionic sedge]

mmmm you could schedule it

it’s a touch awkward unless you write an extension trait

[slim pulsar]

i tried to do that

for some reason none of my events would pass on

but i think it's something I did? not sure yet

ill issue if i find out

[ionic sedge]

BillyDM preferred to tie the playhead to the state, but I suppose it can lead to some awkward cases like this.

But scheduling should certainly work. I can demonstrate in a bit.

[slim pulsar]

that's actually kind of difficult to deal with

because you can't know where the playhead is, using the same api, when it was commanded to pause by the executor. not where it is actually paused

esp if you have something that doesn't exist now, then u pause soemthing, and then u want for that thing that didn't exist to exist, and to stop when it reaches the paused thing's playhead. you'd have to store when it is paused externally

[ionic sedge]

the param doesn’t tell you where the playhead is at any point though

you have to read the shared state for that

[slim pulsar]

ah right

[ionic sedge]

the parameter is just a command, basically

[slim pulsar]

right right mb i knew that just bad words

uh

I guess what I meant was, the command to pause

lol it's fine and it probably won't matter. you can kinda handle that as your own package author. just that restarting at 0 or even 2 is a bit difficult.

[ionic sedge]

well, restarting itself should be quite easy! but pausing with the intent that you can start playing at some later point at 0 without specifically setting it is tricky

[slim pulsar]

since I'm making basically a "track editor", i.e.

the ability to pause and move to a certain point is significant

I'm working on a system to do this

the idea is to be able to map certain events inline with the sampler's playhead

[ionic sedge]

you could pretty easily store a component that tells you where to restart, if you don’t mind the extra component

[slim pulsar]

I think that's what ill do

[ionic sedge]

but it’s definitely something to consider, i think the api would be nicer overall of we could do it that way, despite potentially increasing the complexity of the audio processor

[slim pulsar]

i found this to be really funny

[ionic sedge]

oh yeah i think that's due to bevy_inspector_egui unconditionally accessing parameters mutably, which messes with the change detection

[static quest]

What exactly is the problem?

[slim pulsar]

essentially an absolutely-positioned "pause at location command" that will

1. pause the sample
2. move the playhead to a location in the sample

namely, restarting a sample paused

not starting a sample paused. that works really well hahaha

its np ive already stitched over it

[static quest]

But if it's paused, why does it matter what position the playhead is in? You can just send a "play at zero" event to restart it.

[slim pulsar]

:o why wouldn't it matter to know what position the playhead is in? the playhead is always stateful

I'd always assume that there is some frame the playhead is always pointing to once in memory

and that there would be an api to RW

regardless if some system is constantly interacting with the head to push it forward

that's all from a user-interface perspective. I know the truth of the matter (which im trying to avoid atm) is more difficult to handle

in other words, it's a conflation of two things.
1: the ability to write to the playhead
2. the state of the system that will move the playhead forward when actually processing samples

that's kinda why the Option<Playhead> is awkward. because whenever you pass a None into any parameter here, you're saying "keep going." And that's powerful. But controlling the state to just "resume", and even "resume at X" can be met by separating the concerns into these buckets

(strong opinions weakly held)

[static quest]

The reason I did it the way I did is because it's easier to write a sampler as a state machine with as few states as possible. Either the sampler is playing from a certain position, or it isn't playing at all.

If you decouple the playing state from the playhead state, it increases complexity quite a bit.

[slim pulsar]

I get that, and like totally makes sense. The problem is the transition between those states (Paused, Playing, Stopped). All three states, in theory, have a pointer to the frame. Precluding that information is fine. In fact, the interfaces directly on playback settings and audio events are very clean. love it. The underlying variants of playback state, however, do not cover a decent amount of available capability. And I've kinda hit that bump in the road

[static quest]

This all being said, I am planning on eventually making a new sampler engine from the ground up. (One that can properly handle loops.)

[slim pulsar]

that's to say, it's not a difficult problem on my end to deal with, but the enumeration differences are not used to cover up invariants. all variants are covariant under a single playhead

that's cool, really love it lol

I'm passionate! 😆

though...stopped and paused are kidna the same thing. the former being rhetorical about its playhead position

or maybe not idk

[static quest]

The only difference between "paused" and "stopped" is where you want playback to pick up again when playing.

Yeah

[slim pulsar]

are you picking up what im putting down or am i being ridiculous 😆

[static quest]

I can see how it would be helpful to have some way to denote a "stopped" state directly in the node's state in the ECS.

[slim pulsar]

well... wouldn't these two be equivalent?
Stopped == Playhead::Paused, PlayheadFrames(0)

[static quest]

Yeah, that would be equivalant.

[slim pulsar]

damn it idk what to say

okay cool

usually im wrong 😆 . agh maybe I should make what im trying to say a little more formal then

[static quest]

Now that I think about it, I think it would be pretty simple to add a PlaybackState::Paused { playhead: Option<Playhead> }

[slim pulsar]

holdup...that's coming from firewheel?

that changes things

omg it is

I'm sorry

I thought this was seedling's enum

[static quest]

Yeah, I'm thinking it could be possible to add that to firewheel.

[slim pulsar]

you've essentially created the equivalent of an instantaneous bevy event

this makes a lot of sense. was thrown off by that

[static quest]

Though that would be a breaking change. Though it's still in rc so it's probably fine.

[obsidian tusk]

Do you think that maybe the enum and the playhead state could be separate? Like, it could be a struct with 2 fields: Playing/Paused + Option<Playhead>?

I think I’ve seen (more-or-less) that in other audio APIs

[slim pulsar]

I think you were able to say the words I couldn't find

+1

[slim pulsar]

rereading this morning...if this is the expected behavior, I may be experiencing a bug

stopped recordings do not play again

ill verify in a few to verify

[ionic sedge]

ya they’re despawned

[slim pulsar]

yeah but I have preserve

[ionic sedge]

oh

[slim pulsar]

and i can see it's not despawned

[ionic sedge]

well they’re preserved 😅 but that may be a seedling issue

[slim pulsar]

yes

english words

[slim pulsar]

nw see I might've found something i can fix. also will look into IndexMap later if I can find the willpower

that stuff puts me to sleep 💤 consideration of non-local behavior 🤢

actually can we talk about that? ill move to general

[static quest]

Maybe. The main thing is that it makes things a lot more complicated to implement if the PlaybackState and Playhead are allowed to be scheduled at different times.

What about an enum like this?

pub enum PlaybackState {
    Start { playhead: Playhead },
    Stop { resume_playhead: Playhead },
    Resume,
    Pause,
}

Hmm actually, it might be possible to have them separate. I think there is enough information to convert the state of both parameters into that enum internally.

I'm not even sure we need an Option<Playhead>. Would Notify<Playhead> be enough?

[trim belfry]

huh, I did not see any comments from you

[obsidian tusk]

Oh sorry I don't actually mean that they'd be set separately, just that it'd look like this:

pub enum PlaybackState {
    Play,
    Stop,
}

pub struct PlaybackCommand {
    pub state: PlaybackState,
    pub playhead: Option<Playhead>,
}

(or something to that effect)

Bc that expresses the similarity between start/resume and stop/pause

[slim pulsar]

Adding onto this, in this case, the None variant of playhead essentially is saving the user the step of having to retrieve where the playhead is currently I believe. Is that right?

[obsidian tusk]

Yeah exactly

I guess you could have a custom enum that makes that explicit but I don’t think that’s necessary

[ionic sedge]

Well, to be clear, doing something like this introduces opportunity for error. Or at least it leans in that direction. Fetching the playhead (performing an atomic read on the shared state) and then writing it to the component leaves a lot of time for values to diverge.

In this case, if the sample is already paused, it shouldn't be much of a problem since the playhead won't be moving. But it's something to consider in general.

[slim pulsar]

right right

ok cool!

[ionic sedge]

Now that doesn't mean we can't separate them or anything -- just that the optional playhead isn't strictly equivalent to the user providing it manually.

[static quest]

And also, the way the diffing/patching system works, each field is treated as a separate parameter.

[ionic sedge]

(Well we could make the pair a "leaf" if we want, sending both at once if either changes. Which shouldn't be too bad for performance because you'll rarely set the playhead or playback every frame!)

[static quest]

Hmm, yeah, I suppose.

But I also think it might be possible to have them separate if it was a Notify<Playhead>. The audio processor should have enough information to convert the state into the enum I proposed above.

I'll work on that later today.

[static quest]

@ionic sedge @obsidian tusk Ok, I think I got the new sampler playback API working! Let me know if you run into any issues! https://github.com/BillyDM/Firewheel/pull/72

[ionic sedge]

Awesome! I should be able to take a close look tomorrow :)

[slim pulsar]

huge. thank you

wow im so excited for this 😆 i can get rid of my terrible patchwork

when i can get my mpmc broadcast channel working it's all gonna be so speedy

[static quest]

@ionic sedge @slim pulsar Last night I thought of something to make the playback API more intuitive.

What do you think about replacing the playhead: Option<Playhead> field with play_from: PlayFrom, where PlayFrom is defined as:

pub enum PlayFrom {
    /// When [`SamplerNode::play`] is set to `true`, the sampler will resume
    /// playing from where it last left off.
    Resume,
    /// When [`SamplerNode::play`] is set to `true`, the sampler will begin
    /// playing  from this position in the sample in units of seconds.
    Seconds(f64),
    /// When [`SamplerNode::play`] is set to `true`, the sampler will begin
    /// playing from this position in the sample in units of frames (samples
    /// in a single channel of audio).
    Frames(u64),
}

That way users don't confuse this with the actual playhead that is read from SamplerState.

[slim pulsar]

That definitely would've saved me a few minutes of confusion!

big fan!

wondering though

is it easier to use seconds, or to use a Duration?

you don't use duration internally right? I don't think there's much precision loss either between micros and f64 secs

so not a biggy

[static quest]

Yeah, I don't use duration internally. F64 is easier to work with.

[slim pulsar]

gotcha. it would cover the use case of passing in negative seconds but like, who would do that?

(certainly not me a few times)

[static quest]

(Plus it would make things easier if I decide to ever make C bindings.)

[static quest]

Ok, I made the change!

[static quest]

@ionic sedge Some people were asking about your custom web backend for Firewheel.

Are you in the Rust Audio discord server? If not, I can send you an invite.

[ionic sedge]

oh ya i dont think im in there yet

[static quest]

https://discord.gg/jGN2j6pc

[lapis stone]

Is there a difference between DummyNodeConfig and EmptyConfig?

[ionic sedge]

EmptyConfig is recommended in particular because it implements Component with the bevy feature. This is really helpful for compatibility with bevy_seedling / bevy. Really, you could supply any configuration type there.

If default associated types ever come around, we'd definitely just provide a default there for convenience. But for now, you're required to set it to something.

[lapis stone]

this makes sense, but my followup question is why is there also a DummyNodeConfig? Would it make sense to unify on the one that implements Component?

it appears they are both do the same thing on the surface

[ionic sedge]

Hm, I'm not actually sure on that one.

Possibly for parity with the DummyNode?

[lapis stone]

Mm maybe, that might make sense. IMO if it is the same, I would vote for unifying on EmptyConfig. I did happen to footgun myself slightly (saw dummyconfig in examples so copied it, realized there was no derives and made my own type, not knowing emptyconfig was a thing)

though this is moreso in the firewheel field i suppose

[ionic sedge]

Yeah at the very least, if the documentation isn't sufficient there, we could improve it a bit.

[neon sphinx]

Is there a sample on how to use SamplerNode?

I currently try to use simple music crossfading. It seems like crossfade_on_seek but im not sure how I would use it.

[ionic sedge]

In most cases, you don't need to interact directly with the SamplerNode. We abstract over it in bevy_seedling. Also, its volume parameter isn't smoothed, so modifying it directly can result in stair-steppy volume changes.

The simplest way to do a crossfade now is to work with the effects on a SamplePlayer. For example:

//! Here's a complete example as it would exist in the
//! `bevy_seedling` repo.

use bevy::{app::ScheduleRunnerPlugin, log::LogPlugin, prelude::*};
use bevy_seedling::prelude::*;
use bevy_time::common_conditions::once_after_delay;
use std::time::Duration;

fn main() {
    App::new()
        .add_plugins((
            MinimalPlugins.set(ScheduleRunnerPlugin::run_loop(Duration::from_millis(16))),
            LogPlugin::default(),
            AssetPlugin::default(),
            SeedlingPlugin::default(),
        ))
        .add_systems(Startup, startup)
        .add_systems(
            Update,
            crossfade.run_if(once_after_delay(Duration::from_secs(1))),
        )
        .run();
}

#[derive(Component)]
struct MusicA;

#[derive(Component)]
struct MusicB;

fn startup(server: Res<AssetServer>, mut commands: Commands) {
    commands.spawn((
        MusicPool, // spawned in the default configuration
        MusicA,
        SamplePlayer::new(server.load("selfless_courage.ogg")),
    ));

    commands.spawn((
        MusicPool,
        MusicB,
        SamplePlayer::new(server.load("midir-chip.ogg")),
        // Each sampler in the music pool has a volume node.
        // We'll initialize this one to zero.
        sample_effects![VolumeNode {
            volume: Volume::SILENT,
            ..default()
        },],
    ));
}

fn crossfade(
    music_a: Single<&SampleEffects, With<MusicA>>,
    music_b: Single<&SampleEffects, With<MusicB>>,
    mut volume_nodes: Query<(&VolumeNode, &mut AudioEvents)>,
) -> Result {
    let fade_duration = DurationSeconds(5.0);

    // fade out A
    let (volume, mut events) = volume_nodes.get_effect_mut(&music_a)?;
    volume.fade_to(Volume::SILENT, fade_duration, &mut events);

    // fade in B
    let (volume, mut events) = volume_nodes.get_effect_mut(&music_b)?;
    volume.fade_to(Volume::UNITY_GAIN, fade_duration, &mut events);

    Ok(())
}

I should probably create an example for this

[neon sphinx]

thank you!

[ionic sedge]

You can also freely manage the fading logic yourself -- there's no obligation to schedule the events like this. You can just update the volume every frame.

The scheduling has a couple nice advantages

1. Easy
2. Won't stutter if the frame times are inconsistent
3. Optimized for our perception -- it'll create exactly as many volume steps as are required to sound smooth, but no more

[neon sphinx]

I get:

thread 'main' panicked at
.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/bevy_seedling-0.5.3/src/node/events.rs:376:15:
an event timeline should never be empty
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: bevy_seedling::node::events::EventTimeline::new
   3: <firewheel_nodes::volume::VolumeNode as bevy_seedling::node::events::VolumeFade>::fade_to

[ionic sedge]

oh exciting
what's the context of the call?

As in where you call fade_to.

[neon sphinx]

pub fn on_play_music_track(
    event: Trigger<OnPlayMusicTrack>,
    mut commands: Commands,
    mut music_player: Single<&mut Music>,
    sample_effects: Query<&SampleEffects>,
    mut volume_nodes: Query<(&VolumeNode, &mut AudioEvents)>,
    game_assets: Res<GameAssets>,
) {
    let fade_duration = DurationSeconds(5.0);

    // fade out active
    let Ok(sample_effect_active) = sample_effects.get(music_player.active_player) else {
        return;
    };
    let Ok(sample_effect_reserve) = sample_effects.get(music_player.reserve) else {
        return;
    };
    let (volume, mut events) = volume_nodes.get_effect_mut(sample_effect_active).unwrap();
    volume.fade_to(Volume::SILENT, fade_duration, &mut events);

    let song = game_assets.music.get(&event.music_track).cloned().unwrap();;
    commands.entity(music_player.active_player).insert(SamplePlayer::new(song));
    // fade in reserve
    let (volume, mut events) = volume_nodes.get_effect_mut(sample_effect_reserve).unwrap();
    volume.fade_to(Volume::UNITY_GAIN, fade_duration, &mut events);

    (music_player.active_player, music_player.reserve) =
        (music_player.reserve, music_player.active_player);
}

very similar to the example above.

[obsidian tusk]

Did you ever implement that idea of mapping animations to events?

[neon sphinx]

its probably because I replace the sample player.

[ionic sedge]

That's essentially what the fades are, really! They animate the values in the ECS in step with the audio types.

Now there isn't a more sophisticated ingegration with a general animation system, but it's certainly a possibility.

[ionic sedge]

That might be possible. I actually just worked on a patch for something mostly unrelated that made this process a bit more robust. I'll investigate this later today.

[neon sphinx]

So it seems that getting the effect like this:

    let Ok(sample_effect_active) = sample_effects.get(music_player.active_player) else {
        return;
    };
    let (volume, mut events) = volume_nodes.get_effect_mut(sample_effect_active).unwrap();
    volume.fade_to(Volume::SILENT, fade_duration, &mut events);

and settting the fading with the effects seem to be the issue, if I do it like in the other sample:

 commands.entity(music_player.active_player).insert(
        sample_effects![
            VolumeNode {
                volume: Volume::SILENT,
                ..default()
            },
            fade_in(5.0, &time),
        ],
    );

it works

[ionic sedge]

Hm, yeah it looks like there are some ways you can interact with the audio entities that aren't fully accounted for. Some of the work in setting up the effects is deferred as well, which may be exacerbating this issue. We could certainly perform more setup in observers.

[neon sphinx]

Ah I think its if you fade from the same volume to the same volume

(I was fading from Volume::SILENT to Volume::SILENT

[ionic sedge]

Oh, that would make sense. Sorry for the trouble! I'll bundle a fix for this in with the next patch release.

[neon sphinx]

seems there is still an issue if I replace the SamplePlayer

[ionic sedge]

Oh this is the one I (hopefully) just fixed! Feel free to patch the crate to test out the fix:

[patch.crates-io]
bevy_seedling = { git = "https://github.com/CorvusPrudens/bevy_seedling", rev = "78dde3b" }

[neon sphinx]

That fixes the panic, but somehow my volumes now don't work if I replace it.

[ionic sedge]

It may need a bit more work then. If I'm not able to reproduce this myself, I might ask for a bit more context later.

[neon sphinx]

Currently despawning and then spawning a new one works

[static quest]

DummyNodeConfig is not empty, it contains a field for the number of input/output channels.

@ionic sedge Would renaming it to something like ConfigForDummyNode help solve the confusion?

[ionic sedge]

haha well that's very clear!

[static quest]

Plus the dummy node is rarely used anyway.

[ionic sedge]

Although honestly I don't personally mind the current name. Do we have docs explaining its purpose?

[static quest]

Yeah, the docs say The configuration for a [DummyNode], a node which does nothing.

[ionic sedge]

Hm, and what's the utility of it again? The actual node?

[static quest]

It's used internally for the graph input/output node and for unit tests.

[ionic sedge]

A little sentence of this for config and node might be super helpful!

[static quest]

Hmm, I suppose it could make sense to make it a private struct in firewheel-graph.

[ionic sedge]

That makes sense. At least at the moment, I can't think of a very compelling reason to use it externally.

[static quest]

Although maybe it could be useful for users if they want to have a placeholder for something they intend to add later. But that could be a niche use case.

[ionic sedge]

I was thinking even a volume node would be kinda okay for that purpose. It's bypassed at unity gain anyway.

[static quest]

True

Ok, I'll make it private then.

I also realized a lot of the nodes are missing docs as well. I'll add those too.

I also just realized PanLaw would make sense to have the more generic name FadeType, since I need it for the crossfade node as well.

[ionic sedge]

maybe FadeCurve even? does that make sense?

[static quest]

Ah yeah, that's probably a better name.

[ionic sedge]

could just be Curve too of course, but I think that would be too generic

[static quest]

Yeah, I think that's too generic.

[static quest]

(Crossfading is actually exactly equivelant to panning, but instead of mixing left/right channels together you are mixing two signals together.)

[lapis stone]

thank you!

makes sense

[neon sphinx]

There already exists a random pitch, is there also already a simple way to play one out of multiple sounds?

[ionic sedge]

No, that's something that could definitely be added! @short fossil has a crate designed for this purpose though https://github.com/janhohenheim/bevy_shuffle_bag

Since it's so common for audio, I'd love to get Jan's crate upstreamed if and when bevy_seedling is.

[static quest]

Ok, the crossfade node has been added!

Oh yeah, I just remembered I was going to change the SilenceMask into a ConstantMask. I'll do that real quick while I'm thinking about it.

Man, so many little details 😅

[ionic sedge]

One step at a time haha

[short fossil]

same 🙂

it's soooo useful

I want it upstream!

[static quest]

Ok, I added the "constant mask" hint. I realized that it would be easier to have it in addition to the silence mask, instead of replacing it. Replacing it would have actually resulted in more overhead due to dereferencing each buffer to see if it is silent. https://github.com/BillyDM/Firewheel/pull/73

I haven't thoroughly tested it though.

[ionic sedge]

okay time to make a seedling DSP library to test it haha

[static quest]

In theory the logic for silence masks should be exactly the same. It's just a matter of whether I made a mistake or not. 😅

[lapis stone]

im a little confused still what this does after reading the issue 🤔 is it just outputting a constant instead of silence?

so you could efficiently send constant values without actually processing?

[ionic sedge]

Yeah in this context, "silence" is not just constant, but also zero. Constant could be any value, but it's unchanging over the course of the processing block.

[slim pulsar]

kinda hype

the midi sf2 plugin is now public, hope to get to the latency measurement stuff a lil bit later
https://github.com/RustMIDI/bevy_midix/blob/main/src/synth/mod.rs

[slim pulsar]

hmm I can't get it to run...would you be opposed to making the TimePlugin public? @ionic sedge that plugin is a dependency of mine and I need that to be registered before I can add my plugin systems

[ionic sedge]

wait so what’s the problem?

[slim pulsar]

because I otherwise have to know what backend they use

https://github.com/CorvusPrudens/bevy_seedling/pull/63 moved to here

it's not a great solution I admit

[ionic sedge]

Hm, could you simply check for the presence of the Time<Audio> resource?

[slim pulsar]

I could...but then my plugin would silently fail

wouldn't it be better to throw up instead? idk.

[ionic sedge]

no like check it as a way to see if the plugin has been added

[slim pulsar]

well if I do, then that means i will override the config of their seedling plugin, and then their plugin will panic

[ionic sedge]

Instead of checking the plugin itself

[slim pulsar]

because the plugin must be unique

it'll say "hey seedling has already been added, you can't do that"

I could pass in your own plugin settings into my plugin, but then that makes the usage of the seedilng plugin dependent on my own

and my plugin is garbage, you don't want ur plugin to be represented by me I promise

additionally, if someone comes along and builds a dependent seedling plugin, then that author and i are gonna get into a fist fight

about who should handle seedling

and id lose

[ionic sedge]

Well, wouldn’t this solution still result in the time plugin being added twice?

[slim pulsar]

it wouldn't, because I check

well, hmm actually

one moment I need to be very sure

[ionic sedge]

But if it’s not added by the time you check, will it not be added after? Or is the expectation that seedling will never be added in that scenario?

[slim pulsar]

you're right, it's still order-dependent. it works, but only if your plugin comes first.

what if you had a marker plugin...gross

[ionic sedge]

you could just panic 😅

[slim pulsar]

I need to see how other plugins handle dependent plugins

nothing immediately comes to mind though

the plugins should be order-independent imo

[ionic sedge]

I mean it’s just not a completely mature system. There’s no good way to manage this

[slim pulsar]

lemme see if I can find something better. anything come to mind about a plugin dependent on another? :o

you could always upstream my plugin :3

jk

[ionic sedge]

@short fossil does it with a few plugins i think

i can’t remember which ones

[slim pulsar]

I'll scour through his repos and then bevy's own and update accordingly

thanks

[ionic sedge]

oh probably yarn spinner

[slim pulsar]

yarn spinner has illegal code wtf is it doing. the search continues...

[short fossil]

avian_pickup depends on avian 😉

[short fossil]

Did you find the command / function traits

[slim pulsar]

indeed i did

jesus

[slim pulsar]

ok I see that now

you just panic

that's fine, I thought order independence was possible

[short fossil]

Wellll there are hacks

But Cart says to not do them until we have proper plugin dependencies

So I don’t 😄

[slim pulsar]

Yeah so essentially, I could silently fail, but I'd rather panic if the SeedlingPlugin doesn't exist if possible. Updated the PR notes

but doesn't seem like there's an approved way to have independent ordering of plugins 🪦

[ionic sedge]

check out flec's modules to see how it could work

imo, this is obviously how we should do it

it's very simple and perfectly robust

it also requires systems as entities, componentns as... entities 😅, resources as entities, and plugins as entities

but all of those are so obviously massively beneficial that I don't see them as a downside, just necessary precursors

[slim pulsar]

oh wow definitely

I don't think there's a working group for that or anything

hmm

[ionic sedge]

well... we'd need a working group for like five other things first 😅

[slim pulsar]

this design scares me and I have no clue where to begin

that's like the fabric of the ecs

[ionic sedge]

flec's docs aren't that great here -- the idea is that each item that a plugin adds, including resources, systems, observers, etc. is a child of the plugin

furthermore, adding plugins only registers it if it's not already added

Therefore, you can freely add some third-party plugin within your own, and if it was already added somewhere else, nothing clashes.

[slim pulsar]

right right...

[ionic sedge]

Furthermore, since all a plugin's items are children of the plugin, you can remove the plugin just by despawning it.

[slim pulsar]

for bevy's purposes, the plugin probably couldn't take app as a parameter anymore

I have an idea about how this would be done I think

[ionic sedge]

it's actually kinda nice to just take the world, because then you could very easily dynamically add plugins, which imo is really important!

I really want to be able to dynamically add and remove plugins.

[slim pulsar]

so uh...marker plugin? 👀

maybe TimePlugin might not be the right thing to expose

[ionic sedge]

Again, why not just panic if Time<Audio> isn't present? That serves the same purpose -- it can only be added by seedling.

[slim pulsar]

Because my plugin would silently fail essentially

I could add a startup check ig

[ionic sedge]

that's what i mean ya

[slim pulsar]

:(

[ionic sedge]

the resource would be visible at the same time and in the same context as some marker plugin

[slim pulsar]

ok ok let's say I didn't need the Time<Audio> resource right

what if all I did was register nodes

cuz it's not necessarily that I need to have the Time res

i just added it because it was last in the plugin additions

[ionic sedge]

you could add a system when debug_assertions are active that checks if seedling has been added

[slim pulsar]

ig that's fine, something feels wrong, but it's an 80/20

ill do that

[heady robin]

@ionic sedge the same playtester keep encountering these
(On my machine)

[ionic sedge]

That's frustrating. Is there any kind of reproduction? Or is it just infrequent and difficult to predict?

[heady robin]

I have no idea what she's doing to cause that. I tried replicating it myself but honestly it just feels so random

I also asked her to play more after that but it didn't happen again

[static quest]

Dang, I found an annoying bug with the crossfade node where if one input source is marked as silent, then there is sometimes a strange buzzing noise. This happens in both the main branch and the "constant_mask" branch, so it's not the new code I wrote causing the problem.

[static quest]

Hmm, it appears to be a bug with the silence flag system itself. Debugging this is going to be a bit tricky.

Oh, I figured it out! The silence flag system isn't accounting for the number of frames changing across process cycles.

That case totally slipped my mind.

[ionic sedge]

oh nice!

[static quest]

All right, the silence flag bug has been fixed! Also going over the new constant mask again, I'm very confident the silence flag logic is exactly the same, so I'll go ahead and merge it.

@ionic sedge Is the new playback API PR good to you? If so, I can go ahead and merge that as well. https://github.com/BillyDM/Firewheel/pull/72

[ionic sedge]

To be honest I haven't had the chance to look 😅 but I should be able to get to it today after work. Definitely very insterested since it will have consequences for bevy_seedling's API

[static quest]

Cool, I just want to make sure things are ready before the next bevy release. 🚀

[neon sphinx]

It would be neat if the Volume type would implement Serlialize (to save volume settings easily).

Its not difficult to work around, but would be convenient

[ionic sedge]

I know ur gonna say reflect instead

[short fossil]

FYI you don't need Serialize if you have Reflect

[short fossil]

you predicted it

[ionic sedge]

it should be reflect 100%, let me double check that it is

[neon sphinx]

ah its behind a feature flag

[short fossil]

https://github.com/bevyengine/bevy/blob/main/examples/reflection/serialization.rs

[ionic sedge]

coincidentally i just made a little reflect deserializer mere moments ago

[neon sphinx]

But if a crate I use expects a Serialize can I still work around that?

[short fossil]

generally no, but crates AFAIK shouldn't expect that

[neon sphinx]

currently using: https://github.com/johanhelsing/bevy_pkv

for simple save files, and that one does I think>?

[short fossil]

then imo it shouldn't 😛

[neon sphinx]

or... hear me out... Volume could be serialize behind a feature flag? 😉

[ionic sedge]

i might be crazy but I think you can work with erased serde maybe?

[ionic sedge]

it can certainly be done, but that'll need a PR or issue on firewheel

[neon sphinx]

Should I make an issue, or you think thats not really in scope?

(its easy enough to work around it)

[viscid plank]

This is my preference 🙂

[ionic sedge]

It's certainly very reasonable for Firewheel. While a number of us prefer bevy_reflect over serde for this, Firewheel is engine agnostic and might therefore benefit from serde derives on its core types.

not that bevy_reflect is really tied to bevy in any particular way

[static quest]

Ah yeah, serde derives

[short fossil]

it is, as long as it publishes a semver breaking update in lockstep with the rest of the engine

Hence why I have to bump my engine-agnostic rerecast crate to 0.2 for the Bevy update

[brazen monolith]

there's been desire to split reflect out into a completely separate crate

[ionic sedge]

well we've had to do that for firewheel too 😅

[short fossil]

yeah I believe said we should be good to even put that crate on 1.0.0

[ionic sedge]

mainly because of the component derives though

[brazen monolith]

maybe? idk what 1.0 means from that perspective. I feel like it still gains features and such

[short fossil]

it means crates like rerecast can depend on it without having to bump their own version when bevy_reflect publishes an update

1.0 doesn't mean "no new features"

[short fossil]

I wrap all components in the bevy crate for that reason

but it's meh

[ionic sedge]

but 1.0 doesn't give bevy_reflect any more leeway -- it still wouldn't be able to push semver breaking changes without everyone updating

[brazen monolith]

(I don't have an opinion on what 1.0 means in this scenario, but it will be different to different people like it always is)

[short fossil]

it's more about signalling a commitment to not break stuff

[ionic sedge]

Hm, is it at that stage?

[short fossil]

Alice said that the derive macro could possibly be

I have no insight

[neon sphinx]

Created the issue, hope its clear enough

https://github.com/CorvusPrudens/bevy_seedling/issues/64

[ionic sedge]

mm, I mean it seems like it
although there are some corners of the crate that seems a little underbaked even now

[short fossil]

yeah the idea is to split off the derives into a stable crate

[brazen monolith]

I think the ecosystem gating serde/serialize implementations for everything is a fairly large problem. it means serialization changes in ways that aren't programmatically discoverable

[short fossil]

so that other crates can depend on that safely

[ionic sedge]

Yep that's clear! Although sorry for not clarifying myself -- Volume is actually a part of Firewheel. Feel free to move the issue over there!

[brazen monolith]

(but I'm absolutely not intending to block say, the above issue in any way. just communicating that it does in fact become an issue)

[ionic sedge]

Why do we do that at all, though? Surely that should just not be the default behavior of bevy_reflect when serializing or deserializing types. You should have to opt in to the serde representation at the ser/de call site.

[short fossil]

agreed, I'm lowkey considering dropping them in my crates

[brazen monolith]

you do have to opt-in, but "everybody" thinks serde's Serialize/Deserialize is required for serialization, so it propagates everywhere as a default

avian only has a serialize feature, for example, because it was thought to be required for de/serialization

[short fossil]

yeah, when you upstreamed that serde-less example it was quite the revelation for me

[static quest]

I'll add serde derives here soon. I'm going to be busy this weekend, so it may be Monday when I get to it.

[brazen monolith]

you can always go from <file> -> serde_json::Value -> bevy types without Serialize

[brazen monolith]

yeah that's definitely why I did it, so happy it is communicating the right thing 😆

[short fossil]

I tried posting it whenever I could to people when they were talking about Serialize hehe

[brazen monolith]

for context: I've been gathering my thoughts on the topic to make better upstream suggestions with concrete examples, so this whole firewheel/seedling/Volume discussion is useful for that

[ionic sedge]

Oh is that the standard flow (as in an intermediate representation before spawning into the world)?
My reflect deserializer just shoves entities into the world as it goes 😅

[brazen monolith]

would be interested to see what you're using. part of what I'm thinking of is basically documenting what "should" be the standard flow, and then suggesting we advise that to the ecosystem

right now everyone's kinda guessing at what's supposed to happen, which is how a lot of people land on serde::Serialize/etc

[ionic sedge]

Yeah 😅 I do feel like we're lacking a kind of standardized approach.

For example, what do people do even with serde when they want to serialize and deserialize relations? My serializer and deserialize just hold a reference to the world to follow the relationship edges.

In other words, I'd want something like this:

commands.spawn((
    Name::new("A"),
    my_relationship![
        Name::new("B"),
        Name::new("C"),
    ],
));

to serialize to:

{
    "Name": "A",
    "MyRelationship": [
        { "Name": "B" },
        { "Name": "C" }
    ]
}

ya know

[brazen monolith]

(we're getting a bit offtopic so lmk if you want to move out of seedling's channel)

Relationships in, for example, the current scene example in the bevy repo, require the entity ids to serialize, then get MapEntities called on them

if you throw a relationship in the scene example and check the file you'll see how it sets up

all of the entity ids for a scene already exist, and are mapped (whether from key names or in component values) into the new world when spawning

[ionic sedge]

ohhh is that where a lot of that code is? i'm just running bevy_ecs atm without scenes

I guess I should reference that.

[brazen monolith]

the reflection examples have their own folder, the scene example is just a "quick reference" for what relationships look like when serialized

[lapis stone]

thinking through a convolution node - in the design doc for Firewheel it mentions the ability to blend between multiple impulse responses. I was curious how to approach storing IR samples in an ECS friendly way to fade between. I could just allow for 2 IR samples in the Convolution node & allow for mixing between them, but not sure if that is a good idea. Any suggestions? I see the sampler node has a fade on seek option, but it looks to just be a declicker, so I don't think it stores multiple samples like I imagine this might.

[static quest]

Yeah, the fade on seek is just a declicker.

Though actually, the ability to blend between multiple IRs may not be necessary. The user can just spawn two convolution nodes and a crossfade node to accomplish that.

[lapis stone]

oh, that's true, and would be way easier to interface with

[slim pulsar]

@ionic sedge hey I think I've got everything published to open source to run that benchmark

1. midix updated
2. soundfont synth published
3. bevy plugin published
4. mpmc broadcast channel published

I don't quite know what to benchmark though, would appreciate some guidance!

it could be a good blog post! hehe excited. lemme know what you have in mind, I cannot find your og message

[ionic sedge]

The easiest way to test the latency is to record yourself pressing a key, waiting for the corresponding sound. You can do this with your phone for example. Then, measure the delta between the key press’s transient compared to the sound it triggers.

If the latency is very low, they’ll basically be indistinguishable.

[slim pulsar]

sounds good! hopefully examples can have submodules, i'll put it on the plugin repo

[trim belfry]

Finally got fade_in/fade_out to work ok-ish. @ionic sedge if you want I can add an example

[ionic sedge]

I don't think there's any desktop audio 😅
But did you feel that it was difficult to achieve? Were there any roadblocks?

[trim belfry]

huh, there is, but I should've made music louder.
Yeah, the biggest for me I guess was figuring out when to run crossfade system. With your suggestion (once_after_delay 1s) it did not run at all, and per frame it eventually crashed because event queue is not supposed to be empty.

I ended up with this condition for run_if, not sure how expensive it is:

fn crossfade_is_active(
    fade_in: Query<(), With<FadeIn>>,
    fade_out: Query<(), With<FadeOut>>,
) -> bool {
    !fade_in.is_empty() || !fade_out.is_empty()
}

But this does not feel right either since it leads to this sometimes:

2025-09-28T11:11:25.523488Z ERROR firewheel_core::log: Firewheel scheduled event buffer is full! Please increase capacity to avoid audio glitches.

Other issue I had is I tried to just modify the VolumeNodes of the entities directly, but it did not affect the volume of said entities. My guess its because the only thing that you can change after you spawn volume node is SampleEffects, right?

[ionic sedge]

Other issue I had is I tried to just modify the VolumeNodes of the entities directly, but it did not affect the volume of said entities.

Hm, which entities? Can you clarify on this point? (I'm curious if I can make the process any clearer.)

But this does not feel right either since it leads to this sometimes:

Nah we can probably stand to just increase the size of the buffer by default.

and per frame it eventually crashed because event queue is not supposed to be empty.

Hm, were you running it every frame? i.e. calling fade_to constantly? The idea is that it should just be called a single time to start the process, and then bevy_seedling will handle the rest.

[trim belfry]

here is my setup

[trim belfry]

So my first attempt was to just simply to subtract volume from volume node manually each frame, but for some reason it was not happening - I mean I saw the values going down and eventually the audio despawned because it hit my threshold, but I was kinda expecting to hear that but the audio that was supposed to be faded out was not getting decreased

[trim belfry]

yeah I got that eventually and will refine my approach to just call it once, but its working at least!

[ionic sedge]

ya that's definitely not how it should be
if you happen to have a repro on hand, I'd love to take a look and see if we can get to the bottom of it!

[trim belfry]

oh, nice to hear, lemme see

here is an MRE, do I do something stupid here?

[ionic sedge]

Ah, I see. And you mentioned you updated it to this, right:

    commands.spawn((
        MusicPool,
        SamplePlayer::new(handle.clone())
            .with_volume(settings.music())
            .looping(),
        sample_effects![(
            VolumeNode {
                volume: Volume::SILENT,
                ..default()
            },
            FadeIn,
        )],
    ));

[trim belfry]

yep

[ionic sedge]

Yeah just to reiterate -- node components on the sample player are not considered in processing.

but

[ionic sedge]

this should be totally acceptable

I'd only expect it to overfill the buffer if the framerate is extremely high

otherwise it shouldn't be able to get close

like anything less than, say, 500fps should be good

[trim belfry]

well I do run 160fps, so I guess I might hit that 😅
oh, nvm

Yeah, I think fade_to is much better approach though, so, I might just play with the system scheduling a bit to make it work in a sane way

anyway, after I do that - do you want an example to bevy_seedling?

[ionic sedge]

yeah even at 160fps that's like... 6 events per (default) audio processing block (when manually fading two volume nodes)

and the buffer's several hundred events by default

[ionic sedge]

hm possibly, although it depends on the complexity of the example
I think optimally we'd just slightly adjust the example I gave above to mirror a more real-world example

for example, triggering fading by a key press

(That would also be a good test for managing fades that may interrupt each other.)

[trim belfry]

oh, might be fun

[slim pulsar]

wait

omg

super rust hack discovered

you can define variables in let chains

which is not immediately obvious

at least ime

I mean tuple interiors make sense

but wow cool

yeah makes sense checks out good for it

[ionic sedge]

oh yeah ig because let var is a pattern
it just happens to be infallible

[trim belfry]

another dumb question on fade_to: if I fade to Volume::SILENT of the entity in the MusicPool, it will try to silence the whole pool, right? but if I at the same time fade some other sample in, like Volume::UNITY_GAIN will the two fade instructions fight each other?

[ionic sedge]

Well there are maybe two "entity in the MusicPool" that you're referring to.

Pools are structured like this:

┌───────┐┌───────┐┌───────┐┌───────┐
│Sampler││Sampler││Sampler││Sampler│
└┬──────┘└┬──────┘└┬──────┘└┬──────┘
┌▽──────┐┌▽──────┐┌▽──────┐┌▽──────┐
│Volume ││Volume ││Volume ││Volume │
└┬──────┘└┬──────┘└┬──────┘└┬──────┘
┌▽────────▽────────▽────────▽┐
│Volume                      │
└┬───────────────────────────┘
┌▽──────┐
│MainBus│
└───────┘

This is what the music pool looks like anyway. If you were to reach in and perform a fade on the individual volume node of each sampler, that will not affect the whole pool. (It's not recommended since any changes outside of the pool will overwrite that node, but you could do it.)

If you triggered a fade on the terminal volume node -- the one that exists on the SamplerPool entity itself -- that would silence the whole pool. So it wouldn't strictly "conflict" with another fade on one of the individual nodes. It'll just silence them all outright.

lmk if this clarifies anything 😅

[trim belfry]

just out of curiosity - terminal volume node is not the one I get with Query<&VolumeNode, With<MusicPool>>(I get that MusicPool is just a SamplerPool), its all the volume nodes that sink into it, correct?

[ionic sedge]

Well, that one is "terminal" for that pool (and what I was referring to above). But yes, there is another one (the MainBus is also a VolumeNode) that you could consider terminal for the whole graph.

[trim belfry]

soo, I tried setting up an observer OnAdd, FadeIn but I guess it's firing too early and effects are not yet spawned?

// another system
...
    commands.spawn((
        MusicPool,
        SamplePlayer::new(handle.clone())
            .with_volume(settings.music())
            .looping(),
        sample_effects![VolumeNode {
            volume: Volume::SILENT,
            ..default()
        }],
        FadeIn,
    ));
...

fn crossfade(
    _: Trigger<OnAdd, FadeIn>,
    settings: Res<Settings>,
    fade_in: Query<&SampleEffects, With<FadeIn>>,
    mut volume_nodes: Query<(&VolumeNode, &mut AudioEvents)>,
) -> Result {
    let fade_duration = DurationSeconds(FADE_TIME);

    for effects in fade_in.iter() {
        let (node, mut events) = volume_nodes.get_effect_mut(effects)?;
        node.fade_to(settings.music(), fade_duration, &mut events);
    }

    Ok(())
}

Encountered an error in observer crossfade: audio effects query matched no entities

[ionic sedge]

Yeah relationships can be tricky like that

I would expect it to work, but it may be down to the ordering of bundle effects

or, hold on a sec

hard to read on phone

[ionic sedge]

This observer doesn't seem right. It shouldn't be iterating over all fade_in:

fn crossfade(
    trigger: Trigger<OnAdd, FadeIn>,
    settings: Res<Settings>,
    fade_in: Query<&SampleEffects>,
    mut volume_nodes: Query<(&VolumeNode, &mut AudioEvents)>,
) -> Result {
    let fade_duration = DurationSeconds(FADE_TIME);

    let effects = fade_in.get(trigger.target())?;
    let (node, mut events) = volume_nodes.get_effect_mut(effects)?;
    node.fade_to(settings.music(), fade_duration, &mut events);

    Ok(())
}

Note that this won't work for sample players that don't mention their effects. That's because effects are added / corrected in a deferred way in Last. I'd really like to adjust that so it happens immediately, but it's kinda tricky.

[trim belfry]

yeah, it results in the same audio effects query matched no entities unfortunately. shame

[short fossil]

With a bit of luck I'll be experimenting with seedling + audionimbus later. Should I come crying here or in #1236113088793677888 ?

[ionic sedge]

hmmmmm idk it might be interesting and constructive for #1236113088793677888 tbh

[ionic sedge]

So I've been looking at this today, and I'm not actually sure how this might be happening!

In my own tests, it works as expected. Replacing the sample player does not perturb the effects. I'll note that there might be a number of reasons for why this is happening that aren't necessarily something wrong with bevy_seedling.

One thing that does seem to be happening (and maybe this is contributing to your issues here) is that if a sample takes a long time to load, a fade animation can get clobbered by the deferred loading.

[ionic sedge]

Whew okay sorry for the delay @static quest -- the playback changes look great! there should be no issues integrating them on my end. There could always be some tricky aspect with stop / completion detection (we've run into those before), so I can't say it's 100% perfect yet. But it certainly looks good to me.

Let me know if you'd like to publish a release in the next couple days. I can integrate it and verify there aren't any gaps.

[static quest]

Cool. And if you could go through and add/remove bevy derives here soon, that would be great! I mainly want to try to avoid any more breaking changes to the core library and the essential nodes.

[neon sphinx]

I used the patched version for now and havent encountered any more issues.

(but also implemented it in a slightly different way)

[ionic sedge]

Okay turns out fuzzing worked great 😅 it failed immediately.

Can you guess what the result of -1.2271447e-13.rem_euclid(31.0) is? (Spoilers, it's not the mathematically correct value.)

I fixed this particular issue, but also adjusted the wrapping logic to happen in integers anyway so it truly can't produce an out-of-bounds index. Sorry for the trouble, but it's definitely fixed now.

I just published bevy_seedling v0.5.4 with this fix and a number of others.

[static quest]

Yay, floating point shenanigans! 🥲

[short fossil]

Is the negative duration thingy also released yet?

(no pressure if not, I can live with the patched dependency for now)

[static quest]

Oh yeah this reminds me, I guess we should think about how to deal with denormals. https://mu.krj.st/denormal/

We could simply add a field to the firewheel config to disable denormals globally for the whole processing loop. (Though technically Rust considers disabling denormals as undefined behavior, but I've personally never run into any issues with it.)

[ionic sedge]

That's in firewheel-web-audio, and I haven't published that mainly because it's a pain (the fix is on the rc banch, etc.). If it takes much longer to get the Bevy rc out, I'll get another version published before then.

[short fossil]

Oh no in that case I can definitely wait until 0.17 proper 🙂

[ionic sedge]

There's an unstable feature for this I believe -- the algebraic_* family.

https://github.com/rust-lang/rust/issues/136469

not sure what the status is on it atm

seems to be moving along

[static quest]

Ah ok. I think what I'll do for now is have a feature flag for firewheel-graph to disable denormals and call it something like unsafe_disable_denormals (And not have it enabled by default of course.)

[ionic sedge]

ugh, had to fix a failing docs build (note to self, update the doc_auto_cfg feature everywhere)

v0.5.5 🥲

[ionic sedge]

Okay so for component derives, the only ones that are strictly necessary are those on nodes and their configurations, more or less.

So, for example, the following are not necessary:

• DistanceAttenuation
• DistanceModel
• SvfType
• ChannelConfig (not a node configuration itself)
• Vec2 / Vec3
• MusicalTransport / TransportState

Now, I think it's important to stress that these are not strictly required, but that doesn't mean it's not (potentially) useful. Components are very powerful, and so it's often very useful that a type can be one. Also, Cart in particular really wants an audio API that feels natural and uncompromising for Bevy. That means we'll want as few wrappers as possible, which is what you'd otherwise need to do to insert a non-component.

With that said, I think only one of these items is actually important: the transports. The rest are not all that valuable on their own, and can safely be removed.

While I haven't integrated the transport in any special way in bevy_seedling yet, I anticipate that those types being Components would besuper helpful. I wouldn't remove them quite yet.

Keep in mind that the Component macro isn't recursive; a Component's fields don't all need to implement Component.

This is not true for Reflect. Reflect will naturally propagate through all the fields of any top-level type that needs it.

I'll have to continue this tomorrow, but hopefully this is a helpful start.

[heady robin]

Thanks

And it's for bevy 0.!6 still, right?

[ionic sedge]

oh, no this is 0.5.5 (that’s what you’re using, right?)

for Bevy 0.16

[heady robin]

Could you please make the crash-fix-patch available for 0.16?

[ionic sedge]

It is

Unless we're talking about some other patch?

The only fix not on 0.16 is for firewheel-web-audio, which only seems to occur on newer versions of Chrome on Linux.

[heady robin]

Oh ok thanks

I meant the index out of bounds one

So I should route to 0.5.4

[ionic sedge]

ya, a cargo update should fix that up

[static quest]

Hmm, do you think I should rename CrossfadeNode to MixNode?

[ionic sedge]

i think that's probably better tbh

Crossfade gives the impression of narrower utility that it actually has

[static quest]

Ok, I renamed CrossfadeNode to MixNode node. I also renamed a few DSP methods to make it clearer what they do.

[slim pulsar]

opinions on fundsp integration?

[ionic sedge]

pretty straightforward, I believe I have an implementation lying around somewhere

as long as you're chill with non-audio inputs coming from the audio graph (like for a filter frequency for example), it's quite easy to do

As soon as you want non-audio-rate parameter changes, it's kind hard to abstract over

[slim pulsar]

ahha

that's where trotcast comes in

that sounds good, thanks. am planning on trying it

[ionic sedge]

hmmmmmmmmmmm well now that i think of it.... it might be possible actually?

but let me see if i can dig up the current impl

[slim pulsar]

that'd be lovely, thanks

[ionic sedge]

oh ya i was making a little radio effect demo
but then i was disappointed that fundsp didn't have quite everything i wanted

[slim pulsar]

damn

[ionic sedge]

it really needs a compressor
but i bodged it with a limiter
it sounds okay actually

here it is if you wanna give it a run https://github.com/CorvusPrudens/bevy-radio

it's pretty easy to slap a fundsp processor anywhere:

fn radio() -> impl Bundle {
    let input = highpass_hz(400.0, 2.0)
            >> bell_hz(1200.0, 4.0, 1.5)
            // poor signal quality simulation
            // >> shape(SoftCrush(2.0))
            >> shape(Tanh(16.0))
            >> lowpass_hz(2800.0, 4.0)
            >> limiter(0.005, 0.250) * 0.25;

    let noise = white() * 0.1
        >> highpass_hz(400.0, 2.0)
        >> bell_hz(1200.0, 4.0, 1.5)
        >> shape(SoftCrush(2.0))
        >> lowpass_hz(2800.0, 4.0) * 8.0;

    let amp_adjustment = map(|i: &Frame<f32, U1>| (0.9 - i[0] * 12.0).clamp(0.0, 1.0));
    let branch = pass() & (meter(Meter::Rms(0.1)) >> amp_adjustment * noise);

    FundspConfig::new_downmix(input >> branch)
}

fn play_sound_with_radio(mut commands: Commands, server: Res<AssetServer>) {
    commands.spawn((
        SamplePlayer::new(server.load("divine_comedy.ogg")),
        sample_effects![radio::radio()],
    ));
}

kinda based if i do say so myself

[slim pulsar]

thank you! very cool

why does ProcessStatus::outputs_not_silent() always hit so hard

[ionic sedge]

the processor is in the config because it doesn't necessarily make sense in the parameters itself

and updating the config will cause bevy_seedling to recreate the node

so it's conceptually nice

[trim belfry]

I was just about to search how to make a radio effect! AWESOME

[ionic sedge]

There are many ways you could do it (and definitely lots of room for improvement), but it should at least serve as a starting point!

[trim belfry]

would you like a 0.17 bump PR?

[ionic sedge]

Well, really I want to land a two things:

1. Firewheel's new playback API
2. the fixes from 0.5.5 (which branched off from 0.6.0 in an annoying way)

both of these aren't the most simple
i may have to wait until the weekend to get enough time!

I'd like to get the playback API in there in particular because it's more user friendly and definitely breaking.

[trim belfry]

no worries, I seem to have enough stuff with migration for the whole week

[static quest]

@lapis stone Actually, there is a way to get some declicking without needing to crossfade between two IRs. It works by enabling a slew rate limiter for a brief period of time. It's not as good as crossfading, but it should be a lot better.

I can add a DSP helper for that declicking method.

[static quest]

Actually I found using a lowpass filter was easier to implement, same idea though. Anyway, I added a LowpassDeclicker struct. Let me know if it works for you! (I haven't really tested it yet.)

[static quest]

@ionic sedge When do you think I should publish a new release? Should we wait until the convolution/reverb nodes are added?

[ionic sedge]

I'd love to get something out this weekend! (We'd have to bump the Bevy version to 0.17 proper anyway to get Firewheel in stable Bevy.)

idk what the timelines on fancy reverb might be

although I think that could be done with a patch release

[static quest]

True. The convolution node is nearly done. I'm not sure how far along your reverb node is.

[ionic sedge]

oh yes that, i mean that shouldn't take long at all

moving over freeverb

[static quest]

Also feel free to add/edit any docs you want. (I'm a bit sloppy when it comes to docs.)

[neon sphinx]

Is there a way to delay playing of a sound?

[ionic sedge]

ya

https://docs.rs/bevy_seedling/latest/bevy_seedling/sample/struct.PlaybackSettings.html#method.play_at

[lapis stone]

ah so not storing any crossfade data, but basically interpolating between two samples & ensuring its band limited?

[static quest]

Yeah, pretty much

[lapis stone]

@static quest taking a look at the new helper - this goes on the IR samples as they are changed, correct? If so I am not sure if that is possible as the fft-convolution copies the given IR sample into its own buffer, unless i am misunderstanding. (i imagine it is possible to modify the crate to enable this, assuming partitioned convolution doesnt do anything funky)

[static quest]

It goes on the output of the node.

When you change the sample, just call begin(). The rest will be handled for you.

It's literally just applying a lowpass filter on the output for a brief period of time.

[lapis stone]

okay huh alright, for some reason i assumed that wouldnt declick after the convolution was processed

like it would still be audible but not horrific crackling, right?

[static quest]

"Clicking" is just an impulse at the nyquist frequency (or maybe it's half nyquist, I can't remember). All frequencies are effected by this impulse, but high frequencies are effected much more than low frequencies. So adding a lowpass filter will remove most of the click.

[lapis stone]

hmm, im not sure that it is improving things, unless i need to fiddle with the fade time or something (1st no lp declick, 2nd with lp declick). (ignore the weird bg noise in the first lol)

[lapis stone]

i think this was touched on recently, but is there a preference between using const generics vs. config for channel count? I imagine the latter is a bit more user friendly

its easy enough to store things as a vec instead of an array if based off config, but the downside is it would seem more breakable since users could technically resize vecs and mess up expectations of the processor

i suppose another alternative could be to simply alias some types to the generics, like type EchoStereoNode = EchoNode<2>or something

but in any const generic case, it also means every node with a different const also must be registered individually, which seems not very friendly

[viscid plank]

I would default to config unless you're confident that the added performance / correctness of const generics are correct in general

[obsidian tusk]

There’s a pattern where you use both, but make the runtime one optional and fallback to the const generic. Then LLVM can, if it likes, generate a fast path with const channels and a slower path with dynamic channels. The runtime one can be Option<NonZeroU32> (or whatever int size you want) which makes it just a jnz at runtime for the fast path. Const loop variables make a huge difference in my experience, although for channel count it might not matter

I work on audio software and in our product we have the actual per-impl work done in a function which uses runtime variables but then have a default-implemented trait method that takes const params and just passes that to the dynamic impl, that way LLVM can trivially inline the runtime-var-taking call into the const-taking one and you get both versions at max performance while only having to implement the behaviour once

[static quest]

Do you have the code for me to look at?

[lapis stone]

this is the relevant change

with settings change_ir_declick: LowpassDeclicker::new(sample_rate, 0.2), (tried with other values than 0.2 without much difference)

i mean, i feel like that makes sense though? If I take a unit impulse in bitwig, add a convolution, and slap a LPF on the end of the chain, i would expect to still hear a lot of the click, albeit with the high end attenuated

[static quest]

Hmm, ok. I'll look into it more when I'm back at my PC.

[ionic sedge]

okay porting freeverb is taking a little longer than I thought, but it's almost done

should have that up tomorrow

[lapis stone]

are you handling anything with mono/stereo or is it always stereo? curious how youre approaching if so

im assuming the use case outside of stereo is pretty low but no idea really

[ionic sedge]

The code I lifted is a fixed stereo processor, and I haven't changed that aspect tbh

[lapis stone]

makes sense, wasnt even sure if freeverb could abstract over n channels either

honestly kind of debating if i should just rip different channel support out of convolution. on one hand convolution is expensive, so mono could be nice, but also i cannot imagine a single use case where it would be required in the context of a game, and it does make the end user api not as friendly

ig maybe limited hardware or something but i think in that case theyd probably be using their own audio solution anyways. ambisonics is the only thing over stereo that i can tell

[ionic sedge]

what about typical 5.1 or 7.1 channel configurations?

[lapis stone]

is convolution typically processed/generated with all inputs, or is it downmixed? like i imagine samples in a surround setup are still recorded in stereo, right? im not sure if MixDSP supports past stereo looking at the api but i might be missing something

[static quest]

@lapis stone Ok, after taking a closer look at the convolution node, I have quite a few notes. 😅

• My assumption that you have to delay the dry mix was wrong. FFTConvolver is actually zero latency, so there is no need to buffer the dry input (in fact doing so produces a crackling noise).
• Reading the code, FFTConvolver does its own buffering, so I think the correct way to use it is to pass in a constant block size to FFTConvolver::init (instead of a info.frames). I'm not sure what the optimal block size is (there's probably an optimal size that balances buffering overhead and cache efficieny). We might need to ask the developer on that, or run our own benchmarks.
• You have will_pause logic, when the correct way is to just check if the play/pause declicker has settled before returning ProcessStatus::ClearAllOutputs.
• The play/pause declicker doesn't declick if impulse_response is None.

Also yeah, my lowpass declicker doesn't work as well as I hoped. There is one more declicking method I want to try. I'll work on that here soon.

Oh yeah, and it looks like FFTConvolver::init allocates. We might need to ask the developer to add a way to preallocate a maximum IR size.

OH wait a minute! There might be a much better solution that would also solve both the declicking and the allocation issue!

Instead of sending an impulse response to the node processor, the node could just create a new FFTConvolver and send that to the node processor. That way, you can just crossfade between the new convolver and the old convolver to declick, and then drop the old one when declicking is done. (ArcGc will make sure that the old one is dropped properly).

Oh yeah, and we need to make sure that the sample rate of the IR matches the sample rate of the stream.

It also looks like initializing an FFTConvovler is quite expensive, so it's probably best to do it outside of the audio thread anyway.

[lapis stone]

thanks for the notes! got most of these cleaned up.

as for the fft convolver, want to ensure I understand correctly; as I understand, instead of the pub impulse_response: Option<ArcGc<dyn SampleResourceF32>>, field on ConvolutionNode, i could instead create a wrapper over FFTConvolver & make that the field and pass that into the processor - is that right>

[lapis stone]

e.g., i am thinking ConvolutionNode could have a impulse_response: Option<ArcGc<ImpulseResponse>>, field, with signature

pub struct ImpulseResponse(Vec<FFTConvolver<f32>>);

impl ImpulseResponse {
    pub fn new(sample: impl SampleResourceF32, partition_size: usize) -> Self {
        let num_channels = sample.num_channels().get();
        let convolvers = (0..num_channels)
            .map(|channel_index| {
                let mut conv = FFTConvolver::default();
                conv.init(partition_size, sample.channel(channel_index).unwrap());
                conv
            })
            .collect();
        Self(convolvers)
    }
}

though, I need mutable access to the impulse response, which I dont think arcgc provides 🤔

oh maybe i actually keep the arcgc over sampleresource and just impl diff/patch for ImpulseRespose?

[lapis stone]

@static quest is it intended that update_memo always calls notify on Notify<()> types? I noticed that my stop event was triggering any time any other param changed

[ionic sedge]

Hm, it shouldn’t (unless you modify that value)

But I can double check that.

[slim pulsar]

hey @ionic sedge , on the topic of passing in a handle into a config for an AudioNode::Configuration, since I need the the actual value of the asset, I'm thinking of making the real AudioNode::Configuration private, and updating it in a system once the asset has actually loaded. thoughts?

[ionic sedge]

Yeah that would work exceptttt it does need to implement Default, which is kind of annoying.

So if it's not ready, idk I guess you'd just create a synth without any sound font.

I'd love to improve this situation, but it's a bit tricky!

[slim pulsar]

no worries! ive done a mem::transmute(ptr::null()) before

don't sweat it

[ionic sedge]

oh good!

[slim pulsar]

for real, if it becomes a serious issue as I work through sfz, I'll see if I can make a helpful adjustment :)

I was trying to see what SamplePlayer does, but it seems well-embedded

not really conformant to the node types

[ionic sedge]

ya it's deliberately different, but the SamplerNode might be something to reference? although all it has to transfer is the sample asset

Which doesn't require any kind of allocations within the processor.

[ionic sedge]

Looks like this is coming from an optimization that doesn't handle the notify counter as expected.

This is also causing issues for me.

It can receive such optimizations, but they need to include the counter. I'll include that in my PR for the freeverb node.

maybe tests really are good

[slim pulsar]

rq. if I just set output samples to all 1s, is my speaker cone just gonna be fully pushed in

[ionic sedge]

yes
but also silent

[slim pulsar]

does that mean i could somehow stop all other audio from playing

[ionic sedge]

DC offset can be problematic

but usually it'll just distort things a bunch, even if you try to make it really high

[slim pulsar]

i noticed that with my lil saw wave sandbox, if I have a phase from [0,1), and the frame val is 2 * phase - 1, and I set a lower value for the phase on reset (so now it's [-0.5, 1.5)), i get a nice different sound. i imagine this has to do with harmonic exposure or something.

oh wow listening to that for a minute and taking out my earbuds really does something

[obsidian tusk]

I dunno if the OS audio stack does it but I’ve seen some audio tooling do DC offset normalisation precisely to stop this. No idea how common it is, for sure the software I work on doesn’t do it

[ionic sedge]

yeah lmao

[ionic sedge]

it do do that

[slim pulsar]

ill try to not do that

[ionic sedge]

i wonder if we should do normalization (we can just put a high pass filter on the output)

[obsidian tusk]

This might be because -0.5 to 1.5 is clipping, that’d change the sound. Kinda a janky way of morphing between saw and pulse waves

[slim pulsar]

new wave unlocked

failed successfully

i didn't think it'd actually cap out at 1

[obsidian tusk]

I kinda feel like game audio might be a good usecase for doing DC offset normalisation by default

[obsidian tusk]

Yeah it caps at 1 but the intermediate values will usually be stored with infinite range and it only gets clipped on final output

[ionic sedge]

it's certainly cheap

[slim pulsar]

lol i could hear my earbuds decompress after holding them at 1 for a few seconds

[obsidian tusk]

It’d be easy to implement as an option in the seedling plugin too

[ionic sedge]

We have an initial graph configuration enum now, so we could just add it to the default (same as the limiter)

[obsidian tusk]

Ooh cool, nice that the limiter is on by default now, if I already knew that then I’d forgotten 😅 I was def advocating for it being part of the default config

[ionic sedge]

While in practice people ought to probably use a more minimal graph and build up what they need, I think many people will just stick with the default configuration. So we do need to be thoughtful about it. But a simple high pass is cheap and probably a good default, much like a limiter.

[obsidian tusk]

I’m not sure what the API looks like right now but I wonder if the plugin struct should not implement Default and only have constructors like with_default_graph and with_empty_graph (with the latter having docs mentioning that you might want the former), so it’s clearer to people that they might want to check what’s already added. Like, in case they're somewhat familiar with audio and think "I should add a master limiter"

Like for example, I haven’t updated my project that uses seedling and I manually add a limiter. I haven’t kept so up to date though so maybe you’ve already considered that

[ionic sedge]

i am quite pleased with the initial configuration docs at least but we could probably make it a bit more visible

[obsidian tusk]

Ooh that’s fantastic

Really great docs there

Yeah I reckon that covers what I was thinking of

[ionic sedge]

We'll have to have some default if we want to upstream, so we're kinda stuck in that respect

[obsidian tusk]

Fair enough 😅 Well the game config seems very reasonable to me

[ionic sedge]

although maybe Game isn't the best name

we can workshop that

[obsidian tusk]

I think it’s alright but yeah it’s def not game specific

[lapis stone]

do you use anything neat to generate those diagrams or is it by hand?

[ionic sedge]

im honestly kinda surprised there aren't more tools for this, but i use this https://diagon.arthursonzogni.com/#GraphDAG

[lapis stone]

oh neat this is great

[static quest]

Shush, don't listen to the voices! 😛

[ionic sedge]

yeah, sorry, you're right

[short fossil]

Can a sample player have multiple effects of the same kind?

Like multiple SteamAudioNodes?

[ionic sedge]

ya

[short fossil]

cool

[ionic sedge]

assuming they can be chained

[short fossil]

then we could have a steam audio library that assumes you're using steam audio like a normal human being, and my sample players can just have an additional copy of the effect per NPC that is listening to them. Those additional AI-only nodes can then be fed into an empty node with no outputs

would that work?

[ionic sedge]

Hm, wouldn't you only need the decoder to be duplicated?

[short fossil]

yeah that too

both really

[short fossil]

ooooh you meant chained "into" each other

yeah no

that's not it

[ionic sedge]

There's a number of things you could do though that don't require any adjustment to the crate's API I think. For example, you could add a send node to every sampler in a pool and route that to a "copy" of the pool for simulation purposes.

[static quest]

@ionic sedge One piece of feedback (pun not intended) for your reverb node is that it is common practice in DSP to put #[inline(always)] on every function that operates on a single sample/frame at a time. While the compiler is generally good at auto-inlining, in more complex DSP scenarios like this it is better to make sure. Function calling overhead really adds up when you have 48000x2 samples to process every second (in addition to preventing the compiler from performing auto-vectorization optimizations).

[ionic sedge]

There was some interesting discussion about this recently in the engine dev chat #engine-dev message

Apparently, the compiler is very eager to inline private functions and methods. I'm not denying that this is common in DSP, but I do want to get a nice profiling setup going so we can easily validate these sorts of things.

(I'd love to see number go up haha)

[short fossil]

The rust docs say to almost never do that unless you are very very sure and benchmarked

[ionic sedge]

well, tbf DSP is an area where you really want to make sure certain operations have minimal overhead, so the somewhat strict guidance there may be overly strict in this case

[short fossil]

how would that look in the API?

like, how do you set that kind of graph up

[ionic sedge]

Well you'd need control over the pool, but it's probably pretty easy

[short fossil]

Maybe there should be an example for various funny graph setups for people like me haha

[short fossil]

I mean I have no clue which methods to call to make sure I get this outcome 😄

[ionic sedge]

ah, i didn't make the send node api convenient for lazy initialization

but something like this

[static quest]

Also, this function may be a bit problematic for performance. https://github.com/CorvusPrudens/Firewheel/blob/76e01f3a852e3d65bb39330c1b8ebfb60b4e2d49/crates/firewheel-nodes/src/freeverb/delay_line.rs#L21 It contains two per-sample branches (one for bounds checking self.buffer, and another with the if statement). I'm not sure much can be done about the latter, but it might be worth using unsafe indexing for the former.

All this being said, we can always optimize later.

[ionic sedge]

Oh actually I think this is fine.

commands.spawn((
    SamplerPool(FunnyPool),
    sample_effects![
        SpecialSendNodeMarker,
        // ... normal spatial effects
    ],
));

fn send_observer(
    send: On<Insert, SpecialSendNodeMarker>, 
    target: Query<(), Without<EffectOf>>,
    mut commands: Commands,
) {
    // we want to make sure to only apply this to "real" nodes
    if target.get(send.entity).is_ok() {
        return;
    }

    let npc_sink = commands
        .spawn(AmbisonicNode::new(...))
        .chain_node(NpcDecoder::new(...))
        .head();

    commands
        .entity(send.entity)
        .insert(SendNode::new(Volume::UNITY_GAIN, npc_sink));
}

This lets you "sneak in" a connection to secondary nodes.

[ionic sedge]

I don't mind either way -- I was very uncritical with the actual DSP code that I borrowed.

[short fossil]

Let's see if I get this right: So the entity using the FunnyPool can hold a regular AmbinosicNode, but also an effect called SpecialSendNodeMarker. That lets me access the node. Then I spawn a new AmbisonicNode, and I send the input to the SpecialSendNodeMarker (i.e. the sample player) to that new node?

[ionic sedge]

Oh hold on let me update it

[static quest]

Yeah, that's fair. That being said, do you think users will typically have multiple freeverb nodes (i.e. one for each audio emitter), or do you think there will only typically be one or two global freeverb nodes? If it's the latter, then we probably don't even need to worry that much about performance.

[ionic sedge]

Mostly the latter I'd think. People may accidentally add way more reverb than they need, but that's a very easy fix. You might see people having a number of distinct reverb nodes for reverb zones, but even then you'd have a relatively limited amount.

Freeverb would definitely be okay for persistent zones.

Especially since I think most of the time they'd be completely optimized out with silence calculations.

[ionic sedge]

Hm actually it might not completely work 😅 I imagined that marker as merely a marker component, but it may result in poor interactions with other checks.

[static quest]

I was thinking of this godot plugin which uses a reverb plugin. Though now that I think about it, I don't think you need a reverb for each audio emitter, just different send mixes for each audio emitter. https://www.youtube.com/watch?v=WALpap5ZyuI&t=624s

[ionic sedge]

SpecialSendNodeMarker would give you information about an entity that we expect to be an effect. The pool uses entity cloning to build the chains from the sample_effects relatinoship. So, any time SpecialSendNodeMarker is inserted without EffectOf, that means we can spawn a new NPC sink to send the raw sampler data to.

The sinks, those npc_sink chains, are just kinda free-floating.

[short fossil]

basic question: so when I spawn a sample player with a pool, it will spawn every sample effect for that sample player, right?

so if I have 3 effects in the pool, and I spawn 4 sample players with that pool, I would have spawned in total 3*4 = 12 effects?

[ionic sedge]

The effects that are added to SamplePlayer are not added to the audio graph. Instead, their values are diffed against the real nodes which exist only in the pool.

That's why I added the EffectOf check -- you'd only want to connect stuff up to the entities that are added tothe audio graph.

[short fossil]

So they get spawned twice?

once for configuration, and once for the audio graph?

[ionic sedge]

oh but the check is backwards fixed

[obsidian tusk]

Hey, do you have an ETA on bevy 0.17 support? I see that you already support the 0.17 RC and there’s an open PR for bevy_seedling 0.6. As of yesterday it’s the last crate I use that hasn’t updated and 0.17 has some super nice features I’d like to use

[short fossil]

the RC is semver compatible

just do cargo update

[obsidian tusk]

Oh! Nice, my bad

[ionic sedge]

Once for the sampler pool's SampleEffects, once for any time a SamplePlayer is spawned for that pool, and once for every time the pool spawns a new SamplerNode.

[short fossil]

no worries, you're already the second person that I taught about this feature just today 😄

[ionic sedge]

Only the lattter actually inserts anything into the audio graph.

[short fossil]

What is a SamplerNode 👀

[ionic sedge]

That's that actual node that plays the sounds haha

[short fossil]

Followup: does chain_node spawn the node once?

Or also spawn it multiple times

[static quest]

Still, I might experiment at some point and see if rewriting it in a more auto-vectorization friendly way will help. More specifically, rewriting the delay-line/comb/allpass to be stereo instead of mono may help the compiler to autovectorize the left and right signals together.

[ionic sedge]

it's just sugar

commands.spawn(NodeA).chain_node(NodeB);

let sink = commands.spawn(NodeB).id();
commands.spawn(NodeA).connect(sink);

[slim pulsar]

a modulo for the latter?

idk if that's faster than a branch

[short fossil]

I see. So in your snippet above, it would

• spawn a SpecialSendNodeMarker in the pool itself
• spawn a copy for every sample player in the pool
• spawn a copy in the audio graph
• then in the audio graph, we spawn a free-floating AmbisonicNode and NpcDecoder
• and finally, we send the output from the audio graph entity to that sink

Did I get that right?

[ionic sedge]

we send the output from the audio graph entity to that sink
Yes, or more precisely from the SamplerNode in the pool to that sink.

[slim pulsar]

answer: goldbolt says no

[short fossil]

Then my next question is how I would mutate the AmbisonicNode to account for each NPC

[static quest]

No, the modulo operation is slow.

[ionic sedge]

If the marker trick doesn't work, you can almost certainly add a marker and a SendNode that just... points to the MainBus or something by default. And then replace the node with that observer. A bit less elegant, but that would work.

[short fossil]

yeah that makes sense

[ionic sedge]

Well, presumably you'd need one AmbisonicNode for however many simultaneous evaluations you want to do per frame.

[static quest]

Unless you know the length of the buffer is a power of two, in which case the compiler can optimize it into and-ing it with a bit mask. (Some reverbs like Vital's reverb do this).

[short fossil]

yeah but the node needs to run once for every NPC (due to the different physical location)

[ionic sedge]

Yeah, but you'd only want to evaluate exactly what you're checking for each frame.

[slim pulsar]

hm, is the length always a power of 2

hehe

oh

wait

you and i are literally on the same page, i didn't even see that comment lmao

[short fossil]

that makes sense, but again I don't see how I could do that with seedling's API in this case

sorry if I'm being dense haha

[ionic sedge]

That is, if only two or three NPCs are actually listening per frame, there should really only be two or three nodes total.

[static quest]

Vital's reverb is really good, although unfortunately it is GPLv3-licensed, so we can't use it for a game engine.

[short fossil]

Oh so

spawn 4 nodes

and then in the ECS populate those 4 nodes separately

[ionic sedge]

Yeah you could make a mini pool for each individual sampler.

[short fossil]

and remember which NPC is "linked" to which node

[slim pulsar]

damn, that was an original thought 😆 whelp

[ionic sedge]

Which, you know, should actually be quite easy because they're just entities

[short fossil]

Oh right. So it would be a regular 1:1 relationship to the node

[ionic sedge]

In other words, each frame a listening NPC can attempt to "acquire" a node for each sound. If there are no available nodes, then it don't get to listen.

[static quest]

(The reason I know is because I ported it to Rust 🙂 ) https://github.com/BillyDM/vitalium-verb

[short fossil]

and that again would be a system I implement in the ECS, right?

[static quest]

Although I did make use of the nightly portable_simd feature instead of relying on autovectorization. (Man, I can't wait for portable_simd to be stabilized.)

(It uses fancy linear algebra to do sub-sample interpolation in the delay line https://github.com/BillyDM/vitalium-verb/blob/main/vitalium_verb_dsp/src/matrix.rs). It also uses damping filters and chorusing.

[slim pulsar]

chorusing means phase offset right?

im looking rn

[static quest]

Though actually, it would be pretty simple to add a damping filter to our freeverb node.

[slim pulsar]

that's some math! haha

[static quest]

Yeah, kind of. It's essentially used to "blur" the feedback to make it sound more natural (and even to make it "shimmer" due to the fact chorusing slightly alters the pitch). https://github.com/BillyDM/vitalium-verb/blob/711228999b377cf13deee0c114772941a2ada75b/vitalium_verb_dsp/src/reverb.rs#L624

Honestly I don't understand fully how Vital's reverb works. I just know the gist of it.

[ionic sedge]

ya!

i guess technically they'd acquire it on one frame and then read the results on the next

(or at whatever pace you're doing the listening at)

[short fossil]

Okay, final questions for now. If I set up a pool like this:

(
  SamplerPool(NpcListenPool),
  sampler_effects![
    AmbisonicNode::new(),
    AmbisonicDecodeNode::new(),
    NpcInfoNode::new()
  ]
)

where NpcInfoNode has no outputs, but contains fields with the relevant Steam Audio output for ECS readback.
Does that flow from top to bottom, or do I need to use the chain_node API? And if I need to use chain_node, how do I go from "this is the entity holding AmbisonicNode" to "this is the entity holding NpcInfoNode?

[slim pulsar]

does _v here mean velocity
i can kinda grok this if that's what it means

lmao not

// Hint to the compiler to optimize loop.
let right = &mut right[0..frames];

that's wild

[ionic sedge]

yeah it just flows

[short fossil]

cool!

So I would do 4 of these pools, right?

So that each sample player can be read by up to 4 NPCs

[ionic sedge]

That would work yes. Technically it'll impose some additional performance penalties for playing the same sample four times or whatever, but that's probably still significantly less demanding than whatever steam audio is doing

[short fossil]

Maybe with a <const N: usize> so they're distinct types:

(
  SamplerPool(NpcListenPool<0>),
  sampler_effects![
    AmbisonicNode::new(),
    AmbisonicDecodeNode::new(),
    NpcInfoNode::new()
  ]
)
(
  SamplerPool(NpcListenPool<1>),
  sampler_effects![...]
)
(
  SamplerPool(NpcListenPool<2>),
  sampler_effects![...]
)
(
  SamplerPool(NpcListenPool<3>),
  sampler_effects![...]
)

[ionic sedge]

Yeah or it could just be dynamic too, like struct NpcListenPool(usize);.

[short fossil]

how would I forego that?

[ionic sedge]

The send node I suggested earlier

[short fossil]

Oooooh okay makes sense

[short fossil]

How would the spawn call look like though?

commands.spawn((
  SamplePlayer::new(...),
  NpcListenPool(0),
  NpcListenPool(1),
))

that would panic due to duplicate components

[ionic sedge]

Well it wouldn't work anyway 😅 each SamplePlayer can only be played in one pool.

[slim pulsar]

NpcLIstenPool<0>?

[static quest]

_v means "vector", as in a SIMD vector.

[ionic sedge]

You can get fancy with it though -- just create an observer for the steam audio pool that clones the SamplePlayer into however many NPC listener pools you want.

[static quest]

Also yeah, it sometimes helps to convince the compiler that all slices have the same length.

[short fossil]

See my comment above 😉

[ionic sedge]

The idea being that any time you spawn a sample player and assign it to the fancy spatialized pool, it'll automatically be copied over to the listener pools.

[short fossil]

That would play the sample multiple times too, right?

[ionic sedge]

ya

[static quest]

Oh wait, it looks like the freeverb node might already have a damping filter. (Although I can't quite tell of the damping parameter means that it damps volume or that it damps high frequencies.)

[ionic sedge]

but it's simple and the whole thing's kinda expensive anyway, so it might be honestly fine to start with!

[short fossil]

Yeah makes sense. Thanks 🙂

[slim pulsar]

sorry one more lil novice question, you're using simd because you are applying the same sample out of phase and with shimmering right? specifically 4 instances
I see uh, 6 vectors though. mmm yeah need to read a book or something

[short fossil]

I like that that way, the hypothetical steam audio library does not need to know about my weird use-case

[ionic sedge]

it'll be super cool if it works with good performance

[static quest]

I think so? Honestly I'm not sure exactly what's going on, I mostly just translated the C++ SIMD code to Rust SIMD code.

[slim pulsar]

i gotta find one of those like audio engineering courses so i know what everything's about. I the deepest ive been is modulation enveloping

[static quest]

It would certainly help to see whatever DSP diagram that the creator used rather than trying to reverse engineer it from code.

[static quest]

https://github.com/BillyDM/awesome-audio-dsp 😁

[lapis stone]

completely unrelated, sorry, but looking into this more - i think I will need to fork the fft-convolver crate for this. I think there are two separate concepts wrapped up in their FFTConvolver - processing the impulse into something that can be convolved, and actually processing the audio buffer. So, I can have the node processor "own" its convolution buffers, but have each Impulse processed beforehand so it doesn't need to mutate. Downside of this is I very much doubt that change would be upstreamed.

[static quest]

The project u course linked there is especially good (it even has a section on reverb).

[slim pulsar]

my firewheel node has been waiting for this

[static quest]

Actually, all I think you need to do is create the FFTConvolver struct and send it to the audio thread.

[lapis stone]

oh like without ArcGc, just consume it?

[ionic sedge]

OwnedGc would let you do that, no?

[lapis stone]

it certainly sounds like it would, i wasnt aware it was a thing

[static quest]

Yeah, that's what OwnedGc is for. Sorry if I forgot to mention that.

[lapis stone]

no worries, im just frightened of wrapper types and never bothered to learn much haha

thanks!

[static quest]

https://mu.krj.st/

[ionic sedge]

btw @short fossil do you feel strongly about additional glam types? we could get some more in before the next Firewheel release

for Diff and Patch, to be clear

[static quest]

You will probably need to create a wrapper for the impulse response parameter and manually implement Diff and Patch for it to achieve this.

[slim pulsar]

im on the ride
edit: "Basic Audio by Norman Crowhurst (3 volumes)" lol

[static quest]

And for diffing the impulse response, you can either wrap it in an ArcGc to compare the pointers, or just store a hidden counting variable. (Diffing the actual impulse itself would be very expensive.)

[short fossil]

In a clean steam audio library it would certainly be neat if users could just supply glam types to the nodes directly

Right now they have to go through a weird Transform wrapper

If glam types are supported, that could just be a Vec3 for the positions and a Quat for the head rotation

But uuh

At that point we would probably read the entity transform anyways

So a user wouldn’t interact directly with the node in that way

So I guess it doesn’t make much of a difference for an end user?

But yeah I would appreciate it for my own internal API

I have 300 LOC of wrappers

All of them just for implementing the firewheel traits

[ionic sedge]

ya i mean since we already have a few.... we might as well grab a couple more that are useful

[short fossil]

Thx!

Vec3, Dir3, Quat for me plz

Oh wait Dir3 is bevy_math IIRC

[static quest]

Oh yeah, someone else said that the structure of Vital's reverb algorithm appears to be similar to the structure of the Dattaro reverb algorithm.

[ionic sedge]

(Vec3 should already be in there actually )

[static quest]

https://ccrma.stanford.edu/~dattorro/EffectDesignPart1.pdf

[short fossil]

Oh? Maybe I misremembered

[ionic sedge]

but we can add quat for sure

[short fossil]

Very unfortunate that you can’t add bevy types

Curse you once again, orphan rule

[static quest]

So my guess is that Vital's reverb takes this design, replaces the lowpass damping filters with high/low shelf damping filters, adds chorusing to smooth out the wet signal and make it "shimmer", and adds fancy sub-sample interpolation for the delay lines for better results.

[slim pulsar]

...so that's why 44khz is standard

that makes much sense

it's theoretically lossless ? must be placebo when i up the sample rate

[static quest]

That's quite a bit of work for a synthesizer plugin (most synths just use an off-the-shelf reverb model). My bet is he was at one point developing his own standalone reverb plugin, but then later decided to pivot into synthesizers.

(Sorry for rambling, I'm mostly just rubberducking with myself.)

[static quest]

Not just theoretically lossless. The sampling theorem proves that it is lossless.

[ionic sedge]

-# ~~damn guess we just gotta upstream ~~

[slim pulsar]

-# just one more wrapper bro i swear

[static quest]

Man, it would be so nice if Rust added a feature that lets you bypass the orphan rule (even if you had to use unsafe to achieve it).

[slim pulsar]

necessary evil

[ionic sedge]

ya i see a lot of people suggesting it at least for binary crates

like where you can do it at the end of the chain, since there's no possibility of overlapping impls

[static quest]

Yeah, I understand that would be a problem for library crates.

[static quest]

(Unless you can somehow hear above 22kHz)

[slim pulsar]

still remember when teachers would prank the class with a 20khz whistle

cuz they couldn't hear it

[static quest]

Though using a higher sampling rate does have some advantages. mainly when doing audio processing. (Namely it is easier to create a stopband filter the higher the sampling rate, and a higher sampling rate is less prone to aliasing artifacts.)

[slim pulsar]

so this really raises a question for me

when spotify offers different levels of quality

nvm it's kbps not 192khz

lossy audio should be dead at this point

it is a bit ridiculous

[static quest]

Though it would certainly make life easier if we chose either 44.1khz or 48khz to be the universal standard instead of having both standards.

[slim pulsar]

which one do you prefer

[static quest]

I mean, Vorbis is really good.

[slim pulsar]

my project for this project is sfz impl

[ionic sedge]

48 is a prettier number

[static quest]

Well, 44.1 takes up slightly less memory.

[slim pulsar]

i shudder when my wav file is 3.8 mb and not 4

[static quest]

Vorbis is practically transparent (unless you have a very hi-fi system and golden ears).

[slim pulsar]

like i get it, but why? like how big is a song actually

like I've downloaded a few flacs and it's like, not that huge

[static quest]

It helps to think of audio compression as a noise floor. If the volume of the noise introduced by lossy compression artifacts is below audible levels, then it is considered "transparent".

[static quest]

What about a few hundred or a few thousand flacs?

[slim pulsar]

do you take pictures with ur phone

[ionic sedge]

i mean 24kB per second is way better on the network than uh, you know like 176kB

[slim pulsar]

ok

terrible terrible example

[static quest]

Yes, and pictures are also stored in lossy format to save space.

[ionic sedge]

(16 bit stereo wav at 44.1k)

[slim pulsar]

point retracted

[ionic sedge]

no
steelman it
no takebacks

[slim pulsar]

wish rust performed lossy compression on build artifacts

[slim pulsar]

I guess my point, prior to the self own, was that, for streaming services, lossy audio is quite annoying

[static quest]

Oh sorry, I meant opus, not vorbis.

[slim pulsar]

because I'm not storing thousands of flacs on device. I'm paying them to do it

[ionic sedge]

i imagine that wouldn't go down so well in the board room 😅
"so you're telling me we'll 5x our bandwidth costs for the sake of a handful of unhappy users??"

[slim pulsar]

where is the costco of audio when you need it

it doesn't seem to be tenor

[ionic sedge]

clearly they should get into the streaming business

[slim pulsar]

fuck me

point retracted again

really batting 1000 here

bliss

[ionic sedge]

can't believe i don't have a loss emote in here somewhere

[lapis stone]

lmao

[short fossil]

[lapis stone]

tempted to get nitro just so i can use that

btw is there any reason why ChannelBuffer has channels/channels_mut but no channel or channel_mut? Easy enough to index after calling all but a single get would be nice (i can pr if so)

[short fossil]

Is there a way to ask a sample player from the ECS which outputs it will send to its effects this frame?

So I can do things like "this sample player is very loud this frame, give NPCs nearby listening priority"

Or "this sample player and this NPC are so far away from each other that with this volume, the sound wouldn’t reach it anyways. Don’t let it listen to this sound”

I know I could write a node's inputs into public buffer and read them the next frame, but I was wondering if there’s something builtin for this

[obsidian tusk]

I really strongly disagree with this. If you encode it right, mp3 is literally indistinguishable from flac. The issue is when you’re processing it further (especially pitch shifting) but if you’re just doing playback/attenuation/summing then mp3 at 320kb/s CBR is more than good enough. Lossy encoding is terrible for images though, I’m glad we’re mostly out of the era of that being standard

[slim pulsar]

ok honestly i wasn't saying that with much heart. I'm wrong

this I know

[obsidian tusk]

Like, def use flac if you don’t care about memory use or file size, it’s the best option when you can use it (which is most of the time) but I don’t think mp3 is dead

Ahaha 🙃

[obsidian tusk]

This rules I want to make a custom icon theme that does this

Oh sorry I didn’t get to all the messages after you said that, I’m suuuuper late to the defending-lossy-audio party 🙈

[static quest]

Yeah, if the mp3 is encoded at a high enough bitrate (like 320kbps). Opus can achieve the same result at like half of that bitrate.

[obsidian tusk]

Damn I need to check out opus, I haven’t really needed to use lossy audio for anything where I actually get to use the format I want (mostly DJing where you’re limited to mp3 or wav)

[ionic sedge]

There isn't anything built in because there's at least some cost involved in calculating this. If you just want to monitor loudness, you could route all SamplerNodes into an associated LoudnessNode and just check that. It's trivial to find which SamplerNode is serviving a SamplePlayer -- just follow the Sampler relationship that's placed on the SamplePlayer. Then, from there, you would presumably relate the loudness node you have attached to each sampler.

[short fossil]

Thx!

[ionic sedge]

Alternatively, you could get the actual raw data by 1. looking at the sampler's playhead, 2. fetching the DSP time range for the last frame using Time<Audio>, and 3. reading the data out of the Sample resource. Then you could calculate its average loudness over the previous frame.

Or its expected loudness over the next frame. But this approach is less robust since it would basically have to re-implement some of the sample playback logic.

[short fossil]

The SendNode docs talk about "sends"

But the link there goes to the regular Rust Send trait

Is that intentional?

[ionic sedge]

no

[short fossil]

Is a "send” some concept I don’t know or is it something like the old name for EdgeTarget?

[ionic sedge]

no

[short fossil]

Same here

[ionic sedge]

Oh yeah that one's correct. A "send" is a musicy thing https://en.wikipedia.org/wiki/Aux-send

[short fossil]

Oooooooh

[ionic sedge]

i suppose that's not helpful if you haven't worked a mixer (or a DAW) or aren't fresh on it, although it probably doesn't need much adjustment to be a little clearer

[short fossil]

Can I imagine a SendNode as a kind of SplitNode? e.g. if I have two sample effects, namely [SendNode(B), A], does the sample get processed by both A and B?

[ionic sedge]

ya it's like a wire tap

[short fossil]

You know, like a splitter in Factorio

[ionic sedge]

ya probably (i don't remember anything from my few hours of gameplay haha)

it's like a
uhhh
it's like a power strip

[ionic sedge]

ya

[short fossil]

Cool!

[ionic sedge]

but instead of distributing the sound it magically duplicates it

two sound for the price of one

[short fossil]

Then this part of the docs is completely incomprehensible to me as an audio beginner FYI

because it does something very simple, but I don’t understand the words used haha

A little ASCII graph could help

(I love your ascii graphs)

[ionic sedge]

you also don't need a node to do this -- it can just be more convenient sometimes

because, you know, you can just route the output of a node to as many places as you want

[short fossil]

Call connect multiple times?

[ionic sedge]

ya

It's most convenient within a sampler pool when you want to send everything to a specific target with individually adjustable volume.