Rendering crate refactoring

[north trail]

is it just the dyn dispatch?

[lilac basalt]

maybe things like materials with color animated on the cpu side, even though ideally that's just done in the shader people are going to take the easy route

[north trail]

cus that is like not at all something to worry about if so

[static cypress]

We were talking about changing the way it is done in ways that would make it slower

Yup

[brazen elm]

we were trying to figure out the worst case performance

[north trail]

i see

[brazen elm]

but i think just gotta try and see

[static cypress]

It started about here

I think we should check for changes of specialisation keys

Rather than changes of materials

So in designing we want to make something that makes checking for changes to specialisation keys be low cost

Hot specialisation would look up/rebuild specialisation keys per frame and per entity and that wasn’t so fast either

But that was doing unnecessary work

Cold specialisation avoids a lot of unnecessary work

And I guess we’re still doing some unnecessary work because we’re checking if the material changed and then re-specialising (or gathering keys and checking if we need to re-specialise)

[lilac basalt]

if the main cost from cold-spec is hashing the keys, I think it'll make no difference? (still worth testing tho)

since you'd still check hash/equality when you decide to update the key or not, which you do with every change to the material/mesh

[north trail]

yeah this sounds like it would be peanuts compared to everything else going on at least for vj applications

[static cypress]

It’s also the iteration and such. If mass-animating material data for many material instances, that incurs the specialisation check cost

When it shouldn’t

But we can probably draw the line there

Assets as entities, put material specialisation key in a separate component. Check if component changed

[north trail]

ohhh wait i didnt think it would respecialize if the shaders were the same

you can safely assume all uniforms are always animated in vj

[static cypress]

vj is also likely only one material right?

[north trail]

no, it can be more

fullscreen shader vj is just one setup

[static cypress]

Ok. Is it ever many?

[north trail]

yes

[static cypress]

How many?

[north trail]

probably top out at 100

10-50 id guess is a complicated scene

[lilac basalt]

the issue there is deciding whether to update the key or not I think

which would be calculating the new key whenever the material changes, and assigning if the new key is different, so you still have to do the checks

[static cypress]

I’m suggesting we design something where the key component is what is explicitly modified

Rather than checking a bunch of things and creating the key

Maybe not feasible

Could be a set of components

Like AlphaMode etc

If that affects the pipeline blend state

And so on

Main point is perhaps that data impacting the key is separate from let’s say uniform data

So that we can change detect the data impacting the key and it only cause respecialisation when needed

Rather than changing uniform data and it causes respecialisation

[lilac basalt]

maybe materials should be split between pure bind group data and cpu-side metadata at the end-user struct level? + required component or smth.

also interesting that there's two facets of cpu-only data here, one for queueing (not material specific) and one for specialization (material specific)

[static cypress]

Mmm

Queue data, uniform data, and specialisation data

Though queue data may be a subset of specialisation data, and I think that is ok

Queueing only needs to read it

I guess it’s probably other stuff too

[lilac basalt]

could be independent I think, either way doesn't matter too much

[static cypress]

Mmm

But good point that’s it’s a separate usage pattern of the data relating to a single material

[brazen elm]

putting stuff in the ecs would really help making these concepts generic so much, 80% of the queue/specialize systems atm is just checking whether thing are available in resources and bailing if not, which could literally... just be a query that gets matched or not depending if prior systems inserted data on the visible entity

it's also part of what makes working with custom render code confusing and complicated because there's all these magic resources you have to know about

[static cypress]

Do we risk these becoming magic components if they are required components that you can override?

[brazen elm]

yeah, absolutely not a panacea, but at least would benefit from shared tooling like discoverability in the world inspector and patterns like required components, relations, etc. obv there are the big performance questions we've discussed many times whether its even possible

[static cypress]

Required components in the render world could solve the table moves. Just that we need a cheap way of identifying whether the component data is initialised and a cheap way of resetting the initialisation

I think

[turbid forge]

Totally. Technical Artist is a core role on game dev teams. The Technical Artist role is basically someone who is both an artist and a programmer who helps bridge the gap. They work on asset pipelines, art-related tooling, shaders and shader tooling, etc...

[north trail]

hello im a tech artist!

[turbid forge]

hell yeah

[brazen elm]

need cute bevy stickers asap

[north trail]

yes

[turbid forge]

Was just mentioning this in case someone wasn't aware of the role, realizing now that the way I wrote it makes it looks like I'm explaining to @brazen elm in particular what the role is which was not my intention 😅

[north trail]

smh, mansplaining

[static cypress]

Nice!

[north trail]

i agree though, bad tooling is the main thing that keeps ppl out of writing shaders

i think most tech artists can put up with bad tooling if they have to, at least until they get time to make better tooling

artists should have the shaders in their hands though

[brazen elm]

omg no not at all, you are a great supporter of the arts 👩‍🎨✨ i think its extra important when in spaces that are mostly engineering focused to remind folks not to be too prescriptive about what artists are/are not capable of. our goal is to remove barriers not make assumptions

one of the things i've loved about teaching people to code with art/games, particularly kids, is that when they are driven by this creative need they'll learn literally anything in order to express themselves and be unbothered by tools that realistically should bother them. it's so amazing to watch someone interact with some tool and just blow past the weird sharp edges because they are driven

but also we should remove those sharp edges :p

[static cypress]

I got into this through Processing and The Coding Train. Even though I’d been programming for years, game dev and rendering felt like a black box of scariness.

I think that sensory approach to learning programming is really powerful

[north trail]

i usually get my friends into programming via shadertoy

its really good for learning syntax and semantics without worrying about memory and datastructures, and has a smooth onramp to control flow

i feel like a lot of CS courses unnecessarily stress datastructures way too early on

[lilac basalt]

way back when I was first learning programming in high school, they taught it through a weird java game engine called greenfoot

it was incredibly cursed but actually let you some pretty cool stuff

[lilac basalt]

I have a cursed but maybe really cool idea for modelling camera composition

[north trail]

tel

[lilac basalt]

Similar compositor idea from before, except I think I'm convinced that views/cameras should only render to a single compositor, and that all views are managed by a compositor:

Compositor is what actually contains a RenderTarget, and only hands back a handle to a sub-section of that target to its child Views. Then, in the new analogue of CameraDriverNode, each compositor manages the order to execute its children, assuming Compositors are 1:1 with render targets so there aren't any conflicts

#[require(Composites, RenderTarget)]
struct Compositor { layout }

// Composites, Compositor each require each other

#[require(CompositedBy(self))]
struct View {
  //not exposed in public api, managed by `Compositor`
  render_target: Option<RenderTarget>,
  viewport: Option<Viewport>,
}

#[require(View)]
struct Camera {}

#[require(View)]
struct UiCanvas {}

which might be a bit convoluted but I think would still let you spawn cameras without explicitly mentioning the compositor: commands.spawn((Camera3d, RenderTarget {...}))

[north trail]

so Camera is simply one of the processes which can draw to a Compositor's target?

oh wait Camera only requires View

ohh right, cus viewports

[lilac basalt]

yep, its main function would be to add projection, frustum, etc to a View

[north trail]

hm. but wont we want multiple viewports into the same render target for different cameras?

like for VR for example or split screen

[lilac basalt]

I think then you'd do something like Compositor::split_screen_horizontal() and spawn the left and right cameras as children under CompositedBy

[north trail]

will that render to two different render targets and then copy them into one though

[lilac basalt]

no, the idea is it's camera stacking like we have now but managed by a (possibly) separate entity

[north trail]

i see

okay i dig that

[lilac basalt]

sick

it seemed like it would work in my head, but I'm often wrong about that lmao 🙂

[north trail]

how would clears be managed

and does RenderTarget hold like, buffer config like texture formats and depth buf etc

[lilac basalt]

good question, do we currently share ViewTarget between cameras/do we need to?

[north trail]

also is RenderTarget owned by View or is it a reference of some sort

[lilac basalt]

I probably wasn't going to change RenderTarget too much

owned by View in the rust sense but not in the "which object actually manages this" sense

[north trail]

so two RenderTargets can point to the same underlying buffer

cool

[lilac basalt]

yep, though the idea is that no two compositors should write to the same render target

[static cypress]

@lilac basalt have you seen that video about adding juice to your games through things like camera shake? Specifically the part about a local coop split screen method that is based on where the characters are relative to each other?

https://youtu.be/tu-Qe66AvtY?t=1638&si=EQ3xyFBpejChAIb1

Just working how you imagine that being done

Also, I know using msaa or not for different cameras/views that are compositing into the same target is challenging to get right

[quasi vapor]

kek, it probably will be neither of the options that i had on the poll

[lilac basalt]

Good point, msaa writeback is weird. I’ll be thinking about the split screen example but I think (?) that should be fine with this design

[static cypress]

Specifically I meant where there are multiple cameras rendering to a single texture where some use msaa and others do not. I think we have an example that does that. I don’t remember if it was robtfm or icesentry that got that working properly (I think it was rob) but I know it wasn’t the easiest thing 🙂

[lilac basalt]

mmmmm

[brazen elm]

the way we do it right now is a blocker for using bevy on xr devices iirc

[lilac basalt]

oh how so?

tiled renderer things?

[brazen elm]

it’s slow and we run it even when it’s not necessary

the first fix is just to do some analysis when we extract cameras to understand what cameras actually need to write to it

been on my todo list forever

[static cypress]

Hmmm, I don’t reject how it was solved. What makes it slow? Multiple resolve steps?

[lilac basalt]

Oh my idea might make this easier then 🤔

since views are already grouped by their render target

[brazen elm]

here's the issue https://github.com/bevyengine/bevy/issues/14672, i think it's just potato gpu plus we do the writeback even when we don't need to

[static cypress]

I just don’t remember exactly what msaa writeback is nor why it’s needed. But I’ll look at the code sometime unless someone else fixes it

[brazen elm]

ooo it just makes sure that we write the internal texture from a previous camera back into the msaa buffer before the second cam starts rendering so we don't step on it, otherwise we'd overwrite the previous cameras work when we upscale into the final output texture

basically if you render two fullscreen views both using the same internal texture we waste teh work of the first upscaling pass if that makes sense

at least that's how it works in my brain the multi-cameras stuff is confusing 😭

[lilac basalt]

oh do we share ViewTargets for cameras with the same render target?

[brazen elm]

its not based on the render target but on (hdr, texture usages, msaa)

cameras sharing those render into the same internal textue (view target)

then we upscale into whatever render target you tell us, that's just the color attachment for the final blit

[quasi vapor]

pub struct PbrMaterial;

impl Material for PbrMaterial {
    type Components = (Color, Specular, Clearcoat, Anisotropy, ...);

    ...
}

#[derive(Component)]
pub struct Color {
    tint: bevy_color::Color,
    color: Handle<Image>,
    ...
}

#[derive(Component)]
pub struct Clearcoat {
    clearcoat: f32,
    clearcoat_channel: UvChannel,
    clearcoat_texture: Option<Handle<Image>>,
    clearcoat_perceptual_roughness: f32,
    clearcoat_roughness_channel: UvChannel,
    clearcoat_roughness_texture: Option<Handle<Image>>,
    clearcoat_normal_channel: UvChannel,
    clearcoat_normal_texture: Option<Handle<Image>>,
}

pub struct PbrMaterialKey {
    ...
}

impl MaterialPipelineKey<PbrMaterial> for PbrMaterialKey {
    fn key(components: PbrMaterial::Components) -> Self {
      let (color, specular, clearcoat, anisotropy, ...) = components;
      ...
    }
}

[brazen elm]

pipeline keys are used for pipeline descriptor specialization, not for generating concrete bindings

[north trail]

@lilac basalt charlotte and i have been looking at zuiyu's frame graph PR and it reminds me a lot of your prototypes from a while ago, id be interested in hearing what you have to say about it https://github.com/bevyengine/bevy/pull/19224

[brazen elm]

here's the talk btw for anyone who wants an overview of the architecture https://www.gdcvault.com/play/1024045/FrameGraph-Extensible-Rendering-Architecture-in

me and vv are trying to figure out what parts are implemented in the poc atm

[lilac basalt]

It reminded me of that as well, I definitely have some thoughts 😅. Will write them out, but I tldr I think my original prototype failed for some good reasons, and this seems to have those same attributes but more.

[north trail]

i forget, your prototype, did you get it to a semi-working state such that 3d_scene built and ran on it using it?

[lilac basalt]

I don't believe so, but there's no reason it couldn't have iirc

I tested it fairly well and ran some simple stuff, but got burnt out on it before I could migrate half the engine lol

[lilac basalt]

Mainly, an ideal frame graph makes passes easier to configure/more dynamic, allows automatic resource lifetime management, memoization, and caching, and would help perf through things like pass merging and automatic barriers. This works, really well, for something like use.gpu, a framework that has full vertical integration with their resource management (iirc), and where everything lives inside the graph. It also works for things like unity that built their own HAL and actually have access to resource lifetimes and barriers.

Bevy can't claim either of those though, so what I focused on for my prototype was ease of use, encapsulation, and type safe (but anonymous) resource passing. I wanted to solve the issue of plugin compat, and let us design actually reusable systems. Part of the reason this failed was that bevy isn't really amenable to stuff like this in-engine (and that's a good thing!). I failed to solve the memoization problem in a way I was happy with, and it didn't really seem to fit in with the rest of the ECS. Cart thought so as well, and pretty much vetoed the design.

My initial reaction to zuiyu's pr was "oh, this is like my design but with a heck of a lot more types for me to learn". But, it actually seems pretty close to what we have now? Resources are still passed through components (I really tried to avoid this for mine), graph structure is still fairly static, some nice utilities for pass setup but ehh, it seems like the main benefit would be lifetime and barrier management... if we didn't have wgpu for that. I need to look at it closer but I'm having a hard time seeing what it would really unlock? + All the new types, terms, and systems at a time when I think we want to cut down on a lot of the tech debt in the renderer

I'm also really hopeful about using more "bevy-overall" improvements like generalized schedules and reactivity in the renderer, on top of more targeted, structural changes.

Ex: Assets as entities, returning things to the ecs, restructuring Camera, unified materials, and then the crate reorg of course

My general vibe is that bevy-ish things turn out in our favor more often than not, and we should maybe be wary of big magical systems even when they're proven in other places

(zuiyu I hope I don't sound mean, I just think it's worth being skeptical about this big and important a change)

[brazen elm]

the transient resource cache stuff is important to look at here, although it's unclear how much it's implemented / used in this pr. there's also a logical command encoding step that happens first prior to actual command encoding which is interesting but am not sure how much that helps in terms of building the graph topology. there's also the idea of pass culling that is currently unimplemented in the pr but is potentially interesting in terms of helping to build decoupled passes

the talk also references these, which don't appear to be implemented atm but would be major features

that's covering the advantages afaikt atm

[brazen elm]

i agree that reactivity is immediately what comes to mind in terms of an alternative

[lilac basalt]

oh yeah that's another thing I tried to do with my prototype, usage inference

definitely important but I think there's other ways to do it besides a frame graph

[brazen elm]

i said to @north trail that this feels very familiar in some ways to other decisions we face in terms of the balance between examining prior art (in this case AAA) and using bevy as an ECS research platform to discover better next-gen patterns. i personally tend to lean much more towards the latter in these discussions, but want to acknowledge the huge amount of work already present here which is why i want to do due dillegence in terms of reviewing the effort

esp because some of our hopes for what an ecs forward render graph might look like art still pretty vague

but i agree the tech debt risk is very high, i want to be clear about what the concrete benefits atm are

there's a lot of abstraction here

i think at the very least this is immensely helpful in terms of bringing us to discuss what our goals are

"as a user, i want to be able to allocate a texture that automatically gets resized when the window changes size"
"as a user, i want to be able to allocate a buffer that automatically gets resized when a dependent buffer changes size"

i could go on

[brazen elm]

i'm falling asleep, but i'm going to leave a longer comment on the pr tomorrow with some questions i have. i do want to recognize the enormous amount of effort on the part of this contributor to get it this far. it's kind of a lot to try to digest and i remain not totally certain about the benefits and trade-offs here. but at the same time nothing changes unless people step up to do the work.

i'd appreciate some more feedback from others too

[lilac basalt]

I'll do a more in-depth review this weekend too

[static cypress]

Personally I would appreciate alignment on things before doing a lot of work. Otherwise it risks going to waste

I think we should look at screen-13 as well https://github.com/attackgoat/screen-13 which I don’t think is the fake thing but that @turbid forge had found to be very productive and concise for building renderer features

I do think that we want things to enable defining and managing render resources, bindings, passes, and a graph of them to enable much less boiler plate to build render features and have the boring stuff just taken care of

Things like automatically recalculating resolutions of things, and taking into account whether those textures are multisampled, and whether they’re rendering into a viewport subsection or using a stencil or whatever… without the majority of the binding and pass code having to be written, but being cached into something changes, would probably save a lot of pain for all of us

And checking against api/adapter/device features/limits

Often things don’t work with msaa or break in subtle ways when configured in some way or other

And I seem to remember that msaa is pretty cheap on mobile when using tile deferred? Or maybe just generally

I’ll watch the presentation anyway

[brazen elm]

i asked griffin if he'd weigh in here for this reason, i haven't played with it nearly as much but it felt really refreshing

[north trail]

whats the fake thing?

[brazen elm]

same* i imagine

[north trail]

i think https://discordapp.com/channels/691052431525675048/1262090782211309629 should be reopened, this working group has become a catchall for random rendering initiatives. there should be some semblance of scope kept, the title implies its mostly about reorganizing existing code, not replacing it

(i am aware i contributed to this issue, just asking for that thread to be reopened if possible)

like i dont think dynamic materials or mesh api overhaul or reactivity driven rendering or frame graph stuff or etc should go here

this thread should be for splitting stuff out of bevy render and bevy pbr, and related structural organization changes that dont really affect functionality much

[brazen elm]

i'm going to start sounding like the most annoying person over the next year or so, but i've become pretty convinced that trait driven abstractions are just a bad fit for bevy. i mean, obviously there are uses. but my refrain here is going to be, data driven, dynamic, configurable at runtime, ui introspectable, etc. all of which are natural strengths of our ecs. i think my hesitation here is just what i said before. we don't want to be so naive as to ignore prior art. or that ecs is always the panacea. but i'm also powerfully motivated by treating these as legitimate open research questions.

[static sierra]

agreed. I realize there is a big gordean knot to cut, but I think it’s really important to keep the scope here. I would generally propose we do this refactor first, then try to re-open the render graph wg after this one closes.

And I feel so strongly that this is the way to do things I’m posting on my vacation

[lilac basalt]

One thing is I think we've kinda ended up with a checklist to complete before we can actually do the refactor as we want it, so ending up talking about other possibilites for restructuring the renderer makes sense. But I agree the graph rewrite in particular is a whole can of worms we can delay for awhile lol

(in fact imo it's way less immediately valuable than a bunch of other stuff we have planned, and if we rush into it it might get in the way of those)

[static sierra]

Sweet! Is that checklist anywhere I can easily review it?

[north trail]

• bsn

lmao

[lilac basalt]

here's my version, I think rob posted a similar one somewhere

[north trail]

finishing hotreloadification too

[lilac basalt]

oh true

guess it's not a blocker, just that it'd get real confusing to do that in the midst of the reorg

[north trail]

i think we also need to consider structured plugins

[lilac basalt]

structured?

[north trail]

as in, plugins that are not just a function that does arbitrary stuff

[static cypress]

Mine was more general about 0.17 if it’s the one I think

[north trail]

like, plugins that specify dependencies and requirements and orderings

for example, minimum wgpu featureset required to work

[lilac basalt]

async plugins 🥴

[north trail]

im not sure we need async plugins

just an actually opinionated plugin trait api

[lilac basalt]

mmm ok

[north trail]

like, id like to "add" a bunch of plugins and know they're going to work regardless of the order i add them in, because they know about each other roughly and can order themselves and specify interdependencies, and i want to know that removing one plugin wont make everything explode

[static sierra]

We have async plugins they are just janky

[lilac basalt]

hank 😔

[north trail]

contacting Hank

[lilac basalt]

regardless of changes to Plugin itself I want to kill PbrPlugin and friends

[north trail]

im going to give birth to Hank

the reason i said bsn earlier is because i had an idea for how the render graph should work

i mean bsn render graph is something char has mentioned a few times but it occurred to me that its a convenient format to centralize in while keeping good ux, in the sense that the user can just add a bunch of plugins, and if they are structured, there can be an api for them to mutate a bsn render graph to add their passes in the right places, and that graph can be then used. this is built dynamically every time, satisfying the "no central config tweaking" requirement, but for power users, we can have an option to dump out the bsn, and have the app load the render graph instead of generating it

this permits editing the graph by the editor in the future too

[static sierra]

It would be good to reify this in the design doc

[lilac basalt]

yeah probably

I am in a cabin in the mountains rn, we'll see if I remember to do that in the morning :)

[north trail]

tell the mountains i say hi

[static sierra]

do cabin things instead. go chop some wood or something

[wintry pier]

Flagging up this PR because it's related to render refactoring and also plugin configuration: https://github.com/bevyengine/bevy/pull/19190. It's basically "how does bevy_render tell bevy_gltf/bevy_image what texture formats it wants without being a crate dependency", and the proposed solution is agreeing to register a resource. Needs one more review, but there could also be design questions around the fragility of plugin initialisation order.

[brazen elm]

approved but it's a little sketchy for sure. the warnings are helpful to ensure it doesn't break in the future but really don't love relying on this kind of stuff. i think the better ecs solution is above our paygrade here though so i'm fine with this if it unblocks y'all

[grim inlet]

Hey guys, I am a distant observer of the bevy project and I recently glanced at the whole renderer part, was a interesting read, nice ideas about using a ECS in a rendering context, but I wondered what is for now or in the future the global vision of the Bevy renderer ? A classical CPU-driven renderer, or something more GPU-driven or even more modern using a meshlet-based pipeline perhaps (I say that since I said a prototype hierarchical meshlet implementation in the code)

Could be fun to one day contribute ! 🙂

[static sierra]

as I understand it, our renderer is pretty much gpu driven to the extent wgpu allows

and several members of the community have been working on a virtual geometry "meshlets" implementation for year(s)

[lilac basalt]

The way things are moving it seems like Visibility and friends might fit well in bevy_camera, since I want Camera to cover everything regarding projections, frusta, etc. I think that would be fine? We'd have to move the mesh render code into its own crate to depend on visibility stuff then, but I think we wanted that anyway

[river dew]

Yes I'd like that

[lilac basalt]

cool, I'm hoping to have my camera pr up soon™️ after exam season is over. It should be pretty easy to just move the camera module into its own crate after that

I wrote up a hackmd doc for it as well if you want to take a look, there's a few substantial api changes

[static cypress]

All of the above for as long as they are relevant to enable a good solution in each platform / for different use cases

[lilac basalt]

hmm what about render layers? should all views have access to that or should that be a more camera-specific thing

ig it's its own component, we should prob just add an ExtractComponentPlugin and take it out of extract_views. The view module itself doesn't need to care about that but it can prob stay in bevy_render

[brazen elm]

well minimally lights use it as well rn, and potentially other things in the future?

we had a discussion about maybe letting decals use it?

"view like things"

[lilac basalt]

mhm, though was wondering more like: does anything that's not a camera (ui, things of that sort) need to render anything that cares about render layers?

ig the answer is probably "idk, but we shouldn't prevent it"

[river dew]

UI should also be a camera >.>

[lilac basalt]

In my pr I've added a new primitive called View to handle all the low level camera stuff (composition, viewports, ordering, binding a render graph), and I think ui should be a View rather than a full camera (see the hackmd). Def agree they should use the same primitives tho

since right now we have that weird thing where UI layers itself on top of a camera's render target but isn't really a full camera

also UI shouldn't get a ton of machinery for projection and visible entities added on where it doesn't need it :p

[quasi oxide]

Hi, [I wanted to know](#rendering-dev message) if splitting bevy_render is still a thing. Does anyone has some knowledge in that area or can give me a pointer?

[brazen elm]

this is definitely part of the motivation! we'd love to be able to make some of the core types easier to consume without wgpu

it might take a few steps though. we are tightly coupled to wgpu in a lot of ways so it's not an easy task

[quick spade]

Not sure how things stand right now, but it would be nice if the render graph could be reused for a non-wgpu backend (swap it out for something entirely custom)

[brazen elm]

mmmm, this might be too ecs utopian, but i think ideally we get a render graph that feels like just a very thin layer over the ecs itself rather than a heavy weight abstraction with swappable backends

[static sierra]

agreed. iirc this sort of re-use was part of the design goals for the current render graph, and then no one used it and that property kinda got buried.

the main alternative is vulkan, and that kind of demands it's own scheduling architecture usually

so the people who have wrote their own renderers have not ended up using our render graph at all

[quick spade]

Interesting, I was under the impression that render graphs were backend-agnostic, but I'm more familiar with vulkan than wgpu

[turbid forge]

afaik it's also common across a few of the custom renderers to not use the render world (but still maybe use a different thread for submit), so the whole thing looks a bit different

[turbid forge]

Another potential backend I could see at some point would be the SDL3 GPU stuff since it natively has console support, so if someone got rust to compile on a console I would guess that would be the shortest path to getting bevy on a console.

Regarding splitting up bevy render stuff in a way that works well for other backends, the idea I like is bevy further separating the idea of the 2d/3d "scene" from rendering (somewhat like blender does with cycles/eevee). This is largely already the case and there's just little bits of the renderer sticking into the scene representation here and there.

[cinder stream]

Isn't that already doable today? Iirc the graph doesn't really know about wgpu? With that said, I wouldn't really recommend doing that because if you are already writing a completely new backend you might not want to use the render graph as is, but I think it's already possible.

[ebon lodge]

Just swapping the window system is fairly easy.

[turbid forge]

The SDL3 GPU stuff I'm referring to isn't the window system API, it's more like wgpu and also has backends for consoles.

https://wiki.libsdl.org/SDL3/CategoryGPU

[ebon lodge]

Oh the new one you mean

[quick spade]

Some exampels:

• RenderGraphRunner is hardcoded to wgpu types. Idk, maybe it was meant to be a WgpuRenderGraphRunner.
• SlotValue refers to stuff that refers to wgpu, e.g. wgpu::Buffer

So idk, maybe reusing render graph is simply a non-goal, but if it somehow is, would be cool to take that into account during refactoring

[cinder stream]

Right, for things like SlotValue we don't actually use it anywhere internally so I forgot about it.

[static cypress]

No web support though?

[turbid forge]

I think WebGPU support is in progress, no webgl2 support planned,

[brazen elm]

@lilac basalt i think i've been fully compositing-pilled

will you link me to your proposal again?

[lilac basalt]

https://hackmd.io/Bynv5tv6RqiB6xoha_S-Vw?view

most of it currently is about decoupling View from Camera, wrt the crate reorg

I think I need to go back and think about/diagram the specifics of the compositing model, after the convo in #1379850581253947413 😅

[brazen elm]

had a long conversation with @thick pine and i think we’re all roughly on the same page that compositing being hidden from users is a bad leaky abstraction. we pretend that the user writes to RenderTarget but in reality they write to the internal texture and we handle compositing for them.

i agree that we fundamentally need to make that internal texture explicit and configurable. i also agree that we need some pattern for compositing, ideally that can be configured automatically for users who don’t have complex needs here.

to expand a little, i’ve often wanted to have the render graph be more flexible in order to build the kinds of complex graphs i need for the work i do, which involves loads of compositing. but i’ve kind of realized, the existing abstractions serve me fine. what i want is a “logical” render graph, i.e. something like a camera graph.

i think if we add just one more concept — that cameras can take input textures as well as output textures - we are basically already there. in this model, a compositor is just a camera with a single node sub graph that accepts multiple input textures and does a blit (or something fancier) to the output texture.

i think there’s a lot of naming bike shedding to be had here. but that’s where my head is at right now.

i’ve spent less time thinking about your viewport changes

i think in my ideal world, the camera itself (ie view matrix), the render sub graph (what i’m thinking of as a logical render pass), and the targets would all be different entities. but recognize atm the first two are fused which i think is fine. but yeah, having compositing be “just another camera” means that we would have way more flexibility on when post process effects run and how they are composited with things like ui.

[lilac basalt]

great, seems like we're on pretty much the same page :)

I'm not super attached to the exact thing we end up with, I mostly care about decoupling cameras from compositing/layout/intermediate textures

what I've gone with is more tree-style/top-down, it seems like you've been thinking more like making a DAG of camera inputs/outputs? would love to hear your ideas there

[brazen elm]

i had basically originally envisioned this pattern as being a giant command encoder graph, but realize it’s actually way easier to imagine as a more coarse grained logical graph. we don’t solve for passing internal resources here, for example, but i think that’s okay for now and solving for things like resizing textures based on window changes at a more coarse level in the main world might be the right place to do that anyway.

[brazen elm]

yes exactly i need a dag but really what is a tree if not a dag ^^

[brazen elm]

this way we could also more tightly
target the parallel command encoding use case for the eventual render graph refactor

[lilac basalt]

I think I'll keep working on my pr for now--even if it's not the exact thing we want long-term, it's not a huge jump from the status quo and I've been able to clean up the modules and api a bunch

mostly I just want to keep moving on the crate reorg lol

[brazen elm]

i’d also like to hear more of your ideas for viewports

[lilac basalt]

the core idea is I think cameras shouldn't manage their own composition config (viewport, ordering, etc), but there's two parts to it. Viewports specifically don't matter as much, but they're kinda emblematic of how we leak the "compositor abstraction" rn.

1. Camera (the component) shouldn't dictate the composition config, for the purposes of the reorg mainly. All composition code should stay in bevy_render, and there's other things (like UI) that aren't Cameras but we still want to composite them.

2. nothing on the camera entity should decide what the composition config is, either (at least not directly). IMO cameras should only have to care about themselves, and have as little possible coupling with the overall compositing workflow. Also. if we wanted to add a fancy viewport layout API a la taffy, we'd need a way to dictate viewports from the top down. The way I've done this for now is stupid simple, but also the smallest possible break from the status quo:

• Cameras (Views) are rendered in the order of the relationship source.
• For viewports, I stole the SubCameraView api almost verbatim. Cameras request a proportionally sized viewport, and the parent compositor turns that into an actual Viewport behind the scenes, which is what actually gets used in rendering.

[brazen elm]

ahhhh hehehe a compositor like an old timey type setter doing layout. whereas esp for my work we are (typically) compositing full screen more like blending photoshop layers. i’d like to keep the base api general and not prevent users from doing their own thing but esp the idea of using taffy sounds incredible. it’s amazing how much wasted code does the same painful recalc viewport stuff over and over. i think we could go pretty far with just parenting certain groups cameras under your compositor entity and having that do its magic?

i need to look at your pr (:

like the way i’d imagine it working in a node graph ui is a container that you’d put a series of cameras into. that’s typically how doing layout works in visual programming languages i’ve used. so you have the elements and then the contained defines the layout flow

[lilac basalt]

very WIP still. Got the frontend API pretty much set but I'm in exam season now

[brazen elm]

omg focus on your exams!! good luck 🧡

[molten tartan]

Sorry for the long period of silence (work has been heck and we've been working overtime to prepare for our baby!), but I think it's worth mentioning that wgpu as a whole has merged my MVP no_std PR, and the entirety of Naga (with basically all features) should be no_std compatible once new versions of its dependencies are published. Additionally, Image and bevy_asset are also getting some no_std attention that should be on the near-term horizon.

As such, it might be worth keeping in mind that no_std for some of Bevy's rendering should be quite possible soon.

[brazen elm]

wrote up some thoughts about the above here #engine-dev message

[static cypress]

Mmm I think your dag idea makes sense. A compositor can take multiple inputs and combine them. But also we could composite from the same texture into multiple cameras. And then those back into one. Potentially

[brazen elm]

right! my thought here is just not to be prescriptive. like maybe you want post processing effects on your ui. maybe you want to render 3d after your ui! lol. i totally appreciate carts comment about keeping the most common setup case simple and straightforward but i think implicit compositing and automatically selection of the main texture don’t actually matter for single camera setups and only really make multi camera harder to reason about.

i didn’t bring this up but there are also way more creative options for blending than what hardware provides. i’m not sure how useful that is specifically for games but you can get pretty far creatively just by playing with blending modes when compositing layers back together without needing to write your own post processing effects

[static cypress]

i'm all for building flexible low-level stuff and then convenience stuff on top for common use cases. and having good defaults and documentation for how to customise

[brazen elm]

okay so returning to this, i think we need to spin out a new bevy_material for common material infra that can be shared between 2d/3d

or should this just go in bevy_render?

bevy_core_pipeline?

[river dew]

New crate please

I am not keen on adding more stuff to either bevy_render or bevy_core_pipeline

[static cypress]

I agree. I think having material stuff in a single crate and migrating all material abstractions over to use that single one instead would be great. And then for some reason I feel like if it’s in its own crate it feels cleaner and less like it will creep

[river dew]

Yeah, it feels like a clearly defined scope to me

Maybe this is a skill issue, but core_pipeline doesn't have that to me

[brazen elm]

@lilac basalt how much more do you want to do on https://github.com/bevyengine/bevy/pull/17373 ? seems like there's good alignment on using it

decoupling the specializer is necessary for me to extract bevy_material, i think

[lilac basalt]

I might want to wait till tomorrow, I wanna have the first migration ready to follow it up

Needs some convenience stuff for mutating nested Options

[brazen elm]

oh yeah, no pressure!

[brazen elm]

@lilac basalt hmm yeah i'm still not also sure i totally understand how to use MeshVertexBufferLayoutRef as a key. in the specialization chain for MeshPipeline, would it be that first we specialize on MeshVertexBufferLayoutRef as a "non-cannonical" key and then specialize on MeshPipelineKey?

or the other way around?

[static cypress]

No, I don’t think it’s you. Core pipeline started out sensible but grew into what it is today with all kinds of things in there because that was the most appropriate place to make the code work, in my opinion

So it works but it feels wrong

[static cypress]

Left a comment. Basically I like it but don’t like the naming 🙂

[static cypress]

Every time you all write canonical and non-canonical my brain has no idea what you mean

In this context

https://github.com/bevyengine/bevy/pull/17373/files#r2174136432 <— maybe calling it a base key or something is simpler?

[brazen elm]

yeah it hurts my head a little bit too, i understand the problem it's trying to work around with respect to vertex layout, but i wonder if there isn't an approach where we "canonicalize" the vertex layout prior to specialization? is preserving the ability to specialize based on a specific vertex order versus just general attribute presence worth it?

[static cypress]

I think it’s more that I didn’t study computer science and despite 20+ years of programming, the words canonicalise and normalise in computer science are confusing to me and I’d much rather use other words

Like normalising or denormalising a database. What that means never sticks in my head

[static cypress]

I don’t see any mention of them in the PR. Did I miss it?

[brazen elm]

It's mentioned in the docs comments but not implemented

[brazen elm]

yes this is very real, as another never studied cser (:

[static cypress]

Ah right, MeshVertexBufferLayout contains all attributes, where many may not be relevant to the pipeline. VertexBufferLayout contains only the attributes and layout needed for the pipeline

So MeshVertexBufferLayout can be used to specialise to VertexBufferLayout, kind of

[static cypress]

I can now read this and understand what you’re asking

Though I didn’t read closely enough nor think enough to understand how one could use the PR code

Need more sleep now

[lilac basalt]

That's the approach I wanted to try at first, but @static cypress convinced me we need to be able to select vertex attributes (and respond to which ones are present) at specialization time

could definitely change the canonical/non-canonical language tho, that's just what came to mind first

[static cypress]

Our vertex buffer would always have the same order for the same set of attributes, for now. But I think we probably want to just load vertex data as stopped in an asset, and then we would need to sort different orders/layouts

I think vero wants to implement that

[lilac basalt]

is there another good word to refer to the process of taking a key and making it into something more "representative"?

[north trail]

isnt canonical w.r.t. file paths? like a canonical file path is one which is not a symlink but the actual real file path, and is usually absolute

[static cypress]

Isn’t the only difference that the key is compound or not?

Hmm. No. I guess saying that MeshVertexBufferLayout is non-canonical doesn’t mean it’s compound

So what is it? Whether the key only represents the possibilities that that specialisation can leverage, versus supporting multiple possible specialisers that only use a subset of the key?

[lilac basalt]

It's about making keys into better keys, is the simplest way to think about it

for meshes, a MeshVertexBufferLayoutRef maps to a pipeline, but not uniquely. Multiple refs can contain the same vertex attributes in different orders, or maybe some of the info from the key is unused by the shader. In both cases, you'd get redundant pipelines, which is where canonical keys come in. Where the main key represents some form of input to the specializer (MeshVertexBufferLayoutRef), the canonical key represents the actual state of the descriptor after specialization (the VertexBufferLayout)

So, if we get a cache miss from the primary cache (from plain keys -> pipeline ids) we'll always end up recomputing a specialized descriptor. But, we can check the secondary cache (from canonical keys -> pipeline ids) to decide whether or not to actually compile the pipeline

[brazen elm]

mhm mhm that makes sense. i think i need to spend some more time thinking about this problem space but understand the approach your pr took now

[static cypress]

Hmmm. What exactly is a plain key and a canonical key? Concrete examples are helpful

Trying to understand what the primary and secondary caches are

[lilac basalt]

for a minimal-ish example:

#[derive(FromWorld)]
pub struct SpecializeVertexLayout;

// "plain key"
impl SpecializationKey for MeshVertexBufferLayoutRef {
  const IS_CANONICAL = false;
  type Canonical = VertexBufferLayout; // "canonical key"
}

// still very open to better terms than canonical/non-canonical

impl Specialize<RenderPipeline> for SpecializeVertexLayout {
  type Key = MeshVertexBufferLayoutRef;

  fn specialize(
    &self,
    key: Self::key,
    descriptor: &mut RenderPipelineDescriptor
  ) -> Result<VertexBufferLayout, BevyError> {
    // pretty much same logic as in `impl SpecializedMeshPipeline for MeshPipeline`

    let mut vertex_attributes = Vec::new();

    if layout.0.contains(Mesh::ATTRIBUTE_POSITION) {
      vertex_attributes.push(Mesh::ATTRIBUTE_POSITION.at_shader_location(0));
    }

    if layout.0.contains(Mesh::ATTRIBUTE_NORMAL) {
      vertex_attributes.push(Mesh::ATTRIBUTE_NORMAL.at_shader_location(1));
    }

    ...

    let vertex_buffer_layout = layout.0.get_layout(&vertex_attributes)?;

    descriptor.vertex.buffers.insert(0, vertex_buffer_layout.clone());

    Ok(vertex_buffer_layout)
  }
}

the monomorphized Specializer would look like:

pub struct Specializer {
  specializer: SpecializeVertexLayout,
  ...
  primary_cache: HashMap<MeshVertexBufferLayoutRef, CachedRenderPipelineId>,
  secondary_cache: HashMap<VertexBufferLayout, CachedRenderPipelineId>
}

if you squint, this is just SpecializedMeshPipelines ^^^

(in this example missing the S::Key, the "rest of the pipeline key", since you'd probably want to use something like this composed with other specializers)

[brazen elm]

part of me wonders if the whole two level cache thing isn't just a huge unnecessary optimization. basically, it serves to make that that two specializations that have different "logical" vertex layouts but the same physical layout share a pipeline. when would this actually happen in practice? like i'm genuinely having trouble coming up with a scenario

[lilac basalt]

one could be gltfs loading with attributes that don't get used by the pbr pipeline

[brazen elm]

hmm

[lilac basalt]

but in practice yeah it could be entirely unnecessary, I was mostly going off of status quo

[brazen elm]

no for sure i think your approach is right in terms of modelling the status quo

and that is a good counter example

i'm just trying to push on the assumption here

because just putting the arc'd layout in the key and getting a new pipeline per logical layout sureeee would be simpler

[lilac basalt]

the "why not" for that is that it seems like we want to be able to choose the specific layout during specialization

• use present logical attributes to drive shader defs

at least when I asked rob and others, and it mostly makes sense to me too

[static cypress]

We need to support multiple different vertex layouts per shader, even with the same set of attributes, I think

[dense kraken]

In my math experience, canonical means something like a "unique" or a "specially selected, privledged" choice.

Ex: If there exists a canonical operation between two objects, then it has a special place and is usually the default, even though you could define many other such operations.

For the filepath example, there's many ways to write down a single path, but the canonical path is chosen as a special form that always exists and is unique, and you can/should convert your path to the canonical form; canonicalising it.

Canonicalisation is useful for comparisons, because while nonequal objects may be semantically the same (say, two equivalent trees that have their vertices shuffled in memory, but represent the same structure), their canon forms are most likely equal verbatim and it's thus easy to check for equality there.

Not a rendering or a database person though, so take this explanation with a grain of salt. 😅

[north trail]

yeah im also familiar with this

canonicalization is idempotent

and is a way to define an equivalence class

[static cypress]

Ok. I understand the structure of this code. Just thinking about the consequences. Is the point that a non-canonical key can be used to pass into specialisation, and canonical can come out of specialisation based on what the specialisation actually finalises as being needed? And then both the non-canonical and canonical keys are used as cache keys or only the non-canonical one or only the canonical one? I guess only the canonical one would make sense. So specialisation is done to understand whether the canonical key changed, and if not then the pipeline will be the same (assuming the material type or type id is also part of the caching somehow)

[static cypress]

If Specializer isn’t generic over the material type, and the cache keys don’t contain the material type id, then I don’t think this will work generally. But perhaps you were just throwing out the majority of an example to demonstrate the APIs?

I don’t understand the need for both non-canonical and canonical caches of pipelines

[static cypress]

And @brazen elm The specifics are that the pipeline descriptor contains the specific vertex buffer layout and we may need to support multiple different vertex buffer layouts from different mesh assets that have their mesh data loaded directly without us shuffling attributes around in our mesh code like we do today. So even if the same attributes are used by the shader, they may be in a different order in the different meshes, and so different layouts and different pipelines are needed

[lilac basalt]

Yeah I just monomorphized it to show it’s the same as SpecializedMeshPipelines

ish

[lilac basalt]

(1) is correct, yeah. For (2), both keys are used: using just the canonical key would be correct, but it wouldn’t be as fast. If we know the non-canonical keys match, then we can just take what’s in the cache and don’t have to make a new descriptor.

Note that this is only in the case when S::Key::IS_CANONICAL = false. Otherwise, the key is canonical from the start and we can just use the primary cache for everything

[static cypress]

I don’t know what monomorphism is either, though I would guess ’single form’ so like a concrete example of a type for generics

[static cypress]

Ah I suppose that is true. If the non-canonical key is the only input and it completely matches, and specialisation only uses its arguments to produce output, then it can only produce the same output. A pure function I guess, to use a term

[north trail]

monomorphization is the compile-time expansion of generic code into concretely typed code

[brazen elm]

that totally makes sense. didn’t mean to send us tooooo deep down this rabbit hole just also wanted to understand how to use a non canonical key in @lilac basalt pr

[static cypress]

It’s important for us to understand the API changes and also as the mesh stuff is a key part of specialisation

[static sierra]

Is this complete or still in progress?

[brazen elm]

oh no it's so in progress just has been blocked waiting for 0,17

[ebon lodge]

Waiting for godot bevy 0.17

[brazen elm]

we’re going to pick this back up with some work @north trail and i have been planning for a bevy material crate

[thin drum]

@north trail https://github.com/bevyengine/bevy/pull/21205

[north trail]

yes!!! thats exactly it :D

tysm

so with these changes bevy_material is possible, i have another wip branch up for that i can rebase onto this soon hopefully

but the main thing now is checking that its not a performance regression/not too bad of a regression (i think 1-3% slower is acceptable, 5% maybe not and we gotta rethink a bit)

[thin drum]

it's still WIP, I've added comments on the bits that I still have to do

would appreciate direction, as bits like material.unprepared_bind_group scares me

[north trail]

i appreciate the efforts

i'll check it out locally soon

[thin drum]

Okay made some progress, but hit a roadblock with AsBindGroup derive, see https://github.com/bevyengine/bevy/pull/21205#discussion_r2378266811 for details

[brazen elm]

super awesome work on this, i’ll do a closer review and maybe some benchmarking later

[thin drum]

@north trail I've taken a crack at moving MaterialProperties out from bevy_pbr into its own crate bevy_material, and while bevy_material looks pretty good , bevy_render and bevy_pbr have some issues. This is mainly due to orphan rule, but also some parts are unclear to me what should and should not be in bevy_material . Could you have a look at just crates/bevy_material/ and point out anything that should not be there?

[north trail]

sweet! i'll take a look today, thanks so much :)

and sorry that the BGL change hasn't gotten more attention, we're just a bit busy

[north trail]

https://github.com/bevyengine/bevy/pull/21533 this might help, still reviewing

[thin drum]

New version: https://github.com/bevyengine/bevy/pull/21543 - this compiles, and there's not any shortcuts (i.e. todo!() )

[north trail]

oooo ill look soon, sweet!

[north trail]

this looks really good

[brazen elm]

awesome work, can't wait to review >:)

[cinder stream]

Is this the PR that will add bevy_material or is it meant to be a PR on top of a PR that adds bevy_material?

The naming and descriptiong makes it sound like it's just a simple case of moving a struct around but it seems to me like the core of the PR is bevy_material itself

[north trail]

this is a possible bevy_material, built on the bindgrouplayoutdescriptor change

its roughly what i had in mind for bevy_material

[thin drum]

Yeah it's 3x PRs stacked, your limits, then bind group layout descriptors, then material

Material one is just moves

[river dew]

Limits PR should be merged now 🙂

[thin drum]

I will resolve conflicts for BGLD - this one needs a proper review

[north trail]

@thin drum https://github.com/Zeophlite/bevy/pull/4

[river dew]

Yay 😄

[north trail]

im getting Jasmine to review too

[thin drum]

Thanks, I've merged that and added the clippy fixes

[north trail]

thanks!

[thin drum]

Ah yes, now I recall, I was getting this error beforehand, hence my TODO 😅

error: redundant closure
   --> crates/bevy_render/src/render_resource/bind_group.rs:601:38
    |
601 |             label: Self::label().map(|f| f.into()),
    |                                      ^^^^^^^^^^^^ help: replace the closure with the method itself: `std::convert::Into::into`
    |

[north trail]

Self::label().map(Into::into::<Cow>)

or maybe Self::label().map(Into::<Cow>::into)

[brazen elm]

the story of bevy development 🥲

[river dew]

@brazen elm @ebon lodge @north trail are we good to merge https://github.com/bevyengine/bevy/pull/21205 now?

[brazen elm]

i'm down!

[river dew]

Assuming CI passes :p

[river dew]

Thanks ❤️ This PR is a bit over my head still, but I can broadly follow it and I would really like to unblock bevy_material

[brazen elm]

we feel good about being able to address performance and despite being a large diff it's relatively easy to back out

[river dew]

Yeah, and Vero's migration guide / review comments are really helpful for me for understanding the actual point of the change

[brazen elm]

if ever in doubt, just remember wgpu-types= good, wgpu = bad

that's our mantra

we ❤️ wgpu-types

[river dew]

Noted, noted :p

Would alternate rendering backends pull a dep on wgpu-types?

[thin drum]

Yep pretty much

[river dew]

Also relying only on wgpu-types should help compile times right?

[brazen elm]

yup! basically the way to think about it is that wgpu-types is what we would invent if we were doing it ourselves anyway

[thin drum]

This worked

[river dew]

Oh so it's a bevy_math sort of situation :p

Where it's actually framework agnostic but everyone is scared of it because of branding

[brazen elm]

i'm scared of it because i'm stupid and don't understand all the big math words

[river dew]

False!

|| also come on, it could be nalgebra||

[north trail]

thanks yall 💚

@thin drum i forgot to check, solari and meshlets both work right?

like they compile

[thin drum]

Compile, but my mac can't run them

[thin drum]

https://github.com/bevyengine/bevy/pull/21572 removes unused stuff that helps bevy_material

[north trail]

https://github.com/bevyengine/bevy/pull/21573

[thin drum]

bevy_material is rebased on main

[brazen elm]

@north trail @thin drum i’m free next week to review and work on whatever. i’m starting some relevant materials work for processing and would love to build on top of this

i’m wondering if we can articulate the current blockers vis-à-vis the dependency graph

[north trail]

currently reworking bevy_material on top of this pr

[north trail]

https://github.com/bevyengine/bevy/pull/22417

^ needed for this scratch that its not actually needed

[north trail]

~~ https://github.com/atlv24/bevy/compare/ad/temm-ci...atlv24:bevy:ad/material2?expand=1 ~~ https://github.com/tychedelia/bevy/compare/type-erase-more-materials...atlv24:ad/material2?expand=1

^this is where i left off, bevy_material builds but the rest of bevy hasnt been updated

my workday is starting now, so i'll get back to this in around 8hr

[brazen elm]

time for me to pick it up! 🙂

[north trail]

https://github.com/bevyengine/bevy/pull/22426

[north trail]

btw @thin drum i tried cherry-picking https://github.com/bevyengine/bevy/pull/21543/commits/c0688a4025d5734b9c8c232f6a86f1876a21d55d but it doesnt seem to work on main, it crashes on some missing resource access. did you include some other change in another commit/file to get that to work?

itd be nice to get that commit on main too

[thin drum]

Hmm, I definitely tested it back when I first wrote it, I guess somethings changed

@proven cedar could you have a look at https://github.com/bevyengine/bevy/pull/22443/changes -

Running with cargo run --example animated_mesh --features debug gives

Encountered an error in system bevy_pbr::deferred::init_deferred_lighting_layout: Parameter Res<MeshPipeline> failed validation: Resource does not exist

But strangely, DeferredPbrLightingPlugin (which has init_deferred_lighting_layout) is after MeshRenderPlugin (which has init_mesh_pipeline ) so I would assume it would exist?

[proven cedar]

Adding a plugin after is not sufficient, you need to add system ordering constraints with .after to anything that uses the MeshPipeline

[brazen elm]

@short quarry this is the other one i'll staff. but i think this is also worthy of reflecting on how to frame re: the new goal system. the overarching goal is "make bevy materials better". this specific wg/workstream is to (a). create a bevy_materials facade and (b). unify our internal rendering architecture for 2d/3d/ui. i think both the WESL wg and this are therefore sub-goals of a broader (potentially multi-year, many sub-goaled) "materials" goal?

[cinder stream]

I think I'd prefer smaller/iterative goals instead of a massive one like that

Just the unification alone is a big goal

[short quarry]

Yeah I think an uber Improved Materials goal is probably overkill here

Unification feels like a very reasonable standalone Goal

And importantly: well defined

WESL Shaders also feel well defined / standalone / self-justifying

Whereas we will be "improving materials" forever 🙂

[brazen elm]

yes, for sure

i'm totally fine with that and agree

[short quarry]

I acknowledge that I said that something like Improve Materials is worth considering as a Goal.

I think, especially before we know more, that kind of thing is ok

[brazen elm]

but they are related in some meta sense, which maybe doesn't matter

[short quarry]

But its also the kind of thing I'd want to break up and redefine as we go. And I think we have enough knowledge of the path forward to be more specific at this point

[short quarry]

I do agree!

I'm also hesitant about something like a Bevy Editor goal, as that also feels pretty undefined / we'll be building that forever

[brazen elm]

yes, for sure

i think perhaps what this is gesturing towards is perhaps something more like a "sub area" of A-Rendering

[cinder stream]

To be fair, depending on how you look at things a lot of things can be related in some meta sense 😆

[cinder stream]

It would also go against one of the goal of goals which is to better communicate what is being actively worked on right now

[brazen elm]

oh for sure, which is why i'm not sure if it's worth tracking

[short quarry]

If we were to make this, it would be in the Blocked (Approved) state

[cinder stream]

Right, fair, but casual viewers would just see "Goals > Editor" then think we are working on it 🙃

[short quarry]

Yup! (ex: A-Editor is a thing)

Although I'm not sure the Materials VS Rendering distinction is worth it

[brazen elm]

yeah wasn't exactly proposing that

but okay totally, we're on the same page

this is clarifying

[short quarry]

We can do what we want / this is the type of thing we'll narrow in on as we go. Glad we're discussing it.

[brazen elm]

i think the next sub goals in this path are something like:

• wesl generics, probably with "deferred vs forward not sucking" as the mvp deliverable
• work on a different material "front end", which will probably require a few r&d iterations

[cinder stream]

What do you mean by front end in this context?

[brazen elm]

user facing material api basically

[cinder stream]

Also, is wesl generics really a bevy goal? It seems to me like it's all on the wesl folks, not us. Unless you mean, like, using generics in wesl once we are using wesl

[cinder stream]

Right, that's what I thought, but just making sure

[brazen elm]

i think we want to be closely involved in the spec process, but also i think that we'll want to prototype whatever comes out of that to inform the process. so yeah using it

not the actual language transpiler implementation, that's on them

but with generics esp they are very open to being informed by our needs/use cases

so i think it'll be a close collab

[cinder stream]

Yeah, I guess being involved in the spec process can be a goal too. Or sub goal of the wesl shaders goal.

[north trail]

Can we have a Render Recovery goal? IMO It is necessary for the editor's crash resilience. (I would propose it but I am supposedly not allowed to.) The initial support is already merged but we'll need some coordination to figure out how to reinitialize gpu resources to not immediately re-explode after recovery