#Texel Anti-Aliasing

146 messages · Page 1 of 1 (latest)

hollow owl
#

This plugin is meant to improve the visual fidelity of model previews in Blockbench.

Texel Anti-Aliasing replaces the shader programs for the Textured and UV View Modes to introduce anti-aliasing between texels.

This plugin is heavily inspired by t3ssel8r's anti-aliasing method that they explain in one of their videos, which I can highly recommend giving a watch if you're curious about how this works!
https://www.youtube.com/watch?v=d6tp43wZqps

Load Plugin from URL:

https://cdn.jsdelivr.net/gh/nestorboy/blockbench-plugins@texel-aa/plugins/texel_antialiasing/texel_antialiasing.js

Repo/Branch:

https://github.com/Nestorboy/blockbench-plugins/tree/texel-aa

Texel_Anti-Aliasing_Preview.png
shrewd mirage
#

talk to jannis and see if you can implement this into default blockbench

hollow owl
#

And I spent so much time learning how to write TypeScript/JavaScript and working with the Blockbench API to make this CryInside

#

In all seriousness, I definitely wouldn't mind having it merged into the base. The plugin does modify shaders in a way that makes it quite incompatible with other similar plugins

shrewd mirage
#

if jannis agrees, it would be a case of you then implementing it into blockbench and making a pull request on the repo, so the code wouldnt change too much

hollow owl
#

My initially implementation was off of a fork of Blockbench since I found it easier to simply replace existing behaviour over writing a new plugin, but then I was informed that I should probably make a plugin instead, and I ended up putting it off for several months due to other work 😅

shrewd mirage
#

nice lol

#

maybe instead of replacing it it should be added alongside it (if it has some downsides against the old one that is)

#

then in blockbench, instead of an antialiasing toggle, you instead have Off, Fast, Fancy or something

hollow owl
#

Yeah the behaviour I've got implemented right now actually ends up reverting the shader programs when the plugin is disabled/uninstalled, since it just has a function to get the vert/frag programs with a bool for if it should have anti-aliasing or not

#

Sorry about the extremely late pings, @cosmic scroll @nimble magnet, but I finally got around to turning this thing into a plugin, if you want to check it out still. As mentioned before, I will probably have a quick discussion with Jannis about merging it into the base, but for now you can get early access to it here!

hollow owl
shrewd mirage
#

@echo pewter

hollow owl
cosmic scroll
ripe warren
#

this is AMAZING!

nimble magnet
thorn coyote
#

Three includes this kind of shader pass in one of it's internal add-ons by default too, so it might be a good idea to look into that

hollow owl
thorn coyote
rich turtle
#

I'm not sure a post-processing effect is adequate for this tbh

#

Well at least not for a plugin. It might be a possibility for a "native" Blockbench impl

hollow owl
#

Yeah a post-processing pass wouldn't be suitable for this particular effect as it mostly modifies uv coordinates before texture sampling is done. Looks like a very nice API for post-processing effects though! Might use it for some other stuff in the future

rich turtle
#

There should be ways that could be done as a "post-processing" step in a deferred rendering-like pipeline? But well, that would require pretty significant changes to Blockbench's rendering afaik

hollow owl
#

Yeah I have been thinking of how one might use a deferred pipeline in order to also do the same for texel shading, but it wouldn't be possible to make it 1:1 due to overlapping geometry obscuring fragments that might be part of the final sum in a forward pipeline

rich turtle
#

I feel like there's something to be done if you get the original UV coords & texture

#

But at the same time I wonder if that wouldn't just result in basically SSAA but worse™️ ig?

hollow owl
#

Possibly 😅

rich turtle
#

Yeah idk and it's not really worth it tbh

#

Also, does ThreeJS even let you do deferred rendering?

hollow owl
#

No idea, I'm new to all this JS/TS stuff

rich turtle
#

It's weird, they had something for it at one point, then it got removed, and it seems to be possible to implement it manually but I can't really find information on how to do it nowadays

thorn coyote
#

The difference between your two before and after pictures is AA

#

That's usually accomplished via a post-processing effect

#

THREE.js has a SSAA renderpass built-in, and several other versions of AA

image.png
#

You can even change the render passes post setting up the canvas, which is highly useful for configurability

rich turtle
#

SSAA usually has significant performances penalties tbh

thorn coyote
#

Which is why I mentioned the other AA options

rich turtle
#

But I guess it wouldn't really be an issue in Blockbench, actually

#

It's not like it's doing particularly heavy rendering

thorn coyote
#

Performance is still a concern, so adding an option to disable / change the AA method would be a necessity for lower-end devices.

#

There are people complaining that the current release of BB drops below 120hz on their monitor

rich turtle
#

Huh

thorn coyote
#

I think that was also due to another plugin that added really bad AO but the point is we can't just add something like that and assume it'll be fine for performance

shrewd mirage
#

thats jannis' plugin lol

thorn coyote
#

oh, he wrote a custom AO shader

hollow owl
#

SSAA is just one way to tackle anti-aliasing, but it's also a more general solution where a bespoke one can do a better job. The way I've implemented it makes the fragment shader take advantage of weighted averages in order to introduce anti-aliasing at a much finer level for texel art rather than something like SSAA

image.png
rich turtle
thorn coyote
#

The comments are different, and his shader code is much smaller than the THREE version

thorn coyote
#

The video only shows NN, and Bilinear filtering vs PAA.

hollow owl
thorn coyote
#

Adding all three as an option would be the best solution then

#

I want to see a comparison before hand though, as just saying "this does it better with less overhead" without visual proof is a little silly imo

#

I get that it's a purpose built solution, but I want to see the results compared to other general and well tested solutions

hollow owl
#

The visual difference would be more quantization due to limited sampling points as well. Not sure what a fair comparison would be since you can't really compare them by setting them to different levels either, like I could crank up SSAA all the way in order to achieve similar results but what would be the point then?

thorn coyote
#

The difference would be in performance in that case

#

It would be very cool to see how well PAA does against SSAA with less overhead

hollow owl
#

I'll look into it

rich turtle
thorn coyote
#

PAA is more of a filter than SSAA though isn't it?

hollow owl
thorn coyote
#

More similar to Bilinear and Nearest texture filters compared to a post-processing effect like SSAA

thorn coyote
#

It's a bit confusing to compare it to filters if it's not itself a filter

hollow owl
#

Yeah, that's why we usually refer to it as fragment or texel instead of pixel, since there's ambiguity there

rich turtle
#

Let's not mention how a fragment isn't even a pixel most of the time (rendering is weird)

thorn coyote
#

BB's rendering is weird in the before after you posted

hollow owl
#

It is?

thorn coyote
#

Some edges are smooth and some are not before-hand

hollow owl
#

Ah yeah, that's regular MSAA working in conjuntion with texel AA

rich turtle
#

BB's has MSAA smoothing the edges of quads I'd assume

thorn coyote
#

What's strange to me is the upper left has smoothing and no smoothing at the same time on the same element with the same background

hollow owl
rich turtle
#

It only has AA at the border of the quad, that's MSAA

thorn coyote
#

oh that's because it's texture edge vs element edge

#

yeah that makes sense

hollow owl
thorn coyote
#

Why doesn't it effect the texture though?

rich turtle
#

That's just the way MSAA works

#

It's usually assumed you don't use nearest-neighbour sampling on your textures, so that doesn't need anti-aliasing

#

So MSAA only does AA on the edges of elements

thorn coyote
#

Ah, an optimization assumption then

rich turtle
#

Mhm

#

Otherwise it would just be, well, SSAA, basically

hollow owl
#

The term you're looking for is adaptive supersampling, where it only computes additional samples around edges of triangles, and it's for optimization as you mentioned

rich turtle
#

yeah this

hollow owl
#

This stuff is just too interesting

thorn coyote
#

So Texel is just a different filter for textures?

rich turtle
#

pretty much yeah

thorn coyote
#

cool

#

We could implement this, and the texture edge-leak tech in the same BB update

hollow owl
#

Texture edge-leak tech? Like texture padding?

thorn coyote
#

Yeah, basically what plaster does but as an internal step that doesn't affect the actual UVs

hollow owl
#

YES

#

If you weren't doing it already that was probably going to be my next plugin

thorn coyote
#

I can work with you if you need any help navigating around the BB source

#

I've already helped contribute a few big internal changes to the tool

hollow owl
#

I've only implemented it GPU-side before for my projection texture painting, but that meant I had to do manual conservative rasterization using a geometry program due to constraints for the platform I was making it for

thorn coyote
#

I think we can do it much simpler in BB, and just shrink the UVs given to THREE when constructing geometries

hollow owl
#

Plaster definitely has been bugging me and I've been manually doing my padding, might also be worth averaging the blocks of 2^n for better mip maps when exporting for other platforms

#

Should we create a thread somewhere for this?

hollow owl
#

I've got some pictures using the SSAARenderPass.js example now, let me just trim them

#

Might be easier to diff using some online tool

#

The main visual difference, as mentioned before, is that you end up with a limited amount of intermediate anti-aliasing colors when using ssaa, but it is quite a clean implementation and a more standard approach to anti-aliasing

#

If this were to be merged into BB I would probably want to have a dropdown for "no aa", "texel aa + msaa" and "ssaa" then

thorn coyote
#

I think texel should be default, I can see exactly what you're describing in terms of extra colors and it makes it look more blurry, while texel looks much sharper while keeping the lines from getting too jagged

hollow owl
#

I did also turn off the weighted sampling for this particular test since I felt like comparing the linear one was more accurate next to the SSAA ones

#

Either way, having any form of anti-aliasing for the texels would be quite a nice improvement

thorn coyote
#

yep

echo pewter
#

I think it would be cool to have this as an option directly in Blockbench. Maybe not as default since I did notice some visual issues, but there are definitely use-cases.

#

I would have made made no anti aliasing the default in Blockbench if it weren't for the weird line rendering without aa. That makes editing kind of annoying, and makes things like outlines and grids difficult to see. Other than that, it's the most consistent and has the least artifacts like texture bleeding.

#

Would also be cool to have a supersampling mode, maybe with an additional quality setting.

hollow owl
#

Also for the texture bleeding, it can be completely prevented when using MSAA by adding the centroid modifier to the uvs you pass from the vert to frag shaders, which I did in my case, but since I do anti-aliasing between texels it reintroduces a slight bit of bleeding again

#

The centroid modifier just prevents MSAA from extrapolating the data outside of the triangles

echo pewter
#

Regular MSAA would still be good to keep as an option. SSAA will be more taxing on performance

hollow owl
#

I'm rewriting this plugin as part of the Blockbench source instead, and I'll be doing a draft PR to get some directions in terms of implementation, like what settings we want to expose and how we should implement the shader changes

rich turtle
#

It might be worth looking into the glsl preprocessor? ThreeJS allows you to pass #defines to your shaders per material (see docs), so you could use the same shader code for two different materials in which you simply have an #if to include or exclude the custom texture filtering

#

That would mean replacing the shader material on setting change tho

hollow owl
#

Yeah, a macro would mean a recompile. Currently I've just got a uniform for the anti-aliasing since branching across that shouldn't cause any major overhead, and the code inside of that branch is simply the uv modifications either way

#

However, something which is more of an issue is that it doesn't work properly with semi-transparent texels, as it ends up reducing the precision for them, so for the past hours I've just been trying to figure out the proper blending modes or some way to remedy this issue

rich turtle
#

Yeah a uniform should be fine as well, and is definitely easier to use...

hollow owl
#

I think this is the best I can do whilst retaining the regular alpha blending behaviour..

image.png
hollow owl
#

The only proper way to tackle this that I can think of is to only pass the GPU premultiplied textures, but I don't believe that's how things are currently set up to work, so you'd need to keep track of an additional texture which is copied into from the main one

jolly birch
#

Would be nice to see this pulled into Blockbench in the future

#

Blockbench's current AA can cause some texture bleeding issues

hollow owl
#

I beelieve those texture bleeding issues can be remedied by just using the centroid qualifier for the UVs

jolly birch
#

well i have no idea what centroid qualifier is

hollow owl
#

It's a 1-line change to the shader they use ww

jolly birch
#

bruh
maybe you could make a pull request if it's that simple to change
(unless there are side-effects)

jolly birch
#

either way

hollow owl
#

Yeah I probably could, but I'm not sure if it should be a setting or not at that point since it's a visual artifact other platforms could suffer from too unless they implement the same fix in their shaders

#

The Texel Anti-Aliasing will definitely need more work though as I need to figure out a way to push a pre-multiplied texture to the GPU whilst still painting the raw colors on the CPU

#

Might just need to poke Snave about that or something

hollow owl
jolly birch
#

excellent

hollow owl
#

I ended up making it a setting too, so I'll have to update the shaders in this plugin once that ships, for people who still want to test this out