portal shader help

[dull harness]

Ok weird question that is kind of complicated, imagine you have a mesh, i want to have 3d noise remove the mesh and then replace it with a look into a diffrent scene, like it disolves to show a portal effect. how could i acheave this?

[tawdry spade]

The difficult part there would definitely be the portal. Using a noise texture to mask/mix other textures is trivial compared to that.

This video explains whats required really well in my oppinion. And as stated there a "perfect" portal wouldn't really be possible without a custom engine build, as it requires more finite control over the transformation matrix of a camera which godot doesn't provide by default.
However, if it is truely supposed to be a different scene, then the camera probably wouln't have the problems clipping into a wall anyway. Just make sure you enable "own_world_3d" on the SubViewport so it is actually separate from the main world.

[dull harness]

ok cool

i dont need the teleportation part of it, just the rendering part lol

[tawdry spade]

oh i mean i didn't really touch on that yet anyway.

[dull harness]

Im gonna see if i can get it working with this as a basic guide

every pixel thats affected by the noise texture should instead use the portal camera stuff

[tawdry spade]

i mean a noise texture is just a value between 1 and 0, so you would have to define a threshold where to switch, or i would suggest having some sort of function so the transition is smoother.

[dull harness]

I will add something inbetween, but i want it to be kinda harsh in the transition. like a void

I dont know what im gonna put outside the reality of my dungeon, maybe a black hole, we will see

[tawdry spade]

well i mainly meant smooth in the sense of antialiasing, so you don't get the stairstepping effect from pixels only being either one or the other.

[dull harness]

ahh ok, that makes sense

Ive got that on my toon shader, idk how to fix it....

ill learn one day

[tawdry spade]

these days many probably don't bother because the game will probably use some sort of Screen Space Anti Aliasing like Temporal Antialiasing or FXAA anyway.
Adding AntiAliasing into the shader is mainly necessary when using MSAA, at least in 3D.

[dull harness]

I have no idea what any of those works are, i might just add it into the shader to cover my basises

[tawdry spade]

Those are general Antialiasing methods that are used to address those issues in the whole project. All have their own advantages and disadvantages. You can enable them inside the project settings or when using a SubWindow/Viewport inside its properties.

[dull harness]

oh sweet

btw when im trying to create the viewport texture it gives me this error

so i created a resource file, and the same error happened

how do i make a resource belong to a scene

I have a shader that effects everything in my game, my dungeon, and objects, so i want to apply this portal effect to the global shader

and have the void/portal scene in the backround loaded for the effect

[tawdry spade]

you have to use it on a material, which you probably have saved as a file. As a viewportTexture only really exists at runtime, it can't really be saved with the resource to a file (not sure how scenes handle it, but they probably have a built-in workaround for that).
Not sure if enabling local to scene on the material would work, but i would give it a try.

Alternatively i would suggest assigning the Viewporttexture when the game starts using code.

[dull harness]

would i have to do that for every object that uses the shader?

or just to my shader.tres

[tawdry spade]

only if each object uses a unique instance of that material.

[dull harness]

they do not

each object uses 1 material, mainly because im lazy, but also beacuse it keeps everyting in the same style

[tawdry spade]

then you just have to change it once on the shadermaterial, as they are by default shared by every scene and instance that uses it (at least when saved as a file like you did here).

[dull harness]

how would i turn a viewport into a viewport texture via code?

[dull harness]

so i did it i think, but now everything is yellow..

Ive created a resource that is a subviewport texture, then on ready im doing this

func _ready() -> void:
    var outsidedungeon = OUTSIDE_DUNGEON.instantiate()
    node.add_child(outsidedungeon)
    TOON_LIGHTING.set_shader_parameter("texture_albedo",OUTSIDE_DUNGEON_SUBVIEWPORT)

with the toon_lighting shader looking like this

void fragment() {
    vec2 base_uv = SCREEN_UV;
    vec4 albedo_tex = texture(texture_albedo,base_uv);
    ALBEDO = albedo_tex.rbg;
}

the camera is on layer 2, with everything on layer 1

but layer i mean cull mask

[tawdry spade]

hm weird

[dull harness]

its everything that has nothing on it

the purple cube is the cube here

everyelse is yellow

[tawdry spade]

you would use Viewport.get_texture(), then assign that texture using Material.set_shader_parameter().

[dull harness]

that fixes the issue

doesnt explain why the cube is purple thou lol

[tawdry spade]

is it still purple? usually a kind of rurple/magenta is used when a texture doesn't exist.

[dull harness]

do i have to do this every frame or just on startup

[tawdry spade]

no just once.

[dull harness]

the cube got no texture, so that would be why

not i just need a way to have it done via noise

[tawdry spade]

well but it also has no material that would require a texture.

[dull harness]

yeah thats what i mean lol

[tawdry spade]

so is it still purple for some reason?

[dull harness]

its got no material so yes

[tawdry spade]

in my experience just because a mesh doesn't have a material that doesn't mean it gets purple. usually the engine has a default material.

but i mean you could test it by giving it an actual material.

[dull harness]

still purple...

even with a material

[tawdry spade]

hm, the shape doesn't really fit.

[dull harness]

its behind a wall

[tawdry spade]

oh okay, but why can't we see the wall?

maybe it's because you messed with the cull masks?

[dull harness]

all of the walls have the shader

so its just black unless there is a object

it might be unlit, that could be why

I dont think its working as intened.

[tawdry spade]

the shader that is supposed to display the thing inside the subviewport?

[dull harness]

I think it was just rendering the cube in the world, not the screenspace

[dull harness]

yes

[tawdry spade]

how does your setup look

with the subviewport and the scene you want to render in it

[dull harness]

do i add_child the viewport scene to the main enviroment?

or is there a way i can run both at the same time in seperate areas completly

[tawdry spade]

just show how it currently looks like.

[tawdry spade]

yes by enabling "own_world_3d" on the SubViewport like i said at the beginning.

[dull harness]

that was prolly my issue

[tawdry spade]

you also talked about changing cull masks, i would also suggest reverting that to the default.

[dull harness]

this is waht i have right now

[tawdry spade]

yeah looks fine, except that you seem to instantiate the scene in ready but i don't see you using it. or is that just cut off from the screenshot?

[dull harness]

that was me doing something stupid

well something has happened...

[tawdry spade]

that does look like something being put om the walls in screen space

[dull harness]

It moves with the camera

its more like a overlay

so i gotta rotate the camera with the main camera

i think

[tawdry spade]

yeah

[dull harness]

why is everything else being rendered thou

[tawdry spade]

what do you mean with everything else?

[dull harness]

the enviroment

[tawdry spade]

you mean the outlines?

[dull harness]

I thought that this would replace everything (everything with the shader) with what the camera is seeing

lemi show you my shader setup

cause im doing something werid

[tawdry spade]

yeah, but the outline isn't using that shader it seems.

[tawdry spade]

you are using 2 next passes?

[dull harness]

I have 3 shaders on 1 matieral, the portal effect, the toon lighting, and the outline, how can i make the portal one effect them all

[dull harness]

yes

have i misunderstood what those meant

[tawdry spade]

each next pass renderes another copy of that object. it doesn't somehow "overlay" the color outputs.
so to combine those you have to combine the shader logic.

[dull harness]

into the same shader?

[tawdry spade]

yeah, at least your toon lighting needs to have logic to use the viewport texture. then also the outline shader.

so at least 2

[dull harness]

ok...

but if i add the logic to those 2 shaders, it should work?

[tawdry spade]

yeah

[dull harness]

ok, for the maybe final step i reckon, how can i make it so it only happens if the noise texture in that area is greate then x whiteness

or something

[tawdry spade]

i think the best way to do that would be to just sample the color of the noise texture, and then use its value (just one of the color channels as its greyscale) to mix between the color the terrain should have and the viewport pixel color.

[dull harness]

can i do a 3d noise texture? is that a thing i can even do?

[tawdry spade]

hm, i don't think those exist in godot unfortunately

wait no they do

https://docs.godotengine.org/en/stable/classes/class_noisetexture3d.html

[dull harness]

THEY DO???

YES

[tawdry spade]

then give it a gradient as the color ramp, where you can move the black and white positions together so it actually has regions that are fully one of the two.

[dull harness]

now i just have to take the vertex position and check if its white or not compared to the noise3d texture

[tawdry spade]

yeah, although it's not the vertex

[dull harness]

oh rip

since its in the fragment shader thing

[tawdry spade]

wait, i said you would use the value of the noise texture at that pixel to mix the color from the toon shader and the color of the viewport texture together

[dull harness]

i just want to cut to the viewport texture if its above a certain value, or is that a bad idea

[tawdry spade]

well with the method i suggested with the color ramp that behavior would be done using that gradient. so it wouldn't be hardcoded into the shader.
if you want it to change above a speciffic value, just change the interpolation of the gradient to constant.

and adjust it accordingly

[dull harness]

and how do i read the noise 3d and do the whole position thing

[tawdry spade]

hm, in that case since it's a 3d texture it's probably better to use the Vertex to sample it.

instead of some kind of UV

[dull harness]

but im using SCREEN_UV in the fragment shader to make the portal effect work

[tawdry spade]

those things can be independent from eachother

[dull harness]

they are?

[tawdry spade]

or wait, do you want the noise that blends between the two "worlds" to also be in screen space?

[dull harness]

I want the noise to determine what world is shown, if the noise is white/above a certain value, it shows world 2, if not it shows world 1

and i want the noise to kinda be overlapping the actual world, if that makes sense

so everything intercecting the white would be world 2

if that makes sense

[tawdry spade]

no what i was trying to ask is that how should that noise move in the world? should it be locked to the objects, or should it be locked to your view?

[dull harness]

im not sure what either option would be

[tawdry spade]

should a certain part of the wall always be showing the other world, no matter from where you're looking at it? Or should it show the other world when being at a speciffic position on your screen?

[dull harness]

it should allways show the other world

[tawdry spade]

okay in that case the sampling should be done using the Vertex position and a 3d noise.

[dull harness]

how do i sample the noise texture?

well spesific cords of the noise texture

[tawdry spade]

kinda like any other texture inside a shader, just using the Vertex instead of UV.

[dull harness]

could you explain that to someone who has never used shaders before?

Ive been stumbling through this whole proccess like ive had both of my feet chopped off

[tawdry spade]

don't you have that shader to display the viewporttexture?

[dull harness]

I do

void fragment() {
    vec2 portal_base_uv = SCREEN_UV;
    vec4 portal_albedo_tex = texture(portal_texture_albedo,portal_base_uv);
    ALBEDO = portal_albedo_tex.rbg;
}```

[tawdry spade]

the texture()-function is used to get the color value of a texture, or in this case called sampler, at a speciffic position.

usually specified with uv, but it's just a vector.

and in the case of a 3d texture there isn't really uv for that anyway, and the Vertex-positon is usually used instead.

[dull harness]

so something like this?

    vec4 portal_3d_sampler = texture(noise_texture,portal_vertex );

[tawdry spade]

yeah, although you can just use VERTEX directly inside the function, you don't have to define it as an additional variable first.

[dull harness]

so now that ive sampled it, how would i render the world 2 if the sample is above a value

[tawdry spade]

well for that you would check if it is above a certail value, and with that change between the two colors you have.
But because of the method using the gradient to already put that threashold inside the noise texture, you just have to mix the two values together.
so ALBEDO = mix(<toon_color>, <portal_color>, portal_3d_sampler.r)
replace the variables with the names you are using inside the shader.

[dull harness]

and this is still in the fragment shader? because my portal color shader uses a uv texture to work which doesnt work in fragment, if its not in fragment, how would i even do it?

[tawdry spade]

huh, why would that not work in the fragment shader?

[dull harness]

I mean vertex shader

cause vertex is required in the sample

[tawdry spade]

yeah this is still fragment shader. Vertex is also a variable inside the fragment shader.

[dull harness]

oh

[tawdry spade]

it's confusing i know

[dull harness]

this is a view

this is fun

shader_type spatial;
render_mode blend_mix,depth_draw_opaque,cull_disabled,diffuse_toon,specular_toon;

uniform sampler2D texture_albedo : source_color;
uniform sampler2D color_gradient;
uniform sampler2D fresnel_gradient;

uniform sampler2D portal_texture_albedo: source_color;
uniform sampler3D noise_texture: source_color;

void light() {
    float dotproduct = clamp(dot(NORMAL, LIGHT), -0.9, 0.9);
    float sample = clamp((dotproduct + 1.0) * ATTENUATION / 2.1, 0.05, 0.95);
    vec4 shaded = texture(color_gradient, vec2(sample,0.0));
    DIFFUSE_LIGHT += clamp((shaded.rgb * LIGHT_COLOR), 0.0, 1.0);
    // Uncomment to replace the default light processing function with this one.
}

void fragment() {
    //The toon shading
    vec2 base_uv = UV;
    vec4 albedo_texture = texture(texture_albedo,base_uv);
    
    //The portal shader
    vec4 portal_3d_sampler = texture(noise_texture,VERTEX);
    vec2 portal_base_uv = SCREEN_UV;
    vec4 portal_albedo_tex = texture(portal_texture_albedo,portal_base_uv);
    ALBEDO = mix(albedo_texture.rgb,portal_albedo_tex.rbg,portal_3d_sampler.r);
}

this is what i have as my setup

[tawdry spade]

looks good, but i would suggest enabling seampless on the noise texture, so it loops better.

oh and i just noticed that you didn't give the noise texture a noise generator yet at the bottom.

[dull harness]

ok, how do i get the current camera?

I wanna set the viewport cameras rotation, to be the current cameras rotation

[tawdry spade]

you can use get_viewport().get_camera_3d() as long as the node you're calling from and the camera they share the same viewport.

so just not inside the subviewport.

[dull harness]

so the noise is moving with the camera

[tawdry spade]

hm, do you now want the noise to move with the camera or not?
or do you mean the camera inside the subviewport moving with the camera?

[dull harness]

i want the subviewport camera to move with the actuall, camera. and the noise is moving with the camera view

these are 2 diffrent issues

the shader is broken and based on the camera when it shouldnt

this is the portal shader

    //The portal shader
    vec4 portal_3d_sampler = texture(noise_texture,VERTEX);
    vec2 portal_base_uv = SCREEN_UV;
    vec4 portal_albedo_tex = texture(portal_texture_albedo,portal_base_uv);
    ALBEDO = mix(albedo_texture.rgb,portal_albedo_tex.rbg,portal_3d_sampler.r);

[tawdry spade]

hm, did you give the noisetexture a noise generator? maybe that's some behaviour because of that.

[dull harness]

I did

[tawdry spade]

how does it look now?

[dull harness]

the noise texture is just moving as the camera moves, it even moves in the editor

[tawdry spade]

that's bizarre. sounds like vertex is in view space for some reason.

although given the documentation it should be in object space.

[dull harness]

i think its something to do with portal_base_uv

im not sure thou

[tawdry spade]

but you aren't using that to sample the noise texture

[dull harness]

im not

[tawdry spade]

well one way would be to add this

varying vec3 noise_position;

void vertex() {
    noise_position = VERTEX;
}

and then use noise_position in the fragment()-shader instead of VERTEX.

that way there should be no way for it to be in view space.

[tawdry spade]

try the code i showed above

[dull harness]

that seemed to have fixed it

[tawdry spade]

hm, so vertex is in camera coordinates for some reason

[tawdry spade]

oh wait, the docs state that it is in view space. i seem to have mixed up the vertex and fragment entries, my bad.

[tawdry spade]

what exactly are you showing me here?

[dull harness]

now its just having issues displaying what the viewport is showing

[dull harness]

just the setup of the viewport, and a video showing weird behavour imo

[tawdry spade]

in what way weird now?

[dull harness]

the square is cutting of the black, to show the main texture, instead of just showing the square

[tawdry spade]

could you maybe try changing the shader to unshaded?

i have the feeling this comes from the fact that your light()-shader also runs on the regions that are supposed to be displaying the portal without any shading applied

[dull harness]

these are all of the render modes on the shader right now
render_mode blend_mix,depth_draw_opaque,cull_disabled,diffuse_toon,specular_toon;

I just tried chaging specular_toon to be unshaded, and it did the same thing, but without lighitng

[tawdry spade]

also removed diffuse_toon?

[dull harness]

same issue

but just unlit now

[tawdry spade]

well could you make another video?

oh, what happens when you comment out your light()-code?

[dull harness]

wait... it might be working

let me try something

[tawdry spade]

sure

[dull harness]

it is working

the white cube just looked like the walls

and because only a small amount of the walls was white, it made it look like it was the original texture

anyway to fix this issue?

and to make the viewport unlit from the enviromental lights?

without affecting the base texture lighting?

[tawdry spade]

i don't understand why that should be the case.

[tawdry spade]

i also can't really see anything you're describing inside that picture.

[dull harness]

phycological glitch

my brain thought it was the texture when it was just white

so it wasnt a issue

[tawdry spade]

well i can't see any white in that image

[dull harness]

ignore it, it wasnt a issue

there is now the weird unseemless texture, and the lighting issue

[tawdry spade]

the unseemlessness should be fixed by adding world_vertex_coords to the render modes

and for the lighting, you should be able to re-sample the noise texture inside the light()-function. then multiply the light you calculate with the value from the noise before adding it.

[dull harness]

like this?

    float dotproduct = clamp(dot(NORMAL, LIGHT), -0.9, 0.9);
    float sample = clamp((dotproduct + 1.0) * ATTENUATION / 2.1, 0.05, 0.95);
    vec4 shaded = texture(color_gradient, vec2(sample,0.0));
    
    vec4 portal_3d_sampler = texture(noise_texture,noise_position);
    DIFFUSE_LIGHT *= portal_3d_sampler;
    
    DIFFUSE_LIGHT += clamp((shaded.rgb * LIGHT_COLOR), 0.0, 1.0);
    // Uncomment to replace the default light processing function with this one.
}

cant multiply a vec3 (difuse light) with vec4, the samepl

[tawdry spade]

no, multiply what you're adding to the diffuse light, not the diffuse light itself.

also you should probably only multiply it with a single value, not a vector. so only the red channel again.

oh but wait. the noise texture is white when it's supposed to be the portal, right? in that case you have to use the "inverse". so 1.0 - texture(noise_texture,noise_position).r.

[dull harness]

ah so ```void light() {
float dotproduct = clamp(dot(NORMAL, LIGHT), -0.9, 0.9);
float sample = clamp((dotproduct + 1.0) * ATTENUATION / 2.1, 0.05, 0.95);
vec4 shaded = texture(color_gradient, vec2(sample,0.0));

vec4 portal_3d_sampler = texture(noise_texture,noise_position);
DIFFUSE_LIGHT += (clamp((shaded.rgb * LIGHT_COLOR), 0.0, 1.0))*portal_3d_sampler.r;
// Uncomment to replace the default light processing function with this one.

}```

[tawdry spade]

almost, i also added the fact that we need the inverse in this case so

void light() {
    float dotproduct = clamp(dot(NORMAL, LIGHT), -0.9, 0.9);
    float sample = clamp((dotproduct + 1.0) * ATTENUATION / 2.1, 0.05, 0.95);
    vec4 shaded = texture(color_gradient, vec2(sample,0.0));
    
    vec4 portal_3d_sampler = texture(noise_texture,noise_position);
    DIFFUSE_LIGHT += (clamp((shaded.rgb * LIGHT_COLOR), 0.0, 1.0))*(1.0 - portal_3d_sampler.r);
    // Uncomment to replace the default light processing function with this one.
}

[dull harness]

IT WORKS

maybe

wait.

turning up the threshold gives this result

Ignore the looping of the texture

instead of being black it should be blue

[tawdry spade]

hm, that is weird

[dull harness]

it is..

[tawdry spade]

oh!

[dull harness]

oh?

[tawdry spade]

how do the render_target settings of the subviewport look like?

[dull harness]

like this

[tawdry spade]

maybe set the update_mode to always?

[dull harness]

nope

same issue

[tawdry spade]

hm, and you do have that code that assigns the viewporttexture to the material

[dull harness]

const TOON_LIGHTING = preload("res://graphics/shaders/toon_lighting.tres")
const OUTSIDE_DUNGEON = preload("res://graphics/outside_dungeon/outside_dungeon.tscn")

func _ready() -> void:
    var outsidedungeon = OUTSIDE_DUNGEON.instantiate()
    node.add_child(outsidedungeon)
    var viewporttexture = outsidedungeon.get_node("SubViewport").get_texture()
    TOON_LIGHTING.set_shader_parameter("portal_texture_albedo",viewporttexture)

wait

[dull harness]

this is the issue

removing it i get this
but its now lit

[tawdry spade]

it seems like it's quite often just that you didn't add something i noted will be needed XD

[dull harness]

possibly

[tawdry spade]

but it's still shaded for some reason...

[dull harness]

this is removing what you said

using the old light func code

[dull harness]

using what you wrote gives the error of this

the diffrence

[tawdry spade]

okay i think i misunderstood a bit how the lighting works (not that experienced in it as i primarily work in 2d). where in the lighting function is the color the surface has from the fragment-shader?

[dull harness]

i dont think it does

the lighting doesnt take in a texuture or color maybe

the color gradient is the toon lighting/shadow

[tawdry spade]

right, lighting is additive, so when we have something that isn't supposed be be shaded, we instead want to set the diffuse to white (vec4(1.0))
so it would be another mix

void light() {
    float dotproduct = clamp(dot(NORMAL, LIGHT), -0.9, 0.9);
    float sample = clamp((dotproduct + 1.0) * ATTENUATION / 2.1, 0.05, 0.95);
    vec4 shaded = texture(color_gradient, vec2(sample,0.0));
    
    vec4 portal_3d_sampler = texture(noise_texture,noise_position);
    DIFFUSE_LIGHT += mix(clamp((shaded.rgb * LIGHT_COLOR), 0.0, 1.0), vec3(1.0), portal_3d_sampler.r);
    // Uncomment to replace the default light processing function with this one.
}

[dull harness]

the only time it uses color is the difuse lighitng

cant mix a vec3 and vec4,

[tawdry spade]

oh it's vec3. my bad in 2d it also has alpha.
fixed it

[dull harness]

seems to work

[tawdry spade]

hm it's still at the front lighter for some reason.

[dull harness]

Hmmmm

This is making the noise max white basically

so it would only show world 2

[tawdry spade]

oh right! because the light shader only runs if something gets affected by a light

[dull harness]

isnt that everything?

[tawdry spade]

no, lights use culling so not every pixel has to use every light, only those which could affect it based on the lights bounds.

[dull harness]

-# Yess......

so is there a fix?

[tawdry spade]

i'm unsure if there is a fix

like i said, i'm not that experienced when it comes to lighting in shaders. So it might be better to ask in #shaders-and-vfx if someone knows how to achieve the effect of unshaded but controllable for each pixel/fragment.

[dull harness]

ok ill have to ask prolly tomorrow, cause its getting quite late. but thank you for the help

Ill hopefully get it fully working tomorrow