genaray raycasting on texture

[latent solstice]

Starting a thread to avoid continue flooding flood the main channel.

Myabe check the position values of the raycast :

1. RectTransformUtility.ScreenPointToLocalPointInRectangle already returns a boolean value, so it makes your "Raycast condition" only doubling the check.
2. The localPoint output should be in the local space of the RectTransform, but it looks like you are still trying to transform it afeterwards. I'm unsure exactly what the doc states as "the local space of the RectTransform", it might need a bit of transformation but maybe not the one you are doing.

Like, what does the final debug log of the function look like for different clicking positions ?

@red jay

[red jay]

First of all thanks for helping me, i actually partly generated that script with chatgpt since i have no experience in working with shaders and extracting pixel values from it. Im gonna remove the extra condition for the first.

A few outputs are looking like this:
Top Left: LocalPoint: (-205.39, 453.69), UV: (0.001488102, 0.9946524), Texture Pos: (1, 1293), Color: RGBA(0.804, 0.804, 0.804, 0.804)

Top Right:
LocalPoint: (205.39, 450.63), UV: (0.9985119, 0.9913102), Texture Pos: (1298, 1288), Color: RGBA(0.804, 0.804, 0.804, 0.804)

Down Left:
LocalPoint: (-205.39, -454.92), UV: (0.001488102, 0.004010681), Texture Pos: (1, 5), Color: RGBA(0.804, 0.804, 0.804, 0.804)

Down Right:
LocalPoint: (201.10, -458.60), UV: (0.9880952, 0), Texture Pos: (1284, 0), Color: RGBA(0.804, 0.804, 0.804, 0.804)

So this is a bit weird. Since the UV is correctly, atleast it looks like it. The local point is odd tho. So probably you are right with the localpoint being the issue? The texture position also looks valid or?

[latent solstice]

The UVs look correct, as well as the texture pos.

Something that could help it to actually look at the readableTexture. You could simply make it public so it will show in the inspector, and when double clicking on the object field (or ctrl+click) you can actually display it.

[red jay]

So when we remove the x and y and replace it with just the localPosition whe Texture Coordinates/Location is huge, so some sort of calculation is still needed. What is even more weird is that the texture2D looks like this... The screen on the left is that the shader shows (the webview) and the tiny screen right is what the Texture2D looks like. Why? Thats kinda odd

[latent solstice]

What does the input texture of the shader look like ?

[red jay]

Like this

[latent solstice]

And, it's a regular texture or a rendertexture ?

[red jay]

Uhm... since that here works:

 Texture mainTexture = rawImage.material.GetTexture("_MainTex");
                Debug.Log("rawImage mainTexture: "+mainTexture);
                if (mainTexture is Texture2D tex2D) {
                    Debug.Log("READABLE MAIN TEX: "+ mainTexture.isReadable);
                    if (readableTexture == null || readableTexture.width != tex2D.width || readableTexture.height != tex2D.height) {
                        readableTexture = new Texture2D(tex2D.width, tex2D.height, TextureFormat.RGBA32, false);
                    }
                    Graphics.CopyTexture(tex2D, readableTexture);
                    Debug.Log("rawImage copied to readable Texture: "+readableTexture);
                }

I assume its a Texture2D

I just dont get why the texture is sized like a square 1300x1300. Its all so confusing, what are we missing? I bets its a small and dump detail

[red jay]

Its getting interesting. I made the generated Texture2D public and the one from the shader. The one from the shader looks like the picture attached, the generated one like i showed you earlier. So i assume that something here is getting messed up.

[latent solstice]

That's the one assigned on the material ?
What is the size of this texture ?

[red jay]

This here is the texture2D from the material.mainTexture. It has a size of 1300x1300. The generated texture has a similar size, however both look completly different.

This here is the copy we generate

from...

This here, the material.mainTexture

Using this code:

 if (mainTexture is Texture2D tex2D) {
                    
                    if (readableTexture == null || readableTexture.width != tex2D.width || readableTexture.height != tex2D.height) {
                        readableTexture = new Texture2D(tex2D.width, tex2D.height, TextureFormat.RGBA32, false);
                    }
                    Graphics.CopyTexture(tex2D, readableTexture);
                    Debug.Log("rawImage copied to readable Texture: "+readableTexture);
                    this.mainTexture = mainTexture as Texture2D;
                }

No idea why it copys it incorrectly

[latent solstice]

Would be nice to see more of this, like the texture inspector, and see its sizes.

[red jay]

This is what we generate:

[red jay]

And this is what the mainTexture looks like (Texture2D)

Both have identical sizes and identical formats, why the heck is the copy so messed up?

[latent solstice]

Ok, so it makes sense that the readTexture is 1300x1300, since the source is of that size.
But the result of the copy is more surprising :/

Note that since the source texture is already readable, you don't have to copy the data over to an other one, you can simply read from it, it might help 🙂

Simply doing readableTexture = mainTexture should be enough.

[red jay]

GetPixel not allowed on native textures.
UnityEngine.Texture2D:GetPixel (int,int)

Thats why i copied it initially... looks like you cant call GetPixel directly on it for some reason :/

How hard can this be got damn it

[latent solstice]

Oh, I've never seen that error before, and I don't find relevant information about it 🤔

May I ask what is the final end goal of the project here ?

I wonder if Texture2D.CopyPixels would have a different behaviour 🤔

[red jay]

The goal is to use a transparent webview as the UI-Layer for my game (To use JS,CSS and Html since those are far superior compared to UGUI and UI-Toolkit). The UI already works fine, however i can not cast trough it. Thats why im trying to do that CanvasRaycastFilter, to determine where i can click trough (all transparent pixels). Im gonna try Texture2D.CopyPixels next, however using a RenderTexture and copy it to the Texture2D works fine, thats incredible slow tho. I hope .CopyPixels is faster :/ Unity is so weird sometimes

I assume that something with the Texture2D creation or copying is just off. Like something missing there, not sure what however

[latent solstice]

If you want something fast, you could use a call to a compute shader with the rendertexture.

But I wonder if a webview doesn't ahve more performance impact since you need to do this raycasting, compared to using UGUI or UI-Toolkit ...

[red jay]

A computeshader? Can that one only target one specific canvas or layer? Since i only want the result pixel of that canvas, not the game in total ^^

It has a small performance impact, but tbh not that dramatic. Its like you have one tab open in chrome. It uses more memory and a bit more cpu but thats it, compared to the posibilities its worth it, and it speeds up development drastically. And who cares if the game runs at 300 fps with ugui or 200 with a webbrowser ui? 😄 (Furthermore, a scrollable list in ugui with pooled items was in my experience always incredible slow, with JS and a proper UI framework its much faster now).