Select object in 3d space via depth buffer test

[topaz current]

I'm trying to recreate the selection system from Mine Imator. As far as I can tell, it is a depth buffer test.

/// view_click(view, camera)
/// @arg view
/// @arg camera

function view_click(view, cam)
{
    var surf;
    
    surf = surface_create(content_width, content_height)
    
    render_camera = cam
    render_ratio = content_width / content_height
    
    surface_set_target(surf)
    {
        draw_clear(c_black)
        render_world_start()
        render_world(e_render_mode.CLICK)
        render_world_done()
    }
    surface_reset_target()
    
    var tl = surface_getpixel(surf, mouse_x - content_x, mouse_y - content_y);
    
    if (tl > 0)
    {
        // Find timeline to select
        if (!tl_edit && !keyboard_check(vk_control))
            while (tl.parent != app && !tl.parent.lock && tl_update_list_filter(tl.parent))
                tl = tl.parent
        
        // Select
        action_tl_select(tl)
        
        // Jump in list
        if (setting_timeline_select_jump)
            tl_jump(tl)
    }
    else
        if (!keyboard_check(vk_shift))
            action_tl_deselect_all()
    
    surface_free(surf)
}```

this is what mine imator does

it saves the viewport to an image and compares what pixel your mouse was hovering over

[dense bison]

Just to confirm, you don't want to use the built-in physics to perform a raycast and get the owner of the collider, correct?

[topaz current]

correct

[dense bison]

Okay, just making sure. I'm not sure how I'd go about such a thing personally. My first idea would be to have each object have 2 meshes, one that renders visually, and one that renders to a second non-viewport-attached camera. The second mesh would use a simple shader, no light simulation, just vertex colors that are unique to each object. The second camera would render to a texture, and when you clicked on the screen a script would read the pixel in that same place from aforementioned texture, and depending on the color it would know which object you clicked.

This approach obviously has limitations and is probably about the least-elegant solution to your problem, but it's the best I've got and it seems nobody else has weighed in yet.

I wish you luck in your endeavors! 🙂

[topaz current]

problem with that solution

there is a completely transparent mesh on steve's head in mine imator, this is used for the second layer of the skin when needed

so i literally cant do a thing that detects what mesh you click on

mine imator lets you select parts of something once you select anything else in the scene, so if i select steve, it will let me select his arm if i click it

and it ignores alpha == 0 pixels in the click test

however perhaps i can adjust the solution slightly