I'm building an editor-style application

[stray elk]

To further explore the RenderTexture idea. I could:

1. Create a secondary camera that renders to a RenderTexture
2. Add a Raw Image UI element to your Canvas and assign the RenderTexture as its texture
3. Use Layout Groups to position your Raw Image and other UI elements
4. Handle viewport interactions by raycasting through the Raw Image to the 3D scene

Or maybe a better approach:

1. A "ghost" UI panel (RectTransform with no graphic component) that lives in your Canvas layout system
2. This panel automatically resizes with your UI layout
3. You extract its screen position and size at runtime
4. Convert these values to normalized viewport coordinates (0-1 range)
5. Set your 3D scene camera's Viewport Rect to match

[sand fractal]

Im interested in doing something similar at some point so I’m gonna follow along. Don’t have any concrete help to give right now. My concern for any solution is the same as yours; how to ensure you’re only drawing frames in the viewport when you need to.

Basically you’d want to build unitys built-in editor 3d model viewer (the one used for auditioning prefabs in 3d)

I’ve made a custom editor window that “borrows” that editor feature and I wonder if it might be possible to access it in runtime somehow.

[stray elk]

It’ll be a bit before I have time to try this out… cuz you know work and stuff.

Interesting idea on if you can package Unity editor functionality into your build and communicate with it. But also, I bet that pulling just that part of the editor out an packaging it depends on like 1k dependencies.

I think ultimately for my case I’m just going to implement what I need using some shaders and maybe an asset from the store for the gizmo. But even that I’ll probably implement. I’m big on simpler the better cut the fat from the ground up even if it takes more time up front.

But of course that’s always tricky when you are building in a box inside of a box instead of just the box itself

[stray elk]

Another bike shedding point: probs also want to setup a “redraw policy” basically set some state like “is dirty” and “is interacting” then manually control when the camera renders. That way it’s not running all the time

[mint pine]

Hello. This is a broad topic and as usual there are multiple options available with Unity. But for multi-resolution and UI heavy projects, UI Toolkit will provide a more modern and familiar workflow. Although UI positioned in the 3D world will require uGUI.

UI Toolkit was designed for projects with significant amount of screen space user interfaces, to provide a familiar authoring workflow for artists and designers, and provide textureless UI for better scaling.

The Unity Editor is developed using UI Toolkit, and we recommend UI Toolkit for community created Editor extensions.

Our documentation still recommends uGUI for Runtime, but this is mostly because UI Toolkit does not work out-of-the-box in 3D space and does not support customized shaders and materials. Note that both of these features are on the 6.x roadmap.

You can compare our UI frameworks [here] (https://docs.unity3d.com/Manual/UI-system-compare.html). We also provide a great ebook User interface design and implementation in Unity
for more in-depth coverage.

One great feature of UI Toolkit for business applications is the Data biding system. Mini tutorial here.

[sterile light]

Hey! Matt from Unity 🙂
For the camera interaction you will likely have to create something yourself. But using the Cinemachine Free look camera with the new input system is a great starting point. You can extend it simply with a script to block orbit interaction over UI and to add things like 3rd mouse scroll or double click to centre. I can share some code if you need. For the runtime gizmo we have used this in the past and it works really well https://github.com/HiddenMonk/Unity3DRuntimeTransformGizmo

[stray elk]

Thanks for the resources.

@mint pine I was hoping for some guidance/advice on the approach to positioning the 3D scene camera within the UI layout. Regardless of if I use UItoolkit (a web browser like renderer) or (uGUI a more traditional imgui widget toolkit) I’m going to need a robust solution for managing the 3D scene camera so it renders in only within the ui. Do you have any resources / examples of how my application would position and manage the 3D scene camera in a UToolkit layout? That would be super helpful. Or if you could validate/invalidate my above approaches?

@sterile light thanks for the resources as well. When you say blocking orbit over the UI that sounds like you would be referring to a system where I’d use screen space UI on top of a full screen 3D scene camera. Can you confirm this is a better approach to what I suggested above? I’m not really looking for a quick and dirty solution more of a “my specific use case” solution.

Thank you for your time.