buoyant tangle
Idea Engine provides a way to create and share interactive VR stories and games. I’m integrating AI to generate stories, extract key objects and add them to the world, then create extended descriptions for each that can be used to generate images. Later I hope to add support for interactive characters that players can talk to. https://twitter.com/JumbliVR/status/1604170006044237825
nova lark
How do you do it
buoyant tangle
I pass in the user's prompt for a story but ask that it be returned in a json structure with additional properties for locations, objects and characters and that each has an extended description. If the user chooses to create a DALLE-2 image for one of those, the extended description is offered as a default prompt but that can be edited further if required.
charred abyss
This generates images or full 3d scenes? If it's just generating 2D images, what's the advantage of having it in VR?
charred abyss
Also, I saw you go into Microsoft's Founder's Hub - congrats. Do you know of any other interesting AR/VR startups integrating generative AI? I know it's early, but Point-E and other 3D-asset trained models should arrive in 2023. There will be some interesting use-cases when people can truly design their own spaces.
buoyant tangle
Point-E is exciting but too computationally expensive for my needs at the moment. I will be ready and waiting for it though. I don’t know about other AR/VR startups using generative AI but I expect it will be receiving a lot of thought.
My app supports importing 2d and 3d assets and audio. The 2d assets may be more interesting than you expect in VR and easier for users to create from photos, drawings or AI. I allow the 2d assets to be placed in 3d space and interacted with just as you would a 3d object. Imagine picking up a 2d (two sided) key. You can carry it and place it in the lock of a 2d chest and the chest then changes to an opened image. You can carry a 2d sword and hit enemies with it. I can detect when you are looking at an image and make things happen too. 2d can be a stylized approach to a 3d world. Next I will be looking at AI generated 360 backgrounds (equirectangular).
