Azure Spatial Anchors just needs to get the image and pose data, it doesn't need to write anything to the session, although it very well could be an access violation inside the native plugin code
#🤯┃augmented-reality
soft compass
I only know that Unity gets real pissy about accessing stuff that it owns "ilegally"
Have you checked for something so fundamental as a nullptr?
I don't know, just spitballing
sonic ivy
The pointer is definitely pointing to the right session. Null checks for days
The most likely culprit is either an access violation because of something I don't understand or can't control, locked memory or something like that, or it's a difference between the ARKit/Core packages you can download from the website and the new ones in the Package Manager
sonic ivy
You're definitely allowed to access it from inside Unity, but yeah maybe once it hits the native plugin it's an access violation
soft compass
Unity is very particular with memory access
It would not surprise me if ARFoundation just hands it off to a blackbox
And that's why you won't get to access it
sonic ivy
ARFoundation calls ARKit.Api.GetNativeSessionPtr -- so fundamentally it shouldnt work any different than UnityARKitPlugin, which is also creating a native session and passing the pointer, and works just fine with Azure Spatial Anchors
soft compass
Is it some DLL/SO nonsense then?
sonic ivy
Its worth noting that we got this all working without ARFoundation, just going directly to the ARCore and ARKit plugins, it's when we get the pointers through the ARFoundation abstraction that we run into trouble
Entirely possible. Azure's stuff is a black box though
soft compass
I think Unity takes ownership and then passes it off to the plugin, which is where it goes outside your domain because ARF then
sonic ivy
BUT a good clue is that ARFoundation won't work with the downloadable ARKit and Core integrations
Only it's own package dependnencies
soft compass
The integration is likely fundamentally different
sonic ivy
So maybe it's just like a plugin version conflict
Yeah, but even then at the end of the day the native session pointer is actually just a thing handled in Swift or Java. But this would illuminate why it works on Android and not iOS
Oh, that's the super annoying thing. We haven't gotten full progress, but it's at least running without crashing on Android, and it takes the session pointer just fine 😃
sonic ivy
The DLL doesn't work on desktops so you can't test it
soft compass
ARF does though, no?
sonic ivy
Nope
soft compass
huh
sonic ivy
I think it has a mock data struct
But no, it only runs on iOS and Android. It fails gracefully, gives you Vector3.zero for pose, etc
soft compass
So wait, ARF is not running on macs?`
sonic ivy
It's iOS and Android only
soft compass
Yeah
I get that
It's that to build for iOS you need a mac
It states so in the documentation no?
sonic ivy
Yeah, we do iOS dev on Mac
I'm on a Mac right now 😃
And Android dev on the PC and Mac because you never know
soft compass
hm
sonic ivy
The ASA plugin definitely works with ARKit and iOS. ARFoundation definitely works. The two don't like each other
soft compass
See like
From my understanding ARF is stricly iOS/Android. So I didn't really consider an MS tech to work with it out the box
Though; I've never used ASA
Lol
sonic ivy
You are correct. Azure Spatial Anchors are iOS/Android/Hololens. For Hololens, they just use the built in Hololens DLLs in Unity
soft compass
Welp
There we go then
I've had to deal with some headache from Unity back when I tried to do voice recognition in real-time in a VR experiment
I needed to use the Google Voice-to-Text service. A coroutine could not make it run async, the HTTP request always sync
So the game was always paused for like 0.25 seconds with every request. Super jarring in VR.
sonic ivy
Yikes
soft compass
So I solved this with good old C# Threading. But man was it god damn nightmare to get it working
Because Unity takes ownership of everything that is Unity types
So if you try and pass a Unity object on to a C# thread
Unity says "Nope, access violation"
The way to get around it, was to take the Unity object and split it into purely system types in my own classes and then pass a new object of that class to a thread
sonic ivy
Right. But theoretically, I'm accessing these pointers so that I can pass my AR Session directly to ASA.dll from ARKit or ARCore, skipping Unity completely. In theory, that's what should be happening.
soft compass
You are still using ARF though, no? So you are using Unity :U
sonic ivy
Yeah, but ARFoundation is now pretty not black-boxed. The new XRSubsystem abstraction layer is great. I think that the most likely thing is that the actual UnityARKitPlugin you download and the one compiled in the Package Manager might just fundamentally work differently and return different data. Android doesn't seem to mind, probably since ARCore is a uniform compiled plugin where the ARKit plugin relies heavily on Unity's objective c bridge
soft compass
Right
Yeah I suggested they might just be fundamentally different
further up
Could sound like that's the case
sonic ivy
Yeah, just not much I can do about it 😕
sonic ivy
The problem is that I can't find an ARCore LWRP integration for the life of me
sonic ivy
ARCameraBackground needs to implement IBeforeRender and some other stuff
But it's confusing as hell because the API breaks every time they change LWRP, most people are on 5.6.1 for ARFoundation even though 6.x is out
soft compass
A friend of mine told me to just not rely on those packages until they stop being "preview".
sonic ivy
I've also considered that we could use the old ARFoundation with ARExtensions, but then we've got no image target support for the fucking Magfic Leap 😛
soft compass
For production any way
sonic ivy
On the LWRP side, they're out of prevew
sonic ivy
Not the new one, no
But this is AR development, my choices are either use the previews and betas or don't make apps
soft compass
Oh I get you
It's just that it's gonna be an issue for a bit longer, I reckon
I wanna do AR stuff too
I don't have much under the belt yet
sonic ivy
ARKit/Core means rolling back a week of progress and writing everything and debugging twice, plus no LWRP support
soft compass
And I reaaaaaaaally wanna get my mittens on a Magic Leap...but I'm not American nor have 2300 dollars ._.
sonic ivy
ARFoundation means everything is great, code it once, but Spatial Anchors crashes
Wait for the Hololens 2, and get it on credit. ML1 is a cool toy but as a develop it's hot garbage
soft compass
hm
I'd just like affordable AR Wearables for developers, so that when consumer version comes around I'm ready
sonic ivy
Wait for NReal, $500 at the end of this year, runs Android
soft compass
That's still 6 months
That's 6 months of me being uncompetitive
Lol
The hardware is not nearly as important as the experience of developing for wearable tech
sonic ivy
Exactly
soft compass
The tech will change the year after any way
So getting the tech now
Is better
Than wait 6 months
sonic ivy
That's the plan. But I've dev'd for almost every AR headset. And you spend SO much time on stupid bullshit
For the Magic Leap– every time you press the bumper button, it calls OnBumperOn and OnBumperOff. For every on AND off. And they know about it, and it's been a bug in their SDK for two months.
soft compass
Well
Again like
I get that's it frustrating
But I still have no hardware to even attempt it on
sonic ivy
Their input pointer doesn't actually land on the canvas, it scales out depending on how far you are from the canvas
Supremely hot garbage
soft compass
I'd go with hand gesture recognition instead of controllers any way
sonic ivy
No I get it. I didn't buy mine. My cofounders all came from ML, we got the hookup 😃
The controller is surprisingly good. I'm making an AR laser tag game. Biggest concern with porting to hololens is makig a gun game that doesn't feel suck.
soft compass
=_=
gibe moni ;_;
I mean
Can you make lasertag not suck?
You can make AR Laser tag not suck
sonic ivy
If I can get ASA to work with ARF, then yes
It already feels great, but the calibration with an image target was horrible
We have IK so you can see everyone else as robots and it works insanely well
soft compass
perhaps it's just me who suck ¯_(ツ)_/¯
sonic ivy
ARCore Cloud Anchors are basically the same as Azure's, I would have gone that way but back to the damn LWRP problem
We use the anchor abstractly -- users can't see it. It's just for localization
sonic ivy
We just finished this project: https://www.ft.com/content/1fcbac14-49d4-11e9-8b7f-d49067e0f50d
20 minute volumetric piece with 120 networked Magic Leaps. But it was all synced with an image target. Yuck.
sonic ivy
I'm just so tired of gluing janky shit together
😛
Magic Leap dev is a million times better than Meta dev, though
soft compass
It's still new
I'd kinda expect it
AR in its current form is from like
2016 or something
I'm writing my masters so had to go over the history of AR
😛
I wanna reach a point where I can turn a city into an Arcade, using AR
sonic ivy
Yeah I mean, worldscale is the idea. Google and Azure are getting close, the cloud anchor roadmap for both of them is much more ambitious
The roadmap is: you open your app and know exactly where you are in the world
soft compass
Well
When wearable AR becomes normal
I imagine we'll optimize hardware for prolonged AR sessions
Google should be working on their spatial map service I think
I had hoped they'd reveal it at I/O 19
But they didn't
From what I saw at least
sonic ivy
They are. Google Cloud Anchors are actually pretty great, but not by themselves, you really need to store them with meta data as a cloud service for world mapping. And there are a lot of big data problems -- every coordinate is essentially going to be relative to every other one until a service that establishes an absolute position is built, and the over way I can imagine that is essentially with GPS and lots of averaging
soft compass
That's true
But
Also they wanna make sure they can't locate you with the sparse map they actually store
That doesn't help either really
It likely means that two people could scan two gravel parking lots in two different places in the world
And see the same AR experience
sonic ivy
Well, with the meta data, I can tie your sparse map to your relative position in the world, and then split you up into sharded servers or databases from there so I don't need to query my whole server and db every time I want to pull something, and it should avoid a lot of that confusion
If I know where you are I can give you all nearby spatial anchors so you only search for those
soft compass
Well as long as you can't locate me, I think is the point
From the sparse data alone
sonic ivy
I mean, I certainly could write that service right now, but it'd only be as accurate as the averaged GPS coordinates of every user who had picked up that marker
soft compass
Yes.
If we didn't care about ethics it would be easy to just store an almost literal coordinate
But I'm happy we do care
Lol
sonic ivy
less than three
soft compass
But could you, from that data, locate where I am in the world?
soft compass
Could you recreate an actual physical location of where I did that map?
sonic ivy
So then I just grab any spatial anchors within, say, 150 feet of you, and start scanning. And when I find one, I take your GPS, and compare it to anyone else who's seen that marker.
soft compass
The point is, for spatial mapping services to work in a way that doesn't require massive cultural shifts
It should be possible to store sparse maps
Without being able to use it to backtrack where I was in the world
sonic ivy
It totally is. But the only people with enough maps to map enough space that a relative coordinate system could resolve absolutely are Microsoft, Google, Apple, Amazon and Facebook
But literally you can already be tracked down to the street level inside of an Unity app today, this is fairly common. See Pokemon Go
soft compass
I know
sonic ivy
And by definition, Niantic knows what street every player is on
soft compass
But Pokemon GO did not try and spatially map out the world. They literally just used google maps.
Knowing where I am vs Knowing exactly what my surroundings are, are two very different things.
sonic ivy
I don't understand the difference. I don't need a spatial DB, I just need a DB I can query for distance from other things
Right but I can use one to dramatically reduce the number of possibilities for where you are down to a near absolute point
By tying them to GPS coordinates and doing relative distance queries. You see?
Also, my anchors are placed automatically and you can't see them
soft compass
But that's the thing. Do you even need a coordinate for spatial mapping?
sonic ivy
Not if it's relative., But with enoouogh GPS data, it's a near absolute value
And if you have enough maps you can start matching them to build a whole relative world on top with a single offset value
soft compass
The sparse maps could be tied to an arbitrary space then, which a machine can map, but which shouldn't give you any indication on where in the real world that is.
I don't know. I'd have to get more into that section of tech to understand it I guess.
Right now, I'm thinking about a medical app, of sorts
That would need wearable ar glasses to work efficiently.
sonic ivy
You can localize an arbitrary space with enough GPS readings to be an absolute latlong coordinate
And as long as you have one of your anchors tied to an absolute, and and it knows it's relative distance from the next anchor and so on, you have a mesh of absolute space
soft compass
An app to aid those who suffer from Face Blindness
sonic ivy
story of my life
soft compass
Well, it's a real medical condition where people can't recognize faces.
sonic ivy
Too much time spent staring at computers and not enough at people's faces
Propognasia, I know it well
soft compass
Yea
sonic ivy
But it's definitely not a coincidence that computer nerds tend toward it more
It's like if you study C# all day every day you get sick at it, other people study faces all day
sonic ivy
I think there's probably an autism predisposition (which I definitely have) but I also think that it's a learned intelligence. Nature and nurture
You've done more research, you're probably right. But I have had so many awkward encounters with people I don't actually know but think I do or actually know but don't recognize
soft compass
But it has nothing to do with you spending a lot of time on a computer. It's much more likely you might be on the spectrum somewhere for this condition.
sonic ivy
It's tough when you're intelligent and social and have a girlfriend so you seem normal, but when you get down to autism on a page and you check all the boxes.
soft compass
I could imagine.
sonic ivy
Helpful disposition for networking code 😃
soft compass
heh
sonic ivy
I've literally thought about this as a practical life solution
soft compass
There are many people who suffers from this who develop their own mnenomic devices 😃
sonic ivy
You could pull from Facebook and Instagram, so you already have a decent starting database
soft compass
Oh I would make it much less invasive than that
I understand the appeal of using Facebook/Instagram for it
But I'd much rather have it be a local database thing on the users computer to save on ethical questions.
Training a network to recognize faces can't be hard nowadays. With the Tensorflow Lite I saw at I/O that should be a peach
Can even run offline then
er
Not computer
phone
sonic ivy
You would want to cache it locally
soft compass
I'd just wanna make it as little invasive as possible through social media platforms.
sonic ivy
You're talking < 1000 image dataset for most people, you could run that in OpenCV in unity at 60fps on an iPhone
Social media integration is just for database population
soft compass
I get it
But again it's a matter of ethics I guess. We see differently on this it sounds like, but that's okay 😃
sonic ivy
I'm working on an SDK called LifeScope with a friend that is an open source social graph so you could host your own server for this
What I'm saying is that you would pull from those platforms and keep it locally. Scrape them., Don't use their services or APIs after that
sonic ivy
With LifeScope we're building a login auth system for users so they can create their account with Facebook, which populates their friendlist, but then it never talks to Facebook again, it builds you a new profile on an open source platform that the app developer can control
soft compass
And the thing is that this app can be used by people who don't suffer from face blindness and I don't know what the implications of that are, yet.
sonic ivy
But Facebook has all the user data, and asking for user to enter their manually is a massive friction
So you deal with the devil to get what you need
Our hope is that devs who don't care about data sovereignty just get a nice login and ELO system out of it
soft compass
I guess from my user interaction when making software, some of the first questions that come up are "What about my privacy?" or "I don't wanna be surveilled" or "Facebook shouldn't leak my data!"
sonic ivy
But that most people can start getting off the "log in with facebok!"
soft compass
I know that it's fear mongering that made people say that
But you still need to make people trust your app won't be malicious
Not having it interact with any social media or the internet will help prove that case
Hopefully you see what I mean
sonic ivy
I do, but the reality is that people trust Google and Facebook with their data a lot, which is why those two companies can even make a play in this space. Lifescope is the first thing I've seen that could open source a spatial graph, and we're going to talk to MongoDB about a new spatial database for storing information in the sway that you'd imagine
sonic ivy
The big problem is users
soft compass
As long as it's not one of those "There are 12 standards, we need to unify!" now there are 13 standards.
sonic ivy
Users hate facebook but they really like facebook. Literally account creation is the second step after app download where you lose almost all users
It's really more just about have a flexible object containering system that can create and destroy server instances when and where they are needed. Which means Azure, Amazon, or Google Cloud for hosting no matter what. And since Apple doesn't trust Microsoft who doesn't trust Google, everyone's getting the raw image out of ARKit or Core and doing their mapping algorithm on it. Which is why I'm in the shit I'm in right now with ARFoundation
So at best we're creating a system to host a social graph and collect data about you just like Facebook, but you can see and permanently remove data as you want, you can delete data as you want, you can store your data on Lifescope's server or spin up your own and host graphs just for your users, whatever you want. It's not a unifying solution, but we don't necessarily need one
soft compass
This sounds an awful lot like that service which tried to replace twitter
sonic ivy
But the reality is that there are some really great social media platforms out there and nobody uses them because nobody uses them, and login and auth have the same problem. Because Facebook's login is a strong point of trust,
Yes, exactly
You can't sell people on that angle
soft compass
If I am not mistaken
sonic ivy
You can sell them on a "download this SDK to have login, auth, user storage and verification all rock solid handled for you on open source that you can control"
soft compass
It appeared to me, when I tried out those kinds of services, that most of the people who use mainstream social media and the likes
sonic ivy
Absolutely
soft compass
So because of this
I have a hard time seeing why your solution should be any different.
Because it sounds like it has the exact same problem
sonic ivy
Well, we're not trying to build a social media platform. We're trying to build a game with ranking of our layers
And in an AR context, we're really just trying to know where our players are
Or, in the case of a facial recognition app, we really just want the user to havre a frictionless experience -- oen the app and it works. Lots of set up and they'll just uninistall it
soft compass
Just a lot of mistrust flying around
sonic ivy
So if you download my app, log in with Facebook or Google -- I've just connected and gotten the relevant info to populate a new profile, then Facebook never gets used again. And it'll log you in with anything. You can create a new profile with your github
soft compass
Like you said, it's kind of a love/hate relationship
sonic ivy
At some point, someone is going to need to track user data in a centralized way. We give all of our info away for free because we get an incredible experience out of it
So if we build our own system it'll only work if we aggregate existing data, due to user psychology
soft compass
Eeeh...
We gave all our information away because consumers wanted free
And now we can't go back
sonic ivy
We gave it away because it meant connecting with long lost friends. Facebook is scary, but it's the 9th wonder of the world
soft compass
I'm sorry, no. That's simply not true.
The platform Facebook was something very different when it started out. It got popular super fast and to mitigate this sudden demand in servers, they came up with the idea of ads and minning users for data because consumers did not want to pay for Facebook.
That's where we are today
Same with YouTube, Twitch and all the others
sonic ivy
I live in New York because I connected with a long lost friend on Facebook and we started dating. That wouldn't have happened without the platform or similar. That's what I'm saying
soft compass
Sure, that's anecdotal of how it helped you. I'm glad. Honestly. Not gonna try and invalidate that.
But that's not the nature of why we gave up our data freely for this platform
sonic ivy
No, we gave it up freely because we didn't realize what we were giving it, or how it was being used, but fundamentally the reward of giving it up still outweighs the risk for 90% oof people
soft compass
But, sorry, I'm side-tracking this convo.
A lot of people don't trust Facebook. Or Google. Or similar.
But some do.
sonic ivy
But the point is that you really need a company that big to pull off a system as advanced as cloud anchors
soft compass
And honestly, I'd rather start with a face recog app that doesn't use Facebook, than one that does.
sonic ivy
Using a Facebook API to recognize a face is very different than pulling a database of faces from a user's friendlist
soft compass
You are still tapping into Facebook. Nothing is free.
sonic ivy
You're talking one API connection call to tell Facebook you're connected to the app
And you can destroy your token after that
soft compass
Yeah and then what happens when the faces needs updating?
Better check the old facebook again
sonic ivy
Well, ideally you're running an algorithm locally that adds new faces
soft compass
Adds new faces how?
I am only interested in faces of people that the user knows of
Not random people
sonic ivy
OpenCV. Just recognize faces and capture them
Yeah. If someone is looking at you and engaged in conversation, photograph them
sonic ivy
We already got over that hurdle with Snap specs. It's not 2011 any more
People are okay with being photographed for assistive devices. I mean, microsoft has glasses for blind people that do all this
soft compass
The Snap Specs never caught on from what I saw. And few liked them at release, no?
soft compass
There is a social aspect to that
People would be more okay with a blind person than one who can see
I've never seen anyone with the snap specs though. Whether I traveled or was in Denmark.
Could just be coincidence I guess
sonic ivy
It was mostly a new York / LA thing
sonic ivy
I mean for most of the launch you had to buy them at a vending machine in LA
soft compass
Not "widely accepted"
sonic ivy
They were widely accepted culturally. there was no "glasshole" backlash
sonic ivy
I mean, you could say the same about homosexuality. It's America, we're VERY diverse here.
soft compass
I just wanna understand where these glasses took off because I never heard of them after launch
They just faded away
sonic ivy
They didn't take off in that they ended up overmanufacturing them and losing a ton of money
soft compass
That's not what I said either
sonic ivy
But there was a noted cultural shift in how people feel about being photographed
sonic ivy
No. But the idea of being photographed in public has been normalized
sonic ivy
And by dumb I mean hateful and bigoted, obviously
What I mean is that the actual cultural positioning toward what a camera is and means has changed even in the last 5 years quite a lot
sonic ivy
And that a lot of things we took for creepy -- facial recognition verification at ATM machines, for instance-- is totally normalized and invisible
But also that world tracking cameras, etc, are basically fair game like IoT devices
soft compass
Well, most people don't being surveilled. Just because it's there doesn't mean that people have accepted it. It's been pushed on them.
sonic ivy
I can't point to a specific piece of research that says "humans aren't freaked out by cameras on your face" but I was really pointing to the difference between Google Glass and Snap, culturally
Most human beings outside of LA, SF and NY have never seen either product
But most think glass is creepy and snap isn't
soft compass
There are some layers there.
Google is creepy
So Google Glass is creepy
by extension of being from Google
Snap is that weird service your grandchild told you off for "selfies"
sonic ivy
I don't think SNAP is any less creepy, they were holding onto everyone's photos for years and there was a scandal, since it's supposed to be ephemeral messages
sonic ivy
Because kids don't acually care
soft compass
They are not told about it until they grow up
sonic ivy
As long as their friends aren't hacker enough to get their noods
sonic ivy
By the way, Facebook knows we're having this conversation, Discord has their tracker installed 😄
sonic ivy
They know almost everything. They buy our purchasing history from every large store we go to
soft compass
Amazon knows what goes in Blizzard voice chats
sonic ivy
They're probably running alexa algorithms on it
soft compass
And now Discord knows what goes on in Steam voice chats, if I am not mistaken
sonic ivy
I can't wait for the google voice assistant that sounds like an asshole xbox gamer
soft compass
Oh actually
sonic ivy
using machine learning, of course
soft compass
During my 3rd semester at university
We had to play with technology
One of the things we did was voice recog
So one of the suggestions for a project was a mansplaining dildo
xD
I ended up making a voice assistant which asked you questions instead because it was tired of being a passive servant to every one of your stupid questions
sonic ivy
That's pretty good
I've been generating all my announcer sounds with Google's text to speech
Sounds very good
soft compass
I also did Street Preacher VR, which was that Voice Recog + VR project I mentioned earlier
You are a crazy person in a park on a soapbox
In one hand is a megaphone, in the other a sign that says "The end is near" drawn very poorly
Then we run voice recognition on what the users are saying to look for keywords that might make people turn heads towards them and join them at the soap box
Who says academia can't be fun?
sonic ivy
google voice announcer
soft compass
Yeah I'd only use that because Microsoft Sam is technically copyrighted.
sonic ivy
Plus some audio processing to make it sound sci fi of course
I've actually recorded a lot of voice actors, and using DeepMind's voice stuff is actually just better in most cases
soft compass
It depends on your usecase
It's easy to cover up synthetic artefacts with a bit of sound editing.
If you just need precise line readings
sonic ivy
I used a bit of it when the default timing or inflection was weird
Nope
I mean, yes, but I've played it back for people and they didn;t know it was synthetic
I've been asked who did the voice acting for most of it
soft compass
Well I mean, the synthetic voice won't have any of the flaws that humans have while speaking. Which kind of gives it away :U
sonic ivy
It all sounds like cheesy explainer video voice, and if you just ran a book through it you'd get all kinds of inflection problems
Did you do the captcha?
sonic ivy
For voice language localization, it's the only viable way for an indie dev to do announcer voices IMO
Strange
soft compass
Well
You can find thirsty people
But I get what you are saying
And when I mean thirsty, I don't mean to exploit.
But for reasonably low rates
sonic ivy
Yeah, but actually getting the right voice is the hardest damn thing
soft compass
It is
But it gives so much more, imo
I've have to cast voice acting talent myself
I think I went through...7-8 different people for a single role
And +10 for two other roles
Listening to the same take over and over is hard. But when you finally find that one voice...it's so good. Much better than what synthetic services can currently deliver. imo
sonic ivy
Wait til you've heard the Wavenet stuff 😉
soft compass
If only I could
:b
The circle just goes around inside the button
Nothing is played
Nope
Firefox, Brave, Chrome, Edge (lul)
None of them plays the sound
I see no console requests gone wrong
It just doesn't play
sonic ivy
very weird
soft compass
And I know my browsers sound works
I'm listening to YouTube
My first AR experiment btw
Originally "hot and cold" game looking for treasure. The assignment was to put the phone on a selfie stick to explore how that transforms how people interact with the device naturally
We made a second version where you catch butterflies
So the selfie stick could be a metal detector in one game and a butterfly catching net in another
Pretty cool what such a simple extension tool can do for modes of interaction
soft compass
Didn't you already post this?
sonic ivy
Got some help from the Unity Gods. Not enough to get going, but enough to show hope. The ARFrame pointer that ARF gets is actually a pointer to a struct containing a version number and the actual pointer. So I have to literally do (IntPtr) arSession.subsystem.GetNativePtr() + 4
Or get the pointer, cast it to an object, get the pointer from that object 😃
I don't think ARKit will work, there are some problems, but if we can get ARF working on Android (since ARCore is a black box, so it's a uniform API) then we can hold fast until the devs catch up
bronze silo
@soft compass https://pastebin.com/raw/L9iQq1Df
soft compass
okay?
bronze silo
open your mind dawg
soft compass
I do not understand why you tagged me for what is in this file @bronze silo
It's about a slideshow for what AR can bring a company.
Cool.
I'd love to try the platform
But I am not sure why I was tagged for the slideshow outline
sonic ivy
@bronze silo yeah, doing azure spatial
@bronze silo there are a bunch of great pose solutions now. Check out OpenPose, there's a Unity sdk. Tf-pose could probably be adapted to run on lightweight devices through unity also
drowsy oracle
Hi everyone. When building my LWRP + ARCore app and running on the device I just get a black background instead of the camera feed.
Please don't tell me ARCore does not work with LWRP?
drowsy oracle
Well it seems it doesn't...
latent thunder
Anyone have thoughts on the MagicLeap AR platform? With competition from Microsoft and soon, hopefully Apple, AR headset market is increasingly competitive.
soft compass
@drowsy oracle There was someone who posted a "fix" for that
Somewhere
But I can't for the life of me remember where
@drowsy oracle
drowsy oracle
Thanks but that's ARFoundation
soft compass
But maybe it could help you with ARCore
drowsy oracle
I got that working but I was previously working with "true" ARCore
Maybe yes. But for now ARFoundation does what I need
I feel like Unity really needs to step up the game with all the AR stuff and rendering pipelines
soft compass
I think that is all they are focusing on right now, really
From what I've seen
A lot of focus on AR
That and ECS
drowsy oracle
True. But somehow it takes forever to get proper LWRP support
soft compass
It's a relatively new way of doing the pipeline
drowsy oracle
But I understand they have lots of stuff to do
soft compass
You'll likely have to wait some before using it for production
Unless you know what you are doing
drowsy oracle
I never know what I am doing 😀. My main reason to with LWRP was so I can use shader graph which works great
Thanks for the help!
soft compass
np
latent thunder
Trying to understand object tracking. If you have a 3d model of a real world building can you somehow overlay that model onto its real world counterpart?
drowsy oracle
Well yes you can. Depends on how good your tracking solution is. Also to track a building you're gonna have to stand pretty far away from it.
You could try and track only a specific part of the building
latent thunder
Hey guys I’m interested in exploring large scale tracking. Are there any Unity Arkit demos or tutorials to get me started? For an example of what I want to accomplish See the architecture demo here https://www.wikitude.com/blog-wikitude-large-scale-3d-tracking/
latent thunder
Seriously? Isn't this handled somehow thru object tracking or point cloud mapping with Arkit? I'd be surprised if it wasn't.
dry sierra
@latent thunder Placenote SDK will do it for you 😉
soft compass
@latent thunder You might be able to map a building with point cloud mapping, but ARCore and ARKit won't know what a building is. ML can help you identify buildings.
latent thunder
Hi
I need a user to precisely rotate a placed 3d object.
Options I am aware of include
1- voice commands (not accurate enough)
2- finger gestures, movements (ditto)
3- slider in gui
4- using the phone itself like a steering wheel and getting gyro feedback
I am leaning towards the last option.
Any other ideas of feedback?
wide steeple
LeanTouch - take a look, it was pretty usable for this task when I looked
@latent thunder
wide steeple
Slow?
soft compass
@wide steeple Yeah try and check out LeanTouch in the profiler 😃
wide steeple
OK) I used it for all of scale, rotate and translate and didn't notice any perf issues
latent thunder
Anyone play with latest ARFoundation 2.1? Having issue with Environment probes.
The sudden jumping of the maps instead of a smooth transition.
Another user mentioned it here as well
https://forum.unity.com/threads/arkit-glitch-with-environment-probe-texture.580537/
Also, does anyone know if they be manually placed and if so, what would be the advantages over letting Arkit do it automatically?
modern sparrow
Is this a shadow plane that changes intensity through animation?
Do you think the shadow plane is somehow generated or it's manually drawn for each model?
soft compass
@summer quest Make it so that if I press any of the elements that comes out about the phones specs, those elements come close to my screen
3D interfaces suck because you have to move close to them to read stuff like the text you have there
You need to provide accessibility options so that if I press the text box, it'll come closer to my screen
It's especially necessary because the text is so small
summer quest
You mean to make the canvas screen space from world space?
soft compass
Either that
Or enlarge it so that even from a distance
I can read it easily
3D effects are cool for AR effects
But for information it sucks because you need to apply real world distance
summer quest
Yeah ideed
soft compass
If you want me to read a sign from across the street you'd also make the font big enough that I could read it without going to your store
You follow?
summer quest
Yes
soft compass
Same principle here. Information needs to be easily accessible and readable.
😃
Especially since you are working with products
summer quest
Wouldn't be better if I add only the important stuff like I did on the left side?
soft compass
If you pressed the specs text for example, perhaps the box will expand to take up all the horizontal space you are working with while everything else moves out of the way.
Then enlarge what is in that area so it's easier to read or make the area itself larger as well as increasing font size.
Then if a person wants to go back they either swipe the text away (a right swipe for example) or press some back button
summer quest
And maybe add a button to show the entire spec list in 2d
soft compass
You could do that too.
There are options
If you need to pack a lot of information somewhere, make it super easy to read.
If you are going for a sales pitch, list a few "buzz" words.
Like
- 2.93 GHz Snapdragon 845
- 16 GB RAM
-256 GB Storage - Double Sim
summer quest
Yeah, I'm still learning these this
soft compass
I'm doing my Master's Thesis atm with focus on AR :3
So I've spent a lot of time looking into best practices and whatnot xD
summer quest
🤬😮
I will finish my master next year
And I want to make a thesis about AR too
So until then I have to practice a lot
soft compass
My master's will concern itself mostly about physical space and AR.
What it means to use it and what assumptions are made about the technology
As most thesis writing and papers focus more on AR capabilities and implementations than it does space
burnt root
Trying out ARFoundation, can't get raycasting for planes to return any results on an android build.... followed this tutorial: https://www.youtube.com/watch?v=Ml2UakwRxjk
I tried connecting the profiler and console and it's not raising any errors
var hits = new List<ARRaycastHit>();
sessionOrigin.Raycast(Camera.main.ViewportToScreenPoint(new Vector3(.5f, .5f)), hits, TrackableType.Planes);
hits is always empty
wide steeple
Check your collision matrix and layers
Also try adding mesh collider to detected planes (that is for ARCore though)
strong quiver
@burnt root try changing TrackableType.Planes to something else. Personally I am useing TrackableType.PlaneWithinPolygon and have not found a hitch with it yet.
burnt root
I was able to solve the problem by adding a PlaneManager component to the scene, which was apparently a critical step that the tutorial skipped over, lol
strong quiver
latent thunder
Hi. Anyone play with Microsoft's mixed reality SDK? https://github.com/microsoft/MRLightingTools-Unity
Just wndering, would this work n mobile or only the Hololens?
latent thunder
Feature request - sensor recording! https://www.ittybittyapps.com/blog/posts/2018/11/recording-arkit-sessions/
gritty kettle
@latent thunder I'm not sure, but afaik, peopel are only using MRTK for hololense, and the rest of the Windows Mixed Reality line, which is other AR like the lenovo mirage, and and VR headsets.
It builds out to the Universal Windows Platform, so anything that uses that, y'know?
latent thunder
Exploring Environment Probes and Shadergraph. https://vimeo.com/338537753
summer quest
arkit or arcore?
jolly tiger
Hi! I played a little with ARCore Augmented Faces which is kinda new :) so I thought some of you might be interested in looking at some of my tests to see how it works! I made a video (which is in french but I added subtitles!)
If you have questions do not hesitate :) (also tag me in that case :p)
https://www.youtube.com/watch?v=gbKcdNEvuGc
limpid storm
tropic cedar
@jolly tiger have you played with snapchat lens studio and facebook/instagram ar studio by chance?
jolly tiger
no I haven't, do you know if there is a Unity SDK for those ? :o
soft compass
There is not if I am not mistaken
It's like a studio they made themselves
At least for Snapchat right?
I say studio, I meant suite
tropic cedar
yes it's a whole different system for each thingie, but i have a feeling the things you're working on would be at home on those (there are examples for each platform that you can try out and modify if you want)
jolly tiger
oh okay! I might try one of the studios some day when I have time, looks pretty fun 😁
copper socket
Has raycasting changed for AR Foundations in 2019?
arOrigin.Raycast(screenCenter, hits, TrackableType.Planes);
Is throwing errors on calling Raycast
severe spear
@copper socket yes - there's a ARRaycastManager now that has these calls, see https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@2.0/manual/migration-guide.html
copper socket
Ah, thank you! ❤
latent thunder
Motion Tracking
People Occlusion
Anyone have more info 2 share on these 2 new features of AR-Kit announced during the WWDC keynote?
Do they require depth sensing capabilities of iPhoneX?
Is there an ETA for their support in Unity?
sonic ivy
@latent thunder -- these will use the back camera and probably won't rely on any technology not contained in the iPhone XR which is single lensed. Once you've got the beta you can start building the plugin into Unity, chances are the UnityARKit sample repo will get updated a 4-8 weeks after the code is available, and if you want you can probably write hooks for the new stuff and push a PR yourself
I would expect much longer for ARFoundation support, though, especially until Google announces anything. Since you're using a native plugin bridge, theoretically anything they announce is immediately available in unity if you can write the bridge 😃
copper socket
Speaking of ARFoundation, just so I'm sure, there is currently no image tracking available for it and that it's currently still in development? In that, if we wanted to just spawn something on a tracker, we would still have to do it either in ARCore or ARKit depending on the platform?
And if so, is there much difference in how that's handled? Or could we build for Android for the sake of speed/ease, then relatively easily port it to iOS?
modern sparrow
@copper socket There is a preview package
I haven't tested it yet though so can't help
Did anyone try to use AR Remoting? I'm having trouble connecting to my phone ->
SocketException in StartListening: No connection could be made because the target machine actively refused it.
Sorry, one last question about AR. I thought I recalled seeing some demo where models in the world respected the space around them, meaning people could walk in front/behind it rather than being super imposed into the world like traditional AR.
Was I wrong about that, or is that a thing? If so, is there a term for that sort of feature?
modern sparrow
that's feature of new arkit 3 which was announced yesterday 😄
you're lookgin for "occlusion" I guess
copper socket
Wow. Talk about perfect timing haha.
We're thinking of doing some skeletal renderings for a client project, but it sort of ruins the experience if it's not occluded (it's a multi person experience).
Thanks for the info!
copper socket
Wait, will this update be immediately available on Unity?
bronze silo
naw
only human occlusion
not object occlusion
they use pose estimation to get the depth position of you vs the object
and occlude
they dont have it for objects like a table and a 3d object behind it
my body is ready for markerless motion capture
live actor animations replayed with a 3d model in the same spatial positions saved to a map that can be loaded at all times
live theatrical performances
of 3d models in real life places
unborn atlas
Just when we thought ARFoundation was the way forward for all our projects, ARKit3 mixes things up a bit. Looks like it's a lot of fun to play around with. This guy is using the pose estimation on a TV https://twitter.com/albtaiuti/status/1135812065053085696
latent thunder
So far have used target and planes to place my AR objects. I have seen demos where a user was able to place an object in a fixed position as though it were floating in the air.
How is this accomplished and is there any demo I can reference?
soft compass
Well it depends
One way of doing it is to decide length from camera somehow, then decide how far the object is from the nearest plane in the Vector3.down direction
And then place it relative to that
Is a guess
latent thunder
soft compass
Huh.
Cool
I still don't have an iPhone
Lol
Not that I want one but...for making apps it's probably something I should get
modern sparrow
Hey does someone have vuforia 7.1.35 unity installer? ;d
I need to make some changes to an old project and I'm using latest release which doesn't work and I don't really have time to fix vuforia...
And vuforia doesn't have archive for older releases 😦
copper socket
It's great, because we are literally starting a quick client AR project.
And were hoping that this tech existed (I've never used AR before now).
So, thanks Apple / Unity ❤
floral summit
@soft compass just remember that you need the mac too 😛 Its not like android where you can just dive right in
soft compass
Oh I have a mac already sitting around
That I borrowed from work and they never requested back when they laid me off
:U
Though I'm wondering how long that thing will last. I'm sure Apple will upgrade to some bullshit OSX and make the hardware artificially obsolete
gritty sleet
Anyone tried building to iOS through virtual machine hosting services?
soft compass
Yes
Kind of
I rented a Mac computer once online to do it, which was a VM as far as I know
It is doable but
I had a lot of trouble because I had no admin rights
So it was super bothersome to get anything installed
But other than that, if it's recognised as a legitimate Mac VM then you should have no issues otherwise
gritty sleet
@soft compass Thanks
shell python
hey guys
I'm trying to use the following asset https://assetstore.unity.com/packages/vfx/shaders/starfield-shaders-136597 with vuforia but the stars won't render around me
is there any special configuration I need to do or something? the forums are good for nothing, already tried there
like, any reason why the AR camera wouldn't render something? I'm using the ground plane example you can get from the vuforia core samples https://assetstore.unity.com/packages/templates/packs/vuforia-core-samples-99026 . And I realize stuff only render when you detect a plane and tap to place something on the plane, but even doing that the starfield won't render.
frozen bronze
Hi there
i got problem with the rendering some stuffs inside unity.
here is the setup
unity 2019.20b1
LWRP 5.6.1
Shader graph 5.6.1
ARFoundation 2.2.0
I have made the game in AR before in 2018.3 and i have migrated it to 2019.20b1 so this is giving me some strange result in the rendering.. later on i have tried the basic setup and i guess the problem is with renderer it self for the mobile device.
in the editor play mode the entities looks clean.
but as soon as i build it for android plate form it sticks with each other..
Editor
mobile device.
does anyone have the same problem?
frozen bronze
update on above question.. i have took one more project and enable multithreaded rendering there.. it does render separately no more sticking together things. now there is some problem with child object they are sticking with each other.
frozen bronze
or may be animated object sticking together.. who knows but it certainly sticks together...i think the problem is related to multithreaded rendering ...
copper socket
Does anyone know of any good tutorials/guides for spawning on object from image recognition with ARFoundation? Or does that still have to be done via ARCore and ARKit, depending on the platform?
I'm still a little confused as to which package is used for which feature. In this case, we just need to spawn something based on a tracker.
soft compass
ARFoundation is a unified way to use either iOS or Android
So you can have one code-base for two platforms
In terms of Image recognition though, from what I remember, that is decided by the session you use (if ARCore at least)
Some sort of Session file has to exist that tells ARCore what kind of ARSession you are doing
I don't know if ARKit needs the same
You should download the ARFoundation package from the package manager
Then depending on whether you develop for ARCore or ARKit or both
You download the appropriate package from the manager as well
copper socket
Gotcha, thanks for the info.
shell python
anyone who can lend me a hand?
soft compass
Rather ask what you need to ask instead of asking if someone can help
shell python
I did if you scroll up a little
I just don't want to spam since it's against the rules
last and only person to post on june 11
shell python
ok forget about my previous question on june 11
I need to make a ground plane experience in a dark environment. How can I do this, if when vuforia loses the anchor/tracking it wipes everything from the screen?
I want a way to make an experience where I turn on the lights once to place the object, then I can turn off the lights that even if the anchor/tracking whatever is lost the object will remain there.
prisma elbow
Has anyone been to able to LWRP with ARfoundation after it came out of preview?
frozen bronze
@soft compass
Turning off Multithreaded rendering doesn't seems to be working .. they are still sticks with each other. but i found out one more thing that we need to add vulkun graphics api to render it separately so it's working when there is no AR going on ...but in AR it doesn't seems to be working they are still sticks with each other.
copper socket
Has anyone experienced a black camera feed when building a basic empty AR project with the latest AR Foundations?
Basic in that there's only an AR Session and AR Session Origin
hybrid perch
How dark is your environment @shell python? For AR tracking to work it needs to see the textures of the surfaces around you. Vuforia’s fallback is to hide everything when tracking is lost. If you use AR Foundation I think they don’t disappear but if tracking is lost they might get ‘glued’ to the camera or drift around
copper socket
Ah, it's probably LWRP related. Removing the package works. I guess they'r enot really compatible at the moment?
hybrid perch
I haven’t tried that yet but I’d think it should work
No actually I have, and it worked for me. Have you tried the example scenes?
copper socket
No, I just ran my own. But it was completely bare bones.
I read online that you have to set a custom background on the AR Camera, by using a RenderPipelineBackground asset (or something named similar), but there was no option to create one.
Really, we don't need LWRP for this project. We were just interested in trying ShaderGraph, but I think we're just going to fall back to the standard renderer and continue using Amplify for now.
latent thunder
Anyone playing with latest ARFoundation 2.2 +ARKit3? Having issues with Depth mode.
First off, Stencil mode seems to compile and work fine at all resolutions.
But if I disable Stencil Mode and test with only Depth Mode I get the following>
Full Screen Resolution > crash on launch
Half Screen Resolution > crash on launch
Standard Resolution > doesn't crash however the RawImage is showing only white.
and see the following repeated message in Xcode console
2019-06-13 11:30:28.215227-0400 StandardDepth[2189:83955] [] nw_connection_receive_internal_block_invoke [C8] Receive reply failed with error "Operation canceled"
Is the above expected behaviour? I assumed I should be seeing some type of depth image as implied from the WWDC presentation. (attached)
copper socket
I haven't, but we were hoping to use it for our current project. 😩
shell python
@hybrid perch the darker the better, it's an experience that is thematically dark
so is there any place where I can change this behavior of hiding everything?
oh I'd use AR Foundation if I could
copper socket
I'm really struggling with understanding image tracking in ARFoundation.
I downloaded the ARFoundation examples and ran their example project and it seemed to work well. However, I attempted to replace the prefab with a simple 1cm (0.1) cube, and it doesn't show up at all.
Is my understanding correct:
-
Scene contains an ARSession and ARSessionOrigin
-
The ARSession has ArTrackedImageManager
-
Create a ReferenceImageLibrary and populate it with one image
-
Apply the ReferenceImageLibrary to the ARTrackedImageManager
-
Apply a prefab to TrackedImagePrefab
-
Build and profit?
As an aside, for the project I'm working on, we simply are spawning a model (a skeleton to be exact) at the position of an image tracker. However, once it spawns, we really just want it to exist in the world without the need of the tracker.
Would it make sense that, when I detect the image, I jump over to the plane detection/point cloud system and raycast to the center of the image to spawn the model? Then disable image tracking entirely?
wary fiber
Hello, Im having some issues with my unity + vuforia app.. issue with loading times on iOS (android works fast and nice).
Normally my app is loading really long (ios - 18 seconds) but one day I have used "screen recording" tool to record and post it in forums and ask people why is that.. WHILE recording the app loads in 5 seconds! It wasn't only one time behavior but it is like this all the time! I need a bit of help to fix long loading times.
link to video showing it:
https://photos.app.goo.gl/sUjtCDFucRvbv4k7A
😱 😦
shell python
is anyone having problems with making the flashlight work in vuforia?
I downloaded the core samples, and tried the image target scene where I can toggle the flashlight, and it doesn't work
hollow meadow
On Android the flashlight won’t work if you’re using Vuforia Fusion w/AR Core. Google disables the flashlight when you’re using AR Core because they don’t want lighting changes screwing up trackable features.
hybrid perch
Good to know
half basin
Can someone help me with setting up ARFoundations ReferenceImageLibrary and how to set it up so it detects different images and shows different prefabs?
I found this thread but I don't know what to set instead of "myPrefabsDictionary":
https://forum.unity.com/threads/arfoundation-get-images-clear-names-out-of-referenceimagelibrary.682015/
private void Awake()
{
var arTrackedImageManager = gameObject.GetComponent<ARTrackedImageManager>();
arTrackedImageManager.trackedImagesChanged += OnTrackedImagesChanged;
}
private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs ev)
{
// Check the new tracked images
foreach (ARTrackedImage image in ev.added)
{
XRReferenceImage refImage = image.referenceImage;
string imgName = refImage.name; // this is the name in the library
GameObject prefab = myPrefabsDictionary[imgName];
Instantiate(prefab, image.transform);
}
}
Or is someone aware of another solution on how to handle this?
hollow meadow
@shell python The flashlight will work on iOS, and it will work in situations where Vuforia isn’t using ARCore for extended tracking (vanilla image targets?). It’s a pain in the butt, yell at Google.
latent thunder
Does anyone know how would you would mask the Background Camera Image so that it only appears in a masked off area?
I played with the mask component but have had no luck so far.
half basin
Hey guys what is the best way to get into AR? is it through ARKit, google Ar or Web AR or some AR SDK like Unity?
I want to do this as a profession for a long run so I do not want to constantly switching around
severe spear
@half basin Mobile AR is still pretty new and experimental on a lot of platforms. Best bet for "stability" is probably AR Foundation (Unity's attempt at unifying ARKit and ARCore in one solution). There's better/easier ways if you want to only serve a single platform (say, iOS/ARKit).
copper socket
Has anyone had any issues with Vuforia's extended tracking? From my understanding you just turn on Positional tracking in the config. However, I'm still losing the objects when the image is out of view?
vocal folio
hello... has anyone here deployed to Microsoft Hololens? I'm getting this error message:
Severity Code Description Project File Line Suppression State
Error DEP0100: Please ensure that target device has developer mode enabled. Could not obtain a developer license on 127.0.0.1 due to error 80004005.
I have verified that the device has developer mode enabled and also it's paired with the computer, any ideas?
hybrid perch
It might help to turn developer mode off and on again. And have you trusted the computer? @vocal folio
vocal folio
@hybrid perch I did that, thanks for your reply, I was able to deploy the app creating a UWP package but trying to install directly from VS didn't work
hybrid perch
I never did that, I just deployed from unity 😊 either from the build settings or the UWP build tool
unique sparrow
anyone here have worked with Vuforia?
sonic ivy
Sure
unkempt heath
https://www.youtube.com/watch?v=Qi3h18wJJiI anyone know any tutorials to make something like this?
quasi hearth
@unkempt heath I would say just look for "marker based ar". Seems like most is done with vuforia.
dark frost
I think too, Vuforia do it pretty easily for image target tracking. Maybe some research for the collision between two target, i don't did it yet.
After that, it's more animation than AR
unkempt heath
i have done till building all 3
but the merging script i dont know how to do it
and then changing to the combined molecule
severe spear
@@unkempt heath Vuforia is one option (we made https://prefrontalcortex.de/projects/daskalos-interaktiver-chemielehrer/ with that), but costs license fees for deployment; Image Targets in ARFoundation are the best bet for mobile right now.
unkempt heath
hmm
copper socket
Unless you're moving something on Android ... it's super unstable out of the box , from what I tried (just a basic test)
Well, apparently the newer versions of ARCore fix that, but ARFoundation isn't using it just yet?
latent thunder
Machine learning capabilities of latest iPhones + ARKit 3 = Game Changer!
https://vimeo.com/343272414
copper socket
How hard is it to implement @latent thunder ?
That is, the people occlusion/pose recognition.
Is it more or less an automatic feature for occluding objects in space?
Also, is that Brookfield Place in Toronto?
open pollen
I've been writing some articles on Computer Vision - the latest one is on Convolutions - for anyone interested: https://medium.com/@vad710/cv-for-busy-developers-convolutions-5c984f216e8c
thin egret
Tracking is working in basic project but Camera background is black in basic AR project on iOS 😦 help!
Unity 2019.1.6, AR Foundation 2.2, ARKit 2.10, LWRP, 1.6.1 (also tried with 5.7.2).
App has permission to use the camera. Dunno what to do.
copper socket
I don't know much about it, but I experienced the same issue recently when starting to learn AR. It was due to LWRP, which from my understanding you need to override the Custom Renderer on the Ar Camera Background component (AR Session Origin camera) with an ARBackgroundRendererAsset. Though, I wasn't actually able to find out how to make one of those as there didn't seem to be an option to create ...
Anyway, I ended up just using the standard renderer and it worked.
thin egret
@copper socket This was the EXACT same process and conclusion I've come to.
@copper socket thanks for confirming, I'm currently just using standard as well.
copper socket
Okay, good luck! 😄
thin egret
👌
latent thunder
@copper socket project is latest ARFoundation samples in github. Look for Human Segmentation. And Yes:)
copper socket
Oh righ,t they released a new samples. I'll check it out as soon as we get our new iPads lol.
And yeah, I thought I recognized it. My wife works for them.
raw thunder
For anyone using ARKit from the package manager & not from BitBucket Repo. Can you help me figure out Tracking Reason in UnityEngine.XR.ARKit;[https://bitbucket.org/Unity-Technologies/unity-arkit-plugin/src/default/Assets/UnityARKitPlugin/Plugins/iOS/UnityARKit/NativeInterface/ARTrackingStateReason.cs]
half basin
@odd geyser hi! I saw your AR tutorial, and I need some advice. I'm trying to make a procedural AR forest for architectural visualization. I got a procedural tree asset from Houdini, and my goal is to make it so, that when I draw a marker somewhere, the software recognizes it, and places a tree with random values (height, trunk, foliage, etc.) onto the marker. I'm pretty new to unity, and I haven't found a tutorial yet that would help me out.
Here is a "faked" demonstration of my goal, done with a simple pre-exported obj that matches perfectly with the image target
credit/source for the trees: https://www.youtube.com/watch?v=PfTwUxAgJXs
odd geyser
Hey! That sounds like a great project - a rough rundown of how that'd work:
- Recognise the marker - Vuforia can definitely do this, though we also have support for markers in the latest version of ARFoundation (cross platform AR with unity)
- Take the world position of your marker and define the region - this will take some tweaking, though for reference, unity has a 1:1 scale with units/metres for AR
- Write a spawner that instantiates the models with random values
copper socket
If you're using an image target once to just position a static model, would it be more stable to:
-
Disable image tracking after spawning
-
Create a ReferencePoint at the position of the image
In either case, is there a way to get the accuracy of the position of the prefab to it's reference image? So that disabling or creating a referencepoint would be more or less on the center?
hollow meadow
@half basin The mark your using on the paper in the example is probably too simple to be used as a good AR marker. Usually AR markers want some asymmetrical details so it can detect orientation as well as position.
Vuforia’s web-based developer tools will grade your image marker when you upload it to let you know if it’s good or not. Even if you don’t end up using Vuforia that might be useful to you.
raw thunder
How many here working with ARWorldMap?
open pollen
For those curious about how Edge Detection works: https://medium.com/@vad710/cv-for-busy-developers-finding-edges-836aa58e0bb5
dark frost
Hi there ! I'm using ARFoundation 1.0 and I don't understand why the texture material isn't update when planes are.
I'm trying to create a procedural map that can be update the more we scan the room. (procedural only for props like flower, tree, rocks... I don't need to deform the plane mesh)
If anyone has advice about it ?
ivory relic
Hello all, just wondering if anyone can point me in the right direction to make an augmented image business card with ARCore? 😃
crystal ridge
@ivory relic what do u need ?
ivory relic
@crystal ridge just trying to figure it out really. The only thing I need to make app do is scan an image (which is displayed on the clients business card) spawn an fbx animation and have it anchored to the card, then have the abililty to tap on that object removing it away from the scene and spawn another fbx animation in it's place. Any help would be appreciated.
ivory relic
Also, I'm aware that arcore augmented images already can do this but I'm not really sure how to change the object and place it in the centre instead of those corner picture frames.
copper copper
Anyone knows where i can find an up to date version of this ?
Originaly posted in 12/2018 here https://blogs.unity3d.com/2018/12/18/unitys-handheld-ar-ecosystem-ar-foundation-arcore-and-arkit/
hollow meadow
@ivory relic the barrier that makes augmented business cards pointless is you need to hand someone a business card and convince them to download an app. That’s why my friend made an augmented business card using a Snapchat lens. It’s an app that millions of people already have on their device and you can easily build what you’re asking for in Snapchat’s lens studio. Something to consider.
ivory relic
@hollow meadow it's I shouldve given a little bit more light on the situation, it's for a tv show so things like snapchat would be difficult get a license from . Essentially it would be a form of playback for an actor to use on set.
half basin
we actually had faced a similar problem, but solved it without unity using a web-based approach. This was nice because it didn't force people to download an app.
IIRC, webGL unity can still access mobile device cameras. Just briefly thinking about it, I'd imagine that may be feasible for your problem?
gentle gate
Does anyone have any experience converting a YUV image into RGB?
lunar ice
hey would anyone happen to know how I can create a camera object that doesn't get controlled by my Hololens when using holographic emulation?
frozen bronze
@copper copper most of the features are implemented now. you can find them in the latest unity 2019.. through package manager.
copper copper
@frozen bronze Thanks ! And what about oriented feature points ? Is it available on AR Foundation ?
frozen bronze
@copper copper i think in the ARFoundation terms they called it ARReferencePoint..https://github.com/Unity-Technologies/arfoundation-samples/tree/lwrp-support-2019.1-with-2.1
Samples to the features are in the git hub repo..
copper copper
@frozen bronze thanks 😃
lunar ice
Does anyone know how to get a raw image to work on the hololens? I can get to to appear in the camera in Unity but it doesn't appear when I emulate the hololens or try to run it on the hololens itself
shadow basin
Hey guys, if I want to build an AR app only for iOS, is it better to use arfoundation or arkit? Is there a difference?
ionic hull
ARFoundation is just the base for ARKit/ARCore implementation, afaik. so you'll actually need both Foundation and Kit
odd geyser
Spot on Ryan!
copper socket
Two questions:
-
Is there are way to place a world anchor based on image recognition and do it with some sort of accuracy that the position will be at the center spawn of the image
-
Does anyone have experience with the new ARKit 3.0 human tracking manager? For occlusion, are we expected to use the texture from the stencil to create some sort of screen shader?
Thanks!
past crypt
someone else is using arkit 3 on ios 13 beta 3? we've found this bug and we need help https://github.com/Unity-Technologies/arfoundation-samples/issues/188
normal beacon
hey guys, is anyone here able to run the foundation samples for body tracking? 2d or 3d
those dont seem to work for me
im on ios 13 (downloaded the latest public beta)
@latent thunder it seems you were getting the same errors messages
were you able to fix that?
nw_connection_receive_internal_block_invoke [C9] Receive reply failed with error "Operation canceled"
nevermind, seems like stuff is broken for the latest public beta
and they are in the process of fixing
ivory relic
Is there anyway to change googlesarcore hello ar script to recognise keystrokes instead of touch? But still utilising the plane detection? So a wireless keyboard would be able to trigger the touch input.
frozen bronze
@ivory relic i think so you can replace touch input api with this https://docs.unity3d.com/ScriptReference/Input.GetKeyDown.html one. it should probably work.
dark frost
Hello everybody, is there someone friendly with AR Foundation who can explain me what's wrong with plane texture ? I'm trying to update texture when planes updates bounds, but It's always the same result, may I forget something.. ?
frozen bronze
@dark frost The functionality that you want to achieve is already there in the samples...you just need to replace the texture with the one you want to use.
dark frost
@frozen bronze , As you can see, my texture do not repeat with the update of mesh, that's what I'm looking for and what I don't understand :/
I get a weird result when I replace with a random texture
For both pictures it's the same texture.. :/
frozen bronze
@dark frost May be you need to set the texture properties first....click on the texture and look for wrap mode...and set it to repeat.
dark frost
Damn.. So easy ><"
Thank you very much ! ^^ I lost so much time for a silly reason 😦
frozen bronze
@dark frost No problem any time.. 😀 some time silly mistakes are good for the future..
dark frost
Yea for sure, like you see, I didn't spend all my time on it, I try to use boundary points of meshes to create Virtual world on differents planes. A character will move from a plane to another. It will be a little experience for myself ^^
frozen bronze
It's nice to see that... never tried to use multiple planes for creating environment... is it stable? or plane flies off from the point it has been created?
dark frost
I'm on a Samsung s7 Edge, and at the moment, that's not really stable. As you say, planes flies off when I'm looking for multiples planes. I will show a demo when it will be a little better.
I'm hoping it will become more stable with time and technologie. 😃
At the moment, the only thing I saw was people which scan surfaces and drop a point. From this point they create procedural map with defined lenght. I would like to go further like we can see in Hololens trailer but with mobile ^^"
frozen bronze
@dark frost Hope so.. in the android plane flies off problem is still persist.. after little bit of research on why it flies off i found out that..the phone is not getting enough feature point to make the object stand where we have told them to stand.. so when the system didn't get enough feature point it reset the session.. and tries to scan the environment again for stabilising. so to work around that problem..i tried to put the object a little bit above the surface so it always get enough feature point so object stays there. when the user tries to see object from really closely then this kind of problem happens..happy to hear your findings too...and some of the people on the thread says to use ARReferencePoint..i have tried that too..so it's much more stable than the normal way but there is problem with the scaling..when we scale the origin object doesn't;'t scale with the origin which is created from the reference point ..
dark frost
Oh ! Interesting, I didn't saw that. Nice information, thanks, I will look for it too.
By the way, I saw a "vertexChangedThreshold" which can help to claim more accurency I think
It will wait for more points before creating planes, It's not a final solution for the project, that's why you get my curiosity and I will take a look
frozen bronze
vertexChangeThreshold say that "The largest value by which a plane's vertex may change before the mesh is regenerated. Units are in session-space meters." so what will happen is it will stops to regenerate the plane when user get's smaller values than threshold.. so user may be needs to move more to generate new plane or regenerate existing plane..as per doc and my understanding.
https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@1.0/api/UnityEngine.XR.ARFoundation.ARPlane.html
so basically it's good when we want them detect bigger plane i think so. Looking forward for more accurate solution for plane flies off problem.
dark frost
That's the point ^^
That's why I'm interested by your solution.
frozen bronze
@dark frost Yes try that out and share your experience here.. so all of us get to know if it's proper solution or not.
till then..looking forward to see more update from unity side.. to make proper fix for this problem.
dark frost
@frozen bronze Can I ask you what's your device ? Does new Iphone with depth sensors are better for this kind of work ? And yes, I will share if I get any news 😉
frozen bronze
@dark frost for android we have nokia sirocco.. and for iphone we have lots of different phones from iphone 6s to iphone x. what i think is.. it doesn't matter as soon as IMU sensors are in calibration.. but it's better to have good phone in hand while developing.😁
latent thunder
summer quest
Hello, I want to show you my AR app that uses Vuforia model target https://twitter.com/_Eduard26/status/1148322965269950464?s=19
acoustic swallow
sorry that I posted the question in the wrong place.
latent thunder
I've read this thread
https://forum.unity.com/threads/how-to-setup-people-occlusion.691789/
Anyone have an eta on when updates to HumanSegmentationDepthImages sample scene can be expected?
The last addition over a couple weeks back was the addition of a skeleton demo
https://github.com/Unity-Technologi...s/tree/master/Assets/Scenes/HumanSegmentation
The existing human segmentation sample scene is IMHO not really useful except as a proof that depth and segmentation info can be leveraged from ARKit3.
I ( and I'm sure many others!) have ideas involving depth and occlusion I want to execute but cannot proceed until there are updates.
obsidian zephyr
Are there any updates on fixing the HumanBodyTracking3D sample for devices running Beta 3 (or Beta 2) w/AR Foundation? This is especially troublesome since there isn't a straightforward way to downgrade iOS 13 devices.
https://github.com/Unity-Technologies/arfoundation-samples/issues/170
latent thunder
Anyone know the status of Project MARS – Mixed and Augmented Reality Studio ?
twilit harbor
I'm trying to use a camera's RenderTexture with ARCameraBackground
and its coming out skewed
from what I understand, it might be because the Projection Matrix is not correct on the cameras RenderTexture
but when trying to mimick the AR Camera's Projection Matrix, it don't work
from
that be inverted, skewed, and all round don't look right
twilit harbor
ok, pulling from the AR Camera's projection matrix rather than using ARCameraFrameEventArgs projection matrix seems to help
I have no idea why that would be the case though, i would have assumed they would be the same
severe spear
@twilit harbor please report a bug
formal totem
Hi,
Who can give me a solution : I have a problem with my macbookPro and Vuforia
When I play he crashes my unity thanks a lot
severe spear
Thanks for being so specific. The solution is to solve the bug
...
Jokes aside, you should really explain what your problem is and what you're trying to achieve and what you tried already
proud hatch
i'm just getting started on an ar project in unity and i've been having trouble getting ar foundation to recognize my phone as a supported device, it works fine with the ar core sdk test package though
but whenever i try to start up an ar foundation project and check the state it just goes from checking availabilty to unsuported immediately
session state*
proud hatch
i'm considering learning to use opencv with unity instead, the support for ar core seems really limited
i have 2 phones, one i play pokemon go with which handles ar just fine but is not supported at all by ar core
i'm just looking to track an image in the world and display some graphics relative to it
like this:
proud hatch
ar core sdk for unity apparently just does this sort of thing by default, but there is nearly no control over the process
and my results aren't nearly this smooth
i've got an image with a score of 95/100 in their quality rating system, provided the real word dimensions and have an image database with only one image in it
that's the only optimizations i can see that are available and the results were a very small improvement in recognition time
atomic temple
Can I ask for help with a problem I am having with my AR app here?
atomic temple
O nvm I got it
copper socket
@proud hatch Yeah, we've been using ARFoundation for our project which just spawns a skeleton on a 59cm image target on the floor. It's ... okay. Sometimes it freaks out a bit, but it's never as stable as these gifs/videos make them out to be.
It does help having the latest iPad pro's though. That was a huge increase in performance.
But other than that, there's not much else you can do it seems.
Personally, I've been wanting to use anchor points to just lock our model in space (since we only use the image tracker for spawning), but even then I'm not sure how to evaluate for when the tracking is considered "accurate" so that we can place a world anchor at its position.
proud hatch
one thing i wanted to try to do was adjust the time in which it checks for the target because it tends to roam a lot especially if you pan to the point where it is off camera for any length of time, seems to rely a lot on the motion sensors, maybe too much
but there doesn't even seem to be any way to tweak anything at all in the sdk
its very black box, or i'm missing something
i'm using the ar core sdk, because ar foundation just isn't working
also yea i wish i had access to an ipad pro :( i'm poor and just have two hand me down phones, a galaxy s7 and a nexus 6
it really doesn't even feel good thinking that what would be released would be limited to people with enough money for nice phones
copper socket
Yeah. Well, my work bought the pro because I insisted on it lol. I definitely can't afford my own.
But you're right, it does feel very black box.
It's basic plug and play stuff, at the cost of not being able to do much else.
proud hatch
ok cool so i'm not alone and probably not missing anything, dang
gleaming tundra
Hello!
How can I can make and video follow where the player is watching?
With Vuforia
latent thunder
Have an idea for a current project but before i start would like to get feedback.
2D image tracking allows you to track an image in your scene that is either stationary or moving.
Instead of an image I need to track a stationary video screen, replacing the content that appears on the video screen with augmented content and videos.
Can anyone suggest how this might be accomplished?
Does it help that the size or format of the video screen is always a 16:9 ratio (ie 1920x1080)?
copper socket
Is there actually going to be video playing on the screen?
Or it's just a blank screen that you're augmenting on top of?
latent thunder
There will be random video content playing that I want to replace. So essentially do the same thing you might do with canned video in After Effects and 3d tracking + masks to replace video but live in AR.
Guessing I might need to make use of some type of static image marker, maybe a logo off screen, but was hoping there might be another way.
copper socket
If it's a playing video, you'll definitely need to use markers on the TV frame or something.
While a newer gen tablet might pick it up quick, you couldn't possibly create an image database of frames from the video anyway.
unborn atlas
you could put a frame around the video and use that for rectangular opencv tracking
or track a watermark in the corner of the video
copper socket
I wonder how well AR will pick up on a screen though. Definitely something to test.
unborn atlas
it picks up ok in a living room - because it's relatively dark, and screens are bright these days
if it was a bright room with reflections it probably won't play as nice
another issue that can happen is that if you don't know the size of the screen, then your AR app won't know the difference between a small screen and a big screen that's far away
copper socket
Yeah, you definitely will need to know what scale your TV is going to be, and ideally keep it the same throughout.
Don't know if it's for a show or personal use though. If personal, maybe an option to input your TV size.
little zephyr
Hey guys, I'm new here
Forgive me if this isn't the right place to ask, but I need some help
I'm a complete novice in AR, but I need to make a phone application that recognizes when a certain pattern is displayed in the phone's camera
The pattern in question is word written in braille, so it would be just circles in a certain order
How should I approach this? Any help would be greatly appreciated
wide steeple
@little zephyr is there an example photo of what you need to recognize?
latent thunder
For a current project I need to replace a IRL billboard image with augmented video content on iOS only.
Between Vuforia and ARFoundation-ARKit what is the better solution?
Where should I go to get started?
Thanks!
velvet locust
Hey everyone! I'm working to create a collaborative drawing app using AR foundation, ARkit XR and collaborative sessions. I have gotten the drawing component to work and the networking component to work, but Im having trouble getting the drawn lines (im using a line renderer) to show up on both the local and remote devices. Does anybody have any ideas? Thanks!
gaunt torrent
I'm using ARCore to do marker tracking and spawning a 3d model on top of it. I've gotten everything working properly except that once the camera looses tracking, the 3d model just teleports to the middle of the screen and freezes there until you scan the marker again. I'm using the default scripts (the augmentedimagevisualiser is tweaked) that the example scene came with. I looked through the scripts and it looks like it should delete the model once it stops tracking but it doesnt seem to work. I've tried fiddling with the code but it didn't help. Has anyone had this issue?
proud hatch
if (TARGET.TrackingState != TrackingState.Tracking) MODEL.SetActive(false);
then back to true when it is being tracked again, ie in view
in the update function they just set it to true each time update is called and falst when tracking is lost
in the arcore sdk quickstart thing i mean
@gaunt torrent
gaunt torrent
I tried this setup and still have the same problem. I wonder if the problem is that it is actually not loosing its tracking state when it should be and just teleporting the image to the middle of the screen because it doesnt know where to put it?
And when I mean teleport, It basically shows the 3d model stationary from top view in the middle of the screen.
latent thunder
The 2D image tracking works extremely well with latest ARFoundation.
However a very basic question
Right now images in your ReferenceImageLibrary act as both 'target-trackers' and as augmented content that gets placed on top the matching image IRL. There is a 1:1 correspondence between tracker image and augmented image.
How do you swap out other images or videos to display as your augmented content using the same image trackers?
latent thunder
Experiencing some unexpected results with 2D image tracking when the image being tracked (in real life) is on a video display.
The augmented or tracked image exhibits huge amounts of jitter or vibrating movement.
Why is this?
Is there a workaround?
gaunt torrent
I'm no expert but reflections on the display could cause issues with the tracking. Especially if the image is dark.
proud hatch
i had the exact same issue and that is how i fixed it
is the object in question "FrameLowerLeft" ?
if not, or if the object is just another model in that same prefab, that is the problem
just make sure you are deactivating the right object
gaunt torrent
Well, yes I put my model in a empty GameObject called "FrameLowerLeft" which is referenced in the code. Is this wrong?
How did you go about putting in a custom model? Did you just put it in under a GameObject like mine?
trail cargo
so what are people doing for ARKit facetracking right now
the cool shit that drew me to it like facial AR remote all seems to use the old discontinued arkit plugin
is it still as easy as it was
gritty sleet
Anyone tried Mapbox's Indoor navigation example? https://blog.mapbox.com/indoor-navigation-in-ar-with-unity-6078afe9d958
I'm stuck on getting the sync points to show under the "Find a waypoint"
gritty sleet
Got it all fixed, but now it's just crashing on the phone due to " Abort message: jni_helper.cc: 19 'vm' Must be non NULL". Works fine in Editor. Guess they used a pretty old arcore version. Updating also pretty much breaks everything.
amber venture
Hi does anybody know what happened to the AR Remote tool ?
sonic ivy
@amber venture They are refactoring it for the new ARFoundation, last I heard
@amber venture there are still remotes built into the native ARKit/ARCore versions
amber venture
@sonic ivy That is good to hear, thanks.
tidal wyvern
Anyone knows how to know which image target is currently in view using AR foundation ?
copper socket
Check the ImageTracking example in the scenes folder of the ARFoundation package.
tidal wyvern
It doesn't have an example for changing
copper socket
Specifically the TrackedImageInfoManager script
tidal wyvern
I've tried doing TrackedImageManager.referenceLibrary.name.Equals(), but it doesn't work
copper socket
You could compare against it's name, maybe?
Again, look at the example script they provide. You can see how they display the trackedImage.referenceImage.name.
tidal wyvern
Oh, they are using a reference to the trackedImage, I was using a reference to the trackedImageManager
tidal wyvern
Nope, still not working as intended.
I'm trying to change the imagetrackermanager prefab depending on which image is being tracked.
copper socket
You weren't able to match names?
crystal dawn
@tidal wyvern I'm not sure what that would accomplish. An image is detected -> the prefab is instantiated -> event is fired. If you're changing the prefab after you are notified of a particular image being detected, the ARTrackedImage has already been instantiated and won't be affected by changing the prefab. What are you trying to accomplish?
tidal wyvern
@crystal dawn That's a good point and probably what is causing the problem.
I'm trying to attach a different prefab to different images, any idea on how to accomplish that ?
TrackedImageManager.referenceLibrary.name doesn't return the name of the images, I don't seem to find the method in the referencelibrary to accomplish what I want
copper socket
For our project, we don't even have anything assigned to the TrackedImagePrefab. Instead, we use OnTrackedImagesChanged to determine when an image is detected and then instantiate our prefab as a child of that ARTrackedImage.
Which is why I'm wondering if it's possible to create a dictionary of image names and prefabs. Then, when an image is tracked, take it's name and compare it to the list, spawning the correct prefab.
tidal wyvern
So something like if (trackedImage.referenceImage.name == "Gnomo1" Instantiate(Prefab1, trackedImage.transform);
?
copper socket
Yeah, basically.
tidal wyvern
Building now to test (since I can't seem to get unity remote to work with 2019)
tidal wyvern
It's finally working as intended, many many thanks guys !
crystal dawn
There are a couple of options:
- Create a new MonoBehaviour and put it on the prefab that is instantiated by the ARTrackedImageManager. In your MonoBehaviour's
Startmethod, have it instantiate another prefab as a child of itself according to your own logic. This could be based on aListofstring-GameObjectpairs, for instance. - Subscribe to the
ARTrackedImageManager'strackedImagesChangedevent and perform whatever logic you need to based on the name of the detected image.
TrackedImageManager.referenceLibrary.name doesn't return the name of the images
That is the name of the image library (the set of images). If you want the name of a particular reference image, use trackedImage.referenceImage.name, e.g, in a MonoBehaviour attached to the ARTrackedImage:
GetComponent<ARTrackedImage>().referenceImage.name
You can also iterate over all the reference images in an image library. The XRReferenceImageLibrary is enumerable, e.g.:
ARTrackedImageManager manager = ...
foreach (var referenceImage in manager.referenceLibrary) {
if (referenceImage.name == someString) {
...
}
}
You can also get the count and there is a subscript operator[]. See https://docs.unity3d.com/Packages/com.unity.xr.arsubsystems@2.2/api/UnityEngine.XR.ARSubsystems.XRReferenceImageLibrary.html for the script API reference.
tidal wyvern
decided to change Vuforia for ARFoundation and I after uploading the APK (with armx64 included) I get that most of the devices are not supported
Citing that <uses-feature> android.hardware.camera.ar is not supported
Maybe having had vuforia before altered the manifest ?
crystal dawn
@tidal wyvern ARCore is not supported on many (most?) devices. Vuforia will use ARCore where supported and fall back to their own tech on other devices. This is Google's list of supported devices -- it grows over time: https://developers.google.com/ar/discover/supported-devices
If you want to have a non-AR fallback experience, you can specify that AR is "optional", which will allow your app to be installed on unsupported devices. Then, you can detect whether AR is supported at runtime and provide a non-AR experience if not.
Docs: https://docs.unity3d.com/Packages/com.unity.xr.arcore@2.1/manual/index.html#build-settings
hybrid perch
EasyAR workd quite well, we ended up ditching it for Vuforia though. I can't remember why. But it works fine for image tracking
proud hatch
i think saying arcore doesn't work on most devices is like saying most of the people who ever lived are dead
wary fiber
Hey, I have a vuforia AR in my learning project( im newbie). How should look script if I would like to trigger a link whenever a target is detected? TrackableBehaviour.Status.DETECTED?
i have tried:
using Vuforia;
if(TrackableBehaviour.Status.DETECTED)
{
Application.OpenURL ("https://www.seboldsg.com/");
}
tidal wyvern
Well, amazed how easy was to do augmented images in EasyAR, no watermark and no fees for what I'm looking for, for anyone wondering.
wanton turtle
Yup their license server goes offline sometimes and your apps say you haven't paid your bill for a few days. But other than that it is usually solid. I recommend it for rapid experiments that aren't being done for clients.
But it works on almost all platforms, much more than vuforia, and is super easy to get started with. The price is nice too
tidal wyvern
Hopefully I don't have many problems, since it's an adgame, but good to know beforehand.
proud hatch
is there any way to stabilize the position of a tracked image in ar foundation besides extending the trackedposedriver and overriding the performupdate method with some kind of smoothing logic (not sure how i'd go about that either)?
i've got a cube that just floats in the scene and one tracked image, the tracked image jitters a LOT more than the cube, do i need to use an anchor? not sure what the analogue for that is in ar foundation or if it handles that itself orif the tracked image is just being detected every frame or what
copper socket
That's something I've never figured out.
We have a project where we spawn a human sized skeleton off a floor marker. Once it's spawned, we never really have to deal with the image tracking, but I'm not sure how to determine when to switch over to world anchors (which seem much more stable)
proud hatch
what do you use for anchors? i keep thinking i need to use some class that i'm not finding, but maybe i can just drop a game object into the world relative to the session origin
feel like a lot of the jittering might be due to constantly updating the tracked image found by the camera
gonna try that i think
copper socket
It's definitely because the tracked image is moving. If you're using Android, just note that ARFoundatibons is not up to the latest version I don't think.
Anyway, you can create anchors via Reference Points:
https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@1.0-preview.8/api/UnityEngine.XR.ARFoundation.ARReferencePointManager.html
But the thing is, you can't just pop one into the world where the image is detected, because there's not really a way to determine when the tracking is considered accurate.
proud hatch
ah ty
i was confused by that the first time i went over it
i was going kinda crazy looking for Pose in the ar foundation documentation, but its actually just a class in unityengine
modern sparrow
Anyone familiar with MAXST AR visual SLAM tool for creating object trackers?
wary fiber
Hi,
Im having issue with metadata:
string URL = targetSearchResult.metadata;
Is giving me error:
Assets\SCRIPTS\SimpleCloudHandler.cs(43,41): error CS1061: 'TargetFinder.TargetSearchResult' does not contain a definition for 'metadata' and no accessible extension method 'metadata' accepting a first argument of type 'TargetFinder.TargetSearchResult' could be found (are you missing a using directive or an assembly reference?)
im making cloudreco script
any ideas?
wanton turtle
@proud hatch fwiw, I usually put a tiny smooth on the objects. I usually average the location over the last 3 frames. It makes it feel like it jitters less at a cost of lagging a tiny bit when they move their phone away from the location. The reason it works is people usually aren't trying to look away, they are intending to look at the object.
For some frameworks I've smoothed it over 10 frames. But 3 is usually sufficient
Also you can do that in your base GameObject without needing to do it for the ArFoundation itself if slam is working well the image tracking is what is gittery
proud hatch
ahhh so i'm not crazy (was thinking about trying something like that), awesome tyty
but what is slam?
severe spear
The science behind ARCore/ARKit - simultaneous localization and mapping
copper socket
Has anyone had issues with the camera feed becoming super jittery / tearing?
jade barn
Can anyone help me with AR Foundation and ARCore
Im getting black screen on my phone (Its basically the Camera Solid Color)
Im using LWRP
Unity 2019.2
hallow idol
@jade barn I think there is currently no way of using ARFoundation with LWPR on 2019.2 as it doesn't allow you to use LWPR 5.6.1
Try 2019.1 and get LWPR assets from this branch:
https://github.com/Unity-Technologies/arfoundation-samples/tree/lwrp-support-2019.1-with-2.1
Don't forget about downgrading LWPR to 5.6.1 as even unity 2019.1 automatically upgrades it to higher versions sometimes
copper socket
So turns out the camera ripping jitter issue was a result of being on the latest ARFoundation.
We rolled back to and older version we were using before, and it went away, which is annoying.
jade barn
@hallow idol Thank you, will try that later today
dawn moat
Scape SDK is out Live in London!
ScapeKit is a package that makes creating location-based AR apps easy. It handles all the complexity of giving you the most accurate position and orientation in a real-world via your device's camera while providing you with the freedom to create a stunning AR experience.
For more info check out www.scape.io
hybrid perch
Wow, that is super cool and also super relevant for some things we are working on at work. Very interested in seeing this in use
dawn moat
@hybrid perch Thanks this been holy grail of AR. Till now its only been possible to rely on the GPS to anchor the content but we all know how in-precise that is. Now you can use VPS and get super precise placement and not even talking about persistent content that can now be seen by multiple people in exact location
I posted in #archived-works-in-progress the demo too 😉
azure knoll
Wow, that's dope! Amsterdam coming any time soon?
dawn moat
@azure knoll It is on list we have data of over 104 cities next we will target will be San Fran
azure knoll
Hah that makes sense
half basin
Yo @keen mango
dawn moat
o/
frail rain
oi
unborn swallow
proud hatch
i still get a lot of jittering when i average the position and rotation of a tracked image over a given amount of frames due to BIG leaps in where the phone seems to think the image should be in the world
its like the slightest shake of my hand sends it all over the place
i feel like there should be some way to try to counter that with maths but i'm not sure what it is
the more frames i use for averaging, the slower the object moves back to the correct position too, so it will fly off to the upper left and then creep back if i'm averaging enough frames
proud hatch
a kalman filter would help
copper socket
Suffering the same issue.
It's definitely worse on Android due to ARFoundation not being on the latest ARCore.
iPad pro 2018 definitely is far more stable, and we haven't really suffered tipping or popping muc,h but it does shake quite a bit very slightly like it's vibrating whenever the image is in view (assuming it's trying to correct).
I did consider smoothing it out, but like you said, I'm not sure how to filter out extreme errors in tracking.
proud hatch
kalman filter is made for this, i'm trying to figure it out still though
there are different forms, as well, this was suggested to me by someone on freenode in #math
since it only takes into account the position and rotation
also yea the image tracking itself introduces some jitter it seems https://discordapp.com/channels/489222168727519232/497874498373156894/607600955101282308
i figure the big spikes are the motion sensors
i'm going to output all the motion data i can to a canvas and see if that helps confirm that suspicion, still not sure if there is any better way to debug ar foundation currently without unity remote which isn't supported atm?
copper socket
Yeah, gotta pump it to the canvas for sure
proud hatch
i don't think thats the way to go but i cbf to figure out the right way atm
copper socket
I'm just wondering if a solution would be to:
-
Use image tracking as normal
-
Instead of childing the object to the marker, simply position it to the recognized point
-
Cache the position of the object each frame
-
Over the course of [x] frames (time), get the average position
-
Create a world anchor at said position, child the object, turn off image tracking
But you would need some sort of threshold to consider as a cacheable point. Ie, if the object in [x] frame is >distance than the threshold, discard it (ie, it popped). But then the question arises as to what is the initial caching point ...
proud hatch
i was thinking along the same lines but i figured someone had a more clever solution and the kalman filter is it
specificaly for 5
i've just been polling the position of the object created by the trackedImageManager and not giving it a prefab
then taking the average position and rotation over a variable amount of frames to place my object
i added some buttons to increase and reduce the amount of frames taken into account when averaging to see how it comes out with different values, the pops really kill everything though