Eventual consistent management application on Auth0 and legacy systems using Wolverine

[wide jacinth]

This year, one of my projects will be to migrate a set of (legacy) systems to auth0. Currently these systems sync user data between them (without outboxing, so it is buggy). Now I want to centralize these users in Auth0, but as a lot of querys are dependent on the exising user tables in these systems, I want to sync a readonly copy to these systems (a projection sort of speak). Now in order to do that, I was thinking of making a chain of cascading wolverine handlers, that do this:

1. endpoint for creating user, store logging event
2. create user in auth0, store logging event
3. create user in legacy system 1*, store logging event
4. create user in system 2*, store logging event

*Syncing users might be using queuing in later stages, but at first I will go for raw SQL inserts there.

The management screens will hit 'create', and list users from auth0. But how would I now that at least creating the user in auth0 is finished, such that my list view can get updated. I was thinking websockets or so (as step 2a), but maybe there are easier ways to do it? 🤔

[bleak cradle]

What about making the Auth0 user within the endpoint instead of it being a cascading handler? Creating the user within Auth0 is the most important operation here and needs to be consistent with your UI, so it doesn't make a lot of sense to shove off to the background.

If the UI will list auth0 users as the source of truth then any operation against auth0 should be strongly consistent as a rule.

[wide jacinth]

Yeah ofc, but then it could be that you persist in auth0, but do not persist the outbox message

[rapid hearth]

I agree with @bleak cradle. Wouldn't it make sense to start with Auth0 and use trigger in post registration flow https://auth0.com/docs/customize/actions/flows-and-triggers/post-user-registration-flow ?

[rapid hearth]

You could use side effect for that.

[wide jacinth]

I asked Okta about that flow and they told me that it is not durable. It tries x times and then it stops. They are working on introducing SCIM for this.

[rapid hearth]

Yes, that's why it's worth doing both, so storing the event in the stream and calling auth0.

Still, you'll always have the two-phase commit issue here and chicken vs egg problem.

[wide jacinth]

yes exactly

[rapid hearth]

I think that the most important is to have auth0 set up and then some action to ensure that it's replicated.

Might be that even this is not needed, this could be potentially the manual or semi-manual operation.

My semi-manual is that if you start with auth0 you could detect upon login (or other place) that you don't have user information in the database and then trigger synchronisation.

[wide jacinth]

So I should just have to ways to sync so whenever one fails the other might do it. But no gureantees on either one of them

In my first solution I have guarantees and a DLQ to back me up

but eventual consitency to deal with

[rapid hearth]

Yes, pick your poison.

you can either use web sockets or long polling (https://event-driven.io/en/long_polling_and_eventual_consistency/).

I'd still suggest both

[wide jacinth]

Since there are multiple systems involved, on login is not the best option as well, but it would have been in a lot of other systems 🙂

[rapid hearth]

It may or may not, depending on the likelihood of the failure.

It may be much simpler solution if the error rate is low.

In the worst case someone might not be able to login and will need to retry.

[wide jacinth]

given wolverine/marten in place. Would long polling have benefits over websockets?

[rapid hearth]

Websockets are not so easy to configure (especially in load balancing)

[wide jacinth]

I know from experience in these system error rate could be a problem

That's why I am keen on the durable part

[rapid hearth]

Plus they need to have the connection open and it still may fail.

Regarding the API endpoint and long polling, you could do:

https://github.com/oskardudycz/EventSourcing.NetCore/blob/main/Sample/Helpdesk.Wolverine/Helpdesk.Api/Incidents/ResolutionBatch/IncidentsBatchResolutionHandler.cs#L29

[wide jacinth]

Yeah so, it might be better to have some status projection that you can ask whether creation of the account is ready?

ah ofc you have an example 😆

Thanks!

[rapid hearth]

So return the location header with the expected version, I'm using here current, but you could pass there the expected after all is processed (e.g. +1)

Then if you don't see this record with the specific version, you could retry that on the backend or return retry after header to UI.

[wide jacinth]

oh that stuff using the version is smart Oskar

[rapid hearth]

Yeah, and quite simple 🙂

[wide jacinth]

This would return null if it is not there yet?
https://github.com/oskardudycz/EventSourcing.NetCore/blob/main/Sample/Helpdesk.Wolverine/Helpdesk.Api/Incidents/ResolutionBatch/GettingIncidentResolutionBatch/GetIncidentsBatchResolution.cs

[rapid hearth]

Yes, or 404

I don't have example in C#, but in Java this would look like that:

https://github.com/oskardudycz/EventSourcing.JVM/blob/a2905f2aa78c8f5e177ab65a3c37b817ed21d85f/samples/event-sourcing-esdb-simple/src/main/java/io/eventdriven/ecommerce/shoppingcarts/gettingbyid/GetShoppingCartById.java#L11

This is the query: https://github.com/oskardudycz/EventSourcing.JVM/blob/a2905f2aa78c8f5e177ab65a3c37b817ed21d85f/samples/event-sourcing-esdb-aggregates/src/main/java/io/eventdriven/ecommerce/shoppingcarts/gettingbyid/ShoppingCartDetailsRepository.java#L13

[wide jacinth]

I could also just do long-polling on auth0 ofc

[rapid hearth]

I'm adding >= not to be vulnerable if there were some follow up operations in the meantime.

[wide jacinth]

But I do not know if I can set the users id upfront. So that might be the killer there

[rapid hearth]

For sure you can send some custom field to Auth0

[wide jacinth]

Yeah I suppose. Then it only depends on whether my SPA is going to be connected directly to auth0 for listing users or not. But I do not see reasons why not yet. Thanks for the usefull discussion as always!

[rapid hearth]

Happy to help 🙂

When you have the final solution, it'd be great if you could do a quick follow up 🙂

[wide jacinth]

I will. But it could be couple of months. I am in exploration phase.

[rapid hearth]

Btw. you don't need to pass the version, you could pass e.g. expected status as url param

This could be more maintainable if you expect potentially multiple steps.

Version is especially useful for the read models.

[wide jacinth]

Yeah, I get that. I think I was to easy on going websockets because of signalr being so easy to use. But you are right, on AWS and using scaling you will have to do a lot more.

[rapid hearth]

https://developer.mozilla.org/en-US/docs/Web/HTTP/Conditional_requests - this is a gem

[wide jacinth]

I will first have to move away from our own e2ee based on the users password, since auth0 will not let us access the password and it was never a nice solution either way

[rapid hearth]

From my experience, the prefered option is websockets, but you still need to have some sort of fallback. Even SignalR doesn't have the full delivery guarantees.

User may refresh page, have connectivity issues and bang.

Plus websockets in mobile can drain battery quite fast.

[wide jacinth]

Going to start a proof of concept in two weeks to use virgil security for that to back-up user specific random keys

[rapid hearth]

Plus if your API is not backend-for-frontends then it may be also a challenge as not all may want to use websockets.

[wide jacinth]

in this case it is an admin application, so not really end user fletched

But still, I want their experience to be good as well

[rapid hearth]

Got it. Just sharing my experience, I once overhyped websocket usage and I was doomed because of that 😄

[wide jacinth]

And in a later stage we will let users self manage organizations. So better be prepared

[rapid hearth]

So I'm now treating websockets as an UX/performance optimisation than bullet-proof solution.

[rapid hearth]

Yeah, some key vault and maybe integration with password manager would be nice.

[wide jacinth]

Which I think is a fine way to think about it (websockets), but it also depends on costs and time ofc 😉 Welcome back to the real world :-p

[wide jacinth]

I was first looking into AWS secret manager, but they charge one dollar per user per key....

[bleak cradle]

Our identity system is a effectively a big proxy & cache around Auth0's APIs, we run with a model that Auth0's APIs are the priority operation in all transactions and don't store anything in our own system that would cause issues if it were missing. Some critical paths have fallback logic whereby if a DB entry is missing for a given user/tenant we can fill it in with the default values and continue without breaking anything.

[rapid hearth]

https://twitter.com/davidfowl/status/1756177856693174489

David Fowler (@davidfowl) on X

The idea is to have a convention around the "environment" providing a key vault resource that can be used to store secrets.

PS: Now I'm posting bicep. I might be an SRE now 😅.

Twitter · Feb 10, 2024, 4:46 AM

[wide jacinth]

I am not on Twitter anymore, but I think this uses Azure key vault and that one is one dollar per key/user as well 🥲

[rapid hearth]

Yeah, they align pricing 😉

You can host your own hashicorp vault, but as always tradeoffs...

[wide jacinth]

no I want to be able to say we (developers) can not access the keys

That's why I went on to virgil security since they do pricing on bulk

This would have cost me 30K in AWS or Azure and 600 dollars here

[bleak cradle]

virgil security looks pretty neat, haven't seen them before.

[wide jacinth]

A year ago I was comparing chatengines as a service and we chose for https://getstream.io/ in the end. Competitors where SendBird and PubNub, but they all had articles about e2ee and pointed to virgil. So that's how I know about them

[rapid hearth]

Btw. in your domain @wide jacinth you may be interestin in this https://www.youtube.com/watch?v=WsR-UeA1-sA

I saw that live, really intriguing and inspiring piece of work and design.

They're using https://solidproject.org/

[wide jacinth]

I will look into it, thank you

[wide jacinth]

You think there might be benefits in storing the keys there over virgil?

[rapid hearth]

Dunno, I didn't do a deep dive. My understanding is that solid protocol is more for the general privacy related data, like PII etc.