Multiple databases in a single application

[cloud sable]

I'm about to build a new event sourced application. I am being challenged by the higher ups to not build a new application, but to extend an existing application that already has its own marten db. There is some chance that this new functionality may be decoupled into its own application down the track, but not straight away

I think it's a bad idea, but If I absolutely have to, I'd rather add a dedicated marten store for the new functionality and scope any events to that so I don't have to dig them out from the main store later on if we do decouple. I read the article on spinning up multiple databases, but note this conversation "We don’t yet have a way to register a matching IDocumentSession or IQuerySession type to go with the separate document store". If I read this right, does that make it a non starter for an event sourced system? How do I make sure I get the right document or query session in the new functionality code?

Is a better approach to just keep the one document source and if we need to decouple, to restream the affected streams (which won't share events) into the new infra and just clean up those streams at the appropriate time?

[naive oak]

That's a longer conversation. I think Marten's "separate store" functionality is relatively weak in its usability. I think that the document store functionality is fine as is within a modular monolith, and you can just move document by document storage over later w/ minimal friction.

The event sourcing is a little harder the way it's commingled. I think I'd recommend you to be adamant about tagging streams with an aggregate type just to make it easier to move events to another system if that becomes valuable. It may mean that Marten needs some extra work to make that easier for users, but that sounds like a good thing for us to have anyway.

Any of that make any sense>

?

[full cairn]

I did it before the module per schema, which also mean a different store per module. That worked well for me.

I was integrating modules through Outbox and Kafka

I wouldn't recommend using a single Document Store as the final solution, more like a temporary solution.

You could use a custom session factory to resolve the proper store definition.

[naive oak]

I wouldn't go down that path because of the potential confusion. Any kind of runtime IoC logic can end in tears. What @full cairn said about using separate schemas per module would help for easing future moves and making it clearer where the boundaries are.

[full cairn]

In theory with .NET 8 keyed services it should be a bit more plausible, but still tricky.

That’d probably require some work on our side to make that work with AddMarten. v7 already can support keyed Npgsql data source registration. But that’s a first step.

[naive oak]

I wouldn't do that either -- and that's the voice of 20 years experience supporting people doing things like than w/ IoC tools:) I think we need to revisit the "separate store" stuff some day. I wish in retrospect we'd done something so that the IDocumentSession was different rather than just using a marker for the IDocumentStore type. We went back and forth quite a bit, but I wish now we'd made it more like EF core where you can have subclasses of DbContext so you know what store you're addressing.

But not that anybody has mentioned this, the keyed service approach might help the wolverine integration later. Wolverine doesn't yet have any support for the Marten "separate store" thing. Or for more than one EF Core targeted database in an app either

I've got to think more on that one. My experience -- and remember that all the original DI containers like StructureMap did "keyed services" when David Fowler was still in middle school -- is that depending on using named service injection into a class w/ an IoC container inevitably leads to confusion because the type signatures no longer tell you anything definitive.

Like folks doing something like saying "if the class is in this namespace, inject this named registration for NpgsqlConnection"

In a perfect world, I'd like to see type info like IModule1Session where that implements IDocumentSession

[full cairn]

yeah, but currently one DS could mean one Data Source, which wouldn’t be that bad as DS is singleton

Of course, I agree, that making IoC it too clever is not a great move.

I think that the EF DB context also has a 1:1 relation to data source (although I may be wrong here)

Of course, abstraction you mentioned is also an option.

[naive oak]

It does, and that's its advantage here. I havent' looked too closely, but I'm betting they use a lot of static variables to store the kind of expensive dynamic compilation stuff that we bury in DocumentStore somehow

[full cairn]

Right now after data source changes we also have such relationship

[naive oak]

When we did the separate store thing, we considered what we have now vs. using a marker interface that implemented IDocumentSession , and went the way we did because it seemed easier at the time.

[full cairn]

So it could be a bit easier now, but we’d need to add keyed data source registration for DocumentStore

As an option for advanced users could be something to unblock modular monolith people. We could show that as scenario before coming with something better

Of course, that’s just a thought

[full cairn]

Yes, with keyed services we could remove the requirement for custom interface

Although for sure interface is more explicit and less chance for a footgun

Of course, that’s a food for thought

[naive oak]

Or I'd reverse it. Keyed service for additional document stores, then let them use an interface for the IDocumentSession w/ some way to configure the session creation, then generate an ISessionFactory of sorts to match for the new session type.

But this kind of thing is why I think the pendulum will shift back to at-least-smaller-services where you can use a single IHost in the future and away from modular monoliths when folks really wrestle w/ the extra complexity of actually making your app modular

[full cairn]

In general modularity in .NET is broken

[naive oak]

But hey, I'm not ruling out in my own mind what you're suggesting

[full cairn]

Or almost not existing on the framework level

[naive oak]

I don't see this being any easier in Java world.

[full cairn]

In Java you can have package scope (so folder)

That gives more options than creating assembly

[naive oak]

Huh, kind like what "Child Containers" were in StructureMap years ago? That has its own set of complexities if they're doing that w/ container tricks

[full cairn]

I was referring in general, for IoC is probably similar. Of what I noticed they usually go with auto registration with Spring Boot nowadays

[naive oak]

Funny, but I've been down on that myself for the past several years. Helped someone use that w/ Lamar yesterday, and that's the first I've worked with that since I moved it all over from StructureMap years ago

I'm at a point where I just want to tell folks to use their IoC tool very simply and explicitly, then there's no real problems.

[full cairn]

And water is wet 😄

[cloud sable]

Thanks for the responses, I think I'll go with the tagged aggregates if I have to (though I'm still pushing back on the idea for a whole raft of other reasons as well)