Hello, I am considering Marten for a project where I am tracking items in a production line: I need to track the position and related process data of each single item as time progresses.
Event sourcing looks like a good candidate and I have already started thinking about how to convert the relational architecture to an event sourced one (e.g. each item has its own event stream, each movement of an item is a new event in that stream).
What I am missing though is a solution to this requirement: I also need to query for specific event types and their values across all streams, i.e. looking for all items which have been in some location within a defined timespan.
The documentation advises against using QueryRawEventDataOnly/QueryAllRawEvents but it doesn't look like event projections would solve my problem (or I haven't understood them). Are the performance implications that bad?
Could anyone shed some light on this? Thank you!
#Marten: querying by event across all streams
grim rock
minor hamlet
You would absolutely wanting to be using a projection to build a view for that data ahead of time that you could then index the way you'd want. There would definitely be performance implications by trying to query through the raw event data. Why are you thinking that EventProjection wouldn't work in this case?
vital crest
I'd consider if Event Sourcing is actually the good match.
Check this discussion #1138414928357769246 message
I think that for your case, I'd consider a mixture of TimescaleDB and PostGis plugins
grim rock
You would absolutely wanting to be using a projection to build a view for that d...
In my case, streams are unbounded (think serial number for each item) while events per stream practically top out at 50. Locations (the destination of an item in an "item moved" event) are only a handful and are predefined.
As far as I understand it, an event projection would need to create and update an aggregate document/stream that contains all "item moved" events ever across all items, with a { Item, Location } structure.
Even if my event projection were to create different streams per location, their contents would just keep growing and I don't think this is really much better than querying all events anyway...
grim rock
Check: https://event-driven.io/en/postgres_superpowers/
I agree that this also looks like a good fit for a timeseries database, in fact I am also considering TimescaleDB but am finding some limits for my use case. I'll have a look at what you linked, thank you
vital crest
50 sounds indeed good enough. Yes, you'd probably need to find a respected document (e.g. set the id to string and find some way to build it on the location name).
Still, it's hard to say for sure, as I don't know precisely your use case. Projections and read models should be fine-tuned for the querying needs. So they're highly contextual.