Scaling The Database

[north karma]

I'm trying to scale the database due to high traffic concerns.

I am plannig to split the initial schema into 4 schemas (Orders/Cart, Product, Customers,?) ps:I just extracted the related tables (didn't change them)

The main idea is each service would connect to its dedicated database instance instead of a single monolithic instance which would make scaling much easier and efficient.

I managed to do this by extending the service repository and using a different datasource than the default one that loaders/database provide

However I am concerned if this would cause any issues or if there is a better way of doing it.

This concern stems from the fact that the default datasource is being registered in the container as well as a pg_connection object which may be used somewhere else that would cause some misbehaving.

Ps : I am fairly new to the Medusa codebase I tried tracking down where they are used but couldn't find anything notable. However since they are being registered in the container their has to be a reason behind that.My main assumption is that third party plugins and modules are using that ?

[weak valve]

Nice problem to have! Congrats!

My personal opinion is that you should avoid vertical partitioning of the db if possible. If you've maxed out the obvious way of scaling (biggest and bestest instance available), you should probably look into streaming (real-time) replication first, with some way of splitting the load between instances.

Maybe someone else can add more concrete experience with scaling the medusa db in particular.

[north karma]

sadly there is a limit on how much you can scale vertically . and with the default database setup it's just not built for horizontal scaling (without spliting the initial schema).

[sonic badger]

Hi @north karma how did you solve this problem? and aprox how much traffic/records did you had then you started havind this issue?

[north karma]

I used dbeaver to visualize the schema and split it into services then I specified where each table/model belongs (a different database) it's supported by typeorm and it does most of the heavy lifting.
I didn't really stumble on any issues for the present time but I do anticipate it would be a problem in the future and I like knowing what I am getting myself into before hitting the actual issue

I was especially concerned with this due to MedusaJs being a defactro microservices architecture when the database is not

And it just felt weird and raised this natural question in my head

[sonic badger]

thanks for your response! I'm now on that same question, estimating the traffic we are going to have and choosing which architecture is best for us. Do you have any advice for me?
Currently we have deployed our service on railway but I want to move it to AWS or GCloud as I have free credits

[solar trail]

What load figures are we talking?

I’m thinking you should just be able to create more postgrea read nodes which is trivial