Strategies for Managing Monorepos

[slate wigeon]

Hello everyone, I've been delving into the world of monorepos and exploring some intriguing monorepo tools like Nx and Turborepo. I'm quite curious to learn about how you all manage your monorepos.

Currently, I'm working on a personal project that leverages Kubernetes and Skaffold for building and deploying my NestJS, python, and Next.js microservices. I've set up ingress for traffic routing to avoid the hassle of specifying unique ports for accessing my APIs. However, I've encountered a few challenges with this setup:

• Code Sharing: While I understand that I could use npm to publish my libraries, I find it cumbersome to have to republish the library every time I want to debug or test new changes.

• Slow Refreshing with Skaffold: Skaffold has been a helpful tool for local Kubernetes development, as it ensures that changes take effect without requiring the reapplication of Kubernetes objects. This approach is also language-agnostic. However, I've noticed that it can be slow, particularly when dealing with a large number of microservices that need to run simultaneously.

I've been contemplating using Nx since it does address some of the issues I've had with my monorepo, but I've struggled to find a suitable example for translating my Kubernetes setup, especially the ingress part, into Nx. This is crucial for me because I want to reroute all my microservices to a shared domain (e.g., http://foo.dev) instead of having to specify http://localhost:3000, http://localhost:3001, and so on for local development.

[potent turret]

Sounds to me like a proper tools to work with k8s problem. Before I continue, do you have a continously running k8s setup? Like in a homelab, or in the cloud?

[slate wigeon]

I don't have a k8s setup continuously running. My development setup is just minikube and I make use of skaffold to run all my applications. It's quite nice since skaffold can detect code changes. I did use GKE for development, but found minikube to be sufficient

[potent turret]

Yeah, and that's the problem. k8s needs to be always running. Otherwise, you'll continually have the issue of the "fireup time" to get developing. This is just one of many reasons why microservices for a starting project are a hugely bad idea. If it is for learning purposes, ok. But, you'll always have to take that start-up time hit to do the learning. If you are serious about your project, you might want to rethink your path to success. 🙂

For instance, I'm attempting (and learning too) to build a platform for basically Nest apps (along with frontend too of course). I've been working on it for a few years now (as a hobby) and came to the conclusion some time ago that there is no way I can offer devs a platform, it be flexible enough for building practically anything, without k8s.

And, after fiddling for a few months with local dev and how to make local dev work to move the code to a k8s environment, I ended up creating a constantly running k8s environment. I pay now about 150 Euros a month for a 7 node cluster (on 4 bare metal servers with proxmox creating 8 VMs, the 8th VM working as a load balancer). I see the monthly cost as an investment in my learning and my goals and when I look back, I've learned a lot and I'm getting slowly but surely to my goal. 🙂

Also at one point, I realized, with all the other outside services I need to run i.e. a mail server, an IAM provider, the workflow engine, etc, local devlopment would be a real pain in the butt. There is absolutely no way to coordinate this cleanly (not even with Skaffold).

So, I found a way to develop 100% remotely and inside the cluster with Coder. There is another technology out there for remote development from Ambassador labs called Telepresence, but I didn't like this approach for a number of reasons.

At any rate, first thing I'd suggest you learn is what devOps is and how it works with k8s (if you haven't already). For instance, now that I have the remote dev environments working, I'm working on the "workflow" engine I mentioned above. It's using the Argo suite of apps. Check them out. There is no way you can just fire these things up and then break them down all the time and be successful. 🙂

At some point you'll see you need to run the services constantly in or around your cluster, even as your first prototype and then the fun really begins. 🙂