Pillar, Redis messaging between your servers

[balmy copper]

Pillar exists to make communication between your servers easier: send information and get a response back, fast and reliably. One of the biggest challenges in a server network is making sure that exchange arrives intact, with an equally solid response coming back. Pillar brings server presence, task dispatch, messaging, and distributed locks together in a single set of tools.

It's practical in proportion to what you need. You don't need a network with hundreds of servers for it to be worth it: it solves anything from a single specific problem to becoming the pillar of an entire network.

You can use it to send players to the server with the least load, make sure only one server loads a given resource at a time (like loading a Skyblock island), or ask any server a question and get a fast response back.

For more use cases, pls read the documentation.

GitHub: https://github.com/vinicius-paraujo/Pillar
Docs: https://pillar.markineo.com.br
Hangar: https://hangar.papermc.io/markineo/Pillar

[blissful gust]

Have you ever thought of supporting more than redis?

and supporting more platforms

[gloomy gust]

what other platforms are out there? (i'm a beginner so please don't roast my ignorance 🙏 )

[blissful gust]

fabric, forge, neoforge, sponge, bungee, etc.

also: https://pillar.markineo.com.br/dev/messaging/#requestresponse, your example here is talking about an economy node, but since the api only runs as a plugin for velocity/paper you'd have to spin up an entire velocity server with an open port just to manage economy?

a standalone impl would be nice as well

[gloomy gust]

ohh i suppose i mean like redis alternatives, not server platform alternatives...mb

but yeah if you build a plugin on top of pillar I presume you still need the plugin installed on the network?

[blissful gust]

yeah that too, could also support raw UDP and TCP connections, https, http? idk, I feel like there's more than just redis

[gloomy gust]

ohh like expanding this beyond like even just minecraft servers and maybe even using it for web servers...?

[blissful gust]

like it would open an https server and you can handle the open port as a web server

[blissful gust]

sure I mean why not, the only real reason I see in using a library like this is because it would be write once run everywhere, with then ofc the nice api as a bonus

[gloomy gust]

I suppose I only saw it for its redis capabilities (i'm currently fighting my own homemade redis system and i'm wondering if this would be an adequate outsource that lowkey solves our server communication problems), but now I see its use for general multi-server applications

although to that point, are there not other proven tools/frameworks/whatever that are made for those specific use cases (e.g., web server, streaming service, etc)?

[blissful gust]

btw MessageContext.java could hold a util method respond(PillarMessage<T> type, T payload) where the serverId would be the sending server

[balmy copper]

yeah, for sure. in future versions I plan support folia, purpur (already partly the case), probably Fabric, and others too. I'm still refining the version, but the plan is to expand as much as possible and according to demand

[blissful gust]

anyways the codebase looks strong, good job, if I ever get a network big enough I'll use it xd

[balmy copper]

that's a good point, I'd like to explain why redis:

so, redis is the choice because it solves the infrastructure problems of servers talking to each other, it's good to recover messages that get lost along the way, and it kinda knows what to do when two servers try to do the same thing at the same time

the economy thing was an example, and maybe not a great one on my part since it wasn't that clear, but the idea is that you can send messages to servers that have specific responsibilities and they handle those tasks from there

[blissful gust]

redis is not the only protocol that does that

tcp has promised delivery and order of packets

and https is based on tcp

I'm not saying redis is a bad option, I'm saying that there's other protocols that can do the same and your api could support those too flawlessly with a simple abstraction layer.

[balmy copper]

that's interesting, but look: tcp might not solve in all cases

if a server goes down and a message over tcp fails, normally there's no persistence or resend once it comes back. the pillar ideia is to handle that through redis streams, where the message persists and picks back up from where it left off if something goes wrong

well, I like redis, it has good native tools like scripts, keys and some things like that. anyway, I could considerer that abstraction layer in future

[tough canyon]

There are a lot more message brokers. Postgres could also be fun or grpc

[gilded tide]

Can it use plugin msg? :D

[tough canyon]

Doesn't that defeat the purpose of having such system?

[gilded tide]

Useful for testing environment and supporting Redis alongside Plugin Msg. There are still many people who prefer Plug Mag for whatever reason. Then I could just use that but recommend setting it up with Redis

[balmy copper]

hmm... nope, the whole idea of Pillar is to not use it. it uses redis instead, since has persistence and is useful for messages that you don't want to lose, or when you need an good way to communicate

[tough canyon]

plugin messages should not be used for cross server communication anyway. I know many misuse it that way but that doesn't make it right

[sage hatch]

yeah plugin messages require a player on the server to receive it, so if you want server to server communication for servers without players you just can't have it