#Paper Plugins

1 messages · Page 1 of 1 (latest)

vague mason
#

@modest barn Seeing if it lets you type here…

modest barn
#

I can't type here at all /j

vague mason
#

Yay awesome!

#

The main topic right now is what we should do about classloading. Currently spigot allows you to access plugin classes even if not explicitly defined as a dependency. We want to make it so this behavior is more strict on paper plugins, and instead only allow them to resolve dependencies.

So feedback and opinions are welcome.

odd frost
#

plz don't touch it, thx

vague mason
#

Obviously the biggest issues are PAPI and vault, but we need to figure out if it’s possible to somehow have a nicer way to support what they do.

odd frost
#

remember when Spigot added the warning to later implement this exact case and people lost their minds about it? This would be like the apocalypse for everyone who want's to use Vault wrongly.

#

Also what is even the benefit supposed to be here?

#

You can still access everything if you really want to...

vague mason
#

The advantage is properly encapsulating plugins, cause less iteration over plugin classloaders and in general promote better practices here.

fluid wigeon
vague mason
#

A reminder that spigot plugins will stay the same here.

odd frost
#

Not sure how you want to encapsulate plugins. Pretty sure the JVM doesn't support that.

modest barn
#

One plugin that would be affected by this heavily is PlaceholderAPI, so my feedback is mostly around the case of this plugin in particular.
First of can it not define every plugin an expansion exists for as a dependency. With the amount of expansions would this result in a dependency list with entries up to several hundreds if not even thousands of plugins, which simply isn't good.

Next electroniccat's suggestion of just making the expansions be separate plugins.
This isn't optimal either, especially for the server owner:

  • It fills the plugins folder with tons of plugins that only serve a single purpose. Organizing that is a nightmare.
  • Plugins like that are just bloat.

My personal ideas so far would be one of these:

  • Add a system that allows a dynamic defining of dependencies at the plugin's load/enable/runtime. PAPI could then check expansions for specific info (i.e. required plugins) and define it like that during its loading.
  • Allow plugins who depend on a plugin, that those dependencies can class load them. This could allow an expansion to hook into a plugin which depends on PAPI. Downside is obvious: Expansions of plugins that don't depend on PAPI (i.e. Vault) would not work.

Personally, aproach 1 is more optimal.

vague mason
odd frost
#

For PAPI Extensions the solution is very simple: Make them plugins but load them from the extensions folder?

alpine robin
#

that would still fill the /plugins list, maybe non-bootstrap dependencies could be added by the bootstrap(per)

odd frost
#

I mean I'm all for this but I don't see any benefits that are big enough to justify the amount of work a proper dependency system would require.

#

(Not to mention the whole issue of the ServiceProvider API being a thing)

modest barn
#

Only other idea I have is, that a plugin could define a folder for "semi-plugins" (Jars that are treated as plugins but wouldn't show up in the plugins list)... Or define their own loader stuff or whatever...

fluid wigeon
#

mb if I pinged

modest barn
#

There will be a paper-plugin.yml

alpine robin
tepid gazelle
#

Maybe a lower-level api to access the class loaders of other plugins would help. This way, you can build a more complex system if you want, but "normal" plugins would still be separated

odd frost
#

pretty sure you can't stop anyone from getting all class loaders that are avialable anyways

tepid gazelle
fluid wigeon
modest barn
#

It has been discouraged for a while now and Paper iirc did have you confirm the reload anyways...

vague mason
#

Oh yeah reloading is yikies. That’ll be another convo 🙂

fluid wigeon
#

Yeet reloading?

vague mason
#

Paper plugins will most likely not be reloadable anyways. Can’t reload bootstrappers at least.

#

But, for now talking about this classloader stuff is better.

tepid gazelle
#

I mean, having a central point to reload a plugin might be a good idea too, but with clear restrictions (e.g. the bootstrap stuff is called again, but you can't exchange jar files, etc), but also a lot of work I guess

vague mason
#

It’s possible that we can open up the classloader api, but, I’m not sure.

#

Another possibility, what if a plugin could mark itself as needing to be global? As in, instead of marking of a plugin can use OTHER peoples classloaders what if a plugin could define if it wants its classloader to be available in ALL OTHER plugins. It’s kinda a hack but eh.

#

@modest barn With your point, it’s an issue because placeholder api doesn’t explicitly define its extensions as dependencies right?

odd frost
#

The issue with PAPI is that you can't add dependency-declarations at runtime

fluid wigeon
#

Not sure how PAPI handles it right now but I think just like in Bentobox (I think) if a plugin has some expansions they should perhaps be in their plugin folder to not clog stuff up, that would probably need an API though

odd frost
#

Also defining as global would only solve very specific issues imo. Maybe if you define a service provider but in that case it could just happen automatically? As a service should be available for everyone.

modest barn
modest barn
vague mason
#

But like if we were wanting to go that far, I really wonder if there is a better way to have plugins define services for other plugins

tepid gazelle
modest barn
#

Expansions should define required plugins which PAPI checks for before trying to load it

#

So if the expansion is made properly will PAPI not load it if a required plugin is missing

#

Alternatively can expansions override the canRegister() method to return true/false based on whatever, and PAPI acts based on the result

tepid gazelle
#

Okay, so you basically have a dependency graph of the expansion before actually loading it?

modest barn
#

But that's beyond the topic here I would say

fluid wigeon
# vague mason But like if we were wanting to go that far, I really wonder if there is a better...

What if a parent plugin could have a method on their JavaPlugin like registerSth(Plugin childPlugin) and the child plugin would just have the parent plugin defined somewhere in the plugin.yml? That way maybe the parent plugin can still provide their stuff for the child plugin and can still know it got registered
Default implementation could throw an exception to fail-fast if the parent doesn't implement it, but that's probably to discuss if it's actually a viable solution

tepid gazelle
#

I see. Yeah, so basically loading the extensions with a class loader that can access the class loaders of the required plugins wouldn't exactly work as it seems, because the loading happens before the requirement check

vague mason
#

@modest barn i just wanna make sure I fully understand here. So since plugins can register extensions, the issue is PAPI won’t be able to access the plugin but the plugin can access PAPI?

modest barn
#

I feel like you missunderstood the situation? Idk

vague mason
#

Yeah I haven’t used papi much tbh

modest barn
#

Plugins can have expansions yeah, but then they most likely have PlaceholderAPI defined as their (soft)dependency so that won't be the issue

#

The issue is with stand-alone expansions (Separate jar files) from PAPI's eCloud. Those can hook into other plugins (See the Vault expansion as an example) while not being a plugin themself

vague mason
#

AHH I see

modest barn
#

And since PAPI is loading the expansion, is Spigot/Paper treating it as PAPI accessing another plugin without defining it as a depend or whatever

vague mason
modest barn
#

How an external expansion hooks into a plugin is completely up to them. It's nothing PAPI checks or controls here

modest barn
#

I don't get where you got this idea they would be

tepid gazelle
modest barn
#

The only difference is, if it is a separate jar, or part of an actual plugin (Which then would be registered by the plugin using the expansion's register method)

vague mason
#

Yes but these separate jars need to have access to all the classloaders to properly resolve them correct?

#

I appreciate you explaining here x)

modest barn
#

The only thing I know is, that we have a FileUtils.findClass(...) method which creates a new URLClassLoader for loading stuff

modest barn
#

The class loading stuff was made by another Guy and I have 0 idea what the understanding for all of it is...

vague mason
#

Hmm okay well I appreciate it regardless

modest barn
#

And I know for a fact that forks such as Pufferfish had caused issues by cloasing the loader of PAPI

#

And another thing that is sure is, that Spigot is still triggered by expansions hooking into plugins, so not sure if that gives any clues

#

Either way maybe ask @solemn galleon here or @craggy sable as they for sure have more knowledge on this

vague mason
#

Looks like it sets a parent classloader for the extension tho

#

That makes sense

#

Yeah maybe if we expose a global classloader y’all can use that would work?

#

I donno obviously that’s very technical

craggy sable
#

Would prefer PlaceholderAPI not instantly break on everything Paper related.

#

Though I'll be honest I have not read this thread.

#

If you can remind me in a week, I can. I graduate in 5 days so this is hell week for me.

vague mason
#

Yeah no it won’t break

#

This is for paper plugins only

tepid gazelle
latent shore
#

I don't really get the benefit of separating plugin classes. It seems like it's something that will break things and cause headaches, without any obvious benefits

#

I'm sure it's more neat and orderly and whatever from a software design pov. But realistically, I really don't get why it needs to be done

#

So my opinion would be "pls don't touch it" as well

finite snow
#

I mean, we commonly see issues from plugins and shading libraries

#

hence why you have to relocate stuff, bu, relocation is not always possible or viable

#

and then you end up with cases where, say for example, the Kotlin std; which cannot always bre relocated, all of a sudden you have two plugins practically sharing classes between them; We added some prioritisation to mitigate this, but it's far from perfect due to how classloading works, and then you end up with joyous linkage issues on random plugin updates

#

The classloader isolation stuff will generally always need some form of "get out of jail" card, for things like debugging plugins which need the ability to access a wider scope, so, we can't always offer isolation

#

BUT, if we're going to the levels of creating a new system so that plugins can actually do things like manipulate registries and do the stuff that datapacks do, it makes sense to try to adopt this level of seperation which has been desired for years, especially if we can also work on the other things; I do want 3rd party repo handling, so imagine being able to use a gradle plugin or whatever in replacement of shade/shadow; which will generate your paper-plugin.yml, populate libraries/repo metadata, and it magically just works

latent shore
#

That sure is nicer than the current way of doing it, but it doesn't really allow for anything new. Just nicer workflow. And imo that's not really enough to break existing behaviour

finite snow
#

We're not breaking existing behavior

#

Nothing will change with plugins using plugin.yml

latent shore
#

So it's effectively opt-in? Then it's fine with me

finite snow
#

Yes

odd frost
#

I mean, creating a more limited system than the existing one can't really be the goal? I feel like it should at least be able to provide the same kind of functionality as the bukkit plugin system. (Of course it doesn't need to do it the same way...)

finite snow
#

it can

#

The only restriction by default is that it won't be seeing every other plugins classloader, here will be an option to not have that limitation, however

odd frost
#

I've yet to see an explanation how that will work with the service provider

finite snow
#

service provider?

odd frost
#

I assume the idea is to just expose everything inside the same dependency tree to each other? So e.g. every plugin that provides a vault economy would be exposed to everyone who depends on vault?

#

I guess the fact that there are no services shipped in the api actually helps there lol

finite snow
#

I don't think that they'd be grouped like that

#

Plugins would depend on vault and get the vault interfaces, etc

#

The implementation behind that shouldn't matter

odd frost
#

Well but the implementation behind it does matter for the question of whether or not its classloader should be exposed to you...

#

Because that would be necessary in that case and should be considered/explained how to do it right so that the mess that Vault is nowadays doesn't happen again 👀

finite snow
#

Biggest issue for things like Vault is the lack of a lifecycle for plugins

#

idk if we're aiming to resolve that or if there is any good capacity to; ideally we'd throw a few calls or events or whatever and set guidelines for when to do stuff; like, ideally for Vault, you'd have 1 stage where it's expected that plugins will generatelly register their own providers, and nobody should be reading from the system, and then you'd have another point at which in time you'd expect to be able to safely get stuff from it without worrying that something else is going to register itself

odd frost
#

I mean, the current system works well too if people use it properly 🤷 Just listen to the service provider (un)register events to get notified of changes.

#

Main issue in my eyes is that the priorities of the service providers are specified by the dev and can't be changed by the server admin to what they actually need

finite snow
#

Yea, outside of the default impls theres basically no entire issue outside of the thing just being old

vague mason
#

The relocation issues are a good point. Obviously the real only issues with this isolation has to be plugins that actually use this wacky behavior. For 90% of plugins, I don’t think an isolated classloader would be an issue. Working on some kind of way for these plugins to still somehow handle things would be nice.

An api to expose some global classloader is possible perhaps. The whole standalone extension thing PAPI has could be solved as long as there is still some way to get a “unified” clsssloader.

latent shore
#

Is that just an issue with PAPI, or in general with plugins that load .jars during runtime?

vague mason
#

It’s just a jar that contains an extension class

versed cloak
#

soft-utilizes basically solves my concerns. I have a bunch of stuff where I deliberately don't list the plugins in plugin.yml to avoid the other plugins doing something goofy, and just hook into them either during my load (if already loaded) or as they load with the plugin enable event. Keeps things simpler when I don't care about order and don't trust the 3rd party dev. 😄

Plus this gives me some fun ideas about being able to easily split out my FactionsUUID legacy support 😄

vague mason
#

But I mean, what papi does is legit impossible to support if we want to isolate classloaders in any form.

#

And although yes we can offer some api for a global combined classloader or something, I question WHAT we even want to encourage.

versed cloak
#

I don't understand why you'd want to restrict papi from being able to do it. Like, sure, isolate by default but allow this.getServer().getPluginManager().canIPleasePokeThisPlugin("PluginName") when it loads extensions and let it hug other plugins. ❤️

solemn galleon
#

i’m not sure that’d be possible

#

I don’t think you can just add other classloaders as parents when the classloader for papi has already been constructed

#

unless there’s another way, not sure, i’m kinda rusty

finite snow
#

We're already using custom classloaders, can do whatever you want with them more or less

versed cloak
#

I basically want to see a "Yo dawg I herd you like classloaders so we put a classloader in your classloader so you can load classes while you load classes"

modest barn
#

Something different I can't recall getting a reply for:
Any chance that the api-version setting in the Paper-plugin.yml can be a list?
I feel like it's a bit annoying if a dev wants to indicate that their plugin supports multiple versions, but can only define old ones.

Like if I have set 1.19 and plugin is used on 1.20 is the server complaining that the plugin "uses an outdated/unussported" version. But (from what I know. Never tested it) when I set to 1.20 and plugin is used on 1.19, the plugin won't load(?)

Like it limits the plugin to either not define a version, resulting in warnings that it doesn't define one, or having it be considered "outdated" despite it supporting latest version, but having an older version listed.

Alternative (And to kinda keep compatability? Idk) could a format similar to what mod loaders allow be used like f.e. >= 1.19

#

Also, while at this topic of versions.... I just had this come to mind for me:
Maybe in the future allow to define a dependency with a specific version range? Like if your plugin only supports v3 of plugin X, have a way to set it as (soft)depend and only for v3+ (So that older versions result in the plugin not loading or smth...)
Not sure why I had this idea, but here you go.

finite snow
#

No need for api-version to exist

modest barn
#

Tell that md_5 :^)

vague mason
modest barn
#

I can understand it to an extend. Plugin devs may want to define a plugin as only working for one version onwards (Maybe due to missing features in old ones) without having to implement their own entire version check BS

finite snow
#

The thing doesn'teven care about the api-version

#

Like, the literal only logic around that thing is "is it set"

#

if not, it does a bunch of legacy BS logic

#

otherwise, it just has some hard bytecode transforms for renamed stuff

modest barn
#

Another random thing I currently think about is, if it was of any use to provide a way for plugins to define an API version they themself provide. Like when f.e. Luckperms enables, that it could tell paper "this is my (major) api version!" so that other plugins could retrieve it through that... Tho at that point could the plugin just expose the version throught the API itself anyways... shrug

#

The stuff I think about when I'm hungry... See you after diner perhaps.

vague mason
#

@modest barn I’m certainly planning on implementing your dependency changes, I do like your new format you proposed.

versions are a little tricky, I defo see a benefit for fine tuning the version list but I’m not 100% sure how to do so nicely here. And well, if it’s even needed.

Lastly I wanna bring up the open classloader again. I think that the majority of plugins are simply not going to need this open behavior, and in general, the niche behavior that you guys are relying on is quite advanced in its nature. However, it’s very much possible to do the same thing because paper will still have a “global” classloader stored that you guys will be able to tap into.

So basically, you’ll just need to reflect into the classloader storage and call a method and you’ll be able to resolve classes.

For vaults case… obviously it’ll have to be a little different. If plugins simply make their own vault api implementations, what they would have to do is say that they provide Vault in their paper-plugin.yml… I think?

So in general very technical stuff will still be possible, but this will get rid of the problems with relocating dependencies and so much more.

finite snow
#

they don't "provide" vault

#

they depend on Vault and register their own thing

vague mason
#

Ahh, but they all correctly say they “depend” on vault correct?

finite snow
#

issue is that Vault generally has some headaches, and so it creates some issues; i.e. plugins basically need to depend on vault to get a thing from it

#

BUT, also, some of them basically do stupid stuff by tryna load from Vault early so that they can register their own stuff

#

it's an entire mess

vague mason
#

So in general do you think that it’s better to ward people away from doing whatever vault does with paper plugins?

finite snow
#

Vault is imho broken by design

#

Issue is that without a lifecycles mechanism it's somewhat of a headache to solve

vague mason
#

The thing I cannot stress enough is that spigot plugins will work identically to how they have before, but I don’t wanna create a mess by starting new standards we haven’t fully thought out yet.

vague mason
finite snow
#

as said, basically, ideally something like vault woul have a stage in which it broadcasts out and says "GIVE ME YAH HOOKS", and then another after that which is like "HOOK ME"

#

i.e. during "give", Essentials would provide its own eco provider implementation to it

#

and then in the 2nd phase, stuff should be registered, and so it should be safe to get stuff

vague mason
#

I see,

finite snow
#

I mean, ofc, the proper solution would be that plugins query the service manager on demand, or implement some form of caching which allows for updates to occur

#

Vault is really something that we just need somebody to make a sane modern replacement for

vague mason
#

Plugins have a method JavaPlugin#onLoad they can override which allows code to be run before on enable but after all dependencies should be loaded too

#

Is using an event for lifecycles kinda yucky then?

finite snow
#

Not really, events are fine

#

I mean, really it's a case of ensuring that stuff is in play

#

i.e. you can't call events in onLoad iirc

#

it's worth noting, that Vault on itself, i.e. the API of it is fine

tepid gazelle
#

Besides the bundled stuff, I think the way the impl can be accessed using the service api from spigot isn't that bad. Plugins only need to depend on Vault and do everything else works by using the events

finite snow
#

Biggest issue is that their default ipml guide is broken af

#

Nobody uses the services API properly, and so a good chunk of the bs with vault is people working around that

tepid gazelle
#

yeah the suggested setup is broken

finite snow
#

I've been meaning to look into writing a better alternative for a while, like, iirc the services API has events for when stuff is in/registered which is defi something you should listen to if you're gonna cache it

modest barn
# vague mason <@204232208049766400> I’m certainly planning on implementing your dependency cha...

Maybe with the new format define a version key that contains the min version.

Preferably would it allow patterns to be defined. Some examples:

  • 1.0.x Supports every minor version of 1.0
  • >=1.0.0 Supports every version starting from 1.0.0
  • <2.0.0 Supports every version below 2.0.0
  • <2.0.0,>=1.1.x Supports every version from 1.1.x onwards that is below v2
  • 2.x,1.x Supports all versions in v1 and v2

This is inspired by the version thing fabric and forge mods allow which I find interesting.

The only issue I can see is, that this would require the plugin to follow a SemVer structure which isn't guaranteed.
An alternative could be FlexVer which would support more variants of a version, but obviously not all.

Just a quick mockup to give a visual idea:

dependencies:
  - name: SomePlugin
    required: true
    version: '>=2.x' # Requires version above or equal to 2.x

Alternative idea could be a structure you may know from tools and/or sites like jsdelivr where you have the patter name@version with some stuff like m taking latest version m into account.
For example SomePlugin@2 would mean "supports all version in Major version 2". Tho how to define certain version ranges would be the big question here.

odd frost
modest barn
vague mason
#

@modest barn I've implemented your new dependency syntax.

The tricky thing is that inorder to make it so that bootstrapping/during runtime have separate dependencies i've kinda added support for dynamically adding/removing dependencies. So, this will prolly be something you will need too. 🙂

#

The only thing I am also not sure on, what if we want separate LIBRARIES during bootstrapping too? In general, i'm leaning towards perhaps making an entirely separate file for the bootstrapper.

#

The bootstrapper/server plugin classloader is shared right now, but, I don't know if that's the best thing honestly.

#

Thought about this again. Bootstrapping dependencies is good but I don’t want to make too much of a distinction since they are still technically shared.

modest barn
#

Bit unrelated to this stuff, but does Paper provide/expose the Brigardier library? Maybe even with Lucko's commodore one?
Could perhaps be an idea to allow defining command syntaxes in the paper-plugin.yml so that the command has native brigardier support. The question would be how... Read a .commodore file from the resources? A String to define the syntax using [], <> and alike to define optional, required and literals?
I feel like it would be a huge benefit to have a native way to define brigardier syntaxes in a user-friendly way for paper-plugins... Tho that's more a "nice to have" than a required thing, so maybe for a future update.

vague mason
#

@modest barn idk if I want plugins to be able to define commands in plugin yml snymore. Brig api is coming soon, that’s another PR x)

#

Not sure why it copied the link as issues

modest barn
vague mason
versed cloak
#

Just adopt cloud

vague mason
modest barn
#

Yeah, but I feel like it would have its benefits of being a native paper plugin

#

For once no shading of already available stuff like Adventure

vague mason
#

Then in that case, all you need to do is for https://github.com/PlaceholderAPI/PlaceholderAPI/blob/46d9a695346e989c768cc8a2fb9c58e1a7d6a025/src/main/java/me/clip/placeholderapi/util/FileUtil.java#L45
instead of passing the plugin classloader you will want to reflect into a paper plugin class that basically is a global classloader.

GitHub

The best and simplest way to add placeholders to your server! - 1M+ Downloads - 2.5k+ Placeholders - PlaceholderAPI/FileUtil.java at 46d9a695346e989c768cc8a2fb9c58e1a7d6a025 · PlaceholderAPI/Placeh...

vague mason
#

@modest barn If you wanna prompt any conversation regarding the links.

#

I certainly see benefit 👍

modest barn
#

Yeah

#

I'm in favour of a simple <name>: <url> aproach that accepts certain names as issues links for errors.
Like when f.e. issue or wiki is set as name would it use that URL in certain exceptions.

#

Tho, that could become a bit complicated if you want to include multiple variants (i.e. issue and issues)

#

Only other idea I have is having it as a list like this:

links:
  - url: https://issues.example.com
    type: issues # Tells Paper what type of URL this would be
edgy rose
#

it's a paper contrib thread about a PR which is already merged, what are you doing here?

near dagger
#

Ok.

#

tx for the redirect. New in the server and thought this was the right place

idle vigil
#

After playing around for a bit with the API, I have a very small proposal: expose in the PluginProviderContext a Path pointing to the plugin jar file
My use case? I am bundling dependencies inside the jar file (cuz shading stinks when certain things conflict), and I want to extract them into tmp to add to the classpath in the pluginloader.
In my specific case, by accessing them from a FileSystem for the plugin jar (zip), and Files.list/copy etc the dependencies, but the same could be done with the regular ZipFile API; but to do that first, one needs to access the Path/File to the jar file in question.

balmy trench
#

That sounds doable pretty easily.

versed cloak
#

Acknowledging that asking for an ETA is a nightmare: Is there an approximate hoped-for timeline on making paper plugins not fully experimental? I'd love to release a paper plugin but I don't want to release something that's going to result in users yelling "OMG IT STOPPED WORKING" some random day a commit breaks something like the dependency or load order formatting.

#

e.g. "days" "weeks" "months", not looking for more than that. Just trying to sort out what I should do. I want to be one of the first paper plugins and to support this sort of thing but I don't feel comfortable doing it yet. ❤️