Would like to open this up for possible third party conversation regarding paper plugins. Please keep outside talk out of here.
#Paper Plugins
1 messages · Page 1 of 1 (latest)
I can't type here at all /j
Yay awesome!
The main topic right now is what we should do about classloading. Currently spigot allows you to access plugin classes even if not explicitly defined as a dependency. We want to make it so this behavior is more strict on paper plugins, and instead only allow them to resolve dependencies.
So feedback and opinions are welcome.
See: #paper-dev message
For some context
plz don't touch it, thx
Care to explain a bit more?
Obviously the biggest issues are PAPI and vault, but we need to figure out if it’s possible to somehow have a nicer way to support what they do.
remember when Spigot added the warning to later implement this exact case and people lost their minds about it? This would be like the apocalypse for everyone who want's to use Vault wrongly.
Also what is even the benefit supposed to be here?
You can still access everything if you really want to...
The advantage is properly encapsulating plugins, cause less iteration over plugin classloaders and in general promote better practices here.
Wasn't that supposed to break plugins like Vault?
A reminder that spigot plugins will stay the same here.
Not sure how you want to encapsulate plugins. Pretty sure the JVM doesn't support that.
One plugin that would be affected by this heavily is PlaceholderAPI, so my feedback is mostly around the case of this plugin in particular.
First of can it not define every plugin an expansion exists for as a dependency. With the amount of expansions would this result in a dependency list with entries up to several hundreds if not even thousands of plugins, which simply isn't good.
Next electroniccat's suggestion of just making the expansions be separate plugins.
This isn't optimal either, especially for the server owner:
- It fills the plugins folder with tons of plugins that only serve a single purpose. Organizing that is a nightmare.
- Plugins like that are just bloat.
My personal ideas so far would be one of these:
- Add a system that allows a dynamic defining of dependencies at the plugin's load/enable/runtime. PAPI could then check expansions for specific info (i.e. required plugins) and define it like that during its loading.
- Allow plugins who depend on a plugin, that those dependencies can class load them. This could allow an expansion to hook into a plugin which depends on PAPI. Downside is obvious: Expansions of plugins that don't depend on PAPI (i.e. Vault) would not work.
Personally, aproach 1 is more optimal.
Maybe I used the wrong terminology here, but we just want to split up classloaders properly and respect their actual dependency trees
For PAPI Extensions the solution is very simple: Make them plugins but load them from the extensions folder?
that would still fill the /plugins list, maybe non-bootstrap dependencies could be added by the bootstrap(per)
I mean I'm all for this but I don't see any benefits that are big enough to justify the amount of work a proper dependency system would require.
(Not to mention the whole issue of the ServiceProvider API being a thing)
Only other idea I have is, that a plugin could define a folder for "semi-plugins" (Jars that are treated as plugins but wouldn't show up in the plugins list)... Or define their own loader stuff or whatever...
How do you plan on differentiating between Spigot and Paper plugins?
If it would also make classpath scanning easier (e.g. ClassGraph) I'm all for it, last time I tried sth I believe it did found classes from other plugins without limiting it to some specific jar
mb if I pinged
There will be a paper-plugin.yml
it shows a @ before their username in the reply if it pinged
Maybe a lower-level api to access the class loaders of other plugins would help. This way, you can build a more complex system if you want, but "normal" plugins would still be separated
pretty sure you can't stop anyone from getting all class loaders that are avialable anyways
But - I don't know what the plans are in terms of reloading and stuff, which would be even more painful with such API
With unsafe/reflections you can even change 1 Integer's wrapper value to 0, so if someone really wants sth they will do it
afai understand we're talking about teaching people some good practice
Just remove the reload at that point imo
It has been discouraged for a while now and Paper iirc did have you confirm the reload anyways...
Oh yeah reloading is yikies. That’ll be another convo 🙂
Yeet reloading?
Paper plugins will most likely not be reloadable anyways. Can’t reload bootstrappers at least.
But, for now talking about this classloader stuff is better.
I mean, having a central point to reload a plugin might be a good idea too, but with clear restrictions (e.g. the bootstrap stuff is called again, but you can't exchange jar files, etc), but also a lot of work I guess
It’s possible that we can open up the classloader api, but, I’m not sure.
Another possibility, what if a plugin could mark itself as needing to be global? As in, instead of marking of a plugin can use OTHER peoples classloaders what if a plugin could define if it wants its classloader to be available in ALL OTHER plugins. It’s kinda a hack but eh.
@modest barn With your point, it’s an issue because placeholder api doesn’t explicitly define its extensions as dependencies right?
The issue with PAPI is that you can't add dependency-declarations at runtime
Not sure how PAPI handles it right now but I think just like in Bentobox (I think) if a plugin has some expansions they should perhaps be in their plugin folder to not clog stuff up, that would probably need an API though
Also defining as global would only solve very specific issues imo. Maybe if you define a service provider but in that case it could just happen automatically? As a service should be available for everyone.
Yes. Expansions can hook into plugins and since PlaceholderAPI loads those expansions, it's treated as PAPI not having the dependencies defined
Yeah. A plugin could define itself as a global service thing or whatever...
But like if we were wanting to go that far, I really wonder if there is a better way to have plugins define services for other plugins
What's currently happening if an expansion is loaded that then tries to load from some plugin (because it depends on it), but the plugin isn't present?
Expansions should define required plugins which PAPI checks for before trying to load it
So if the expansion is made properly will PAPI not load it if a required plugin is missing
Alternatively can expansions override the canRegister() method to return true/false based on whatever, and PAPI acts based on the result
Okay, so you basically have a dependency graph of the expansion before actually loading it?
But that's beyond the topic here I would say
What if a parent plugin could have a method on their JavaPlugin like registerSth(Plugin childPlugin) and the child plugin would just have the parent plugin defined somewhere in the plugin.yml? That way maybe the parent plugin can still provide their stuff for the child plugin and can still know it got registered
Default implementation could throw an exception to fail-fast if the parent doesn't implement it, but that's probably to discuss if it's actually a viable solution
I see. Yeah, so basically loading the extensions with a class loader that can access the class loaders of the required plugins wouldn't exactly work as it seems, because the loading happens before the requirement check
@modest barn i just wanna make sure I fully understand here. So since plugins can register extensions, the issue is PAPI won’t be able to access the plugin but the plugin can access PAPI?
I feel like you missunderstood the situation? Idk
Yeah I haven’t used papi much tbh
Plugins can have expansions yeah, but then they most likely have PlaceholderAPI defined as their (soft)dependency so that won't be the issue
The issue is with stand-alone expansions (Separate jar files) from PAPI's eCloud. Those can hook into other plugins (See the Vault expansion as an example) while not being a plugin themself
AHH I see
And since PAPI is loading the expansion, is Spigot/Paper treating it as PAPI accessing another plugin without defining it as a depend or whatever
So the extension doesn’t really specifically say it’s any plugin, it’s kinda just in the middle ground and relies on the global classloader behavior in order for it to properly resolve classes?
This here shows some of PAPIs loading behaviour.
How an external expansion hooks into a plugin is completely up to them. It's nothing PAPI checks or controls here
Expansions are just separate and not really a plugin at all.
I don't get where you got this idea they would be
yeah and at that point they rely on the currently global classloader behavior
The only difference is, if it is a separate jar, or part of an actual plugin (Which then would be registered by the plugin using the expansion's register method)
Yes but these separate jars need to have access to all the classloaders to properly resolve them correct?
I appreciate you explaining here x)
The only thing I know is, that we have a FileUtils.findClass(...) method which creates a new URLClassLoader for loading stuff
I have no idea to be honest... I only participate in PAPI and while I have a lot of knowledge about it (Honestly consider myself the core maintainer at this point) is it still something I do not get
The class loading stuff was made by another Guy and I have 0 idea what the understanding for all of it is...
Hmm okay well I appreciate it regardless
I know for sure that this is where PAPI is loading those separate jars (Not those inside plugins. They are registered by the plugin) and from what I can tell is it using a separate URLClassLoader here I guess.
And I know for a fact that forks such as Pufferfish had caused issues by cloasing the loader of PAPI
And another thing that is sure is, that Spigot is still triggered by expansions hooking into plugins, so not sure if that gives any clues
Either way maybe ask @solemn galleon here or @craggy sable as they for sure have more knowledge on this
Looks like it sets a parent classloader for the extension tho
That makes sense
Yeah maybe if we expose a global classloader y’all can use that would work?
I donno obviously that’s very technical
Would prefer PlaceholderAPI not instantly break on everything Paper related.
Though I'll be honest I have not read this thread.
If you can remind me in a week, I can. I graduate in 5 days so this is hell week for me.
Loool
Yeah no it won’t break
This is for paper plugins only
That would work I guess, passing it as parent for the URLClassLoader should do the trick
I don't really get the benefit of separating plugin classes. It seems like it's something that will break things and cause headaches, without any obvious benefits
I'm sure it's more neat and orderly and whatever from a software design pov. But realistically, I really don't get why it needs to be done
So my opinion would be "pls don't touch it" as well
I mean, we commonly see issues from plugins and shading libraries
hence why you have to relocate stuff, bu, relocation is not always possible or viable
and then you end up with cases where, say for example, the Kotlin std; which cannot always bre relocated, all of a sudden you have two plugins practically sharing classes between them; We added some prioritisation to mitigate this, but it's far from perfect due to how classloading works, and then you end up with joyous linkage issues on random plugin updates
The classloader isolation stuff will generally always need some form of "get out of jail" card, for things like debugging plugins which need the ability to access a wider scope, so, we can't always offer isolation
BUT, if we're going to the levels of creating a new system so that plugins can actually do things like manipulate registries and do the stuff that datapacks do, it makes sense to try to adopt this level of seperation which has been desired for years, especially if we can also work on the other things; I do want 3rd party repo handling, so imagine being able to use a gradle plugin or whatever in replacement of shade/shadow; which will generate your paper-plugin.yml, populate libraries/repo metadata, and it magically just works
That sure is nicer than the current way of doing it, but it doesn't really allow for anything new. Just nicer workflow. And imo that's not really enough to break existing behaviour
We're not breaking existing behavior
Nothing will change with plugins using plugin.yml
So it's effectively opt-in? Then it's fine with me
Yes
I mean, creating a more limited system than the existing one can't really be the goal? I feel like it should at least be able to provide the same kind of functionality as the bukkit plugin system. (Of course it doesn't need to do it the same way...)
it can
The only restriction by default is that it won't be seeing every other plugins classloader, here will be an option to not have that limitation, however
I've yet to see an explanation how that will work with the service provider
service provider?
I assume the idea is to just expose everything inside the same dependency tree to each other? So e.g. every plugin that provides a vault economy would be exposed to everyone who depends on vault?
I guess the fact that there are no services shipped in the api actually helps there lol
I don't think that they'd be grouped like that
Plugins would depend on vault and get the vault interfaces, etc
The implementation behind that shouldn't matter
Well but the implementation behind it does matter for the question of whether or not its classloader should be exposed to you...
Because that would be necessary in that case and should be considered/explained how to do it right so that the mess that Vault is nowadays doesn't happen again 👀
Biggest issue for things like Vault is the lack of a lifecycle for plugins
idk if we're aiming to resolve that or if there is any good capacity to; ideally we'd throw a few calls or events or whatever and set guidelines for when to do stuff; like, ideally for Vault, you'd have 1 stage where it's expected that plugins will generatelly register their own providers, and nobody should be reading from the system, and then you'd have another point at which in time you'd expect to be able to safely get stuff from it without worrying that something else is going to register itself
I mean, the current system works well too if people use it properly 🤷 Just listen to the service provider (un)register events to get notified of changes.
Main issue in my eyes is that the priorities of the service providers are specified by the dev and can't be changed by the server admin to what they actually need
Yea, outside of the default impls theres basically no entire issue outside of the thing just being old
The relocation issues are a good point. Obviously the real only issues with this isolation has to be plugins that actually use this wacky behavior. For 90% of plugins, I don’t think an isolated classloader would be an issue. Working on some kind of way for these plugins to still somehow handle things would be nice.
An api to expose some global classloader is possible perhaps. The whole standalone extension thing PAPI has could be solved as long as there is still some way to get a “unified” clsssloader.
Is that just an issue with PAPI, or in general with plugins that load .jars during runtime?
It loads stand-alone jars during runtime, not tied to any plugin.
It’s just a jar that contains an extension class
soft-utilizes basically solves my concerns. I have a bunch of stuff where I deliberately don't list the plugins in plugin.yml to avoid the other plugins doing something goofy, and just hook into them either during my load (if already loaded) or as they load with the plugin enable event. Keeps things simpler when I don't care about order and don't trust the 3rd party dev. 😄
Plus this gives me some fun ideas about being able to easily split out my FactionsUUID legacy support 😄
But I mean, what papi does is legit impossible to support if we want to isolate classloaders in any form.
And although yes we can offer some api for a global combined classloader or something, I question WHAT we even want to encourage.
I don't understand why you'd want to restrict papi from being able to do it. Like, sure, isolate by default but allow this.getServer().getPluginManager().canIPleasePokeThisPlugin("PluginName") when it loads extensions and let it hug other plugins. ❤️
i’m not sure that’d be possible
I don’t think you can just add other classloaders as parents when the classloader for papi has already been constructed
unless there’s another way, not sure, i’m kinda rusty
We're already using custom classloaders, can do whatever you want with them more or less
I basically want to see a "Yo dawg I herd you like classloaders so we put a classloader in your classloader so you can load classes while you load classes"
Something different I can't recall getting a reply for:
Any chance that the api-version setting in the Paper-plugin.yml can be a list?
I feel like it's a bit annoying if a dev wants to indicate that their plugin supports multiple versions, but can only define old ones.
Like if I have set 1.19 and plugin is used on 1.20 is the server complaining that the plugin "uses an outdated/unussported" version. But (from what I know. Never tested it) when I set to 1.20 and plugin is used on 1.19, the plugin won't load(?)
Like it limits the plugin to either not define a version, resulting in warnings that it doesn't define one, or having it be considered "outdated" despite it supporting latest version, but having an older version listed.
Alternative (And to kinda keep compatability? Idk) could a format similar to what mod loaders allow be used like f.e. >= 1.19
Also, while at this topic of versions.... I just had this come to mind for me:
Maybe in the future allow to define a dependency with a specific version range? Like if your plugin only supports v3 of plugin X, have a way to set it as (soft)depend and only for v3+ (So that older versions result in the plugin not loading or smth...)
Not sure why I had this idea, but here you go.
No need for api-version to exist
Tell that md_5 :^)
Bytecode stuff?
I can understand it to an extend. Plugin devs may want to define a plugin as only working for one version onwards (Maybe due to missing features in old ones) without having to implement their own entire version check BS
The thing doesn'teven care about the api-version
Like, the literal only logic around that thing is "is it set"
if not, it does a bunch of legacy BS logic
otherwise, it just has some hard bytecode transforms for renamed stuff
Another random thing I currently think about is, if it was of any use to provide a way for plugins to define an API version they themself provide. Like when f.e. Luckperms enables, that it could tell paper "this is my (major) api version!" so that other plugins could retrieve it through that... Tho at that point could the plugin just expose the version throught the API itself anyways... 
The stuff I think about when I'm hungry... See you after diner perhaps.
@modest barn I’m certainly planning on implementing your dependency changes, I do like your new format you proposed.
versions are a little tricky, I defo see a benefit for fine tuning the version list but I’m not 100% sure how to do so nicely here. And well, if it’s even needed.
Lastly I wanna bring up the open classloader again. I think that the majority of plugins are simply not going to need this open behavior, and in general, the niche behavior that you guys are relying on is quite advanced in its nature. However, it’s very much possible to do the same thing because paper will still have a “global” classloader stored that you guys will be able to tap into.
So basically, you’ll just need to reflect into the classloader storage and call a method and you’ll be able to resolve classes.
For vaults case… obviously it’ll have to be a little different. If plugins simply make their own vault api implementations, what they would have to do is say that they provide Vault in their paper-plugin.yml… I think?
So in general very technical stuff will still be possible, but this will get rid of the problems with relocating dependencies and so much more.
Ahh, but they all correctly say they “depend” on vault correct?
issue is that Vault generally has some headaches, and so it creates some issues; i.e. plugins basically need to depend on vault to get a thing from it
BUT, also, some of them basically do stupid stuff by tryna load from Vault early so that they can register their own stuff
it's an entire mess
So in general do you think that it’s better to ward people away from doing whatever vault does with paper plugins?
Vault is imho broken by design
Issue is that without a lifecycles mechanism it's somewhat of a headache to solve
The thing I cannot stress enough is that spigot plugins will work identically to how they have before, but I don’t wanna create a mess by starting new standards we haven’t fully thought out yet.
I’d love to further explain on this, wdym?
as said, basically, ideally something like vault woul have a stage in which it broadcasts out and says "GIVE ME YAH HOOKS", and then another after that which is like "HOOK ME"
i.e. during "give", Essentials would provide its own eco provider implementation to it
and then in the 2nd phase, stuff should be registered, and so it should be safe to get stuff
I see,
I mean, ofc, the proper solution would be that plugins query the service manager on demand, or implement some form of caching which allows for updates to occur
Vault is really something that we just need somebody to make a sane modern replacement for
Plugins have a method JavaPlugin#onLoad they can override which allows code to be run before on enable but after all dependencies should be loaded too
Is using an event for lifecycles kinda yucky then?
Not really, events are fine
I mean, really it's a case of ensuring that stuff is in play
i.e. you can't call events in onLoad iirc
it's worth noting, that Vault on itself, i.e. the API of it is fine
Primary issue is that it bundles its own providers, which causes headaches due to dep order, etc: https://github.com/MilkBowl/Vault/issues/789
Besides the bundled stuff, I think the way the impl can be accessed using the service api from spigot isn't that bad. Plugins only need to depend on Vault and do everything else works by using the events
Biggest issue is that their default ipml guide is broken af
Nobody uses the services API properly, and so a good chunk of the bs with vault is people working around that
yeah the suggested setup is broken
I've been meaning to look into writing a better alternative for a while, like, iirc the services API has events for when stuff is in/registered which is defi something you should listen to if you're gonna cache it
Maybe with the new format define a version key that contains the min version.
Preferably would it allow patterns to be defined. Some examples:
1.0.xSupports every minor version of 1.0>=1.0.0Supports every version starting from 1.0.0<2.0.0Supports every version below 2.0.0<2.0.0,>=1.1.xSupports every version from 1.1.x onwards that is below v22.x,1.xSupports all versions in v1 and v2
This is inspired by the version thing fabric and forge mods allow which I find interesting.
The only issue I can see is, that this would require the plugin to follow a SemVer structure which isn't guaranteed.
An alternative could be FlexVer which would support more variants of a version, but obviously not all.
Just a quick mockup to give a visual idea:
dependencies:
- name: SomePlugin
required: true
version: '>=2.x' # Requires version above or equal to 2.x
Alternative idea could be a structure you may know from tools and/or sites like jsdelivr where you have the patter name@version with some stuff like m taking latest version m into account.
For example SomePlugin@2 would mean "supports all version in Major version 2". Tho how to define certain version ranges would be the big question here.
That's what I do in ChestShop and will be the suggested way to hook into Tresor 👀
I recall Vault dev mentioning that somewhere as the recommended way... Idk. Has been to long to remember correctly
@modest barn I've implemented your new dependency syntax.
The tricky thing is that inorder to make it so that bootstrapping/during runtime have separate dependencies i've kinda added support for dynamically adding/removing dependencies. So, this will prolly be something you will need too. 🙂
The only thing I am also not sure on, what if we want separate LIBRARIES during bootstrapping too? In general, i'm leaning towards perhaps making an entirely separate file for the bootstrapper.
The bootstrapper/server plugin classloader is shared right now, but, I don't know if that's the best thing honestly.
Thought about this again. Bootstrapping dependencies is good but I don’t want to make too much of a distinction since they are still technically shared.
Bit unrelated to this stuff, but does Paper provide/expose the Brigardier library? Maybe even with Lucko's commodore one?
Could perhaps be an idea to allow defining command syntaxes in the paper-plugin.yml so that the command has native brigardier support. The question would be how... Read a .commodore file from the resources? A String to define the syntax using [], <> and alike to define optional, required and literals?
I feel like it would be a huge benefit to have a native way to define brigardier syntaxes in a user-friendly way for paper-plugins... Tho that's more a "nice to have" than a required thing, so maybe for a future update.
@modest barn idk if I want plugins to be able to define commands in plugin yml snymore. Brig api is coming soon, that’s another PR x)
Not sure why it copied the link as issues
Yeah. Feel like defining it through commands or a dedicated file would be better
Tbh I think it’s better to register during runtime.
Just adopt cloud
So looking and talking about this again. (I hope I didn't forget this convo)
But papi won't break at all as long as it sticks as a spigot plugin.
Yeah, but I feel like it would have its benefits of being a native paper plugin
For once no shading of already available stuff like Adventure
Then in that case, all you need to do is for https://github.com/PlaceholderAPI/PlaceholderAPI/blob/46d9a695346e989c768cc8a2fb9c58e1a7d6a025/src/main/java/me/clip/placeholderapi/util/FileUtil.java#L45
instead of passing the plugin classloader you will want to reflect into a paper plugin class that basically is a global classloader.
@modest barn If you wanna prompt any conversation regarding the links.
I certainly see benefit 👍
Yeah
I'm in favour of a simple <name>: <url> aproach that accepts certain names as issues links for errors.
Like when f.e. issue or wiki is set as name would it use that URL in certain exceptions.
Tho, that could become a bit complicated if you want to include multiple variants (i.e. issue and issues)
Only other idea I have is having it as a list like this:
links:
- url: https://issues.example.com
type: issues # Tells Paper what type of URL this would be
it's a paper contrib thread about a PR which is already merged, what are you doing here?
go to #paper-help
After playing around for a bit with the API, I have a very small proposal: expose in the PluginProviderContext a Path pointing to the plugin jar file
My use case? I am bundling dependencies inside the jar file (cuz shading stinks when certain things conflict), and I want to extract them into tmp to add to the classpath in the pluginloader.
In my specific case, by accessing them from a FileSystem for the plugin jar (zip), and Files.list/copy etc the dependencies, but the same could be done with the regular ZipFile API; but to do that first, one needs to access the Path/File to the jar file in question.
That sounds doable pretty easily.
Acknowledging that asking for an ETA is a nightmare: Is there an approximate hoped-for timeline on making paper plugins not fully experimental? I'd love to release a paper plugin but I don't want to release something that's going to result in users yelling "OMG IT STOPPED WORKING" some random day a commit breaks something like the dependency or load order formatting.
e.g. "days" "weeks" "months", not looking for more than that. Just trying to sort out what I should do. I want to be one of the first paper plugins and to support this sort of thing but I don't feel comfortable doing it yet. ❤️