Gradle takes ages to end

[ashen silo]

<@&987246399047479336> please have a look, thanks.

[orchid geode]

gosh dammit

i didnt mean to do that

basically, i have a gradle task that extracts a jar:

// extract jar
        try (var jar = new JarFile(jarPath.toFile())) {
            Enumeration<JarEntry> entries = jar.entries();
            while (entries.hasMoreElements()) {
                JarEntry entry = entries.nextElement();
                Path destination = outputDir.resolve(entry.getName());
                if (entry.isDirectory()) {
                    Files.createDirectories(destination);
                } else {
                    Files.createDirectories(destination.getParent());
                    Files.copy(jar.getInputStream(entry), destination);
                }
                
                System.out.printf("Extracted %s%n", entry.getName());
            }

            System.out.println("Extracted jar!");
        } catch (IOException exception) {
            exception.printStackTrace();
            throw new IllegalStateException("Failed to extract jar!", exception);
        }```

and it hangs for roughly 1 minute after Extracted jar! is printed

[orchid geode]

i've also noticed, the time it hangs is definitely proportionate to the amount of files its copying

[delicate rock]

Well, extracting a zip may take some time

[orchid geode]

yeah but its already extracted the files

it hangs afterwards

i mean im fairly confident that it must have to do with closing either the jar file or the jar entry input streams

[ashen silo]

@orchid geode

Your question has been closed due to inactivity.

If it was not resolved yet, feel free to just post a message below
to reopen it, or create a new thread.

Note that usually the reason for nobody calling back is that your
question may have been not well asked and hence no one felt confident
enough answering.

When you reopen the thread, try to use your time to improve the quality
of the question by elaborating, providing details, context, all relevant code
snippets, any errors you are getting, concrete examples and perhaps also some
screenshots. Share your attempt, explain the expected results and compare
them to the current results.

Also try to make the information easily accessible by sharing code
or assignment descriptions directly on Discord, not behind a link or
PDF-file; provide some guidance for long code snippets and ensure
the code is well formatted and has syntax highlighting. Kindly read through
https://stackoverflow.com/help/how-to-ask for more.

With enough info, someone knows the answer for sure 👍

[orchid geode]

bump

[wanton bloom]

Seems it's doing quite a bit of cleanup at method run() in class CleanableResource in class ZipFile

Dunno how that would hang for so long though...

Probably also depends on the file size, since it 'loads' all the contents and has to discard the unused stuff (at least from what I can see in my jdk)

This is like the only reference I could find of a proper usage

Are you sure the code after it isn't causing this issue?

[orchid geode]

there is no code after it

so no, its nothing to do with the code after it

i mean im not sure if theres anything i can actually do about this

this is basically what its doing

[severe spindle]

Maybe xou should try to use workers

[orchid geode]

what's a worker?

[trim wing]

do you perhaps start up a thread pool/use futures?

that is all I can think of

[grand portal]

which statement does it hang on?

[orchid geode]

None of them, it's hanging on a separate thread I believe

[ashen silo]

@orchid geode

Your question has been closed due to inactivity.

If it was not resolved yet, feel free to just post a message below
to reopen it, or create a new thread.

Note that usually the reason for nobody calling back is that your
question may have been not well asked and hence no one felt confident
enough answering.

When you reopen the thread, try to use your time to improve the quality
of the question by elaborating, providing details, context, all relevant code
snippets, any errors you are getting, concrete examples and perhaps also some
screenshots. Share your attempt, explain the expected results and compare
them to the current results.

Also try to make the information easily accessible by sharing code
or assignment descriptions directly on Discord, not behind a link or
PDF-file; provide some guidance for long code snippets and ensure
the code is well formatted and has syntax highlighting. Kindly read through
https://stackoverflow.com/help/how-to-ask for more.

With enough info, someone knows the answer for sure 👍

[orchid geode]

so i'm back to this again after having left it for ages. i tried to do some inspecting using visualvm, and i have found that the issue probably pertains to gradle trying to cache things

this is the jfr

and the reason i believe its gradle caching stuff is because gradle has a lot of cache threads running at that time

im thinking this one specifically. cuz im extracting a jar with ~22k files. so adding that many hashes would definitely take a bit of time i assume

(this is all just an assumption though, cuz i dont really know what im looking at lmfao)

[grand portal]

why does your jar have 22k files?

[orchid geode]

well. minecraft is pretty big

has a lot of stuff

so thats why

[grand portal]

are you packaging every asset too? 22k files still seems pretty big

[orchid geode]

yeah it has all the assets too 💀

well not all of them technically but yeah

[grand portal]

assets are typically downloaded. when you launch a game, the client receives assets from a server. they're downloaded once, then updated as needed

[orchid geode]

nah

well yeah that happens with some assets

like sounds

but the rest come packaged with the jar

[grand portal]

22k files worth? doesn't seem right

[orchid geode]

there isnt really anything i can do to strip them out

[grand portal]

what are you doing?

[grand portal]

where did you get your code base from?

[orchid geode]

mojang

[grand portal]

send the github link

i cant find anything like that released by mojang

[orchid geode]

no, its from their piston meta

its not on github 💀

that'd be against their own eula

but anyways, just trust me lol. 22k files is correct

22.6mb jar file

i mean its not really too much of a problem, because ideally this only needs to happen once per minecraft version

but it'd just be nice to try and cut it down a bit cuz it seems unnecessarily long especially since if you extract the jar normally, it takes like 3-5 seconds total

[delicate rock]

1min for 22k files sound like a good score

[delicate rock]

hmm

and what if you use ZipFile instead ?

[orchid geode]

ok well that takes 9 seconds to finish extracting

and then after that it still appears to be hanging

so its basically about 6x faster at the actual extracting

and then it still hangs for over a minute afterwards

problem obviously being, the next gradle task has to wait for that one to finish before it can start (even though from everything i can tell, the files are all extracted perfectly fine after those first 10 seconds)

[delicate rock]

@orchid geode can you try with ZipFile instead ?

[orchid geode]

yeah that is with zip file

[delicate rock]

and it's the same ?

[orchid geode]

static void extractArchive(Path archiveFile, Path destPath) {
    try (var archive = new ZipFile(archiveFile.toFile())) {
        Files.createDirectories(destPath);

        List<? extends ZipEntry> entries = archive.stream()
                .sorted(Comparator.comparing(ZipEntry::getName))
                .toList();

        for (ZipEntry entry : entries) {
            Path entryDest = destPath.resolve(entry.getName());

            if (entry.isDirectory()) {
                Files.createDirectory(entryDest);
                continue;
            } else if (Files.notExists(entryDest.getParent())) { // have to do this for MANIFEST.MF for some reason
                Files.createDirectories(entryDest.getParent());
            }

            Files.copy(archive.getInputStream(entry), entryDest);
        }
    } catch (IOException exception) {
        throw new IllegalStateException("Failed to extract archive!", exception);
    }
}```

[delicate rock]

And same performence problem right ?

[orchid geode]

yes

[delicate rock]

it's very fast but takes ages to close ?

then

can you try ZipInputStream ?

@orchid geode

[orchid geode]

er

i have no idea how to use zip input stream tbh

oh nvm i was just being an idiot

[delicate rock]

It's very similar

[orchid geode]

seems to have the same problem

[delicate rock]

so it's the closing which is so slow ?

[orchid geode]

yeah. after 8 seconds it finishes extracting and then it takes 85 seconds before it ends

[delicate rock]

can you show where you put the prints ?

[orchid geode]

its after that method is called

    // extract archive
    long start = System.currentTimeMillis();
    ExtractServerTask.extractArchive(jarPath, outputDir);
    System.out.println("Extracted jar in " + (System.currentTimeMillis() - start) / 1000 + " seconds!");
}```

[delicate rock]

wait

but it already ended here

the jar is closed here right ?

[orchid geode]

i mean i assume so yeah, since the try-with-resources would have exited

[delicate rock]

so the problem isn't the unziping

(Also don't use currentTimeMillis but nanoTime)

[orchid geode]

does it matter?

[delicate rock]

It might, it's not the cause of the problem here

but you should switch to nanoTime

[orchid geode]

even if im only using a resolution of seconds?

that seems unnecessary

[vernal nimbus]

currentTimeMillis()

<:Class:871140776900440074> java.base/java.lang.System

public static native long currentTimeMillis()

Returns the current time in milliseconds. Note that while the unit of time of the return value is a millisecond, the granularity of the value depends on the underlying operating system and may be larger. For example, many operating systems measure time in units of tens of milliseconds.

See the description of the class Date for a discussion of slight discrepancies that may arise between "computer time" and coordinated universal time (UTC).

see

Date

return

the difference, measured in milliseconds, between the current time and midnight, January 1, 1970 UTC.

✨ Click the white name at the top to open the online javadoc.

[orchid geode]

interesting

i'll take that into account for future

but yeah its obviously not the cause of the problem here lol

[delicate rock]

i mean, no it's wrong

but

anyway

like the doc said, use nano

also

note that in java, you can do / 1_000_000_000

this is valid

[orchid geode]

yeah ik

[delicate rock]

so yea, try to see the source of the problem

you should run a profiler

[orchid geode]

well i tried that. but profilers dont really like gradle from what i can see

[delicate rock]

-_-

then go blind and try to see the method causing this problem

[orchid geode]

(this is the profile report)

[orchid geode]

im convinced its probably an internal gradle thing. im not sure i'll ever be able to find that

i might have to ask on the dreaded stack overflow and see if any gradle nerds on there know whats causing it

[delicate rock]

hmm

can you try to disable your antivirus on your project ?

[orchid geode]

well i dont have any antivirus 💀

apart from windows defender ofc

[delicate rock]

yea windows defender

usually intellij asks you automatically to disable it

[orchid geode]

yeah i think i clicked that. not sure how i check though

[delicate rock]

also be sure that you are not in one drive

[orchid geode]

definitely not. i hate onedrive with a passion

breaks absolutely everything

im tempted to move my gradle cache onto my ssd and see if it makes a big difference

that would tell me if its some big io operations

i mean the jfr said there wasnt any io operations. but tbh i dont really trust that profiler stuff

especially since this file seems to be getting proportionally bigger

[small cave]

If this is on windows then it's a possible file lock issue. If you run with -no-daemon option what does it do?

[delicate rock]

@orchid geode maybe ⬆️

[ashen silo]

Activities have been reset.

Gradle takes ages to end

Changed the title to Gradle takes ages to end.

[delicate rock]

I changed the title so it matches your issue better

[ashen silo]

Changed the category to Build Tools.

<@&987246554085740594> please have a look, thanks.

[orchid geode]

nah that doesnt seem to make a difference unfortunately

[orchid geode]

ok so yeah its 100% a gradle caching thing or whatever. because if i move it to somewhere outside of the gradle caches directory, it finishes much quicker

so really, the question is, how do i stop gradle from doing whatever its doing to the files? lmao

[small cave]

This really sounds like a gradle Worker that doesn't release its handle on a file. If it works in WSL or linux box then you know this is the case. You never showed the full gradle task or if you are using weird plugins.

[orchid geode]

public abstract class ExtractClientTask extends DefaultTask {
    @Input
    public abstract Property<String> getVersion();

    @InputDirectory
    @Optional
    public abstract DirectoryProperty getJarDir();

    @OutputDirectory
    @Optional
    public abstract DirectoryProperty getOutputDir();

    @TaskAction
    public void extractJar() {
        System.out.println("Extracting jar!");

        Path versionPath = getJarDir()
                .getOrElse(getProject()
                        .getLayout()
                        .getBuildDirectory()
                        .dir("minecraft")
                        .get())
                .getAsFile()
                .toPath()
                .resolve(getVersion().get());
        if (Files.notExists(versionPath))
            throw new IllegalStateException("Version directory does not exist!");

        Path jarPath = versionPath.resolve("client.jar");
        if (Files.notExists(jarPath))
            throw new IllegalStateException("Client jar does not exist!");

        // get or else use user home dir
        Path outputDir = java.util.Optional.ofNullable(getOutputDir()
                .getOrNull())
                .map(Directory::getAsFile)
                .map(File::toPath)
                .orElse(Path.of(System.getProperty("user.home"), ".testGradle"))
                .resolve(getVersion().get())
                .resolve("client");
        if (Files.exists(outputDir))
            ExtractServerTask.deleteDirectory(outputDir);

        try {
            Files.createDirectories(outputDir);
        } catch (IOException exception) {
            throw new IllegalStateException("Failed to create output directory!", exception);
        }

        // extract archive
        long start = System.nanoTime();
        ExtractServerTask.extractArchive(jarPath, outputDir);
        System.out.println("Extracted jar in " + (System.nanoTime() - start) / 1_000_000_000 + " seconds!");
    }
}

thats the whole task

its not really anything special

these are the plugins im using:

plugins {
    id 'java-gradle-plugin'
    id 'maven-publish'
    id 'org.gradle.gradle-profiler' version '0.0.1'
    id 'idea'
}```

[small cave]

Using gradle long?

[orchid geode]

not really. im pretty new to it

i only started making plugins and tasks and stuff with it in october and its been very on and off since then

[small cave]

Generally you don't need to do this, you just make a jar task in your build.gradle

Does it work in linux?

Plus you can delete your gradle cache anytime. That sometimes is a fix.

You're basically reinventing the wheel here.

[orchid geode]

idk. i dont really have a linux machine to hand

[orchid geode]

wdym? theres an existing task to extract jars?

[small cave]

yes

I think you should read some gradle documentation.

There's a task for just about everything.

[orchid geode]

the gradle docs aren't exactly great 💀

[small cave]

Yep, gradle is for people that are willing to invest a lot of pointless wasted time and effort. You should see their source code.

I knew Groovy before I started using gradle, which was a long time ago so I suspect it's much harder now that Groovy is out of mainstream. They support kotlin too but less example of how to do stuff in that.

My advice, search github for gradle builds that do what you are also doing.

But do figure out the docs first, you'll always have to make sure you see a date somewhere because they've changed stuff so much. Lot of old examples that don't work anymore.

[orchid geode]

oh that worked actually

that was much faster

[small cave]

Just for some context, I've been using gradle since it came out and I've never once had to define a DefaultTask in a java file ever.

[delicate rock]

what did you do ?

[orchid geode]

damn

well i didnt use the jar task actually. but i did this

getProject().copy(copySpec -> {
    copySpec.from(getProject().zipTree(jarPath.toFile()));
    copySpec.into(outputDir.toFile());
});```

[orchid geode]

also, i think most of the tasks i have are not gonna be something gradle has by default

but rn everything im doing is kinda just a proof of concept rather than anything final. cuz im probably gonna come back to it and rewrite it all in groovy or kotlin once im done

[small cave]

There is already a jar task in your build file. You can override that one by just saying

      // override inputs, outputs, etc
}```

Gradle is declarative, you don't have to tell it how to make a jar.

The point of doing it this way is so the build system knows when input and outputs change, so you don't have to redo every task.

[orchid geode]

well the problem is the input depends in the previous task though

[trim wing]

curious

you said you don't think gradle is gonna have most of what you need by default

what are you doing?

[orchid geode]

making a mod loader for minecraft pretty much

so like. the plugin downloads the jars, remaps them, decompiles, applies patches, other remappings, all the other stuff

i mean idk maybe there are tasks for some of the simpler things i already have such as downloading and parsing the piston meta. but i dont really see how that could be the case

[wanton bloom]

paperweight does about all of that

for the plugin side of mc

[orchid geode]

yeah but i prefer to write it myself tbh. its more fun

and a good learning experience

[wanton bloom]

fair enough- could've known that
||ya making a video series on that? /j||

could take inspiration I guess

it's quite a task

[orchid geode]

plus, i've got it working already in the past using MML (forge's mod launcher stuff)

[orchid geode]

heh. something like this is probably out of scope for any video

its hard to even boil it down to simple steps

[wanton bloom]

haha yeah joking, just funny to see you here

[wanton bloom]

only the crazy people would want to do this
and they would want to do it themselves anyways

[orchid geode]

yeah, no-one seems to have boiled it down to simple steps before me either (at least that i can find). so i'm very much figuring it out as i go along

[wanton bloom]

mhm

but the point of you using gradle is that it can be used by your modloaders mods?

because you could also do something similar to spigot's BuildTools outside of the gradle env

Or, write a prototype without gradle (cmd invoked r smth) and then use that in gradle

Juggling gradle + the actual core logic sounds daunting to me

[orchid geode]

i mean yeah ig thats a possibility. but being in gradle means basically what you said. mods can use it, and when it comes to applying access transformers/wideners or mixins and stuff. its just a lot easier when using gradle

[wanton bloom]

👍 I wish you much luck

[orchid geode]

thanks lol. i'm already pretty far with it tbh

[wanton bloom]

any plans to make it like.. 'production' ready

if you can call MC modding production

or just fun/learning

[orchid geode]

eh. for now its literally just a proof of concept. im not sure if i'll ever go anywhere with it cuz its quite time consuming

[wanton bloom]

hmm can imagine

I'm not that experienced, I wouldn't know where even to start :P

[orchid geode]

but if i do decide to use it in production, i do already have an org setup for it https://github.com/brassmc

lol

[wanton bloom]

xD

that is funny

okay

[orchid geode]

(everything already on there is using MML, not the stuff im writing now)

[orchid geode]

i mean really, the starting bit is just getting the game to run

[wanton bloom]

exactly... 😅
I'm gonna go for a mc clone in Java at some point, to me even plugins' mc interactions are sometimes a pain

partially because of bukkit ig

[orchid geode]

which is a matter of

1. downloading the piston meta
2. parsing the piston meta
3. grabbing the meta for the specific mc version
4. parsing that version's meta
5. downloading the client (specified in the version's meta)
6. downloading the libraries (specified in the version's meta)
7. downloading the assets to the correct hashes and whatnot - from the asset index which you have to parse from the piston meta)
8. running the jar with the libraries in the classpath and the assets in the correct place

[wanton bloom]

hmm

so like

not that hard but just boring ol' work

[orchid geode]

its just a matter of downloading some files and then parsing them

[wanton bloom]

fair

gradle adds more complexity tho

[orchid geode]

but yeah, unfortunately thats the easiest part of the whole process

[gaunt raven]

gradle does not like what developers need to do to minecraft

[wanton bloom]

I'm really surprised paperweight is as stable as it is

[orchid geode]

i've never heard of paperweight before this discussion tbh

ig thats just cuz i havent really touched plugins before. i've always been on the modding side

[wanton bloom]

basically everything mc internals for plugins (server side only ofc)

plugins ecosystem is a mess of garbage rn :P

worse than mods

[orchid geode]

surely cant be worse than mods. cuz thats very very messy atm lmao

[wanton bloom]

eeeh

the plugin community is very okay

but the code aint

maybe that's just my perception from digging too much into internals

[orchid geode]

ah fair enough

[wanton bloom]

I wanna make a MC clone

at some point

and see myself make the same mess

then I'll know I shouldn't complain

until then I will be screaming for MC public modding API 😆

[orchid geode]

i mean i initially started this whole thing cuz quilt came out and then silica was announced and i was like fuck it, everyone else is making mod loaders, it cant be that hard, would be a fun challenge to do it myself. and 3 years later, here i am

(not that its taken me 3 years, i've just been distracted with other stuff)

[wanton bloom]

I mean quilt did cheat

[orchid geode]

sounds fun tbh. but idk, a lot of work

what r u gonna use? lwjgl? libgdx? jaylib?

[wanton bloom]

I'm building the empire by making pure Java libraries that can be used in my plugins, and eventually in the clone

[orchid geode]

or directx or vulkan or something?

[wanton bloom]

I would love to try vulkan

lwjgl easy route.. uh maybe

'easy'

wait but

lwjgl has vulkan

bindings I believe

[orchid geode]

does it?

oh wait yeah ofc it does

mb lmao

i somehow completely forgot about that

[wanton bloom]

yeah my research is not at the front of my brain rn
I am basically sleeping

but yeah lwjgl probably

no reason not to

[orchid geode]

you looked into libgdx at all?

[wanton bloom]

yes

but

the word 'framework' makes me scared

haven't seen implementation in it tho

[orchid geode]

ehhh. yeah i mean its a little bloated. but idk it does have a lot of useful stuff

you can still do all of the raw stuff

(granted i've only used it twice for small dumb projects)

[wanton bloom]

hmm fair

I'll note it

[grave ravine]

tim

[orchid geode]

although ig it might be better built for non-procedural games

[wanton bloom]

and apps in general it seems

as they show in the Features tab

[orchid geode]

anyways, i think we're kinda going off-track

[wanton bloom]

very much so

[orchid geode]

and this problem is solved now

so thanks to everyone that helped

[wanton bloom]

I just made another problem by asking you stuff >:P