Reforger Workshop Crawler

[harsh night]

After a lot of hard work, we are happy to finally announce the release of Reforger Workshop Crawler.
Inspired by the original workshop crawler for A3 and DayZ, we've invested time and resources into this passion project, that helps everyone protect their hard work and IP.

Just look up your mod by its name or by its ID, select it and wait see who broke your license and reuploaded it.

https://reforger.workshopcrawler.com/ <-- this will take you there

If you select a big mod, it might take a while, up to 20 seconds.

There's definitively gonna be quirks, we didn't catch during our testing. Feel free to report them here.

We have a way to whitelist a hash, so if you think a hash should be whitelisted, like an empty file, basic file etc, please add your suggestions.

Known issues:
Addons removed from the workshop are still listed in the DB and returned - fix will most probably come this upcoming week (depends on backend guy time availability)

[prime gorge]

Love this

Initially reading this, I genuinely thought 'ah shit, someone's beat me to the punch' as I am making a program that archives assets

I do hope Enfusion Workbench really does push some of these assholes away reuploading entire workshop content, although, there's a lot to be desired with Workbenches limitations for modding mods

[final quest]

the other side of the argument to is alot of people in the community dont read what each one means kinda why i always ask before do

[harsh night]

But... Why would you need this?
You can just run the server executable and have it download it for you

If by assets you do not mean something else than the entire mod that is

[woven swallow]

Interesting, currently on mobile due to being at work, but when "duplicates" are found, what is this referring to?

Ahh, so it would still pick up if someone has a similar override as a duplicate I see.

Would be interesting with scripts, I imagine that would be a little more involved with searching

[mossy mist]

Cool project, can you make it only check against actual source files and not inheriting from vanilla? Lots of false positives on localization files and commonly inherited base prefabs due to this

[harsh night]

I could theoretically whitelist all the files that are in the vanilla game.
Not a bad idea, will look into it later this week

[mossy mist]

Yeah I think stuff inheriting from vanilla would only be relevant to show if the hashes match, otherwise 99% of the time it's just someone else inheriting from the same vanilla resource but for something completely different

[harsh night]

It only does show files that hash match

[mossy mist]

ah ok so I guess there's just a ton of mods with empty localizaiton.en_us.conf then 😅

Do you mean hash match on the source file file they are inheriting?
Because there is NO way all of these world files have the same hash as mine

It seems to match any mod that inherits from a file with a hash match, even if the inheriting file itself doesn't match

Oh wait it's probably just comparing the actual .ent file and not associated world layers so I guess that makes sense

In that case might be worth omitting .ent and comparing layers instead or something because every one will be a false positive when the files look like this:

SubScene {
 Parent "{5D71F496D28056E0}worlds/MP/CTI_Campaign_Cain.ent"
}

[prime gorge]

My ideas isn't so much about finding duplicates.

But I want to archive all the meta data of every assets, prefabs.. ect in every mod in every version.

My idea isn't for mods and version, but actually each item.

If I want to make X mission, using X, Y, Z mods with X, Y, Z versions and I want to know, what guns are avaliable, what's in the loadouts already, ect.

I shouldn't need to go through a clunky Workbench UI; I want it neatly presented in front of me.
"What damage does a gun do?" / "What items does a loadout have?"

Just click it, list accepted bullets / mags (With the mods you have selected)

• Mine isn't just for developers.
• It's meant to be for users curious about why a gun isn't doing the damage it should.
• Or mission makes trying to understand what has changed.
• Maybe a modder would like a tabled overview of the stats they're going to inherit for another mod.
• Maybe a server admin wants to make sure that a specific version of a mod doesn't contain a version of a 7.62 magazine that goes in a gun. (Real thing by the way)

I was also... Debating on using this system to then... Maybe create simple configuration mods with just a few clicks and drags

• Want to re-pack a mag from one mod with another mods ammunition because it's broken?
• Maybe a mission-maker has realised a gun that was added doesn't have valid ammunition and needs to create one quick

I wont keep dragging, but I do love this, because it'd definately help in reverse searching mods.

If a unit is using mods that clearly aren't there's, I can use this to reverse search that.

[timid dawn]

This is great for the Reforger team to see how susceptible to hash collision their resource ID system is

[harsh night]

Yeah, it compares the actual files.
You can filter out .ent files yourself for now. I'll look into it

[mossy mist]

Yeah all good mate I realised my mistake haha 😅 I think since the engine kinda treats the world layer files as an extension of the world .ent file, maybe it would make sense to group them together?
Possibly could represent it as a heirarchy with the layers nested under the world file so the world .ent is like a "folder" containing the layers? Would probably make it a bit easier to visually grep than listing layers separately

[harsh night]

It isn't using Reforger system
The actual packed files are read, hashed and saved.

[timid dawn]

Ah ok so I have some hash collisions on textures etc

But some of them are normal like empty texture or default emat

[harsh night]

Send over hashes of these empty ones, I'll whitelist them

[timid dawn]

b598b371bc8f9d76acc25c2cea4df6ce83912cc5af9478db7cc74b3312172cb661ffd1fa3f8bc5e4f175a2910a3a7afe992d9154726c818f25310e712d2dfd77
c8462306867e0c706f24fa9c80b28872e7538ecc779c912f9ee6beacbb719ea391c5324e337d3489d026243ba51accc2d28d7ff420d8586e11a800272cc331f2

[untold stump]

Thread blocked due to unauthorized usage of our online services.

[harsh night]

we are so back

[unique hollow]

So glad to see you back

[timber plank]

Not sure if it's just me or if it is because I use a VPN 24/7, but searching anything or directly clicking on a mod in the search seems to search for addons forever:

[harsh night]

which addon did you search for?
there's currently a bug, that will not update the UI if it returned a "no duplicates found" case

[timber plank]

RHS, and then my own mod to see if I was just bugged.

Neither loaded though.

[harsh night]

RHS just loaded for me

[timber plank]

I came back to it just now and attempted RHS and it loaded; i'd just chalk it up to me using a VPN.

If I have any further errors or issues, I'll highlight them here in an attempt to provide bug fixes; unless you have a github I can submit reports at. :)

[harsh night]

wouldn't say VPN would cause an issue, not doing any blocking
if you find anything else, send it here please, I will look into the UI hopefully today

(And you can blame the slow DB I am using currently for RHS taking ages to load, now it is fast because someone already spent a minute waiting for it and indexes are cached)

[harsh night]

fixed on local, will update later tonight

[harsh night]

Well, the frontend release will have to wait a bit.
Updates to the DB are currently paused, old data will stay available.

A lot of the architecture was changed, sometime tomorrow/two days from now, it will be switched to a newer version. Migration is in progress

[harsh night]

new system vs old system, guess which is which
a request returning duplicate addons for ACE Carrying