Unreal Essentials

[echo current]

i will in a bit

i was on VR

im back on desktop

https://danu.is-ne.at/6pFakiGOW

https://danu.is-ne.at/6pFaDtpYd

[modest shale]

IS THAT UM JAMMER LAMMY

[boreal fiber]

It parses your Steam Library, finds the AppId for your game, drops a steam_appid.txt, and also hooks 2 functions related to rebooting/checking if need to reboot

Off the top of my head

Of course, if there is non-Steam related logic for rebooting, it won't handle it

[wild abyss]

Yeah, I had a look and I don't think steam is the problem (or if it is, it's not the only problem). I edited the source to force steamhook to work (just hardcoded the paths) and the reboot still happened

[boreal fiber]

oop

[echo current]

its a mod

[rough rose]

yeah, no, HiFi devs just trolled a little too hard

that said, I assume the HiFi specific fix can still be applied globally?

[wild abyss]

Yeah

It's just writing 0s to the struct every time the function is called. There's no harm in doing that for all games

[wild abyss]

I still haven't worked out the reboot problem yet but I did find where the branch name is. This might be useful for more specific matching of signatures, I'm not really sure. For example, I now know that Scarlet Nexus is built on ++UE4+Release-4.25Plus M3

The signature for it in Scarlet Nexus is E8 ?? ?? ?? ?? 45 33 ED 4C 8B F0 4C 89 6D ?? 41 8B C5 if either of you want to find it in other versions. You might also be able to just search for the string UE4+Release, hopefully that's a consistent part of it

[dusky steppe]

BuildSettings::GetBranchName right?/

[wild abyss]

Yeah

[dusky steppe]

++UE4+Release-4.27

[wild abyss]

Actually if that start of the name is consistent it'd probably be easier to just scan for that. Code may change between versions

Then instead of those other values we could use this string which should be more helpful (and hopefully actually work with all the things)

[dusky steppe]

tested with all my normal 4.27 games (GBVSR, Ishin and Stray) and they're all reporting the same string

[wild abyss]

Nice

[dusky steppe]

hifi rush is being a troublemaker as usual though

[wild abyss]

😔

What's it report?

[dusky steppe]

i haven't gotten the sig to match

[rough rose]

always the troublemaker

[wild abyss]

Did you try just searching for the string

Surely they wouldn't change the branch name that much

[dusky steppe]

[rough rose]

[wild abyss]

What about a less specific one

UE4+Release or something

[dusky steppe]

nothing so far, wow

[wild abyss]

bruh

[dusky steppe]

tried bsimming the caller and no good matches so far

[wild abyss]

There were a few functions that call it, maybe another one will give you better luck

[rough rose]

well, at the end of the day this means its no more consistent than the older method

because you'll still run into the HiFi rushes of the UE4

[wild abyss]

I think it is a bit better, with the last versions like 4.25+ weren't distinguished from 4.25.

[dusky steppe]

oh hello there

[wild abyss]

bruh

[dusky steppe]

oh i love this game

[wild abyss]

I guess I can search for both uppercase and lowercase ++ue4+ and it'd work

Oh wait

Is that assigned at runtime

bruhhhh

wtf are they doing

I see it now lol

[dusky steppe]

the full name's ++ue4+hibiki_patch+4.27hbk

i found it stored as a global somewhere

[wild abyss]

Oh, so I could search for ++ue4+?

[dusky steppe]

that would work in this circumstance

[wild abyss]

Cool, well I'll make something that does that and we can see if it works

[patent belfry]

I’m not a modding expert but this seems like a real pain in the ass to make work lol

I wish y’all luck tho!! Not sure how close to working this is but I believe in you guys

[wild abyss]

It's not that close tbh. I've been getting sidetracked with things like getting the version and fixing the reboot

It'll get done though (I hope)

[dusky steppe]

man I wonder what FF7R reports since that's heavy modified too

[wild abyss]

Damn, only 15 days until reload

I think it should be done by then, depends on how side tracked I get (and whether I get a job in that time which would nuke all my free time)

[rough rose]

well, scarlet nexus is 4.25+ and the sigs from 4.25 still worked but i get what you mean

IO Store format change moment

[wild abyss]

They actually didn't lol

[rough rose]

what about this

[wild abyss]

I used bsim to get scarlet nexus sigs from 4.25

That sig is from scarlet nexus, not the 4.25 project.

I think there was one that worked between the two but most I've had to use bsim. When I change to this new version string I'll update the sigs I have to not be lies

They were all really close but not identical

[rough rose]

whats the pinned one for 4.25 then

thats the one i used for scarlet nexus

[wild abyss]

It's from scarlet nexus, so I guess it should be ++UE4+Release-4.25Plus M3, I'll get the actual ones for 4.25 in a bit and update you with what they should be

[rough rose]

i see i see

what the fuck is plus M3 lmao

[wild abyss]

idk

[rough rose]

they only list a single 25 plus on epic

[wild abyss]

I guess M3 is what they added to denote their modifications

[patent belfry]

The cheering u on dance

[dusky steppe]

i'm still actively working on my own thing, if that ever ends up being needed

[wild abyss]

True, that might be our saviour. I guess we'll see

[patent belfry]

I do wonder how easy it’ll be to mod music for this game considering that the DLC BGM can actually be used to create an actual playlist to shuffle between

Assuming it can be modded, I mean

[frosty terrace]

i'll try to help out as much as i can

i have a decent amount of experience with ue modding, as deathchaos can attest to

[rough rose]

oh hi there

who kidnapped you here

[wild abyss]

Oh, hello. Sewer mentioned I should reach out to you

(I then proceeded to not do that :D)

[dusky steppe]

[wild abyss]

Could one of you check that this still works for hi-fi rush

[patent belfry]

I’m simply here to cheer on (I do not know the first thing about doing this shit)

[rough rose]

its ok, I simply showed up and inserted myself into the help without asking

[marble orchid]

ive just been lurking

[dusky steppe]

the spectators ‼️

[wild abyss]

I appreciate your help, I probably would've pinged you eventually if you hadn't shown up on your own

[patent belfry]

FemSpectator

[rough rose]

tbf this would help a lot of people if successful

[patent belfry]

no I’m not coping about femc not being in reload what’re you talking about

[dusky steppe]

yes (it works)

[marble orchid]

i...probably should try and do something to train myself in this sort of thing, but...need to clear ssd space...

[ebon shard]

Same; don't really have much to say in here, at least nothing that would be helpful at all

when it comes to threads like this with active work being done I personally try to avoid "useless" comments ig just to avoid being a potential bother or anything unless there's a really good joke opportunity sometimes I can't help myself then)

[brave sinew]

I'd love to try and help out but I'm stuck under 5 tons of imposter syndrome

[patent belfry]

See the most I could do is provide testing muscle for Hi-Fi Rush and even then I mainly use Steam Deck (though I imagine knowing whether or not it works on that would probably be beneficial)

[marble orchid]

mood

[frosty terrace]

it was sewer

[rough rose]

ah

the criminal is at it again i see

[frosty terrace]

real

[dusky steppe]

*sewer56 sonic riders gif*

[wild abyss]

The actual signature for 4.25 is E8 ?? ?? ?? ?? 44 39 60 ?? 74 ?? 48 8B 08 48 85 C9 btw.

[rough rose]

wait thats the one i got from KOF XV

lmao

[wild abyss]

What version is it?

[frosty terrace]

4.26

[wild abyss]

But you've got the 4.26 one listed as E8 ?? ?? ?? ?? 48 8B D8 39 78 ??

Is that wrong?

[rough rose]

odd

are we getting another hifi moment

[wild abyss]

[rough rose]

ill check again after i finish this bsim

[rough rose]

shouldnt be

thats what i got from "gartenofswine.exe"

[wild abyss]

Hmm ok

Could you check if the KOF XV branch is different?

I hope it is otherwise branch isn't a consistent way to deal with signatures either

[rough rose]

give me like 10 days

[wild abyss]

(Run the game with Unreal Essentials and it'll print it out)

[frosty terrace]

ah i see yall are using bsim now

i just found out about it yesterday

[wild abyss]

Yeah, bsim is awesome

[dusky steppe]

yea it was our christmas present from the nsa

[frosty terrace]

i usually use ida but bsim is a very strong argument to make me switch to ghidra permanently

[rough rose]

i hate how ida handles making structs so i always stuck to ghidra

and i also dont have to download potential malware to get it

[frosty terrace]

lmao.

and yeah making structs in ida sucks

[rough rose]

the math isnt mathing i think

[wild abyss]

rounding™️

[modest shale]

Decompiling FUN!!! 😃

[rough rose]

kof

[wild abyss]

Huh, ok

Well I guess branch isn't enough 😔

[rough rose]

well, the thing is

the listed sig works

and it takes me to the right spot

but the sigmaker generates a longer one

wonder whats up with that

[wild abyss]

Oh

Maybe that listed one matches multiple things

[rough rose]

well the first match is the correct one

sooooooo

[wild abyss]

Good enough for me

[dusky steppe]

just hope the first match is the right one

[wild abyss]

We'll cross that bridge when we find a game where that isn't the case

[rough rose]

kof gives this

[dusky steppe]

how long is the generated sig anyway?

[rough rose]

E8 ?? ?? ?? ?? 48 8B D8 39 78 ?? 0F 84 ?? ?? ?? ??```

[dusky steppe]

that's fine

at least it's not that one sig we got where it was the entire function lmao

[rough rose]

well, i checked on the default project exe

that sig works too

[wild abyss]

Ok, then I'll just use the longer one

[rough rose]

ok swine

we may have a problem

i checked the listed regular 4.25 sig on DNF Duel

no match

[wild abyss]

😔

[rough rose]

but the one you said was scarlet nexus only

match

E8 ?? ?? ?? ?? 48 8B D8 39 78 ??```

i am 100000% sure this matches across regular 425 and plus

above is dnf duel

below is scarlet nexus

[wild abyss]

Oh, wait. It does match my 4.25 project???

[rough rose]

lmao

i cant believe youd lie to me like this!!!!!!

I will never emotionally recover

[wild abyss]

But it's not in a function (at least not one that's defined)

[frosty terrace]

wacky

[wild abyss]

Which doesn't really make sense, I had symbols for this

[rough rose]

guilty gear strive is also 4.25 but i need to upgrade to open it

[wild abyss]

It's just after a real function, I think it might just be coincidence

[rough rose]

i mean, DNF Duel is regular 4.25

and it works there

oh right i have the demon slayer exe here too from that one time i helped that one person

also 4.25

[wild abyss]

Alright I'll just let you work it out, maybe something is off with my blank project

I'll just put it as whatever you tell me it is

[rough rose]

yeah im suspecting that

well right now its the "right" one

the one that works on both scarlet nexus and DNF Duel

ill check ggst once it finishes upgrading

matched Strive too

matched on Strive arcade exe (which has symbols)

[wild abyss]

On a different topic, I found out what's going on with the rebooting on launch. It turns out it is a problem with steam, but also unreal engine...
In FOnlineSubsystemSteam::InitSteamworksClient bRelaunchInSteam is true, however, making it false doesn't help. i also tried lying to it and returning true in thee function that calls that but no dice. somehow in what i've done i've also lost my ability to use the ctrl and shift keys

[rough rose]

[wild abyss]

Well I restarted my pc and I'm now allowed to use ctrl and shift again. That's something lol

[rough rose]

oh right demon slayer had denuvo

well it works on all the other 4.25 exes I have

so i think its fine

dont need a separate sig for 425+

[wild abyss]

Yeah, I'll believe you

I've switched it to that sig

[rough rose]

do you still plan to mess with the pakfolders function?

[wild abyss]

No, not for now

Maybe later on

[rough rose]

sad!

[dusky steppe]

ik it's something we'd have to get to eventually if we want each mod's assets in their reloaded mod folder

[wild abyss]

Would we? I was thinking it was only so I don't have to place a dummy utoc and ucas in the actual game's files. Instead they'd go in Unreal Essentials' folder

[rough rose]

guess that's my cue to vanish for tonight

[dusky steppe]

that's an option too

[rough rose]

timezones moment

[wild abyss]

Oh, well gn

[dusky steppe]

but yeah i had a look at GetPakFolders and saw that TArray's push function was inlined

so you'd have to reimplement that

at least in my blank project there was a call to TArray::ResizeGrow which handles allocation when length > capacity, and then the stuff that gets inlined is mostly just things like incrementing the length field

the main problem with hooking ResizeGrow is that it's signature's it generated is the entire function lol

48 89 5C 24 ?? 57 48 83 EC 20 48 63 79 ?? 48 8B D9 B9 04 00 00 00 48 8B D7 83 7B ?? 00 74 ?? 48 8D 04 ?? 48 C1 E8 03 48 8D 4F ?? 48 03 C8 EB ?? 48 3B D1 48 0F 47 CA 48 C1 E1 04 33 D2 E8 ?? ?? ?? ?? 48 C1 E8 04 B9 FF FF FF 7F 3B F8 0F 4F C1 89 43 ?? 48 8B 0B 48 85 C9 75 ?? 85 C0 74 ?? 48 63 D0 45 33 C0 48 C1 E2 04 E8 ?? ?? ?? ?? 48 89 03 48 8B 5C 24 ?? 48 83 C4 20 5F C3

[rough rose]

I mean

we're already scanning for a function call for keysdelegate

just scan for another function call here

[wild abyss]

Yeah, when I tried before I realised I'd have to scan for a bunch of functions to make it work since it gets angry if you allocate stuff on your own. I didn't really want to deal with finding all of these for multiple versions

[rough rose]

well if you list them down we could tackle them slowly but surely

[dusky steppe]

my concern is mostly that the length of that sig is so long that it'd be a lot more fragile

[rough rose]

hmm

does ResizeGrow get used anywhere else

[dusky steppe]

thousands of times

[rough rose]

oh baby

then that shouldnt be an issue

go through the xrefs and see which function call has shortest sig

[dusky steppe]

E8 ?? ?? ?? ?? 85 FF 49 8B C6

the first one lol

FString::AppendChars

[rough rose]

ez

[frosty terrace]

couldn't we just use ue4ss

[wild abyss]

What's that?

[frosty terrace]

https://github.com/UE4SS-RE/RE-UE4SS

it contains a modified fork of unreal for stuff like interacting with tarrays/fstrings

it is the tool for code modding unreal games

[wild abyss]

I guess we could copy the necessary code from it

[frosty terrace]

i'd just recommend using ue4ss in general for modding ue4 games

for tarray stuff it should be fine to just reuse the code??

it relies decently heavily on finding gmalloc for interacting with reflected memory though

[wild abyss]

I might look at it for reference but for now, I'd already actually written code for working with TArrays and FStrings yesterday. In theory I think it'd work but ue doesn't like me using c# for allocating stuff, I'll have to scan for the one in ue itself.

[frosty terrace]

fair

you can't allocate memory "regularly" in unreal, you have to work with the fmemory functions via gmalloc

https://github.com/trumank/patternsleuth

here's a useful tool for finding several things from ue games btw

it's a very sophisticated pattern scanner

[wild abyss]

Cool, thanks

[frosty terrace]

here's the code used in ue4ss for finding fmemory::free and consequently gmalloc

[wild abyss]

I guess I'm trying this again. Before I start though I'll say that I found the reboot is caused by steam as I mentioned here. I tried not running FOnlineSubsystemSteam::Initialize at all and just returning true, in that case the game did launch without rebooting but I had no save data, presumably because the steam stuff wasn't initialised. Anything else I tried just caused a reboot or it to not launch at all.
From this just nuking the entire initialisation clearly won't work, I'll have to come back to this later...

[frosty terrace]

ah

[wild abyss]

(That wasn't directed at you btw, I'm sort of just putting my thoughts to text so I don't forget what I was doing)

[frosty terrace]

no i understand dw

[wild abyss]

Thanks for the suggestions so far though, those tools seems like they'll be useful even if I don't directly use them

[frosty terrace]

np

i'm pretty in tune with the general ue modding scene, so if nothing else i'll be helpful there

unreal is a wack ass engine

[boreal fiber]

7.6MB DLL 💀

In any case, not bad

so those guys have many symbols too

damn, I feel cursed, reversing everything from scratch for each game.
I've mostly dealt with native C games, so I never had these kinds of opportunities

I wonder what these guys will do about the ImGui versioning problem, since they let C++ mods use their ImGui.

If they update ImGui, all those C++ mods will break
(Note: Technically speaking they may break even on same version if different compiler is used, due to risk of struct size differences, though I doubt that would happen given how carefully ImGui is made)

Nobody brought that up even in the PR that merges the functionality, I just had a look.

Willing to guess though they'll just update it eventually and break mods the hard way, that's the way people operate in the Unity & Unreal world from what I've seen, they just expect things to break, and as the games themselves get updated and break things, that can sometimes be acceptable.

[wild abyss]

Same, it feels so weird having symbols and source code. It's a really different experience, sometimes it feels like I'd be better off without symbols since it's what I'm used to lol

We come from a different world than unreal modding people lol

[boreal fiber]

Some of us are more spoiled than the others :p

[frosty terrace]

we're lucky cuz unreal is twofold with symbols:

1. reflection (though not everything gets exposed to reflection, it's an opt-in system)
2. the engine itself is source available, so as long as the devs don't fuck with the engine too badly it's pretty easy to find things

[boreal fiber]

Free Engine Symbols for Disasms

Build your own engine, make the symbols, apply symbols to new disassembly

Also standardised unreal file formats across games (with official dev tools)

Free stonks, yeah.

[frosty terrace]

unreal modding is quite a bit more difficult than unity modding, but we're still priveleged

...unless you mod ue3. god help you then.

[boreal fiber]

The easier the engine the less powerful the mod loading solution

runs

(I joke but it seriously always felt that way, there's no incentive when people have it easy)

[lapis parcel]

huh, I was always under the impression modding UE3 was easier

at least judging from the insane amount of mods for Mass Effect Trilogy that alter gameplay elements and aren't purely audiovisual

[boreal fiber]

This was also an UE3 game :)

https://i.imgur.com/ciQjc7V.png

[lapis parcel]

like-- i'm not experienced full disclaimer, but I see a lot more gameplay and story sequence alteration mods in ME alone than... any UE4 game i can possibly think of

maybe it's due to ME's popularity and my bias is showing

I've always been told that kind of mod is not possible in UE4 rule of thumb

[frosty terrace]

ue3 modding is easier in certain ways

especially if the dev supported modding, in which case it's super easy

but in the case of, for example, guilty gear xrd, the engine was heavily customized so no asset mods were possible

since ue3 isn't source available

ue3 modding is very crazy when it's possible, but when it's not possible

yeah

in the ue4 days, engine modifications are rarely as heavy

so you can usually get away with just messing with stock engine versions, and in the worst case scenarios where engine modifications do break stock engine versions you can always make a custom engine build from source

plus, ue4 is just a much more well-understood engine than ue3

[lapis parcel]

i see

[frosty terrace]

i feel like the main thing holding back ue4 modding is the lack of communities surrounding most such games

[lapis parcel]

hopefully reload doesn't suffer that fate

[frosty terrace]

yeah

ue4 modding, at its worst, is about as hard as modding a normal c/c++ game

and at its best, is almost as easy as unity modding

[lapis parcel]

i want reload to be my main game for modding-- but again, GFD modding is so advanced I fear people are gonna dip lmao

[frosty terrace]

gfd?

[lapis parcel]

P5R's engine

[frosty terrace]

ah gotcha

i don't know much about persona modding, i was brought here because i'm an unreal engine modding expert plus i love jrpgs

[lapis parcel]

asset mods for P5R are very straightforward, can also decompile and modify game scripts very easily

[frosty terrace]

asset mods in ue4 are very easy as well, custom formats are incredibly easy to recreate too

ue4ss allows for lua, bp, and c++ modding

interacting with reflection makes modding practically any ue4 game very easy

[lapis parcel]

i need to make documentation my top priority when I get around to modding P3R

[frosty terrace]

the main problem is that most of the things we know now were only advanced enough to be usable within the last two years

so before that ue4 modding was quite... simplistic

[lapis parcel]

i mean ive constantly been told "models, textures and audio is the most you'll see"

i do none of those 🥴

[frosty terrace]

fair

that was the case a while ago

now we're far beyond that point

it doesn't help that a lot of ue4 games have modding scenes filled with people who don't know anything about programming

only recently have i gotten people to make code mods for ue4 arcsys games

https://fxtwitter.com/i/status/1744031827260797051

as an example, introducing frame data tools into gbvsr

[lapis parcel]

well yeah

[frosty terrace]

this was based on my initial work on ggst

it's not surprising, just disappointing imo

there's only three people in my community who make code mods

out of thousands of modders

[lapis parcel]

that's why I'm very focused on documentation

you can't teach everyone but if you can facilitate it, it might make it more appealing

[frosty terrace]

yeah

unfortunately my modding scene has a very distinct lack of documentation, as i'm pretty much. the only person who knows how to do 99% of things

it's mainly on me for being bad at documentation tbh

i hope that by contributing to knowledge here, i'll be able to take some documentation back to my own scene

[lapis parcel]

BF decompilation from previous Persona games was seriously such a blessing that I am mentally preparing myself to lose lol

[frosty terrace]

blueprint disassembly is a thing we have

[lapis parcel]

doesnt it return gibberish?

[frosty terrace]

it's actually fairly readable

https://github.com/trumank/kismet-analyzer

we have a control flow graph type system

[lapis parcel]

oh nice

[frosty terrace]

and most of it is just function calls or basic operations

[wild abyss]

After way too long I've gotten paks loading from Reloaded mods at least somewhat working. It's only for scarlet nexus rn and for some reason the other mod I'm trying to use (simple ps4 ui replacer) will only work if it's the only one that's on.
I'll try and fix that but I'm really happy with the progress (I feel like this took way too long, it's literally just adding strings to a list)

[lapis parcel]

I saw this used for Shin Megami Tensei V https://github.com/tge-was-taken/KismetKompiler

[wild abyss]

(sorry for disrupting your convo)

[lapis parcel]

sorry for hogging the UTOC thread

[frosty terrace]

it's alright, if you want we can move this somewhere else

[frosty terrace]

i have never heard of that before, but if it's like you say and gibberish then kismet analyzer is most likely better

looked at the samples and yeah that's much harder to understand

[lapis parcel]

iirc it was used to modify AI procedures for one of the enemies in the game and nothing else

[frosty terrace]

makes sense

[wild abyss]

If I can get this working universally then I've realised it'd technically be a minimum viable mod. With it people would be able to use Reloadeed for theeir mods even though they'll still have to build utocs and stuff
Obviously it'd be nice to get utoc emulator done but if I can't in time this should do for a while

[frosty terrace]

fair

[lapis parcel]

[frosty terrace]

good work

[lapis parcel]

I don't really intend to mod it day 1, I'm going to dive into the game and vanish for days lol

[frosty terrace]

that's fair yeah

[wild abyss]

Thanks, that pattern scanning thing you linked was pretty helpful for getting gMalloc stuff working

[frosty terrace]

gotcha

for basic asset edits there's also uassetgui

https://github.com/atenfyr/UAssetGUI

[lapis parcel]

i see

[frosty terrace]

https://github.com/atenfyr/UAssetAPI

uassetapi is a c# library for messing with assets

though, i just remembered that asset editing doesn't work with utoc

someone's going to want to get zentools working

https://github.com/Archengius/ZenTools

zentools can convert io store packages to normal cooked packages

but it only supports ue5

[wild abyss]

I've used UnrealUnzen to open Scarlet Nexus utocs. I assumed it'd work for packing them as well but maybe that assumption's wrong
There must be some way to do it since mods exist for the game

[frosty terrace]

i should try unrealunzen

[dusky steppe]

i had a look at unzen and it appears to be possible to use it to pack into a utoc but it requires a manifest file

that's in a json format

at least that's what I've gathered looking at how uecastoc handles the manifest

[wild abyss]

It's weird, only certain mods are consistently working and it seems to depend on their location, not the actual file. Does the path name affect whether files are loaded or not?
Files in scarletNexus.ge1Alisa\Unreal consistently work, those in scarletNexus.ps4UI\Unreal only work if it's the only mod and those in scarletNexus.yuitoKinglyRaiment\Unreal never work. The order I have the mods in Reloaded and whether there are others doesn't seem to matter

[dusky steppe]

when I tested having multiple patch utocs that patched the same file, it would load the one that was in alphabetical order

so AAAA_P.utoc would load it's asset over BBBB_P.utoc

[wild abyss]

Something weird is definitely going on, I renamed the scarletNexus.yuitoKinglyRaiment mod folder to scarletNexus.ge1Alllllllllisaaaaaaaaaaaa (as well as one's with less characters, that was in case it's a length problem) and the two mods (yuito and kasane skins) both work fine together.

[dusky steppe]

oh

[wild abyss]

I didn't touch any of the actual files, just the folder name

[frosty terrace]

hm

in the meantime i am trying to get zentools working with 4.27

since i assume reload is 4.27

[wild abyss]

Pretending stuff isn't broken for now, the two signatures I need if we want to test this with other games are: FPakPlatformFile::GetPakFolders and GMalloc (the global variable), I got that from FMemory::GCreateMalloc but I suppose anywhere that's consistent is fine.
The signatures for scarlet nexus are 48 89 5C 24 ?? 48 89 74 24 ?? 48 89 7C 24 ?? 4C 89 74 24 ?? 55 48 8B EC 48 83 EC 40 48 8D 4D ?? for GetPakFolders and 48 89 05 ?? ?? ?? ?? E8 ?? ?? ?? ?? E8 ?? ?? ?? ?? 84 C0 74 ?? for GMalloc

I don't think I'll do much (if any) more work on this tonight, other than putting in those signatures if anyone finds some.

[rough rose]

I'll get to digging tomorrow maybe, been busy with P5 beta

(friendly advice, dont manually do assembly patching on ps3, it SUCKS!!!!!!)

[dusky steppe]

GetPakFolders
4.27, same as Scarlet Nexus
48 89 5C 24 ?? 48 89 74 24 ?? 57 48 83 EC 40 48 8D 4C 24 ?? - Hi-Fi RUSH

GMalloc
48 89 35 ?? ?? ?? ?? EB ?? 48 8B 3D ?? ?? ?? ?? - 4.27
48 8B 0D ?? ?? ?? ?? 48 85 C9 75 ?? E8 ?? ?? ?? ?? 48 8B 0D ?? ?? ?? ?? 48 8B 01 48 8B D3 FF 50 ?? 48 83 C4 20 - Hi-Fi RUSH (returns several results but they're all pointers to GMalloc)

[hoary basin]

documentation of some sort is one of the highest priority things that should be looked at when it comes to P3R (and honestly Persona modding in general) - it is always going to be so much harder for people to get familiar with things and potentially help out if they don't have a good starting point.
(which I think you address that later on but this was the first message I saw about it lol)

once people start modding this game, we've got sniwe's modding documentation pages that could probably use some filling out lol

[lapis parcel]

ideally i'd make guides even for "simple" tasks, as signalling "hey, this is a thing you CAN do and here's how" is a very big motivator

[boreal fiber]

I'm personally a fan of that approach, for instance this is how I did the Getting Started guide for code mods https://reloaded-project.github.io/Reloaded-II/DevelopmentEnvironmentSetup/

[lapis parcel]

[wild abyss]

Thanks, here's a version with those added.
If anyone wants to test out loading files from Reloaded mods just add Unreal Essentials as a dependency then put your utocs and stuff in the Unreal folder of your mod. It's still the whole utocs, paks, etc so if you download a mod from nexus or whatever you can probably just extract it into there (what I've done so far).

I'll be out for a few hours but it's easy enough to edit the source yourself to add the signatures if moree are found or those don't work (they're just in Signatures.cs)

[wild abyss]

That's roughly what I was aiming for with my modding docs. I wanted everything to be very hands on, lots of pictures showing you what to do, that sort of stuff. My aim was that you would generally not need prior knowledge to go through them (or if you did, resources to gain that should be clearly shown first)

[echo current]

Ill test once I get on

[frosty terrace]

trying to backport zentools to ue4 with

mixed results

main issues right now:

• cooked package summary is busted
• at least one file sets off an assert in zentools at check( SerialDataStart <= ChunkDataEnd );
• i can't figure out how filenames are stored

[wild abyss]

So, I'm looking into how priority works with files to try and fix the weirdness with some mods not working. What I've found out so far:

• UTOCs are loaded in FPakPlatformFile::Mount, specifically when FIoDispatcher::Get().Mount() is called in it. This means utocs will only be loaded if there's a corresponding pak with the same name (this was presumably already well known as it's how mods are formatted)
• The priority is done using a PakOrder variable in here. This is passed into Mount when it's called in FPakPlatformFile::MountAllPakFiles and the initial value is determined by GetPakOrderFromPakFilePath which gives priority in the order: game main pak -> game content -> engine content -> saved dir -> anything else
  • If the pak ends with _P.pak the priority is increased by 100 * CunkVersionNumber (seeemingly if you name a file ..._2_P.pak it would have a version of 2)
• In the case where files have the same priority I'm fairly sure it should go in alphabetical order as there's a call to PakFiles.StableSort in FPakPlatformFile::Mount
• I'm fairly sure utoc priority is identical to paks (which makes sense). It applies in FFileIoStore::Mount when the file is added to the OrderedIoStoreReaders and is stored as Order which is a part of the FIoStoreEnvironment variable.

[wild abyss]

All that said, I'm still not really sure what's going on with mods not working as I described here...

All of the folders I add should have a base priority of 0, and then that's increased to 100 since the paks have _P in them.

[wild abyss]

I'm now logging utoc files being mounted and it turns out it doesn't even try to mount the yuito mod's folder, even when it's the only mod 😕

[wild abyss]

I'm logging both the function for mounting paks and utocs, it doesn't even attempt to load it in either. I'll have to go further up and log stuff from FPakPlatformFile::MountAllPakFiles

[rough rose]

that's super odd, maybe there's a problem in the file discovery function

[wild abyss]

Well, I found the problem. I logged the folders it's searching (separated by a new line) and this is what I got. Something is definitely going wrong with that third line lol

[wild abyss]

Turns out it was a skill issue. I was setting the length and capacity of my FStrings to be the number of bytes where it should be the number of chars (I had it correct in my TArray code). It working with the other mods was just coincidence, they were being allocated somewhere that was zeroed out so it read them fine.

[rough rose]

[wild abyss]

Here's the current version. With it you should actually be able to load mods from Reloaded.
To do so, just put the pak, utoc, and ucas files in an Unreal folder in your mod and set Unreal Essentials as a dependency of it.

[wild abyss]

UwU

[boreal fiber]

uwu!

[wild abyss]

I guess I'll work on the actual utoc emulator now. I just realised there are only 11 days until reload

[echo current]

this progress is already insane

[rough rose]

this is some terrible timing for p3r too because some of us are busy with the p5 betas stuff lmao

[wild abyss]

So true

Good thing I don't care about p5 modding that much

If it was p4g I'd have to drop everything for it lol

[rough rose]

but it means you're mostly alone right now lmao

[wild abyss]

Yep 🙃

[echo current]

p5 (and R) betas revealing themselves specifically to mess with everyone's plans

https://tenor.com/view/breaking-bad-chemistry-experiment-scientific-walter-white-gif-17820266

[hoary basin]

fortunately (or unfortunately?) it probably will never happen just bc the interest isn't there + P4 had its quick development time...

[lapis parcel]

atlus inside job

clearly this just means we need a p3r proto too

[wild abyss]

Definitely unfortunately, I would love a p4(g) prototype 😄

[lapis parcel]

there's debug stuff for fes and 4 its just nowhere near the level of, well, this

[visual bramble]

og p3 debug build is pretty close to this

p5 just has a metric fuck ton of leftover data and dev tools

[boreal fiber]

My plans of finishing rewriting x86 implementation of hooks library this weekend got thwarted too :v

I happened to receive a ROM for a future prototype release ahead of time

[wild abyss]

I love ue 4.25+ source, it doesn't have implementations for FIoDirectoryIndexReader and Writer (unless I'm just blind) so I'm switching to 4.27 for now and I guess I'll just hope it hasn't changed much 🙃

[obsidian moon]

Can this be tested on older unreal engine games?

Like those which only load from pak

[wild abyss]

Yeah, it should work with theem in theory

The function it hooks is used to load both paks and utocs

(Since a utoc is only loaded when it has a corresponding pak of the same name)

[obsidian moon]

Got it. I will try to ask if the other people who also were making unreal engine games mod interested on testing this

[wild abyss]

Cool, the more testing the better

[obsidian moon]

I wonder if idolm@star starlit season was tested. Not too many were working on this game

[boreal fiber]

Well, since the game broke street date, you might be able to get hands on some real files ahead of time. Maybe that might help

[rough rose]

xbox doesnt count lol

actually none of the systems its on would allow filedump aside from pc

[boreal fiber]

You saying you can't get files from a Jailbroken PS5?

[rough rose]

who even has one capable of opening p3r

[boreal fiber]

I dunno, I've not done anything in that space before. Guessing there's cryptographic keys involved that only ship with newer firmware then

[zenith sparrow]

i know a guy with one but i don’t wanna talk with them

[rough rose]

thats always the case with consoles

i.e. switch games need you to update your fw (sometimes re-dump keys) to dump new games

[boreal fiber]

oh the irreparable damage caused by letting the users access the files of a product they bought

how will the economy ever recover??

[obsidian moon]

I don't think not intended early access count lol

[visual bramble]

it woulnt matter console ue games ship with special shit

and afaik, no ones cracked opening ps5 ones lol

[steel fractal]

Cheering on you for the utoc emulator!
Has been a pain dealing with those for other ue games

[wild abyss]

Thanks, I'm hopeful that I can finish it by launch (can't wait for that to backfire on me lol)

To give a bit of a progress update, a few days ago I worked on making an imhex pattern for utoc files and now that I've done that I think I have a good understanding of the format. I'm decently confident that I could build a valid utoc at this point (at least for Scarlet Nexus, I'll need to check if anything changed in 4.27)

I'm going to try and actually make progress on utoc emulator today, we'll see how far I get...

[steel fractal]

Dealing with the workflow of making the utocs with unreal is not the end of the world.
So don't stress too much

[wild abyss]

I was way too ambitious with my plans for today, turns out I don't understand the format as well as I thought. I'll not try and set any plans like that (at least publicly) for now, I'll just work on it and see how it goes.
If it's not finished by reload I'll hopefully just release the current version and that'll do for a while

[steel fractal]

God speed to ya
And even if you don't crack it
If you're able to. It would be a big win if you can handle the chunk numbers of different mods so they don't conflict

[wild abyss]

How does conflicting chunk numbers actually work? When you say that do you mean if two mods tried to replace the files from the same utoc it'd conflict?
Like I have two new utocs in a mods folder and they both edit different files that were originally in pakchunk0-WindowsNoEditor, then only one works?

[frosty terrace]

essentially the one with the higher priority would work, and the other one wouldn't

unless you're asking about something else

[wild abyss]

That's all I was asking

[frosty terrace]

gotcha

priority is determined by pak name

[wild abyss]

I just hadn't seen conflicts for myself

I guess I just happen to have ones from different chunks

[frosty terrace]

it gets weird sometimes when two mods overlap in one area but not another, and they get mixed together essentially

[wild abyss]

So can one utoc file contain stuff from multiple chunks?

[frosty terrace]

there's nothing stopping you from doing such a thing

[wild abyss]

Ok

[frosty terrace]

ue games are generally modded by making one big pak with all of the things you want to replace

and then forcing the pak to have a higher priority

if you know about the ~mods folder in unreal games, there's no implicit behavior in that

[wild abyss]

Cool, that was pretty much the plan

[frosty terrace]

it's just that the name has a good priority and is easy to read

[steel fractal]

When you import content into unreal engine, you assing it to a chunk number when you make the ucas files
So you can have both a mesh and a texture contained in fx chunk 22
But if another mod wants to mod a music track. And also assing it to chunk 22
Then there will be a conflict

[frosty terrace]

does renaming the pakchunk not work

that's what i usually do

[wild abyss]

It sounds like nothing's forcing you to use any particular chunk so couldn't everyone just be careful not to use one someone else has? (In theory that is, I realise that might be hard in practice)

[frosty terrace]

also i'm fairly sure that you can just. rename the pakchunk and at that point the chunk index doesnt matter anymore?

so like instead of pakchunk22-WindowsNoEditor, just call it MyMod

after you package

[wild abyss]

When you say rename the pakchunk do you mean something in unreal when building or the literal file?

[frosty terrace]

the literal file

[wild abyss]

Hmm ok, I don't really see why conflicts would be a problem then

[frosty terrace]

like, i've never seen the pak name have any hardcoded behavior

[wild abyss]

I'm fairly sure it doesn't. The only thing is checking for _P for higher priority

[frosty terrace]

the other one is that the closer to the end of the alphabet, the more native priority it has

_P just makes a pak have priority over all paks that don't end in _P

[wild abyss]

Yeah but if I understood the code correctly you can add a number before for even higher priority

so _999_P would have 9990 priority

[frosty terrace]

you are correct

i was just simplifying

[steel fractal]

The default unreal engine only allows up to 300 chunks

So might be a problem might not

[frosty terrace]

oh i wasn't aware of that

[steel fractal]

For the Hogwarts Legacy game they made a costum engine instance to remove that limit

So overlap was less likely

I believe that Menv's UEcastoc was supposed to take the mod name's hash as a chunkid when packing
So practically impossible for overlap.
But not sure how much success that had

[frosty terrace]

oh yeah, i'm working on implementing c# mods into ue4ss

so look forward to that future

[hollow glacier]

So how's development coming along

[azure gorge]

What do you mean? It's basically all in this thread. Last "progress update" is here:

Cruel Alchemist

[Reply to:](#1195558171377422346 message) So how's development coming along

AnimatedSwine

To give a bit of a progress update, a few days ago I worked on making an imhex pattern for utoc files and now that I've done that I think I have a good understanding of the format. I'm decently confident that I could build a valid utoc at this point (at least for Scarlet Nexus, I'll need to check if anything changed in 4.27)

Jump

[Go to message!](#1195558171377422346 message)

[hollow glacier]

Right

[wild abyss]

You can have a look at the original post if you to see what's done and to do, I've tried to keep it relatively up to date although because of word count it isn't as specific as I might like

[hollow glacier]

I didn't know the post was made yesterday (I live in the UK and that post was made 12:07AM my time lol)

I'm sure you'll have the emulator almost complete by the time next week rolls around, GL with that btw

[wild abyss]

Thanks, hopefully that comes true 😄

[dusky steppe]

i've still been working on this on the background btw
my efforts are at the point now where it's generating an almost complete utoc, missing only the hashes for TocEntryMeta and a mostly complete ucas, missing only the container header

(since the container header's missing, this utoc still won't be able to load)

[wild abyss]

Awesome, the ucas and small bits of the utoc are where I'm stuck as well. Do you have an understanding of ucas, I'm kind of confused about it. It seems to me like it's just compresion blocks all one after the other but I'm sure there must be some kind of header because the file size doesn't add up if that's the case

[dusky steppe]

ucas has a header that's at the end of the file

[wild abyss]

Ohh

I never would've thought to look at the end lol

[dusky steppe]

it is the opposite of a header

[wild abyss]

Do you have a template or anything like that for it?

[wild abyss]

Thanks

[dusky steppe]

[wild abyss]

Is it actually working???

[dusky steppe]

i've gotten it to work with one (1) texture uasset

the caveat is that you can't currently use a cooked asset directly

i'm making the utoc, then unpacking it with unzen and using the uasset that produces since the structure of package assets and io store assets is completely different

I had a crack at trying to convert between those two formats and got partway through but that was taking a while

[wild abyss]

Oh, ok

I do remember seeing something about the format on the uecastoc github page

Yeah, at the end of this it's got stuff on the different format
https://github.com/gitMenv/UEcastoc/blob/master/CasTocFormats.md#uasset-file-format

Honestly, seeing you're version working makes me want to just give up on mine and just leave it to you...

[steel fractal]

[wild abyss]

Since I don't think I'll keep working on utoc emulator directly (I didn't get far with my version anyway) I'm trying to make myself useful elsewhere.
I've gone back to the problem with the game restarting and found that putting a steam_appid.txt in the folder with the exe does actually work (I guess I was doing it wrong before).

So, I'm going to try updating Reloaded's built in steam hook to work with unreal games. I don't think it should be particularly hard

[wild abyss]

Here's a modified loader that should make steam hook work with ue games. That means you should be able to launch the shipping exe directly from Reloaded, no asi loader necessary

If you wanted to test this you'd just extract it into your Reloaded folder, replacing the existing Loader folder in there (there's no real need for me to upload the entire program)

[boreal fiber]

(Depending on the nature of the patch, you might be able to ship this as a mod, or PR it to the loader)

[obsidian moon]

Thanks. Should this be pinned?

[wild abyss]

I was planning on making a pr for it. All it does is check an additional place for the steam api dll

[steel fractal]

There is also still going to be a problem with chunkid overlap

For mods made with the engine

[steel fractal]

A tool was made utilizing UECastoc to resolve the conflicts for Hogwarts Legacy.
If Reload can have the same functionality so users don't have to manually deal with it
https://www.nexusmods.com/hogwartslegacy/mods/1155

[wild abyss]

I don't think it'll be a problem if utoc emulator is finished. In case it isn't though, I guess it'd be useful and hopefully not too hard to add

Is the tool open source? I can't even find the download... I see there's a link to uecastoc but not the actual tool 😕

[modest shale]

is this what you're looking for?
https://github.com/gitMenv/UEcastoc

[wild abyss]

That's the source for UEcastoc (I already had that) I was asking specifically about this tool that uses it

[modest shale]

ahh, i see

also kinda seems like you wouldn't be able to use the tool regardless

[wild abyss]

I wasn't going to actually use the tool, just replicate what it's doing

Based on that though I'm guessing it probably is closed source 😔

[modest shale]

yeah

[wild abyss]

Well I can probably work out how it's doing stuff

[wild abyss]

I now know why I couldn't find it, the author took it down. Thankfully they also linked to the source on GitHub which is still up

[modest shale]

oh that's good

open-source?

[wild abyss]

Yeah

[modest shale]

[wild abyss]

I'm assuming you're familiar with hogwarts legacy modding based on this. Do you think you could link me some mods that would normally conflict that are fixed by the chunk id thing?
If so, I could try making Unreal Essentials fix the conflicts and if possible, you could test it out with Hogwarts Legacy

Actually, I think I've found some mods I could use to test it with Scarlet Nexus, if that doesn't work though I'd appreciate the help

[patent belfry]

It’s funny how modding tools probably made by people who want me dead will help mods be made for me to use for one of my new favorite series lmaooooo

(This ain’t me throwing shade on y’all for using the tools btw)

(It’s just ironic and funny)

[wild abyss]

Well this is interesting. I found two mods that use the same chunk id and gave the one that edits Kasane priority, the model change shows for thee other characters but no textures (Kasane's works fine)
Would that be due to the chunk id thing?

[steel fractal]

I've made some example mods for HiFi-Rush
One that changes the Guitar on the titlescreen
And one that changes Chai's Model on the titlescreen
When both meshes are assigned to chunk 121 they work fine when loaded individually.
But loading both of them will crash the game

And by then changing the Chai mesh to chunk 69 instead. the game can load them both just fine

With seperated chunks

[steel fractal]

Yes. this is also a behaviour that can happen.
If it doesn't outright crash the game

[wild abyss]

Awesome, thank you for those. I'll see if I can get Unreal Essentials doing that automatically

Another thing I realised that I should do is hook where ue decides priority. With the current system, the order you put mods in Reloaded makes no difference since it's based purely on file name. That's not a particularly good experience for users (this is only with full files though, the emulator would probably remove the need)

[steel fractal]

True, being able to set the load order is a nice plus

[hollow glacier]

here from reloaded hub

[patent belfry]

I’m not able to help with coding but if a lab rat is needed to test this stuff lemme know 🫡

[wild abyss]

Welcome, I'll add back the check for exe name in a minute then look at sackboy

[wild abyss]

I think I'll try put out a call for some lab rats soon, I'd like to get Unreal Essentials tested with as many games as we can in the hopes of finding any bugs before P3R (like the one with Sackboy that M&M found)

Probably once I've dealt with the chunk id and priority stuff

[patent belfry]

👍

[wild abyss]

(And sackboy)

[hollow glacier]

yea, was a good thing i caught the engine branch stuff now

incase P3RE ended up doing that too

[patent belfry]

I’ve barely played Hi-Fi Rush but I’m sure there’s something I could test with that

[hollow glacier]

luckily i have several UE4 games
just a matter of installing them

ah, stray is 4.27 with no utoc

okay huh, it also can't load mods

[wild abyss]

Do your mods have _P at the end of the name of the pak files?

[hollow glacier]

nada, i can try that though

[wild abyss]

That'll probably fix it then

[patent belfry]

Turns out the recent SpongeBob platformers I own are UE4, and I own hi fi rush

So I can try with those

[wild abyss]

When I do stuff to fix priority then it shouldn't be necessary but it is for now

[hollow glacier]

lmao that did it

[wild abyss]

Ah, well that's good

[hollow glacier]

good to know then

will check out stray with the same thing

same thing there

[patent belfry]

Yoooo hi fi rush in stray

Wow I’m realizing how many of my games are Unity lol

[hollow glacier]

i'm just realizing that we can make use of demos too

both scarlet nexus and like a dragon ishin have them

[wild abyss]

Oh, I didn't think of that. Good idea

[hollow glacier]

ishin has ++UE4+Release-4.27 which should work as-is

ff7r is ++UE4+Release-4.18

though the first instance of that is ++UE4+Release-4.18-CL-0

[patent belfry]

Dumb question but how do I check what version of UE4 a game uses?

[hollow glacier]

i've been checking in hex but it can also appear under details in properties

[wild abyss]

Yeah, but that's not a guarantee

[hollow glacier]

yeah

hi-fi is a good example of that

[wild abyss]

PC Gaming Wiki also usually has it listed (although for forks I don't think it lists the full branch name)

[patent belfry]

Oh this is helpful, thank you! SpongeBob BFBB Rehydrated is 4.23.1

[steel fractal]

7R is very custimized cobbled togehter engine. Do not reccomend 😂

[hollow glacier]

doesn't surprise me given how bad the pc port is

[wild abyss]

We'll need to get signatures for these older engine versions if we want to support them...

[patent belfry]

I can just test with hi fi if that’s easier

[wild abyss]

Yeah, that's fine but it would be nice to support as many versions as possible.

[patent belfry]

Sorry for being a bit obnoxious, I’m just really excited for P3R lmao

[wild abyss]

All good, so am I lol

[rough rose]

7R doesnt use utoc though, does it?

[steel fractal]

Nope

[dusky steppe]

yeah it's "4.18" that's far too old

[rough rose]

no point worrying about it then

[steel fractal]

Kh3 and 7R does not ise IO store
But crisis core did

[patent belfry]

Like I don’t need mods for initial playthrough besides switching out mass destruction and FeMC but I’m of the opinion that P3 has the best story/main cast of newsona and with modern gameplay I think it might be the best persona game ever

On an objective level

[wild abyss]

I'm kinda focusing on Unreal Essentials for now, leaving utoc emulator to rirurin

[rough rose]

a

well uh, you're not going to get much out of 7R

[wild abyss]

For Unreal Essentials there's no reason we can't support older things that don't use utoc

[rough rose]

you should really ignore it

I've tried myself already

not a single symbol matched with bsim

and i had some methods to manually find some stuff

[wild abyss]

Fun

[rough rose]

i.e. findfileinpakfiles

that shit got destroyed beyond recognition that its unsearchable

[steel fractal]

If Reloaded can handle any Unreal game and deal with IO store's chunk problem
Very appealing mod maneger

[rough rose]

even with the method i got that worked for like 20 diff games

its too custom

does GetPakSigningKeysDelegate exist in older ue4?

i have quite a few games analyzed so I could look in older ones if that's your goal

[wild abyss]

I'm not sure

Supporting more games is sort of a sidequest, if people want to search for signatures or send me exes that I can search for I'll do that, otherwise it can be left for later

My current proper goal is to make the game respect Reloaded's mod load order

[hollow glacier]

[hollow glacier]

oh god

[wild abyss]

Woah, you weren't kidding

Well if you want to find signatures I'm happy to add them lol

[rough rose]

if you want to go even further

if you have non utoc games, you can just enable loose file loading

and do something akin to persona essentials i guess

although there's no "bind files" function afaik

[wild abyss]

Oh yeah, you'd mentioned that before

How does loose file loading work for paks?

[rough rose]

thats why i have all these games lol

its super easy

you hook FindFileInPakFiles, and make it return 0, this makes UE4 search for the file outside of pak (because 0 return means file doesnt exist in PAK)

you also have to hook IsNonPakFilenameAllowed and just make it always return 1

[rough rose]

obv though you'd have to check the file exists loosely first

[wild abyss]

I see, well that sounds like it'd be worthwhile to add to Unreal Essentials

utoc games do still put stuff in paks so it'd be good to have for reload as well

[patent belfry]

Was R2 designed for Persona? It’s funny how Persona is about to make R2 so much more usable for games

[rough rose]

the method is the same for all versions, however the struct changed ever so slightly I think starting on 4.19 or some such

[wild abyss]

It was designed to be universal, Persona games are just one of the big franchises that picked up on it

[hollow glacier]

R2 is designed for any game

[rough rose]

so you get filename by doing **filename rather than just *filename

[wild abyss]

I think it was used for some sonic games before that

[patent belfry]

Ah

[rough rose]

thanks discord

[patent belfry]

Well it’s still cool that it’s about to become much more accessible as a mod manager since a lot of games are UE4

[marble orchid]

a bunch of the documentation has sonic-related example images so i figured those were being eyed early on (though obv it was meant for all sorts of games to use it)

[hollow glacier]

can confirm

[rough rose]

const TCHAR* TargetFile = *(TCHAR**)Filename;

if (wcsstr(TargetFile, L"../../../") != NULL && FileExists(TargetFile) && config::enableLooseFileLoad)
{
    printf("Redirected File: %S\r\n", TargetFile);
    return 0; // file exists loosely, return false so the game thinks that it doesn't exist in the .pak
}

return orig_hook_FindFileInPakFiles_NoLog(PakFile, Filename, FileEntry);```

youve no idea how many times ive copypasta'd this

lol

its

void* FindFileInPakFiles (void* PakFile, TCHAR* Filename, void** FileEntry)```

and ofc IsNonPakFilenameAllowed you just make it always return 1 since there's no harm in doing so

ok yeah, in older versions a2 is just the char* pointer to filename

i dont know which version was where i started needing to do that first line

but everything after that uses it, so its not like they changed it again

[hollow glacier]

seeing MHOJ2 here made me go and check MHOJ1
++UE4+Release-4.18

lmao, curiosity hit me and wanted to see what fortnite's engine branch showed
++Fortnite+Release-28.10-CL-30676362

so even epic themselves forgoe the ++UE4+

[patent belfry]

I also have BPM: Bullets Per Minute, which is UE4 4.26.2.0

[wild abyss]

What did you use to actually change the chunk ids?

I'm fairly sure I've got priority working properly now. I just need a signature for FPakPlatformFile::GetPakOrderFromPakFilePath for other engine versions

(It's 48 89 5C 24 ?? 57 48 83 EC 40 48 8B D9 48 8D 4C 24 ?? for Scarlet Nexus)

I'm downloading the like a dragon ishin demo so I can at least do 4.27 myself

[rough rose]

it has denuvo btw

have fun with that one

[hollow glacier]

the demo doesn't

[wild abyss]

With this would the loose file need to actually be in the game's folder though?

[rough rose]

yes

[hollow glacier]

does 48 89 5C 24 ?? 57 48 83 EC 40 48 8B D9 48 8D 4C 24 ?? E8 ?? ?? ?? ?? 83 78 08 00 also work for scarlet nexus? sackboy ran into a sig conflict

[wild abyss]

Ok, well I'll have to do some extra work

[rough rose]

youd have to symlink it like how old essentials did

[wild abyss]

I'll see if I can avoid that, it can be a last resort

[wild abyss]

Yeah, that does work. I can switch to it

[visual bramble]

Are you looking to test this with other games? I have a fuck ton of ue4 games I've modded I could test this on if need be

[wild abyss]

Yeah, the more games the better

[visual bramble]

It omega dies on Sifu, I can test a couple other games I have downloaded too (Sifu is 4.26)

Do you want that .txt it spits out

[wild abyss]

Nah, I can tell from the screenshot

I guess it uses some non standard branch name

[hollow glacier]

4.26 only has GetPakSigningKeys too

[wild abyss]

Yeah, I just didn't have signatures for it

Scarelt Nexus was the only game I was getting them for myself, I was kinda relying on other people for the others 😅

[visual bramble]

I think I have one on 4.27

[hollow glacier]

i'm upset i don't have my hi-fi rush idb anymore because that means having to analyze that denuvo ridden exe again

[wild abyss]

Do you think you could send mee the exe for Sifu Mudkip?

[hollow glacier]

hi-fi rush GetPakOrder = "48 89 5C 24 ?? 48 89 6C 24 ?? 48 89 74 24 ?? 57 48 83 EC 40 48 89 CF 48 8D 4C 24 ??"

[rough rose]

alright im pooped and ill check more tomorrow

it seems roughly up until 4.19, a2 itself is just char* for filename, but then somewhere between 4.20 or 4.21, a2 became an FString* which is why you have to do the bit of trolling with pointer to pointer to pointer thing

[rough rose]

this is for 4.21 and above

coincidentally, in the same way, 4.21 and above you're no longer hooking FindFileInPakFiles

its too "late" in the chain of command

you have to go 2 levels depeer now into

EFindResult __thiscall FPakFile::Find(FPakFile *this,FString *param_1,FPakEntry *param_2)```

its still the same exact thing/result

just thought you might want to know the distinction incase you go looking for symbols

you can tell because the function you're hooking should always be 3 args, and at some point (i have 4.21 as earliest to compare) it became 4

anyway

[hollow glacier]

at least with stray (4.27) you can use the same GetPakOrder signature as the other 4.25 games

[wild abyss]

I've got loose pak file loading working with Scarlet Nexus. No symlinks, they're loaded directly from the mod's folder 🙂

The way I did it feels a little jank so I definitely want to test it on lots of games to see if it works with them all. If file structure is consistent between them it should work fine though

The signature I need for this is FPakPlatformFile::OpenRead which is 48 89 5C 24 ?? 55 56 57 41 54 41 55 41 56 41 57 48 8D 6C 24 ?? 48 81 EC B0 00 00 00 48 8B 05 ?? ?? ?? ?? 48 33 C4 48 89 45 ?? 66 0F 6F 05 ?? ?? ?? ?? for Scarlet Nexus

I'll be able to get ones for 4.26 and 4.27 myself eventually (I'm waiting an eternity for analysis) but if someone gives me them before then it could be tested quicker.

[hollow glacier]

let me check that against sackboy

no results, hm

[wild abyss]

Try this, it's the 4.25 one 48 89 5C 24 ?? 55 56 57 41 54 41 55 41 56 41 57 48 8D 6C 24 ?? 48 81 EC D0 00 00 00 48 8B 05 ?? ?? ?? ?? 48 33 C4 48 89 45 ?? 66 0F 6F 05 ?? ?? ?? ??

Scarlet Nexus is on a custom branch which is closer to 4.25+ (it's sort of a mix of 4.25 and 4.25+ if I understand correctly)

[hollow glacier]

nada

[wild abyss]

rip

Well I've got Sifu and Like A Dragon Ishin analysed, now just to wait to add them to the BSim database and I can get sigs for 4.26 and 4.27

[hollow glacier]

should i send the executable for sackboy then?

[rough rose]

if you can wait 8-9 hours i can help again

i am going to sleep, this message has been brought to you by "certified timezone moment"

[hollow glacier]

have a gnight

[wild abyss]

If you could, yeah that'd be helpful

[hollow glacier]

would be fine to just put it here then, right?

[wild abyss]

All good, I can wait. Sleep well

Yeah just do it, we can delete it after I've downloaded it 😄

[hollow glacier]

alright

[wild abyss]

Thanks

[hollow glacier]

np

[steel fractal]

You assign the chunkID in Unreal Engine before you export/cook the ucas

How to change it after it is cooked is the big question

[wild abyss]

Oh, ok. So you just remade the mods, I see

[steel fractal]

Yeah

I've never gotten UECastooc to work myself

[wild abyss]

Well, I think I'll just leave this chunk id stuff for now. I'm pretty sure utoc emulator should fix it anyway.
I was hoping it'd be something relatively simple that I could do but it seems like it probably isn't

[dusky steppe]

oh yea, the emulator

when I posted the first image where I had the emulator working, that was only emulating the utoc portion

the ucas was still a real file that I made by copying the contents of each file into it, which ofc was never gonna be the solution I would ship
so I've done the work needed to also emulate the ucas, but I've come across a weird problem where it doesn't report any errors, but the game refuses to continue when using the emulated ucas

It only works if I dump the contents of the emulated ucas to a file and let the game read the actual file

[wild abyss]

That's fun. My guess just based off of what you said would be that the emulated file size is being reported incorrectly (or something along those lines). I had that problem with bf emulator although I thought it was fixed (it was a problem in the base file emulation framework mod)

Another idea is maybe the stream for the ucas is being closed and it's failing to read it that way

It's kinda hard to tell if there aren't any errors reported

[dusky steppe]

after you mentioned that, I remembered that all ucas streams are sized as a multiple of 0x10, which I hadn't done, so I added that in without any success

[dusky steppe]

i imagine that'd have to be the case then

[wild abyss]

Worst case scenario you find where the game is reading from the ucas and just step through until you see where it fails, it'll probably be a pain but it should work

[dusky steppe]

[wild abyss]

Fun

What debugger is that?

[dusky steppe]

x64dbg

[wild abyss]

Cool, I've always just used cheat engine

I've tried to use the debugger in Ghidra a few times but have never really had any success with it

[dusky steppe]

i mostly use cheat engine too, this just has a couple of useful features such as pausing at the entrypoint by default

yea I've tried to get ghidra's debugger working too and didn't get anywhere with that

[wild abyss]

If you launch the game directly from cheat engine it breaks at the start

[boreal fiber]

and pausing on crash

that used to be useful before I discovered crash dumps

which funnily enough I never knew about for many years

[rough rose]

@wild abyss FPakPlatformFile::OpenRead changed omega drastically somewhere after 4.18 lol

4.17 and 4.18

4.21+

thats not even the whole thing lol

[rough rose]

@wild abyss here, I hunted down most of them manually just to make sure it wasnt a signature collision thing

[wild abyss]

Thanks, as long as it still does the same thing it shouldn't really matter if it changed drastically (we'll have to test it to see if that's the case though)

What I do is check the file that it's trying to open and then if we have a loose version, call it with the path to the loose one instead. I doubt any change would break that

[rough rose]

i see

while you're at it I'd suggest a file access log toggle

(although i have yet to find where you'd get such a thing for utoc)

[wild abyss]

Yeah, it currently logs them if you're in "debug mode". I'll put a separate toggle for file access to make it clearer

I don't have signatures for versions prior to 4.25 (apart from the one you just sent). Did you send them at some point or do we still need to find them?

[rough rose]

for what?

[wild abyss]

The other things I use:

• GetPakSigningKeys
• GetPakFolders
• GMalloc
• GetPakOrder
  (Not the actual names, that's what I called them in my code)

[rough rose]

i see

i dont know if getpaksigningkeys is needed for pre 425?

[wild abyss]

Ok, well if it's not that's fine, I'd still need the other ones

[hollow glacier]

i have GetPakOrder for hi-fi rush here if you haven't seen it yet

[rough rose]

i can look for some of these later if they havent been found yet

although ill need the full name for the gmalloc one

the others i assume are from FPakPlatform class?

[wild abyss]

I didn't see that one. Thanks

Yeah, I'll double check and get you the full names of them

GMalloc is a global variable (that's what it's called) but I can get the name of the function I scanned for it (there are hundreds of options iirc)

• FCoreDelegates::GetPakSigningKeysDelegate
• FPakPlatformFile::GetPakFolders
• FMemory::GCreateMalloc (there's a pointer to GMalloc in there, I need the address of the instruction that loads it, not this function. Any function that uses it would also work fine)
• FPakPlatformFile::GetPakOrderFromPakFilePath

[wild abyss]

In theory this should work for all games on 4.25-4.27 (I'll post another when I have complete sigs for older versions). The additions are:

• You can now load loose files from paks. To do this, just put them in the Unreal folder of your mod copying the file structure in game. For example: Unreal\ScarletNexus\Content\UI\01_Common\00_Font\AGMENAW1G-SEMIBOLD.ufont
• You can log file access (from paks only for now) by enabling the new setting in the mod's config. If you're having trouble working out where loose files should go you can use this (just ignore the ../../../ at the start of what's printed in the log and that's what you'd put in the Unreal folder)
• Your load order in Reloaded is now reflected in game. If a file is lower in your mod list it will take priority over other mods that edit the same files (chunk id conflicts can still happen, this doesn't fix that)

I'll write some proper usage documentation later today but hopefully that'll do for now if anyone wants to test

[visual bramble]

sifu still seems to get same error (2.26)

[rough rose]

lol did they troll with custom version

[visual bramble]

no

its like the most default ue4 a project can get lol

[rough rose]

it says unable to find unreal engine version

that means its not in the exe

which usually only happens when they mess with the code

[visual bramble]

swine could probably definitively tell you he has exe

[wild abyss]

Oh right, I forgot to get the version for it

I'll do it when I get home

[wild abyss]

That doesn't make any sense, Sifu does have the standard ue4 version. It should find this just fine

[visual bramble]

I just retried again and got this

[wild abyss]

Umm that's weird. I should be able to fix that though

Did you change anything since you last ran it?

[visual bramble]

I re-pasted files in, ig windows didnt replace files correct first time or something?

[wild abyss]

I'm having a lot of trouble finding the function in Sifu for some reason 😔

[rough rose]

send exe

[wild abyss]

K

Could you also check if this signature matches with your 4.27 games, the shorter one that you had was conflicting with Like a Dragon Ishin Demo
4C 8B DC 55 53 57 41 54 49 8D 6B ?? 48 81 EC B8 00 00 00 48 8B 05 ?? ?? ?? ?? 48 33 C4 48 89 45 ?? 66 0F 6F 05 ?? ?? ?? ??

Oh, I was going to send the Ghidra archive for it to save you analysing but it's huge

[rough rose]

thats why i made note of a longer one lol

the longer one still worked

oh wait 4.27

my brain read that as 25

ill check

lets see, matches granblue

matches crisis core

matches on default project exe too

so it should be fine then

[wild abyss]

Cool, thanks for checking

I'm still trying to upload Sifu btw, it nearly finished but failed for some reason

[visual bramble]

I think you sent him the wrong one

[wild abyss]

Oh yeah

[rough rose]

this is too tiny-

lmao

[rough rose]

what the fuck did they do on SIFU lmao

ive tried backtracing from FindFileInPakFiles and IsNonPakFilenameAllowed (because both are used by OpenRead) and i cant find any function that matches

[wild abyss]

bruh

[rough rose]

i actually had this same issue on KoF XV

I think retail 4.26 is different from what they give now

in fact, I can even tell you

the FindFileInPakFiles here

it looks like the 4.18/4.19 one

but i cant find anything that matches how OpenRead was in those versions

[wild abyss]

If you havee FindFileInPakFiles then open read should call it...

[rough rose]

in the sifu exe you sent its at 0x143025540

(findfileinpakfiles)

[wild abyss]

Thanks

[rough rose]

isnonpakfilenameallowed is at 0x143029fb0

finding them both is easy because FileExists is an easy to find function (relatively) and it calls them both

FileExists is at 0x143024f20

[rough rose]

thing is, like i mentioned earlier, after 4.21 or so there's 2 FindFileInPakFiles

i think openread calls the one thats inside FindFileInPakFiles

fun isnt it

[wild abyss]

Oh...

[rough rose]

and this one doesnt seem to do that?

it only has the top level one like older UE4

[wild abyss]

I found one that looks like it might be it at 143026b10

The general structure looks about the same to me

[rough rose]

hmm, idk

it shouldnt be calling FindFile first thing

[wild abyss]

Oh, true

Yeah, you might be right

[rough rose]

and ghidra is usually good at guessing types

and it didnt guess a3 as a bool

usually it guesses "undefined" which is just 1 byte

[wild abyss]

Yeah, I think you're right. nvm

[dusky steppe]

yeah I haven't yet found a solution for why it hangs on startup, so for now I'm testing by just writing the ucas to disk right after the emulated utoc's created
which sucks because other than that, things have been working great so far. I can add and replace files, it respects reloaded's mod order and it has worked fine with hifi rush

I've only tested it with 4.27 so far, so I'm planning to check how it performs with 4.26 later since I've found out that normally the container header on 4.25+/4.26 is located at the top, so I wonder if that may cause any issues

[wild abyss]

Well I'm glad to hear it's going well apart from that. Sounds very promising

If you want I could try looking into it not working with the emulated file, maybe some fresh eyes would help

[dusky steppe]

also I'm wondering if it may be a good idea to bundle the UTOC emulator inside of the Unreal Essentials repo instead of FileEmulationFramework since the IO Store format changes and the only way I can think of to check for that would by getting the exe/branch name from Essentials

because I know that the format's quite different in 4.25 and in UE 5

[wild abyss]

Does it change without changing a version number or something in the file?

[rough rose]

Tekken 8 is UE5, so it'll be very fun if someone runs into this and tries to use it

[dusky steppe]

there's no version number field stored in the files to indicate such a thing

[wild abyss]

I thought there was... Maybe I'm thinking of something different

[dusky steppe]

cooked packages do

but its not in io store packages since it's just assumed that they're formattes for the target engine version

[wild abyss]

Ok, well if you want to include it directly in Unreal Essentials that's fine by me. Either that or expose an api that Unreal Essentials can use to tell it the version and stuff (you'd probably need to do that anyway)

Oh, I see what you mean. the version is either initial, directory index, or latest. Latest is not a particularly useful version number lol

[rough rose]

ok i found something in SIFU that i dont know if its the right one, but the args match at least

check 0x14302e3b0

[wild abyss]

We can just try and see

@visual bramble could you try this with Sifu.

[rough rose]

there are exactly only 2 functions that match the argument count that call FindFileInPakFiles

[visual bramble]

[rough rose]

the other function is this which doesnt even look remotely similar

[visual bramble]

[rough rose]

ah wait that says undefined8 nvm

[wild abyss]

[rough rose]

so yeah only the one i sent

[visual bramble]

i will test a mod

[rough rose]

this was a pain in the ass to find

wtf are they doin

[wild abyss]

Do we have any other 4.26 games we could test it on?

[visual bramble]

I think so, yeah

the two im thinking of have eac though so

[rough rose]

this matches on KOF XV and that one actually vaguely resembles the function from other retail games

[wild abyss]

Ok, well it's probably right. I guess we'll find out eventually if it isn't consistent

[rough rose]

not much we can do at this point lmao

it doesnt even match symbols build

[visual bramble]

So I can't seem to get mods to load on Sifu, however I also don't really see it logging alot of files

[wild abyss]

Does it use utocs? (I feel like I've asked this before lol)

[visual bramble]

nah

[wild abyss]

Weird, it should log everything then

[rough rose]

but it seems to be logging files here

very odd

[visual bramble]

yeah but not all

just some like, ui stuff

[wild abyss]

Could you see if this logs more stuff

[visual bramble]

ye

yes

yes it does

[wild abyss]

Ok, see that seems like it's logging all the files?

[visual bramble]

very much so

[wild abyss]

Alright, well I guess there must be something other than OpenRead used. Maybe I just hook FindFileInPakFiles instead (that's what this logging is from)...

[visual bramble]

mods no worky, but its logging everything

[wild abyss]

Yeah, that was just logging to see if it's the right function, I'll work on making mods actually work

[visual bramble]

[rough rose]

hook isnonpakfilenameallowed

make it always return 1

literally just kill the entire functionality and make it just return 1

[wild abyss]

That goes back to files needing to be in the game's folder though, doesn't it?

[rough rose]

well findfileinpakfiles has the filepath

but being an older game you get into the whole

"this is an fstring so you need do to **(*char)" or however it went

[wild abyss]

I've already got code for dealing with FStrings, that's not really a problem. I guess I'll try doing what I had before but on FindFileInPakFiles and just see what happens

[wild abyss]

I think I have an idea based off of this actually, those file types are ones that would be stored in utocs if the game used them. So, maybe I need to hook whatever the function is that tries to read from utoc files

Actually no, FindFileInPakFiles isn't used in the io store code anywhere that I can see so that'd be wrong...

[dusky steppe]

I made the assumption that the UTOC file will always be read before the UCAS, and that held true for my 4.27 tests

but it's the other way around in 4.26

but it does read the 4.27 formatted UCAS just fine (4.25+ and 4.26 put the header at the top with a fixed size of 0x10000 while 4.27 puts it at the bottom, and it doesn't use the partition count and partition size fields in the toc header)

[ebon shard]

Lmao 😭😭 small revisional changes between versions that have major impacts moment or smth fjfjn

Makes sense why theyd change it tho, bit odd to read the data file before the table of contents file haha

Sounds like it'd be an easy enough fix for that since ud just have to make the emulated UCAS file write first but then I could see that just inverting the problem where 4.27 games start hanging or failing to read files BC the UTOC wasn't made in time or smth

[dusky steppe]

which is why I mentioned earlier that the emulator's gonna end up depending on unrealessentials to get the right version from

i knew this was gonna happen eventually since 4.25 and 5.0+ change the format significantly, but I didn't think I'd have to pull that trick this early

[ebon shard]

Sometimes thats just how it be 😔

At least for that it might be simple enough to just go "if 4.27 or equivalent behavior, utoc first, else UCAS first" but then u never know if any devs decided to be cheeky on those older versions and change the read order for no reason lol

Regardless tho it's just gonna have to be one of those "this is a kinda dirty workaround to fix an issue but no other option is better so ¯\_⁠(⁠ツ⁠)⁠_⁠/⁠¯ "

[patent belfry]

I believe in y’all!!!!

[wild abyss]

I just had an idea for the emulated ucas files not being read. Maybe the stream of the file isn't being set to position 0?

I acquired Sifu and it has symbols

I found out where those other files are being loaded thanks to that, FPakPlatformFile::OpenAsyncRead

[wild abyss]

Sorry that took ages, could you try Sifu with this one @visual bramble (won't work with non 4.26 games for now)

[wild abyss]

If anyone wants to look for signatures (assuming that works for Sifu) FPakPlatformFile::OpenAsyncRead is what I hooked for it,
I'll try and do them myself tomorrow in any case

(I would do it now but Ghidra keeps deciding to completely freeze and then eat my cpu alive and I don't know why, it's not even analysing anything)

[rough rose]

oh it did this for me a few times too

i wonder whats up with that

had to completely kill it with task manager and reopen

specifically only happened yesterday too

[visual bramble]

infinite loads at this point always, the files not modded so its not that

just refuses to load stuff beyond this point

[visual bramble]

oh nop it is dying from mod specifically

mod works fine as a .pak so ik thats not the issue

yeah ive deleted files, the only way to make it load is to just not have a mod enabled lmao

[wild abyss]

Fun, could you send the mod that's breaking it. I'll try and see if I can work out what's wrong

[rough rose]

sounds to me like you'll end up eventually just going with my original idea lol

findfileinpakfiles has the filepath

[wild abyss]

Maybe...

[visual bramble]

you can immediately test btw

start a new game, adachi first person you see

[wild abyss]

I tried going further up to change the path in FAsynchArchive but still no dice 😔
I'm going to try FindFilesInPakFiles now...

Ughh why does this work

This needs files in the game's actual folder though. I'll see if I can work out why this works when I'm fairly sure what I was doing should be equivalent

[rough rose]

cant you just replace the filepath string?

its in a2

well, a2 is an fstring that contains the path but you get the idea

[wild abyss]

I can try

Nope, that just caused a crash unless I did something wrong

(It's also not an FString btw, just a regular string)

[rough rose]

the symbols list it as fstring

odd

[wild abyss]

At least in 4.26 it isn't

[rough rose]

is that the top one or the bottom one

remember that there's 2

theres the FPakPlatform one

and that one in turn calls another one

you can check FileExists in FPakPlatform

that one calls the "top" one