[BETA v0.1.6] LethalModManager https://thunderstore.io/c/lethal-company/p/kyxino/LethalModManager/

[hollow epoch]

at least you know what a region is 💀

[dull hawk]

Reminds me of the time I forgot to use static classes because it said internal class, so I kept thinking 2 words = static class. Lol

Got to love coming back to a static programming language after many months/years.

Well, not even that, just me being blind and seeing internal as static just because there was another keyword behind class.

[hollow epoch]

and this is why your making mods and not me :) (i have no clue what you just said)

[dull hawk]

Lol, all good.

[pearl plover]

JetBrains tools let you collapse regions in the middle of a line 😎

Entire selections can just be collapsed using ctrl+.

[dull hawk]

In the middle of a line?

[pearl plover]

yeah so like you can easily skip over parts of a long line

basically reduces the use of home/end

it’s also very nice for long lines of text

[dull hawk]

Ah, so you don't have to scroll all the way below where the function ends?

[pearl plover]

No like:
if ((conditionA) && … {

where … has been collapsed

[dull hawk]

OHHHHHH

Okay, that is actually pretty cool.

I typically don't need something like that because of my ultrawide monitor, but still cool to have.

[pearl plover]

same but it’s useful for skipping content in large if lines

[dull hawk]

Yeah, the other way I handle that is just newlining it. Lol

[pearl plover]

AceJump is great, it lets you start typing and you can jump to the start of words

it’s a plugin for jetbrains tools

[dull hawk]

Ah, cool.

I'd look into JetBrains again had I not already adapted so much to Visual Studio 2022. LOL

I know those two are like most people's top choices.

[pearl plover]

Granted you’re probably using Rider which is paid

But assuming LMM is open source and you’ve been working on it for 3 months, you can request an open source licence

and then you get rider for free

[dull hawk]

Ah, I see. Well, unfortunately I'm not sure if I'll be doing LMM as open source yet, just concerns with mod sync and it being compromised. When LMM gets bigger and the security concern is lessened with more updates, then it will be open source.

[pearl plover]

Yeah that’s fair

[dull hawk]

The concern has already been fairly mitigated, so I could probably do it now.

[pearl plover]

it’s only if you want the open source licence ¯_(ツ)_/¯

[dull hawk]

Yeah.

Thanks for that info though, I forgot some of that stuff was paid, I already used a free trial for some of its features a while back iirc.

I might actually use it for Unreal Engine, or at least its plugin for VS2022.

[dull hawk]

Ah, yep, I see exactly what you mean now.

[pearl plover]

It’s not particularly cheap either

[dull hawk]

DANG, I know VS2022 was a crazy upgrade from Roblox Studio, but this is even crazier.

[pearl plover]

No need to switch if you’re familiar with VS

I personally really like JetBrains tools

[dull hawk]

Yeah, I could probably download some plugins that mimic some of the same features, but still aren't that needed.

[pearl plover]

as in I almost exclusively use JetBrains tools when developing

[dull hawk]

It definitely looks pretty nice.

[pearl plover]

it feels it too lmao

I can practically navigate the entire IDE with only the keyboard

[dull hawk]

Dang.

I also like some of these debugging features with Unity and JB's .NET.

[pearl plover]

I haven’t tried it 👀

[dull hawk]

It looks awesome.

If I was purely a Unity developer, I would've switched over immediately, likely.

[pearl plover]

I pay yearly for WebStorm

i think it’s close to $150?

[dull hawk]

Haha, oh gosh, not too bad, just that I need to actually start working.

[pearl plover]

not entirely sure because I’m gbp

[dull hawk]

Oh, shoot, it supports Vue out of the box.

I'll be using that soon for some stuff, that might be helpful.

[pearl plover]

Yeah, and a ton of other web languages / frameworks

But im pretty much a web dev so it makes sense for me

[dull hawk]

Jeez, it's funny to think how good VS is compared to everything else, and then see this compared to it.

[dull hawk]

Ah, yeah.

Probably would be best for me to stick with my current stuff since I'm not very consistent with web development.

[pearl plover]

Well yeah that’s why i mean for webstorm

[dull hawk]

I'll just be using Vue for a project, if the project is still greenlit.

[pearl plover]

Rider would make sense in your case

[dull hawk]

Yeah.

Still looks cool though.

[pearl plover]

Maybe just hit up another free trial

[dull hawk]

JetBrains is working on wild stuff.

Idk what that free trial was for exactly.

[pearl plover]

There’s also Fleet which you might be able to use?

[dull hawk]

DotPeek?

OH

[pearl plover]

It’s basically VSC but JetBrains-ified

[dull hawk]

Yeah, it was DotPeek.

[dull hawk]

Ah, I see.

[pearl plover]

So yeah you should able to still trial Rider

[dull hawk]

Yeah.

Fleet would likely be something I'd use as well.

It says JS and TS, I'm assuming Node.js as well?

[pearl plover]

Node is JS

[dull hawk]

Yeah, just that some companies consider it different.

[pearl plover]

Node is just the runtime

[dull hawk]

Alright, cool.

When I'm not too busy, I'll probably give JetBrains (overall) a go.

Fleet would be good for me to mess with another Discord bot probably.

At some point I want to mess with Golang as well, so I could do both in one. Lol

3 in 1 (Fleet + Golang + DiscordGo)

That's a way to make me want to die.

[pearl plover]

Fleet (last time i used it) didn’t feel as nice because some of the functionality isn’t there

Like, it’s missing the usual shortcuts to create multi-line cursors for example

[dull hawk]

Oh, not too important.

[pearl plover]

But if you’ve never used them then it doesn’t really hurt

Oh i use multi line cursors all the time

[dull hawk]

I'm never really able to. Lol

The few times are short-lived.

[pearl plover]

Brilliant for writing tests with dependency injection

For example, using Spring Boot you may have a bunch of dependencies that need to be declared in the test with an @Mock annotation

You can multi-line, move to the start of each line, insert a new line after your cursor on each one, and then write @Mock against all of them in one go

[dull hawk]

Ah, nice.

Yeah, the most common time I'm using multi-line cursors is when I'm just trying to tab align things. LOL

[pearl plover]

Oh I just auto format code

[dull hawk]

I tab align when a lot of things are very similar, and I want it to be easily readable.

If it is all over the place, idc.

(Mainly for ConfigEntries in LMM as an example.)

Oh, I forgot the bottom. Lol

but yeah, this right here is annoying.

(If it wasn't tab aligned.)

The one below that is much more subtle, I'm just adding consistency at that point.

[pearl plover]

Oh I don’t like that format lmao

As soon as you add a new one that’s longer you have to reformat everything which kinda sucks

I’d instead have them split into classes and treated them like singletons

And then you’d just access it by doing something like this.apiSettings.updateTime

[oblique marten]

(which also causes git blame to show all of those lines on a commit adding just one line :( )

[dull hawk]

That's the plan, I've been working with a fork of BepInEx's Configuration class to do something like that for my other mod, and eventually I want to add it to an API that both LMM and others can use.

[dull hawk]

Yeah, it's not too terribly favorable, but it's not hard to keep up.

[pearl plover]

And hurts with merge conflicts if there were more people working on it

[dull hawk]

Just multi-line the others and it's all good.

[pearl plover]

Fair, although is it something you need to fork?

Like, you can split them out pretty easily so what do you gain with your change?

[dull hawk]

Yeah, I'm likely going to just patch current methods, only reason I forked it originally was because it couldn't support what one of my mods were doing.

It was just that I couldn't pass a type through without explicitly writing it out.

(So if I had a variable for a certain type, I needed to pass it through to construct a ConfigEntry, so I had to fork the entire thing just in order to add another Bind method that allows a Type parameter since I couldn't use generics or anything.)

Luckily BepInEx is never updated anymore, so I'm not missing out on any new features. Lol

[pearl plover]

Rip

[dull hawk]

Forgot to mention, yesterday my dad's new computer case (and second new PSU after I stole his first when mine blew) arrived, so we set that up yesterday, but are still doing a few more things today.

[hollow epoch]

any updates?

[thin bane]

#1186595897052774472 message

[hollow epoch]

k.... doesnt mean he didnt do anything at all...

[thin bane]

i was trying to be funny 😓

nevermind

[hollow epoch]

oh

without context its confusing

[dull hawk]

A little more stuff was done for mod sync, but that was it, been busy, this has been my busiest month in a long time, and most of it isn't even LMM.

Especially since a lot of friends want to hang out and play because they are out of college for a bit and stuff.

[hollow epoch]

Personal life is better than the mod 👍

Idk if this came of sarcastic or smth but I wasn't trying to be :P

[dull hawk]

Nah, I get it.

Luckily most people are taking a break from LC until the next update comes out, or some other big mod comes out.

It's relatively calm for now.

Which is great because I'm stressed and tired, and that isn't even from me working on LMM, haha.

[tepid sedge]

shit, I might qualify for an oss license from jetbrains

[pearl plover]

do it

[dull hawk]

OK I FINALLY ESCAPED FRIENDS AND FAMILY, MODDING TIME.

Zeekerss has been grinding on v50 as well, so I need to get this (and LethalUtilities' rewrite) out before then.

[dull hawk]

Okay, the biggest issue is that suddenly there is a memory leak.

I don't need this affecting lower-end PCs.

Alright, I think I got it down like 50%.

[lethal gyro]

Noice

[dull hawk]

I don't get how when I do something that should actually help reduce memory usage, somehow it uses more, and now GC is just not working anymore.

Idk if that is just VS2022 reporting the wrong number (I do see a significant difference between that and Task Manager), or if I'm really doing something wrong.

[dull hawk]

Alright, well apparently most of it is from me getting the latest Thunderstore data, but it won't release the old data, which I figured, so I switched from WebClient to HttpClient and it still seems to have the same exact effect.

Wait, I forgot something, let's see.

Okay, that's better, but still have a memory leak.

[copper turtle]

im always here for the updates

ty

[dull hawk]

Np, every time I can settle down, I try working on LMM for as long as I can before being dragged back into something else. Lol

[dull hawk]

Luckily it does seem like this memory leak stops at some point (possibly due to GC), so I might be able to just continue for now, but I'll try a few more things until then.

[dull hawk]

Alright, it's decided, I'm going to just skip this, everybody that is modding should have enough memory to handle it, especially since GC should be more aggressive if you have less memory to spare (assuming C# does the same as some other things I have worked with), it isn't too much, so it should be fine.

I just need to get things done and out the door at this point.

[dull hawk]

Never mind, it seems like it was adding up way too much once I made some other changes.

I have an idea, but it will require another slight rewrite, yippeeeeeee.

If this doesn't work, idk what will.

[rapid viper]

GC is lazy giving the memory back to OS

it will only force the memory back if OS warns that memory usage is high

[dull hawk]

Yeah, it's lazier than some of the other things I've worked with.

[dull hawk]

My testing probably isn't the best in this case because I have 64 GB of ram.

[dull hawk]

Never mind, I just realized that "slight rewrite" wouldn't end up being a slight rewrite, I would literally have to do 5 other slight rewrites because of that rewrite.

Ehhh, I wouldn't have to, it would work fine without the extra ones, it would just run over some stuff. Lol

[sage socket]

@dull hawk pssst, look at #announcements

[dull hawk]

Yep, I was expecting this today.

[sage socket]

Time to stay off lethal internet for a bit lol

Don't wanna get spoiled anything

[dull hawk]

Lol

LethalModManager is still functioning properly, which means no extra hinderances.

[lethal gyro]

is it out yet?

[dull hawk]

No, I have to fix a memory leak before I can release it.

It's a severe memory leak as well, and it would only get worse over time if I don't take care of it now.

[lethal gyro]

F

[surreal cove]

so close yet so far

[dull hawk]

LMM's beta may be releasing when v50 releases so that there are no annoyances with v49 users right before v50 comes out, and then mismatched issues all over the place.

Especially because LMM doesn't have it's own server list yet.

[full ether]

https://tenor.com/view/rage-i-quit-angry-table-flip-studying-gif-4905184

[hollow epoch]

v50 is in beta rn :>

[dull hawk]

Yeah, that's why I said it. Lol

[dull hawk]

Man, it would've been funny to actually release LethalModManager today, everybody think it is just an April Fool's joke, and then it literally be the real version.

If only the memory leak wasn't such an issue.

[dull hawk]

[BETA SOON] LethalMemeManager https://thunderstore.io/c/lethal-company/p/kyxino/LethalModManager/

[ripe sky]

ah yes i can finally manage my lethal memes

[lethal gyro]

Beta soon or beta now?

[sage socket]

How does a memory leak even happen?

<--- not smart enough to know

[lethal gyro]

He didn't unclog the pipes properly

[ripe sky]

lemme make you even more lost

it happens when you allocate memory to a heap but dont remove it later

[hasty fog]

have you tried the command fixmemoryleak LethalModManager

[dull hawk]

When programming languages are lazy.

(or the programmer itself)

((In this case, it is the former, and I can't do much about it other than changing the cause completely.))

[maiden bridge]

I don't understand how GC languages get memory leaks, isn't not worrying about memory leaks like the whole point of GC

[dull hawk]

I mean, it works, just very poorly.

So immutable classes (like the lovely strings we come across all the time) can be a PITA.

Idk, I'm having the growing suspicion that it is VS2022 not reporting it properly, but even checking Task Manager is giving me concerning numbers.

[pastel geyser]

Watch this entire project need a whole rewrite due to v50

[lethal gyro]

That's not

Do you know what this project is?

[pastel geyser]

Ik what it is, I’m js making a joke how a lot of mods break due to updates. And how a lot broke due to v45 even v47 I think

[dull hawk]

.

Nothing else was broken anyway.

[pastel geyser]

Chat

Did I read that?

Hell nah🔥🔥🔥

[dull hawk]

Ah, nothing like a bunch of extra programming to try to mitigate memory leaks as much as possible because you are working with hundreds of MBs of data that builds up because of immutable classes, lovely.

[dull hawk]

[BETA SOON] LethalModManager https://thunderstore.io/c/lethal-company/p/kyxino/LethalModManager/

Should've named it LethalMemoryLeak, ngl. Lol

[dull hawk]

how did this get worse...

1.2 GB from updating... how...

I think this somehow became like 3-4x worse suddenly.

Yeah, I had 1/4th of what it is now.

[oblique marten]

is it the mod listings from the store that are consuming all that memory? would it maybe be better to write it to some fairly efficient format on disk and load it lazily?

I'm not sure exactly what the issue is with immutable classes tbh

[dull hawk]

Say you have a 130MB string just lying around after converting it to another class that is actually readable like List<CustomClass> that the rest of your code uses... that's literally it, that string is still around, even if you aren't using it. Even worse, if you were to add another character to it, it would basically duplicate.

[oblique marten]

why isn't that string being collected?

[dull hawk]

Now, I'm not adding anything else to it, but still.

[dull hawk]

Beats me.

[oblique marten]

you would have to be keeping a reference to it

[dull hawk]

Exactly, and I'm not, so this is just ridiculous.

[oblique marten]

have you tried a memory profiler yet?

[dull hawk]

I did see an improvement when I switched from WebClient to HttpClient, so there are some sort of references going on, or GC is just extremely lazy.

[dull hawk]

I'm just using numbers from VS2022 and Task Manager.

[oblique marten]

there should be one way or another to figure out exactly what objects are alive, and hopefully also what's referencing them

I'll see if I can find any options for that

[dull hawk]

I have no idea, I'm honestly just tempted to rip one of my sticks of memory out to see if it is just greedy. (I have 64GB of RAM.)

[oblique marten]

that seems doubtful, keeping a massive string around if it's not in use is very strange behavior

[dull hawk]

Exactly, everything is just weird.

[oblique marten]

the GC can certainly be more aggressive under memory pressure, but it wouldn't do that

are you able to run LMM in editor?

[dull hawk]

I have yet to try that.

[oblique marten]

that's one option, or convert a copy of the game to a debug build and attach the editor's memory debugger to it

[dull hawk]

Basically, every time I download the results from Thunderstore, it's a 158MB file, but for some reason, both VS2022 and Task Manager reports it as double.

[oblique marten]

I'm gonna see if that provides enough information about non-scriptables

[dull hawk]

Ohhh, you mean independently?

[oblique marten]

well, just in any way where you can reproduce the excessive memory consumption

[dull hawk]

I was meaning to do that a while back, and I had at the beginning, but then I changed everything.

[oblique marten]

(I wouldn't really call it a leak unless it increases indefinitely)

[dull hawk]

Considering it went up to 10GB, yeah, definitely a leak.

[oblique marten]

and it did so gradually?

was it doing any processing of the large data strings you mentioned?

[dull hawk]

Yeah, I undid the changes, so it shouldn't be as fast anymore, but it still did it before either way, just took longer.

If I don't do any processing, it just uses double the RAM it should, and then just never gets GC'd.

[oblique marten]

so this is caused by some background processing of the thunderstore cache?

[dull hawk]

I was wondering if it could be some caching going on, I actually didn't check HttpClient for that, hold on.

Because in some cases, it would mitigate itself.

[oblique marten]

oh, you mentioned an update, do you mean memory increases every time it updates the Thunderstore cache?

[dull hawk]

Everytime it updates the data from the API, yeah.

[oblique marten]

oh, yeah, the Unity memory profiler will do what you need to track this down I think

[dull hawk]

Alright, well, ig time to try to put everything back into Unity. Lol

[oblique marten]

it gives you a list of all objects and their references

no need

just build the mod, convert a copy of the game to debug, then attach the memory profiler from a blank Unity project

[dull hawk]

I don't know how to convert a copy of the game to debug. Lol

[oblique marten]

https://github.com/dnSpy/dnSpy/wiki/Debugging-Unity-Games#turning-a-release-build-into-a-debug-build

it's not too hard if you already have the appropriate version of Unity installed

[west surge]

could also just use the built in profiling tools in editor on a #1199473521265475745 right?

or nah

[oblique marten]

Kyxino doesn't seem to be using that at the moment

but honestly, I wouldn't do that anyway

having that much memory usage inside the editor is a recipe for non-responsiveness

[dull hawk]

Well, it has been slowed down again, so it should be good.

[oblique marten]

anyway, once you get the memory profiler (have to install it in the package manager, it's not a default package) attached to the game and take a screenshot, you'll want to go here:

see what's consuming all that size, presumably it would be in Scripting Memory > Managed Objects unless you're creating GameObjects

[dull hawk]

Oh, shoot, this is actually really helpful.

Just got to figure out how to do this. Lol

[oblique marten]

yeah, lemme know if you get stuck anywhere

that dnspy link is the first step

[dull hawk]

Partially blind rn (thanks to allergies), so it will be a moment. Lol

[oblique marten]

oh rip

I feel you, I have that sometimes too

[dull hawk]

Yeahhhh. Lol

[surreal cove]

How’s progress going

Btw this “m”

Got a rebrand

[gray snow]

hey where did the "lmm this weekend" go

[dull hawk]

It was no longer promised weeks ago. Lol, unless you want a memory leak.

[gray snow]

yeah I'm adobe, I love memory

[dull hawk]

Nah, it's a Chrome moment.

https://tenor.com/view/gigachad-chad-gif-20773266

[dull hawk]

Yeah, the person who made these instructions really didn't make them clear.

[oblique marten]

the debug build ones?

[dull hawk]

https://github.com/dnSpy/dnSpy/wiki/Debugging-Unity-Games

The entire page is confusing.

I've looked up other stuff, but Google doesn't understand what I want per usual.

[oblique marten]

you only need to worry about the specific section I linked

[dull hawk]

What about the patched DLLs?

[oblique marten]

I didn't need any of that myself

just the debug Unity runtime executable and dlls

[dull hawk]

.

[oblique marten]

so basically, you just need to navigate to that unity editor directory it mentions, copy WindowsPlayer.exe and UnityPlayer.dll to the copy of the game folder and replace LethalCompany.exe with that WindowsPlayer.exe, then add the player-connection-debug=1 to the boot.config

that was all I had to do to get it working

although you do want the 64-bit version of the runtime, not 32-bit

[dull hawk]

This entire time, I was thinking you were highlighting the most important part that I have to get to or something, oml, I need sleep. LOL

[oblique marten]

understandable, it's not overly apparent on that page

especially since the title of the first section seems very relevant given that this is a release build

[dull hawk]

Yeah.

uh, LMM just broke

[lethal gyro]

mood

[dull hawk]

I got the thing working, but then LMM just broke.

[dull hawk]

Well then, some bug just bit my finger or something, but I flung it off before I even knew what it was. (little dude snuck up my leg, I swiped it, got on my finger, and just freaked out because of the weird feeling)

Never had some sort of bruising from a bug bite tho, so that's weird.

Anyway, thanks @oblique marten, trying to figure a few other things out, but I think I got it now.

[oblique marten]

glad to hear it, hopefully you can figure out the cause with that

[dull hawk]

OHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH

I think I know why now.

(This is still ridiculous.)

(Also, it didn't take me half a day to get to this point, I was waiting for a while to restart my PC because Unity would no longer load.)

It's just reserving a crap ton of memory, it isn't actually in use.

This is still crazy, and excessive, but okay ig, idk.

That is aligning very well with the behavior that was occuring before, so I don't doubt it, but huh.

[oblique marten]

does that reserved memory keep growing though? it should reuse it if there's no objects in that memory

[dull hawk]

It grows a bit, and there does appear to be a little memory being added each time that should no longer be in use, but it's deeming it so.

Alright, time to put capacity limits on everything ig.

[oblique marten]

it might be good to compare two memory profiles, one before and one after updating the mod listings, before doing invasive changes

at least imo

I'm a little suspicious if it grows

if it was reserving all that memory and then only growing perhaps a tiny bit for a few runs and then stopping that would perhaps be normal

[dull hawk]

Yeah, luckily if there is a memory leak, it looks like it is a few MBs each time.

Which is not that bad considering I'm working with hundreds of MBs each time. Lol

[oblique marten]

yeah

comparing two profiles would let you know if you have a problem there

any increase in actual occupied memory would be concerning eventually I would think

especially if whatever is sticking around grows in later updates

you mentioned you had it get worse at one point

[dull hawk]

Yeah, but it may of been just a lot of reserved memory, idk.

Yep, as I thought, when I don't process the data after downloading it, it's taking up DOUBLE the RAM.

It's not bad because it doesn't add up, but still annoying.

It seems like it is adding a few extra MBs at a time before processing.

It could very well be both doing it, but I'm observing it with no processing other than saving to the file.

Oh, it looks like those few extra MBs went away.

Could be some random junk from the HttpClient object and some other things from downloading, idk.

It could just very well be the game too.

So margin of error.

Alright, well, this is great, I'm going to see if I can lower the reserved memory, and then LMM may actually release this weekend.

Oh, right, I'm going to test without invoking GC manually.

Reason why I probably never came across this type of situation before was probably because I never had 64GB of RAM and worked in C# with large amounts of data until now. Lol

[oblique marten]

when you download the data from Thunderstore, do you have it in RAM or on disk? and how large isthat file?

the managed heap growing so large may be a symptom of loading too large a string into memory and forcing that to take up residency at the end of the heap and making it grow

https://docs.unity3d.com/2020.1/Documentation/Manual/BestPracticeUnderstandingPerformanceInUnity4-1.html

this page indicates that memory fragmentation isn't actively prevented, so there can be a lot of free memory in the heap without it actually having a place to put a large allocation

if that's what's happening, streaming the json from disk instead of loading it all as a string and parsing it all at once may help considerably

I guess if it's anything like r2modman's last_api_request.json file, it's something around 155MiB?

that would probably do it lol

[dull hawk]

RAM so that the data is accessed immediately.

Then it saves it to a file for when you may reopen Lethal Company, it loads that into RAM instead of requesting the data from the API again.

[dull hawk]

Yeeeep.

So I am actually doing some things to reuse existing objects to help mitigate it due to how updating works.

(The way it works rn is that it just does everything from scratch every time, along with the main variable being the same and just referencing the new data instead.)

I could see if it would be better to keep a reference to the original string data at all times though.

[dull hawk]

I was considering such, but I don't know how well it would work, nor could I find any supporting methods for it.

Actually, wait, never mind, something I previously looked at was something I overlooked.

It did exactly that, just weirdly, found a more convenient way though.

[oblique marten]

it looks to me like JsonSerializer.Deserialize(System.IO.Stream, ..) should let you deserialize the data directly from the HttpWebRequest.GetResponseStream(), which hopefully will mean you won't be storing a massive string at all

[dull hawk]

Is that apart of JSON.NET?

[oblique marten]

there's no need for a JSON reader to have more than the current character essentially, so keeping the whole string is a waste

it appears to be?

System.Text.Json.JsonSerializer

[dull hawk]

Yep.

I'm switching from Newtonsoft anyway.

[oblique marten]

NewtonSoft actually has a similar thing

but yeah, I would use the built-in

[dull hawk]

Json.NET is a lot faster, so yeah. Lol

[oblique marten]

unless it's missing features

[dull hawk]

I think it is, but idk, I might be able to easily get around it without too much more work.

This may have to be something I would do later though because of enough complications with needing almost the entire JSON at times.

Otherwise I would have to save it with additional variables and stuff, so complexity is just rising anyway.

[oblique marten]

you can use the Stream to load the json with NewtonSoft too

it shoudlnt' be too invasive

[dull hawk]

Oh, huh, I couldn't find that at all yesterday.

[oblique marten]

I saw mention of it looking for info about streaming json yeah

create a StreamReader and pass it to https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_JsonTextReader.htm

[dull hawk]

StringReader?

[oblique marten]

no StreamReader

[dull hawk]

It says StringReader.

[oblique marten]

JsonTextReader expects a TextReader, which a StreamReader is a subclass of

iiuc anyway

[dull hawk]

Lol

[oblique marten]

https://www.newtonsoft.com/json/help/html/M_Newtonsoft_Json_JsonTextReader__ctor.htm

JsonTextReader(TextReader)

StringReader is also a subclass of TextReader

[dull hawk]

Ah, never came across a StringReader before, so I didn't know. Lol

[oblique marten]

yeah

gotta pay attention to those class hierarchies

[dull hawk]

Well, for now, due to time (and speed) reasons, I may have to wait before making those changes and just see how much I can mitigate the reserved memory.

[oblique marten]

hmm? why wouldn't you switch to reading it from a stream now? it should completely solve that

if that's the issue anyway

[dull hawk]

I still want it to be instant searching and stuff, idk if it would stay instant if I did so.

[oblique marten]

if not, it narrows down the cause to other things

wait, why would searching not be instant? are you querying the deserialized JSON when you're searching instead of creating data structures in memory?

[dull hawk]

Hold on, reprocessing everything in my brain.

Maybe I can sneeze it out.

Ah, wait, yeah, I just need to stream in the beginning, and that's it.

[oblique marten]

[dull hawk]

I might not even need to make all of my other changes for now too.

[oblique marten]

I'm hoping that's the root cause of your reserved memory issue, it would certainly make sense

[dull hawk]

Yeahhh.

Got to love working with large amounts of data, especially processing it.

Along with processing what the heck is happening half the time.

[dull hawk]

Alright, cool, part 1 done, it saves to the file without upping the memory at all.

Now I just need it to convert via Stream.

Realized I could just switch how I open the file to write, then do CopyToAsync.

Works perfectly now.

Alright, first test, let's go.

Things broke, let's go.

Second test, let's go.

Things somewhat working. Lol

I'm going to have to make it parse in pieces.

Since it ends up parsing the entire file all at once via the Stream anyway.

Oh shoot, just downloaded again in the same run, it doesn't grow anymore.

Holy crap, I may not have to do a bunch of additional crap after all.

[oblique marten]

ayy, glad it's working!

[dull hawk]

I could practically leave it as it is, but I'll make it parse in pieces for minimal reserved memory. Lol

Even though it barely does anything else already.

(Compared to before at least.)

[oblique marten]

I think with you using the Http Stream directly it should already be parsing in pieces. It should take whatever piece of data it needs next to parse the file fully, but it'll be taking it in very small parts, likely tens of characters at a time at most

so I wonder what you mean by making it parse in pieces?

do you mean it provides an API to read top-level elements of the JSON without reading from the deeply nested ones?

the way JSON has to be parsed, there's not really a way to skip the internals of an object or array, so I don't know if that would benefit you

seems to me it's best to just parse the data directly from the stream, transform it into what you need to do your queries, then throw out the parsed JSON

[dull hawk]

Yeah, I was parsing it directly from JsonTextReader, but it was still using up quite a bit of memory, it no longers adds up a lot though.

My plan was to parse each entry one at a time so that it wouldn't take up too much memory, but I think it is parsing all of it at once. Lol

[oblique marten]

yeah, technically you might be able to lazily load each entry in an array json but I don't think it's really worth doing that

it won't benefit your memory footprint at least

[dull hawk]

It still uses 1GB for it. Lol

Well, ish.

Actually, going to check Unity for more specific statistics.

[oblique marten]

you mean that's what the managed heap reserved size is after loading the json?

[dull hawk]

Quite possibly, checking to see if Unity is saying so.

After processing/converting the JSON.

[dull hawk]

Alright, well, LMM only adds 1.1GB, and barely adds any extra upon further updates, I think this is very reasonable, especially because TMM uses like ~1.5GB (easily goes up to 2GB+ though). and r2mm uses like ~0.9GB. I will obviously want to lower the additional overhead due to it stacking with Lethal Company's memory requirements with it being an in-game mod manager.

Btw, now that the memory "leak" has been addressed (ironic term), LMM is soon to be truly beta released.

[lethal gyro]

woo

[hollow epoch]

taking up my hole potato computer 💀

[dull hawk]

Ah... wait, crap, somehow I didn't realize that I had some other code commented that messes with that data.

Let's hope that metric doesn't change.

Thank goodness, jeez. I think it went up a little, but everything else is good.

[dull hawk]

never mind...

[lethal gyro]

F

[dull hawk]

Alright, well, I figured out how I'll have to handle it, the metrics aren't too high with it rn, but I have to make a few more internal changes to really get it working.

[surreal cove]

How’s karate?

[dull hawk]

I couldn't go yesterday because they cancelled due to a really bad storm, going tomorrow.

[surreal cove]

Alr maximize use your jujitsu karate Chop technique

[dull hawk]

Lol

[dull hawk]

I don't know how much longer it will be until v50, but if Zeekerss states it would be at the end of the month, I may just go ahead and release LethalModManager.

[hasty fog]

im not gonna say no to a v50 beta release, but why not release a version for v49 too

unless the v50 version just works on v49

[dull hawk]

I mentioned before that it was because of lobby finding will be unrestricted, meaning you would see lobbies of any version, which would cause mismatch problems and wasted efforts of downloading from hosts only for it to be the wrong version. (Along with other things.) Basically it would be simpler to just release as soon as v50 comes out so that there is far less human error.

I will have a setting for filtering the version, but the reason I don't have that is because there are some mods that changes the versions, which can be very annoying.

[surreal cove]

So would that be for v49 or v50?

[dull hawk]

uh what

[surreal cove]

Lmm

[dull hawk]

I'm saying that if v50 won't fully release until like the end of the month or something, I'm just going to go ahead and release LMM right now.

[surreal cove]

yes, im asking if it will release for v49 or v50 beta

[dull hawk]

...

[dull hawk]

Once again, if Zeekerss confirms the release date for v50, and it is towards the end of the month, it will release now, if not, it will release with v50.

This is just because I'm trying to mitigate human error as much as possible.

I may have to check SteamDB to see if more players are playing again, but there should be far more when v50 fully drops.

Either way, it works with any version, I'm just trying to make sure that there isn't going to be much version entanglement since v49 is the current release, but many are also playing v50.

Maybe I should go ahead and create the separate lobby browser now.

That would actually help this situation a lot.

[west surge]

its hard to figure out what to do yeah haha

[maiden bridge]

imo a beta branch on git/beta releases on git are the best option, lets people test out things to find if anything is broken and the only people who are willing to jump the hoops to install mods from git tend to be the more savvy types that want to try and break things anyway

[dull hawk]

Yeeeep.

[dull hawk]

That's not exactly the situation. The problem is just version mismatching for lobbies, which is mostly fine when no updates are happening.

[lethal gyro]

So what I'm hearing is

LMM tmrw 100% no taksie backsie

!?

[dull hawk]

Is v50 releasing tomorrow?

[hollow epoch]

is it?

[lethal gyro]

no im just being sarcastic

:>

i take all credit to being a confirmed futuresight if it does drop tmrw tho

[dull hawk]

Lol

If it does, then crap, there are literally a few more things I have to finish, but I'm house shopping right nowwwww. LOL

Me and some friends are going to probably be buying a house soon, and I've been doing a lot of spreadsheets and stuff.

You have to plan this stuff out months in advance.

[hasty fog]

"just a few more things" is this project's motto

[dull hawk]

More exact, it's like a few more lines of code, idk why I don't just finish it already.

Oh, right, I was testing some other things before.

[dull hawk]

Zeekerss is awfully silent suddenly.

He hasn't even done any changes to the actual game's private beta either.

[lethal gyro]

theres a private beta?

[thin bane]

https://steamdb.info/app/1966720/depots/#:~:text=299.55 MiB.-,Branches,-Name

[lethal gyro]

oh interesting

[thin bane]

password-protected beta (probably) for his testers

[dull hawk]

Yep. Lol

[west surge]

he's talked abit in patreon chat

[dull hawk]

Ah, I'm not subbed to that.

[dull hawk]

Zeekerss is back on the grind yesterday, nice.

[lethal gyro]

Oh?

[dull hawk]

SteamDB

[dull hawk]

[BETA AT V50] LethalModManager https://thunderstore.io/c/lethal-company/p/kyxino/LethalModManager/

Soon after beta release, I will be open sourcing LethalModManager once I verify there are no security vulnerabilities.

The reason for me open sourcing it is to contribute to the community, but also for the community to contribute back.

I obviously cannot mod full-time nor do I have the mental capacity to along with life/work hitting recently.

I also would like to get back to LethalUtilities, which will soon be open sourced as well.

I am also waiting for the full release of v50 because I don't have the time to do anything with LMM right now, which is another reason why I want to open source it for later on when that may happen again. Had I gotten the free time, LMM would've been completely finished a month ago, but obviously that is not the case.

[tepid sedge]

@dull hawk hf, v50 just dropped

[sage socket]

Oh shit?

[dull hawk]

Wait, what?

I thought that was an update to the beta release?

[dull hawk]

Oh shoot, I'm out right now.

[tepid sedge]

lmao, I bet; currently on sorta-maybe vacation rn

[dull hawk]

Well, just had to put all of my other stuff on hold, even the contract work I was supposed to finish a few days ago until other things screwed everything up.

Alright, cool, v50 still didn't break anything, everything should work, just need to do a few lines and some testing in regard to that, and LMM will be up.

[sage socket]

Woo

[hollow epoch]

i will share it with every server ever

[dull hawk]

Lol

[hollow epoch]

so if im reading this right,
everythings ready just you need to look over some code and maybe add alittle?

[dull hawk]

Yep.

Got to do some testing before.

I was expecting the release to be possibly next week, so I was going to do that before (but not immediately since I am busy), but then it released today instead.

[dull hawk]

Alright, well, this is annoying, I will be gone just about the entire day tomorrow as well.

[lethal gyro]

f

[dull hawk]

Technically today as this point.

I should be asleep right now.

[lethal gyro]

then go

[dull hawk]

I am, just having to transfer a bunch of files from my main drive to my USB because I replaced a hard drive in my old laptop that I gave to somebody.

(They have to set up their Microsoft account all over again, then I can transfer their files back.)

I also may or may not have messed up the touch pad, so now it doesn't register the clicks as easily. Lol

(I'm not going to unscrew everything and peel the chassis off to fix it, takes way too long.)

[dull hawk]

Luckily they are an involuntary victim always available tester so that I can have them test the last few things for me before release.

[surreal cove]

With more more company dying maybe your lobby expansion

[oblique marten]

wdym more company dying? I'm pretty sure it got an update for v50 already

[surreal cove]

thought I read somewhere it died, nvm then

[oblique marten]

we as a community really need to make an effort to stop sharing third hand accounts of compatibility and mod status lol

[surreal cove]

true that

[oblique marten]

if I had a nickel for every time, I certainly wouldn't be rich but I could at least get a bubblegum

[surreal cove]

I would probably more, much more

amount of times I have seen people say to use cysnc to automatically sync all configs is crazy

[oblique marten]

oh yeah

[full ether]

https://tenor.com/view/me_listening_to1700_sea_shanties-pirate-radio-dancing_pirate-gif-14077384

[hollow epoch]

Kyxnio progress update :D(didnt want to ping :P)

[dull hawk]

As I said, I was gone just about the entire day yesterday, so not really.

[dull hawk]

Well, just played v50, holy crap that was crazy, I didn't even play it that long.

I'll be finishing up LMM some more today, we'll see where it goes from there since I still have to mow the lawn today, but only takes an hour max for mowing/showering/etc.

Yep, had I released, mod sync wouldn't of worked because I forgot to change the name of a variable, lovely.

I also had something run at the wrong time, so it wouldn't have even applied things properly even after that fix.

[surreal cove]

Oh Alr

I am hyped

[west surge]

how'd you like it?

[dull hawk]

It was awesome, I'm looking forward to more updates as Zeekerss's life settles. I also saw a lot of new features for me to add to LethalUtilities. Lol

[west surge]

yee

big fan of adamance

[dull hawk]

I love how annoying, but fun it is. Lol

It definitely gets interesting when the mechs are chasing you.

[dull hawk]

Yep, mostly just testing that is left.

[dull hawk]

The one time they AREN'T available.

I got a few others things I can finish up for testing though as well.

[dull hawk]

Ironic timing, now they are.

[surreal cove]

Tbh i wasnt a big sucker for the moons, but artifice is amazinf, and the 3 new enemies

[dull hawk]

I love the additional chaos that was previously only achieved via mods.

[dull hawk]

What a lovely way for me to remember to come back to this line of code.

(That is one of the most important methods there is that I need to finish after I can test a million times.)

[lethal gyro]

cuz obv we cant do ctrl F "todo"

[dull hawk]

I'll forget. Lol

I need code to practically scream at me at this point.

[dull hawk]

Finally, things are FINALLY coming together.

Luckily the dude I work for is understanding, so I'll get that work done after this release, but oml.

I only have to bounce between 3 scripts now, mostly for testing, so I'm happy.

I'm also done with overcomplicating things for now.

[dull hawk]

I love having to ignore errors for some stuff because they contribute to nothing other than slowing me down and I can no longer be bothered to handle them if it doesn't make any difference. Lol

I'm becoming the programmer I swore to destroy. LOL

I guess this project became a bit of a wake-up call for me to realize I can't keep trying to fix every little issue, and just have to ignore and move on if it makes no difference.

I'm just stubborn.

https://tenor.com/view/i-started-blasting-so-anyway-i-started-blasting-danny-devito-always-sunny-in-philadelphia-blasting-gif-24425727

Yeah, no, I'm going to bed now, I'll deal with more tomorrow.

[surreal cove]

lol

[dull hawk]

Got to love that I forgot where I disabled some features.

Oh, never mind.

@pure halo, I see you. 👀

(Ignore the 0 downloads thing, been doing a lot of tinkering.)

[pure halo]

Hahaha

I am dropping a big update soon.

[dull hawk]

Fixed this, then I saw LethalModManager is at 180k downloads for some reason. LOL

[dull hawk]

Nice.

Well I just realized that was the light version.

[pure halo]

Yeah the patch gets applied to core and picked up by the other packs.

Excited to try what you have been cooking

[dull hawk]

Nice.

[dull hawk]

Yeah, well, unfortunately despite my best efforts to fix a lot of things, I'll just have to leave them be for now.

I will have to block updating after already loading the data once because memory usage still goes up a lot for some reason.

I've spent way too much time trying to fix it, but it's whatever, it's not going to be the end of the world for 99.99% of people.

[sage socket]

Me hoping I'm not that 0.01% lol

[dull hawk]

It's a beta, so obviously not everything typically works, but I'm a bit annoying when it comes to fixes everything right off the bat.

[dull hawk]

https://tenor.com/view/i-will-destroy-you-patrick-star-kamp-koral-ill-ruin-you-ill-wreck-you-gif-20601997

[sage socket]

Hehe

[surreal cove]

the title makes it seem like irs released

[hollow epoch]

lmao

[dull hawk]

anddddd I may or may not be working on it again

[dull hawk]

LMM will be out before MrBeast passes T-Series confirmed.

[dull hawk]

Testing to see if a specific thing is the cause, if it is, then I will fix it since I'll know how, but if it isn't, then never mind for now.

It does seem like I found the cause.

[dull hawk]

Yeah, never mind, this will take too long, I've got to stop.

[grizzled plover]

sorry if these questions have been asked before..
1 - i assume, this will auto add the mods to your thunderstore pack?
2 - will it also update mods or downgrade mods, to the same versions the host has?
3 - load host configs?

i was mainly wondering if this would be "seperate" from thunderstore or not, ie, would you only need this mod and the clients would all get full installed via your mod, love the idea btw!

[dull hawk]

Think of it like TMM/r2mm, but in game. Not as advanced for searching (yet), but it shows more stats, and allows you to go to the online listing via a web icon at the top right of the mod. It has profiles that you can switch between as well. As for the mod sync, when joining a lobby (via the Sync button in lobby search), you are prompted with the mod downloads, which you can determine what mods you want to download, it will then download them and (when finished) relaunch the game under a temporary profile and immediately join.

It does sync configs and stuff- albeit forcibly for now, so personal configs may be screwed until I finish that part of the UI.

[grizzled plover]

"relaunch the game under a temporary profile and immediately join." this is basically what i was aiming to understand, it would basically, create a profile area for the mod, and re-launch with that profile, sounds cool, and means non-hosts would effectivly only need your mod installed to play in any lobby, the mod would then get all the other mods, from the host, as a list.

love it 🙂

[dull hawk]

Yeah, as long as the host also has my mod installed.

The good thing is that a lot of people are already aware of LMM, and are waiting for it, but a lot of people think it may be dead because it has taken FAR LONGER than it was supposed to.

So it will help with everybody having the mod for mod sync.

[grizzled plover]

that makes sense 🙂 but it means for "non smart users", they can just use the same pack to join their friends lobby every time, and the mod will do the work for them.

its like what valve did for Counterstrike back in the 1.6 days, you connect to a server.. you dont have the map? download it, you dont have the mod? download it, you dont have the resources/music? download it. 🙂

[dull hawk]

Yeah, pretty much, I tend to describe it as Garry's Mod for everything. Lol

Except you get a choice for the mods you download, or to just disconnect instead. LOL

[grizzled plover]

i really need to buy garrys mod. keep meaning to xD

[grizzled plover]

yeah, same with valve, looks like its gonna take years, you see somethign you dont like, hit cancel, and d/c, that all sounds amazing 🙂

[dull hawk]

Garry's Mod was made by Valve, so I'm not that surprised you still don't have a choice at first.

[dull hawk]

Yeah, it's fun.

[dull hawk]

Wait, double checking this.

Never mind, just published by Valve, not made, but it does use their engine and stuff.

[grizzled plover]

haha, its hard to know when it comes to Valve, at one time they seemed to make everything on steam.. oh how times change.

[dull hawk]

Yeah. Lol

[grizzled plover]

well i look forward to seeing it work in the future 🙂

[dull hawk]

It may finally be this weekend since I'm just ignoring some mostly non-critical bugs.

(After like a month of trying to fix a lot of them.)

It will all still work, just the things that made it even more convenient will not.

I'm somebody who likes making things as convenient as possible, so I stressed over it way too much.

[grizzled plover]

thats how i am, more pain for myself, more easier for others who join me 😛

[dull hawk]

Lol, yeah.

The most convenient things are the most complicated for devs, typically.

That's why you don't find many convenient things in corporate programs. LOL

[grizzled plover]

i know right, before i worked for my current company, i thought all companies were like that, in my work, im lucky, cause we develop programs to do hard work, to make our own lives easier, so luckily in a corporate sense, i dont run into this issue, but ive seen it many times within the modding of games xD

[dull hawk]

Little confused by what exactly you mean, but what I'm at least hearing is that they make programs to help the programmers have an easier time working on some stuff? Lol

If so, I do the same for a lot of stuff I do, especially non-programming things.

[grizzled plover]

oh no, i mean, we the programmers, write programs, that help our customers, as well as helps ourselves, we basically write AI to do heavy lifting for us, so we do less work.

[dull hawk]

(I got a task where I have to rig a tank with realistic wheels and everything, so I made a script that I can run to automatically retrofit the rig to it.)

[dull hawk]

Ahhhh.

[grizzled plover]

im sure i dont make much sense... im fairly new to my job, so, explaining it is hard... and i try to make it understandable for everyone, so i keep it vague xD

[dull hawk]

Nah, I get it.

So you are saying luckily your job isn't as lazy as other companies when it comes to making convenient features for the customers as well as the developers?

[grizzled plover]

yes, because, my old company, i wasnt officially a programmer, but i was writing programs for them, also helping myself, but they did things VERY manually
my new company, they dont need people like me at my old company, because they cut out that whole side of things... by automating 99% of everything we do, with programs.

so ive basically gone from automating stuff at a company that doesnt like automation and easyness because "hard work", to a company that only does automation 🙂

[dull hawk]

Ah, okay, that's good- them caring about making both the lives of the customers AND the developers easier.

[grizzled plover]

indeed 🙂 which is also why your mod is so nice, because it means everyones life is simpler

[dull hawk]

I try. Lol

My other mod is so much more convenient, but for a different purpose, and VERY outdated since I've been working on LMM for so long.

[dull hawk]

Alright, final list of things to finish.
I know I've mentioned some of these things before, but I had to redo a lot of logic, so many of my work was undone.

+ [COMPLETED] Profile selection - When switching profiles, this shows the mods under the selected profile. (THIS IS NOT THE SAME AS RELAUNCHING UNDER ANOTHER PROFILE, THIS JUST SHOWS OTHER PROFILES' CONTENTS.)
+ [COMPLETED] Mod toggling testing - Whether or not the mod is enabled or disabled properly. (This was changed around a lot, I need to do a final bit of testing to verify this is working properly.)
+ [IN PROGRESS] Mod Sync testing - I have to verify that configs are being transferred properly, other than that, everything is done.
+ [IN PROGRESS] Mod extraction testing - I need to make sure that everything extracts in the proper order while maintaining speed and stability, I do have to finish a bit more code, but most of it was finished a month ago.
+ [COMPLETED] README will have to be revised a bit, not too much, so it's fine.
+ [MOSTLY FIXED, API UPDATE TIME HAS BEEN RAISED HOWEVER.] API updating - I can't fix this without spending even more time, so I'm done messing with it for now. (I may leave this as an option that you can use at your own risk, increased memory usage is very likely.)
- [CANCELLED, PROBLEM WAS MOSTLY FIXED] Option for ignoring immediate dependency loading. (This is an option that greatly benefits lower end computers because the current JSON parsing system is taking up a lot of memory when loading every single dependency and stuff, but I can't do this right not because there is no stable solution, so it will just be completely solved when I redo the JSON parsing.)
* Lobby resizing - I have to check new code to see if it will or will not fully work, but it is not a priority due to MoreCompany being a lot better than it used to be, along with LMM's lobby "ecosystem" of features not being very full yet.

There may or may not be a few more tiny details that I'm forgetting, but AFAIK, this is it.

[grizzled plover]

if you want i can look at your json parsing, i dont know why it would take up a lot of memory xD

[dull hawk]

It is because of the amount of dependencies a lot of mods (aka modPACKS) has, so when the same strings (that are converted to a custom class of mine that breaks that string up into 3 pieces: the owner, mod name, and mod version) are being remade a million times (not literally I hope), it takes up a lot of memory.

Modpacks equate to over half of the Thunderstore listings, and who knows how many versions there are compared to mods.

Mods -> Versions -> Dependencies

So it grows a lot.

Soon I'm going to redo the parser to allow it to reuse those already made (custom) classes based off of the string. (So if it already exists via string, it will just give the custom class instead.)

Currently I'm just using Newtonsoft's JsonTextReader with Streams.

[grizzled plover]

so, its more the searching over thunderstore for the dependencies thats making it take a lot of memory?

[dull hawk]

It takes all of the data from the API (which is a 160MB JSON file), then parses it all at once.

[dull hawk]

Either I do some sort of progressive parsing that only parses when needed, or I cache a lot of the custom classes as I mentioned here.

[grizzled plover]

can you not use the api to search for the exact mod, and only pull and download that one result?

[dull hawk]

A lot of tiny API calls can be worse than one big one.

Plus, you can't really get all the data you need from just one.

Especially since there are no batch ones.

It's either one at a time, or all at once.

[grizzled plover]

not as the library expands.

something i learned with programs i wrote for my old company

somethign has 1.. easy
somethign has 1000.. bloated file, everything slows down as size increases.

1 would take 2 seconds
1000 would take 4000 seconds... for example.

link me to the api? (sorry, im really not familiar with lethal company coding xD

or thunderstore xD

i guess i can look it up myself actually, nvm

you are using the dependency strings like thunderstore does, yes?

[dull hawk]

Yep.

Example: kyxino-LethalModManager-0.0.2

https://thunderstore.io/api/docs/

[dull hawk]

The file downloads pretty quickly (though not for all, likely), but once it downloads, everything else is really fast.

Especially when searching, which is another reason why I download everything.

TMM/r2mm does the same.

[grizzled plover]

ok, so rather than downloading, why not use the file already downloaded by thunderstore every time it opens?

[dull hawk]

I am downloading that file, the same way the others are, I just parse it a bit differently to make it work better with LMM.

(It also allows more features, and makes more things run faster.)

[grizzled plover]

yes im saying, dont download it, use the one thunderstore downloaded.. as it already exists?

[dull hawk]

Not everybody has TMM/r2mm.

[grizzled plover]

true, but if they dont, then you can download it, it makes it faster for people who do.

[dull hawk]

Plus, that doesn't make a difference on memory usage.

[grizzled plover]

well true.

[dull hawk]

Yeah, and that is a plan if people want to enable it.

I have multiple plans for that stuff, along with a custom API for those with more restricted speeds.

Though Cloudflare (according to multiple people) requires compression, so things actually download a lot faster since it gets compressed a lot. (Thunderstore uses Cloudflare.)

[grizzled plover]

i was writing a lot, reading what i wrote, and thinking to myself, not really much i can suggest.. if you think the download all way is better, then hopefully the compression makes life easier.

a lot of my coding for work makes use of multithreading... but my experience with bepinex admittedly hasnt led to me seeing much implementation with multithreading, and if that is not possible, then you maybe right, but an application i wrote bulk processed files, i split it into queues for every core, it was much quicker than doing a big file one by one.

well much quicker if the user had 8+ cores... didnt really test below that.

[dull hawk]

Well, either way, the main concern is memory usage, I'm just either going to have to do some caching or just parsing dependencies and stuff when things are needed.

The only real solution will probably be caching because more and more modpacks will just render the progressive parsing useless.

Well, eh, not really, just another stat wouldn't be available, unless I have an option for both, in which case, yeah, it would eventually be useless.

[dull hawk]

Profile selection is now working, you can see your list of mods regardless of your active profile.

Going to update that README again. Lol

README is updated.

So far everything left is just a bunch of testing and a bit more programming depending on the results of those tests.

[surreal cove]

Do we have an ETA for release

[dull hawk]

If I can get the testing done near immediately (as some of it requires others for mod sync), I'd say Sunday to be safe.

Only after today will I really be sure to say.

If I can knock 2 more things done, then definitely Sunday or before.

I am admittedly taking things a bit slowly now because I don't want to rush, and I really need to make sure there is nothing else crucial missing, but at this point, I've checked so much where there shouldn't be ANYTHING crucial missing.

[dull hawk]

I JUST PULLED A MAJOR BIG BRAIN.

HOLY CRAP

I don't need to change the actual JSON parser at all.

I CAN CHANGE THE CLASS PARSER TO CACHE INSTEAD.

OVER 1GB HAS BEEN SAVED EASILY.

[surreal cove]

not the first time you said, hopefully the last🤣

[surreal cove]

yippeeeee i have no idea what you said, but lets goo

[dull hawk]

Yeah, it may really be considering how things are going, I found a few more things I got to redo, but they aren't crucial for release. Lol

[surreal cove]

also not the first time you said that 🤣 but keep up the grind, it is def amazing how you were able to keep devolping the same mod for so lomg

[dull hawk]

Well, it is the first time I said I found some things that I got to redo that aren't crucial for release.

[surreal cove]

ye

[dull hawk]

As for the considering how things are going, yeah, that definitely isn't. LOL

(because immediately an issue would slam me in the face)

[surreal cove]

lol

[dull hawk]

and that issue has been bugging me for over a month, and I've yet to fix it

UNTIL NOW

I wasn't even trying to fix it, it just came to mind suddenly for the solution. 😭

[oblique marten]

class parser?

[surreal cove]

lol

[oblique marten]

I am intrigued

[dull hawk]

Idk how to word it.

The parser that I have for a class when the JSON parser is parsing a string.

[oblique marten]

like the deserialization method that takes some JSON object and turns it into a class instance?

[dull hawk]

YEP.

[oblique marten]

making it lazy or something?

[dull hawk]

My brain fried. Lol

I made it cache the previous results.

[oblique marten]

oh why was not doing that causing a leak?

was it not GCing the ones that you were done with?

[dull hawk]

I have everything loaded at once, but the amount of dependenices that got parsed into its own ModInfo class shot memory usage up like crazy.

[oblique marten]

ohhhh

[dull hawk]

Yep.

[oblique marten]

yeah that would make sense lol

[dull hawk]

Yeah, CONSIDERING MODPACKS ARE MORE THAN HALF OF THUNDERSTORE.

[oblique marten]

right

[dull hawk]

Who knows how bad it is counting versions for mods vs modpacks. LOL

I don't even want to swallow that pill of how bad it is.

[oblique marten]

perhaps it would even be worth converting the json into some other form of database that's more query-friendly

(assuming the memory footprint is still sizeable after this change)

that way, you should hopefully not have to even have everything loaded into memory, just query the things you need when you need them

idk how worth that is though

[dull hawk]

I mean, this works great now, it's not even that bad, I probably still have to change the JSON parser to handle it all better, but this was a HUGE improvement, even if you update.

[oblique marten]

I believe it for sure

that recursivity had to have been pretty awful

[dull hawk]

Mainly just the dependencies. Lol

This might even be faster for loading the extra data LMM uses.

[dull hawk]

Wait, yeah, idk why I keep thinking I need to change the JSON parser, I can just do the same thing with my other classes.

[west surge]

Dumb question that your probably already doing, but you can cache/predict certain dependencies right?

[dull hawk]

Uh, what?

[west surge]

like if LethalLevelLoader is dependent on LethalLib, any mod dependent on LethalLevelLoader is going to be dependent on LethalLib

[dull hawk]

Oh, yeah, I call that flattened dependencies.

[west surge]

less a suggestion more curiousity

[dull hawk]

Mod A requires Mod B, which requires Mod C.
Mod A therefore requires Mod B and Mod C (with a different level unless Mod A directly requires Mod C)

It's the way I handle mod downloading as well, that way it doesn't have to search through a bunch of dependencies and stuff.

It just has the list ready for the order and everything.

It's very helpful for modpacks that list dependencies manually.

Along with informing users what they'll actually be downloading, but I don't have a popup menu for that, so they have to check Thunderstore's page if they really want to know. Lol

[hollow epoch]

@dull hawk question, just from your testing, how long does it take to, join a lobby, download the mods, then relaunch the game?

[dull hawk]

I haven't really gotten there yet, but I have really fast internet, so when I downloaded a huge modpack (TMLC v7.3.0), it downloaded and extracted REALLY quickly.

As for relaunching the game, it is painfully slow depending on how many and how big the mods are.

[west surge]

will be a lot less of a problem in the next couple weeks dw

[dull hawk]

That would be awesome.

I also will be helping with that by recontinuing LethalUtilities so that there aren't as many tiny mods again.

(A lot of tiny mods SUCK for load times.)

[west surge]

experimental LLL can auto async**** load related bundles for all types of content

and people wanting to do stuff w/ code can register to get callbacks when i load it and stuff

[dull hawk]

Nice.

Looking forward to it, it will help greatly.

For both LMM and Lethal Company modding in general.

[dull hawk]

Well, I think LMM may now be better than both TMM and r2mm in terms of memory usage, which I never expected to get to this point.

After all, memory is being taken up by the game ALSO running.

Very little, but still.

[dull hawk]

Actually, almost, but the moment you start doing a lot more stuff, it does add up a bit more.

It does give a bit of a headache though since everything is RED.

Can't wait to add themes. Lol

(I technically already have, but it is nowhere near done.)

(It was just some gimmicky thing.)

[dull hawk]

Oh, that was mostly just Task Manager being the brightest thing on my screen, never mind. Lol

[west surge]

btw completely unrelated but figured worth mentioning since you seem like a nerd in a good way and have experience w/ the codes

this server would massively gain from a bot command that takes a profile code and outputs a modlist .txt

[dull hawk]

I could easily make that, actually.

I know exactly how.

[west surge]

I figured 😛

right now we get so many people being like

"i have x problem" + code

and its like, people aren't gonna go download what might be a couple gb's just to tell you to uninstall LCAPI yknow

[dull hawk]

Yeah.

Well, I could ask Robin or Albino about it, if they want it, but Albino could probably make it really quickly as well since he has a bit of experience (I think) with how to make that work as well.

[solemn terrace]

sadly not very easy

[dull hawk]

It is very easy?

[solemn terrace]

the server has no concept of what is inside the profile files

❌

[dull hawk]

I've messed with it before.

[solemn terrace]

it's easy to do w/ a client script

it's not easy to do on Thunderstore's side

because they would need to create a new database, process all the existing profiles to populate the database

[dull hawk]

uh what

[solemn terrace]

when I say 'process' I mean, base64 decode the file they have stored, and then unzip

[dull hawk]

Yeah, it's easy.

[solemn terrace]

oh 'THIS server'

[dull hawk]

You don't need to create any new databases.

[solemn terrace]

I read 'THE server' and assumed he meant Thunderstore's API

[dull hawk]

im confused

[west surge]

https://media.discordapp.net/attachments/807809192537882647/1037946021533204550/1CFC6F34-BB31-40F6-AA72-DE250F097A90.gif

[solemn terrace]

don't worry, what I'm talking about would only apply for adding the functionality to thunderstore API

[dull hawk]

AH, I see what you mean now.

Yeah, I was confused at first.

[solemn terrace]

since doing the decoding/unpacking isn't really viable for every API request

[dull hawk]

Nah, the bot would just do that instead.

[solemn terrace]

Yes, I realise that

[dull hawk]

I've done it before when testing LMM's mod sync a few months back in Unity. Lol

Jeez, the old UI sucked.

The last (publicly known?) iteration before it was completely redone. Lol

It was so bad. LOL

Thank goodness I redid it.

[solemn terrace]

😬

speaking of game integration UI, my favourite new discovery is

LuckPerms

https://luckperms.net/

[dull hawk]

Seems nice, a lot of whitespace though.

A lot of bulky elements that could be slimmed down.

[solemn terrace]

https://luckperms.net/editor/demo lets you edit permissions for your minecraft server in a webapp, and push the perms to the server hassle free

[dull hawk]

Definitely isn't my favorite UI, but still very convenient.

[solemn terrace]

the UI might not be perfect, but the execution of the ease-of-use is

which is what excited me about it

[dull hawk]

Yeah.

[thin bane]

my favourite as well

it's saving so much time

[solemn terrace]

it's been a few years since I had to do anything perm related, I think the first time round was my first ever time seeing JSON ... I was lost as all hell 😂

It was just such a refreshing experience

like 'holy shit, this is so much better than it used to be'

[thin bane]

you send one command in game and everything is done

so easy to use 🥹

[dull hawk]

Well, had my computer blue screen, got to love spending hundreds of dollars on cooling, but the 13900k still overpowers it.

(At least I think it blue screened because it overheated, I did have my CPU at near 100% for quite a bit.)

[dull hawk]

Though I think I had something wired wrong, so maybe it should work better (if not, properly) now.

[sage socket]

I really hope LMM is backed up to like GitHub or something lol

Your computer does not seem to like you working on it

[dull hawk]

Nah, I was just doing quite a bit of stuff, everything is backed up though. Lol

[tidal fossil]

is the backup backed up too?
👀

[dull hawk]

Lol, ask Github.

[dull hawk]

I'm confidently not confident in today's progress, but literally only 3 things left to do. Problem is, I'm gone later, and XFINITY IS DOWN AFTER A POWER OUTAGE.

So I can't test mod sync out.

(I'm running on my hotspot rn.)

Somebody explain to me how this makes sense...

[dull hawk]

XFINITY IS BACK UP.

[thin bane]

the drivers not downloading on metered connection is the best and worst thing at the same time

[dull hawk]

The thing is, I've had this camera plugged in for months, any drivers should've already been downloaded. Lol

[thin bane]

l m a o

that's even better

[dull hawk]

Me: tells friends and family I'm finally finishing LethalModManager today
Friend: asks to play
Family: asks to watch movie
Me: finally gets time
Also me: uses bathroom for a while
ALSO MY FAMILY: PROCEEDS TO SUDDENLY DO THINGS AND ASKS ME FOR HELPI
Me: only has 6 hours left in the day...

I should just never tell anybody I'm close to finishing anything so that this doesn't happen again because what the heck.

[surreal cove]

so

prob not releasing today?

[dull hawk]

Good thing I never promised weekend (like usual), but it might still be today, just eating before I get back to it.

[dull hawk]

Just replying to this for me to look at again.

I can definitely knock out mod toggling and mod extraction testing today.

We'll see about mod sync.

[dull hawk]

Alright, mod toggling works perfectly, I forgot about mod deleting, but I literally figured it out within minutes, just adding better visual indicators, I'll soon add proper replacement icons for such.

On a side note: I got to redownload a mod, but luckily I am literally working on a mod manager. Lol

@pure halo, how am I constantly coming across your modpack now? Lol

Found an "issue" with mod downloading, but instantly fixing.

[pure halo]

Because I constantly update it. hahahaha

[dull hawk]

Maybe I'll have to sort by how many versions to check what rank you are with updates. Lol

[pure halo]

Rank?

[dull hawk]

Just wherever you are showing.

[dull hawk]

[BETA TOMORROW] LethalModManager https://thunderstore.io/c/lethal-company/p/kyxino/LethalModManager/

[full cliff]

👀

[pure halo]

https://tenor.com/view/right-yeah-right-yeah-oh-yeah-ohh-gif-14553729619893238098

@dull hawk I believes it when I sees it!

[dull hawk]

I'm only doing it tomorrow because I didn't have sufficient time to test today, other than that, everything is done within the hour.

[BETA 4/22] LethalModManager https://thunderstore.io/c/lethal-company/p/kyxino/LethalModManager/

That's a bit better.

[surreal cove]

im sorry i dont beleive it

[dull hawk]

You don't have to, it's happening tomorrow.

[surreal cove]

its not possible

[dull hawk]

I don't know if you are messing around or not.

[surreal cove]

nah the real question is, are you messing around with us or not 🤣

[dull hawk]

No, the main problem (that memory leak) has been fixed, it was the main reason for it being delayed because I thought I had fixed it many times before, and it turned out I didn't, but now I really have.

That problem had been around for over a month, maybe even 2.

Part of the reason for me practically remaking LethalModManager was because of that issue.

(Mostly because of lacking support for other features I was wanting, but it was also harder to add that support when resources were constantly being taken up by a memory leak.)

Then between the end of March and a week ago, I started really slowing development down and focusing on other things because I needed a break.

Considering how I fixed the problem within a week that I previously tried fixing within a month tells me that I should've taken more breaks or spent less time on LMM every day. Lol

[dull hawk]

Honestly kind of ticks me off, but then again, my fault for not being healthy about it, especially with how bad my sleep schedule was back then.

[surreal cove]

great to hear but im happy that we are finally gonna get it, you eorked hard for this moment

[dull hawk]

Yeah, things will finally be a lot easier for me. Lol

(and for others as well)

[burnt sky]

Just checked today after goin weeks without check in this and it’s really comin out today! Exciting, gonna allow people to join lobbies with ease and play modded with strangers

[dull hawk]

Yeppp, working on the final 2 things right now.

[dull hawk]

ok i just realized another issue, im fixing it after release, it shouldn't be the end of the world, idc

It's funny that I was going to make a simple config syncing mod back in December, and then it led to one of the most complicated things possible.

[hasty sequoia]

pls release it in 2 years it would be funny

[dull hawk]

I would probably not even be in the modding community at that point.

[hasty sequoia]

make a bot that auto releases it in 2 years

lol

[dull hawk]

oml

[odd knot]

When will this release

[dull hawk]

Later today.

[odd knot]

So what dose it do. I heard some people say that it will auto install mods that other people have

[dull hawk]

Essentially, yeah.

Think of it like Garry's Mod when you are joining lobbies, but instead it prompts you with the mods to download or whether you just want to cancel and try another lobby.

[odd knot]

Oh ok

[dull hawk]

If you accept the mods to download, it will download and install all of them to a temporary profile and then relaunch the game in that profile.

[odd knot]

Can you add support for advance company

Since it’s not on thunderstore anymore

[dull hawk]

If there is an easy way for me to do so SECURELY, possibly, but I'm trying to get this to full release as soon as possible.

[dull hawk]

Well, I just found out a VS2022 feature by mistake.

[pearl plover]

good luck with that, mod manager work is hard to escape

[dull hawk]

Nah, I meant if I am still working on trying to get a beta out.

[pearl plover]

oh lmao

[dull hawk]

Yeah, if it took 2 years for me to get a beta out, I'd just call it quits.

Especially because who knows if Lethal Company will last that long to make that much of a difference.

Alright, yeah, I'm not spending anymore time trying to make multiple mods download at once and stuff, I'll figure it out after release, I have it mostly done, but I know it will probably have some little additional issue, and then it will delay, so I'm just making it download one at a time.

That conversion was fast. LOL

I'll finish all of the "gimmicks" after beta release.

Fixed profile loading not working properly with the main directory and temp profiles (if they are still showing for whatever reason).

[dull hawk]

It still downloads and extracts mods really quickly, actually.

[hollow epoch]

hes gone insane from working to much... talking to himself

[dull hawk]

Getting people to help test mod sync, if this works out, LMM will be out, and I am placing my bets on it working since the main problems were just getting the mod manager working, which it is.

[odd knot]

I can test

[dull hawk]

I'm waiting on 3 private testers in my Discord right now.

If they can't, sure.

[odd knot]

k

k

[dull hawk]

Got somebody to help test.

Here's an accurate representation of what I see when I'm programming.

That isn't even much of an exaggeration, that's pretty realistic. Lol

[tepid sedge]

blinds/curtains go brrr

[dull hawk]

My mom doesn't believe in them practically. LOL

I'm finally getting some after complaining for a long time though.

[dull hawk]

(We only have one curtain and one set of blinds in our house: curtain in my bedroom, and blinds in their bathroom.)

[surreal cove]

How’s progress?

[tepid sedge]

blinding

[hollow epoch]

whos dat in the bottom right lmao

[dull hawk]

BigPuffer

[surreal cove]

Looks like a streamer

[dull hawk]

Great, just one problem that I'm trying to fix with a tester rn.

Well, one big problem.

It's the configs, everything else seems to work.

[surreal cove]

Bet is the goal still today?

[surreal cove]

What’s wrong?

[dull hawk]

Yep.

[surreal cove]

Bet

[dull hawk]

Configs aren't decompressing properly.

I think, at least.

[surreal cove]

Oh Alr

[dull hawk]

Yeah, something is going wrong with decompression.

WE'RE ALMOST THERE.

[west surge]

Watch us release the same day lol

[dull hawk]

You about to release the new version of LLL? Lol

[west surge]

ye

Probs tomorrow

[dull hawk]

Ah, then probably not the same day. Lol

[surreal cove]

YESS

u actually motivated me so I can finish all my work before it comes out lol

[dull hawk]

Lol, nice.

[burnt sky]

Hope this gets its well deserved recognition, crazy convenient and multiplayer modding compatibility will definitely change the lethal company random lobby experience

[surreal cove]

absolutely I beleive a public announcment should be made

also sorry to keep bothering, but progress 🙂

[surreal cove]

also should i make a brand new profile with just LMM

[dull hawk]

You can, I have it where it sets up your main directory (the Steam directory) with LMM only so that you still are playing vanilla, and can quickly load into another profile from it.

LMM has its own profiles as well.

Other mod managers are not yet properly supported by LMM for downloading mods and such, so yeah, I don't know how that would work, but for the basic mod sync stuff, everything should be perfectly fine.

[dull hawk]

Thanks! I have been trying to get this as convenient as possible, unfortunately some things remain a bit janky since not everything has been rewritten properly, but that will be happening soon after release, I actually already have a bit of the code ready for such.

[surreal cove]

bet bet

[dull hawk]

Alright, things are mostly working, just getting configs properly synced up and we should be good.

Oh, right, an issue with the menu staying up for the host after opening a lobby, got to fix that.

[maiden bridge]

https://tenor.com/view/dawn-majora-finalday-gif-9607331

[dull hawk]

Minus almost an entire day.

[surreal cove]

consisering the day starts at 6:15am yea

[dull hawk]

I have less than 2 hours left in the day.

[hollow epoch]

so is it realising tomorrow or later this night? its 8:31pm for me

[dull hawk]

Today, I'm pretty dang sure I can get this out today, if not, I'll stay up until it is released.

but I'm very sure it is within these 2 hours.

[hollow epoch]

https://tenor.com/view/shameless-attention-yes-sir-soldier-army-gif-10238483

[dull hawk]

Figured out the config situation, everything should be working, fixing the last few bugs.

WE ARE ALMOST GREENLIT, PEOPLE.

BREAKING NEWS: AN ASTEROID IS HEADING FOR A LETHAL COMPANY MODDER NAMED KYXINO.

man if that actually happened, i wouldnt be scared, i'd be annoyed

Already had a PSU blow, multiple power outages, a blue screen, components overheat (both main and alt drives along with my CPU), internet go out, etc.

[maiden bridge]

Summoning 50 billion new bugs in the code through mental projection

[dull hawk]

LOL

I really don't know how I threw myself into one of the most complicated things possible.

I could've done a program that attached to Lethal Company instead and it would've been so much simpler.

[dull hawk]

YEP, CONFIGS VIA MOD SYNC SHOULD BE WORKING NOW.

About to test with one of my private testers.

[dull hawk]

IT WORKS

Truthfully, I could release right now, but I got a few more bugs, and it's already 13 minutes away from 4/23 for me, so nobody really cares too much I'm sure, so I'll just finish them off for the night, and release.

Which means LethalModManager is coming out in a few minutes or so.

Oh yeah, I got a lot of debugging messages to get rid of. Lol

[hasty fog]

i can't believe it's finally happening

[dull hawk]

Don't because I'm deleting everything. jk jk, i would hate myself if i gave up on it

but yeah, it's finally happening, and I can now have peace of mind until the bug reports on everything I know is broken is reported a million times.

Should've just released on the dot and released a patch right after. Lol

Having to make a few things a bit inconvenient for stability sake, but I'll restore proper functionality of everything within the week probably.

(I'm instantly rewriting a few last things after I release tonight.)