Production Rates Calculator Extreme - Fa...

[hasty vortex]

Well well well

Is the extreme part that it uses simplex?

Also, how do you do the mod scripting compatibility?

[deep fulcrum]

"extreme" because it uses simplex algorithm, and also because it supports most space age stuff, plus mods

scripting compatibility is done via lots of remote calls. everything is just under the hood.

https://github.com/SWeini/sw-rates/blob/main/sw-rates-pyalternativeenergy/configurations/py-digsite.lua
here is an example: the digsite from PyAE
(ignore fill_progression and fill_basic_configurations, those are not used at the moment)

[hasty vortex]

Interesting. Normal rate calc might get a solver too I think, but might work differently.

[deep fulcrum]

normal rate calc has a fork with simplex solver
but it still has a lot of issues with fluids/heat of multiple temperatures

[hasty vortex]

Oh don't we all

(I do Factory Planner and do have those issues still lol)

[deep fulcrum]

I tested my mods heavily with PyAE, and that mod is known for heavy use of temperatures. Most of the time there are no issues.

[hasty vortex]

I'm asking about the mod compat stuff because I have something in mind that sounds very similar to what you've done. Maybe there'd be a point to standardizing this format in some way.

[hasty vortex]

Nice, it's no easy feat for sure.

Oh you already set it up as a framework of sorts I see

[deep fulcrum]

here is some heavy-temp-usage proof

yeah, that API has evolved over multiple iterations of my own
I really tried to have it as extensible as possible

[hasty vortex]

Yep that is one recipe that breaks on FP lol

[hasty vortex]

I imagined it where the mods would carry the compatibility code, and thus updated it themselves when they change their code. That both makes it always synced and doesn't put the support burden in a central place. Is yours meant to do that or are you collecting them centrally?

[deep fulcrum]

when everything is built things become easier, because the fluid ingredients know their temperature
things become tricky when everything done with ghosts because then the fluid temperatures can't be resolved early

yeah, mods doing the compatibility themselves would be ideal state
just not possible without an API available in the first place

[hasty vortex]

Yeah for sure. Very cool, I'll have to have a look at this in more detail sometime.

[hasty vortex]

Yeah it's funny, it's the same problem from a different direction than Factory Planner, so it shares some issues and has some different ones.

[deep fulcrum]

yeah, please take a look and give feedback. doing something together is definitely possible.

just disclaimer: some parts are more finished than others

[hasty vortex]

Happy to hear it!

Oh no worries, I haven't done any work on this from my side yet.

[deep fulcrum]

you can see lots of stuff in there that is totally inaccessible at the moment, e.g. there are recipes for spoiling or sending items to orbit, cross-surface transfer, and stuff - all of which is not accessible from just selecting entities

all in preparation for doing something like FP

[hasty vortex]

Oh no, competition

[deep fulcrum]

btw, would it be possible to provide access to async google or-tools simplex solver via API?
many mods of this kind could benefit from it, would be massively faster, more robust, and async

[hasty vortex]

I appreciate the comprehensive README for sure, esp. for such an early stage.

[deep fulcrum]

early stage = in the making since many weeks 🙂

[hasty vortex]

That's early! FP is almost 6 years in development by now :p

[deep fulcrum]

also, I was in contact with raiguard and hoped he could give feedback

thus the extensive readme

[hasty vortex]

With a program running on the player's machine yes, but kind of tedious and also that reduces your audience to almost 0.

[deep fulcrum]

google or-tools is a simple C library you could link to - not doable as a mod
I already fear some performance issues for large selections - probably have to do the simplex calculation async (little bit per tick) anyway - if it was running async in the engine (similar to requesting localizations) that would be so much less effort to do in the mod

why would google or-tools reduce the audience to 0?

[hasty vortex]

Because practically no one will install an additional exe to run a mod.

[deep fulcrum]

that's why I asked to provide this in the factorio install

[hasty vortex]

Oh I see, didn't get that

Very very unlikely, but you can always ask on the forums.

Also running stuff async would be great, but is also unlikely to be added. I'd love it though.

And yeah my matrix solver does have performance problems for larger setups, so the worry is warranted 😅

[deep fulcrum]

your matrix solver is not simplex though, is it?
you have to select the "free" items upfront and then it solves a normal set of linear equations, right?

[hasty vortex]

Yes

Is simplex slower than that?

[deep fulcrum]

simplex has a very bad theoretical worst time, even though in practice it runs quite well

the problem is that the matrixes become quite huge
maybe not for FP or selecting parts of the factory

but long-term I'm planning to add milestones and cost calculation (similar to YAFC), and for the last one you put EVERY recipe in EVERY machine with all meaningful module combinations into the matrix

the thing with simplex performance is:
a performant implementation needs performance from an algorithmic point of view (revised simplex method, sparse matrix, few other tricks), and low language overhead (C vs LUA, who wins?)

[hasty vortex]

I see. Well when I said 'performance problems', I meant that it might hitch slightly when it recalculates. Which is okay-ish, it's just that it recalculates a lot, after every change. I assume simplex would potentially take several seconds or more?

[deep fulcrum]

first one is doable with enough effort, but second one will be out-of-reach for mods

[hasty vortex]

Yeah Lua isn't slow, but it's not C-speed.

[deep fulcrum]

funnily, when I tested in my current PyAE save the larger performance cost was reading all the entity info
probably because I look for beacons around every entity (mod supports beacon ghosts)

[hasty vortex]

If we ever get LuaJIT that'd help with speed.

[deep fulcrum]

selecting my 400 moss farms takes roughly 1 second

[hasty vortex]

Yeah that and accessing API properties is sort of slow, relatively. Since it's a function call and the data needs to be pushed from C to Lua.

Could always do some profiling though to see what takes the time for real.

[deep fulcrum]

I've never done lua profiling, when the mod gets some audience I will probably invest more time into it

[hasty vortex]

Oh for sure, would just be for curiosity. There's this pretty much https://lua-api.factorio.com/latest/classes/LuaHelpers.html#create_profiler, or FMTK can do profiling as well.

[iron tapir]

not serious but lol

[hasty vortex]

With regards to which part? lol

[iron tapir]

this

[hasty vortex]

Man I feel weird about it, if ever there is a better planner mod than FP, will I be relieved that my job is done and I can do some other side projects besides this one for years on end, or will I be sad. Not sure tbh.

There's definitely a market for a good planner mod though, either way! It's not a new standard, it's competition!

[deep fulcrum]

doing all the math for the planner is no issue for me
it's the GUI that gives me headaches

[iron tapir]

that I can help with

[hasty vortex]

Well the GUI is the hardest part :p or at least the part that's the most work.

I spent maybe 10% total dev time on the solver side (which is why the solver kinda sucks but yeah it's not the main part, paradoxically)

[iron tapir]

I'm still amazed that you did FP without some sort of library or framework

[hasty vortex]

There weren't any back in my day!

[deep fulcrum]

even getting this 2-column layout was a lot of effort (first mod with GUI for me)
and I couldn't do it as intended (both columns with full height, but window height dynamic based on content, with scrollbars if too large)

[hasty vortex]

I wrote the mod for pre 0.17 GUI stuff originally, it was incredibly basic times.

[thick mountain]

kudos to fp's gui code, ive peeked once or twice and the structure doesn't reek of technical debt, nice and modular even though its a crap ton of total code

[hasty vortex]

This is it looked like back then, for reference 🫣

And that was one of the nicer GUIs I'd like to think.

[thick mountain]

ah the old drill icon, that takes me back

[deep fulcrum]

production rates on top of the icons - too little space in my opinion. I tried it, didn't really fit

[hasty vortex]

Thanks :) I don't like the GUI code details, it's kinda messy to do it vanilla, but I'm not convinced frameworks are better.

[iron tapir]

is that pre- gui refresh?

[thick mountain]

yeah im still a little on the fence too

[iron tapir]

I can agree with that sentiment a little bit, though the two main gripes are much nicer with at least a tiny framework than handling it myself every time

[hasty vortex]

Weell it's on 0.17, but was written for 0.16, so it's weird. I don't know if I have a 0.16 screenshot, that was pre-release of the mod. Would look even worse tho.

[deep fulcrum]

I found the flib gui part OK to use

[hasty vortex]

Well I do have a whole very involved thing for event handling, that part is definitely crazy otherwise. But just defining the GUI elements I don't really feel the need for a framework.

Also I just have one giant mod, if I did many smaller mods I might do a framework. But because my mod has so much GUI it makes sense to do more things custom imo.

I'd probably use a framework if I did a smaller GUI mod.

[iron tapir]

the two main gripes I have with vanilla gui, event handling and multi-element creation - I don't want to have to make an entire event handling system every time, and I also don't want to have to save every single element with children I make into a variable that I have to name

[hasty vortex]

Yeah makes sense.

[deep fulcrum]

these are very good reasons. and flib solves them, so I just used it.

[hasty vortex]

One thing also is that I have to be careful with dependencies, if I used flib for GUI I'd have had to rewrite like 8k LOC of GUI code a few times by now since it changed several times.

But yeah that's big mod problems, I'd use flib for a small mod probably.

or another framework idk

[deep fulcrum]

I'm wondering how often the sw-rates-lib API will change - probably a lot before 1.0
really trying to stabilize it and get some feedback

[hasty vortex]

No need to worry for now since only you are using it.

[deep fulcrum]

for now yes
and if I change it every week no one ever will

[iron tapir]

for example this is the entire export section of my ui for recipe tweaker 2.0, besides the actual event handlers

[hasty vortex]

Afterwards you might need some versioning of the API. Or just not do breaking changes.

[hasty vortex]

Doing it declaratively is nicer on the eyes for sure.

How's performance?

Because GUI performance is definitely an issue in FP and this is probably slower.

[iron tapir]

performance is barely worse than if you had done everything crudely

[deep fulcrum]

GUI performance an issue? what exactly, and are you redrawing everything on every change?

[hasty vortex]

Yes, everything else leads to madness.

[iron tapir]

as far as the event handler goes, this is the whole thing

[deep fulcrum]

I'm doing the same currently, didn't want to go mad, too. I can relate.

[hasty vortex]

Well not everything, but if something changes it gets redrawn, not modified.

[iron tapir]

so just checking a couple things and then calling the function directly

destroying and recreating elements is certainly a hit, I try to avoid it on larger uis

[hasty vortex]

It's also only an issue in multiplayer, in singleplayer you don't notice.

[deep fulcrum]

oh, I haven't looked at multiplayer so far at all

[hasty vortex]

It's just that mod GUIs are calculated for everyone in everyone's game, so if you do very heavy GUI stuff everyone's game lags even if they are doing something else. It's not a huge issue but you can't really take too too many milliseconds for your GUI.

[deep fulcrum]

ok, here goes my dream of having full py base in single GUI

[hasty vortex]

I was just thinking that creating the table, then parsing the table to create the elements must be slower than just creating the elements. Not sure by how much tho.

[hasty vortex]

Well you can do that if you want to build your GUI over multiple ticks :p

[iron tapir]

the args field in the element structure gets passed directly to LuaGuiElement::add() so it's not as bad as you think

[deep fulcrum]

oh dear god
not enough of running simplex over multiple ticks, now this

[hasty vortex]

Yeah mods aren't ideal for super heavy stuff unfortunately.

[deep fulcrum]

you know, I have done some YAFC stuff, and external programs also have their downsides

[hasty vortex]

Oh for sure, I much prefer in-game. I'm actually planning on adding more stuff that only ingame calculators can do, just to provide value that the others can't.

[deep fulcrum]

getting the data (correctly), dealing with all the edge cases, bringing the API to another programming language, syncing with progress in a save, ...

[deep fulcrum]

oh nice, what did you have in mind?

[iron tapir]

like yeah there's a tiny bit of overhead from creating the table structure and iterating over it to make stuff but that's all regular lua stuff, the biggest perf hits from gui stuff is the api calls, which there's the same number of

possibly even less since you're not indexing style all the time after making the element

(unless you're baking your styles in which case never mind, but you can do that with the lib too)

[hasty vortex]

Lots of vague things, stuff like tracking existing mines and comparing that to your factory's needs, similar thing for research labs and power production. Also tracking real-time research progress and adjusting accordingly in a few ways. Some stuff where you can select your existing factory to verify it, or import it, or to generate a blueprint for your plans. Most of these will probably not happen but there's a lot there with enough time and effort.

[hasty vortex]

That's a very good point, I'm very lazy about creating style prototypes.

[iron tapir]

I can imagine production statistics would be the main way of going about that kind of thing

[hasty vortex]

So it probably comes out about even.

[deep fulcrum]

comparing with real production stats will be difficult, because you can't easily separate the stats from a small build from the rest of the factory, right?

[iron tapir]

true

[iron tapir]

flib's element creation performance is a bit worse because it doesn't have the args table, it just extends the LuaGUiElement.add paramters, and it has to store + remove the keys from the args table, add the element, then re-add the stuff back into the args table as to not modify the table that's passed into the lib add function

[deep fulcrum]

"select your existing factory to ... import it"
that one I will definitely do for the competition - already done with the selection part

[hasty vortex]

Wellll you'd think that but there's probably some parts that aren't quite so easy :p

Like what fuel a machine uses, if there's none in there atm.

[deep fulcrum]

my library works with imaginary fuels if necessary

[iron tapir]

do you cache the fuel categories -> items list?

[hasty vortex]

Yeah it's probably pretty doable for you. It'll be a nice feature. I'll probably never get to it, I have so many more good ideas than I'll ever have the time to implement.

[deep fulcrum]

most work for my mod went into supporting ghosts, because you have to do everything yourself

[hasty vortex]

Fixing bugs takes so much of the time once a system gets this big, so rare to work on proper big features nowadays.

[deep fulcrum]

like beacon coverage, module filtering, neighbour connection calculation, ...

and ghosts don't support everything, see this request for example
https://forums.factorio.com/viewtopic.php?t=129217

[hasty vortex]

Yeah I know from Rai that ghosts are very annoying

[deep fulcrum]

btw, kudos to @crude crag for adding most of the requested API access

really busy the last few days

[thick mountain]

i am most excited about the utility constants one 👀

[hasty vortex]

Caught him at a good time 😏

[iron tapir]

there's also this
https://forums.factorio.com/viewtopic.php?t=103378

[hasty vortex]

And yeah I need rocket lift weight too, just hardcoded it atm

[deep fulcrum]

the one thing that even rate calculator smuggles in from data stage

[hasty vortex]

What a cheater

[thick mountain]

oh whoops made a duplicate 😅
https://forums.factorio.com/viewtopic.php?t=129382

[iron tapir]

I don't blame you, it's an old thread

clearly it's a long time requested feature

[thick mountain]

a workaround would be creating a real entity and then calling copy paste onto the ghost, but thats so janky

[iron tapir]

a lot of these look like entity prototype property reads which is really easy to add, so that explains it

[deep fulcrum]

can't you build a blueprint string yourself and paste that?

yes, they are all easy reads, the not easy ones are still left 🙂

I have to go soon, was nice chatting with you guys
if you have feedback regarding this mod, or the library/API powering it all, please don't hold back

[hasty vortex]

Same same, thanks for the convo and very interesting library

Will definitely look into this more at some point

[iron tapir]

good stuff

[hasty vortex]

Kind of took over your mod‘s thread but I guess that‘s fine 😅

[spice trout]

Love that it solves based on constraints instead of showing a naive calculation.

Would be nice to have it pop-up in a corner instead of the center of the screen. And also, if I could tell it to display all numbers in /s or /m consistently.

[fallen cliff]

This does the hard part of the rate calculator redesign that I've been wanting to do for years!

[thick mountain]

oh?

[iron tapir]

he's been meaning to add a matrix solver into rate calculator for a while now

[thick mountain]

oh that part, redesign made me thing of the gui 😅

[fallen cliff]

Well, that too.

I want to do both at once.

I keep getting confused at my own GUI when playing pyanodons

[sweet galleon]

Production Rates Extreme doesn't support presence of multiple beacons at once if complex enough setup is present.

[wary ivy]

Sorry, there was a problem completing your request.
No value present

[sweet galleon]

Here is blueprint string

enable planets lite << is being used there along with infinite stacking

[lone oxide]

🌐 Enable Planet Mods Lite

A more streamlined, Lite version of the Enable All Planet Mods, removing several planet mods which either warped the early game too much, or simply did not add enough unique content for the mid to late game. A more playability-focused planet pack. 🌐

Author

Kryzeth

Downloads

1088

Factorio version

2.0

[deep fulcrum]

I've released a new version of the library, can you please confirm the bug being fixed

[sweet galleon]

Yep, it works now