Thread for LP nonsense! | Factorio | Page 1

dapper parrot Nov 3, 2024, 9:56 PM

#

Created

#

What solver setup are you using? I usually use sagemath

distant igloo Nov 3, 2024, 9:57 PM

#

I'm using the sage 'get_solver' API but all of the setup is handrolled Python code

dapper parrot Nov 3, 2024, 9:58 PM

#

Nice. So it doesn’t know about blue belts because they have a fluid ingredient?

#

Or just not coded in yet

distant igloo Nov 3, 2024, 10:02 PM

#

I've manually added recipes, because IME adding every recipe in the game makes it impossible to understand if the code is doing the right thing

#

I added a bunch of belts because I suspected they would be useful, but didn't bother to do blue belts

dapper parrot Nov 3, 2024, 10:02 PM

#

Ok that makes sense

#

Neat, so belts are one of the best options for an early game grind

distant igloo Nov 3, 2024, 10:04 PM

#

Yup

#

Another thing I keep seeing is buildings with one quality module in them. Apparently the first one is "more cost effective" than the later ones?

#

pretty sure this drastically increases the material consumption, though... let me try a hybrid cost metric

dapper parrot Nov 3, 2024, 10:06 PM

#

Hm yeah. I guess getting a little qual at multiple steps might stack better than a lot in one step? Though I would think that would kick in past 10% qual

distant igloo Nov 3, 2024, 10:08 PM

#

the weird thing is that it's still using it only in one step. it's just using four buildings with one module each instead of one with four modules

#

maybe this is literally just the speed penalty

dapper parrot Nov 3, 2024, 10:12 PM

#

Oh yeah probably that. If you’re trying to do it in one step instead of a loop

#

Or if it’s a loop but the main path is to get lucky in one step

distant igloo Nov 3, 2024, 10:16 PM

#

okay, if I mix in some ore cost, then I start to get normal looking setups. e.g. to make rare module 2s, you use quality just at the ending step and recycle the low-quality results.

dapper parrot Nov 3, 2024, 10:21 PM

#

ok that makes sense

#

any idea how asteroid crushing compares?

#

you only get two modules but there's a lower chance of losing stuff

distant igloo Nov 3, 2024, 10:28 PM

#

I could try it out. give me a few minutes, that's a lot of recipes to add...

dapper parrot Nov 3, 2024, 10:29 PM

#

could just add reprocessing and crush-to-iron

distant igloo Nov 3, 2024, 10:33 PM

#

the script is definitely interested in those recipes

#

it's using an unholy mix of stuff, but asteroid crushing is in there

dapper parrot Nov 3, 2024, 10:33 PM

#

yeah figures. LP

#

especially because asteroid crushing makes ore

#

you could make rare ore, but if logistics are free (because LP) it's probably better to make a mix and then have a chance to upgrade later

#

and when you get lucky and hit rare then you use prodded furnaces, but when you don't get lucky you use qual ones, and then ...

distant igloo Nov 3, 2024, 10:36 PM

#

hmm. I'm seeing that you should recycle the low-tier asteroids but reprocess the high-tier ones

#

recycling is more lossy but WAY faster, so that makes some sense

dapper parrot Nov 3, 2024, 10:36 PM

#

oh huh that makes sense

#

because asteroids are almost free

distant igloo Nov 3, 2024, 10:37 PM

#

I wouldn't trust the exact tradeoff the LP is making, but it seems plausible in general

#

recyclers are also furnaces, which is convenient if you ask me

#

yeah, I'm pretty consistently getting "recycle normal tier, reprocess higher tiers". though sometimes I see weird things like "reprocess normal non-metallic without quality, then recycle metallic with quality". presumably this is to avoid making high quality non-metallic asteroids? not sure I trust that

distant igloo Nov 3, 2024, 11:04 PM

#

maybe I should investigate beacons...

#

uh-oh, it's using beacons...

In [557]: quality.do_lp({"legendary/stone-brick": 1.0})
Out[557]: 
{'solver': <sage.numerical.backends.glpk_backend.GLPKBackend object at 0x7fa653538fa0>,
 'objective': -541.663205999954,
 'rates': {"stone/big-miner/('empty-slot', 'empty-slot', 'empty-slot')/normal": 4756.611544620971,
  "stone-brick/electric-furnace/('empty-slot', 'empty-slot')/normal": 19026.446178483882,
  "wall/assembler-3/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')/normal": 617.140648020885,
  "wall/assembler-3/('rare/quality-2', 'rare/quality-2', 'rare/quality-2', 'rare/quality-2')/uncommon": 17.787758798866943,
  "wall/assembler-3/('rare/quality-2', 'rare/quality-2', 'rare/quality-2', 'rare/quality-2')/rare": 2.8819057801495034,
  "wall/assembler-3/('rare/quality-2', 'rare/quality-2', 'rare/quality-2', 'rare/quality-2')/epic": 0.5397828317058248,
  "recycle-wall/recycler/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')/legendary": 0.032187231926702226,
  "recycle-wall/recycler/('rare/quality-2', 'rare/quality-2', 'rare/quality-2', 'rare/quality-2')/uncommon": 4.8471642726912405,
  "recycle-wall/recycler/('rare/quality-2', 'rare/quality-2', 'rare/quality-2', 'rare/quality-2')/rare": 1.4256786418499485,
  "recycle-wall/recycler/('rare/quality-2', 'rare/quality-2', 'rare/quality-2', 'rare/quality-2')/epic": 0.3148753614011402,
  "recycle-wall/recycler/('rare/quality-2', 'rare/quality-2', 'rare/quality-2', 'rare/quality-2', 'rare/speed-2', 'rare/speed-2', 'rare/speed-2')/normal": 86.09663058327078}}

#

⚠️ speed and quality ⚠️

dapper parrot Nov 3, 2024, 11:22 PM

#

hm neat even though it hurts quality

#

i found with asteroid flipping that in theory speed beacons help, but in practice you lose time to insert and remove stuff so it might not really help

#

i'm trying to test asteroid recycling. it's hard to feed it enough asteroids tho

distant igloo Nov 3, 2024, 11:28 PM

#

Yeah, the recipe is crazy fast

#

If you can't make full use of one recycler it might not be a win

dapper parrot Nov 3, 2024, 11:43 PM

#

so, with some fiddling: I think it is better in theory but my ship isn't fast enough

#

and i should wind down and head to bed instead of fiddling with it more tonight

#

but yeah if you have a ship that is collecting excess asteroids it's probably better to recycle some/all of them at normal, and reprocess at uncommon+

fathom iron Nov 4, 2024, 7:50 PM

#

cool thread!

fathom iron Nov 7, 2024, 4:31 PM

#

@dapper parrot @distant igloo check out my new script! https://github.com/scottmsul/FactorioQualityOptimizer

GitHub

GitHub - scottmsul/FactorioQualityOptimizer: Optimizes prod/quality...

Optimizes prod/quality module ratios for creating high-quality products in Factorio - scottmsul/FactorioQualityOptimizer

distant igloo Nov 7, 2024, 4:55 PM

#

Very cool!

#

The annoying thing about LP is that it has no chill. By which I mean it doesn't care about simplicity at all, and it spits out things that are very hard to understand. Actually making it balanced IRL can be really hard (for instance, what do you do if you end up skewed from the expected recycling probabilities?)

dapper parrot Nov 7, 2024, 5:07 PM

#

fathom iron <@287047189430140930> <@689143847435108401> check out my new script! https://git...

Neat, will check it out

upbeat hatch Nov 8, 2024, 12:26 AM

#

OK, so, according to my script, Kovarex would be as following with 0%/100% productivity

#

... time to see why

#

Ah, OK.

#

Multiplying the ingredients probably doesn't make sense

#

This seems more sensible

fathom iron Nov 8, 2024, 12:31 AM

#

I don't think they'd be ints

upbeat hatch Nov 8, 2024, 12:35 AM

#

At 100% productivity?

fathom iron Nov 8, 2024, 12:35 AM

#

oh at 100 yes

upbeat hatch Nov 8, 2024, 12:35 AM

#

It's 1.0, not 1%

#

For 0.5 you get 41.5, as expected

#

OK, this looks decent

#

I suppose I'll have to expand it in order to get the module slots and so forth properly done

#

Because for you, basically every combination of (machine, recipe) is an own "recipe", isn't it?

fathom iron Nov 8, 2024, 12:52 AM

#

yes, you could run it with all combinations, or you could pick which machine you'd use with each crafting group

upbeat hatch Nov 8, 2024, 12:54 AM

#

Yeah, but we're going for the full expansion for now

#

Filtering can come later

#

Hm, still, this is tricky

#

I suppose I'll just have to cheat a bit

#

can I even cheat?

#

Yeah, I don't think I can, given that the items can be differing

#

Since the output of an ingredient is depending on the productivity, and is not linear, I can't really set an expected value, can I?

#

If we're using Kovarex, the expected value is anything between 41 at 0% productivity, and I suppose 44 at 300%?

#

But that depends on the prod modules you throw in, which I can't control

#

I don't think we can really model that with the current data model of yours?

#

📎 recipes-ls.json

#

This would be the current state of the thing - I added more properties to the amount to show the different expectations at 0/50/100/250% prod

#

Kovarex is, as expected, weird

#

{
  "key": "centrifuge#kovarex-enrichment-process",
  "allow_productivity": true,
  "module_slots": 2,
  "additional_prod": 0,
  "ingredients": [
    {
      "name": "uranium-235",
      "amount": 40
    },
    {
      "name": "uranium-238",
      "amount": 5
    }
  ],
  "results": [
    {
      "name": "uranium-235",
      "amount": 41,
      "amount_50": 41.5,
      "amount_100": 42,
      "amount_250": 43.5
    },
    {
      "name": "uranium-238",
      "amount": 2,
      "amount_50": 2,
      "amount_100": 2,
      "amount_250": 2
    }
  ]
},

fathom iron Nov 8, 2024, 1:08 AM

#

yeah I think I'll just add all the amount_x fields to the python script

#

then you don't have to worry about computing any EVs

#

plus that's the only way it'll work for catalyst recipes

upbeat hatch Nov 8, 2024, 1:08 AM

#

I mean, we can't really compute the EV, since they're a function of productivity

fathom iron Nov 8, 2024, 1:09 AM

#

yeah

upbeat hatch Nov 8, 2024, 1:09 AM

#

You could calculate it yourself, theoretically

fathom iron Nov 8, 2024, 1:09 AM

#

yes that would be the plan, loop over all the prod/qual combinations, and calculate EV for each one

upbeat hatch Nov 8, 2024, 1:09 AM

#

public float GetExpected(float productivity)
{
    float baseAmount;
    if (Probability is > 0)
        baseAmount = Probability.Value * (0.5f * (AmountMin ?? Amount!.Value) + (AmountMax ?? Amount!.Value));
    else
        baseAmount = Amount!.Value;

    if (IgnoredByProductivity != null)
        baseAmount += Math.Max(0, (baseAmount - IgnoredByProductivity.Value) * productivity);
    else
        baseAmount *= (1 + productivity);

    if (ExtraCountFraction != null)
        baseAmount += ExtraCountFraction.Value;

    return baseAmount;
}

You'd need to get the ignored_by_productivity and extra_count_fraction values though

#

baseAmount I can supply using the EV above, that's not a problem

fathom iron Nov 8, 2024, 1:10 AM

#

I can update the python script to use those from the config file

upbeat hatch Nov 8, 2024, 1:13 AM

#

📎 recipes-ls.json

#

^ file with those two fields present; amount is the expected value based on the min/max/prob

#

^ and this ones does not write nulls

📎 recipes-ls.json

#

Hm, the additional_prod for the EMP seems wrong

#

📎 recipes-ls.json

upbeat hatch Nov 8, 2024, 2:10 AM

#

{
  "key": "big-mining-drill#mining-tungsten-ore",
  "allow_productivity": true,
  "module_slots": 4,
  "additional_prod": 0,
  "ingredients": [
    {
      "name": "patch-tungsten-ore",
      "amount": 1
    }
  ],
  "results": [
    {
      "name": "tungsten-ore",
      "amount": 1
    }
  ]
},

Do you think this is a good idea

upbeat hatch Nov 8, 2024, 2:40 AM

#

Huh, okay, so for some reason some items are... not there? It's "just" 257 items, which seems way too few

#

I'm not entirely sure what is missing, but rails would be definitely one of them

#

This looks better.

upbeat hatch Nov 8, 2024, 8:09 PM

#

@fathom iron: I see you've changed the format a bit around? What are the crafting machines for?

#

oh, it's a n:m mapping between recipes and machines, I see

#

Still doesn't solve the productivity issues though, does it?

fathom iron Nov 8, 2024, 8:10 PM

#

I haven't solved that part yet, no.

upbeat hatch Nov 8, 2024, 8:22 PM

#

Updated JSON in the new format with recipes, crafting machines, and the recipeVars I think

📎 recipes-ls.json

fathom iron Nov 8, 2024, 8:30 PM

#

I do plan to keep updating the config format, so apologies if it's not backwards-compatible

#

I'm trying to make it closer and closer to the factorio format

#

so eventually those fields could be passed in without changing anything

upbeat hatch Nov 8, 2024, 9:30 PM

#

It's not a big deal

#

I'm using a JSON serializer and have the data structures as C# types

#

I can very easily shift it around

steel lagoon Nov 9, 2024, 9:03 PM

#

what does it mean when the solves says the problem has no optimal solution...?

steel lagoon Nov 9, 2024, 10:00 PM

#

apparently my problem is "infeasible" by LP standards

#

📎 one_step_optimize_modules.json

fathom iron Nov 11, 2024, 10:23 PM

#

I made some more updates to the script if anyone's interested

shell meadow Nov 12, 2024, 5:56 AM

#

No way to enter productivity research yet, is there? Also, it's adding legendary assembling+recycling of landfill or stone furnace just to sink the stone byproduct from molten iron and copper

fathom iron Nov 12, 2024, 2:06 PM

#

shell meadow No way to enter productivity research yet, is there? Also, it's adding legendary...

yes, both good ideas. I might even be able to add a "allow-byproducts" option that makes anything voidable from the solver's perspective

distant igloo Nov 12, 2024, 2:20 PM

#

That's typically easy and convenient in a linear solver, yeah

fathom iron Nov 12, 2024, 3:36 PM

#

I added support for byproducts

fathom iron Nov 12, 2024, 4:31 PM

#

I think there's a bug, I can't make legendary holmium plates from scrap while allowing byproducts, it just says that the problem does not have an optimal solution

#

but it works fine for epic holmium plates

#

or if byproducts are turned off

#

even weirder, it works if the module cost is set to 0.91, but fails at 0.92

fathom iron Nov 12, 2024, 5:04 PM

#

adding a building cost seemed to help, I think it didn't like that recipes with zero modules could be added for free

distant igloo Nov 12, 2024, 5:30 PM

#

huh, no optimal recipe chain is... surprising. if you only include the module cost, do you get the same error?

#

you could also try a constraint of "cost >= 0.01" and see what it spits out. that gives a hint to what the unbounded solutions look like

#

is it possible that you're allowing recycling into scrap?

#

other possibility: are you rewarding the setup for making modules?

fathom iron Nov 12, 2024, 9:35 PM

#

I think infinities just don't mix well with zero-cost recipes. Even just setting the upper-bounds to a very large number like 999999 fixed it. I'm guessing there was probably some intermediate step where the solver wanted to set a couple variables to the max possible, even if it brought them back down again later.

upbeat hatch Nov 12, 2024, 9:47 PM

#

I'm wondering about the implementation - wouldn't it be possible to model this as a (rather large) graph too, where each item is a node, and the edges are recipes (with different chances), then simply run some pathfinding/filling algorithm on it?

fathom iron Nov 12, 2024, 10:54 PM

#

You're not really finding a "path" though, its a balancing act between all the possible recipes

#

You can represent the production chain as a graph though, and some tools print that out at the end

distant igloo Nov 12, 2024, 11:02 PM

#

Some recipes involve three items (eg two inputs, one output), so it's not really a graph. It's kind of a multigraph, but multigraphs are terrible

fathom iron Nov 12, 2024, 11:31 PM

#

If anyone else has written their own linear solver examples it might not be a bad idea to cross check results

distant igloo Nov 13, 2024, 3:29 AM

#

it's tricky because there are so many different assumptions you can put in, and the chains you get are so complicated

#

here's a test case that's maybe easy to confirm: what do you get as the most ore-efficient way to make legendary iron plates from normal nauvis ores? (ignore oil and water costs, don't allow scrap or asteroids, no productivity research, but all buildings available.)

I get the following (picture because it's too long to send). Basically, cycle batteries. Make batteries in cryo plants with 8 prod modules (except that normal tier ingredients get 7 prod 1 qual instead). I get 4.6136 ore consumed per plate, but that's not including the 50% mining drain modifier of big drills. (it is including the prod bonus in those drills, though.)

(see what I said about lots of assumptions?)

#

it also makes sulfur with a little bit of prod and recycles legendary batteries with a little bit of quality, but those recipes are both really underconstrained, because the modules have no impact on the output at all

#

if you have a test case on your side I can try and get my side to do it too

shell meadow Nov 13, 2024, 3:49 AM

#

Huh, with quality only coming from recycling but not the battery recipe? Fascinating

distant igloo Nov 13, 2024, 3:52 AM

#

yeah, the more slots you get, the better prod is

#

8 slots is craaaaaaaaaaazy

fathom iron Nov 13, 2024, 3:57 AM

#

I'm basically forcing the mining drills to use quality in my script, and also not accounting for resource drain, but I'm getting around 24 ores/plate. Though I am seeing the battery recipe in the outputs.

📎 message.txt

distant igloo Nov 13, 2024, 3:57 AM

#

I'm not sure quality in miners is ever correct tbh. It costs a lot of modules AND it doesn't save ore consumption

#

I see the same module mix on the battery cryo plants too!

fathom iron Nov 13, 2024, 3:58 AM

#

I guess the reason for me is I'm not sure how much one needs to worry about ore patches running out

shell meadow Nov 13, 2024, 4:00 AM

#

if you're at a point of mathing out quality grinders, you are probably not too worried about finite ores

distant igloo Nov 13, 2024, 4:00 AM

#

I suppose, but I'm not sure what other point there is in measuring ore cost

#

If you're not worried about that, you should measure buildings or modules or something

fathom iron Nov 13, 2024, 4:01 AM

#

yeah I added that as options to the script

distant igloo Nov 13, 2024, 4:01 AM

#

but measuring modules is haaaaaard. I don't want to input all the beacon arrangements that might matter...

fathom iron Nov 13, 2024, 4:01 AM

#

I don't worry at all about beacons

shell meadow Nov 13, 2024, 4:01 AM

#

oh!! I just realized that distinct recipe count would be a good regularization hyperparam

distant igloo Nov 13, 2024, 4:01 AM

#

becaons definitely matter a lot for saving prod modules

fathom iron Nov 13, 2024, 4:01 AM

#

usually it's just (num module slots) x (recipe var)

distant igloo Nov 13, 2024, 4:02 AM

#

and they might matter for saving quality modules sometimes

fathom iron Nov 13, 2024, 4:02 AM

#

yeah I've thought about adding beacons to the script

#

and maybe just not giving them a cost

distant igloo Nov 13, 2024, 4:03 AM

#

do you have an understandable test case that goes from module cost?

#

(also are you fixing one tier of module when you do module cost? otherwise you need to decide the relative cost of different tiers, which seems tricky)

fathom iron Nov 13, 2024, 4:04 AM

#

yeah I just always run the script with only one allowed prod module and one allowed quality module that you set in the beginning

distant igloo Nov 13, 2024, 4:05 AM

#

I get that with 11.89 modules you can make 1 legendary iron plate per second. (imagine if you could have fractional buildings in real life...) you use express underground belts, but the exact mix of what things you recycle is kind of complicated.

fathom iron Nov 13, 2024, 4:06 AM

#

distant igloo do you have an understandable test case that goes from module cost?

I did make a test case with just two items, four module slots, t3 legendary modules:

python .\scripts\linear_solver.py --config .\examples\one_step_optimize_modules.json
Solving...

Solution:
Objective value = 539.3072768635545

Inputs used:
input__normal__item-1: 122.7832364500082

Buildings used: 218.60347845336278

Modules used: 539.3072768635545

Recipes used:
normal__craft-1-to-2__assembling-machine-3__3-qual__1-prod: 101.3363483499971
uncommon__craft-1-to-2__assembling-machine-3__2-qual__2-prod: 4.916175331737391
rare__craft-1-to-2__assembling-machine-3__2-qual__2-prod: 2.63497636827681
epic__craft-1-to-2__assembling-machine-3__2-qual__2-prod: 0.8371174698170993
legendary__craft-1-to-2__assembling-machine-3__0-qual__4-prod: 0.14686537324283389
normal__recycle-2-to-1__recycler__0-qual__0-prod: 83.77665923747415
uncommon__recycle-2-to-1__recycler__4-qual__0-prod: 19.067612656339616
rare__recycle-2-to-1__recycler__4-qual__0-prod: 4.5604476009472865
epic__recycle-2-to-1__recycler__4-qual__0-prod: 1.3272760655305031

distant igloo Nov 13, 2024, 4:08 AM

#

If you allow recycling the ingredient into itself, you get a better result I think

#

{'solver': <sage.numerical.backends.glpk_backend.GLPKBackend object at 0x7ffa44ebd8e0>,
 'objective': -330.64804154944665,
 'rates': {"big-miner/mine-abstract/normal/('empty-slot', 'empty-slot', 'empty-slot')": 1219.281151007839,
  "assembler-3/abstract-recipe/legendary/('legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3')": 0.21055090519758612,
  "assembler-3/abstract-recipe/epic/('legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 1.0828332267304426,
  "assembler-3/abstract-recipe/rare/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 9.474790733891378,
  "recycler/recycle-abstract-two/rare/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 0.683278177924859,
  "recycler/recycle-abstract-two/epic/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 0.34481458049445207,
  "recycler/recycle-abstract-one/normal/('empty-slot', 'empty-slot', 'empty-slot', 'legendary/quality-3')": 261.9293557481932,
  "recycler/recycle-abstract-one/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 5.383403826074644}}

#

though I suppose that depends on the self-recycling speed...

#

Are you allowing buildings with empty slots?

#

I think they're actually correct sometimes

#

In [178]: quality.do_lp({"legendary/abstract-two": 1.0})
Out[178]: 
{'solver': <sage.numerical.backends.glpk_backend.GLPKBackend object at 0x7ffa44211220>,
 'objective': -300.8081003820032,
 'rates': {"assembler-3/abstract-one/normal/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')": 769637.5301016847,
  "assembler-3/abstract-two/normal/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')": 1005.2408556430165,
  "assembler-3/abstract-two/legendary/('legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3')": 0.587843251204259,
  "assembler-3/abstract-two/rare/('legendary/prod-3', 'legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3')": 6.193382191149627,
  "assembler-3/abstract-two/epic/('legendary/prod-3', 'legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3')": 1.94601208744113,
  "assembler-3/abstract-two/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 20.561744774516228,
  "recycler/recycle-abstract-two/normal/('empty-slot', 'empty-slot', 'empty-slot', 'legendary/quality-3')": 165.33566704654882,
  "recycler/recycle-abstract-two/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 2.409579465763625,
  "recycler/recycle-abstract-two/rare/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 1.6754692451061044,
  "recycler/recycle-abstract-two/epic/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 0.49407731868263294}}

#

[I added a suuuuper slow recipe that makes abstract-one so that recycling it would be super slow]

fathom iron Nov 13, 2024, 4:13 AM

#

I'm not checking all possibilities with empty slots, but I am looping the quality modules from 0 to slots, and if the recipe allows prod then it always fills the remaining with prod

#

my solver doesn't have any hard-coded knowledge about recycling, instead the recycling and crafting recipes are all lumped together as "recipes"

distant igloo Nov 13, 2024, 4:15 AM

#

what speed are your recipe and recyling recipe in this case?

fathom iron Nov 13, 2024, 4:17 AM

#

the recipes themselves have "energy_required" fields of 1 and 0.25 for crafting and recycling respectively (this is how factorio represents data, and is the number of seconds basically)

distant igloo Nov 13, 2024, 4:17 AM

#

I believe recyling recipes are usually sixteen times faster, not 4 times faster

fathom iron Nov 13, 2024, 4:17 AM

#

and the crafting machine/recycler have factorio stats too, so 1.25 and 0.5 respectively

distant igloo Nov 13, 2024, 4:19 AM

#

yeah, the pipe recipe has energy 0.5 and the recyling has energy 0.03125, so 16 times less

fathom iron Nov 13, 2024, 4:23 AM

#

hmm, iron gear wheels are 0.5 and 0.0625

distant igloo Nov 13, 2024, 4:24 AM

#

are you on the latest version?

fathom iron Nov 13, 2024, 4:24 AM

#

in the factorio data file it says pipe recycling is 0.0625

#

no

distant igloo Nov 13, 2024, 4:24 AM

#

they fixed a bug where default-speed recipes had the wrong recyling speed

fathom iron Nov 13, 2024, 4:24 AM

#

did they change it?

distant igloo Nov 13, 2024, 4:24 AM

#

basically the quality lua code interpreted "default speed" as 1, but the engine uses 0.5. so that caused a factor of 2 difference for any default-speed recipe

#

oh no, I have bad news. if I let my solver use speed modules, even without beacons, it likes them 😦

#

In [185]: quality.do_lp({"legendary/abstract-two": 1.0})
Out[185]: 
{'solver': <sage.numerical.backends.glpk_backend.GLPKBackend object at 0x7ffa44e86700>,
 'objective': -225.1630418617982,
 'rates': {"assembler-3/abstract-one/normal/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')": 359614.3449373728,
  "assembler-3/abstract-two/normal/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')": 454.84818331999674,
  "assembler-3/abstract-two/legendary/('legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3')": 0.5823369608363039,
  "assembler-3/abstract-two/epic/('legendary/prod-3', 'legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3')": 1.8787352305817517,
  "assembler-3/abstract-two/rare/('legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 5.529242886969374,
  "assembler-3/abstract-two/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/speed-3')": 9.394454732943137,
  "recycler/recycle-abstract-two/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 3.227050246399163,
  "recycler/recycle-abstract-two/rare/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 1.3312925829706916,
  "recycler/recycle-abstract-two/epic/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 0.5047770420112728,
  "recycler/recycle-abstract-two/normal/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/speed-3')": 33.842870782737855}}

fathom iron Nov 13, 2024, 4:26 AM

#

what's a good way to get the factorio data? I got it previously by downloading a file off kirkmcdonald's github, but he hasn't updated it yet https://github.com/KirkMcDonald/kirkmcdonald.github.io/blob/master/data/space-age-2.0.11.json

GitHub

kirkmcdonald.github.io/data/space-age-2.0.11.json at master · KirkM...

Simple web-based calculator for the game Factorio. - KirkMcDonald/kirkmcdonald.github.io

distant igloo Nov 13, 2024, 4:27 AM

#

there's a mod that has a command to dump a json file, but I'm not sure it's updated for 2.0

#

but if you fix the abstract recipe you're using, we can at least compare those results

fathom iron Nov 13, 2024, 4:29 AM

#

this is with the recycler energy set to 0.0625:

python .\scripts\linear_solver.py --config .\examples\one_step_optimize_modules.json
Solving...

Solution:
Objective value = 3442.3444906506215

Inputs used:
input__normal__item-1: 1105.448118428236

Buildings used: 1461.1766572200288

Modules used: 3442.3444906506215

Recipes used:
normal__craft-1-to-2__assembling-machine-3__3-qual__1-prod: 726.4750370756356
uncommon__craft-1-to-2__assembling-machine-3__4-qual__0-prod: 8.023681051538485
rare__craft-1-to-2__assembling-machine-3__3-qual__1-prod: 3.140666188774729
epic__craft-1-to-2__assembling-machine-3__4-qual__0-prod: 0.6846364967148661
legendary__craft-1-to-2__assembling-machine-3__0-qual__4-prod: 0.08058437133014072
normal__recycle-2-to-1__recycler__0-qual__0-prod: 600.5905345573731
uncommon__recycle-2-to-1__recycler__4-qual__0-prod: 106.69788632365011
rare__recycle-2-to-1__recycler__4-qual__0-prod: 13.575685484424904
epic__recycle-2-to-1__recycler__4-qual__0-prod: 1.9079456705866382

#

no wait that's completely wrong

#

disregard that

distant igloo Nov 13, 2024, 4:30 AM

#

I was gonna say, that number sounds pretty high

fathom iron Nov 13, 2024, 4:30 AM

#

I changed the output instead of the energy...

#

maybe this?

python .\scripts\linear_solver.py --config .\examples\one_step_optimize_modules.json
Solving...

Solution:
Objective value = 367.64283330973575

Inputs used:
input__normal__item-1: 84.15177514007344

Buildings used: 91.91070832743394

Modules used: 367.64283330973575

Recipes used:
normal__craft-1-to-2__assembling-machine-3__3-qual__1-prod: 64.03495415971946
uncommon__craft-1-to-2__assembling-machine-3__2-qual__2-prod: 7.590132010447865
rare__craft-1-to-2__assembling-machine-3__2-qual__2-prod: 2.7374027764924347
epic__craft-1-to-2__assembling-machine-3__2-qual__2-prod: 0.8410756914115006
legendary__craft-1-to-2__assembling-machine-3__0-qual__4-prod: 0.14705707906456278
normal__recycle-2-to-1__recycler__4-qual__0-prod: 11.028936929006369
uncommon__recycle-2-to-1__recycler__4-qual__0-prod: 4.086186881734061
rare__recycle-2-to-1__recycler__4-qual__0-prod: 1.1141224382986739
epic__recycle-2-to-1__recycler__4-qual__0-prod: 0.33084036125900995

distant igloo Nov 13, 2024, 4:32 AM

#

hm. using the same rules as you, I'm getting a worse result, and your result SHOULD be viable in my solver. odd.

#

wait, here's a really dumb question: are you trying to make item 1 here or item 2?

#

and you're making one per second, right?

#

given that you're not recyling legendaries, looks like you're making item 2

distant igloo Nov 13, 2024, 5:07 AM

#

.... This is a wild hunch, but I think you're turning speed penalties into speed bonuses. If I constrain my solver to use exactly your recipes it uses exactly 13/7 times as much of the normal forward recipe, and notably that assembler should be at a 30% speed penalty

fathom iron Nov 13, 2024, 12:24 PM

#

distant igloo .... This is a wild hunch, but I think you're turning speed *penalties* into sp...

turns out I was doing this by mistake. I defined the penalties as negative numbers and then subtracted them...

#

new result:

python .\scripts\linear_solver.py --config .\examples\one_step_optimize_modules.json
Solving...

Solution:
Objective value = 621.1603493504524

Inputs used:
input__normal__item-1: 96.31209734078664

Buildings used: 155.29008733761316

Modules used: 621.1603493504526

Recipes used:
normal__craft-1-to-2__assembling-machine-3__4-qual__0-prod: 112.1702833146833
uncommon__craft-1-to-2__assembling-machine-3__3-qual__1-prod: 13.303287769657612
rare__craft-1-to-2__assembling-machine-3__2-qual__2-prod: 6.121944220734775
epic__craft-1-to-2__assembling-machine-3__2-qual__2-prod: 1.952256431966584
legendary__craft-1-to-2__assembling-machine-3__0-qual__4-prod: 0.5873769995429726
normal__recycle-2-to-1__recycler__4-qual__0-prod: 13.180008289475289
uncommon__recycle-2-to-1__recycler__4-qual__0-prod: 5.76257666066572
rare__recycle-2-to-1__recycler__4-qual__0-prod: 1.71446337471775
epic__recycle-2-to-1__recycler__4-qual__0-prod: 0.49789027616910436

distant igloo Nov 13, 2024, 1:31 PM

#

I get the same result using your constraints 🙂

In [258]: quality.do_lp({"legendary/abstract-two": 1.0})
Out[258]: 
{'solver': <sage.numerical.backends.glpk_backend.GLPKBackend object at 0x7ffa3f191e80>,
 'objective': -621.1603493504472,
 'rates': {"assembler-3/abstract-one/normal/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')": 77049.67787262842,
  "assembler-3/abstract-two/legendary/('legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3')": 0.587376999542973,
  "assembler-3/abstract-two/rare/('legendary/prod-3', 'legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3')": 6.12194422073478,
  "assembler-3/abstract-two/epic/('legendary/prod-3', 'legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3')": 1.9522564319665852,
  "assembler-3/abstract-two/uncommon/('legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 13.303287769657604,
  "assembler-3/abstract-two/normal/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 112.17028331468207,
  "recycler/recycle-abstract-two/normal/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 13.180008289475186,
  "recycler/recycle-abstract-two/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 5.762576660665746,
  "recycler/recycle-abstract-two/rare/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 1.7144633747177505,
  "recycler/recycle-abstract-two/epic/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 0.4978902761691046}}

#

But you can actually do a lot better with empty slots:

In [260]: quality.do_lp({"legendary/abstract-two": 1.0})
Out[260]: 
{'solver': <sage.numerical.backends.glpk_backend.GLPKBackend object at 0x7ffa3f35bd00>,
 'objective': -305.870123189209,
 'rates': {"assembler-3/abstract-one/normal/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')": 785365.8969385906,
  "assembler-3/abstract-two/normal/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')": 1025.951530945252,
  "assembler-3/abstract-two/legendary/('legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3')": 0.5878336863294126,
  "assembler-3/abstract-two/rare/('legendary/prod-3', 'legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3')": 6.264279278493663,
  "assembler-3/abstract-two/epic/('legendary/prod-3', 'legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3')": 1.9577569050289174,
  "assembler-3/abstract-two/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 20.835697372607637,
  "recycler/recycle-abstract-two/normal/('empty-slot', 'empty-slot', 'empty-slot', 'legendary/quality-3')": 168.74202811599542,
  "recycler/recycle-abstract-two/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 2.4481944412813985,
  "recycler/recycle-abstract-two/rare/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 1.6912460753939889,
  "recycler/recycle-abstract-two/epic/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 0.4970160091683563}}

#

most of the savings are from skipping all modules in abstract-two/normal; the gain from dropping down to only one module in recycle-abstract-two/normal is relatively minor

#

if you allow speed modules, then you no longer have empty slots, which kinda makes sense to me:

In [266]: quality.do_lp({"legendary/abstract-two": 1.0})
Out[266]: 
{'solver': <sage.numerical.backends.glpk_backend.GLPKBackend object at 0x7ffa44df9040>,
 'objective': -229.19510821006196,
 'rates': {"assembler-3/abstract-one/normal/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')": 367354.53204708843,
  "assembler-3/abstract-two/normal/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')": 464.85862960719913,
  "assembler-3/abstract-two/legendary/('legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3')": 0.5822736182767487,
  "assembler-3/abstract-two/epic/('legendary/prod-3', 'legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3')": 1.890789772507216,
  "assembler-3/abstract-two/rare/('legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 5.590178851978394,
  "assembler-3/abstract-two/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/speed-3')": 9.520498654494972,
  "recycler/recycle-abstract-two/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 3.2762044100302177,
  "recycler/recycle-abstract-two/rare/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 1.343476059159743,
  "recycler/recycle-abstract-two/epic/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 0.5076600307706743,
  "recycler/recycle-abstract-two/normal/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/speed-3')": 34.58769565529752}}

#

I added some very rough beacon support, but I think it suggests that the standard "8-beacon" setup is good. (I didn't code in the fact that different buildings are different sizes; everything just assumes 3x3, which is explicitly wrong for recyclers...)

[WRONG]

#

the setups I encoded were 0, 1 (with 8 buildings around it), 4 (rows of buildings with beacons on one side), 8 (rows of buildings with beacons on both sides), and 12

#

oh crap wait no I forgot about the quality penalty

#

In [272]: quality.do_lp({"legendary/abstract-two": 1.0})
Out[272]: 
{'solver': <sage.numerical.backends.glpk_backend.GLPKBackend object at 0x7ffa32dbc700>,
 'objective': -115.74848976953588,
 'rates': {"assembler-3/bcn-0/abstract-one/normal/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')": 369605.3827047863,
  "assembler-3/bcn-0/abstract-two/normal/('empty-slot', 'empty-slot', 'empty-slot', 'empty-slot')": 465.9380809389043,
  "assembler-3/bcn-8/abstract-two/rare/('legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3')": 0.4598488632042923,
  "assembler-3/bcn-8/abstract-two/legendary/('legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3', 'legendary/prod-3')": 0.022607026031969703,
  "assembler-3/bcn-0/abstract-two/epic/('legendary/prod-3', 'legendary/prod-3', 'legendary/quality-3', 'legendary/quality-3')": 2.3700651518899787,
  "assembler-3/bcn-1/abstract-two/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 4.808189790193099,
  "recycler/bcn-1/recycle-abstract-two/normal/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 16.00062091136345,
  "recycler/bcn-1/recycle-abstract-two/uncommon/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 0.6213082744515136,
  "recycler/bcn-0/recycle-abstract-two/rare/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 2.6423899432342353,
  "recycler/bcn-0/recycle-abstract-two/epic/('legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3', 'legendary/quality-3')": 0.4314821013967977}}

So beacons with quality only make sense with particularly cheap inputs, but they ARE good when you want a ton of throughput of those items at low cost

#

beacons with prod are of course amazing

#

legendary beacons don't change the recipes used and they only make the setup marginally cheaper

fathom iron Nov 13, 2024, 2:01 PM

#

I'm guessing the best way to add speed modules to my script is probably with beacons only. First because it's probably optimal, since it doesn't take up any slots that could be used for prod/qual. Second because the combinatorics get pretty bad when you start checking every combination of module slots. Since I'm only checking prod/qual there's only O(N) variants of each recipe where N is the number of module slots. If I check speed beacons it's more like O(N^2), one linear term from module slots and another linear term from checking each possible of speed beacon bonus.

#

I should go check the math on beacons though, I remember they changed it a bunch and there's some diminishing effect formula.

#

Wonder if I should just have 2 modules / beacon always, or maybe check 1 module/1 beacon, 2 modules / 1 beacon, 3 modules / 2 beacons, etc.

hazy fractal Nov 13, 2024, 5:51 PM

#

can you do a two-phase solve if building/module costs are 0? find the optimal pipeline, then optimize the building & beacon counts?

hazy fractal Nov 13, 2024, 6:38 PM

#

calculator could definitely use an output total-- I think this means ~1.8 output?

epic__productivity-module__electromagnetic-plant__5-qual__0-prod__0-beaconed-speed: 4.768079627580483
legendary__productivity-module__electromagnetic-plant__0-qual__0-prod__0-beaconed-speed: 0.6697057839050451

upbeat hatch Nov 13, 2024, 7:27 PM

#

fathom iron what's a good way to get the factorio data? I got it previously by downloading a...

https://wiki.factorio.com/Command_line_parameters

--dump-data

Command line parameters

#

It's what I'm using so far, and the fact that it's vanilla sounds pretty promising to me, as it's unlikely to break

upbeat hatch Nov 13, 2024, 8:05 PM

#

Hm. Where/How does it actually define what quality does to modules?

#

Oh, it's just hardcoded based on the tier, OK

upbeat hatch Nov 13, 2024, 9:09 PM

#

So the recipe has a chance of giving sum(higher_or_equal_quality, quality => chance^(quality.tier - ingredients.tier) * expected_items) in total, except this has to be normalized because it can't give all of those

#

So I guess it's smarter to start with the highest quality, and then for each following tier, subtract the items from higher tiers

#

So if we expect it to give 50 items in total, and 0.1 legendary, 0.5 epic, 1 rare, 5 uncommon; then the it would in total give 0.1 legendary, 0.4 epic, 0.4 rare, 3.4 uncommon, and 45.7 common or something?

#

Hm, no.

0.1 legendary
0.5-0.1 = 0.4 epic
1-0.1-0.4 = 0.5 rare
5-0.1-0.4-0.5 = 4 uncommon
50-0.1-0.4-0.5-4 = 45 common

hazy fractal Nov 13, 2024, 9:57 PM

#

upbeat hatch Hm, no. 0.1 legendary 0.5-0.1 = 0.4 epic 1-0.1-0.4 = 0.5 rare 5-0.1-0.4-0.5 = 4...

no:

50*.9=45 common
50*.1*.9=4.5 uncommon
50*.1*.1*.9=.45 rare
50*.1*.1*.1*.9=.045 epic
50*.1*.1*.1*.1=.005 legendary

upbeat hatch Nov 13, 2024, 10:01 PM

#

Hm

#

I mean, I made up the numbers

#

But yeah, using q^(i-1)*(1-q) makes more sense than adding

distant igloo Nov 13, 2024, 11:42 PM

#

One thing you can do: only include the recipes that you suspect are useful in the LP, but then output the implied prices (from the dual model) and check that the recipes you skipped are negative value

hazy fractal Nov 13, 2024, 11:48 PM

#

testing summaries

#

fathom iron Nov 13, 2024, 11:50 PM

#

that's a MUCH better output!!! If you make a PR I'll happily merge

hazy fractal Nov 13, 2024, 11:50 PM

#

I'm not sure how to interpret the solution values so I left them out

fathom iron Nov 13, 2024, 11:50 PM

#

the solution values should just be a literal building count

hazy fractal Nov 13, 2024, 11:51 PM

#

# solved result is equivalent to number of buildings in units where craft_time=1 and craft_speed=1

fathom iron Nov 13, 2024, 11:51 PM

#

but it tries to solve 1 output/sec which I guess results in small numbers

hazy fractal Nov 13, 2024, 11:51 PM

#

does it have proper craft times yet?

fathom iron Nov 13, 2024, 11:51 PM

#

that comment is out-dated, I've started incorporating factors such as the recipe energy and building crafting speed

#

I think it should

hazy fractal Nov 13, 2024, 11:51 PM

#

so rounding up for building counts should work

fathom iron Nov 13, 2024, 11:52 PM

#

but again a lot of that stuff hasn't been rigorously cross-checked with other solvers, so there could easily be bugs

hazy fractal Nov 13, 2024, 11:52 PM

#

would still love to see outputs in items/m or whatever

upbeat hatch Nov 14, 2024, 12:08 AM

#

I'm currently trying to set it up using the Google library thingy for C# I found, but I'm still a bit at a loss as to what is what

#

I guess items are variables, recipes are constraints, ingredients are negatively bound variables on them, and products are positive bound ones?

#

(and you want to, uh... minimize constraints?)

fathom iron Nov 14, 2024, 12:20 AM

#

upbeat hatch I guess items are variables, recipes are constraints, ingredients are negatively...

have you ever taken a linear algebra course?

#

I'd recommend reading through this first, it's a good primer on how to setup factorio solvers in general http://kirkmcdonald.github.io/posts/calculation.html

upbeat hatch Nov 14, 2024, 12:23 AM

#

fathom iron have you ever taken a linear algebra course?

A decade ago, and I was kind of forced into it

#

it wasn't as bad as analysis because it seemed to deal with a finite-ish realm of things, rather than making up stuff as you go, but it was still rather bizarre

#

It's a world I definitely do not get along

fathom iron Nov 14, 2024, 12:26 AM

#

upbeat hatch I guess items are variables, recipes are constraints, ingredients are negatively...

the recipes are actually the variables, and each item has a constraint

upbeat hatch Nov 14, 2024, 12:26 AM

#

I suppose maximizing for the target item variable then would make sense?

fathom iron Nov 14, 2024, 12:28 AM

#

upbeat hatch I suppose maximizing for the target item variable then would make sense?

my solver has a constraint that says "make 1 legendary <user-specified-item> per sec", and a cost associated with each raw input (ores, oil, etc), and minimizes the cost. There can also be costs associated with number of buildings used, modules used, etc.

upbeat hatch Nov 14, 2024, 12:30 AM

#

Hm, let's see - if I can create variables (with a lower/upper bound), and constraints, and constraints can have coefficients with are other variables and a value, then.. uh.

#

Giving it a final constraint of the target makes sense. I'm just not entirely sure how I'll piece the rest together

upbeat hatch Nov 14, 2024, 1:11 AM

#

Okay, I've skimmed through the document and what you've said, but I'm not quite sure it makes sense

#

If the recipes are variables, and you're trying to minimize the cost... the result of recipes isn't unique, though, is it? With recyclers at least, two recipes can have the same result - making it really difficult/impossible to optimize for?

fathom iron Nov 14, 2024, 1:14 AM

#

yeah you could imagine a simple case where you can turn iron ore into iron plates using either furnaces or foundries. We know foundries should be better because of the prod bonus (ignoring calcite for now). Linear solvers can figure that out magically, and pick out foundries as "optimal" since they require less iron ore for the same iron plates.

distant igloo Nov 14, 2024, 1:14 AM

#

Usually "ties" aren't too much of an issue; the solver will pick one somehow

upbeat hatch Nov 14, 2024, 1:23 AM

#

Right, but I'm not quite sure how that would work

#

Like, what do you at the end optimize for?

#

x*foundryPlates + y*furnacePlates = 1?

#

minimize for x/y?

#

(I don't think I had this in linear algebra, but I do remember optimization like that being part of HS maths)

fathom iron Nov 14, 2024, 1:26 AM

#

you'd have variables like numberOfFoundryRecipes , numberOfFurnaceRecipes, ironInputPerSecond, and a constraint on iron where ironInputPerSecond = numberOfFoundryRecipes + numberOfFurnaceRecipes

upbeat hatch Nov 14, 2024, 1:28 AM

#

Right, I got that so far, minus the ironInputPerSecond

#

What are the relations between iron plates and iron ore? I assume those are constraints, too?

#

Is it a literal numberOfFoundryRecipes = ironOre / 1.5, numberOfFurnaceRecipes = ironOre?

#

(i.e. two constraints that link the variables together?)

#

This is where I struggle a bit if recipes are variables - because it means "ironOre" itself could not really exist, but rather, you would have to define (again) every possible producer of iron ore?

fathom iron Nov 14, 2024, 1:32 AM

#

usually most items don't have associated variables. The exception is items that are inputs or byproducts to the system. For these we can construct a fake "recipe" variable that produces the item out of nothing, or voids the item.

upbeat hatch Nov 14, 2024, 1:33 AM

#

Right, that makes sense.

fathom iron Nov 14, 2024, 1:33 AM

#

maybe take a look at the intro comment to this other script I made before I wrote the linear solver https://github.com/scottmsul/FactorioQualityOptimizer/blob/master/scripts/one_step_matrix_solver.py

GitHub

FactorioQualityOptimizer/scripts/one_step_matrix_solver.py at maste...

Optimizes prod/quality module ratios for creating high-quality products in Factorio - scottmsul/FactorioQualityOptimizer

#

this one is a matrix solver, which is a bit easier to grasp, since it can be solved with regular matrix algebra

#

linear solvers are similar, except they have too many columns and don't have a unique solution, but by having a cost function it's possible to still find an optimal solution. The behind-the-scenes math is way more complex, but you can import other solvers to do it for you and treat it like a black box.

upbeat hatch Nov 14, 2024, 1:38 AM

#

Yes, Gauss flashbacks right there

hazy fractal Nov 14, 2024, 1:41 AM

#

neither helmod nor factoryplanner have linear solvers, right?

upbeat hatch Nov 14, 2024, 1:41 AM

#

I mean, at this point, I'm almost tempted to say that each recipe is a constraint and a variable in the form of recipeNum * sum(ingredients->amount) = recipeNum * sum(results->expectedAmount)- with recipeNum being if/how much the recipe is utilized, and the ingredients/results being variables that are utilized in the whole process, too

fathom iron Nov 14, 2024, 1:41 AM

#

I think helmod actually is, factory planner is not (just a regular matrix solver) - side note I actually wrote the FP matrix solver

hazy fractal Nov 14, 2024, 1:43 AM

#

nah looks like a matrix too

fathom iron Nov 14, 2024, 1:43 AM

#

but I thought it did simplex

hazy fractal Nov 14, 2024, 1:44 AM

#

oh, no, that's correct

distant igloo Nov 14, 2024, 1:44 AM

#

Helmod routinely gives weird negative numbers IME

#

Maybe it's been updated since I used it, though

hazy fractal Nov 14, 2024, 1:45 AM

#

it's really hard to use linear solvers without having to learn an uncomfortable amount about their guts

distant igloo Nov 14, 2024, 1:45 AM

#

BTW, there's a good intro to LP on Kirk McDonald 's factorio calculator website

fathom iron Nov 14, 2024, 1:45 AM

#

upbeat hatch I mean, at this point, I'm almost tempted to say that each recipe is a constrain...

to be precise, in the code I wrote, there is a constraint for each item, such that the sum of all its uses in all recipe variables must equal zero

fathom iron Nov 14, 2024, 1:46 AM

#

distant igloo Helmod routinely gives weird negative numbers IME

that was one of my original motivations for working on the FP matrix solver. Since I knew there was always a unique solution I felt it was a bit easier to understand.

#

Though there was no shortage of people confused about selecting free items haha

distant igloo Nov 14, 2024, 1:47 AM

#

Yuuuup

hazy fractal Nov 14, 2024, 2:06 AM

#

wow, the impact of high tier quality modules is absurd

#

10x less iron required for legendary plates going from normal Q3s to legendary Q3s

upbeat hatch Nov 14, 2024, 2:07 AM

#

fathom iron to be precise, in the code I wrote, there is a constraint for each *item*, such ...

I've thrown together something now, and the result is promising so far - I didn't give it an input constraint (yet)

Using 60 variables and 10 constraints
Solve Result: OPTIMAL
iron-plate (legendary->legendary): 0.6666666666666666

#

It successfully found the best way to make legendary iron plates is by taking legendary iron ore and turn it into iron plates

#

... which is true, I guess.

#

I'm only running two recipes so far (iron-plate and iron-plate-recycling), but with all possible module combinations, and in all possible crafting places (for those recipes)

#

And yeah, for each item (... in each quality), there's a constraint that adds up the variables depending on whether they're ingredients (subtracted) or results (added)

hazy fractal Nov 14, 2024, 2:17 AM

#

fathom iron that's a MUCH better output!!! If you make a PR I'll happily merge

here ya go https://github.com/scottmsul/FactorioQualityOptimizer/pull/5

upbeat hatch Nov 14, 2024, 2:24 AM

#

I think I've set up the solver correctly - I just need to be able to tell it to not start with a simple recycler recipe from the get-go, but actually work its way up somehow

#

(Assuming I'm telling it to use normal ore, rather than legendary ore from the get-go, it will properly do normal->normal and normal->uncommon using a furnace; but for normal->legendary fall back to just recycling epic plates that it gets out of thin air I guess)

#

Aah, setting the bounds on inputs freely seems to... do the trick?

iron-plate @ electric-furnace (normal->normal) using [legendary-productivity-module-3+legendary-productivity-module-3]: 1820.4444444444441
iron-plate-recycling @ recycler (normal->legendary) using [legendary-quality-module-3+legendary-quality-module-3+legendary-quality-module-3+legendary-quality-module-3]: 2730.6666666666665

I'll need to counter-check that with your solver to be sure

upbeat hatch Nov 14, 2024, 4:22 AM

#

OK, I think part of the reason why it's doing what it is doing is because the recycler recipes are borked. amount: 0.5, extra_count_fraction: 0.5 has an expected value of 1, which means it can basically endless recycle the same thing to get components to make more using productivity

distant pier Nov 14, 2024, 5:15 AM

#

Is the optimal ratio of prod to quality universal, or situational for like every recipe?

hazy fractal Nov 14, 2024, 5:21 AM

#

situational, it's usually mostly quality though

wild marlin Nov 14, 2024, 5:50 AM

#

I remember that most reddit posts with math claimed that 2P2Q works best (assuming both legendary)

#

With base productivity 50% and more module slots it can change though, like 4P1Q

distant pier Nov 14, 2024, 5:53 AM

#

When I'll gethome I'll try to make a program which takes desired ammount of legendaries/m and putputs a table of different modules combos and other stats

#

I have most of the math in exel table already

wild marlin Nov 14, 2024, 5:56 AM

#

I think people above already kind of did those math

distant pier Nov 14, 2024, 5:57 AM

#

Nice do they have a script which does what i wanna do?

distant pier Nov 14, 2024, 5:58 AM

#

wild marlin I think people above already kind of did those math

I mean the math is a bunch of geometric progressions lol

distant pier Nov 14, 2024, 5:58 AM

#

distant pier Nice do they have a script which does what i wanna do?

I saw one for calculating best quality to prod ratio

wild marlin Nov 14, 2024, 6:58 AM

#

By the way, what does LP mean?

hazy fractal Nov 14, 2024, 7:09 AM

#

linear programming, the method of optimization that many factory planning tools use

fathom iron Nov 14, 2024, 2:27 PM

#

Thinking it might be interesting to play around with a fake module 3 recipe that only uses module 2/red chips/blue chips (no special planet ingredients) to get a sense of which planet makes the most sense for doing quality stuff.

wild marlin Nov 14, 2024, 2:57 PM

#

So I've been using scottmsul's program and was really surprised to see the actual best solution is quite far away from a normal expectation

#

I rembmer when it had fewer features (the legacy one) and it's really changed a lot

fathom iron Nov 14, 2024, 2:58 PM

#

I think it's getting to a point where it has most of the needed features

#

The only major one left is the infinite prod research bonuses

wild marlin Nov 14, 2024, 2:59 PM

#

maybe is there a way to add some 'cost' for having many different recipes?

#

the number of machines don't seem to represent that

fathom iron Nov 14, 2024, 2:59 PM

#

I've thought about it but I'm not sure an LP can do that, as it would be non-linear

#

I think a better strategy is to re-run for different simple subsets of recipes using the allowed recipes feature

wild marlin Nov 14, 2024, 3:00 PM

#

btw I'm the one who opened the second issue

#

if you remember

#

So for me, I just wanted to see how I can make a legendary quality module before I already have enough of them

#

And I was surprised when the solver said 'Hey please breed biters and fish'

#

I disallowed biochambers and it began to make more sense but the results were still a bit too complicated to set up

#

also when I run "python ./scripts/factorio_solver.py -oi quality-module-3 -q normal -ii scrap-resource=0.1 heavy-oil=0.1 -dc biochamber", your program outputs "... legendary 1Q4P" for copper cable but does this make sense?

#

I guess the solution doesn't need more than 4 prod modules for legendary copper cable

fathom iron Nov 14, 2024, 3:14 PM

#

I'm guessing maybe it's over-producing copper and by skipping out on a prod module maybe it has to recycle the legendary copper less

#

are you allowing byproducts? That might help

#

over-producing copper relative to iron or other inputs, that is

wild marlin Nov 14, 2024, 3:15 PM

#

I guess I am, isn't the program allowing byproducts by default?

wild marlin Nov 14, 2024, 3:16 PM

#

wild marlin also when I run "python ./scripts/factorio_solver.py -oi quality-module-3 -q nor...

This is the whole command line

fathom iron Nov 14, 2024, 3:16 PM

#

try adding the flag "--allow-byproducts"

#

I just realized the short version is "-av" I think it was meant to be "-ab"

wild marlin Nov 14, 2024, 3:18 PM

#

much simpler

#

yeah and I see how it's different

#

This is closer to what I'd have expected before

#

Still some surprise remaining though, like making beacon in the middle

fathom iron Nov 14, 2024, 3:19 PM

#

might be worth playing around with the cost functions for buildings/modules/beacons

#

I just set a bunch of things to 1 by default which might not be optimal

hazy fractal Nov 14, 2024, 5:17 PM

#

fathom iron I think a better strategy is to re-run for different simple subsets of recipes u...

could automate it, so it tries to run the solver repeatedly with some of the least used (or nonessential, somehow) recipes removed and tells you what the efficiency loss is after printing out the main solution

distant pier Nov 14, 2024, 11:20 PM

#

fathom iron Thinking it might be interesting to play around with a fake module 3 recipe that...

Dude it's vulcanus

#

I just copy paste huge factories making legendary modules only and they work for free basically

fathom iron Nov 14, 2024, 11:22 PM

#

distant pier Dude it's vulcanus

that wouldn't surprise me. I'm still not actually that far into space age yet, probably spent more time on the solver than actually playing 🤣. Finished vulcanus and fulgora, making my way through gleba currently.

distant pier Nov 14, 2024, 11:23 PM

#

Fulgora will kill you ups i am guessing if you attrmpt to expand to much there

distant pier Nov 14, 2024, 11:23 PM

#

fathom iron that wouldn't surprise me. I'm still not actually that far into space age yet, p...

Sick

#

I sat with exel calculator all day yesterday lol

#

Comparing results to ingame results

fathom iron Nov 14, 2024, 11:25 PM

#

someone found a calculation bug in my code today, was a pretty simple one line fix though. But I still feel like I'm not 100% sure it's perfectly accurate.

distant pier Nov 14, 2024, 11:26 PM

#

Justbuilt an ingame factory and compre results

fathom iron Nov 14, 2024, 11:26 PM

#

I think I'd rather compare with other solvers

distant pier Nov 14, 2024, 11:26 PM

#

You can increadd the game speed 60 timrs

distant pier Nov 14, 2024, 11:27 PM

#

fathom iron someone found a calculation bug in my code today, was a pretty simple one line f...

Do you have the ratio of legendaries to common input materials?

#

I can give you mine

fathom iron Nov 14, 2024, 11:28 PM

#

distant pier Do you have the ratio of legendaries to common input materials?

for an abstract one-stage factory?

distant pier Nov 14, 2024, 11:28 PM

#

Yeah

fathom iron Nov 14, 2024, 11:28 PM

#

yeah that was one of the first things I did, I still have the original code for that

#

but the latest script runs on every recipe in the game

#

and tries to get accurate building counts and throughputs from ores-to-modules

distant pier Nov 14, 2024, 11:29 PM

#

That's advanced..

fathom iron Nov 14, 2024, 11:29 PM

#

and a lot of the results it gives are extremely complex and hard to interpret...

sullen bluff Nov 14, 2024, 11:33 PM

#

in mine you input the prod and quality bonuses for machines and recyclers, some other stuff and input resources/m and it gives number of machines, recyclers, modules that you need

#

and legendaries/m, ration of legendaries to common input

#

very simple but effective, enough to build any factory basically

#

numbers corelate to the game btw

#

also this

fathom iron Nov 14, 2024, 11:36 PM

#

the tricky part though is when there's multiple stages of production, it's usually more optimal to "spread out" the quality bumps, rather than going straight from normal to legendary in one shot

sullen bluff Nov 14, 2024, 11:36 PM

#

true

#

but on vulcanus everything is free

#

so not always

#

like for cicruits it doesn't matter does it

#

only constraint is plastic but you have so much coal that again not really

#

btw if you have some numbers you would want to comare i'll gladly do so

fathom iron Nov 14, 2024, 11:41 PM

#

probably the main number if you want to check for correctness is 79.9 normal inputs per legendary output, assuming four module slots. 2 qual / 2 prod at every quality step, except the last which is legendary. This is a number that's been verified several times now by multiple people.

sullen bluff Nov 14, 2024, 11:42 PM

#

give me a sec, ok

sullen bluff Nov 14, 2024, 11:42 PM

#

fathom iron probably the main number if you want to check for correctness is 79.9 normal inp...

are moduleds legendary?

fathom iron Nov 14, 2024, 11:42 PM

#

yeah, that's all t3 legendary modules

#

a bunch of us calculated it when they first announced quality in the FFF

sullen bluff Nov 14, 2024, 11:47 PM

#

fathom iron probably the main number if you want to check for correctness is 79.9 normal inp...

what;'s on the last step, 4 prod?

fathom iron Nov 14, 2024, 11:48 PM

#

yes

#

but the 2/2 split everywhere else was calculated to be optimal, at least in terms of inputs / output

sullen bluff Nov 14, 2024, 11:56 PM

#

fathom iron probably the main number if you want to check for correctness is 79.9 normal inp...

same modules in recyclers right?

#

i am not sure how productivity in recycklers works

#

by default they spit 0,25 of input

#

if they have 4 leg prod modules will they spit 0,5?

sullen bluff Nov 15, 2024, 12:06 AM

#

fathom iron probably the main number if you want to check for correctness is 79.9 normal inp...

that's withour recycling all the common - epic products it seems?

#

you typed 79,9, not 76,6?

#

i got 76,6

#

btw when you use prod the ammount of crafters you need grows very fast

#

for 2 qual 2 prod you need like 2 more times crafters to consume the same input than 4 qual

#

which matters if you wenna do circuits on vulcanus let's say

sullen bluff Nov 15, 2024, 12:54 AM

#

lmao factorio makes you have fun staring at exel

#

i got some insane results for a legendary green circuit factory gotta build it rn

fathom iron Nov 15, 2024, 2:30 AM

#

It's 79.9

main tartan Nov 15, 2024, 4:37 AM

#

I guess 2/2 is optimal, but using 4 prod and being able to apply speed beacons is considerably more pleasant XD

upbeat hatch Nov 15, 2024, 5:26 AM

#

fathom iron I think I'd rather compare with other solvers

I'd like to help, but I feel like my results are all weird and wrong. It's almost always building the thing with prod3, then just recycles it with quality3 to immediately go from normal to legendary, and that doesn't feel right

distant pier Nov 15, 2024, 7:43 AM

#

main tartan I guess 2/2 is optimal, but using 4 prod and being able to apply speed beacons i...

You know that speed kills quality right?

fathom iron Nov 15, 2024, 12:50 PM

#

distant pier You know that speed kills quality right?

a touch of speed modules can help with throughput, especially at lower qualities, if you're short buildings/modules and you're ok with wasting some amount of input

distant igloo Nov 15, 2024, 12:50 PM

#

But also that person was talking about speed+prod, not speed+qual

#

Speed+prod is still fine

sullen bluff Nov 15, 2024, 9:21 PM

#

btw the more i tried to design a legendary circuit factory recycling everything else the more i think it's better to make a lot of common ingredients and grind only at final stage

fathom iron Nov 15, 2024, 10:02 PM

#

sullen bluff btw the more i tried to design a legendary circuit factory recycling everything ...

care to post your spreadsheet? I can have a look at it to see why you're getting 76.6

hazy fractal Nov 15, 2024, 11:12 PM

#

sullen bluff btw the more i tried to design a legendary circuit factory recycling everything ...

this is certainly simpler but it's more wasteful of inputs. for circuits the best design is probably legendary LDS @ 300% prod + a legendary iron plate setup using belts

main tartan Nov 16, 2024, 4:18 AM

#

@fathom iron I asked your solver how to make prod 3 modules and it's doing something with underground belts?!

#

lol

#

python3 ./scripts/factorio_solver.py --input-items electronic-circuit=1 advanced-circuit=1 biter-egg=1 --output-item productivity-module-3 --output-amount 1

fathom iron Nov 16, 2024, 4:22 AM

#

main tartan <@510531329910374400> I asked your solver how to make prod 3 modules and it's d...

yes, that is optimal

main tartan Nov 16, 2024, 4:25 AM

#

is it supposed to do stuff like "Electronic circuit in Electromagnetic plant: normal 4Q1P" even though I specified that as an input item

#

I guess I don't understand how iron gear wheels, copper cables etc are even getting involved when I specified red & green circuits as fixed inputs

hazy fractal Nov 16, 2024, 5:14 AM

#

it has to recycle them sometimes

distant igloo Nov 16, 2024, 5:25 AM

#

You have normal green circuits as an input, but not higher tier ones, and it wants to make them by crafting them apparently

hazy fractal Nov 16, 2024, 5:26 AM

#

yeah, it also needs to make processing units

distant igloo Nov 16, 2024, 5:27 AM

#

main tartan <@510531329910374400> I asked your solver how to make prod 3 modules and it's d...

Very weird things are sometimes optimal. Take LP output with a big grain of salt! It's hard to encode "what seems reasonable" in an LP

fathom iron Nov 16, 2024, 1:57 PM

#

main tartan I guess I don't understand how iron gear wheels, copper cables etc are even gett...

To elaborate, my guess is it has to do with the difference between upping one quality step vs jumping multiple quality steps in a single production step (i.e. 24.8% vs 10% for a fully-loaded assembler 3). This asymmetry probably favors adding extra stages of production, so for instance making underground belts just to recycle them is better for making legendary, than just recycling iron plates.

main tartan Nov 16, 2024, 4:08 PM

#

Maybe like a “recipe weight” feature would be cool to find sequences that don’t use so many steps

distant igloo Nov 16, 2024, 4:24 PM

#

Difficult to do, algorithmically.

shell meadow Nov 16, 2024, 8:29 PM

#

Is there a way to specify liquid input resources? Getting KeyError: 'normal__lava-resource'

fathom iron Nov 16, 2024, 8:31 PM

#

shell meadow Is there a way to specify liquid input resources? Getting `KeyError: 'normal__la...

you have to specify it as an input item, not an input resource

#

the resource logic is mainly needed to get accurate quality counts when quality modules are placed in miners

fathom iron Nov 17, 2024, 3:54 PM

#

I added productivity research to the script, I know a few users were interested in having that

sullen bluff Nov 18, 2024, 1:45 AM

#

hazy fractal this is certainly simpler but it's more wasteful of inputs. for circuits the bes...

fair but sometimes building another mine is overall a better option than a whole legendary item factory, they are very convoluted

upbeat hatch Nov 18, 2024, 4:17 AM

#

I've realized why my script was doing really weird things - I've had the wrong constraints (a common recipe never generated uncommon results, but rather had a separate constraint, which obviously could be set to zero for "nicer" results). Now I have more convoluted and a bit weird things - like making epic electronic circuits apparently involves... solar panels?

#

I feel like part of it is that it's now assuming that the recipes can be happening simultaneously - i.e. having one furnace with quality, and another with productivity, seems to be OK. Which is... questionable

upbeat hatch Nov 18, 2024, 9:38 AM

#

Can I solve this problem somehow with another constraint, or do I have to do some additional refining step (i.e. take all possible module combinations into the solver; then after each round, eliminate the results for each machine that have the least usage, then re-try again)? I don't think I can realistically say "One variable is > 0, every other variable is 0"?

fathom iron Nov 18, 2024, 1:18 PM

#

upbeat hatch Can I solve this problem somehow with another constraint, or do I have to do som...

there's not really any easy way to encode "simplicity" as a cost function into the solver

distant igloo Nov 18, 2024, 3:08 PM

#

You can probably do that with a MILP solver, but those are a looooot slower than IP solvers

upbeat hatch Nov 18, 2024, 5:49 PM

#

fathom iron there's not really any easy way to encode "simplicity" as a cost function into t...

Yeah, it's why I'm thinking about an iterative approach - group recipes by crafter/thing, then attempt to solve it, and disable the least used recipe

#

(by setting the upper bound to zero, as a hack)

#

It's probably not perfect because it's based on the assumption that "If something is not used often, it's probably a byproduct"

fathom iron Nov 19, 2024, 3:29 PM

#

Now that I'm actually playing through gleba I'm realizing that the solver probably shouldn't be trusted with anything bio-related since it doesn't handle fuel consumption.

#

which is also tricky since "prod" modules are basically anti-prod when it comes to nutrient consumption

#

any optimal solve of quality bioproducts almost certainly uses efficiency modules...

distant igloo Nov 19, 2024, 3:55 PM

#

hm, the fuel doesn't have to be quality, so it shouldn't affect it too much at higher tiers of quality? But yeah, nutrient fuel is important on gleba

fathom iron Nov 19, 2024, 4:02 PM

#

that's a good point, maybe it doesn't actually matter that much

hazy fractal Nov 19, 2024, 8:53 PM

#

fuel production is relatively minimal on gleba though

#

80~90% of the nutrient consumption is in the pentapod egg recipe

distant igloo Nov 19, 2024, 9:32 PM

#

Yeah, quality pentapod eggs are a whole different conversation

#

and quality bioflux for quality ores seems... annoying at best. you'd need a huge setup to avoid the bacteria spoiling before you get your next legendary bioflux

hazy fractal Nov 19, 2024, 10:26 PM

#

yeah, I don't know why you'd do that on gleba instead of vulcanus

shell meadow Nov 20, 2024, 1:29 AM

#

Is the speed beacon option trustworthy? it's slapping speed beacons on machines with quality modules

#

also the script is insisting that I upcycle uranium ore to epic no matter what cost settings I choose, very interesting

fathom iron Nov 20, 2024, 1:38 AM

#

shell meadow Is the speed beacon option trustworthy? it's slapping speed beacons on machines ...

yes but only for the default cost settings, if you ever set the building and module costs to zero (so only inputs have costs) then it should never use speed modules.

shell meadow Nov 20, 2024, 1:38 AM

#

yep, I did see that and it makes sense

distant igloo Nov 20, 2024, 3:52 AM

#

High quality speed modules with quality modules can make sense. Low quality speed modules are terrible, though

fathom iron Nov 21, 2024, 12:08 AM

#

hey everyone I merged a change from @wild marlin that makes cool graphical outputs from the script

wild marlin Nov 21, 2024, 12:08 AM

#

Wait did I make such a PR?

#

No it's zzh8829 not me

fathom iron Nov 21, 2024, 12:10 AM

#

oh shoot nvm

#

I assumed that was you haha with the zz

wild marlin Nov 21, 2024, 12:10 AM

#

My github name is dr-zzt

#

(FYI not a doctor yet)

fathom iron Nov 21, 2024, 12:10 AM

#

is he in discord?

wild marlin Nov 21, 2024, 12:10 AM

#

I don't know

patent pebble Nov 21, 2024, 12:28 AM

#

fathom iron which is also tricky since "prod" modules are basically anti-prod when it comes ...

Sorry for jumping in but what's quality got to do with fuel consumption? I know it slows the machine down but you can stockpile and bank huge amount of spoilage to basically run Biochambers on quality duty on demand.

fathom iron Nov 21, 2024, 12:30 AM

#

patent pebble Sorry for jumping in but what's quality got to do with fuel consumption? I know ...

yeah for quality it doesn't matter as much since quality recipes can run on normal nutrients

versed moth Nov 22, 2024, 7:44 PM

#

fathom iron yeah for quality it doesn't matter as much since quality recipes can run on norm...

what was best outcome for best module mix for superconductor? (only one stage) that way i can use it for processors as well

fathom iron Nov 22, 2024, 9:07 PM

#

versed moth what was best outcome for best module mix for superconductor? (only one stage) t...

this is what I get from the one_step_matrix_solver script:

python ./scripts/one_step_matrix_solver.py --module-slots 5 --additional-prod 50

optimizing recycling loop that turns ingredient quality 1 into product quality 5

q1 input per q5 output: 13.233909931435692
recipe q1 uses 1 quality modules and 4 prod modules
recipe q2 uses 1 quality modules and 4 prod modules
recipe q3 uses 1 quality modules and 4 prod modules
recipe q4 uses 1 quality modules and 4 prod modules
recipe q5 uses 0 quality modules and 5 prod modules

#

btw I just made some updates to the linear solver, there was a bug I didn't realize apparently speed penalties are capped at -80%, it now allows plant inputs (jellynut/yamako w/ default cost of 1), and --verbose prints input/output amounts for each recipe

versed moth Nov 22, 2024, 9:08 PM

#

i think i found the answer somewhere u put as 4 prod/1 qual. but then what was difference between that and 5 prod or 5 qual

fathom iron Nov 22, 2024, 9:11 PM

#

versed moth i think i found the answer somewhere u put as 4 prod/1 qual. but then what was d...

I get 30.1 with only qual, only prod doesn't really make sense

python ./scripts/one_step_matrix_solver.py --module-slots 5 --additional-prod 50 --disable-prod

optimizing recycling loop that turns ingredient quality 1 into product quality 5

q1 input per q5 output: 30.09995129512883
recipe q1 uses 5 quality modules and 0 prod modules
recipe q2 uses 5 quality modules and 0 prod modules
recipe q3 uses 5 quality modules and 0 prod modules
recipe q4 uses 5 quality modules and 0 prod modules
recipe q5 uses 0 quality modules and 0 prod modules

#

but that's just setting it to a recipe that doesn't allow prod, so it uses 0 prod on the last step

versed moth Nov 22, 2024, 9:12 PM

#

hmmm

#

i need more specific. assume super conductor recipe. so u only get 1 step. then recycle step

fathom iron Nov 22, 2024, 9:21 PM

#

versed moth i need more specific. assume super conductor recipe. so u only get 1 step. then ...

interesting, didn't realize the superconductor recycles into itself. Getting 191 of each input per legendary output.

python ./scripts/factorio_solver.py --output-item superconductor --input-items copper-plate=1 plastic-bar=1 holmium-plate=1 light-oil=0.2 --module-cost 0 --building-cost 0 --allowed-recipes superconductor superconductor-recycling
Solving...

Solution:
Objective value = 765.4482825957372

Inputs used:
input__normal__copper-plate: 191.3620706489343
input__normal__plastic-bar: 191.3620706489343
input__normal__holmium-plate: 191.3620706489343
input__normal__light-oil: 956.8103532446714

Buildings used: 493.9353491764136

Modules used: 2230.8908242375664

Machine layout:
Superconductor in Electromagnetic plant: normal 5Q
Superconductor (recycling) in Recycler: normal 4Q, uncommon 4Q, rare 4Q, epic 4Q

#

if you start to allow other recipes it does some weird clever tricks like making copper cables and recycling them, but this is also highly dependent on the cost function

python ./scripts/factorio_solver.py --output-item superconductor --input-items copper-plate=1 plastic-bar=1 holmium-plate=1 light-oil=0.2 --module-cost 0 --building-cost 0
Solving...

Solution:
Objective value = 503.0409394864801

Inputs used:
input__normal__copper-plate: 47.45947334153138
input__normal__plastic-bar: 219.41408689780448
input__normal__holmium-plate: 219.41408689780448
input__normal__light-oil: 83.7664617466988

Buildings used: 149.78230854775015

Modules used: 660.8155444397239

Machine layout:
Copper cable in Electromagnetic plant: normal 5P
Copper cable (recycling) in Recycler: normal 4Q
Holmium plate (recycling) in Recycler: normal 4Q
Plastic bar (recycling) in Recycler: normal 4Q
Superconductor in Electromagnetic plant: uncommon 5Q, rare 5Q, epic 5Q, legendary 5P
Superconductor (recycling) in Recycler: uncommon 4Q, rare 4Q, epic 4Q

versed moth Nov 22, 2024, 9:25 PM

#

blah so many variables now

#

wait, it doesnt use mix modules?

#

i thought in here it would use 4P +1Q

distant igloo Nov 22, 2024, 9:29 PM

#

Crafting superconductors isn't involved in a loop, which changes the math

fathom iron Nov 22, 2024, 9:29 PM

#

versed moth i thought in here it would use 4P +1Q

it would be if recycling gave back the original materials

versed moth Nov 22, 2024, 9:29 PM

#

okay

fathom iron Nov 22, 2024, 9:30 PM

#

wonder why they did that for superconductors

distant igloo Nov 22, 2024, 9:30 PM

#

OTOH this chain would be a pain in the ass to actually build, because you'll end up with an imbalance between the types

fathom iron Nov 22, 2024, 9:30 PM

#

it's not really a plate or bar

distant igloo Nov 22, 2024, 9:30 PM

#

How much worse is it if you craft the inputs to superconductors with prod instead of quality?

versed moth Nov 22, 2024, 9:30 PM

#

so if it would change to supercapacitor (it doesnt follow rule of super conductor) then it would 4P +1Q then recycle? how much is that different vs 5P or 5Q?

distant igloo Nov 22, 2024, 9:31 PM

#

That recipe chain is much less annoying to build in practice

fathom iron Nov 22, 2024, 9:31 PM

#

yeah just using --allowed-recipes with only the one craft/recycle is probably better

versed moth Nov 22, 2024, 9:34 PM

#

actually this is nice, if it recycles into itself, use full Quality, right?

#

which would apply to most first stage items, like holmium plate

fathom iron Nov 22, 2024, 9:36 PM

#

versed moth so if it would change to supercapacitor (it doesnt follow rule of super conducto...

it's 1Q 4P for supercapacitors, but it's pretty different results depending on if you're optimizing for inputs versus number of modules used

versed moth Nov 22, 2024, 9:37 PM

#

yea i figured

hazy fractal Nov 22, 2024, 10:55 PM

#

restricted recipe chains might work nicely. let users specify a tolerable waste %. start with a full solve, then keep iteratively removing recipes as long as the required inputs don't increase more than the specified waste %

shell meadow Nov 23, 2024, 5:10 PM

#

Yeah I’d love support for incrementally removing random recipes for armor and stuff. I suppose I can write a little script

shell meadow Nov 25, 2024, 12:43 AM

#

Not sure if anything changed, but I'm seeing great regularization out of speed beacon setting with building cost setting. Mixed prod/quality modules don't actually make sense because beaconed prod machines have enormous throughput compared to quality modules

versed moth Nov 25, 2024, 1:03 AM

#

better yield

shell meadow Nov 25, 2024, 1:34 AM

#

Sure, but they're less convenient to scale.

versed moth Nov 25, 2024, 1:36 AM

#

similar to full quality basically, just copy paste

#

full quality builds gets large, but it isnt even close to like 2.5k spm vanilla

fathom iron Nov 25, 2024, 1:47 AM

#

I did fix the 20% minimum speed issue, which was causing a lot of weird results previously

versed moth Nov 27, 2024, 9:20 PM

#

Do you know rate for asteroid chunks and/or crafting speed to get 1 to legendary?
From my P graph it seems around 40-50 per legendary.

I'm trying to compare it against mining rate then recycling

meager kiln Nov 29, 2024, 9:00 AM

#

did you do all 3 asteroids or substitute 80%?
cause im getting 90 on average with 80%
i think separate shouldnt matter but not 100% sure

versed moth Nov 29, 2024, 9:04 AM

#

only iron, but i break q2/q3/q4 as well

meager kiln Nov 29, 2024, 9:05 AM

#

oh nvm made a mistake
im getting 48 now

fathom iron Dec 5, 2024, 2:45 PM

#

I haven't checked, but I'm starting to wonder if quality science ever wins when optimizing for modules or number of buildings instead of inputs

distant igloo Dec 5, 2024, 3:09 PM

#

Well science requires zero modules normally, so module count isn't going to favor it

fathom iron Dec 5, 2024, 3:09 PM

#

no, if you use buildings with modules in them to make the science, and "minimize modules" just means making a more compact setup

distant igloo Dec 5, 2024, 3:09 PM

#

Building count might, but speed+prod is better than speed+qual, so I suspect that will also lean prod. (Thought recipes that don't take prod might take quality?)

fathom iron Dec 5, 2024, 3:10 PM

#

oh you mean like if I allowed the solver to check buildings with no modules in them

distant igloo Dec 5, 2024, 3:10 PM

#

Yeah

#

Buildings with no modules are pretty good when modules are your cost function 🙂

fathom iron Dec 5, 2024, 3:30 PM

#

this is making me re-think the whole cost function. Maybe instead of the number of inputs used per output, or the cost of the buildings used in the setup, it should just be the number of buildings? Maybe what we really want is the most compact setup possible?

sullen bluff Dec 5, 2024, 4:59 PM

#

fathom iron I haven't checked, but I'm starting to wonder if quality science ever wins when ...

would it even matter, the "goal" usually is to have higher spm and slapping some more machines is always better than to use quality science i would guess

fathom iron Dec 5, 2024, 5:20 PM

#

sullen bluff would it even matter, the "goal" usually is to have higher spm and slapping some...

To use an extreme example, would you rather build 6x the buildings and use 6x the input resources, or 6x output from converting to a quality build that used 1/10 the buildings but used 6.01 as many input resources?

sullen bluff Dec 5, 2024, 5:22 PM

#

that's an exellent question actually, i don't know, i think i might have went the first rout

#

to utilize every input i can

#

depends on science too

fathom iron Dec 5, 2024, 5:22 PM

#

more generally, I think extra input belts should be thought more of as "buildings" in some sense than as the "thing that must always be optimized", especially if you're not worried about ore patches running out

sullen bluff Dec 5, 2024, 5:24 PM

#

about 1\10th buiding btw, if we are talking about quality modules and prod modules, the difference between them is 10 percent speed (-5 and -15)

#

so for most sciences the ammount of machines wont be more than halfed with any module configuration

fathom iron Dec 5, 2024, 5:24 PM

#

yeah it's a very unrealistic example, more just to demonstrate that it's not obvious how to think about it

sullen bluff Dec 5, 2024, 5:24 PM

#

exept cryogenic plants

sullen bluff Dec 5, 2024, 5:26 PM

#

fathom iron yeah it's a very unrealistic example, more just to demonstrate that it's not obv...

yeah i guess you have a point

fathom iron Dec 5, 2024, 5:28 PM

#

and it's a little annoying arguing w/ romayne, who keeps insisting that "speed always bad" even though the linear solver keeps wanting them even in the quality steps, shrug...

sullen bluff Dec 5, 2024, 5:29 PM

#

well, He is just a perfectionist haha, not a drop of resources should be wasted

fathom iron Dec 5, 2024, 5:29 PM

#

oh no god forbid I waste lava

sullen bluff Dec 5, 2024, 5:30 PM

#

yeah, gotta change your way of thinking sometimes

fathom iron Dec 5, 2024, 5:30 PM

#

I mean even after all this LP discussion even I don't know the best answer lol, just that it's way trickier/more subtle than people realize

sullen bluff Dec 5, 2024, 5:30 PM

#

you know Rimworld, that's a game that changed how i play factorio, on high difficulties it doesn't allow you to be perfectionist at all

fathom iron Dec 5, 2024, 5:31 PM

#

interesting, never played it

sullen bluff Dec 5, 2024, 5:31 PM

#

it forces you to use everything only to survive

#

it's a colony simulator with 137 hours AVERAGE time played

#

very addictive highly recomment

#

d

#

also i am banned from the official surver because i kept accidentally sending my colony name -_-

distant igloo Dec 5, 2024, 6:50 PM

#

Another thing that I don't know how to model: some of these recipes are probabilistic, and you might get an excess of one product. What do you do with it?

sullen bluff Dec 5, 2024, 7:32 PM

#

distant igloo Another thing that I don't know how to model: some of these recipes are probabil...

for example?

distant igloo Dec 5, 2024, 7:33 PM

#

Recycling belts gives you plates and gears. What do you do with extra gears? What do you do with extra plates?

sullen bluff Dec 5, 2024, 7:33 PM

#

No

#

if you grind for high quality belts

#

you recycle and priority input those quality gears in assemblers for belts

#

such systems never overflow and should not overflow if done correctly

distant igloo Dec 5, 2024, 7:34 PM

#

Are you making high quality gears elsewhere? Plates too?

#

What do you do with extra rare plates where you don't have any rare gears to combine them with?

sullen bluff Dec 5, 2024, 7:35 PM

#

distant igloo Recycling belts gives you plates and gears. What do you do with extra gears? Wha...

like, the materials will be given to you from recyclers in the roght proportion, so it's not like one material will fill the chest and clock the system after long enough

distant igloo Dec 5, 2024, 7:35 PM

#

The output is probabilistic and may drift from the right proportion

sullen bluff Dec 5, 2024, 7:35 PM

#

distant igloo Are you making high quality gears elsewhere? Plates too?

i personally do not, i grind from common ingredients

distant igloo Dec 5, 2024, 7:35 PM

#

If you try this with just a short sushi belt, it will deadlock due to this

sullen bluff Dec 5, 2024, 7:36 PM

#

distant igloo The output is probabilistic and may drift from the right proportion

i had a module factory running for 20 hours, the chests were always empty

#

gimme a sec ill show it to you

#

everything i had was made like this

#

for modules too, and never backed up

distant igloo Dec 5, 2024, 7:37 PM

#

Module 2s take in 4 module 1s, so the recycling outputs 1. No probability there. Belts take one gear, so they output 0.25 gears. Probability is an issue

sullen bluff Dec 5, 2024, 7:37 PM

#

hm ok

#

than sorry i don't think i can give good advice here, but some guys here done very good math, ask them ig

distant igloo Dec 5, 2024, 7:39 PM

#

That is, in fact, what I was doing 😛

fathom iron Dec 5, 2024, 7:40 PM

#

distant igloo Another thing that I don't know how to model: some of these recipes are probabil...

were you asking about how to model this in theory? Or what to do with excess of something in practice? The solver sometimes creates byproducts btw, but you can always recycle stuff down if it backs up I think.

distant igloo Dec 5, 2024, 7:49 PM

#

Yeah, in theory

#

Recycling down to nothing definitely works. I'm curious what the LP-approved optimal way is though

fathom iron Dec 5, 2024, 8:04 PM

#

for probabilities I just use EV. Is there a particular recipe or item you were interested in?

fathom iron Dec 6, 2024, 3:00 PM

#

distant igloo Recycling down to nothing definitely works. I'm curious what the LP-approved opt...

I can answer this more precisely about what the linear solver is doing in my code. If you set the --allow-byproducts flag, it gives every item in the game a costless voiding recipe. If you do not set this flag, all items are constrained to zero and must be recycled, which can add additional building or module costs.

fair arrow Dec 6, 2024, 6:32 PM

#

Posting to follow for catching up later

keen thistle Dec 6, 2024, 6:34 PM

#

linear programming is dark magic for me, imma lurk

fathom iron Dec 11, 2024, 2:38 PM

#

is the solver broken? https://github.com/scottmsul/FactorioQualityOptimizer/issues/13

GitHub

Solver recommends putting quality modules in crafting machines with...

An example way to reproduce is running the script with the following options: --input-items 'heavy-oil=0' 'scrap=0' --output-item quality-module-3 --module-cost 1 --productivity-res...

distant igloo Dec 11, 2024, 2:43 PM

#

This seems like another case where you should allow empty module slots

fathom iron Dec 11, 2024, 2:44 PM

#

yeah

#

I don't think it would be hard to add the extra for loop, but I'm wondering how much it would slow it down, basically turning everything from O(N)ish to O(N^2)ish

#

for the combinatorics of modules

#

even checking the speed modules makes it noticeably slower

distant igloo Dec 11, 2024, 2:46 PM

#

You could add a special case of no modules, and not consider partially filled buildings

#

There are probably a few other cases that are unlikely to be useful, e.g. some speed modules and some empty spots, or speed modules without some other kind of module

fathom iron Dec 11, 2024, 2:48 PM

#

speed is fairly easy to check since it only checks it coming from beacons, but it basically just adds an extra O(N) overhead factor for each speed modifier being checked

#

I don't think it's worth checking speed in the building's modules

#

but having empty modules in the buildings could be useful

distant igloo Dec 11, 2024, 7:40 PM

#

Makes sense

astral barn Dec 12, 2024, 8:06 PM

#

+1 to the request of allowing empty module slots. As you have free time of course. It feels wrong see quality mods in liquid recipes. This would always increase the building cost since they'd run slower, right?

RE: performance, maybe it can be optional flag if you don't want to increase the baseline compute time—assuming this doesn't make the whole thing more complicated

fathom iron Dec 12, 2024, 8:10 PM

#

I speculated a bit on why I think it was picking qual over prod in some liquid cases in the issue thread on github. Basically the speed penalty is worse for prod than quality, so if an output is making everything from scrap, and holmium isn't the limited factor, then it might not need the entire prod bonus relative to other ingredients, and so it would be better to make it with a smaller speed penalty if the prod isn't needed, hence the quality module taking its place.

astral barn Dec 12, 2024, 8:13 PM

#

I think I'm missing something. How is speed worse for prod than quality?

fathom iron Dec 12, 2024, 8:13 PM

#

-15% instead of -5%

#

for the tier 3s

#

to clarify, prod has a bigger speed penalty to the machine than qual, so it would end up with more machines

astral barn Dec 12, 2024, 8:18 PM

#

oh oh, okay. Yeah that part makes sense to me. Then I misread your comment here 😅

I'm not extremely familiar with the codebase but skimming over it seems right now there is no possibility for an empty module, which would be a lower building & module cost in some cases. And you are currently deciding whether or not to implement that as a feature.

fathom iron Dec 12, 2024, 8:19 PM

#

astral barn oh oh, okay. Yeah that part makes sense to me. Then I misread your comment here ...

Probably wouldn't actually be too hard tbh, but want to get a basic version of the web thing first. If I did that it would definitely be an optional feature because it would probably make it take much longer to run.

astral barn Dec 12, 2024, 8:20 PM

#

awesome! I totally get priorities too. I will be watching from the sidelines 🙂 thanks for all the work

fathom iron Dec 12, 2024, 8:22 PM

#

new code would likely involve extending this for loop: https://github.com/scottmsul/FactorioQualityOptimizer/blob/master/scripts/linear_solver.py#L339

GitHub

FactorioQualityOptimizer/scripts/linear_solver.py at master · scott...

Optimizes prod/quality module ratios for creating high-quality products in Factorio - scottmsul/FactorioQualityOptimizer

astral barn Dec 12, 2024, 9:05 PM

#

Just realized after reading through the code more carefully that I had been neglecting the "check-speed-modules" arg. Doh. This kinda gets closer to what I expected. It doesn't really give useful results (IMO) for smaller output amounts. Changing to a "building count" cost as you described in the github comment would probably tune this part better as well.

I see what you mean about extending the loop. You'd have to add yet another iterable to the cartesian product, right? The flat 16 from speed beacons already blows up runtime so your point is well made.

fathom iron Dec 12, 2024, 9:07 PM

#

pretty much yeah, though it may need to be implemented as two for loops since the quality modules would only go up to the number of modules, so it's not really a full cartesian product and more like a triangular number

#Thread for LP nonsense!