#Arbitrary throughput universal machine parallelisation with arbitrary nonconsumable sets

235 messages · Page 1 of 1 (latest)

storm charm
#

Full title (it would not fit in the thread box):
Arbitrary throughput universal machine parallelisation with arbitrary nonconsumable sets for any machine using cell shuffling and ME IO ports

See attached text file, images, and this youtube video: https://www.youtube.com/watch?v=rbKXnSovjVw
(the title would not fit on the youtube video so I removed some words from it)
I considered putting the text doc in latex but decided narrowly against it.

text document is just under 3500 words.

main_setup.png processing_machine_array.png 📎 uaal-io-shuffling.txt
YouTube
Ivelieu
Demo: Arbitrary throughput machine parallelisation with arbitrary n...
▶ Play video
hard inlet
#

Important to note it works with block mode nonconsumable

#

That’s not circuits

tawdry vigil
#

This does seem pretty powerful, circumventing the item conduit extraction rate that my setup suffers from, however I don't think it's worth the seemingly 10x complexity over my design - with proper pattern sizes you have 6.4 seconds to cycle the cells which allows you to parallel roughly 32 machine quads before being limited by the conduit speeds (at 1 tick recipes)

One of the main goals of my Universal was to make it as simple as possible so setting it up is easier - if you don't care about simplicity and just want the optimal automation, this is likely better. I did also experiment with using I/O ports but found conduits much more intuitive

hard inlet
tawdry vigil
#

My solution to that was editing the code so lenses and molds are treated the same as circuits by blocking mode 😅

hard inlet
#

Lmao it should be that way

#

How hard was it

autumn hound
#

aren't there some nonconsumables that are neither lenses, molds nor circuits? I remember seeing a copper plate as one somewhere

hard inlet
#

Several

tawdry vigil
hard inlet
#

@storm charm does this setup handle multiple nonconsumable fine

storm charm
#

yes

tawdry vigil
#

It does

storm charm
#

up to 9 itemstacks

tawdry vigil
#

Read it thunkban

hard inlet
#

Just add more to the interface

hard inlet
storm charm
#

I am particularly enthused to try using this to automate multiple bm rituals. fully from assembly to deconstruction.
so i have like a 'ritual area' where 1 of 20 rituals can be summoned. i think thatll look great.

hard inlet
#

Cool idea lol

storm charm
#

automating parallel bm rituals like this is somewhat pointless as far as tier progression but i think itll look cool

#

exactly

#

i would be interested to see how hard it would be to write a hook that auto blacklists nonconsumables for adv blocking card and if such a thing would be performant and/or remotely practical

autumn hound
#

would that then for example blacklist copper plates? seems a bit weird tbh

storm charm
#

yes exactly

#

it does

meager tusk
#

these are going too far lol

tawdry vigil
meager tusk
#

lmao true

hard inlet
#

If only stocking fluid

red reef
tawdry vigil
torpid bloom
#

When there is so much AE that I can't figure out where the actual machine is supposed to connect concern

#

Does this still use the stocking bus to actually start the craft (and is hence subject to the up to 5 seconds delay per crafting step?)

tawdry vigil
#

Yes, it is simulated with an export bus to a trash can here

#

I believe any "cell cycler" -derivative has to use the stocking bus because there is no other way of extracting the non-consumed items from the other item insertion options

torpid bloom
#

yeah

#

I was brainstorming something that would use transvector dislocators to physically swap out input buses with NC items

#

Can't figure out a way to do this at scale though. (Both n machines and m non-consumable items.)

tawdry vigil
#

Does that work on controllers kek2
Could swap PA machines that way, kind of
Although there's basically no gain in this case

torpid bloom
#

Yes it does work, I don't know who it was, but someone showed that a controller can in some cases even keep running when dislocated from its machine.

#

(That is obviously not something that is intended and can be relied on in future patches.)

#

And if you include something like SFM then you can even dynamically break/replace PAs and put in machines inside them.

tawdry vigil
#

As far as I know the controller slot cannot be accessed by automation?

torpid bloom
#

Not even place items into it?

tawdry vigil
#

I've never tried personally but I don't think I've ever seen it being done

torpid bloom
#

Something like this should be at least in theory possible:

  • Place consumable inputs into an input bus.
  • Translocate another input bus with NC items.
  • Translocate PA controller with desired machines.
  • Start processing (can do this immediately with a redstone signal to PA controller cover)
  • Translocate PA controller to a "work location", simply another PA structure with an output bus/hatch
  • Translocate input bus with NC back where it was
  • Repeat for the next recipe with another PA controller
#

So rather than having one decoder per machine (or block of machines connected by a stocking bus), you have one "PA decoder" that transports running PA controllers with machines inside them

dire yacht
storm charm
# torpid bloom Yes it does work, I don't know who it was, but someone showed that a controller ...

it does, ive tested it actually, for PAs its really pointless though because the controllers contain the most expensive part aside from the energy hatches i suppose. its actually a fairly reliable method to run multiple dtpf or eoh controllers with only one multi fully built and the rest just swapped out, you just swap them in just before the machine does structure checks.

although i think some people would call that an exploit, im not sure if its 'bad' enough to try and bug report and 'fix'.

storm charm
dire yacht
#

they're pretty, just a lot of stuff on the screen for eyes to wander onto instead of the material you actually want to showcase

storm charm
#

I want to showcase the fractals too that's why they're in the image 😛
I get your point though and its totally reasonable. if i had of thought of it before making most of the image and showing it to people I would have swapped out the resource pack. its just something ill try to remember in future for complex setups

#

it would probably take about 15 minutes for me to make a vanilla ae2 snippets instead of my texture packed one, plus removing the background. soo... if thats something you want i suppose i could do it.
joys of making images with layers. its so painless to change stuff.

storm charm
# tawdry vigil I believe any "cell cycler" -derivative has to use the stocking bus because ther...

there is actually a way, replace the ME chests on the output machines with IO ports, each machine subnet has an IO port with a storage bus and fluid storage bus guaranteed to hold capacity. nothing else in the setup needs to be modified, so its actually a fairly simple change to not use stocking busses if you so desire.
(and of course because IO ports are blazing fast they will never ever bottleneck you. seriously. I tried three hyper acceleration cards once and it crashed a server it was transferring items so fast.)
the problem with doing this is figuring out how to extract the nonconsumable after the recipe is done, plus this setup requires redstone control so a 2nd cell is not inserted with a possibly recipe conflicting nonconsumable.

#

technically you would still need an ME chest to buffer and guarantee only one cell insertion so the setup would look like..

tawdry vigil
#

I did have a design where an I/O port dumped everything in 1 tick to a machine-specific subnet ME cell which then returned the non-consumed back to the "main" subnet after recipe was done, but it was more complex without much of a gain - I also didn't figure out a way to "block" more recipes being inserted

torpid bloom
#

Honestly with your setup I could see breaking and replacing the input bus (in 1 tick with SFM so you never break the multiblock) and recycling the NC item later to be viable.

storm charm
tawdry vigil
#

Transvectors 💀

storm charm
#

BUT

#

you can use drives instead of transvectors

#

there is a way

tawdry vigil
#

Yeah I know

#

Just need to block 9 sides

#

Or slots

storm charm
#

Yes

#

id really like if there was umm

#

a "cell putty" or "cell jelly" item that fills up the drive slots

#

or a "storage downgrade" drive upgrade

#

hmm.. i could probably implement that actually...HMMMM

torpid bloom
#

Can you format a void cell to some unusable item?

storm charm
#

pardon?

torpid bloom
#

Or even essentia cells or something?

storm charm
#

1k drives are fine

torpid bloom
#

yea

storm charm
#

you just partition them or make sure they have no storage bus to go to

#

its just a pain

#

putting 9 cells that dont normally itemstack into the drives

torpid bloom
#

right I see

storm charm
#

"dont normally itemstack" never thought id say this. but it actually is a timesaver.

#

perhaps you could memory card slot sizes even

#

or maybe "storage limiter upgrade" just a configurable button compatible with memory card

storm charm
torpid bloom
#

Hmmm gonna test planes

storm charm
#

me too

#

tthey are super fast so

#

you could just toggle bus it

#

and the delay would be acceptable

#

omg it works..no way

#

this is genius

#

the input bus does not keep its orientation when being formation planed though

torpid bloom
#

It seems that the formation plane always places the bus with the input facing the plane though

storm charm
#

yes

#

how could this be fixed

torpid bloom
#

You could transvector stuff in?

storm charm
#

you cant bind it

#

the bind will break and it wont reform

torpid bloom
#

TVI should keep the linked position even when the bound block is broken no?

storm charm
#

the bind will break if it checks after a while iirc

#

you could use frames to move the bus

#

wait wait i have a better idea

#

transvector dislocator

#
  • bind the dislocator to the correct location of the input bus
  • when you want to take the nonconsumable out you first swap the dislocator with the currently formed input bus
  • you then break the input bus with annihiliation plane
  • you then replace the input bus with formation plane on that some block location
  • you then send redstone pulse to dislocator
    cause dislocators never unbind
torpid bloom
#

yeah that should work

#

You just need a dislocator + b&r setup for each machine, right?

storm charm
#

okay now its time to design this in the most compact and efficient and tps friendly and loki friendly way possible

#

yeah you can share one input bus per quad

#

orrrrr

#

if you're crazy

torpid bloom
#

yea quad/octa/machine, same thing

storm charm
#

only use one input bus for a lot of quads

#

and just a bunch of dislocators

#

thats so crazy

#

because you can round robin dislocators i think itd work though

torpid bloom
#

UHHH

#

I got it

#

let me test something

#

Key part here being linked input bus.

Breaking a LIB drops anything it was holding on the ground (so also from all the other linked inventories). You can have a LIB in all machines, insert one NC item, start the recipe(s), then break another LIB somewhere else in one central place, and retrieve the NC item.

The only problem is that when you replace the LIB you need to again link it to the same frequency.

Golem with a data stick can do this "instantly" (as soon as the bus is interactible with after placing).
https://i.imgur.com/MoZuw5l.png

Imgur
broken tulip
#

wat

#

it definitely shouldn't drop the item it's holding if there are other linked busses on the same freq

kind lagoon
#

@storm charm another thing that I havent seen mentioned yet in this chat is how much laggier is this setup compared to sampsas enderio conduit swapper or my toggle bus setup? I bet a lot laggier

broken tulip
#

ah I see why it breaks... silly off by one

storm charm
# kind lagoon <@105572816128069632> another thing that I havent seen mentioned yet in this cha...

toggle bus subnet switching is rather bad for tps because network recalculation is expensive -- ae tries to compensate for this by spreading out ticks for network recalculation over a longer period of time afiak. it also adds quite a few render updates for network components lighting up and turning off.

eio conduit TESR is rather notorious for fps drops and I would be inclined to believe ae <-> eio filtered item interfacing introduces a small tps overhead too. eio round robin is very tps performant though.

The only non tps performant item in the demo setup is translocators which I mention I am only using for demo purposes. GT pipes with high tier conveyors are fps and tps performant - especially because the way they are used in this setup is unfiltered.

The only render updates will be on the drives and the depositrd ME chests so it is impossible to improve fps without artificially limiting speed. obviously you wouldn't use storage monitors on a real setup.

A significant advantage of ae2 purist approaches for automation is tps performance. Interface<->storage bus junctions have special handling code whose performance cannot be replicated with other mods. OC and eio and sfm tend to really suffer on performance when scaled up but ae depositing with storage busses is extremely optimised. Because there is no subnet switching I am inclined to believe if I profile this setup in a test world the highest tps consumer would actually be the quartz fiber doing fake power checks -- easily fixed with neutronium energy cell spam. (maybe I should make a pr fixing this long standing quartz fiber issue...)

#

In other words, aside from the overhead introduced by just increasing throughput (which is very modest) I believe this setup is more tps performant than both toggle busses and sampsa's enderio approach.

To truly compare tps performance with both setups side by side is quite easy since it is easy to artificially rate limit my design by just changing the rate cells are transferred into IO port array 1. If you provide test worlds of the setups to compare I can try to do this and run a profiler with them artificially scaled up copied with worldedit mcedit or something. If worldedit mcedit works on everything, anyway.

I have to emphasize that I found manually filtering the eio conduits time consuming and cause it can't be loki'd I don't want to have to manually spam build that for high throughput testing.

#

I suppose the fact that it is easy to rate limit my design to any slower speed by using a combination of fewer drives and slower transfer to IO port array 1 is another advantage of my setup because it seems very difficult to precisely control throughout rate limiting for subnet switching or with eio conduits.

#

It also occurred to me that the white/lightgray subnets can be replaced with gt pipes and high tier conveyors, if the warmup time in the import busses is found to be too slow in practice. But it is just for the warmup time.

dapper crane
#

How did you paralelise fluids?

#

Or did you just... Not?

kind lagoon
#

I have to emphasize that I found manually filtering the eio conduits time consuming and cause it can't be loki'd I don't want to have to manually spam build that for high throughput testing.
in sampsas setup this only needs to be done once for each circuit config. the me chests at each stocking bus does not need any filters. this means the amount of clicking you need to do here only scales by the number of different circuit configs you have, and a ring of loki would not help whatsoever for this, even if it worked, since every single filter needs to be different

storm charm
# dapper crane How did you paralelise fluids?

The dual interface P2P <-> dual interface junction inserts the fluid packet, you just need the fluid discretiser, fluid export bus and fluid storage bus on an input hatch for the local machine. same as sampsa setup.

kind lagoon
#

thats not the same as sampsas, since he uses export bus filtered to packets with a fuzzy card going into a packet decoder -> fluid storage bus -> input hatch

storm charm
kind lagoon
#

also fluid discretizers can't decode fluid packets, it can only convert between drops and real fluid

storm charm
#

oh sorry export bus filtered to packets

#

you're right

#

i just forgot

storm charm
# kind lagoon > I have to emphasize that I found manually filtering the eio conduits time cons...

yeah the issue is not with the me chest for local machines but for the drives the cells are being shuffled from which are connected to the dual interfaces.
if i want to automate 80 mixer quads then I need 8 drives per config, 20 configs, took me ages to do. my setup I just deposit the 20 different configs in the subnet, each subnet is lokid, I only need to open the gui once and do one click versus eio conduits which in this case would be 8 guis and each gui needs a few clicks.

#

it wouldn't be hard to set up an autoclicker macro to do the eio conduit filters, but again, this is really something conduit probe should be able to do, just pull and insert filters from your inventory if they are saved to conduit memory.

kind lagoon
# storm charm yeah the issue is not with the me chest for local machines but for the drives th...

if you want to automate an infinite number of mixer quads then since mixers have a lot of conflicts you will need several ME chests for each circuit. at a guess lets say mixers have 16 different circuits and that 25% of them need 2 circuits due to conflicts (so you have 2 ME chests with circuit 1, another 2 with circuit 2, and so on). this gives a total of 20 circuit configs. this means you need to configure exactly 20 extract filters and 20 insert filters for a grand total of 40 times that you need to drag and drop an item into an enderio filter ui. this setup will then be able to control an INFINITE number of mixers as I said

storm charm
#

if the max cell capacity on the input connected to your p2p duals is 20 cells, then you can have at most 20 mixer quads running simultaneously. am I missing something in your description?

kind lagoon
#

and importantly, each mixer quad needs no enderio filters whatsoever. all the filters are only on the giant tower of ME chests of which you only need one

kind lagoon
#

or even multiple chests

#

compressed chests are pretty large

storm charm
#

and you are saying that chest is depositing items into.. an ME chest hooked up to the dual p2ps?

kind lagoon
#

or you can store the cells inside another ME chest, and since items with the same NBT stack inside AE you could have a million cells of the same circuit config (63 types) in one ME chest

kind lagoon
storm charm
kind lagoon
storm charm
#

my test setup one ticking recipes with a 32 machine array hit a bottleneck using about 160 cells simultaneously
if you want to parallel run, say, 500 engraver quads, it'll be quite inconvenient to use your approach, as each individual subnet for each different nonconsumable will require its own buffer of a few hundred or few thousand cells filled with configs. in my setup, one subnet can supply empty cells to all the different nonconsumable configs, even for different machines.

#

manually stacking a few hundred cells into compressed chests, several times, sounds time consuming to me. and if you instead choose to deposit them automatically into compressed chest there is a modest risk of memory corruption at that volume of cells, even if they only contain one item.

kind lagoon
#

this is sampsas entire setup

#

now you can have an entire compressed chest's worth of cells per circuit

#

optionally replace the leftmost ME chest with an ME drive, also works and may be marginally faster

kind lagoon
storm charm
# kind lagoon if you want to automate an infinite number of mixer quads then since mixers have...

so I realise now what you meant, you were using ME chests to interface with main net, sampsa has used both ME chests and ME drives to interface with main net.
using drives has the drawback of potential recipe underflow/overflow which will jam, which is entirely a non-issue in my setup. drives have the advantage that ae can always push a recipe once per tick, ME chests will have to have downtime, at bare minimum they will take 3 ticks as you need
1 tick to deposit items into cell
1 tick to extract cell
1 tick to insert new cell
with eio conduits of course this would be much slower though.

kind lagoon
#

this means each cell only gets 1 insertion's worth of items hence why you multiply it into the millions

#

but you want to multiply it that far ANYWAY to be able to even calculate stargate components at all

#

so thats a non issue

storm charm
kind lagoon
#

yea but it should, so it should be fixed in code

storm charm
#

I wrote about this a bit in the text doc
it's going to be an ephemeral issue because there are tons of nonconsumables and it's very impractical to blacklist them all, plus there are things which it would never make sense to blacklist to begin with.

#

it's very easy to miss a nonconsumable on blacklist if the blacklist is managed manually too, which drives this cycle of "oh, it doesn't work this patch, but it'll work next patch, but next patch might have new broken things"

kind lagoon
#

give the adv blocking card an option to manually blacklist stuff?

#

anyway gotta go afk for a while

storm charm
dapper crane
storm charm
#

ill draw it out for you excavon

#

it's the same thing sampsa does

storm charm
#

well I guess also acceleration cards

#

I am currently still trying to figure out an efficient way to replace the ME chest with an IO port, because the IO port can push much faster than the export bus

#

the LIB technique is nice but it sounds like an unintended bug that may be patched so im looking for a different approach to remove config circuits

dapper crane
#

yeah, I understand that, but what if it tries to parallelize 4 recipes between 5 machines?

#

it'll put 4/5 of the required fluid amount into each machine and they'll get stuck, no?

storm charm
#

no

#

one recipes worth is pushed into one cell

dapper crane
#

ah.

#

so it's smart about it.

#

I designed half of the same thing and got stuck with fluids.

#

It'll be much easier if the fluid stocking hatch ever gets merged.

storm charm
#

well you need this fluid packet decoder setup once for each processing machine.

#

fluid stocking hatch will trivialise this absolutely, i expect it to be gated to luv at minimum, i saw talk gating it much higher though.
right now its way too buggy and dupe-y to be merged though. you probably know all this

dapper crane
#

I do.

storm charm
#

so yes it does occur to me now I shouldn't have dual P2Ps on the white and light gray nets, they should be just regular interfaces.

tawdry vigil
kind lagoon
tawdry vigil
kind lagoon
storm charm
#

Still gonna be snail pace though.
It's worth mentioning that there is a point where scaling up recipe sizes does lead to a disadvantage because it can make load balancing worse. If your recipe size is really big, then fewer copies of the recipe will be pushed, and if this number is smaller than the number of parallel machines, the time to finish running will be longer than with a smaller pattern by comparison. Small, faster to run patterns benefit more in raw throughput from higher cell shuffling speed (and thus better load balancing), whereas large patterns receive more overall time reduction when doing big crafts when the load balancing is better.

It's also somewhat of a pain to level maintain a lot of different circuits and parts, and if you don't, the delay introduced by 8000 circuits/parts being crafted by one machine, for one item while I'm testing things, has bothered me on more than one occasion.

kind lagoon
storm charm
#

Meanwhile me and my parallel nonconsumable patterned platline in IV :p

I think it's useful pretty early on too. Just my personal opinion though.

torpid bloom
#

It's useful very early on when machines actually cost a non-trivial amount of resources, then it's easier to just spam more, then it's useful again when machines cost a non-trivial amount of TPS

storm charm
#

Yes, I agree with that.

hollow basin
#

@storm charm I'm making a variant of this concept inspired by your design, but I'm having an issue with getting the nonconsumables into the drives via the I/O Port
It seems like randomly the ME Interface stocking the nonconsumables won't be pulled from by the I/O port. The odd thing is that it seems really random, seem to happen towards the beginning not the end of a big request, when the Import Busses are still warming up, and seem more frequent if i run two orders at once, though again it seems random. Any Advice? Best i can think of is to use Transfer nodes with lots of speed upgrades and a hyper rationing pipe to increase the frequency the stock of nonconsumables are refreshed, but it seems roundabout to fix a obscure issue

You can see in the screenshot, two of the drives in the 200 drive order passed the I/O port without picking up a consumable.

image.png
#

Another example. Upon starting the second order, the non-conumable got skipped once then all other drives were correctly loaded

image.png
storm charm
# hollow basin <@105572816128069632> I'm making a variant of this concept inspired by your desi...

It's hard to comment without seeing in detail what your setup is like, here are some troubleshoots I have went though:

  • Are there more stocked NC sets than cells?
  • Are ae subet components power shorting due to IO rapid power drain?
  • Is one cell entering the IO port at a time?
  • Are all cells entering IO port for only one tick? (I used OC to debug this a couple times)
  • Does terminal view on the NC depositing IO port flicker?
#

1.7 interface stocking is tick perfect. But I have decided it's better to replace the import bus with transfer nodes and speed upgrades because their filter works for the cells, they have faster max speeds and no rampup. For the past two weeks I've been working on a more detailed tutorial video reviewing some applications and tradeoffs from Sampsa approach in detail too.

#

In 1.12 when I did this on TJ I was finding interfaces are not tick perfect which required a multi pass system; if the cell doesn't match the NC set it should have, then send it back. In practice this rate was 10% of passes and requires some tricks with extracting the consumable items first in order to get it checked with filters rather than slower level emitters. You should not need a multi pass system in gtnh though.

hollow basin
# storm charm It's hard to comment without seeing in detail what your setup is like, here are ...
  1. Ive got creative cells for NCs for testing, 1 stocked NC in interface
  2. Im using creative power cells for testing, and id expect a power deficit would betsist as an issue
  3. As far as i can tell. I have a 1 import to 1 IO ratio
    4.i think so, this problem has persisted between both unaccelerated and three hyper acceleration cards
  4. It does flicker, faster as the import busses warm up, then stops flickering at max speed

Am definitely resisting a multipass system since that defeats some of the point of avoiding custom cells for each NC

storm charm
#

Yeah. Accel/hyper accel only acts on a throttling system so I wouldn't expect it to change much. Can you try non creative cells on the NC set storage?

The flickering on warmup is a clue at least. Try replacing the IO port with other filtered single extract options like filtered transfer node or conduit just to see if the issue persists

hollow basin
#

I've even completely isolated green(NC Interface) and blue (IO Port) nets to their own dedicated power, and this still occurs at startup

image.png
#

At least it is consistant and reproducable now

hollow basin
#

Using a hyper-rationing pipe to buffer chest to storage bus seems to fix the issue. I suspects there is some sort of lag or tps spike made on orders that freezes the interface's ability to restock itself at the start of a big order

image.png
hollow basin
#

actually a pedestal does the same thing but cooler and without the hyper rationing pipe

image.png
hollow basin
#

actually i kind of like the drying rack aesthetic

image.png
full tusk
#

just use "p" hotkey with item on hand

storm charm
storm charm
storm charm
storm charm
#

I'm also wondering why adding the one slot buffer fixes the issue for you. Because that buffer shouldn't change the IO rate of the stocking interface to a transfer node. And in your earlier screenshot it was the first four cells that were missing the NC, not just the first one.

hollow basin
hollow basin
storm charm
#

Yeah that doesn't sound like much

#

Oh I just realised in your image the transfer node from green subnet is probably backed up with 64 sets of the 1 NC circuit which is why you wouldn't get an issue with interface stocking.

#

I need to go to bed soon but I'll start up an instance and have another look at my setup for reference ingame

hollow basin
# storm charm I'm also wondering why adding the one slot buffer fixes the issue for you. Becau...

What i suspect creates the fix is that im no longer relying on ME to be perfect. The transfer node has an internal stack but can only output one item into the buffer, but does that at 64 operations/ second so it just brute forces being a higher refresh rate than the io port

Id have to check the cells output with a chest not a filing cabinet but i bet that the very first cell is fine, but the skips are in cell 2 or 3 + when there are issues restocking

Do interfaces have to warm up like import busses?

storm charm
#

No interfaces don't have warmup and they are tick perfect in 1.7 which is what inclines me to think the issue can be fixed

#

This is what my current demo setup looks like for reference

image.png
storm charm
# hollow basin What i suspect creates the fix is that im no longer relying on ME to be perfect....

Well the only other thing I can suggest is I can try it if you give a world download or join a server with it. I'm out of ideas at the moment what the issue could be leading to interface stocking not being tick perfect in your demo.
Sometimes I have success overriding handler code using transfectors on full block interfaces but I don't think that will fix the underlying issue here.

Also with the 64 ops/sec that will inevitably get squeezed into 20 ticks, I'm not exactly sure how exu does that squeezing like in pretick and posttick which is why the IO port always picks it up and never gets duplicates either.

hollow basin
#

Yeah that matches what ive got on the interface+storage variant, just with some cable length differences

Ill see about a world download

storm charm
#

Okay, I can have a look tomorrow

hollow basin
#

Doing some more findings
Interestingly, If i just swap out the setup back to storage bus+interface, keeping the transfer node filled cell extraction speed constant, its just straight outcompeted, only 39 cells get correctly formatted

image.png
#

Going down to a no speed upgrade extraction rate, and looking at the order the drives arrive, my theory is correct. Drive 1 is correctly formatted, drives 2-5 are missing NCs, and every subsequent drive is correctly formatted

image.png
#

So i think im likely to go with the 1item buffer approach since its just naturally faster, but its good to get confirmation whats going on here

hollow basin
storm charm
# hollow basin For when you got the time The consistent test is ordering 6400000 drops of meth...

So I've had a look at this. I tested and saw what you meant, I could reproduce your issue. The first thing I did was strip down all the components except for one NC set and move the subnet around a bit. The green subnet in this image has the same creative cell you used. Then I gave each subnet a neutronium cell. Then I ran the test for 6400000 methanol with the import bus<->storage bus junction, and it worked fine. Then I ran it with filtered transfer node with 16 speed upgrades and it also worked fine.

So I'm going to be honest I'm not 100% sure what the issue was. I think the subnets were a bit cramped/tangled together and there might have been some unexpected subnet communication going on which interfered with the green subnet properly stocking the item cause it clearly wasn't doing it tick perfect. I tested just manually taking out the NC from the interface and was seeing 1-2 second refill delays so I think the attached IO port was making green subnet think really hard before doing the stocking. Also one light blue storage bus was unpartitioned and that was probably causing special handler code to slow down the whole subnet. Also you only need advanced blocking cards on the interface touching the drive, you don't need it on the red subnet interfaces (i.e. the main net with the crafting cpu)

image.png image.png image.png
hollow basin
#

Ah gotcha, so segmenting the nets further, spacing it out, and prepartitioned storage busses should fix it.

Thanks for taking a look, definitely odd behavior, wonder if there is a big floating around in it