Track Network Splitting Via Stations

[wet flicker]

An option on a station to not join the two track networks on either side. Since this is marked as a station in both networks, thr train can pathfind to this station, then proceed onto the other network.
Suggested for https://discord.com/channels/706277846389227612/1431762186853683320 to reduce lag from the giant server track network.

[cyan tendon]

Honestly this should just be in the Create mod and I keep considering adding it. I just don't know if I have the technical skill yet.

Certainly an equivalent of a station block that can be placed which separates out the signal and node networks would be incredibly helpful to prevent lag. This would essentially still connect to the track normally but it would make it so trains within one region cannot route to another one. That sounds inconvenient but it could allow the splitting of the network up into smaller collections which is very useful for large servers.

[keen umbra]

The real problem is that the schedule would need to know in what rail network it's currently mapping

[austere vale]

If the station marker is the point at which the networks are split, wouldn't that make it impossible for the train to move onto the next section, cus the graphs aren't linked anymore?

[keen umbra]

If the schedule and train can switch networks at the split, that shouldn't be a problem. (aka jump from 1 network to another)

The issue is actually doing that

[strange bramble]

Currently working in progress with implementing it (as it looks interesting), so far (half) succeed with splitting track graph for special type of track block. (called SplittingTrackBlock internally, done with mixin into TrackPropagator, TrackGraph; very buggy)
TODO for making it work with manual drive and navigation, which is a lot harder.

[wet flicker]

https://cdn.discordapp.com/attachments/706277846829629474/1405973367819079750/p77c7yjh93tbycyp9t7x.gif

[cyan tendon]

That is absolutely dope. If u need someone to test it we desperately need it for the SnR server haha so we can throw it in the deep end

Also you may find this useful https://github.com/Layers-of-Railways/Semaphore

It shows the track sections, train info, active signals etc...

commands are /semaphorec and /semaphore

[strange bramble]

I didn't know such a tool exist, thank you for letting me know. Will definitely need people to test it.

Now manual drive partially-works, it sometimes rotate 360º but it seems to be client sync problem. The real problem now is pathfinding and signaling...

[zinc heart]

epic

[ancient tiger]

Wait this is 1.21

[zinc heart]

my guess is to make a huge list of all the nodes and stations in the other network and just check x is in this network

[raven needle]

It was made for by Slimest for the SnR server wonderful tool. We have such large track networks that some signals get borked. Semaphorec really helps keep everything running.

[ancient tiger]

Finally someones making a cool track util for 1.21 lets go

[raven needle]

Looks like it. The SnR server is 1.20.1 fabric but honestly just this untility thing existing is huge deal. 1.21 neoforge seems to be the spot to mod rn.

[ancient tiger]

Being on neo 1.21 is good and bad at the same time

I get access to that new mod the cdg dev is making but i lose steam and rails

[raven needle]

Yah, hopefully soon that will change. What a wild world we live in despite everything we have people who make cool stuff made possible through something that runs on a machine made of sand as 1's and 0's and can be enjoyed across then entire world again on a machine made of sand and transported by millions of miles of glass all powered by electrons.

[zinc heart]

https://tenor.com/view/factory-robotics-chipmaking-semiconductor-tsmc-gif-24571725

[austere vale]

damn, i'm glad to be proven wrong 😅
Though from looking at the video it seems to jump the car from one side to the other.. how does it handle longer trains?

[zinc heart]

more shocking that it takes months and thousands of steps to make a cpu

[strange bramble]

umm I've never worked with fabric

I figured out the reason why it jumps: while train is both on two graphs, carriage won't update position. IDK why but..

currently it looks quite funky

[raven needle]

Fair, I'm can't code so not much help there but it's up to you if you even want to get it onto 1.20 in fabric. I know dev stuff is a lot of work. I think at somepoint Slimest will see this but I don't want to ping him, very busy person he is. There are a few others who would might know what is needed here, maybe they can help, just literally all of them are so busy.

[strange bramble]

..but most of code will directly about Minecraft Vanilla and Create, so it'll be easy to implement. maybe?

actually it started as one feature ofmy tiny collection mod..

[raven needle]

I hope, but snr s2 isn't even on 6.0 yet

[strange bramble]

aah

[raven needle]

the server is a bit over a year old

https://github.com/Layers-of-Railways/modpack

[raven needle]

That's the whole pack, really the only info I know that might be useful.

https://modrinth.com/modpack/steam-n-rails-modpack here's the modrinth link to the pack as well

[raven needle]

reason why is there are a few mods that just aren't on 6.0 and 1 or 2 major ones that have been abandoned.

[strange bramble]

now works smoothly with manual driving. need to work with pathfinding and signaling though

(stopping animation weirdly showing while exiting graph boundary <- to fix later; lower priority)
Create railway code is getting more complicated as I discover more about it

[zinc heart]

AMAZING

[drowsy wharf]

wait what was abandoned?

[wet flicker]

i know one crwate addon abandoned fabric

forgot which

(if it was even on the server)

[strange bramble]

IDK why it works but now it works with most simplest scheduling. (without signal -- this is my next goal)

[vocal perch]

我以前也有过这个想法，没想到真的有大佬实现了！
I had this idea before too — didn’t expect some genius actually pulled it off!

Coooooooool!

[strange bramble]

...Suddenly I wonder how much track graph causes lag, and in which aspect it causes? If anyone knows let me know.

1. maybe track block place/remove like a) simply place, b) graph merge/split,
2. train schedule/navigation(pathfinding),
3. signaling,
4. world loading/saving, (which this feature will not help much)
5. network overload caused by track graph sync, (when joining world or graph changed)
6. ..or everything lags / cannot figure out what causes lag?

[cyan tendon]

• Attaching track to a large network can cause a tonne of lag. We're kind of testing at the moment if intersecting track especially of different gauges makes the network even larger as well. We don't know if this is because the network has to update all running trains as well or not or if the networks size is the problem yet. Any assumptions I made are just assumptions that need to be tested.

• It only became a problem at a large scale though. It takes thousands of signals (we have 100000+ in one network on the server atm) to replicate something like that.

• The primary cause of lag is pathfinding to the next station within the global railway manager that happens every 100m (which is very frequent for fast trains). This is kind of new information though that Yoku discovered and is backed up by multiple spark reports we took. We're going to look into optimising by reducing the rate at which it's called. The actual pathfinding algorithm seems optimised enough.

• Track splitting should reduce the size of the overall network which should help with lag when adding new track. A network with less overall signals and therefore less areas to search within has to be more optimised in some way. I just can't be certain where yet and i'm being a lot more cautious about making assumptions.

• What we can do as well is test your mod on our server to give you some feedback. I can't promise it will make a drastic difference but if we see anything we can send it back to u.

[ancient tiger]

Talking about gauges are any bogeys that can move between gauges that arent invisible planned, like multiple wheelsets on it

[cyan tendon]

Not really the thread for it hun

u can create a new one if u want though

[ancient tiger]

Yeah

Nvm that already got denied

This mod looks really good

[strange bramble]

Thanks for detailed reply, actually there are lots of code that iterates over every graphs so splitting excessively may not help (though might be optimizable)
Also thinking that changing nodes implementation(curremtly just HashMap, also nodesById is not something like Int2ObjectMap, it is `Map<Integer, TrackNode>) might change performance

[strange bramble]

TODO for TrackGraph performance (for me to read later):

• TrackPropagator: when merging graphs, make it merge smaller graph into larger
• TrackGraph:
  • when adding node, incrementally calculate bounds / adding or removing node -> update checksum incrementally
  • Change nodesById to use Integer2ObjectMap series / make internal maps use more optimized collection type like HashMap/AVLTree/RBTree etc.
  • in connectedNodes(called every time when placing/removing tracks), it iterates over EVERY NODES IN EVERY GRAPH to create TrackEdgeIntersection; some optimization needed, should I add chunk-like system? or splitting down graph then checking graph.bounds will help? (in Create code, it does not check bounds; why they created something like bounds then does not check...)
• GlobalRailwayManager: when player joins, sends every graph. this can be optimized into sending graph on demand - combined with splitting graphs, this will help a lot / but this makes some logics breaK, needs to be careful

[strange bramble]

eww

[cyan tendon]

Merging smaller graphs into larger seems like a bad idea though. You're increasing the size of graphs into ones that could have 100000+ signal or node entries like the ones we have

By contrast we only have about 2000 track graphs which seems like a lot but most only have a tiny amount of entries

Also I doubt the lag is caused on player joining. Pretty much of all the lag is caused in the tick loop through train movement and navigation on SnR

_ _

in connectedNodes(called every time when placing/removing tracks), it iterates over EVERY NODES IN EVERY GRAPH to create TrackEdgeIntersection

Thats really useful to know. Ultimately what that means though is reducing the size of individual graphs will help if you can stop graph updates getting propogated to other networks. Merging track graphs together risks making it even worse.

Using a chunk system doesn't work as well because the railway manager is not chunk specific for the most part. It has it's own rules for the global railway manager

[strange bramble]

This means that, when there is graph with 1 node(A) and graph with 100000 nodes(B) then if place one track between A and B to connect two graphs, which graph is to destroy is not related to graph size. Meaning, in many times B is deleted -> A gets all nodes of B -> new nodes of A(which is from B) is sent to all players. (BTW not certain)
I think this would happen pretty often.

Yeah I think lots of train/track code is quite inefficient

[ancient tiger]

Do schedule trains still see areas across graphs as navigable

[strange bramble]

That is my goal, of course.

[ancient tiger]

Im meaning has that been achieved, just wondering where you are rn

[strange bramble]

This -- everything without signaling works now, I think. Some hidden bugs should exist somewhere however

[ancient tiger]

Yeah hidden bugs are prob unavoidable with creates track thing

Im running into crazy bugs with unmodified create track graphs

This looks like it will be a great utility

[strange bramble]

Now signaling seems to work?

yeah it works

[calm aurora]

it would still be useful to have "chunks" (or some other spacial data structure, such as a quadtree) to lookup track nodes, since then you don't need to iterate through all of them for intersections, but only those in the same spacial region. This would be completely unrelated to minecraft chunks

btw @strange bramble feel free to ping me with questions. No guarantees on response time, but I'll get to them eventually.

[strange bramble]

..?

this mod is touching so many aspect of core Create train graph codes so...

holy carriage sync

[strange bramble]

Development progress: currently it is mostly feature-complete, except for bugs.
Bugs to Fix:

• ✅ Cannot find any routes if inside split node
• ✅ Chained signal does not work
• ✅ Pathfinding: avoiding other trains does not work
• ✅ Pathfinding: goes to inefficient/same direction
• Signal edge group sync sometimes goes wrong
• ✅ Train collision does not work
• ✅ Too much empty graphs
• connectedId is sometimes not properly synced over client
  • As TrackPropagator handles things at once, TrackGraphConnectedIdPacket might sent before TrackGraphSyncPacket.
• cannot relocate over split track
• ✅ cannot target split track / curve from split track
• TODO: Navigation: portalScout

[strange bramble]

I'm mostly done with features of this mod; I think it needs some polishing/backporting/porting to fabric but thinking core of mod is done.
Code is on my github: https://github.com/lhwdev/railx; mixins in https://github.com/lhwdev/railx/tree/new/modules/forge/src/main/java/com/lhwdev/minecraft/railx/mixin/splitGraph and other codes in https://github.com/lhwdev/railx/tree/new/modules/forge/src/main/kotlin/com/lhwdev/minecraft/railx/splitGraph (Kotlin).

Second thought about performance:

• In some aspects, one huge graph over multiple split graphs has advantage; for example, to find which graph contains node at specific location, in one graph most data structures use java.util.HashMap; most lookup is done in constant time. / if using multiple split graph, the time is multiplied by numbers of graph.
  • So, splitting track graphs properly is important. Splitting every single stations from each other is dumb thing.
• Attaching track to a large network: performance definitely improved.
  • But, this: in connectedNode(called every time when placing/removing tracks), it iterates over EVERY NODES IN EVERY GRAPH (to calcuate intersections for signal) is not fixed.
• pathfinding: slight improvement; pathfinding still needs to search over connected graphs (I didn't said all connected graphs).
  • However, navigating to already-pathfound path, is optimized to use only related graphs in path.
  • Having lots of signals on one graph, by itself, does not impair performance, I think. However having lots of signals itself might be a problem. (each signals in every graphs are ticked every time / SignalEdgeGroup is global object; not contained by graph)
  • Currently, DestinationInstruction (where, in Train Schedule, you specify name of station for destination) searchs over all stations in all connected graphs (recursively; this means splitting one graph with 1000 stations into 100 graphs does not impact this)
• lags on player joining is different topic, but if there is optimization to lazily send graph to player than splitting graph will definitely help.

Yeah realworld performance benchmark/testing seems important

[ancient tiger]

How do you build this, it doesnt seem to work the normal way

Oh nevermind i figured it out

What since when were you making off axis tracks

[sacred needle]

i can maybe use some of this for my own train addon
becuase im currently "trying" to make OpenTTD´s Path signals but for Create
i got tried of spamming brass signals in a junction
would just like just 1x path signal at 1x entry of a junction to get the same effect
maybe even more optimized instead of spamming brass chain signals

[strange bramble]

About 6 month? that(so called flexi-track) was my dream car

[ancient tiger]

Pulling that off is amazing

I thought you were just developing the track splitting

This mod just gets better and better

Are there any known instabilities rn

[strange bramble]

Oh discord didn't show me any notification... yeah I think everything is instable as this is just a toy project and I made this only to be used by myself.

If anyone wants, you can download mod jar in here and test: https://github.com/lhwdev/railx/actions (Minecraft 1.21.1, NeoForged 21.1.209+ , Create 6.0.8; select latest item on top of the list -> click NeoForged Mod Jar in bottom 'Artifacts' section)
Note that this mod contains other things I'm experimenting and not feature-complete nor bug-free. Backup your save file before joining.

[ancient tiger]

That was confusing to get the thing, ive never seen multiloader with that layout

Took me ages to decide to look in modules for some reason

[ancient tiger]

So probably dont use this on my main world then

[strange bramble]

umm actually I'm not using multiloader; just gradle monorepo-like thingy. Was planning to split each features(flexiTrack, middleTrack, splitGraph, ...) into other modules

[ancient tiger]

oh ok

it looked like some form of multiloader

[strange bramble]

Yeah in regard of stability I'm using flexiTrak in my own world so I think its quite stable but sometimes it does weird thing

[ancient tiger]

I spent way more time than i should have looking everywhere but modules/forge

[ancient tiger]

But no world or track file corruption yet?

I do take daily backups anyway so doesnt matter too much

[strange bramble]

It does not happen often but it rarely happens(maybe I think).

[ancient tiger]

ok

ive had corruption from some dumb shit on my world before, lead to me making very regular backups

I made the discovery that a train derailing while part of it is in a nether portal corrupts your train data file

[strange bramble]

oh that's interesting,

yeah the system of minecraft makes so easy to corrupt the world (ie. throwing exception while loading/saving BlockEntity makes it deleted)

[ancient tiger]

im just lucky that it only broke that specific train in the file so i just changed a few values to move it somewhere else and rerail it

yeah mc can go wrong so easily

[strange bramble]

I also had hard time recovering my train files, luckily I'm developer so handcrafted create_tracks.dat file

[ancient tiger]

I had to entirely learn how the dat file works from scratch in that moment

At least i know it now incase that would be useful

And ive said this in the create server, it shouldnt have even been possible for it to happen

The train not only went off route onto a dead end line, it went through a red signal and a backwards signal then crashed into another train

This was a scheduled train

It just decided to do that

[strange bramble]

Oh that is stupid

[ancient tiger]

Since that was a manual driven line i have now made it require a turn around so that cant happen again

[strange bramble]

there are definitely some more stupid things in Create code, some of which is fixed in 6.0.7 (like teleporting signal marking)

[ancient tiger]

Yeah

hopefully whatever caused that is fixed in .7 or .8

Whenever i can update to that

[ancient tiger]

oh yeah thats really annoying, i have to switch between flywheel backends constantly

[strange bramble]

also some of internal code structures are changed in .7 so that's reason why SnR or some other mods (railways navigator, etc) break

[strange bramble]

yes that's fortunately fixed in .7 afaik

[ancient tiger]

life without s&r on 1.21

noo theres no gifs in here

Just waiting for railway navigator and copcats plus

[strange bramble]

1.20 -> 1.21 migration looks hard (i started in 1.21 so IDK)
When migrating from 1.21 to 1.22 or more, I know that there are huge breaking changes like DataInput/DataOutput; all serialization code should be updated

oh for that I built my custom mod... aand copycats just work fine?

[ancient tiger]

I wouldnt know rlly at all. I understand the java lang but have no idea how i would even start with mc mods

[ancient tiger]

They work fine? i thought they crash your game if they are on a contraption on 6.0.7 or 8

[strange bramble]

copycats 3.0.2+1.21.1 works fine

[ancient tiger]

Yeah mojang cant stop changing major parts of the game

Everyone going on about deobfuscation, the real thing modders need is for mojang to stop doing that

[strange bramble]

yes... there cannot be some abstraction layer as Create or some mods touch core of minecraft so this is sometimes hard

[ancient tiger]

minecraft doing whatever they did to nbt was sad

cuz theres like no item editing mods anymore

I need to see if this works with create unlimited

The stuff you could do with these two will be crazy