Railbound Solver [Python Script] COMPLETE! Working on Interface...

[fervent tusk]

assuming it cant unpack the variables from the object

[jovial heath]

So assign argslist.pop() to a name and use that in both places

[fervent tusk]

so should i add an __iter__ function that returns the params in a state obj

[fervent tusk]

true

i thought pop returned a list with the last item missing and not just the last item? or

does it modify the list when you use .pop() and then just return the last index to use if you put it in something

[jovial heath]

So if you're yielding a State object you don't need to unpack it, just change generate_tracks to take a single State object

[jovial heath]

Yeah if you have a list [1, 2, 3] and pop from it, pop() returns 3 and the list is now [1, 2]

[fervent tusk]

never knew that thats super cool

[jovial heath]

a = foo.pop() is basically like doing *foo, a = *foo

[fervent tusk]

curious how to go about converting argslist into state_queue (a dict)

to find state id need to get the lowest non-empty value in the dict which means using a for loop unless theres some method i can use

also, id need to iterate over new_states to add each one to its correct spot in state_queue

so instead of having to use a for loop there im wondering if i should just add the states to the queue in the generation function (because thats already using the for loop for every branch so no extra time is wasted) but then i need to make state_queue a global variable and thats kinda messy

[fervent tusk]

(also dont mind the comments i was just trying to figure out how it worked)

[jovial heath]

I had some pseudocode I shared here: #1142318326136180796 message

But then I realised there was a better way to do it: #1142318326136180796 message

[fervent tusk]

i completely forgot about deques

and that whole conversation

thank you for reminding me

the one problem i remember with the deques here is that the states appended are never in order

at least i dont see anything in the pseudocode that would sort it

[jovial heath]

The last line chooses which deque to append to based on whether the number of pieces placed are the same

Using a ternary

It's based on the assumption that you only ever place one tile at a time, so the next state either has one extra piece, or the same number

[fervent tusk]

on levels like 7-5A the tracks available for different generated states could be anywhere from tracks - 1 (only car places) to tracks - 5 (all cars place)

[jovial heath]

If that assumption can be untrue then you need to use a dict to store the deques

[fervent tusk]

why use a deque instead of a list in that case?

theres no sorting to be done so only the top one of the stack would need to be checked, both sides arent needed

[jovial heath]

Using the deque just means you're exploring all paths through the solution space equally, rather than exhausting one branch for a given number of pieces before moving to the next

Whether or not that's useful is unclear though

It's just a purer implementation of BFS

I'd be tempted to implement it with deques, and then once it's working see whether switching to lists changes anything

[fervent tusk]

im not sure i fully understand but i can explain how i plan to do it

[fervent tusk]

i want to have a list for each track count (shown in the code above) and then append new states to its coorelating key (or track count)

and then look at the last list item in the lowest non-empty key and generate off that

rinse and repeat until values is empty

[jovial heath]

So that would be like the first message I linked to

[fervent tusk]

correct

[jovial heath]

Except you're looking to use lists instead of dicts

[fervent tusk]

you want to use nested dicts?

[jovial heath]

Sorry, autocorrect

Lists instead of deques

[fervent tusk]

yeah, i wanna know how deques would be different then

i dont see a reason why both side of the deque are needed, we're only looking to generate off the lowest track count so the branch doesnt really matter

[jovial heath]

Yeah I'm not sure whether in practice it does matter, but in terms of saying it's a bucketed breadth first search, using a deque makes it a more pure implementation, which just makes it easier to talk about and reason about if you've seen other stuff using a similar algorithm

Which means that when you see how it performs it's easier to say "well based on the performance profile, it might be good to switch to this other named algorithm"

[fervent tusk]

i see

[jovial heath]

Like, on a case by case basis is kinda arbitrary

But when you see enough stuff then being able to identify with a label makes it easier to spot patterns in where to look next

[fervent tusk]

as long as deques arent slower i suppose i can use them if that makes it easier to read

[fervent tusk]

this works

im wondering whether i should:
make a new deque for a key if it isnt there (like you did)
or have all the keys but empty ones are empty deques

im thinking if i did the second option id need another variable that just marks the lowest non-empty value (which could be changed when adding/popping deque entries)

first option is probably better

[jovial heath]

You can use setdefault, but the argument is eagerly assigned so you would create a new deque whether there's already a value or not

I wouldn't know without testing which way is cheaper to execute

[fervent tusk]

itd be harder to tell if the dict is empty if all the values were empty deques

[jovial heath]

And yeah, you could definitely go through range(initial_state.available, -1, -1)

(range is a half-open set so the second argument isn't included, and the third argument is the step, i.e what's added on each iteration)

[fervent tusk]

unnecessary for loops are slow

i like the first option

only issue i have with the code you sent is it fully does one track queue before moving onto the next unless i missed something

i could add a break if a new one found is better than the current best but then that requires another if statement for whether or not the key should be removed (it could be empty or it couldnt)

[jovial heath]

I doubt it's any slower than the while loop combined with max(keys) that it would replace though. Possibly faster without the max(keys) going on

[fervent tusk]

maybe

once i finish the first option ill make both and check runtimes

[jovial heath]

You'd also be about to remove the removal of the key

In fact

Iterating over a dict is ordered by key creation, so you should be able to do states = {k: deque() for k in range(initial.available, -1, -1) and then iterate through states.values()

[fervent tusk]

🎉 🎉 🎉 🎉

its faster im happy

it wastes a little time after the solution is found so i can probably change it so the program stops immediately when a solution is found

time to try option 2

[jovial heath]

Guessing there's been a fair drop in the number of iterations because it's not redoing work?

If not, we probably need to look at eq and hash methods to check they're correct

[fervent tusk]

im going to compare the non-obj generation with option 1 and option 2 once im done with all of them

the bucket sort is only useful on levels that have more than 1 solution though, it'll tend to be slower on levels with 1 i think

since the original generation searched the tree postorder it depends on where the solution is for it to be faster

vs bucket sort which is preorder (i think)

[jovial heath]

For levels with only 1 solution then I'm guessing the solution has no pieces remaining, in which case yeah, it'll search most of the possibility space before finding a solution

Which is close to the worst-case performance

[fervent tusk]

i mean we could technically add a button on the website that changes the generation

but thats a bad solution

unless you have any ideas

[jovial heath]

As in switches algorithm?

[fervent tusk]

the user can choose what generation method they want to use for solving the board at the start

[jovial heath]

Can do, I guess it depends what the goal of the project is

I would be very interested in seeing how something like A* works, and once things are refactored a little, implementing that shouldn't be too hard

[fervent tusk]

i just had a thought

for option 2 we can actually pop off the highest keyed empty value because the track count cant be increased

so that way you can get the highest count really easily because its just the first dict key

[jovial heath]

We could, but then we have to check if the available tracks is the current number, whereas if you leave the item in the dict you can just state_deque[new_state.available].appendright(new_state) and save a check

[fervent tusk]

option 1 vs option 2 on 3-10C

ran both a couple times to get the best result

probably need to do a longer level to see more variance

[jovial heath]

For that second option can you try doing for queue in states.values() : with while queue: inside, with your for loop nested in that, and then you can get rid of the while not ... part?

[fervent tusk]

ran it 3 times

[jovial heath]

Huh, I guess the extra level of nesting cancelled out not having to manually manage the loop

Oh wait, the if it's superfluous there too

Doubt it makes that much difference though

[fervent tusk]

gonna try it on a world 4 level with extras

i think 4-9b should work

ill add yours as option 3

[jovial heath]

The execution flow should be about the same between what I was suggesting and your option 2, so if the overhead of an extra level of looping is making a 10% regression (which surprised me) then it's probably not worth including. Unless there's something external to the program affecting performance

I wonder also what adding if state.solved: return state would do for the performance too

[fervent tusk]

oh yeah i forgot about the extra time

i could put a quit(0) and move all the code for ending into the generation function where it prints 'found a new minimum solution!' but im not sure if thats a good idea

what if i yield the state in the 'found a new minimum solution' section and then add a bool to the state saying whether its solved or not

i could also set a global variable and check if its true in the tco code but thats messy

[jovial heath]

Hmm, I feel like it should go on the state object, have you pushed your code recently?

[fervent tusk]

i pushed it once i finished turning gen into objects but it doesnt have the new tco code

[jovial heath]

I'm take a look, because it's the gen function using State objects I wanted to look at

Ah looks like this is before that https://github.com/FoxtrotOnce/RailboundSolver/blob/main/algo%2Fmain.py

[fervent tusk]

huh guess i didnt

ill push it then

should be there now

im just gonna make the minimum solution section of the code yield the solved state

and then tco checks if its solved and returns if it is

[fervent tusk]

i did all of that and theres still some delay after the solution is found

im not sure why

[jovial heath]

Why not set a break point at the solution being found and see what happens after you get there?

[fervent tusk]

i thought when you placed a break point it didnt allow you to step through the code after it

or it ended debugging

guess it does

it doesnt seem like it does anything extra, it just returns as though the generation function finished

oh actually

the extra time is probably because tco needs to generate all new states before it starts doing anything

and then it has to do the tco stuff on all the states before it reaches the solved one and returns

thus to make it instantaneous the minimum solution section in the generation function needs to stop tco and the program or tco needs to stop as soon as it receives the solved board

im really tempted to just put a quit(0) but i feel like thatd cause issues for thinh

[jovial heath]

So at main.py#L606 you don't yield anything, you just exit the generator, so the caller doesn't know you've found a solution, right?

[fervent tusk]

i changed that

[jovial heath]

Ok

[jovial heath]

And the final argument is whether it's solved?

[fervent tusk]

mhm

[fervent tusk]

if you look at the for loop with the new_state thing in it i put an if new_state.solved: return right after it

[jovial heath]

Hmm I mean I'd expect that to do it. I'll have to dig into what's going on at some point, but I'm away for the next week so I don't have my computer, just my phone

[fervent tusk]

opt 1 vs 2 vs 3

take the times as you will

(notice the delay as well)

[jovial heath]

Ok so option 3 isn't necessarily slower than option 2

Did you take out the superfluous if statement in option 3?

[fervent tusk]

im not sure what youre referring to so likely not

ah yeah i see why its useless no i didnt

was on both opt 2 and 3 though

[jovial heath]

Yeah it just might be slightly faster with

for queue in states.values():
    while queue:
        for new_state in func(queue.popleft()):
            states[new_state.available_tracks].append(new_state)```
because it removes an equality check

Unlikely to be that significant though

[fervent tusk]

yeahhhh

option 3 seems like the best so we'll use that one

before i do some timing comparisons between non obj gen and obj gen i wanna see if its possible to remove the delay

any thoughts @jovial heath

[jovial heath]

Not off the top of my head pretty sure I could get it with 5 minutes in the debugger, but that'll have to wait

[jovial heath]

See you've got me at it with the premature optimisations

[fervent tusk]

for comparisons i think ill just ignore the delays on obj gen assuming we'll remove it

im gonna make obj gen work up to world 6 too so we have more extra solution levels to compare

whats next?

[jovial heath]

At some point it would be good to stop unpacking state.params() and reference state attributes directly, which should make it more obvious which variables are local mutable state and which are being referenced from what's passed in - that should make it easier to reason about how to pull out some of the gen algorithm into state methods and make things a bit less... Spaghetti

But mainly we want the new algorithm working for all the levels that already worked, and then we can see how it does tackling some of the levels we couldn't solve before

Then once everything works and things are a bit easier to reason about then we should run it through a profiler and see where it's useful to optimise things

[fervent tusk]

do you want obj gen to work for every world

like up to 12

i suppose it had to eventually and it now is "eventually"

[fervent tusk]

main problem with this is that the attributes need to be modified before being packed back into a state

[jovial heath]

First thing is making sure on the levels with many solutions that it's definitely stopping after the first

[fervent tusk]

theres a world 8 level with 7 extra tracks i can run that

[jovial heath]

Sure, which is where you create local mutable state for that variable - the fact that it's not a reference to a state attribute is then the indication that it's something that's modified

[fervent tusk]

do you mean to say i should have a secondary state class thats mutable

[jovial heath]

Nah, just that things that haven't been altered are direct references to the state class

I mean you absolutely could have a mutable dataclass or something that will put out an immutable State object, but then you end up in the situation of deepcopying a load of stuff where you're not sure if it has changed

[fervent tusk]

im not sure i fully understand what youre asking, it seems like im already doing that?

i mean like the whole mutable state variables

because the variables unpacked by params are the mutable ones and the immutable ones are never referenced

[jovial heath]

Hmmm

I'm just trying to work out, for instance, whether bestBoard does anything in this algorithm

[fervent tusk]

well, some variables are not changed whatsoever but are unpacked into mutable ones and then packed back into immutable

[jovial heath]

Which is a bunch of unnecessary operations

[fervent tusk]

couldnt i just unpack the mutable ones then?

[jovial heath]

Yeah, but I'd only bother creating your mutable variables at the point where you're going to mutate them

I have an idea but it might be overly complex

Basically an object that you create by passing a state object to it, and then you update its attributes and it tracks what has changed and what hasn't, so it can then put out a new state object

I'm also not sure it makes things any easier to reason about

Most of what I'm concerned about with generate_tracks is that it's hard to see the woods for the trees. Having a bunch of state floating about that may or may not be meaningful obfuscates what's going on in a function that's already doing an ungodly amount

The fact that there might be performance benefits is more of a side benefit at this point

[fervent tusk]

maybe what we should do first is make it more readable then?

i remember you saying you wanted to organize stuff in classes etc

i need some instructions on what that's to look like

[jovial heath]

Urgh that might need to wait until I'm back

But basically start by breaking out some functions that generate_tracks orchestrates

Ideally you'll find in some of them that most of the state being acted on belongs to a single object - then it makes sense to make that function a method on that object's class

Most of it is likely on State but you may find that some makes sense on Grid or Tile

There might be some stuff that doesn't make sense to be on any of those, in which case they might be best as functions, or it might make sense to create a new class that manages the state they all share

Thinh might be able to offer some code examples and guidance before I'm home

[delicate nymph]

for more organize I think you should move more function to State class

I think 2 function is enough:

• one for simulation car movement and track that have effect
• one for generation possible state

[fervent tusk]

yeah i think im starting to understand

the function is already split into 3 sections so those would be the 3 functions i suppose

the generation function which is the big one might need sub functions inside it though if you want it to be more organized, not sure if thatll be messier though

[delicate nymph]

while queue:
        iteration += 1
        state = queue.pop()
        result = state.simulate()
        if result['status'] == "simulate_success":
            positions_to_check = result['positions_to_check']
            possible_states = state.generate_possible_states(positions_to_check)
            queue.extend(possible_states)
        if result['status'] == "goal_reached":
            if state.placed_tiles <= best_solution[1]:
                best_solution = (state, state.placed_tiles)
        # if results['status'] == 'crash' or collision or anything that not good, we ignore
    return best_solution

It gonna look like this

[delicate nymph]

so we gonna have a seperate function call solver and we can play with it

we can implement A* here too

you just replace the dequeue with a priority queue and a function to calculate how good a state is

[fervent tusk]

we already have a generation system in tco

[fervent tusk]

look at this

i dont want to implement any more generation methods until the program is finished unless the generation function will drastically improve the performance of the solver

[delicate nymph]

I see

By the way, I am still having trouble solving it; my code is unable to handle some edge cases. I really have to commend you for figuring this out

[fervent tusk]

thank you very much

im proud of it being able to do all the mechanics

i think you might be able to start working on the program so we can get ui up and running soon

[delicate nymph]

sure, update your new code, I'm gonna do some magic soon xD

[fervent tusk]

do you want to work on it right now?

ill push what i have, all thats different is the tco generation is changed

i still need to make the obj gen work for all worlds before i move onto organization though

i also need to make mod icons...

@delicate nymph is there some way you can make a palette for the mod types, like a white mask or something

because then i can just make white masks for all the mods and you can color them and overlay them on the tiles

ill actually just work on mod icons right now

[delicate nymph]

sure I already have it

[delicate nymph]

something like this right?

[fervent tusk]

that would be the mod types

ill finish the icons then explain it further

[fervent tusk]

i want you to add a palette maker on the website where people can pick what colors they want different mods to be

a mask like this that you can change the hue of and overlay it on the tiles

so basically for every tile:
place tile image -> place mod image -> place color shifted mod mask

[delicate nymph]

For more simple, you export it as svg

So I changed the color code and boom, magic happened! xD

[fervent tusk]

actually

can you do something like a green screen hue shift too?

[delicate nymph]

Yeah but svg is fast and easy

Since you already draw in figma

[fervent tusk]

for swapping tracks the rail needs to be color changed so id have to make 8 different 3-way masks or you could just color-shift the rail color

ill change the filetypes to svg

[delicate nymph]

So no need for you to draw many color

I just simply change the color in it

And the car too

I can change many number or color without draw it again with svg

[fervent tusk]

ill just send the images once i finish them and tell me what needs to be changed

[delicate nymph]

Sure

[fervent tusk]

ill make anything that gets color shifted 00ff00

...unless its something like swapping tracks in which case the color shift color would be e2f5ff

stations might be a problem because they have the same tile id as fences

actually you could just add it and itd have the same tile id as fences but a different img nvm

maybe?

[delicate nymph]

Yeah

But we need one more variable to know what is a fence and what is a station.

[fervent tusk]

you could store an invisible array specifically for the image i suppose

with different ids just for stations but thats not very resourceful

[delicate nymph]

Is it easy to add different tile id for the station?

I'm afraid we will have a complex thing xD

[fervent tusk]

hmmm i suppose i could

itd be the exact same as a fence though

[delicate nymph]

Excellent, please notify me when we have the SVG file. I must go now. Work call xD

[fervent tusk]

oh, and on most of the levels with stations the train is actually outside of the border so it wouldnt even be drawn (or itd be hard)

[fervent tusk]

pushed new svg files @delicate nymph

ill finish up other stuff like decoy images tomorrow

[delicate nymph]

Nice

Play with AI and this was so cool xD
https://claude.site/artifacts/6e2f8d7a-0cd3-415e-98d5-2da8f5cb1a20

[delicate nymph]

@fervent tusk can i get what your input data gonna look like?

I assuming something like this:

{
  "grid": [
    [5, 0, 0, 0, 5, 0, 5],
    [5, 0, 0, 0, 0, 0, 0],
    [0, 0, 5, 15, 0, 0, 0],
    [0, 6, 0, 15, 0, 0, 0],
    [16, 15, 15, 15, 0, 16, 0]
  ],
  "destination": [0, 0],
  "trains": [
    {
      "x": 4,
      "y": 0,
      "direction": 2,
      "order": 1
    },
    {
      "x": 6,
      "y": 0,
      "direction": 2,
      "order": 2
    },
    {
      "x": 2,
      "y": 2,
      "direction": 2,
      "order": 3
    }
  ],
  "max_tracks": 12,
  "numberLayer": [
    [0, 0, 0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0, 0, 0],
    [1, 0, 0, 0, 0, 1, 0]
  ]
}

[fervent tusk]

iirc there are 10 different params for a state

i need to make it so theres a mod type layer like you have too so probably 11

if you look at main.py on my github sparams should have all the data types fed into a state

[delicate nymph]

That a lot xd

I will try my best

[fervent tusk]

all 10 are based on the cars and board

i still need to modify a lot of stuff on the obj gen but yeah

[delicate nymph]

Oh wait, the input is 10

But the actual data is less right?

[fervent tusk]

wdym

the 2 most important variables are cars and board

how you have yours set up is about the same as mine but itd have to be converted when we merge

[delicate nymph]

board, cars, maxTracks, modifiers = lvl

Only 4 variables right?

So I only output these and everything match

[fervent tusk]

later levels have more but all variables simplify down to board and cars

[delicate nymph]

[fervent tusk]

...you need the max tracks and max semaphores too though but thats all

[delicate nymph]

Okay got it 👍

[fervent tusk]

keep in mind cars is both cars, decoys, and ncars and board is both board, mods, and eventually mod types when i add it

[fervent tusk]

@tranquil sinew im curious, how did you make the station models appear outside of the board when loading a level? im sure the board is a variable passed when the level is loaded so how is the station outside of it?

[delicate nymph]

So this should be cover all the case right?

levels["a-b"] = [
    np.array([[...],[...],[...]),  # Board layout
    [[0, 2, 1, 0]],                # Car positions
    3,                             # Move limit
    None,                          # Interactions
    None,                          # Decoys 
    1,                             # What is this number mean @.@
    [[0, 0, 1, 0]]                 # Non-car objects
]

[fervent tusk]

the last 2 are the amount of semaphores on the level and the numeral cars

thats the old data though, it gets converted into a car object and board object later on

eventually i have to convert levels_cars into the new data format but thats a very large task given it would be 2000 lines of changing arrays to objects

but yeah

[delicate nymph]

what is the new data format

I can help

[fervent tusk]

the old cars for the data just have [x, y, xvelo, yvelo] which would need to be fixed into [x, y, xvelo, yvelo, num, type]

then decoys and ncars arent needed

board and interactions should be converted into a board object before being put into the world dict too

same with cars once that the array is changed to [x,y,xvelo,yvelo,num,type]

[delicate nymph]

can you provide me some example? I think I can use some tool to do this fast

[fervent tusk]

ill use 12-10 as an example since it uses every parameter

this is the old data version:

board12_10 = np.asarray([[2, 0, 1, 0, 0, 0, 2, 0],
                         [0, 0, 1, 1, 0, 0,10,17],
                         [2, 0, 0, 0, 0, 0, 2, 0],
                         [2, 0, 0, 0, 1,17,10,21],
                         [7, 0, 0, 4, 4, 4, 2, 0],
                         [5, 0, 0, 0, 6,18,16, 0],
                         [2, 0, 0, 0, 0, 0, 2, 0],
                         [0, 0, 0, 0, 6, 0, 7, 3],
                         [2, 0,20, 0, 2, 0, 0, 0]])
ncars12_10 = [[0, 0, 0, 1]]
decoys12_10 = [[6, 0, 0, 1]]
cars12_10 = [[0, 8, 0, -1], [4, 8, 0, -1]]
interactions12_10 = np.asarray([[ 0, 0, 3, 0, 0, 0, 0, 0],
                                [ 0, 0, 2, 0, 0, 0, 0, 5],
                                [ 0, 0, 0, 0, 0, 0, 0, 0],
                                [21, 0, 0, 0, 8, 6,18, 0],
                                [ 0, 0, 0, 0, 0, 0, 0, 0],
                                [ 0, 0, 0, 0, 1, 6,24, 0],
                                [28, 0, 0, 0, 0, 0, 0, 0],
                                [ 0, 0, 0, 0, 0, 0,10, 0],
                                [ 0, 0, 5, 0, 0, 0, 0, 0]])
world12 {
    ...
    "12-10": [board12_10, cars12_10, 26, interactions12_10, decoys12_10, 1, ncars12_10]
}

this would be the new data version:

board12_10 = np.asarray([[2, 0, 1, 0, 0, 0, 2, 0],
                         [0, 0, 1, 1, 0, 0,10,17],
                         [2, 0, 0, 0, 0, 0, 2, 0],
                         [2, 0, 0, 0, 1,17,10,21],
                         [7, 0, 0, 4, 4, 4, 2, 0],
                         [5, 0, 0, 0, 6,18,16, 0],
                         [2, 0, 0, 0, 0, 0, 2, 0],
                         [0, 0, 0, 0, 6, 0, 7, 3],
                         [2, 0,20, 0, 2, 0, 0, 0]])
interactions12_10 = np.asarray([[ 0, 0, 3, 0, 0, 0, 0, 0],
                                [ 0, 0, 2, 0, 0, 0, 0, 5],
                                [ 0, 0, 0, 0, 0, 0, 0, 0],
                                [21, 0, 0, 0, 8, 6,18, 0],
                                [ 0, 0, 0, 0, 0, 0, 0, 0],
                                [ 0, 0, 0, 0, 1, 6,24, 0],
                                [28, 0, 0, 0, 0, 0, 0, 0],
                                [ 0, 0, 0, 0, 0, 0,10, 0],
                                [ 0, 0, 5, 0, 0, 0, 0, 0]])
board12_10 = rb.Board.from_numpy(board12_10, interactions12_10)
cars12_10 = (rb.Car(0,8,0,-1,0,rb.CarType.NORMAL),
             rb.Car(4,8,0,-1,1,rb.CarType.NORMAL),
             rb.Car(6,0,0,1,0,rb.CarType.DECOY),
             rb.Car(0,0,0,1,0,rb.CarType.NUMERAL))
world12 {
    ...
    "12-10": [board12_10, cars12_10, 26, 1]
}

youd have to do this for all 233 levels

rb is if you had import classes as rb at the top

[delicate nymph]

got it

I gonna make some function to save and load that to json. Since the UI cant access directly to python code ( different language x.x )

[fervent tusk]

youre going to make all the data json?

[delicate nymph]

maybe

put it in a single file

so you can do something like

world12 = load_world('12')

Ok here what I got

Look like we dont have tile with id 23 xD

Error converting level 8-8A: 23 is not a valid TrackName

I need you double check some level @fervent tusk

[fervent tusk]

    if ncars:
        for i, ncar in enumerate(ncars):
            cars.append(rb.Car(*ncar, 0, rb.CarType.NUMERAL))

should be

    if ncars:
        for i, ncar in enumerate(ncars):
            cars.append(rb.Car(*ncar, i, rb.CarType.NUMERAL))

the item before the type is the car num

[delicate nymph]

oh mb

[fervent tusk]

it doesnt matter for decoys though... i dont think at least

it shouldnt

[tranquil sinew]

you're correct - the train station tile actually contains just the piece of tracks that's visible on the grid

the model of the station is attached to this tile on one of the sides; if the model is present on the grid we mark it as an obstacle tile (so that you can't drag tracks through it)

but if we spawn the station next to the edge of the grid, the model can just be displayed outside of the grid

the model (building) is pretty much just a decoration, what matters for the in-game logic are the train tracks in front of it 🙂

[fervent tusk]

so the track that looks like a permanent tile (the train station) is just a model spanning 2 tracks

and you overlay a fence on any tile that the station model overlaps with?

[tranquil sinew]

yeah, but we don't spawn any other models under the building, just mark it as an obstacle

[fervent tusk]

huh, interesting way to do it

guess we still have the problem of doing it our way though lol oh well

thinh is there any way you can make the station appear out of the grid or is that too complex

[delicate nymph]

on render it simple xD. I am going to look for two tiles to the left and right. If there are any obstacles or position outside of the map, the station should be there.

[fervent tusk]

hmm

the problem with that is if theres 2 fences adjacent to the station mod you need to know which is a station decoration and which is a fence

[delicate nymph]

I think any of it is okay

[fervent tusk]

should i still make an extra track id for stations

[delicate nymph]

you should keep the logic as simple as possible, on the render it quite simple and we can use some trick

[fervent tusk]

alright i wont then

oh, and im not sure why the track on 8-8A is 23

im looking into that right now

im thinking i forgot to convert a tunnel id into a new id when i did something

[delicate nymph]

do you think that code is correct, I'm gonna do the json soon

maybe we should have the test case now xD

[fervent tusk]

if this track id is wrong im skeptical if all the tunnel ids are correct

i thought i converted them correctly

yeah, its supposed to be a tunnel that i didnt convert right

[delicate nymph]

Therefore, we will save one extra variable for each level that contains the true solution or a list of all possible solutions. Then, whenever we make a modification, we run the solver to see if the level can still be solved.

[fervent tusk]

the 21 on 8-8a's board should be a 17 and the 23 should be an 18

just change those

[delicate nymph]

it work now

[fervent tusk]

what are you talking about?

[delicate nymph]

sorry, I use google translate xd.
I mean we should have a test function

for example when we switch from the old data format to new data format. The algorithm still work

[fervent tusk]

i dont think a test function is necessary

my code switches the old data into the new format when its run anyways, i just dont have the data on the file saved as the new format

[delicate nymph]

sure, is my code is working then I continue to do the json thing

I think I can draw with this now

[fervent tusk]

i would advise against saving the board as tiles but that works
(if you want to make it editable at least)

[delicate nymph]

I will fix that later but this format is easy for me to draw for now

[jovial heath]

Surprised you're not just nesting the tiles dicts like

{"tiles":
  0: {0: {"track": 1, "mod": 0}},
  1: {0: {"track": 0, "mod": 0}},
  2: {0: {"track": 0, "mod": 0}},
  3: {0: {"track": 0, "mod": 0}},
  4: {0: {"track": 3, "mod": 0}}
}

[fervent tusk]

the code already used numpy before so saving the key as a tuple makes conversion a lot easier

also you dont need to key it twice which imo is more confusing

[jovial heath]

The problem is the lack of json support for tuples, so you either have to parse it or eval - the former is slow and the latter is ill advised

[delicate nymph]

this gonna too hard to read xD

btw that just a temporary

[jovial heath]

Although I guess the format is simple enough for a regex

"((\d), (\d))"

I don't think it's possible to have anything bigger than 10x10, is it?

[fervent tusk]

biggest boards are sandbox which is 9x9

actually there are some with 10 on the x nvm

[jovial heath]

Although I guess also just looking at (int(key[1]), int(key[4])) also works if none of the indices are greater than 9

Otherwise a regex is the most sensible way to go about it

[fervent tusk]

trying to implement stations into obj gen and ive been stuck for the longest time

finally realized its because it hashes it twice, not sure how to go about fixing it

adding station stalled as a hash variable doesnt seem like the right decision

[fervent tusk]

i think i can just avoid saving the hash for the state if any stations are stalled actually

because the state directly after it will have the same car as if it were stalled

blahhh logic it should probably work

it works for now so im just gonna leave it at that

[jovial heath]

I'm back, got things to do this weekend, but wanted to make the thread reopen so I can find it more easily when I have time to look at things on my PC

[delicate nymph]

Me too, I’m on vacation

[jovial heath]

That's an interesting new screen handle, I'll make sure not to blink

[fervent tusk]

ive been knees deep in solving all the easy leetcode problems so whenever you guys want to work on it again lmk

i think the main problem and why i stopped temporarily is because in world 6 the program raises an error on some of the levels because it continues generating after the solved path is found

which is already a problem we had to solve but now it needs to be fixed to keep making obj gen work for other worlds

dunno how to fix

[harsh cipher]

I started working on my own Solver in Kotlin. You can find it on GitHub here. I'm adding some UI to help me debug the solving process. The solver can deal with levels up to 1-13 so far.

[fervent tusk]

I think im going to start working on this again sometime soon

been feeling like taking it on again recently

most importantly, i want to add documentation because ive been programming another project and i realize how important readability is for both the reader and the team of a project

i want to hopefully get this whole thing ready for when the railbound level editor comes out.

stay tuned?

It seems Al has left the server and i cannot contact him on discord, but ill see if i can get him back here because he was a massive help to me.

[fervent tusk]

ok, to start working on this again i need to figure out what on earth i was doing initially...

i curse myself for not having done any documentation, everything is a mess right now, but the code is at least still somewhat readable

so the first thing in order for me is to understand how everything works again, and then start documenting it all to avoid future mishaps, and for better clarity for everyone

after i finish documentation, im going to thoroughly analyze every choice i made for the solver to see if i can optimize it somehow

i think the end goal i want is to have this project be converted to javascript so it can be run on the web and be faster than python, but i dont know javascript that well and ive heard its quite notorious for being difficult to bugfix

i would like to write it in c++, though i hardly know that better than javascript, because c++ would be the fastest. however, javascript would be needed for a website version of the solver.

i need to understand this all first though.

it seems i did add comments explaining everything, which does a fine job for documentation itself actually.

i will be sure to be thorough when adding new things this time around though

[delicate nymph]

do you need help? I guess I'm have some free time the next month 😄

[fervent tusk]

yup! you helped a lot the last time, plus youre good at doing the gui with javascript

i need someone else to tell me how understandable the code is and how javascript would tie into everything as well which i believe you can do

plus, you likely know more about coding than i do

im going to try and contact Al as well

[jovial heath]

👋

You might want to look at https://anvil.works/. You could get the existing solution working in the browser and then gradually pull functionality into JavaScript a bit at a time

It's not a framework I've used personally but I have friends who are contractors who rave about it

[fervent tusk]

it looks very useful for using both languages at the same time, ill definitely try it when we need to convert the language

[jovial heath]

Yeah, it just saves the step where you end up having to do everything before you can figure out if the new version works, because you can have something that works every step of the way

If you really want to get into things properly I'd suggest looking into testing frameworks, but I think that might be too big a diversion for you at this point. If I become any less busy I might look at writing some automated tests for you, but at this point I'm unlikely to find time for more than advice

[fervent tusk]

doing code in segments should be enough to figure out whats not working for js i think

i will probably be proved wrong

it only needs to be converted once though

also, i recall that we were trying to make everything hashable to make duplicate boards not happen when we left off, but i also remember saying that it wouldnt have a large impact and that we would have to use buckets for hash collisions, as unlikely as they would be

when we left off stuff was in the process of being converted to classes which was a good idea, but i dunno if hashing is worth it still

[jovial heath]

Yeah I'm less sure of how JavaScript internals work with that stuff, I'd need to look into it

I did a lot of full stack stuff almost a decade ago but all the complicated stuff was on the server in python or .Net while the JavaScript stuff was pretty simple, so I have no idea how things scale

[fervent tusk]

im a little worried the website version wont run that well for the user since its javascript thats being run on their computer(?) and some of the levels take multiple hours. that would still be shortened to like 30 minutes, which isnt exactly friendly to the user i think

would that mean they have to keep the page open for 30 minutes? would it freeze the page?

ideally i want them to still be able to use the browser and the page and just have a progress bar or something along with some time measurement

i hope it scales well

i looked back at our chats about hashes (i love that all of this is in discord, history is so convenient) and it seems that the reasoning was to make it faster via less duplicate boards

the copies required for making changes to the immutable data that was being hashed made the program ridiculously slow versus not hashing, though

not hashing had a LOT more iterations to process though, even if it was faster

my thought is that hashing is unnecessary because the hoops that need to jumped through to work with the immutable data make it take too long, and i want to manually find and eliminate duplicate boards myself because they should be removed from even being generated by the algorithm, i shouldnt have to remove them after all the checks because the hash found a flaw with the algorithm. there should be no flaws with the algorithm

the only use hashing has in my eyes is for finding duplicate boards, which i can then look at and figure out why a duplicate was able to be made, and then stop it from being generated again

[delicate nymph]

do you want to try "test driven development"?

[fervent tusk]

have not heard of that before

[delicate nymph]

It's like you write the test first before write a function

For example, consider a function like add(x,y) designed to sum two numbers. In a test-driven development approach, you would write the test for that function first. For instance, you might write an assertion such as assert add(1,2) == 3, assert add(4,5) == 9....

[fervent tusk]

so would you test unique cases of the add function until satisfied, and then implement it into the greater program (like a calculator)?

[delicate nymph]

Precisely. You would anticipate various scenarios for the function, including edge cases and potential error conditions. While coding, it's easy to overlook these specific situations. However, with tests already defined, running them will quickly reveal if your function has issues. This methodology is prevalent in competitive programming, where platforms like Codeforces or LeetCode present problems with a predefined set of test cases, expecting your code to pass every one.

[fervent tusk]

yeah when i understood the reasoning it immediately reminded me of leetcode

i hadnt thought of a development approach prior to this, it seems smart

[delicate nymph]

It's mostly 90% of what real world developers do 🤓 .

[fervent tusk]

i think how ive been doing the code is similar to that kinda. whenever i work on some part of the code, it always needs to succeed in one of railbound's worlds (like world 5 or 6 or 3) before i can move on with development. the levels themselves already accomplish that well though

for a puzzle game, each level needs to be unique, so i think we have our work cut out for us

[delicate nymph]

Yup! We can do simple tests per level. But for more control and detail, we can test deeper, at the function level. Like a function that checks which tracks connect to a given track, and from which direction.

[fervent tusk]

that is true but it would require more to be implemented simply for debugging

i usually prefer print statements to debug specifically what im looking for, but i dunno how the pros do it

[delicate nymph]

You don't need to change anything; just import your function and test it in another file. You can look up pytest

the test is optional; it will not populate your current code, just a addition

[fervent tusk]

interesting

[delicate nymph]

It is very fun and satisfying to watch the test. Here I'm trying to implement the track class with OOP and test it:

[fervent tusk]

i will try it out when im refactoring stuff i think

cool

[delicate nymph]

When I code a new product in my company. usually there is a whole team just to write the test before start a product; that team never join to code the product; they just write test xD

[fervent tusk]

thats pretty neat actually

[delicate nymph]

But with the AI, they write more tests now. And they did it fast.

[fervent tusk]

AI will definitely change programming

i dont think its capable of creating projects as in-depth as this one though, and it definitely cant bugfix that well

its very nice for smaller and tedious work like that though. sucks for the people who write the tests though, but maybe its also a help? i cant speak for them

[delicate nymph]

Yeah, it helps boost the performance and save time, but sometimes it takes the fun of programming away

[fervent tusk]

use in moderation is how i see it. the main reasons solving problems like that or coming up with unique stuff is to work your brain, and by letting AI do it you dont get the same enjoyment or learning from it. asking AI about more difficult solutions is a massive help though, especially for tedious ones. it gives really, really good insight, at the cost of your own thought

[delicate nymph]

I remember when I first learned to program; watching Coding Challenges from The Coding Train is very fun.

https://www.youtube.com/playlist?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH

[fervent tusk]

this is really cool, is it for javascript?

[delicate nymph]

Yeah, it's good for common knowledge like economics, definitions... I used it every day for these knowledge. But when programming, it not fun . Maybe I'm coding for too long haha

[delicate nymph]

at first it the Processing, it like Java. But he change to JavaScript so more people can use, and can share on web too

[fervent tusk]

would be useful

[delicate nymph]

the natural of code very hit me

it beautiful

[fervent tusk]

message for thought: im not sure where to start off on refactoring the code. theres 2 python files, main.py and no_hashing.py (iirc) and only no_hashing.py works, while main.py seems to be in a dysfunctional state. im tempted to work on main.py since it already has some OOP implementation, but i dont want to work with a system that needs to be fixed. on the other hand, no_hashing.py already works, but i would have to redo OOP implementation

i really wish i hadnt used 2 files

i think i want to restart with no_hashing.py and copy over what was in main.py to it while making sure it is still functional

there are some things i also want to refactor about the code that are extremely tedious, like:

• how levels are represented
  • i want 3 copies of the board to represent a level now, before it was only 2 (tracks and mods), but i want tracks, mods, and then the types for the mods. the old system only allows for up to 3 or 4 types of any mod, but using 3 layers would allow for an unlimited amount which is ideal.
• car x/y stuff
  • at some point in development i made the cars have their x and y positions be opposite of how you would access an array. arrays are usually formatted such that you access them by doing array[y][x] but the cars have the x and y swapped, so its inconsistent with everything else and really confusing.
• make everything more readable
  • very vague, but im aware this is a difficulty with the code that everyone struggles with. im not sure how to tackle it since the inner workings of the code are so complex

i think step 1 will be for me to change everything to be OOP in no_hashing.py, and delete main.py

im gonna sleep now. im glad youre still able to work with me despite the 12-hour timezone difference thinh

working on this project is a good experience for me

[delicate nymph]

g9

[jovial heath]

Some changes would likely need to be made to be able to test different functionality separately, but the good part of that is that by making your code easier to test you make it easier to understand and refactor, and make it easier to extract functionality, especially if you're moving things out into a different language or something (like moving stuff out into JavaScript)

(I'm more used to that in terms of splitting up a monolith into micro services)

[fervent tusk]

Started doing some of the OOP changes. What are your guys' thoughts on how far I should go with it?

Here are some of the changes, I dunno how much should get converted to objects instead of just using numbers:

• Board uses TrackName (Enum) objects instead of track indices
• ALL references of a track by index have been changed to TrackName. For example, something like abc = 0 now becomes abc = TrackName.EMPTY throughout the whole file.
• Direction still uses numbers (0=left, 1=right, 2=down, 3=up)
• Offsets/velocities still use numbers ((0, 1), (-1, 0), etc.)

im kinda iffy on the board (a numpy array) using TrackName objects instead of numbers because it needs a function to print it values to see it properly, and the board itself is harder to read kinda? I think its for the better though because otherwise it causes conflicts with figuring out when a track is a number or a TrackName object

the thing im on the fence about the most is the direction, because leaving it as a number feels fine and is still pretty readable, but all of the other numbers for the most part in the project are now objects. so do i make a direction Enum class to make it more obvious that 0=left 1=right... or do i just leave it as a number?

i think the velocities should stay a number because they need to be used as numbers for indexing the board anyways.

im tempted to make a new direction Enum class but I dont want to go overboard with OOP programming, because I feel like with too many objects the code just gets a lot longer and too verbose

im gonna go with making a new direction Enum class i think

it will be more clear than the numbers, but a little verbose i guess

[jovial heath]

I support going for enums because you can check types, rather than numbers could be anything

I'm kinda pondering whether the web version would be better in typescript rather than vanilla JavaScript given how big and complex a program it is

[fervent tusk]

im worried itll freeze the page while it runs

i want it to run in the background while the page is open

also, i just realized that saving and loading solutions and puzzles would be really good. like, if someone wants the solution for a complex level they made that takes 30 minutes to solve, they have to remake the level and wait another 30 minutes if they close the page. saving would make the experience a lot better. can we use cookies at all for it?

i think ill host the webpage on github as well, ive heard you get to make one for free with each account. if we need cookies, i hope github allows you to use them for your webpage.

[fervent tusk]

how is typescript different from javascript?

[jovial heath]

What you'll probably want to do is write the solver around an event system, and have the UI update based on those events, dropping them when it can't render fast enough, rather than trying to do all the work before showing anything

[jovial heath]

It's a language that's fully compatible with JavaScript but allows you to have a comprehensive type system where you can be sure a given function call is working with the expected types

https://stackoverflow.com/questions/12694530/what-is-typescript-and-why-should-i-use-it-instead-of-javascript/35048303#35048303

[fervent tusk]

would cookies work for saving/loading?

[jovial heath]

https://javascript.info/indexeddb

[fervent tusk]

i see

very convenient

i see the vision!!

typescript seems like better javascript, i like the additions it has. seems better for this project i think unless its a little slower, but i dont see that being the case

will have to see when we get there

[jovial heath]

It shouldn't be slower, but if it is, it's a superset so we could do the inner loop in plain js if it's important

[fervent tusk]

the ability to swap from js to ts will be very convenient at least

writing everything in js first would be smart before doing ts since you wouldnt be able to convert it back

[jovial heath]

Nah, ts compiles to js

So converting to js is zero effort, versus the other direction

[fervent tusk]

wouldnt using ts features mean js cant run it?

[jovial heath]

It means you need to compile before js can run it

[fervent tusk]

ts is a superset of js so it includes js but js doesnt include all of ts right

ah

so ts would compile it into a runnable version for both then

compiling as in... like a .exe?

[jovial heath]

Anything you do in the browser is going to compile to js

Nope, spits out a .js file

[fervent tusk]

but the code is in a .ts file

thats really interesting

so write it in ts before js then

sounds good

[jovial heath]

You might want to have an experiment with it

https://www.typescripttutorial.net/ isn't terrible

You'll probably also need to look stuff up on MDN

[delicate nymph]

I think you don't need a full 2d board map, a dict with (x,y) as key should work

[delicate nymph]

local storage may better than cookie, more storage size

[jovial heath]

Yeah I was pointing at https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API but it very much depends how storage is being used

[fervent tusk]

im not sure if i should use a 2d numpy array or a dict

im pretty sure there are some numpy operations that make it a lot smoother but i could be mistaken

it really depends on which is better for copying

copying boards is the process that takes the most time during generation so its important to make sure its fast

another issue is that a dict would be harder to read than a numpy array because the elements arent displayed as a grid, theyre displayed as keys/entries

that can be fixed with a function that prints it as an array though

on paper, i think a dict would be faster, so ill try it out and see if i can implement it instead of a numpy array

on that note, i can put the 3 layers i want (tracks, mods, mod types) all in the same entry. i couldve also done that for numpy, but then they would be indexed by numbers which is vague as Al said

hm. heres another important thing then: do i put all 3 level layers in one dict so each entry would be like python3 (1, 3): {tracks: T.HORIZONTAL_TRACK, mods: M.OPEN_GATE, modtype: 2} or do i use 3 dicts of the board like python3 tracks = {(1, 3): T.HORIZONTAL_TRACK, ...} mods = {(1, 3): M.OPEN_GATE, ...} modtypes = {(1, 3): 2, ...}

i think that having 3 dicts is the best since theyre separated, but 1 dict also works. whichever one copies faster is the most important factor actually, so 1 dict would probably be the best option

will have to see

[jovial heath]

Dicts will be easier to translate to typescript without significant performance reduction, if that's a factor worth considering

[delicate nymph]

This is me trying branch and bound again

(me) PS D:\Code\me\railbound-solver> python .\solver.py
Solving level: 1-1
Total state visits: 4
Solving level: 1-2
Total state visits: 16
Solving level: 1-3
Total state visits: 3
Solving level: 1-10
Total state visits: 1377

(me) PS D:\Code\me\railbound-solver> python .\solver.py
Solving level: 1-1
Total state visits: 4
Solving level: 1-2
Total state visits: 13
Solving level: 1-3
Total state visits: 3
Solving level: 1-10
Total state visits: 27
Solving level: 1-11A
Total state visits: 1843552

(me) PS D:\Code\me\railbound-solver> python .\solver.py
Solving level: 1-1
Total state visits: 4
Solving level: 1-2
Total state visits: 11
Solving level: 1-3
Total state visits: 3
Solving level: 1-10
Total state visits: 27
Solving level: 1-11A
Total state visits: 319108

[fervent tusk]

the visits for 1-11A... yeesh

[delicate nymph]

what is your Total state?

I dont know if this the best yet xD. I'm try to implement A*

[fervent tusk]

my solver does 1-11A in 2946 iterations

im so annoyed that i made the cars have reversed positions from everything else

there are so many vectors used and i cant tell which are reversed and which arent

[fervent tusk]

i finished turning all of the track stuff into objects, it is certainly more readable

im not too sure about the speed difference because sometimes the OOP version is longer than the non-OOP version, but the non-OOP version also takes longer sometimes (long enough to not be a difference due to runtime)

i think it should be fine though, they are both about the same speed

sweet!

im gonna turn the cars into objects now and fix the godforsaken positions being reversed

[delicate nymph]

Yay, I match your speed

+--------+-------+-----------+--------------+
| Status | Level | Total (s) | State Visits |
+--------+-------+-----------+--------------+
|   ✅   | 1-11  | 0.001111  |      9       |
|   ✅   | 1-12A | 0.001371  |      13      |
|   ✅   |  1-2  | 0.002738  |      13      |
|   ✅   |  1-3  | 0.000320  |      2       |
|   ✅   | 1-14A | 0.559013  |     1378     |
|   ✅   | 1-10  | 0.005404  |      30      |
|   ✅   | 1-11A | 0.444915  |     2186     |
|   ✅   |  1-4  | 0.002215  |      11      |
|   ✅   | 1-13A | 1.076108  |     4359     |
|   ✅   |  1-8  | 0.002414  |      6       |
|   ✅   |  1-9  | 0.002537  |      6       |
|   ✅   | 1-15A | 3.778520  |    13280     |
|   ✅   |  1-5  | 0.003021  |      8       |
|   ✅   |  1-6  | 0.000748  |      4       |
|   ✅   | 1-15  | 0.033559  |     110      |
|   ✅   | 1-14  | 0.038569  |     190      |
|   ✅   | 1-11B | 0.040711  |      77      |
|   ✅   |  1-7  | 0.000922  |      4       |
|   ✅   | 1-13  | 0.035748  |     141      |
|   ✅   | 1-12  | 0.010959  |      26      |
|   ✅   |  1-1  | 0.000965  |      4       |
+--------+-------+-----------+--------------+

my code is OOP with basic tuple and dict (not optimize yet). Look like it not affect the speed too much

[fervent tusk]

1-12A seems to be missing from your list

how did you do the solving? the iterations for my solver is bigger for most of those levels besides 1-15A

your state visits might also be different than my iterations; i count a single iteration every time a new car is generated on a state

[delicate nymph]

I only use straight and curve track, only use the T track if needed

[delicate nymph]

I count a single iteration by everytime I change or place the tracks

[fervent tusk]

we count them about identically

the difference probably comes down to what tracks are generated in what order, then

i believe mine will do a horizontal track, then a bottom right turn, then a bottom left turn

yours might be different

regardless, nice work

out of curiosity, why are you making your own solver?

[delicate nymph]

I just started learning Rust and want to try its speed, but I'm not an expert at it yet, so I'm going to try to solve it in Python then convert it to Rust. And btw, last time I gave up because I was busy, and now I want to try again xD.

[fervent tusk]

ah. thats a good way to practice a language

cool

im assuming you skipped 1-12A because of the crashing? looping?

[delicate nymph]

I'm forgot to type it xD

Look like from level 3 they share a same logic for all component that have same color. I'm brainstorming to implement the data structure for it

[delicate nymph]

Here is level 2:

+--------+-------+------------+--------------+
| Status | Level | Total (s)  | State Visits |
+--------+-------+------------+--------------+
|   ✅   | 2-5A  |  0.041374  |     210      |
|   ✅   |  2-7  |  0.307561  |     1875     |
|   ✅   | 2-7A  |  3.549967  |     15806    |
|   ✅   | 2-3A  |  0.738352  |     3135     |
|   ✅   |  2-6  |  0.017345  |     44       |
|   ✅   |  2-1  |  0.001581  |     5        |
|   ✅   | 2-6A  |  4.062025  |     24435    |
|   ✅   | 2-4A  |  0.061040  |     291      |
|   ✅   |  2-3  |  0.021592  |     61       |
|   ✅   |  2-2  |  0.045854  |     229      |
|   ✅   |  2-9  |  1.508078  |     6446     |
|   ❌   | 2-3B  | 167.557851 |     501408   |
|   ✅   |  2-5  |  0.069696  |     247      |
|   ✅   |  2-4  |  0.042255  |     184      |
|   ✅   | 2-7B  | 17.136340  |     58773    |
|   ✅   |  2-8  |  0.237457  |     1274     |
+--------+-------+------------+--------------+

It looks like my algo gets stuck at this level since it only goes straight and curves (and if it by chance has a connection to another track, it becomes a T-track). It don't know it should place a T-track in the edge case in level 2-3B

[fervent tusk]

i remember having the exact same issue

[fervent tusk]

i just had a thought on how to visualize some of the generation options better: a flowchart

a lot of the generation is a bunch of if statements making sure a bunch of rules are met during generation, so a flowchart would be a good visual guide to see what the program is doing better

i will make sure to have that somewhere for documentation

[fervent tusk]

@delicate nymph could you continue working on the gui stuff some time?

it doesnt have to be immediate since we likely wont get to the web phase of the solver for another 3 weeks(?) but itd be nice to have it as completed as it can be

[delicate nymph]

Sure, what do you need?

[fervent tusk]

i believe where the gui left off was being able to place all the track icons which was really nice, but the mods need to be able to be placed as well, and i want them to be able to be color-coded to a palette depending on what number the mod type is (for example, switch 2 and gate 2 would share a palette)

let me know if i need to make any changes to the mod/track icons, they should all be svg and hopefully able to be hue-shifted

[delicate nymph]

Svg is good enough

[delicate nymph]

this is what i got so far

[delicate nymph]

Level 3:

+--------+-------+-----------+--------------+
| Status | Level | Total (s) | State Visits |
+--------+-------+-----------+--------------+
|   ✅   |  3-1  | 0.001167  |      6       |
|   ✅   |  3-2  | 0.371745  |     975      |
|   ✅   |  3-3  | 0.017609  |      47      |
|   ✅   | 3-3A  | 0.016594  |      72      |
|   ✅   |  3-4  | 0.629756  |     2357     |
|   ✅   |  3-5  | 0.008160  |      32      |
|   ❌   |  3-6  | 0.125230  |     365      |
|   ❌   |  3-7  | 0.095581  |     226      |
|   ❌   | 3-7A  | 0.142335  |     664      |
|   ✅   |  3-8  | 1.607092  |     3177     |
|   ❌   | 3-8A  | 0.066741  |     236      |
|   ✅   | 3-8B  | 0.083322  |     132      |
|   ✅   |  3-9  | 8.261910  |    15782     |
|   ❌   | 3-10  | 1.693720  |     7826     |
|   ✅   | 3-10A | 0.053111  |     253      |
|   ✅   | 3-10B | 1.342353  |     3175     |
|   ❌   | 3-10C | 58.819268 |    105604    |
|   ✅   | 3-11  | 0.051000  |     377      |
|   ✅   | 3-11A | 0.019758  |      89      |
|   ❌   | 3-11B | 0.267949  |     715      |
+--------+-------+-----------+--------------+

[delicate nymph]

I haven't solved this yet, but I will keep my current algorithm despite it missing some cases.

BTW, as long as I add more features, my code gets more spaghetti xD. Lucky that there are only 2 more new components left. Despite using OOP, it's still spaghetti but readable. So I guess when I'm finished, I will refactor the code once and for all and make everything standard and pluggable.

I have 2646 lines of code. include visualize, map editor tool xD

[fervent tusk]

every world gets harder than the last one for the algorithm... definitely difficult to get it 100% right

[fervent tusk]

this looks nice so far. the F icon shouldnt be necessary though, just place an ending track to find the ending. im also assuming permanent tracks can be placed

[jovial heath]

To be fair about ⅗ of that is the level editor, which isn't too bad

UI code always ends up bloated

[fervent tusk]

the only other thing i think it needs is palettes then, and making sure the stations render outside the grid. i would also prefer if the car icons were used to indicate where the cars are instead of the circle icons. the track underneath is going to get covered up anyways, so might as well make the car look nice.

good job

[delicate nymph]

Yes haha

[delicate nymph]

Yeah, im lazy to rotate the car image xD

Btw the beta level editor look very cool, maybe I should copy that

[fervent tusk]

just noticed that railbound's level editor has ending tracks that can face any direction so im gonna have to add some extra tracks

[fervent tusk]

just finished changing everything involving cars to objects

the board used to use ints but uses objects now, but the modifiers still use ints, so im going to change all of that to objects next

actually, im going to change both the board and mods to be dicts instead of arrays first

then i can convert mods to objects and add the 3rd mod num layer much easier

im excited to see how the speed changes

hopefully i dont get disappointed

[fervent tusk]

finished changing all 3 level layers to dicts, making mods objects, and adding the third mod_nums layer

the performance took a slight decrease after all of the oop changes, but it doesnt seem significant enough for me to sacrifice the readability for faster code

most levels generate about the same speed, oop is faster on some, but for levels that take longer to do it takes about x1.5 as long

there is also a slight discrepancy between the oop and non-oop version that i have yet to find, because i noticed that oop does 12-1A in 384,274 iterations while non-oop does it in 384,256 which is a little strange

it produces the exact same results for all of the other levels i looked over, so its probably some obscure edge case. im sure ill find it eventually

what to do next...

[fervent tusk]

i think theres a few things that need to be done before we can convert to java/typescript so im gonna try and figure out what those things are to minimize the js/ts bugfixing

i feel like there has to be at least one optimization that can still be made, and i also want the code to be even more readable. i made the big stuff like cars and the level layers objects, but theres also other arguments that could be objects for better readability

on the note of optimizations, i think one of the things ill do to try and find that is to try and make the level able to be represented as some hashable object. im not going to implement hashing as a part of the generation process, but use it to try and find duplicate boards and see if i can fix them

i also want to make a flowchart of the whole generation process because i think that will ultimately be the best way to understand how everything works

im going to change how levels are created in levels_cars.py to reflect the new system as well. i do have one thing im debating on that i want your guys' opinion on though.
for the new level creation, should the boards and mods be initialized as:

• numpy arrays with ints (like board1_1 = np.asarray([[1, 0, 0, 0, 3])) and then converted to the OOP version during generation. this version is the most concise and easiest for the eyes, but is completely different from how the boards are used during generation
• numpy arrays with OOP tracks/mods (like board1_1 = np.asarray([[T.HORIZONTAL_TRACK, T.EMPTY, T.EMPTY, T.EMPTY, T.CAR_ENDING_TRACK_RIGHT]) so the objects dont need to be converted (though the board needs to converted to a dict during generation still) this version compromises between the old and new systems. the array is easier to understand, but its arguably harder to read due to tracks/mods being long names instead of numbers)
• dicts with positions as the key and OOP tracks/mods as the values (like board1_1 = {(0, 0): T.HORIZONTAL_TRACK, (0, 1): T.EMPTY, (0, 2): T.EMPTY, (0, 3): T.EMPTY, (0, 4): T.CAR_ENDING_TRACK_RIGHT} so the board is accurate to what it looks like during generation and doesnt need to be converted. this version doesn't need to be converted and is accurate to the program, but it's incredibly hard to read.

or what else should i do?

i like the numpy array with ints the most. the levels arent going to be looked at often since they are really just needed to have the data be there for the solver, and someone can reference the numbers to the actual tracks if necessary to read the board. its also the easiest to see imo once you understand the numbers, and the numpy array and numbers makes it look like an actual grid, which the others dont really do.

the only big downside is that the numbers are separate from the objects, so if an object's number changes, all of the boards are screwed up. the numbers allow for more conciseness and grid readability though.

let me know what you guys think

[jovial heath]

I'd be tempted to go for one of two approaches:

1. Exactly as it's used
2. Json files that get converted

[fervent tusk]

i remember seeing how thinh did the levels as json files. given the options of having 1 json (all levels and everything in it), 12 jsons (levels for each world), or 200+ jsons (each individual level) i feel like 12 jsons would be ideal if we were to go that way

i dont really want to go for the "exactly as its used" route since its impossible to read, so having the object conversion happen during json conversion would be good

[fervent tusk]

the only thing then is i want the json files to work like option 2 here, where its an array with objects

i dunno json that well though so im not sure if the object and array thing is possible

[fervent tusk]

just checked, the objects would have to be serialized as dicts (and ultimately numbers) anyways so json actually doesnt seem to be an improvement to me

nvm itd be usable with both python and js

[fervent tusk]

im gonna store the level data in 12 or so json files and use the arrays + numbers stuff, then just convert it to the generation format along with converting the json to python/js/ts

seems like the best route now

[jovial heath]

I'd be tempted to use strings which are keys for something in both python and js, but I don't think the approach you go for has to be permanent. You could do whatever and then write a serialiser for a new format if that's preferred.

There's some cool stuff I can think of that you could do with one big json file, for instance, but I also don't think it's what you'd want to write by hand, and it also might not be what you'd want to store either

[fervent tusk]

i forgot about strings

strings and arrays seems the most logical then

i just want the data to be stored such that i wont have to make many tedious changes in the future

using strings as object names and arrays for readability should be good

the object names wont change which is good

...speaking of tedious changes, 12 json files would make it more difficult to change something

though 1 json is a little bloated but it might be for the best on second thought

hm

[jovial heath]

You can probably switch between in code, just pick one, revisit it later

[fervent tusk]

im gonna have to revisit it soon is the thing

later could mean when the javascript is all done though

i suppose i could just always change it

good to have a plan

[jovial heath]

Either way, changing how you serialise and deserialise is fairly simple

Sometimes no amount of thinking it through is worth as much as trying something and discovering the pitfalls for yourself, it's just a matter of judgment as to which is appropriate in a given scenario. There's not really any better way to learn that than experience

[fervent tusk]

true

[fervent tusk]

ive started working on making the flowchart. i suspect it will be very helpful for me, as it will help me understand the entire generation system again so i can teach you guys how it works better using the flowchart, and i should also be able to see optimizations better with better knowledge of the program

if you guys understand it well enough you should also be able to see some flaws or optimizations as well

thats the hope at least

[fervent tusk]

finished doing about a fifth of the flowchart. woohoo

[delicate nymph]

nice

[delicate nymph]

you can draw on the paper then send the image to chatgpt so it can draw it using mermaidjs xD

[fervent tusk]

speaking of other diagram programs, i wonder what would be best to display it

it could be cool to make it interactive with obsidian or something but i feel like i should just use an image

[fervent tusk]

a little under halfway done with the flowchart

the track selection part is pretty cool to look at

im putting a lot of annotations on some of the more confusing stuff to make it easier to understand

also noticed i forgot an arrow on the "decrease available_tracks by 1" box woops

heres how it looks so far (im going to make it look nicer than just white boxes when its finished)

let me know what you guys think so far and if i should change anything

[jovial heath]

A background might be nice, my phone is set to midnight mode so I can't see any of the lines and arrows, going to have a bit more of a peer at it for any feedback on the content

Yeah I think I need to be able to see the lines to have any useful feedback, it looks fairly clear with where I imagine the lines and arrows to go, but if what I'm imagining is wrong, there might be bits where clarification is useful

[fervent tusk]

will do

[jovial heath]

I'd imagine a light grey like the Discord light mode background would make both the black lines and white boxes pop? But that's only a suggestion in case you can't think of anything better

[fervent tusk]

im going to try to make it look like the editor up here

it exports differently though so ill just have to change some things

have you tried opening it in a browser? on my phone chrome gives it a light gray background and the arrows are visible

[jovial heath]

Ah that did help

So I'm guessing the (?) circles are where you still need to investigate what the code is doing?

Other than the lines drawing attention to that is much how I thought it was

Things like be slightly more legible if you were write

Decrease foo by 1
as
foo--
And similar with ++ for increment, but that's fairly minor
I wonder also whether using a different arrow style when branching would make it more obvious which code is getting run most, which would help for targeting where to spend effort on optimisation?

[fervent tusk]

its fairly easy to tell what code is getting run most based on what youre running imo

the (?) circles are going to be a connection to another < End loop? > like i did with the (A) circle, i just havent gotten to the end of the loop yet so i left it as a ?

similar to the (Z) circle at the start of the program

[fervent tusk]

as i go through the program i keep finding some minor changes that make it more readable or optimized which is nice

im starting to get to the part where crashes are handled and one of the flaws with the program i remember is that same-tile crashes messed up sometimes and i had a failsafe, but the failsafe shouldnt be needed so im looking into it now

turns out that i forgot to consider station stalled cars (though i considered crashed decoys and stalled cars) so sometimes it wouldnt tell that a station stalled car is in front of it due to how the possible crash poses are calculated

but its not that simple to fix

possible same-tile crash poses are calculated as the position of already-generated cars + crashed_decoys + stalled_cars, so i thought to just add station_stalled cars to the mix but the problem is that it cant be implemented as easily as stalled_cars was

stalled_cars is the position of all cars that are stalled before even pre-generation (due to the 7-3B edge case where crashes happen before gates/stalling) but i cant just put station_stalled there because its handled during post-generation

so i thought to move it to pre-generation since thats easy to do, but it still isnt working so im not sure what the issue still is

ah

same-tile poses checks the "already-generated" cars by doing cars_generated[:car_index], so it only considers them that way

however, by putting the station stalled generated cars in pre-generation, they are all in cars_generated before post_generation, but the poses doesnt check it because theyre after car_index

iirc i made it so it does the cars generated before it only because there shouldnt be any after, but there is now so i can change that i think

it still isnt working though

[jovial heath]

I think I know what you mean here, I was just thinking that if you're going to the effort of drawing a flow control diagram it's just as useful to be able to visually pick out the innermost loops and then check what's going on there as it is to be able to follow through the code and eventually note the innermost loops, so assuming draw.io still lets you style the lines like it used to, use, say, a double line for any step where it branches and a dashed line for any step where it returns

[fervent tusk]

so a dashed line if the result is a return (the branch is killed) right? i dont get wym by branching though

do you mean a double line for when multiple boxes go to the same thing?

im not gonna use a double line for if statements though if thats what you mean by branching

[jovial heath]

Yeah, for/while loops, generators etc

Essentially where you're saying running this step will cause multiple of the next step

[fervent tusk]

what line specifically would be a double line then

do you mean everything under a for loop should use double lines?

if so, everything would technically be double lines due to the "for every level" at the start but i suppose that doesnt have to be double lines but it would break the trend

[jovial heath]

Anything under a for. I'm guessing also stuff like "check each generable track"

[fervent tusk]

thats not a loop

just worded readably

[jovial heath]

How do you mean? It's a return to an outer loop?

'cause to me "each" implies branching

[fervent tusk]

"check each generatable track" = tracks_to_check = generatableTracks[car.direction][tile_ahead]

i have a note in the track selection section specifying that "check the track" means add it to tracks_to_check

[jovial heath]

So it's more "mark as generable"?

[fervent tusk]

not really

generatableTracks is a collection of tracks a car can generate depending on direction and the track ahead (also in the note on the flowchart), so the tracks to check just becomes whatever set of tracks in there

[jovial heath]

If it's that we're adding multiple items to a collection that's being looped over, then I'd be tempted to count that as branching as it's an optimisation to avoid increasing the stack, but it's logically equivalent to branching in a TCO runtime

Uh, Tail Call Optimised

[fervent tusk]

i think you misunderstand

[jovial heath]

I think I do

[fervent tusk]

for each car, tracks_to_check becomes something depending on what the car is what track is in front of it

and then it checks each of those tracks and adds them to generation

[jovial heath]

Throwing potential understandings at the wall until one's correct

[fervent tusk]

it does this for each car, so its not function-wide, its local to each car

almost everything done in generation is per-car, the only branching that ever happens is at the very end and thats simply getting the product of each possible combo of generated cars and tracks for that frame

the flowchart isnt complete so im assuming thats where the confusion comes from

[jovial heath]

So is it just that it's returning to the top-left box of that section?

[fervent tusk]

?

[jovial heath]

check the track in front of the car + each generable 3-way track depending on the track in front of the car

(minor spelling note that I think might be in the code too: the word is generable rather than generatable, which saves you a few characters everywhere it's written)

[fervent tusk]

did not know that, thanks

[fervent tusk]

this is just one of the outcomes of track selection. tracks_to_check could become any of the 7(?) options depending on the car's situation

accidental ping

[jovial heath]

It might be that both are valid US English, but the shorter one is valid in both US and UK English while the longer one gives me red squiggles every time I try to write it

[fervent tusk]

what are you confused by? your confusion is confusing me

[jovial heath]

I'm being unsure whether your spelling of generable is valid in US English as some dictionaries seem to have it while others don't. Ignore me

[fervent tusk]

no i mean everything else

you dont seem to understand track selection

[jovial heath]

Oh right. I guess I'm just unsure what's actually happening when you say "check each …", whether you're queuing multiple things or just saying "this is a valid place to generate from", in which case where does the control flow go next?

[jovial heath]

Because it seems to lead here, which goes nowhere

And I don't know where 'each' there is doing multiple things

[fervent tusk]

im assuming were both looking at this

the note should specify what "check each track" means, no?

[jovial heath]

So it's adding multiple potential next steps and then returning to the start of the loop?

[fervent tusk]

no, it doesnt return to the start of the loop

thats only for the ones with the (?)

otherwise it goes to the next step which is at the bottom

[jovial heath]

I hadn't understood that the note was a note, I thought it was an orphaned step that needed to be linked from somewhere

[fervent tusk]

i see

is there anything i should do to clarify its a note

[jovial heath]

Different shape or outline maybe?

[fervent tusk]

i thought about using a dashed outline since it uses a dashed line to highlight what the note is talking about anyways

[jovial heath]

I was zoomed in too close to see that line off the bottom

[jovial heath]

Sure that would make it visually different

Although I hadn't spotted those ones being dashed when I suggested dashed return lines. Maybe returns should be dot-and-dash or something

[fervent tusk]

maybe

other than that... does everything else make a lot of sense now about how the program works before track selection?

[jovial heath]

Yeah, I think you should say mark rather than check though

[fervent tusk]

will do

[jovial heath]

Just makes it clear you're not doing it yet

[fervent tusk]

its fun to look at all of the complexities i put into the program to make everything work perfectly, i love how simple it makes it seem

like the heatmap limits

[jovial heath]

So if you get a 'No' on "track to check is going to the wrong train?" Where does that go?

[fervent tusk]

havent gotten there yet

[jovial heath]

I'm guessing "tile in front of car is empty"

[fervent tusk]

the flowchart is a little under halfway done, there is much to add

[fervent tusk]

no

it goes to "cannot beat best tracks?" after that

and then crash handling

[jovial heath]

"track to check is going to the wrong train?" Might be a loop over the possible marked ones?

There was supposed to be more text at the start of that

[fervent tusk]

nope

[jovial heath]

I'm going to stop trying to understand until you've finished

[fervent tusk]

the line is specifically if (tracks_to_check[0] is T.car_ending_track and car.type is not NORMAL or tracks_to_check[0] T.ncar_ending_track is not NUMERAL)

so if the tile_ahead (track_to_check) is the wrong train for the car, kill the branch

[jovial heath]

Hopefully you can see which bits I'm finding confusing, and you can judge for yourself whether that confusion is likely to be lessened as it becomes more complete

[fervent tusk]

figured out what the issue with same-tile stuff was

stalling got moved from post-gen to pre-gen, and other gate stuff also happens in pre-gen

since all gates need to be handled before stalling, the program fails in situations where a car uses a gate before it should be able to

i think ill just add another "for each car" loop for doing stalling and stations alone and include it as part of pre-gen

that way same-tile crashes can be done easily

i was probably against it initially because it requires another loop over each car, but... stalling does it anyways so

also another loop wont impact performance that much now that i think about it again

the only issue is that if something that ends a car's generation happens before post-gen, i then have to go through ignoring the generation which is somewhat confusing but what isnt confusing about the program

basically a continue in one for-loop means i need to continue in all the others

oh well

[fervent tusk]

just had a thought. crashes could be handled by using more for-each-car loops, but every one i use makes the program slower i think

will see if i can go without them first

[fervent tusk]

found another issue with same-tile crashing, the failsafe cant realistically be removed

the program wont crash a car if it chooses to generate into another car (unless its a head-on collision) because it assumes the other car is going to generate out of the way to avoid the collision. however, decoys can choose to crash, making this collision actually happen

so the only realistic solution is to wait until all of the cars are generated and then check if the decoy is crashed, meaning the failsafe stays where it is

man

i keep thinking im finding optimizations and then i just realize that strange thing i did back then was smart. if only i documented it so i knew that...

[fervent tusk]

im just gonna revert the code to what it was before i tried to mess with stations. it works fine as-is and trying to "fix" one of the things just makes everything take longer

i would say that im going to try to not change the code any more and just finish the flowchart first but... optimizations

[fervent tusk]

got a little over halfway done today, spent way too long trying to fix the same-tile crash stuff, but at least i ended up figuring out how it worked better

track confirmation is next, then solving, branch creation, and results printing

[fervent tusk]

i just finished up track confirmation. semaphore gen is probably the hardest thing to explain out of the entire program, but i tried my best.

hopefully you guys understand it

[fervent tusk]

Finished doing the entire flowchart

here's the whole thing in all its glory

I decided against the return arrows because i feel like its pretty obvious, but i havent done the for-loop arrows since draw.io actually cant do double-lined arrows very well

theres one version that does it but without the arrows which is a no-go, and the other makes the arrows way too big so that there isnt even a line

[fervent tusk]

regardless, i want both of you to look in-depth at this chart and tell me if you understand what the code is doing, because that's the goal.

Ill be sure to put this flowchart somewhere on the github page as well

[delicate nymph]

Look cool

[fervent tusk]

i think the next step is to simplify some of the variables, and then ill quickly (hopefully) set up hashing to check for any extra duplicates

i also want to have a few hyperparameters to control generation better, especially for the web version, so i was thinking:

• heat_limit: the upper limit any heatmap limits can be, its currently 9 in the program.
• decoy_heat_limit: the upper limit any heatmap limits can be for decoys, since they operate differently. currently 15 in the program.
• gen_type: either BFS or DFS. the program currently uses DFS.

as for ending generation early with a max_iterations or max_time, i feel like just having a stop button on the website would be better

[fervent tusk]

oh, theres also one issue with the array to dict stuff that im a little worried about

heatmap_limits is the only numpy array in the program that utilizes numpy stuff to be faster so changing it to a dict would be devastating to the speed

like a lot of the indexing calculations across dimensions that numpy can do quickly will be annoying with a dict

[fervent tusk]

i thought to change some of the list stuff in the program to dicts but the time wasted copying lists and dicts instead of just lists is kinda horrible so im gonna keep it as lists

[fervent tusk]

ok, added hyperparameters and bfs generation. only things left to do before converting to js/ts are:

• try using hashes to find optimizations
• put all of the level data into one massive json
• see what to do about heatmap_limits so it can work with js. numpy array wont work, but it is the most ideal option..

[fervent tusk]

implemented hashing was easier than i thought, and i am absolutely astonished at myself

i am 100% confident that the hashing is working correctly as well

but there is not a single duplicate generated at all

which is phenomenal

i am unbelievably happy at myself

so proud...🥲

the only thing that the hashing caught which was helpful is that the solve conditions for numeral cars dont trigger right, so the board wastes 2 frames before solving itself (due to mvmts_since_solved) so i can just fix that

[fervent tusk]

@jovial heath give me your thoughts on the heatmap array stuff and the flowchart, if you could. the json shouldnt take too long, so i honestly think js/ts conversion could get started tomorrow.
all very exciting!
im not sure what ide to use for js/ts so ill look into it more later, and then the code will start getting translated after i learn the basics of the language.

oh, ill also push the code to github tomorrow once the jsons are converted. itd probably be good for you to look at the code before conversion. i feel like its fine even without optimizations, plus we could always make it faster later

[jovial heath]

I'll take a look when I have time, just a little busy this week as my wife is on annual leave

Leaving myself a link #1142318326136180796 message

[fervent tusk]

why did we make the boards into dicts for ts? both support arrays, do they not? why on earth would dicts be better?

[fervent tusk]

im also definitely going to use ts since im assuming the web version will be able to support it

[fervent tusk]

im going to make the boards into arrays for the json and ts instead of dicts since i dont see why dicts would be better

[fervent tusk]

went on a break for a short bit, back to working on this

finished up turning the stuff into json

level data went from a humble 3172 lines long to an inordinate 42707 lines

definitely not breaking that pb any time soon

830 kilobytes is crazy for what is essentially a text file

anyways

i just need to make the python version work with the json and im also going to switch the python version from using dicts back to numpy arrays... unless i get an explanation for why dicts are better

[fervent tusk]

im pretty sure im finished with the python version for now

ive checked everything i think and it all seems functional and ready to be converted

i noted an issue with 10-7 on it failing before i started working on the program again but the issue doesn't seem to be there any more so im going to pretend like it doesn't exist

im going to edit the README to be better and then update the github so you guys can marvel in the readable oop version

on the note of errors, one thing im going to add to the checklist of stuff to get done is id like it if there was some really easy way to automatically (or if thats too hard, make it easy for someone to manually) file an issue on the github

[fervent tusk]

on the topic of updating the github... i wonder if naming the solver something other than "RailboundSolver" would be nicer

i need your guys' thoughts on these pivotal decisions

[fervent tusk]

alright, i updated the github

https://github.com/FoxtrotOnce/RailboundSolver

i feel like it would be a lot nicer if it had an identifiable name to go with it. something like "Conductor"... i feel like there's a better name i could use though

[delicate nymph]

Cool, I may create a port for javascript now

[fervent tusk]

i feel like i should do the port since i understand the program the most but i dunno how you would make the website using my code, im sure you have to integrate it somehow

i could just convert the solving algorithm and objects to typescript and you could integrate it after?

[delicate nymph]

I will make the UI first

I will combine the level editor and have a button "find solution"

[fervent tusk]

theres a lot of things i want on the ui

some might have to wait until the port is done

[delicate nymph]

Yeah I will try to make it smooth movement

[fervent tusk]

could you add a section for hyperparameters? i want a switch between DFS and BFS, and 2 heat limits. i also want a description area for them so the user understands what they do. i can give the descriptions

[delicate nymph]

Sure

Let me choose some tech first

I don't know if React.js is easy for anyone to read or just javascript is enough

But look like I'm gonna use pixijs for the graphic

I will try Pixijs + typescript first and reactjs is optional (I will use it in case the UI get more complex)

[fervent tusk]

im not really sure how to properly use typescript types so im just gonna use the language and see what happens along the way

i got stuff set up to be able to compile the ts files and run the js ones though so woohoo

[fervent tusk]

typescript is making me go insane

i cant just make a dict of 3-ways that list the 3-way it gets swapped to

i need to type hint everything... and its impossible to easily type hint that dict of 3-ways. Partial<Record<Track, Track>>, and {[key in Track]?: Track} don't work because "oooo what if the key is undefined???" (i know for a fact that the key will never be undefined) and Record<Track, Track> and {[key in Track]: Track} don't work because "oooo the dict of 3-ways doesn't list all track objects even though you typed them" (i know for a fact that the key will never be something other than a 3-way)

cant do {[key in three_ways]: Track} where three_ways is a set of the three-way tracks because sets arent iterable, but a set would be nicer because its O(1) (it gets referenced in another function as well)

so the only other option would be to type each key... which is basically just retyping the entire dictionary to declare the types

thanks for making sure everything is airtight typescript

nvm adding a ! after indexing the "dict" (object cause typescript) magically makes typescript happy

[fervent tusk]

i got some of the classes.py file converted to ts, im probably not going to use typescript's enum class even though the python version uses enums because typescript treats enums like the actual enum values instead of objects, and i'd rather have the enum stay as an object instead of get converted to a number after its used

[delicate nymph]

typescript is hard when you first use it xD

[fervent tusk]

finished converting all of classes.py to typescript

whats important for typescript at least

conversion is so draining

gotta do main.py now

[fervent tusk]

theres so many type checks it is actually crazy

the json of the level data only defines car positions and directions as number[] and string and i have to turn them into [number, number] and ["LEFT" | "RIGHT" | "DOWN" | "UP"]

i can convert number[] by doing [pos[0], pos[1]] but "string is unassignable to "LEFT" | "R..."

i know for a fact that it will never be otherwise but i have to do an unnecessary check now

just gonna type assert it

womp womp

[fervent tusk]

i have absolutely 0 idea how to convert the TCO to typescript, i barely understand how the syntactic sugar works in python

i put it into an ai converter and it doesnt even convert the wrapper so im assuming i cant even do that actually

ill just do it manually so the first call is TCO(generate_tracks(args)). not sure how im gonna do some of the other typing and i dont really wanna just copy and paste what the converter does unless i understand it soooo

[delicate nymph]

I manage to change the car's number on the fly

[fervent tusk]

it looks offputting imo

maybe if the font, weight and outline are accurate to the other svgs id opt for that method

[delicate nymph]

in typescript you can add param to a queue, and pop it to process, it the same logic as TCO

[delicate nymph]

yeah it hard to use custom font in it

[fervent tusk]

nice pull request

i will accept that

merging it created another branch that is identical, i havent used github too much in the past and based on what i looked up it seems like i can just delete it... @delicate nymph can i delete it?

[delicate nymph]

yes you can delete after merge

[fervent tusk]

i believe ive properly implemented the TCO after learning more about generators/iterables in typescript and making my own deque

just gotta check if the deque works

[fervent tusk]

i think it works now

linked lists are actually black magic

especially doubly linked lists

man

good stuff today, i got TCO implemented (not going to test it until generation works) and finished up converting everything besides generation to typescript

so all thats left now is the generation, and then that should be it for the algo. ill probably make a few tweaks once im done but im happy with how its going

i feel like i actually know typescript now. im a wizard

converting generation will be the true test though...

[fervent tusk]

ok i finished converting generate tracks

i would absolutely love to run it and get bombarded with bugs have it work perfectly but the program uses itertools.product to get every new branch and typescript doesn't have itertools.product and i dont feel like downloading a library so im just going to convert the python code itertools.product uses to typescript

...tomorrow... today?

im glad i can type fast. if only i could have originally wrote the entire solver algorithm in 3 hours like i just did...

so as soon as i finish implementing a product function and not have to spend time debugging it it should be all ready to be worked into the website for @delicate nymph

we still need branding though. maybe ill think of a good name and make a silly image for the solver soon

[delicate nymph]

xD

wow the AI can now review the code 😮

[fervent tusk]

incredible

youre the best thinh

copilot overview is kinda funny

[fervent tusk]

nice changes but i dont like that classes and main are refactored

if were adding solutions im not too sure if we should have the solutions in a separate json or kept with levels.json

[delicate nymph]

I just use it for debug

[fervent tusk]

ah

[delicate nymph]

https://media.discordapp.net/attachments/865247647216041996/960925428221108304/954722895182037042.gif

[fervent tusk]

im going to accept it but see if i can reject the algo file changes

[delicate nymph]

Oh maybe my linter auto format the file

I remember I add only 4-5 line of code

[fervent tusk]

yeahhhh

[delicate nymph]

Damm xD

[fervent tusk]

ok i think i did it correctly

[fervent tusk]

finished converting everything, gonna check if the product and copy array functions i made worked and then play roulette with generate_tracks working

[fervent tusk]

finally finished making product work

python list comprehension is hard to convert

well

everything is converted

time to see if it works first try

not a chance 🥲

bugs are not major like generation yet at least

i take that back TCO isnt passing the right arguments to generate_tracks so thats a generation issue

[fervent tusk]

i hate that spread variables need their own argument when passed into a function

python makes everything so easy

im grateful for overloading in typescript though i wish python had that

on second thought im just going to make it such that all arguments are passed as an object since that way spreading is unnecessary, individual args can just be wrapped into the object, and as an added bonus you have property names for each parameter

[fervent tusk]

got generation to work for world 1

python does it in about 9 seconds usually, js does it in about 2.4

very welcome improvement, but i thought it would be a LOT better

im curious if i wrote javascript instead of compiled typescript if it would be faster

not going to mess around with that though

im going to get the generation working on all of the worlds and then try and package it such that you can just call the algorithm with a level configuration and itll return the solution

[fervent tusk]

javascript seems to be 2x-3x as fast as python

very nice

[fervent tusk]

apparently theres an issue with decoys that has existed since OOP conversion, woops

something about the amount of frames decoys can move for after a board is solved i think

cause it gives a 10-tracks-saved solution for 7-5A that doesnt work due to a crash after the board is solved

decoys need to be able to crash into the last car at the ending tile even if the board is solved for those 2 frames, im pretty sure i remove it from generation so that doesnt happen and thats why though

super interesting edge case

i wonder how i didnt convert that when doing OOP

actually i can just check if the tile in front of the decoy is the ending track and not have to deal with car positions that way

curious how that wasnt resolved before

yeah no the line is there actually

strange

ah

one of the conditions for considering a board solved is if the first car in cars_to_use is not a decoy (there are no decoys to consider generating for 2 more frames for) and the car isnt removed from generation even if the board is solved until after the solution is registered

so i think i can just move the car-remove-from-generation-inator before the solution is registered

still did not work

also just realized i probably put the remove from generation thingy after since cars_to_use needs to be indexed to check if a decoy is in it, but if theyre all removed before that check, it raises an error since theres nothing to index

indexing cars_to_use to check if a decoy is in it is also dangerous because the exact same thing wouldnt work for numeral cars

since cars are first in the list, then decoys, and then numeral cars

so checking the first index of cars_to_use to see if there are any decoys works for cars since there should be no cars and therefore decoys is in the front, but if it were just using numeral cars then there could be some still in the list, but the decoys would still get indexed ... something something i need a better way to check if decoys still exist

i think ill just iterate over the whole array because that specific line of code doesnt get triggered very often at all (it is the final if statement before registering a new solution) so i could just do that

yep changing it to check all of them fixed it

woohoo

ive been planning on making some statistics for the solver for a while, i need to check how long every level takes eventually

running the ones that take more than 12 hours will not be fun though

[fervent tusk]

checked all of the worlds (up until the levels that take too darn long anyways) and the generation code seems to work now

[fervent tusk]

alrighty! i made the level solving stuff into a function and it returns the best board, mods, and tracks and semaphores remaining

so @delicate nymph you can go ballistic, everything should be set up for the perfect website now

@jovial heath feel free to check the code out for any improvements

the last few things i want to get done otherwise are some minor things with the algo and then branding (gotta have a name and logo)

it would also be really cool if there were a link to the website for the solver inside of the game itself. it could be an "unofficial solver". would have to see what the afterburn team thinks of that

[fervent tusk]

i added the remaining images to the github and all 4 directions of each ending track

[fervent tusk]

i wonder what i should make for a logo. i could sketch one up and then try my hand at drawing it. ive never really drawn anything digitally though so thatll be interesting (i do have a drawing tablet though)

i feel like there needs to be a search tree on it since its like the main thing the algo does

actually figma would be better graphic design would be nicer

[fervent tusk]

i might just deprecate the python code since keeping both the ts and python versions up-to-date doesnt make much sense any more

i will remove them from the repo next commit

[fervent tusk]

@tranquil sinew can i use this train image for track sprites for the website (the solver will be available on a website hopefully soon)?

im also curious if you would be interested in having a link to the website as an "unofficial solver" in the game, like in the level editor section or something (for example, shown in the image). its totally fine if not.

[tranquil sinew]

Yeah, this is completely fine

[tranquil sinew]

Not really since we prefer to keep all in-game links to only refer to our stuff, to ensure support for years to come; but thanks for asking 🙂

[fervent tusk]

got it
thanks for letting me use the train sprite

[fervent tusk]

@delicate nymph i can help work on the website if you would like me to. im really excited to get this published

[fervent tusk]

im gonna try loading workshop levels into the solver to flatten out more bugs

[fervent tusk]

man i love watching it solve stuff, it never ceases to make me happy

[delicate nymph]

Hi, I'm currently a little busy as the moment and will get back to you soon at this weekend

[fervent tusk]

i ran the solver through as many levels as i could, and it successfully solved all of them except for a few that i skipped because they all take too long:

• 5-5B (should take about an hour)
• 7-5A (should take less than a day?)
• 8-4A (should take about 2 hours)
• 8-6A (probably multiple days...)
• 10-1A (should take about a day...)
• 10-5A (probably multiple days...)
• 11-9B (should take less than a day?)
• 12-8A (should take less than a day?)
• 12-8B (probably multiple days...)

the longest one i successfully ran was 12-10 which took 1hour and 54 minutes but it gave a nice reward of 6 saved tracks

since the javascript version is twice as fast as the python version at minimum the times were all cut in half at least

the rest of world 8 and world 12's saved tracks are kinda funny though, its amazing there is this many in the first place

like, 7 tracks?? saved on 2 levels?? damn

i love all the work ive done on my solver. it makes me so darn happy to watch it annihilate all of the puzzles in this silly little train game, and it gives a perfect solution, first try, every time

[fervent tusk]

going to run these ones for extended periods on my pc

my pc also seems to be double the speed of my laptop, so thats a x4 speed improvement from python

[fervent tusk]

i just realized a potential optimization for decoys

decoys sometimes place a track on the side to get out of the way before they crash themselves like on 7-2

the solver will make 3 copies of this board however, with that side track the decoy goes on to being either a vertical track, or one of the two turns

so what i could do is make it so that the decoy is only able to crash with one variant, not 3

the only issue is how do i determine when the track can crash and when it cant?

placing a vertical track or a turn is perfectly legal and would be able to crash the car in any situation, but i only want one of those variants to be able to generate a crash

i could either allow specific tracks to crash, or restrict specific tracks from crashing

if i tried to only allow straight tracks (vertical/horizontal) then that means a decoy cant crash from a 3-way or tunnel which is problematic

i could restrict turn tracks from crashing the car

i would have to verify theyre placed tracks and not permanent ones though, because then a decoy wouldnt be able to crash on a permanent turn track

i think it should work?

ill attempt to add that in some time

i desperately wish there was some optimization i could find for semaphores... the things make generation take magnitudes longer to complete

[fervent tusk]

my optimization theory worked!!

most of the world 7 levels had their iterations decreased by a few thousand iterations which isnt too bad

a few were sped up by about 25k (100k iterations -> 75k / 94k -> 69k)

always nice to see optimizations despite the program being complete.

i suspect that this optimization will make 7-5A take less time, which is a very welcome improvement given that its one of the levels that would take upwards of a day or longer to complete

[fervent tusk]

theres one semaphore optimization i can think of right now: semaphores get placed whenever possible (based on a bunch of heat checks), and the hope is that a car will somehow trigger the semaphore so that all of the cars solve

one small thing i could do is check if the other tile that can trigger the semaphore is empty when no more tracks can be placed

that would effectively shorten the track count of all states that use a semaphore by 1

...in theory

im not sure how big of an optimization it would be and if it would work so ill check

[fervent tusk]

huzzah!!! it worked!!!