tulip kelp
I'm working on a heightmap and associated .json files with coordinates of towns so that map makers and whomever else can customise a British Isles scenario with their choice of newgrfs, map size and amount of towns placed. This should allow a lot more flexibility for people wanting to create a scenario and won't require manual placement of thousands of towns each time.
Current progress is as follows:
British Isles - All cities
๐ฎ๐ช&NI - All towns
๐ด๓ ง๓ ข๓ ท๓ ฌ๓ ณ๓ ฟ - All towns (In EN or CY)
๐ฎ๐ฒ - All towns and villages
๐ฏ๐ช - All towns and villages
๐ฌ๐ฌ - All towns and villages
& some screenshots of current town placement
The heightmap itself is very high resolution but does need a lot of tidying up around the coastline
still charm
The east coast is a glimpse into the future of Lincolnshire ๐
tulip kelp
Don't worry I will raise England from the sea
tulip kelp
Today I finished plotting all 1483 villages in Eire ๐ . Had to rename about 100 duplicates to disambiguate them. Takes a good 5 minutes to load the villages in from the .json. Will also be possible to choose to import all the villages in Gaelic instead (where available)
plain bramble
if youre planning to do that density for everything youre gonna run into the town limit (5000)
i applaud your efforts though
still charm
Luckily you can make some progress on cutting them down automatically since you're doing it data-driven.
tulip kelp
if youre planning to do that density for everything youre gonna run into the tow...
There's a town limit??
plain bramble
Chris sawyer
tulip kelp
good news, for my sake, you are wrong
plain bramble
Did it get lifted recently? I know new game only lets you do 5000
tulip kelp
๐คทโโ๏ธ
that would have ruined my entire day and week if it were true
but I actually already had 5,287 placed as of last night
plain bramble
Youโre using json import right? I bet it bypasses the usual limit, I wonder if theyโre all actually functional or if anything past 5000 is broken somehow
Yeah 5k is just new game limit, the actual max number is much, much higher
Learn something new everyday
tulip kelp
Youโre using json import right? I bet it bypasses the usual limit, I wonder if t...
yes, I am. It takes many times longer to load the towns from a json than spawning them, when I did 5000 random towns it takes 1 second in the scenario editor, when I load 1500 villages it takes 5-10 minutes! Idk if that is a bug or not because it seems excessive
tulip kelp
In the meantime, I've been finding out how to correct the offset positions of towns that you get when trying to translate latitude coordinates onto a mercator projection such as a heightmap. Essentially this chart shows that while towns at the top and bottom of the UK would be placed correctly (Inverness and Truro), those in the middle would be in the wrong place by nearly 200 tiles vertically. But by comparing the correct locations of towns in-game with their coordinates, I established a clear function that defines the expected error for a town of any given latitude, which means I can subtract that error from each town's coordinates. This reduces the vertical error down to 2 or 3 tiles for the vast majority of towns ๐
it works ๐ฅน one London is the one I plotted manually, the second is the one that was calculated automatically from its real-world location
Previously it would have been 100 tiles out without my correction function
tulip kelp
so, progress report. this is what 11,314 cities towns villages and suburbs imported on a 4k map looks like. (importing suburbs as towns is actually a bad idea, but I wanted to see what would happen)
I do have 3400 duplicate village/town names which is a slight problem
still charm
Are there any duplicates if you suffix the county name (but only for places with conflicts), like Gillingham (Essex) vs Gillingham (Kent)
tulip kelp
If there is, I can fix them manually
In Eire I only ended up with 1 instance of 2 villages in the same county with the same name (2x Ballingarry)
The problem is when I exported the data for England I didn't take any county data so I need to get that in order to automatically fix any duplicates
and it's hard to automatically assign a town to its parent county because of the mixture of ceremonial counties and administrative counties
What I will probably do is take each county's data individually and then I can add the correct county to each row
because I kinda want hamlet data also to have those as labels for more decoration which is not in the data export I took for England
this is the density I get for towns villages and hamlets where I live
tulip kelp
Good news, I now have a complete spreadsheet of all 12868 cities, towns and villages in England, with county suffixes added where there are two or more places with the same name. I also have all 16930 suburbs and hamlets so those can be imported as decorative labels.
tulip kelp
This is what a 4k map of England looks like with all 1189 cities and towns imported
& this is with all villages (attempted to be) imported. About 2500 failed due to the density of villages
clearly not a good idea on a 4k map
delicate plover
& this is with all villages (attempted to be) imported. About 2500 failed due to...
I can see my house from here!
Also why are the fens underwater
It isnโt 2040 yet
still charm
I think grooming the heightmap is waiting until the town data is captured
tulip kelp
Also why are the fens underwater
I haven't been bothered to fix the heightmap yet
It's a nice heightmap but it's not perfect when it comes to the coastline
tulip kelp
I did manage to import all the towns & villages onto a 16k map (took 1 hour to process). Aside from 2 villages that have names too long and some places that are underwater, everything was placed correctly
tulip kelp
thankfully a 16k map with 12000 towns has the exact same performance as one with two
I was worried it would be sluggish
still charm
I should have told you that'd happen lol.
plain bramble
thankfully a 16k map with 12000 towns has the exact same performance as one with...
yeah no the only things that really affect performance afaik is map sizze and number of vehicles
random dew
and trees, its crazy how trees expand the size of save games ๐ and the time to save it...
open grotto
This is exciting, I hope you can sort out the town placement - might just require going into Scenario Editor and deleting towns and placing them further north of the map. Do you have a team helping you as this will be a monumental task to connect everything up with transport?
tulip kelp
@open grotto town placement is pretty much sorted - the algorithm I made to transform real coordinates to ingame coordinates only has a maximum error of 8 tiles at the very north of the map. If need be I can manually change some of the coordinates in places like Orkney which are also to be included. I'm using a json to place all the towns so that people are able to re-generate the scenario with their own newgrfs or even map size as they wish. I'm doing this on my own, and yes it would take a long time to connect everything up but I enjoy having a save I can work on over multiple years. My current UK 4K savegame is 5 or 6 years old now. The latest bit I worked on was getting the Scottish towns, villages and hamlets loaded in. I worked on improving my spreadsheet for generating jsons which basically means you can select the desired language of town names for each constituent nation (+Ireland and IoM) e.g. maybe you want all Welsh places in Welsh and Irish places in Irish Gaelic. If no string is available, it will fall back to the default language, which is usually English. That means I do have more work to do with deduplication, as if somewhere in Scotland and somewhere in Ireland have the same name one would fail to be placed. As I said above it's as much a tool for mapmakers to create their own perfect scenario, which would undoubtedly be better than anything I could create on my own.
verbal vine
Where can I download the scenario?
plush walrus
Where can I download the scenario?
Good question. I had assumed through the in-game menu, but I can't find it...
tulip kelp
@verbal vine @plush walrus I'm still working on it - when it's done I will see if I can make a vanilla scenario available for download in-game (will require jgrpp as it's a 16k x 16k map). I will also put the associated files on github or something as in order to re-create the scenario you need to use the same .json files to load the towns (or generate your own using the spreadsheet).
verbal vine
I hope everything goes fine!
tulip kelp
Thanks! At the moment, the problem is procrastination!
plush walrus
If I ever decide to get into scenario-making, I might consider adding a fictional Island off the coast of Cumbria to this map as a training exercise...
tulip kelp
not Sodor... please not sodor
However, if you had a map overlaid over a map of the UK, you could get the exact coordinates for each of the settlements and create a .json to spawn the towns in the correct locations. You could also modify the existing heightmap to add in this missing island.
tulip kelp
Today I worked on deduplicating the few villages in Scotland and Wales where they were in the same county and therefore could not be automatically differentiated. As you can see I have now successfully got all towns, villages and cities in Wales and Scotland loaded in.
tulip kelp
There are still a few key things that need working on before I'd consider it playable:
verbal vine
So far, it looks good!
tulip kelp
The Faroe Islands have joined the party
verbal vine
It would be interesting to build a rail network there
tulip kelp
If you look on google maps (3d satellite) you can see that the terrain is even more extreme than is possible in OpenTTD. If you have slope steepness cranked up it would be a real challenge for trains. (Something I would probably recommend as you want people to follow the contours of the valleys in Scotland and Wales)
plush walrus
The Faroe Islands have joined the party
That looks like it'd be boats all day long. I doubt you'd manage to get the 8-tile runway needed for a chunnel in JGR...
tulip kelp
You only need 4 tiles, 8 is the maximum
tulip kelp
this is the best I could recreate a real road tunnel that exists there
Sadly it won't be possible to build the underwater roundabout
still charm
I love this project, I can't wait to have a play with it.
verbal vine
this is the best I could recreate a real road tunnel that exists there
It would be great for a nice interurban/tram network...
Though that's unrealistic :/
tulip kelp
That's the beauty, you can choose how realistic or unrealistic to build your network
My 4k map has giant hyperloop tunnels going to the Isle of Man
tulip kelp
One thing I am not looking forward to is picking the set of newgrfs that I'm gonna use for my long play of this map
I can never remember which (british) train and road vehicle sets are the best
verbal vine
I like RUKTS (I think that's what it's called) coloring but there's not many trains
tulip kelp
That's got some nice trains but it's missing the eurostar or equivalent high speed trains
the fastest is 155mph which is the APT (not a real train)
doesn't seem to have the intercity 125 either
verbal vine
It does
But like, the only modern electric train I have rn (I'm on year 2000) it's the Networker :(
Ok, it turns out I was using an outdated version
still charm
One thing I am not looking forward to is picking the set of newgrfs that I'm gon...
I am here to agitate for (my extension of) Timberwolf's UK Trains ๐
plush walrus
I can never remember which (british) train and road vehicle sets are the best
I've been following Fairy's ... something set in here for a while (I can't remember what it's called), which is an evolution/expansion of UKRS2. I plan on using it next time I start a long save.
tulip kelp
I am here to agitate for (my extension of) Timberwolf's UK Trains ๐
I'm not really a fan of how the timberwolf trains go round corners (and the graphics don't match vanilla)
still charm
That's fair enough, I think it's pretty much marmite.
tulip kelp
I believe I have now fixed the cities and towns that accidentally got missed out (damn unitary authorities), namely Nottingham, Leicester and the Tees Valley. That crosses the first 2 items off my todo list.
tulip kelp
If you were to build the M25 to scale in this map it would be 500 tiles tall and 600 tiles across. (I do need to dredge the Thames a bit)
verbal vine
That's crazy!
Time to connect every single town and village by rail
When it comes out ofc
tulip kelp
Time to connect every single town and village by rail
how do you plan on getting a train from scotland to the faroe islands? ๐
it's approximately 2,800 tiles across the sea
plush walrus
it's approximately 2,800 tiles across the sea
That'll be boat or plane territory then.
tulip kelp
To be replaced by a vactrain tunnel in the year 2100
verbal vine
how do you plan on getting a train from scotland to the faroe islands? ๐
Giant tunnel under the sea
tulip kelp
That's a big expense for some islands with a population of just 55,000 ๐
verbal vine
Ik LOL
tulip kelp
I've been doing some spreadsheeting today, and actually figured out a better way of transforming real-world coordinates into coordinates in-game. Basically, my method was a bit hacky using a quadratic function to transform WGS84 coordinates into Web Mercator coordinates (which is what the heightmap uses). This chart shows how my close my function's coordinates are to where the real in-game coordinate should be. For example, the highest point on the graph is Inverness, which was placed 8 tiles lower than where it really should be. The problem is that as I also added the Shetland isles, they are even further north and therefore had even greater errors than Inverness (the northernmost village was out of place by over 110 tiles.)
What I have been experimenting with is a proper formula to convert WGS84 coordinates to Web Mercator coordinates, which then map linearly onto my heightmap. After quite some wrangling, all the test cities I have applied this new method to have errors of fewer than 1.5 tiles (!). The benefit of this is the same formula can be applied to maps created anywhere in the world, not just at the same latitude as the UK!
both charts have latitude of the city on the X axis and error in town placement on the Y axis
Embarrassingly, a lot of my problems today was caused by the fact that my list of coordinates I was aiming for was actually wrong! I had it in my spreadsheet that I wanted to place Inverness at tile 5478, which meant that when my formula suggested 5491 this was out by 11. But the tile I want to place Inverness on is in fact 5492 which means my formula was almost perfect, but my sample data was wrong ๐คฆโโ๏ธ
tulip kelp
I just got the function for y working in excel (instead of doing a manual conversion each time using a web tool) and it works like a charm. Now I can bin off my hacky homemade functions and have perfect town placement across the whole map, and it could be used as easily by someone creating a scenario anywhere in the world.
tulip kelp
This is my final formula for transforming a real world latitude into an exact tile coordinate on the map
=15646+((1/(2*PI()))*2*(PI()-LN(TAN((0.25*PI())+(RADIANS(B15)/2))))-0.685269074)*131088.0386
where B15 is the latitude of the town
Now I can start collecting all my separate lists of towns in England, Wales, Scotland, Faroe Islands, Ireland, Isle of Man and Channel Islands into one big list (necessary in order to de-duplicate any town names which appear in multiple nations)
As I'm taking the town data from OpenStreetMap, the possibilities really are endless for what I could import as labels for decorating the map. The obvious choices are things like mountain peaks and lake names, which won't conflict with any town names. Or, I could do something silly like have a label for all 8000 benches in Wales.
plush walrus
This is my final formula for transforming a real world latitude into an exact ti...
Is there an equivalent formula for longitude?
still charm
just a linear rescaling, (web) mercator is linear for longitude
tulip kelp
Exactly, mercator does not distort longitude.
tulip kelp
This is why when I was originally trying to convert coordinates it was so frustrating, because the calculation for longitude -> Y worked perfectly but latitude -> X was having things in completely the wrong place. This initial attempt was a couple years ago and I had no idea why one was working and one didn't.