#Languages, translations, I18n and L10n

979 messages · Page 1 of 1 (latest)

brave hawk
#

Internationalization [i18n] is the process of designing a software application so that it can be adapted to various languages and regions without engineering changes.
Localization [l10n] is the process of adapting internationalized software for a specific region or language by translating text and adding locale-specific components.
~ https://en.wikipedia.org/wiki/Internationalization_and_localization
Now that we've split from Forge, we need to talk about how to handle translations -- crowdsourcing them on a platform, and integrating them into NeoForge proper.

#

'ello bolbs and bolbettes

serene olive
#

just have chatgpt translate norge into french

brave hawk
#

we have two main questions to answer:

  1. What platform will we use for crowdsourcing translations?
  2. How will we integrate these translations into NeoForge?
brave hawk
jaunty wadi
#

also will chatgpt translations allowed

rustic ibex
#

i don't think where the translation came from matters as long as it has the right copyright and it's correct

jaunty wadi
#

or not chatgpt, but i heard that there is whole ai tool just made for translating stuff

#

lol

brave hawk
#

some historical background

In Minecraft Forge (henceforth MCF), we used the platform known as Crowdin for crowdsourcing translations from the community and integrating them into Forge.
On each build, the CI provides an API key from Crowdin to the buildscript, which then fetches an export of the translations at that moment in time from Crowdin, and integrates it into the JAR.

#

current situation

We have no translations. 🔥

Because we split off from MCF, we don't have access to the Crowdin project anymore (naturally), so we lack translations for any language outside of the base English translations in the repository.

#

current plans

We do have a Weblate instance on the server, though that is currently offline due to resource exhaustion concerns on our server. ( we screm at reposilite and TC ) Self-hosting comes with its advantages and disadvantages, though -- in particular, we would have to start fresh on getting translators, since we'd be disconnected from Crowdin's community.

We could reconsider using Crowdin, but we'd likely need to look into getting their open-source offering if we do so.

#

uhhhh.

thoughts?

short lark
#

Note that even with Crowdin OS, there are severe restrictions in their API which would hinder certain CI/CD components

craggy fulcrum
#

hallo!

weary fossil
craggy fulcrum
#

You can just have people open a bunch of language prs

#

but that'd get messy

brave hawk
zenith cargo
#

how many localizable strings does forge even add anyway

#

or, how often does it add them, since i guess where i'm going with this is "how feasible is it to mandate that any PR that adds any new translation key must also add translations into all supported languages"

craggy fulcrum
brave hawk
brave hawk
#

imagine, if you will, receiving logs from languages around the world -- Spanish, to German, to Russian, to Chinese, to Arabic, etc.

#

hellish for debugging

craggy fulcrum
#

yeah good point

elder pollen
brave hawk
#

iirc, you are referring to the exception messages that FML employs in cases like dependency errors, which is because we show those messages to the user on the mod loading error screen in the game window

elder pollen
#

Yes. The issue is that these messages are also attempted to be translated when they are printed to the log, which regularly fails and produces untranslated messages which hide the formatting parameters and therefore the relevant information

brave hawk
#

hmmm. we should improve that

#

care to make an issue on the repo? thinkies

elder pollen
#

Sure

elder pollen
#

Nevermind, I'm going to hold off on making an issue for this for now, I can currently not reproduce this and the last reference to the issue I can find in this discord where the version is known is from 1.19.0

grave steppe
#

I was about to say - it defaults to en_us if the selected language is unavailable

#

Usually that issue happens if a mod loading error prevents the Forge mod's translation files from loading, which... there isn't a solution for.

elder pollen
#

Yes, the problem is that in the case of translations being entirely missing, it doesn't fall back to a hard-coded message and instead prints a useless message with the raw translation key, hiding any and all of the useful info it was supposed to present. If you search for fml.modloading.missingdependency in this discord, you'll find a bunch of examples of exactly that happening, but, as mentioned, all those examples are really old

grave steppe
#

Yeah. Unfortunately the system for "translation if available, hardcoded string if not" is the problem you're trying to solve, as well as its solution

floral walrus
#

Even if there is an error the forge mod should get loaded

grave steppe
#

That's not quite how mod loading works

floral walrus
#

It is literally what system mods are - and forge is one of them 😛

elder pollen
#

Literally nothing prevents a mod from accidentally breaking the loading of the Forge mod

floral walrus
#

If there is a dependency resolution error, only minecraft and forge get loaded

elder pollen
elder pollen
floral walrus
#

Maybe unrelated but working with FML in general is a painful experience

#

The code is old, hard to follow, and suboptimally organized

grave steppe
#

That's maybe a little out of scope for current situation

floral walrus
#

I figured 😛

zenith cargo
#

what the fuck is this {n} syntax?

#

why aren't {0} {1} and {2} used?

elder pollen
elder pollen
zenith cargo
#

doesn't vanilla support %n$s? did it not back then?

elder pollen
#

It might have not supported that back then, but this is more than just positional parameters, it's about custom argument conversion

zenith cargo
#

i can't find the actual code any of these are printed from

#

like i'm half wondering what the patch footprint of this is and half not sure if it even works if these are used with Components at all

#

//TODO, this should be re-evaluated now that ITextComponents are passed everywhere instaed of strings.

#

Component.literal(ForgeI18n.parseMessage(
lovely

#

like i get it but

#

this whole system needs a long hard look at some point

elder pollen
zenith cargo
#

i still can't find missingdependency

elder pollen
#

ModSorter line 234

zenith cargo
#

i think fml is broken in my workspace

#

can't even find the ModSorter class

zenith cargo
#

idk

elder pollen
zenith cargo
#

sure, fair enough

#

i still can't find ModSorter, what package is it in

elder pollen
#

net.neoforged.fml.loading or net.minecraftforge.fml.loading respectively

zenith cargo
#

is it just not present in a modding workspace due to the way fml is structured? that's annoying

elder pollen
#

It is, I looked at it in a mod dev env

zenith cargo
#

no something is still wrong with my workspace because i still can't find the fucking Mod class

#

i thought invalidating caches would fix it

zenith cargo
#

ugh regenerating runs fixed it but now i can't find references for the EarlyLoadingException it uses

#

it keeps doing the decompiled class file thing

#

what is the caller of ModSorter.sort

digital quest
#

You can use crowdin's github integration and it will add to a pr when there are language changes (we use this on Mekanism)

elder pollen
elder pollen
wicked sable
#

From experience dealing with prominent foreign communities - crowdin seems to be a solid preference in non-english speaking communities

short lark
digital quest
#

its not a plugin? I use the native crowdin integration which handles branches just fine

#

it pulls the translations from the branch, then will commit them to the configured service branch and create the PR

#

the only annoyance I have with it is that it'll do one commit per file per language

short lark
#

But yeah if they improved this, then it might for sure be an option

#

It really was not an option in the past

digital quest
#

yeah we squash it on mek. it triggers the webhook every commit tho lol

weary fossil
floral walrus
#

So there's two things:

  1. Forge extends the translatable component syntax and we might want to preserve that feature, and
  2. FML can't use vanilla Components
weary fossil
#

isn't the error reported by forge

floral walrus
#

Yeah it is, but the translation key is generated by FML

weary fossil
brave hawk
brave hawk
#

i'd apply for our project, but point 5 probably means I'm not the right person to do it thinkies

  1. You are the project lead.
#

someone from <@&1128776748666978355>, perhaps

weary fossil
#
  1. You keep the "News" section of your website up to date.
    9.You release updated builds on a regular basis.
    thinkies
#
  1. You have created a translation project in Crowdin.
#

well step one is a blocker currently harold

brave hawk
#

i'll schedule some time to make the project

weary fossil
#

i can do it in a bit. you must be sniped stabolb

brave hawk
#

if you have the time, sure

#

wait, no

#

I MUST DO IT

#

goes on a hunt

#

damn you internet

#

just realized, we need to request an open-source license to add managers to a project

#

oh well.

weary fossil
#

stab

brave hawk
#

stab'd

weary fossil
#

i was fighting crowdin as apparently my account uses the $125 plan somehow and i have no idea how to downgrade

brave hawk
#

welp. time to ask the top brass if I can be the 'project lead' of the day kek

floral walrus
brave hawk
#

crowdin enterprise? 👀

#

what

weary fossil
floral walrus
#

is that even enterprise? I have no idea, @brave saffron did the setup

weary fossil
#

WTF

brave hawk
#

as I understand it, organizations with a subdomain only exist on Crowdin Enterprise

weary fossil
#

i wanted to set up a password for my oauth-created crowdin account

brave hawk
#

truly a sad day for the British users of AE2

image.png
weary fossil
#

but that promopts me to confirm my password

#

WHICH DOESN'T EXIST

#

the most stupid sudo mode ever

brave hawk
#

ah yes, good ol' catch-22

floral walrus
weary fossil
brave hawk
#

i'd like to see if we can get a bit of clarification on that from shartte thinkies

short lark
#

Does the opensource variant not give you that

#

At least it insinuates that based on the boxes you have to fill in

#

Regardless I can set it up as a steering council member

#

Right now even

#

Work is dead

brave hawk
#

my understanding is that the open-source variant is for the non-enterprise version of Crowdin

brave hawk
#

unless Crowdin is willing to allow you to request the license for a project owned by another

short lark
#

That last box indicates different to me

IMG_0019.png
brave hawk
#

...huh

#

okay then, go for it

#

I'll hold on to the crowdin.com version of the project until we get that setup

weary fossil
#

is it finally the time to snipe

#

hmm, i'm a bit puzzled on how you are supposed to create an enterprise org

short lark
#

Filing it in then

weary fossil
#

i see it wants a name, work email and password. is that for the admin account or the first account created under that enterprise hmmm

short lark
#

I will just fill it in

#

And see what they say

weary fossil
#

orion, do you have an email @neoforged.net

brave hawk
#

slowly the urge to setup a mail server on phoenix rises /jk

weary fossil
#

cloudflare has been dying this week screm

#

hmm yeah @short lark according to the docs, a @neoforged.net email is indeed what crowdin suggests to use for owner accounts of organizations

#

CF is really getting on my nerves screm

image.png
short lark
#

Can somebody set that up for me?

weary fossil
#

unfortunately CF is currently dying, i can't set up new routes

#

but i can use existing routes and me and cpw, curle, me and sci have an exisiting @neoforged.net route set up

brave hawk
#

screm in not having enough foresight to make orion and shrimp their email addresses

weary fossil
#

email routing is operational but their dashboard is practically dead

image.png
#

they're having some real problems huh

#

yeah okay, i tried refreshing and different browsers, the email routing api is completely dead

brave hawk
#

oof

weary fossil
#

@short lark i created an org, since it seems like the initial owner account can't change their email, until CF decides to stop being dead, what email should i send an org owner invitation to (you can change it later when the CF api is no longer dead)

#

@brave hawk do you have mail thinkies

brave hawk
#

i have mail

#

i have two mail

#

the left and the right

weary fossil
brave hawk
#

my internet is why

#

it done

short lark
#

Alternativly you can search for my vriendin account: OrionDevelopment

#

Crowding*

#

Crowdin**

weary fossil
#

invite sent

brave hawk
#

icon'd

digital quest
brave hawk
#

hmmm. let's see if they can support Enterprise though, if only so we don't have to go through hoops if whoever owns the crowdin.com project leaves the team (since crowdin.com doesn't have a project ownership transfer mechanism)

#

plus, maty already made the thing kek

brave saffron
brave hawk
#

neat

brave saffron
#

Yeah the only mails I can find related to that topic are these, and I am certainly not paying for it 😄

image.png
sacred swift
#

there are some quicks with the FOSS plan, like only one maintainer, only one project (unless you apply for more FOSS projects), no private projects

#

but it's definitely worth it to get more features and save money

brave hawk
#

hey @short lark, any update on the Crowdin Open Source request?

zenith cargo
#

hey just dropping in here to say that forge's attribute name localization keys are just "forge.whatever" without attribute.name, might be worth changing whenever we change these things

brave hawk
#

open an issue? thinkies (so we don't forget)

#

heck, I should remember to open an issue on this very topic (translations)

brave hawk
short lark
obtuse shuttle
#

I want to contribute to the Korean translation but there isn't. Can I request Korean translation section?

grave steppe
#

We currently don't have a translation system at all, as far as I know. Once we do, we'll let you know.

weary fossil
grave steppe
#

oh that's a thing?

weary fossil
#

assuming crowdin gets back to us within the next like 2 weeks yes

short lark
#

I am going to send them another mail today

brave hawk
#

it exists, but isn't integrated yet into our systems

grave steppe
#

What I'm reading here is "not yet"

brave hawk
#

we're waiting on the open source request to be fufilled before we push ahead

short lark
#

Yeah

brave hawk
#

in part, so we can best assess our options and decide which one to use

short lark
#

They have been suspiciously quite

#

So I am going to write them another message tomorrow/this evening

wet bluff
#

In the short term people can just contribute lang files, right?

floral walrus
#

Not really, that's a pain to manage

brave hawk
#

also a pain to move to Crowdin with credits, I believe

short lark
#

I send in another request

grave steppe
#

Perfect. Thank you

short lark
#

No problem

weary fossil
#

good news, crowding gave us the OSS plan as of 3 days ago

#

we now have 500.000 words available harold

short lark
#

Ah

#

I did again not get a single email

weary fossil
short lark
#

Very nice

weary fossil
#

we can translate everything now! harold

void rock
lean snow
weary fossil
#

it's the plan

#

oh you mean how are words counted?

#

idk

lean snow
#

Mainly just does each language count separately or does each word count separately

chilly radishBOT
#

Successfully scheduled reminder on <t:1701745796:F> (<t:1701745796:R>)!

chilly radishBOT
#

@brave hawk

Camelot
Reminder

crowdin time

sciwhiz12 ·
brave hawk
#

can't

#

a day more

chilly radishBOT
#

@brave hawk

Camelot
Reminder

crowdin time

sciwhiz12 ·
brave hawk
#

:catjam:

#

@weary fossil did you upload translations to the crowdin project at some point

weary fossil
#

i did

brave hawk
#

screm'd

weary fossil
#

i had to make sure it works

sacred swift
#

so is there an official crowdin project that gets translations pulled to main thing now? where?

brave hawk
#

still contemplating (and then waiting to implement) how to handle translations in NF

sacred swift
#

oh wow, crowdin SSO overview pages look a lot different from the main thing

weary fossil
#

lol, i just noticed

#

we have pending translations thinkies

#

we need to look into how to do translations

#

i would do it with an automated PR honestly

brave hawk
#

weekly automated PRs?

weary fossil
#

no, the crowdin kind of automated PRs

#

(see mek)

#

not sure when they're done

#

but we just have to hit a few buttons

digital quest
#

Crowdin does it automatically

#

It does it every time a change is made on their side about every hour I think (configurable)

brave hawk
#

@weary fossil asking for a second opinion: would there be any forseeable problems if I fork NF into a separate repo in the org, and link the Crowdin integration there for testing purposes

weary fossil
#

no, you don't even need to disable actions, publishing is disabled on the non neoforge named project

brave hawk
#

good, good
I'll look into doing that... soon

weary fossil
brave hawk
#

not yet™️

#

but soon™️

#

(perhaps in a few hours)

brave hawk
#

@weary fossil polling: what do you think of adding a custom domain for crowdin

#

yay or nay

weary fossil
#

sure

brave hawk
#

(note: not doing it now; will do once I have mustered the energy for putting Crowdin into action thinkies)

brave hawk
#

@weary fossil should we translate the test framework's lang file

#

also, could you check what happens when the game tests run and the src/main/resources/(...)/lang folder contains lang files with uppercase letters (en_AU.json)? thanks

merry gust
#

I think that might depend on the set resource pack format

#

or maybe that one runs through the DFU regardless

brave hawk
#

silently screms in Crowdin

#

local download works

#

why in the heck does Crowdin not allow me to make a 1.20.x branch

#

oh. now it shows up

#

okay, new lesson learned: when branches are being used, specify the branch with -b to crowdin download so it downloads only that branch (instead of every branch in the same ZIP, under respective subfolders)

#

crowdin upload and crowdin download work

merry gust
#

is this using crowdin's github integration?

#

or using a different thing

brave hawk
brave hawk
#

but I was confirming the soundness of the configuration file by using the crowdin CLI tool

#

(btw, I requested your review since experience with Mek -- if you want to toss the substance of the review to Thiakil, tis okay)

#

also, disclaimer: I am slightly loopy because I've not slept in 18+ hours thinkies

#

(but it's the best kind of loopy, when I don't have the regular inhibitions)

merry gust
#

if @digital quest is up for reviewing it I will let him as he set it up including adding new languages when mojang adds more as well as doing branch stuff when we change mc version so that it reads from the new branch. I just merge the PRs that it makes before releases

brave hawk
#

oh yeah, we should also set down policy on when exactly to merge the Crowdin PR

#

i did take a peek or two at Mek's crowdin.yml for a bit of reference

merry gust
#

and what I will warn you about is like two or three years ago (so might be fixed by now[?]) it sometimes had issues with making the PRs after we merged one unless after merging the PR we also deleted the generated lang branch. I don't know for certain if that has been fixed or if it was just a one or two off bug (I think it happened twice to us before we changed) as it is simple enough when I hit merge to wait a second and then hit the delete branch afterwards

brave hawk
#

I think I did read a note somewhere about deleting the service branch after merging

#

don't remember where though

#

i'm sure it's fine

digital quest
#

It will just add a number to the branch name if the old one still exists

brave hawk
#

perhaps it would be prudent to enable that "Delete head branches" option on GH

merry gust
#

oh the other thing is

#

it will potentially at least partially spam #neoforge

#

just because each file change is its own commit

digital quest
#

Yup, one commit per language file update

merry gust
#

though given we have four mods so four separate lang files that then when someone is going through and actually translating a language they often touch all or if we add/remove things it can hit many files at once (which may be a bit less of a problem for a project with only one translatable file)

brave hawk
#

@weary fossil perhaps we can thonk a solution for this? thinkies

merry gust
#

maybe between three years ago and now crowdin added an option to push to a fork instead of just a separate branch? kaishrug (if so do let me know, I just know the last time I asked thiakil a couple years ago that wasn't possible)

#

I guess technically you could always use a GHA to clone the repo then setup crowdin with that cloned repo (and make sure that repo has no hook) and then have the same/another GHA open the PR from the crowdin cloned repo to the main repo

#

though that seems like a mess

#

or depending how webhooks work you could potentially basically have the webhook go somewhere then have that filter out the received things from the specific commit message crowdin uses, and then send as a webhook from that intermediary place to the actual discord channel

brave hawk
#

question: why do you have the mek-translations user?

merry gust
#

I forget what thiakil's reasoning was. Whether it was just attribution or whether something was getting on his nerves in regards to notifications

#

he probably remembers better than I do

digital quest
#

I think it was so I could turn the email notifications off

#

Since it makes the prs as the user it was configured with

brave hawk
#

concern I'll need to double-check that later, since I don't want to be getting PRs made from my personal account kekw

digital quest
#

There's a small chance it'd be different with the org setup you've got here, but knowing crowdin it probably still links with a personal access token lol

weary fossil
#

we're already proxying our webhooks with a worker

#

as for forks, can you even make a fork in the same org and then PR from there

#

github uses the org/owner name without the repo name in the api for the head branch of a PR implying that only one repo in the same network can exist for the same owner

#

also I hope we can use a github app instead of pats

brave hawk
#

@merry gust do you still want to affix your approval review on the PR, or should I remove the review request and wait for maty?

#

also, @weary fossil please, review

merry gust
#

I mean all my approval would be is basically proxying my trust in thiakil's review xlurk

#

as I said I don't really know how the system works myself

weary fossil
#

ok sci, need you to do something extremely hard. filter out commits made by the bot in the CF worker

brave hawk
#

@grave steppe @short lark need also your opinions
the Crowdin open-source plan is restricted to 10 total managers -- we occupy 3 slots already (us three; maty I believe is excluded as Crowdin org owner)
how do we want to divvy up manager access to the org/project?

(I think we do have an unlimited amount of developer roles, but that access is more restricted than manager)

brave hawk
weary fossil
#

THEN YOU HAVE TO REMOVE THE EXPLICIT WEBHOOK ON THE NEO REPO, AND SET THE REPO PROPERTY SO IT CAN USE THE CF WORKER WEBHOOKS

#

mhh caps

grave steppe
#

screm

brave hawk
#

CALM DOWN MATY

grave steppe
#

i would say: steering council & subproject leaders

brave hawk
#

NO NEED TO SHOUT

#

coughs anyways

brave hawk
#

good joke, maty

merry gust
grave steppe
#

lol

brave hawk
#

(something for #project-talk? thinkies)

weary fossil
#

în the worker script you'll have the payload as a json object

#

just check the webhook structure from the gh webhook viewer and make sure that the bot and the commit event when together are ignored

#

there's one already for the pr publishing bot and commit comments

#

anyways sci merge your PR, I want to see it explode

brave hawk
#

which block of logic is for commit comments thonk

weary fossil
#

it might not filter out bot and commit comments, just bot actually

brave hawk
#

that looks about right to what I'm reading

weary fossil
#

but you can check the webhook payload in the webhook tab of the org settings

#

see what field is set to what value for a commit

brave hawk
#

ah, I think I see what you mean now

brave hawk
weary fossil
#

also don't send screenshots or snippets of that worker in public lol

#

you have high chances of leaking a secret string stabolb

brave hawk
#

the hardcoded secret alone kek

#

"bad." indeed

weary fossil
#

and the 4 webhook links

brave hawk
#

i wonder if you can make those links configurable with worker variables or something

merry gust
brave hawk
#

fair enough then

#

I'll take off your review request thinkies

#

i'll merge in a couple of minutes, currently doing some other things

brave hawk
#

I think it's time

#

pray for me

#

shit

#

okay, so lessons learned:

  • use a PAT for a machine account instead of my personal account
  • the repo rules require the build workflow
#

@weary fossil need your opinions
do we make a separate machine account for the Crowdin translations, or consolidate with the existing machine account (but renamed, when cpw gets back from his vacation)

#

@short lark I don't suppose you have the creds to the neoforged-machine-admin account, or do you? thinkies

#

@weary fossil add'tl to that: is it possible to exclude the crowdin PRs (perhaps by the machine account login name) from being PR-published automatically?

#

pending questions:

  • what machine account should we use for Crowdin translations
    • neoforged-machine-admin? need to pull creds from cpw (to rename/create PAT)
    • a new machine account? neoforged-translations?
  • what do we do with the PR requirement (by repository rule) that the PR build workflow be successful
    • actually build commits for translations (remove the [skip ci] from the crowdin file)?
short lark
brave hawk
#

screm.

#

we should resolve that one point of failure -- machine account credentials -- particularly if we are to make a second machine account

brave hawk
#

discussed this with cpw a bit in the internal channels

he'll be back after a week or so, for better resolving the problems with shared credentials (see: machine account) because it's further complicated by our required-2FA policy on the GH org (which I personally don't want to disable)

#

and because any machine account we make is also going to be subject to that 2FA policy, we do need to setup a credentials service in some form to share these credentials for the machine account(s)

#

and I'll likely become sparingly available starting next week, due to resuming the 2nd semester and all the activites lined up
so that'll be fun

fleet orchid
#

2fa basically renders shared accounts on GitHub impossible. Kinda the point heh

brave hawk
#

after more internal deliberations, we've decided that I'm going to create a machine account for the translations (neoforged-l10n is tentative name) and use that (via PAT) for the Crowdin GitHub Integration

digital quest
#

Some password managers can store/share totp code secrets, and google authenticator can be used to export and clone it too if needed

weary fossil
#

I can disable the checkbox for pr publishing by default if the authors name ends in -crowdin

weary fossil
#

actually when the branch is updated through the gh ui it will build it since the last commit will be a merge commit

#

so sounds like a non issue as long as the branch is one commit behind before merging

brave hawk
weary fossil
#

sci

#

how did the bot bypass the rule...

#

@brave hawk stab

brave hawk
#

SHIT

#

CROWDIN stabolb

#

...oh well.

weary fossil
#

WHY WAS THE BOT ALLOWED TO BYPASS IT

digital quest
#

What did it do?

brave hawk
#

because our repo rules are allowing the Bots team to bypass em

#

well, that's a minor defect

weary fossil
#

I want to stab whoever thought that's a good idea

brave hawk
#

I think this is the part when we realize it was you who did it /jk

#

i've removed the bypass

brave hawk
#

oh well.

#

time for a sync!

digital quest
#

It does warn you it'll edit it iirc

weary fossil
#

also sci share the account creds and totp token with me, just so if we lose you we don't lose that account

brave hawk
#

I WAS UNPREPARED

weary fossil
brave hawk
brave hawk
#

@weary fossil WHY DOES THE WORKER NOT FILTER

weary fossil
#

because the neo repo doesn't use the worker hook yet

brave hawk
#

i forgot about that

#

damnit

weary fossil
#

set the repo property and remove the explicit webhook

brave hawk
#

aye, it'd be fixed now

#

@weary fossil need your assistance, it seems the webhook isn't working

#

...it didn't save the custom property value

#

should we change the default PR title?

#

@weary fossil if I untick the box for PR publishing before the action runs, does it not publish

#

well, I say that having done it, so time to see kek

#

shall I do the honors of merging our first translations PR? thinkies

weary fossil
#

sci, I task you with doing something much important

#

make it not tick the box by default for the crowdin prs

brave hawk
#

i have no idea where to start looking for that kek

weary fossil
north pelican
#

Could Dutch be added? (nl_nl.json)

sullen cradle
#

and a copy to be_nl potententially? Though realistically nl_nl is fine

digital quest
#

It can do copy languages, if it's not defined in the sub it'll use the parent one.
Last I knew it was flaky with the github integration tho

weary fossil
#

@brave hawk for the record, i need a tutorial on how to do new langs

#

mostly related to your custom metadata stuff

north pelican
#

(also, why are you grand rotater sci?)

brave hawk
weary fossil
#

i added the lang, where do i add the mapping

brave hawk
brave hawk
#

scroll down a bit thinkies

#

i'm sure you'll find it :patpat:

weary fossil
#

there

fickle osprey
#

whats the feelings on pop culture reference translations/languages? For example i was just doing the British English translation and when it came to gravity i wanted to make it a doctor who reference as the latest Doctor who episodes went back in time and changed gravity to mavity

analog perch
#

I don't know where to request so I'll do it here: could you add Italian?

weary fossil
weary fossil
fickle osprey
#

i figured, but no harm in asking 😂

digital quest
#

I removed British from Mekanism cuz some person changed some words to obscure British ones...

weary fossil
#

"heavy wa'er"?

brave hawk
#

waʔer

digital quest
#

Even wierder

grave steppe
north pelican
grave steppe
#

The commit isn't linked directly so I can't see it on mobile

#

I remember annoying you with the British English translations in Mek before lol

analog perch
#

Should I translate abbreviations like TPS?

grave steppe
#

Yes

digital quest
#

Wasn't you curle 🙂

grave steppe
#

OK good lol

digital quest
#

I think a lot of yours were the exact same as the US ones so a bit redundant tho

grave steppe
#

Ye, I didn't understand what Crowdin wanted me to do for ones that don't change

#

Was my mistake lol

bitter gust
#

@grave steppe Hey since I was in charge of Korean translation before I'd like to do the same thing

#

Could you add Korean to the list?

brave hawk
brave hawk
brave hawk
bitter gust
#

wow fast

grave steppe
#

The convention is to translate initialisms where the expanded version still has independent meaning

weary fossil
#

how often do we want to merge translations

grave steppe
#

Ticks Per Second is a very common thing to say

north pelican
#

should I translate Fancy Mod Loader?

grave steppe
#

That's a brand, so no

north pelican
#

kk

bitter gust
#

Hmm

#

was there major changes in translation keys?

#

if not we could reuse the old one

#

It's at 0% percent which previously was at 100%

weary fossil
#

the pregen stuff was ripped and redone

#

keys of forge -> neoforge

#

new keys got added wrt dependencies

#

all the network-related stuff is about to get ripped too this sunday

brave hawk
analog perch
#

What the hell is froge lol

grave steppe
#

Snapshot frog

#

but Forge

#

frog Forge

#

froge

brave hawk
#

another thing to pin:
when a new version branch is created, a Crowdin manager has to enable the integration for that branch at https://neoforged.crowdin.com/u/projects/2/apps/system/github
(it's not automated via the branch pattern matching, because otherwise all of the old branches we inherited from Forge would be included, which would be bad™️

analog perch
grave steppe
#

It's a meme that was used for snapshots a couple versions back

#

When one was about to release

#

Mojang would post a frog

#

So frogs became synonymous with snapshots for a while

brave hawk
#

(and snapshots are usually released on Wednesdays)

weary fossil
#

i hope you know how i read the last part Stabby

brave hawk
#

silent on the webhook front

weary fossil
#

if only you knew

#

i changed something in the wbehook and 2 mins ago it spammed #neoforge

#

i deleted it all

brave hawk
#

oh. that's why I saw unread notifications for #neoforge without any actual new messages kek

#

btw maty, is it possible to use Workers to listen to a webhook and specifically do some action with the GH API automatically

#

serverless automations thinkies

weary fossil
brave hawk
weary fossil
#

tbh if we were to d ostuff like "wait X hours before merging BC PR" that would be done with a server and webhooks, not with actions

brave hawk
#

my thought was to automatically edit the PR description of neoforged-l10n's PRs with a prefilled thingy

#

perhaps something for another time

weary fossil
#

translations go brr

image.png
brave hawk
#

this is so fun 👏

weary fossil
#

...to watch it fail

#

/s

brave hawk
#

i see Romanian thinkies

weary fossil
#

anyways sci

#

yes

#

that was me

#

#squirrels-🦊 i pinged you

#

go look and do kthx

analog perch
#

Italian finished!

#

God, techical stuff is hard to translate

coral mirage
#

Out of curiosity, are all usages of the translation keys (if they are even used) within the NeoForge GitHub project, or are some translation keys also used in other, related projects?

weary fossil
#

FML too

#

also we should decide on a how often we merge translation PRs

brave hawk
#

well, at least

#

that's 52 merges over the course of a year (assuming each week has translations that need to be committed)

weary fossil
#

(bad assumption)

#

i see the bot signed the CLA

north pelican
#

Dutch has been finished if I didn't miss anything :)

tardy creek
#

Hello there!
Is anyone already translating Spanish?

#

I did a similar translation job back some years ago where I (alongside another Spanish friend) made the official Black Mesa spanish translation in Crowdin
So, how can I help

brave hawk
brave hawk
#

we do need to work on adding context to some of those lang keys...

tardy creek
#

alrightyo

#

I'll star on the 2nd of January

placid flint
digital quest
placid flint
short lark
#

Froge is a Snapshot version of Forge

digital quest
#

OH, yeah posting screenshots is not clear.

you seem to be pointing out that the spelling was corrected at some point?

placid flint
#

Why did someone change the correct French translation?

digital quest
#

likely because it looks like a misspelling

stoic inlet
#

I was looking to help translate Neofroge to Danish, but noticed that it was not listed as a language on the Crowdin instance. Is it possible for it to be added, so I can begin translation?

weary fossil
#

goes and locks the translation just in case

#

sure, what's the lang code of danish

stoic inlet
#

da or da_dk?

digital quest
#

yeah kinda what you need to do Maty lol

weary fossil
#

I need to make a discord command for adding new langs to crowdin lol

digital quest
#

or are y'all not using yaml at all?

weary fossil
#

I mean ideally we wouldn't have the mapping in the yml since if mojang/crowdin adds a new language we'd have to update a file in the repo

digital quest
#

which wouldn't exist until the new version of mc..?

placid flint
# 
  "neoforge.configgui.forgeLightPipelineEnabled": "NeoForge Light Pipeline Enabled",```

This translation key is weird, it should be

```  "neoforge.configgui.neoforgeLightPipelineEnabled.tooltip": "Enable the NeoForge block rendering pipeline - fixes the lighting of custom models.",
  "neoforge.configgui.neoforgeLightPipelineEnabled": "NeoForge Light Pipeline Enabled",```
or
```  "neoforge.configgui.forgeLightPipelineEnabled.tooltip": "Enable the Forge block rendering pipeline - fixes the lighting of custom models.",
  "neoforge.configgui.forgeLightPipelineEnabled": "Forge Light Pipeline Enabled",```
weary fossil
stoic inlet
digital quest
weary fossil
#

keep track of langs added. but sci configured it, I don't really care so take it up with him thinkies

digital quest
weary fossil
#

what does hiding a string do thinkies

digital quest
#

uhh that send seems to be the wrong tense

weary fossil
#

it is, a lot of network related messages and exceptions are probably the wrong tense

#

does changing the source in crowdin batch it in the next pr in en_us

digital quest
#

wdym changing source in crowdin?

#

doesnt it pull from github to grab the strings?

weary fossil
#

yes but it seems like you can also change it in the crowdin ui

weary fossil
#

@placid flint youre now a french proofreader

north pelican
#

yay my translations are merged

weary fossil
#

FYI: you might notice a legacyinstaller project, just ignore it for now

#

well, "project" in the loose sense of it, it's actually a subproject of the main project if that makes more sense

soft shadow
#

Hello, do you plan adding more languages into crowdin to be translated to?

lean snow
#

languages are added when someone asks for them aiui

soft shadow
#

Well, then i should rather ask if someone could add Slovak language there, i would like to contribute

brave hawk
soft shadow
#

Thanks!

sacred swift
#

Estonian, please. et_ee

brave hawk
sacred swift
#

ah nice

#

can you also please link crowdin on github 😄

#

nvm, made a PR

sacred swift
#

how does one even go to configgui in-game btw?

#

it's grayed out for NeoForge so that can't be it

image.png
floral walrus
#

there is no configgui by default

sacred swift
#

where is it then 😄

#

or why do the strings exist then

#

I know they existed in the Forge project as well but I think I never checked where they could be in-game

floral walrus
#

there are some mods that add a config screen for all mods

sacred swift
#

wait no

#

I'm talking about those things

image.png
#

that seem to be (Neo)Forge-specific

polar dust
#

Aren't those the Config comments?

#

*As in the raw file not the gui

sacred swift
#

I don't see an overlap
or maybe only for 1-3 strings, not most

image.png image.png
#

and their actual string mentions "configgui"

#

e.g. neoforge.configgui.removeErroringEntities

weary fossil
#

those are shown when a mod install config guis for neo

polar dust
#

Ah, then their there for mods that add blanket config guis for squirrsnipe

sacred swift
#

mm okay, I was able to see it work with the mod somewhat on Froge 1.19.4

#

but still some stuff are obsolete and some missing there

#

maybe they are remaints of 1.12 or something and nobody bothered to make new options' comments translatable? 😄

weary fossil
#

that doesn't make sense

#

plain text isn't translatable

sacred swift
#

well, yeah but from what I understood some of this plain text is generated from a resource string

#

here's how it looks for me

image.png
#

(froge is 90% translated, but this first and last string is not there)

weary fossil
sacred swift
#

well I never translated that forge mod
only forge itself

weary fossil
#

all have translations supposedly

weary fossil
#

no one bothered to translate the new config values in forge ig

#

since those two are rather new

sacred swift
#

so maybe it would make sense to remove the old ones and make new ones localizable?

#

since I copied most stuff from my froge translation anyway, it's not too wasted effort for me

#

but you wouldn't want new users to waste time on stuff that'll never be seen

weary fossil
#

i mean go ahead

#

if you want to spend the time i won't stop you 😛

sacred swift
#

orr... just hide every configgui string in the crowdin project haha

weary fossil
#

that doesn't make sense since some are actually used and can still be used by configured et al

sacred swift
#

woo, 💯% done

weary fossil
weary fossil
north pelican
#

are there gonna be roles for translators in the discord?

weary fossil
#

maybe for proofreaders

north pelican
#

ok

zenith cargo
#

Do we want to invent a convention for translation keys for dimension names?

#

I think I looked into this a while back and there are a couple of map mods that roll their own and have them for the most popular dimension mods, but no common convention

weary fossil
#

this is probably the wrong thread harold

#

#neoforge-github would be better

#

this is more related to how neo itself is translated

zenith cargo
#

fair enough, it seemed related - though we would want to ship translations for the vanilla dimensions if we did do that

patent prawn
#

Hello, can Portuguese language be added (probably both Brazilian and European)?
I'd like to contribute!

#

and do I need a crowdin enterprise account?

brave hawk
#

we'll look into adding Portuguese in a bit, sure
and you do need to make an account on the NF Crowdin Enterprise instance, though you should be able to link your Crowdin.com account IIRC

brave hawk
#

btw @weary fossil, how did you fix it so I appear on the contact page for the Crowdin?

weary fossil
#

demoted you

#

then made you a manager of the project

brave hawk
#

ah

weary fossil
#

then promoted again

#

tho the last step is optional

brave hawk
#

right, I think I can't be made Manager if I'm already an admin of the org iirc

bitter gust
#

I think Maty wanted to have you stay demoted judging from his name

stiff gust
#

So, I think we need to design this with the following goals in mind:

  • Single string (no json objects) so it can be edited on crowdin
  • Support for all styles
  • Support for spans of styles
  • Support for parameters and parameter re-ordering
  • Separation of format and text
  • Separation of vanilla-compatible and "neo format" strings (no "guess which one it is" code) for better compatibility
#

here's my first draft again

image.png
fierce copper
#

I would've proposed using a boolean at the start for determining what kind of format it is (which is how some other people do it), but not having Crowdin compatibility kind of sucks

stiff gust
#

yes, Crowdin is the big trigger for change here.

fierce copper
#

the reason why I think the variant where you add the #format one manually is weird is because it requires writing both the plain and formatted version, which is kinda eh

#

like right now it explicitly works so that if you try to get the translation string and not the Component it just .getStrings the component, which strips all coloring

stiff gust
#

it separates it into a pure formatting string that has no text and only needs to be edited for languages with word-order changes and a simply text that is eay to translate without having to understand format codes

fierce copper
#

oh it works like that

#

hmm

stiff gust
#

yes. there are few translators, so making it easier to translate is a good thing. if we had a professional translation studio, this wouldn't matter...

fierce copper
#

ye I get that

#

would translators see the two strings together though

#

on a platform like crowdin

stiff gust
#

next to each other, I think it sorts by key

#

give me a sec

#

I think it may be file order. not sure

#

in any case, those should be close together I think

fierce copper
#

imo it would be better for translators to just immediately see <gray>File: "<0>"</gray> instead of both the formats though
because we're probably going to have to explain this to translators anyways (I've seen a turkish person talk about the turkish translations for hypixel, which iirc use e.g. %%red%% for colors, and the translators did some incredibly weird things with them)
and I think that explaining an HTML-based format would be easier than explaining how the sidecar translation entry works

stiff gust
#

but then we run into the issue that we don't know which format a string is. we'd need to guess if it's in our format, and if we guess wrong, we destroy strings that target the vanilla format. Unless we use the same trick as my plural PR: make our codes invalid strings for vanilla by using %x as initiator

fierce copper
#

i mean, you could make it so that you can attach a #format to the end to signal that it's in the neoforge format

stiff gust
#

this also has another advantage: it is easy to patch in

fierce copper
#

didn't you have to change the actual stiring translation for this though
like i'd assume the original would be File: "%s"

stiff gust
#

no. what I did for plural was to employ an invalid %x sequence: example: "mymod.dogs": "There %n(one{is # dog} other{are # dogs}) here."

#

%n is invalid for vanilla, so I capture that and do "my thing" instead

#

"%f{gray:File: %s}" or romething like that

#

we just need to think about nesting tags, as parsing nested () is a pain and three halves

#

so I would suggest not allowing nested tags and using a character that's not used elsewhere

fierce copper
#

I wanted to do a more XML-like format because that seems to be where the string-based component formats are headed (see minimesage) but %f sounds like it could work

stiff gust
#

it'd be minimally invasive. But we could also do: "%f: <gray>File: <parens><param 0/></parens></gray>" <--- overly complex as example

#

i.e. use %f as a marker at the beginning and then diverting the whole parsing of the rest to our own parser

#

But then I'd also use ICU MessageFormat to parse it in addition to our style tags. This would automatically yield us better number formats and plural support

fierce copper
#

sounds good

stiff gust
#

should I start a branch and code the %f insertion? we can then work on the parser together

fierce copper
#

👍

#

should I close the docs PR if we're gonna rework it anyways

stiff gust
#

there's no other location we can put that warning....

fierce copper
#

eh no worries, it's not like I spent much time on it (and it has a bunch of review issues so eh)

#

i mean my point is that if it's a weird system then not documenting it would hamper use

#

hmm

#

i'll try to continue it then

#

i'll add the deprecation warning when a replacement is actually cooked up though

#

because 'deprecated with no replacement' is a bit weird

#

the part that most annoys me is that this system will now have to be supported because we thought about this way too late
admittedly I poked you on discord after you commented on the PR and you didn't respond, but you probably had a good reason not to

stiff gust
#

most likely that reason was I overlooked the message...

olive sage
#

what docs PR are we talking about here

stiff gust
#

it's a dud as it fails horribly on crowdin <--- simplified, sorry for the harsh way of saying it, Basique

fierce copper
#

i mean it's still useful for everybody who doesn't use crowdin ¯_(ツ)_/¯

#

i've had some people saying the feature is cool so we'd somehow have to say to the people who saw it and were excited "it's deprecated now we made a replacement"

stiff gust
#

https://github.com/HenryLoenwind/NeoForge/tree/i18next Here. There seems to be something in the json parser that filters out %f (and maybe more of the common %-codes), so I went with %n. Also, I don't think we can use MessageFormat as it either produces a String with all the parameters inserted or we'd need to copy&paste most of its contents to take over formatting. Also, I still can't make heads or tails out of what you're doing to add those styles. It really doesn't help that Eclipse just errors out when I ask it to tell me what calls what here. Those visitors seem to completely confuse it; it can't even jump to the superclass from those.

fierce copper
#

can't you configure the Gson for that

stiff gust
#

?

fierce copper
#

how does it deserialize the json

#

because i think the Gson object has a setting for url escaping or whatever

stiff gust
#

oh, the filtering? I didn't look into that. we can tweak the exact character later

fierce copper
# stiff gust <https://github.com/HenryLoenwind/NeoForge/tree/i18next> Here. There seems to be...

okay lemme explain how it works
the patch into Language adds an optional argument that accepts entries of translation key to component
and adds a method that gets a component from a translation key
the clientlanguage patch also just uses that argument and stores the components and provides them via the method
then the translatable contents patch, when trying to load the translation, checks if there is a component translation
and if there is it just uses the component as the singular part in the array
and when visiting it sets a threadlocal that says "hey i'm visiting this specific translation"
then inserting contents just grabs that threadlocal and gets the argument from there
otherwise it just output a string that corresponds to the vanilla argument insertion format

stiff gust
#

for now, my main question is if net.minecraft.network.chat.contents.TranslatableContents.decompose() ever gets called on something that is then serialised again. I wouldn't think so, as it used the language file to look up the String. So we could return or own FormattedText with whatever we parse

fierce copper
#

wdym?

#

decompose shouldn't affect serialization

#

more specifically the relevant parts should be run when the Language changes

#

we will still require a Language patch with this setup btw

stiff gust
#

what I mean is that decompose is where my hook is. the hook is supposed to produce FormattedText. Later, when the list of FormattedTexts decomposedParts is visited, something something style?

#

i.e. we could produce FormattedText objects that contain the style we parse?

fierce copper
#

because I18n should ideally output non-colored text

#

without the %f stuff

#

(this is specifically tested in my PR)

stiff gust
#

I18N is what's used on the server? If so, I tested my plural patch and it worked for the server log

fierce copper
#

it's a client only class

#

that just gets the translation from the Language and uses string.format

stiff gust
#

ok, then put that back until later. patching a filter in there is easy. What's hard is to produce styled text.

fierce copper
#

you just do the MessageFormat stuff on decompose

#

and all the style should be in the parts

stiff gust
#

ok, we're at very different stages of understanding the vanilla code. How about you add something to my hook that simply makes the whole text red. no parsing yet. The parsing part I have an idea about, it's the styles that I have no idea.

#

I can give you push access to my fork, if you want?

fierce copper
#

uh i'll do it tomorrow
2:39 AM 😢

stiff gust
#

np, nearly as late here

stiff gust
#

🤦 got it; it's way easier than I imagined.

stiff gust
#

this is too easy...

image.png
stiff gust
stiff gust
#

Basique, I got a bit carried away and finished the parser. Sorry.

My test string is "neoforge.configuration.title": "%n <blue>%s</blue><color #7700ff00>gr<underline>ee</underline>n</color> (There <1:plural:one{is # dog} other{are <blue>#</blue>\\> <italic>dogs</italic>}> here.) <ref:fml.menu.mods.config>",, it takes one numeric parameter that I temp-patched in at net.neoforged.neoforge.client.gui.ConfigurationScreen#ConfigurationScreen() / line 262. Also, a translatableformatexception.printStackTrace(); in TranslatableContents line 117 is really, really helpful.

#

I also included the plural format while I was at it. It's using the ICU format, that's why it may look weird, but that's pretty much standard and I see no reason to roll our own.
Should we also include a normal number format? <0:number:##.###>?
Also, <ref:...> or <include:...>?

stiff gust
#

open TODOs:

  • %n -> %f. needs a patch to eliminate the error in the json reader
  • I18N/other places that may access the raw language value
  • an automated test
  • convert all json-format keys in neo's langfile (I volunteer)
  • docs
  • other formatters? numbers, dates, spellout, ordinal, duration?
  • report exceptions to the logfile? (vanilla doesn't)
  • other Style attributes? (click, hover, font)
brave hawk
#

the thread lives!

#

also, screm that I didn't anticipate the 'expanded' translation format coming into conflict with Crowdin

fierce copper
stiff gust
#

ok, I had a look at that list. Most of what I don't have is something I would say is out of scope for neo (rainbow, gradient, ...) or not suitable for a translation value (click, hover, insert). newline already can be done with \n where it can be used. I would add <key>, maybe font (have to look at how this works), and the decoration aliases (b, i, em, grey, ...)

brave hawk
#

document carefully what parts of the MiniMessage format we implement, if we do not implement the format in its entirety

stiff gust
#

I'm currently typing this out, you have clairvoyant timing...

brave hawk
stiff gust
#

I created the draft PR. Still a couple of minor TODOs (see above), but what is there is 99% done.

brave hawk
#

pls post here, for pin

stiff gust
brave hawk
#

embed or nah?

brave hawk
#

thanks

gaunt violet
#

rich translations 2? what was wrong with the first merged PR?

brave hawk
#

Crowdin and other translation platforms are ill-equipped to deal with the previous PR's format

#

and because we use Crowdin for translations, it is imperative our i18n formats are compatible with Crowdin

#

FYI Henry: you should link the MiniMessage format page on the PR desc, as reference

stiff gust
#

can do

gaunt violet
#

would it just throw errors for anything with a rich translation or something?

weary fossil
#

what about MaxiMessage

stiff gust
#

people were asked to translate "1" and "red", but they couldn't reorder parameters

#

to illustrate (pseudo format): people could translate a <red>%s</red> %s to eine <rot>%eins$s</rot>, %null$s but not to eine %1$s, <red>%0$s</red>

stiff gust
#

one question to all: Do we care about these locations (not) being able to handle translation strings in the new format?

image.png
#

I'm pretty sure none of them could even take Styles for their texts

neat canyon
stiff gust
#

I'd say "The format is inspired by html and MiniMessage. It supports:" is quite clear...

neat canyon
#

I mean more in eventual documentation than in the PR description (end users will probably never see that)

#

Just leaving my two cents as an adventure maintainer who has dealt with confused users from other non-standard MM impls 🙂

#

I think it's a great idea to make the general format more universal, just want to avoid confusing users

stiff gust
#

ok. The PR description usually is a template for what the documentation should be. (at least that's how I write it...)

brave hawk
stiff gust
#

btw, I based it on html/xml first and only incorporated some tag names from MiniMessage today. There's one major difference, where MM is linear like vanilla §-tags, my format is recursive like html. <red>x<blue>y<reset>z works in MM, but it'd be <red>x</red><blue>y</blue>z in mine

neat canyon
#

MM isn't linear like legacy text, it parses to a tree

floral walrus
#

this XML syntax is really disgusting 😄

neat canyon
#

<red>x<blue>y<reset>z == <red>x<blue>y</blue></red>z

#

and it will parse into an equivalent Component representation

#

it can insert closing tags implicitly (ie you close the parent of the current tag, it will know to close current as well), and reset is just a shorthand to close all open tags

<red>x</red><blue>y</blue>z would also work and produce a visually equivalent result, but a different Component tree structure (although by default MM will 'compact' output components to the smallest visually equivalent representation, so it may end up the same)

stiff gust
#

So, any opinions on Language.getInstance().getOrDefault()? At the moment this gives you the raw string. Should I insert a hook so it gives you a stripped version of the formatted text that also has the argument markers replaced by %s?

stiff gust
floral walrus
#

what is hard about parsing JSON inside the string?

#

we already have hooks to parse the extended format

#

that looks like a suitable place to inject a string -> json conversion

stiff gust
#

it's not about parsing it, it's that we then have an object in our hand when we're being asked for a bunch of FormattedText. so we'd need to iterate over those objects to convert them to FormattedText.

floral walrus
stiff gust
floral walrus
#

I want the "SJSON" option to at least be on the table

#

then we can consider what is the best; personally, I find the XML-like syntax out of place, and we would be adding a second system (realistically we'd want to deprecate the json array-based one ASAP, but still for now we have it)

#

I will also note that the "SJSON" syntax reuses components for its semantics, giving it:

  • familiarity with other uses of components and their syntax
  • extensibility? (not sure if Component are all that extensible right now)
weary fossil
#

components aren't exactly extensible no

fierce copper
#

SJSON?

#

like the main complaint (and one thing I didn't like about the JSON-like system) is that it's clunky

#

and just stringifying the component json format will be real clunky

fierce copper
fierce copper
#

okay i've talked about this with some other people (the main owo-lib dev, more specifically) and I've come back with some thoughts:
should the JSON-based format be completely discarded?
like there are already a bunch of people who know how this is used (minecraft resource pack makers mainly) and most mods don't use crowdin (which is why, while quite a bit of people have used the JSON format, nobody noticed this issue)
so while it would be nice for a more crowdin-friendly format this basically requires mod makers (who probably already know how the component format works) to learn a new format, thought it is thankfully quite simple (and I think this is due to it being XML-like - stringified JSON would be even clunkier than just JSON)

#

hm

#

I guess the clunkyness of the JSON format (and the crowdin incompat of course) need to be solved though so this would probably be the way forward, yeah

#

~~but then I'd have to implement a version of this format on Fabric or keep the mixins for implementing the JSON format on neoforge once it gets removed 😭 ~~

fierce copper
#

like the reason I was constantly pointing to minimessage is because I don't want people to have to figure out yet another stringified text component format

#

so having it be maximally similar to minimessage would be a huge plus

stiff gust
# fierce copper yeah most likely

ok, I'll do that then. Needs some structural rework as I don't want to double the parsing logic, but I just figured out how to do that.

stiff gust
stiff gust
# fierce copper yeah most likely

done. I also added a test, duplicating yours. Sorry for the finals added to your test, I forgot to disable autoformatting in the tests project. From my end, this looks pretty final, do you have any more suggestions? Also, please review the code, you know the translation engine better than I.

As I said, I do not want to add autoclosing to tags as it would be a mess to define which tags can close which other tags and would require backtracking as there are constructs that cannot be parsed without looking at what comes later. I also left out the features that are only useful for chat messages. Defining those in a language file would be a bit weird, in my opinion. If a mod wants to display rich chat messages, they can do that on their own.

I will add the aliases for <ref> to match MM now, I just noticed these now. done

fierce copper
#

because you can put a translatable component in the sjson

stiff gust
# fierce copper because you can put a translatable component in the sjson

one of the issues I have with the json format is that that's not obvious or discoverable. I went through the deserialisation code and still missed that. For the second PR, I needed to dump the deserialisation result to find out what I get... (but that's not a discussion point, just a remark.)

What is a point for discussion:

I made the 3rd variant, "registerable parsers" ( https://github.com/HenryLoenwind/NeoForge/commits/i18variant/ ), 3 days ago just for fun, not as an earnest suggestion. However, the more I think about it, the more I come to the conclusion that it is the best solution. This variant uses a marker in the translation string that contains a ResourceLocation and a way to register parsers in code. The advantages are that it wouldn't cement a single variant format to vanilla's but keep open what's added by neo, and also allows mods to register their own formats. And those registered Parsers would be independent of the consumer of the translations.

"key1": "%n(neoforge:html) <red>boo</red>",
"key2": "%n(neoforge:sjson) [{\"text\":\"boo\",\"color\":\"red\"}]",
"key3": "%n(mymod:bbcode) [red]boo[reset]",
"key4": "%n(othermod:markdown) **boo**",
"key5": "%n(mm:mm) <red>boo<reset>",
"key6": "%n(rtfmod:rtf) {\\rtf1\\ansi{\\fonttbl\\f0\\fswiss Helvetica;}\\f0\\pard This is some {\\b bold} text.\\par }",

If that would go in, the actual format parser we put into neo doesn't really matter, as anyone would be able to install a mod with any format they like.

Opinions?

raven storm
#

I think the biggest priority should be that it works for the cases in neoforge or that might be in neoforge -- the one I know of here is sensible plural handling cause, you know, the fluid units PR could really use that. Basically, whatever solution you go with should, as provided in neoforge, work with plurals. For me, that rules out the strigified JSON option, but the other options don't seem bad.

stiff gust
fierce copper
#

like imo the fact you have to prefix everything with %n already is a bit meh

#

but ideally if you make it so that you can also add custom parsers this should really be something that applies to the entire lang file

#

because constantly adding %n{adventure-platform-neoforge:minimessage} (doesn't seem to exist right now, but it could be added) would be kinda meh

#

like i understand that there will be problems with crowdin but usability by developers should also be a goal

brave hawk
#

oh dear.

image.png
#

I think it seems it is difficult to obtain agreement on what variant of non-string translation format to implement
and because we ought not to block proper translations of NeoForge's strings, I think what we ought to do for now is to fix up the en_us.json in NeoForge itself to remove all embedded JSON objects and return to regular string-only

#

my priority here, as a maintainer, is to make the translations process be workable for translators that want to volunteer their time to our project
mods can choose to use the 'extended' format on their own, in line with their own decisions on translations, but for NeoForge, we are committed to Crowdin and we therefore need our language files to be consistent with what Crowdin's interface expects

floral walrus
#

I agree. Removing non-vanilla extensions from our translations is a good idea

brave hawk
#

@stiff gust given you're the architect of the three outstanding translation format systems PRs and the config screen which uses the currently-implemented extended format system, would you agree with the above?

stiff gust
#

not quite. While I agree with the conclusion of "having nothing is better than a having broken system", I don't see difficulty to getting an agreement as the cause. Rather, I would blame the lack of constructive suggestions: At the moment, we have a solution that was created collaboratively by multiple people here, a veto to it, a single person's suggestion nobody else liked, and not much more. Aside from the one suggestion mentioned before, not a single person has brought up any suggestion on how to solve this. Instead, all I see are "I don't like this" and silence.

(And, by the way, I do consider having a single system that has multiple problems, i.e. what we have now, a bad thing as long as there is no API that allows people to use their own implementation. Having those patches in places that are critical for injecting any other system is not a good thing. But that's slightly off-topic to the current issue.)

Anyway, if so wanted, I can go ahead and replace the existing markup with hardcoded colours and translation strings that are further broken down into smaller parts. We will lose some functionality with the warnings and RTL-languages, but that can't be helped. No problem at all, aside from the fact that I'm busy today and Friday evening. With some luck I can work on this tomorrow afternoon, but Saturday is more likely.

brave hawk
#

in my opinion, speaking from the same standpoint as my message above, the solution I would like best would involve minimal to no use of stringified JSON because that is just too unwieldy for both developers (who'd have to deal with serializing Components) and translators (who'd have to learn JSON and Component format, probably while editing in a limited interface) alike

#

and yes, if you can, I'd like for you to adjust things so the en_us.json file has string values only

stiff gust
#

indeed, sjson is by no way friendly to use for anyone. That's why we need any kind of markup language. SGML as a base seemed the most logical as many people have basic understanding from knowing a bit of that SGML-dialect called "html", it is by no means the only one that was suggested. But nobody has come forth and said they liked bbcode, rtf, markdown, or any of those I listed more than what we worked out here weeks ago. All I got was that people really, really hate "<" and ">".

brave hawk
#

setting aside my wavering thoughts on whether colors or other special things have a place in language files at all, while SJSONified Components are flexible, they miss the point that the translation keys in language files should be concerned with a single task: take strings, and make them localizable

the inputs to such strings are necessarily hardcoded, since the developer who writes them has an intention in what each string expresses, and what arguments they pass to it to make it express it express better

we can add additional small tasks to that main task -- colors, perhaps, or even plural-aware strings -- but anything that accesses the full power of Components (which is necessarily possible when dealing with stringified JSON Components) is too much in my opinion

floral walrus
#

There is a non-zero chance that translators mess up the formatting of any custom markup language

stiff gust
#

and there is a non-zero chance that programmers mess up the syntax of Java when trying to write mods.

floral walrus
#

Personally I think that it's reasonable to keep the current format, and just not use it in NeoForge so that we can continue to use crowdin

brave hawk
#

there is a non-zero, and likely even higher, chance that translators (who are not expected to be developers) will mess up the formatting of a stringified JSON component
(since that makes them need the knowledge of JSON syntax and Component syntax)

floral walrus
#

Yeah I don't think that sjson is a good idea

stiff gust
#

you mean modders that do not care about translators and could just as well hardcode the strings inside their code.

brave hawk
#

at the very least, I think we can agree that because of our use of Crowdin, using any form of extended syntax that isn't a string value in our language files is not great

floral walrus
#

No? If they handle translations with plain old PRs they might be fine with the array format

brave hawk
#

(unless someone finds a way to teach Crowdin how to handle that extended syntax, but that's a whole 'nother can of tomatoes)

stiff gust
#

as Crowdin would need to allow reordering of json array elements to match word order...yeah, zero chance. not even a sliver.

#

all that Crowdin allows is defining patterns so they can syntax check placeholders

floral walrus
floral walrus
stiff gust
#

I just wanted to show ow irrelevant your statement was. Anything that has any syntax can be messed up. Translators also can mess up %1$s notation

floral walrus
#

The problem is that we have no checking of the syntax

#

%n isn't too difficult usually. More custom stuff might be

stiff gust
#

should be relatively easy to write a gradle task to check. my markup parser is pretty mc agnostic and should be able to be used without a running mc instance (at least the "to text" variant)

floral walrus
#

If we have an automated check that we can run on the translation PR we can fix a lot of problems

#

It doesn't have to be MC agnostic. We can likely write a gametest or something for it

stiff gust
#

(I hate the "sdx" cluster on keyboards. text-test fixed above)

#

I prefer a task we can include for modders in the normal gradle build process, not just use in neo or supply for those few people who set up gametests for their mods

floral walrus
#

I'd prefer if the parser would directly output a Component

#

We can easily expose such a check in our testframework or NeoForge directly

stiff gust
#

we can't output a component directly, that's not how the vanilla translation system works. Translation happens at the very last moment from Component + translation to FormattedText. The json hack to get around that is pretty wild and, in my opinion, unsustainable long-term. Outputting a series of FormattedText when the conversion happens (i.e. where I hook in) is the best compromise

#

Or in other words: The translations from the lang file come into play when Components are decomposed, not when they are composed.

#

However, if you look closely, the parser does produce a list of FormattedText that does include Components (literal ones).

#

ok, when I go ovr the translations to de-json-ify them, I'll also have a look at hooking up a mass test. I'll do that at the "registry" one, as I think that's the best architectural basis and is independent of the actual parser.

#

Oh, PS: Would a PR with just the parser registry and no default parser have a chance? (i.e. PR1470 without the file "JasonTemplateParser.java" and its associated test and the lang file changes)

floral walrus
#

We can of course output a Component

#

Why couldn't we?

#

Just hook wherever the sjson would have been hooked

#

Ah, didn't read everything

stiff gust
#

as a side note: the json solution is pretty hacky. In order to allow for parameter substitution while executing when the lang file is read, it uses stacks in Threadlocals to do late binding. That's why all my solutions hook into the code that decomposes a Component at a time when the parameters are available. That way I don't need miles of patches, Threadlocals, or such things, as this is exactly the moment vanilla would do the same thing. (sjson also hooks into this instead of using the available parameters as that was easier to implement and I didn't expect sjson to make it anyway)

floral walrus
#

Ok then I approve of the approach. You probably know more about the details than me. Things I'd like to see: (up for discussion of course)

  • Unit test system that checks that the translation keys makes sense. E.g. just see if converting to FormattedText works. I'd want it to check our specific markers, not just the basic structure of the format. The junit tests should have access to the client so that wouldn't be a problem.
  • Bonus points if mods can implement a similar unit test with a simple function call.
  • Keep it simple. Writing parsers is fun but the main goal is to turn a string into a Component/FormattedText. 🙂
  • (Can be in another PR) Deprecate the array syntax for removal, with a log message that warns about future removal.
#

We should probably discuss this with @chilly acorn in case he's not aware of the issues with the component array syntax

#

TLDR the component array syntax is not good for translations via crowdin

brave hawk
#

(also, the reason why I brought this back up was because I was doing an inspection of the Crowdin translations PR for potential merge, which I'm now deferring until after we fix the translations in Neo itself to be string values only)

stiff gust
#

Same order:

  • on my todo list (this weekend). Level of checking depends on the parser. The <red>text</red> parser can run with minimal references to mc (only need the ChatFormatting enum), the sjson parser requires mc to have started up at least half-way.
  • that's a given
  • that's why I have been concentrating on the parser registry lately
  • not so sure about this, but that's mainly because we don't have the replacement in place yet. But sure, I know where that log call needs to go, I can make that PR (Edit: I just had an idea how to allow decomposition parsers to work for array-json. I'll investigate that as supporting array-json via third-party parsers would provide a nice transition and allow people who like it to continue using it even if/when neo drops it.)
raven storm
#

Saw your later messages -- yeah, that makes sense

#

I'm not sure having something to check the parsability of translations is necessary per se, since, you know, vanilla translations' replacement format isn't checked in that sense till it's actually loaded -- but there's no huge disadvantage to that if it's plausible to do

stiff gust
#

Anyway, I managed to find time to write lang file test hooks.

            helper.assertValueEqual(TemplateParser.test(key -> true), Collections.emptyList(), "Rich translations valid");
            helper.assertValueEqual(TemplateParser.test("rich_translations_test", "en_us"), Collections.emptyList(), "Rich translations valid (file test)");
#

They return pairs of lang key and error message, so people can use those error messages in any way they see fit. The first one works on whatever language is loaded and takes a predicate, the second one loads a language file.

#

The rest will have to wait for Saturday

raven storm
fierce copper
floral walrus
#

Ok

#

And what did glisco say?

fierce copper
#

Pretty sure she didn't like the new syntax

#

uh

#

the issues here aren't really important to us owo-lib devs, mainly because we don't use crowdin 😅

#

I was planning myself to add an xml-based format via text placeholder api (since modmenu depends and JiJs it now) though

brave hawk
fierce copper
#

github PRs lmao

#

crowdin would be nice but I dunno if people would use it

#

plus the gh pr workflow means that I can look at the translations people submit (I have no idea how this would be done with crowdin)

#

||god damn are some of the people who write translations to russian incompetent||

brave hawk
#

so even if we don't use the proofreading feature in Crowdin (and the corresponding settings to only export proofread translations), we are still able to double-check the translations that are going in

fierce copper
#

right

weary fossil
fierce copper
#

and I can read some ukrainian or whatever

#

so these aren't really fun, they're just 'oh somebody is malicious, yay'

#

it's more fun when it's somebody who is russian and says that they're commited to translating mods makes a really bad translation

#

that looks like they know russian very poorly

weary fossil
#

oh I know the feeling

stiff gust
#

yes, Russian translations stand out but I've seen plenty of bad translations in other languages. I never understood how (presumably) native speakers can have such a bad grasp of their own language. But then, I'm sitting in a room I call a library, and it does have enough books to be called that in good conscience…

stiff gust
stiff gust
brave hawk
brave hawk
brave hawk
#

i should thonk on those four(!) translations PRs soon

fierce copper
#

mfw

brave hawk
#

my standards dictate I have to pass changes through Google Translate to ensure nothing... untoward gets through

#

sometimes I wish we had more proofreaders

polar dust
fierce copper
#

wait

#

drhesperus?

polar dust
#

For reference Russian Translator known for a) poor quality translations, b) sneaking in advertisements to their VK community into the translation

fierce copper
#

yeah I asked because I was asked to review a PR by the guy

#

and it was shit

#

didn't know he'd sneak in advertisements

polar dust
brave hawk
#

i need to cajole people into proofreading our translations

#

but perhaps after I do the ever-present task item of adding context to everything, and sorting out the unused translation keys

fierce copper
#

Turns off chopping, translated by the modherald community, if natural leaves aren't touching logs

#

what the fuck

polar dust
sly cloud
#

Hello, I'm not sure if this is the right place to mention it,
but it seems that the Traditional Chinese translation file has been mistakenly named zh_hk.json.
It should be zh_tw.json.

brave hawk
#

oh, you're right

#

I think when I looked at the list of languages on the Minecraft wiki, I saw first the entry for "Chinese Traditional (Hong Kong SAR; Mix)", and used that language code

#

but Crowdin itself gives the language Chinese Traditional the code zh-TW
(and it has Chinese Traditional, Hong Kong for zh-HK)

#

i've fixed the issue on the Crowdin side, so it'll sync with the PRs, which I merge on a semi-regular basis

brave hawk
sly cloud
#

BTW, is there a way to translate common tag on Crowdin?

brave hawk
#

at the moment? we don't have the language keys setup, no

#

one of the items we have on the TODO list; figuring out what's the best way to go about that, perhaps even communicating with Fabric in some way to sync our translations of the common tags

sly cloud
#

Understood. Thanks for your explanation.

fierce copper
#

clearly there should be a common common tags mod that's multiloader /j

brave hawk
#

one of the idea that was thrown about in the past was moving common tags to a separate org and 'team', which modding APIs such as Fabric and NF can pull from

#

but that's still an idea, and would probably take weeks of discussion to fully flesh out

fierce copper
#

the common letters between Fabric and NeoForge are Fr, so clearly it should be called FrMC

#

I think somebody was working on a common common tags mod though

brave hawk
#

anyone know Simplified Chinese? want someone to go through the current translations on Crowdin and check if they look fine

#

someone asked to be a proofreader for it on Crowdin, and I need a second opinion on things

brave hawk
#

@weary fossil could you give your crowdin account a first name, please, so I don't have to see , Curle, SciWhiz12,... in my emails/messages from Crowdin

brave hawk
#

note to self: need to check if Minecraft loads English translations as a fallback when another language is loaded
for the possibility that I can disable exporting untranslated strings from Crowdin, so the repo only contains translated strings

chilly radishBOT
#

Successfully scheduled reminder on <t:1725449541:F> (<t:1725449541:R>)!

chilly radishBOT
#

@brave hawk

Camelot
Reminder

see above note

sciwhiz12 ·
brave hawk
#

oop

olive sage
#

should be the case afaik?

brave hawk
#

I believe so

#

but I think to adjust that requires me to make a PR to adjust the crowdin file
which I don't want to do this evening
-# maybe

digital quest
#

Yes English is always loaded lol

#

Crowdin added that feature to json export because I asked for it thinkies

#

Least I think that was the one

brave hawk
#

neat

digital quest
#

tho Mek's crowdin github integration has just... stopped working... after we went over the plan limits (and then back under)

coral linden
brave hawk
#

thoughts: renaming all of those fml. translation keys currently in NF away from that prefix?

#

the reason for that prefix is historical: when FML used to be integrated into Neo/Forge, some functions such as networking were done in the name of FML
but when we did the banana peel and then the banana split, anything that wasn't core to FML's mod loading was passed to NeoForge

#

thinking about doing that alongside a general cleanup of unused strings

#

perhaps the cleanup of unused strings first, then the grand renaming come 1.21.2

olive sage
#

Does FML have its own language file?

#

Or more accurately, does FML provide any strings that need to be translated to begin with? And if yes, is it possible to define a en_us.json file for FML that will be read by the game if normal?

weary fossil
#

we have 3 repos that need to have translations updated

sly cloud
brave hawk
#

oop. I'll need to do the same rename fix for FML, right

cunning quartz
#

Hello! Does anyone know how to become a proofreader for a specific language in NeoForge? Or is the project not accepting applications? I actually messaged the owner of the localization project on Crowdin back on May 27, but I haven't received a reply.

brave hawk
# cunning quartz Hello! Does anyone know how to become a proofreader for a specific language in N...

hello! sorry, I only saw your message on Crowdin today and I'm pretty much the only active owner for the Crowdin project harold
current (informal) policy for getting proofreader status is either being someone that the team is familiar with (being around in the community, actively being involved in the project, etc.) and/or having someone who can vouch for one's translating abilities for a particular language (could also be a portfolio or list of projects you've translated that's verifiable)

#

(that doesn't stop anyone from contributing to the Crowdin project, though, since we don't restrict who can add translations)

cunning quartz
# brave hawk hello! sorry, I only saw your message on Crowdin today and I'm pretty much the o...

I see. Then it might not work out. I'm not very social, and I don't think there are any big names who could vouch for me. Of course, I've worked on translations for many projects like Fabric, Forge, Applied Energistics 2, and many others, but few people know about my contributions, mainly just friends.
I've already contributed to NeoForge, yes; I just also wanted to be a proofreader for my language in the project so there would be someone to verify translations. My community agrees that my translation approach is accurate, but it's just a small community. I'm not sure if there's a way I could further prove my reliability?

brave hawk
#

i mean, I don't need big names to vouch for you, just at least someone else (even your friends or people in your community) who knows your work and can tell me that your translations are good (and aren't doing something like inserting ads into translations thinkies)

#

( @cunning quartz )

#

I'm a bit strict on this for languages that none on the team (to my knowledge, at least) can read in, since we ourselves can't check up on what proofreaders approve unless we get someone else (that we know) to check for us as a favor

#

you do seem to be quite active in contributing Russian translations for other projects (based on your Crowdin profile), and that seems to be satisfactory to me
but if it's at all possible, I'd still like someone else you know to vouch for your translations

midnight hemlock
surreal quartz
#

hello, im deflecta's friend, i can confirm too, that his translations are good, in minecraft russian translation community too many machine translations, minecraft mods deserves muuch better than this

cunning quartz
brave hawk
#

done, you should now have Proofreader permission for Russian. thanks!