@pastel plank @cosmic totem I'm assuming you both help with translating
#Placeholders
quaint moon
I did a lot of translation applying in the backend in integrations, and I have rights to download the translations from lokalise, so I can test stuff locally
So I am known with lokalise, but I am far from an expert 😛
So maybe with some experimentating we can make something nice out of this
pastel plank
I think there are only 4 translation keys left across locales, which have problems with placeholders https://imgur.com/a/7aQ7ftZ
quaint moon
huh, why does it qualify as invalid placeholder while its empty
pastel plank
Using multilingual view, the filter is applied across 60 locales (languages). So one or more languages have problem with placeholder for specific translation key, for example translation key: component::nut::config::error::cannot_connect has problem with Danish, Korean, Lithuanian, etc https://imgur.com/a/xNm8Rdo
quaint moon
I found one that's interesting
"deprecated_voice": {
"fix_flow": {
"step": {
"confirm": {
"title": "\u0418\u0437\u043f\u043e\u043b\u0437\u0432\u0430\u043d \u0435 \u043e\u0441\u0442\u0430\u0440\u044f\u043b \u0433\u043b\u0430\u0441"
}
}
},
"title": "\u0418\u0437\u043f\u043e\u043b\u0437\u0432\u0430\u043d \u0435 \u043e\u0441\u0442\u0430\u0440\u044f\u043b \u0433\u043b\u0430\u0441"
}
component::cloud::issues::deprecated_voice::fix_flow::step::confirm::description
They used different brackets
let me try to fetch translations with the unfuzzy
diff --git a/script/translations/download.py b/script/translations/download.py
index 958a4b35a7b..8f7327c07ec 100755
--- a/script/translations/download.py
+++ b/script/translations/download.py
@@ -39,6 +39,8 @@ def run_download_docker():
CORE_PROJECT_ID,
"--original-filenames=false",
"--replace-breaks=false",
+ "--filter-data",
+ "nonfuzzy",
"--export-empty-as",
"skip",
"--format",
That Bulgarian problem is backtick vs single quote issue. The correct one is using backtick.
quaint moon
```json
"deprecated_voice": {
"fix_flow": {
...
oh wait
now I see
it did not fetch it
Do we have an example of an unverified value that doesn't have placeholders?
"deprecated_voice": {
"fix_flow": {
"step": {
"confirm": {
"description": "\u0413\u043b\u0430\u0441\u044a\u0442 '{deprecated_voice}' \u0435 \u043e\u0441\u0442\u0430\u0440\u044f\u043b \u0438 \u0449\u0435 \u0431\u044a\u0434\u0435 \u043f\u0440\u0435\u043c\u0430\u0445\u043d\u0430\u0442.\n\u041c\u043e\u043b\u044f, \u0430\u043a\u0442\u0443\u0430\u043b\u0438\u0437\u0438\u0440\u0430\u0439\u0442\u0435 \u0441\u0432\u043e\u0438\u0442\u0435 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0430\u0446\u0438\u0438 \u0438 \u0441\u043a\u0440\u0438\u043f\u0442\u043e\u0432\u0435, \u0437\u0430 \u0434\u0430 \u0437\u0430\u043c\u0435\u043d\u0438\u0442\u0435 '{deprecated_voice}' \u0441 \u0434\u0440\u0443\u0433 \u0433\u043b\u0430\u0441, \u043a\u0430\u0442\u043e \u043d\u0430\u043f\u0440. '{replacement_voice}'.",
"title": "\u0418\u0437\u043f\u043e\u043b\u0437\u0432\u0430\u043d \u0435 \u043e\u0441\u0442\u0430\u0440\u044f\u043b \u0433\u043b\u0430\u0441"
}
}
},
"title": "\u0418\u0437\u043f\u043e\u043b\u0437\u0432\u0430\u043d \u0435 \u043e\u0441\u0442\u0430\u0440\u044f\u043b \u0433\u043b\u0430\u0441"
}
Without the nonfuzzy it just returns it
so I like this change thus far
pastel plank
Do we have an example of an unverified value that doesn't have placeholders?
Try component::random::config::step::user::description
The English source added last year, and last month is changed because small typo: allow -> allows
quaint moon
what language to check?
pastel plank
Galician, Indonesian, Korean, Swedish, Vietnamese should be on the "Unverified"
quaint moon
Yep, isn't present
pastel plank
(Catalan, Italian also)
quaint moon
I tried to compare with the Dutch one, but that one has even less
{
"config": {
"step": {
"binary_sensor": {
"data": {
"device_class": "Apparaatklasse",
"name": "Naam"
}
},
"sensor": {
"data": {
"device_class": "Apparaatklasse",
"maximum": "Maximum",
"minimum": "Minimum",
"name": "Naam",
"unit_of_measurement": "Eenheid"
}
}
}
},
"options": {
"step": {
"sensor": {
"data": {
"device_class": "Apparaatklasse",
"maximum": "Maximum",
"minimum": "Minimum",
"unit_of_measurement": "Eenheid"
}
}
}
}
}
{
"config": {
"step": {
"binary_sensor": {
"data": {
"device_class": "Enhetsklass",
"name": "Namn"
},
"title": "Slumpm\u00e4ssig bin\u00e4r sensor"
},
"sensor": {
"data": {
"device_class": "Enhetsklass",
"maximum": "Maximum",
"minimum": "Minimum",
"name": "Namn",
"unit_of_measurement": "M\u00e5ttenhet"
},
"title": "Slumpm\u00e4ssig sensor"
},
"user": {
"menu_options": {
"binary_sensor": "Slumpm\u00e4ssig bin\u00e4r sensor",
"sensor": "Slumpm\u00e4ssig sensor"
},
"title": "Slumpm\u00e4ssig hj\u00e4lpare"
}
}
},
"options": {
"step": {
"binary_sensor": {
"description": "Denna hj\u00e4lpare har inga alternativ att konfigurera.",
"title": "Slumpm\u00e4ssig bin\u00e4r sensor"
},
"sensor": {
"data": {
"device_class": "Enhetsklass",
"maximum": "Maximum",
"minimum": "Minimum",
"unit_of_measurement": "M\u00e5ttenhet"
},
"title": "Slumpm\u00e4ssig sensor"
}
}
}
}
swedish
{
"config": {
"step": {
"binary_sensor": {
"data": {
"device_class": "Device class",
"name": "Name"
},
"title": "Random binary sensor"
},
"sensor": {
"data": {
"device_class": "Device class",
"maximum": "Maximum",
"minimum": "Minimum",
"name": "Name",
"unit_of_measurement": "Unit of measurement"
},
"title": "Random sensor"
},
"user": {
"description": "This helper allows you to create a helper that emits a random value.",
"menu_options": {
"binary_sensor": "Random binary sensor",
"sensor": "Random sensor"
},
"title": "Random helper"
}
}
},
"options": {
"step": {
"binary_sensor": {
"description": "This helper does not have any options.",
"title": "Random binary sensor"
},
"sensor": {
"data": {
"device_class": "Device class",
"maximum": "Maximum",
"minimum": "Minimum",
"unit_of_measurement": "Unit of measurement"
},
"title": "Random sensor"
}
}
}
}
Total number of key-value pairs in all JSON files: 55781
nvm, my script is wrong
Total number of key-value pairs in all JSON files: 608266
cosmic totem
Are you trying to count number of translated, unverified keys?
quaint moon
just the total of every key value pair with and without change
Total number of key-value pairs in all JSON files: 601065
cosmic totem
What do you mean with "change"?
quaint moon
Total number of key-value pairs in all JSON files: 608266
this is with the current setup
quaint moon
What do you mean with "change"?
adding the filter on non-fuzzy
quaint moon
Total number of key-value pairs in all JSON files: 601065
This is with the filter on nonfuzzy
so with this change we would lose 7000 translated keys
cosmic totem
So the former includes "unverified" and the latter excludes them?
cosmic totem
If any of those 7000 have placeholder mismatch, they are excluded runtime
quaint moon
If any of those 7000 have placeholder mismatch, they are excluded runtime
that's what's happening at the moment
but with this change, they won't even get in the build process
cosmic totem
Yes, so they should be substracted from your 7000
quaint moon
So whenever the source string changes, they become unverified
cosmic totem
Yes, I learned that part earlied today as well 🙂
I agree it sounds acceptable to exclude. However, should a note be posted on the developers blog?
quaint moon
I think so
cosmic totem
If we're excluding unverified, we can keep the code as is, as we would not expect placeholder mismatch
quaint moon
If we're excluding unverified, we can keep the code as is, as we would not expec...
we still expect mismatch, but not from core anymore
cosmic totem
It's much less likely that a custom component has mismatch at least
quaint moon
oh man
cosmic totem
?
pastel plank
so with this change we would lose 7000 translated keys
Across 60 locales, that sound reasonable
cosmic totem
Hmm - when are new base strings uploaded to Lokalize?
Translations will be lagging
Is Lokalise version aware?
I'm just trying to understand the data flow to grasp the consequences
pastel plank
Well, translations is always lagging anyway. And I don't think Lokailse is version aware (it complicates things AFAIK)
cosmic totem
So there will be mismatches in beta releases?
cosmic totem
Related question (I'm verifying Norwegian translations) - Lokalize does not regocnize placeholders in single quotes, so those can still be mismatch as well
quaint moon
cosmic totem
OK - on push to dev
cosmic totem
on:
workflow_dispatch:
push:
branches:
- dev
lolwat
cosmic totem
Check the yaml file
quaint moon
paths:
- "**strings.json"
So beta releases would not get mismatches with non-fuzzy filtering
cosmic totem
```yaml
paths:
- "**strings.json"
```
That's not relevant for when, only what
cosmic totem
Yes, no need to push strings when not changed…
quaint moon
but that's what threw me off haha
cosmic totem
So if a string is changed [or added - or deleted?], it will be pushed [to Lokalize I assume] when merged to develop
cosmic totem
Fair enough
quaint moon
Might also be a bit problematic sometimes
cosmic totem
In case they are revived?
quaint moon
I don't think there is an explicit reason
cosmic totem
Only a size problem
quaint moon
They skip out the integrations that don't exist anymore
cosmic totem
That should be most deleted strings
quaint moon
I think we could purge those
cosmic totem
From Lokalize? Or just when downloading?
quaint moon
from lokalise
as in, they're removed for a reason
so why bother keeping them
But I have to discuss this with Paulus and Frenck
cosmic totem
A lot of unverified changes are change of keys (to common keys)
So when verifying those, the correct thing seems mostly to be to "insert source"
quaint moon
ah nice
Describing it was difficult than I thought, so feel free to comment and suggest improvements
cosmic totem
Reviewed! Not pushing this for 2024.5.0?
quaint moon
Might as well include it 2024.4.2
cosmic totem
Related question (I'm verifying Norwegian translations) - Lokalize does not rego...
Looks like double quotes always should be used!
cosmic totem
quaint moon
When reviewing docs, please use the suggestion button haha
Then I don't have to copy paste it all
;p
cosmic totem
https://github.com/home-assistant/core/pull/114974
Damn, I broke a lot of tests…
I almost regret going down this path
quaint moon
Just move the hassfest to a separate PR
cosmic totem
Yes, but the broken tests were due to the quote change
I didn't run tests locally, because I didn't change code. What an idiot. 🤦♂️
quaint moon
I didn't run tests locally, because I didn't change _code_. What an idiot. 🤦♂️
It's weekend
cosmic totem
Hmmm. They all pass locally. Maybe it's the GitHub action that had problems
cosmic totem
Argh! I need to re-generate the local translation files!
cosmic totem
When reproduced locally it was trivial to fix 🙂
quaint moon
that's always the case haha
cosmic totem
Would you care to review again?
cosmic totem
I found the list of issues your PR will fix. Should be added to the description.
You are fixing 7 issues with adding 2 lines!
quaint moon
Hah
Like commented, unless there is really a bug
But then it should now happen consistently in every language
cosmic totem
Those 7 are placeholder issues
So they are not used in any case
But yes, "fixing"
"Removing the error messages"
quaint moon
But there is a change that it's a bug
When the English translation expects a placeholder
cosmic totem
Yup
quaint moon
But the placeholder isn't passed
Every time I read your name I get reminded the moment i fucked up git and you sent me a git guide via email
😂😂
cosmic totem
Every time I read your name I get reminded the moment i fucked up git and you se...
We aim to please
cosmic totem
OK, I see another way this error will continue to happen - it is common (actually encouraged) in translations to build the translation based on other keys. When the placeholders of any of those translations are changed, as https://app.lokalise.com/project/130246255a974bd3b5e8a1.51616605/?view=multi&search=key::common::config_flow::title::reauth, the translations using those strings will break.
"Encouraged" is at least how I originally understood rule 6
6. If text will be duplicated across different translation keys, make use of the Lokalise key reference feature where possible. The base translation provides examples of this underneath the states translations. Please see the Lokalise key referencing documentation for more details.
I have seen lots of translated strings that are built up this way, but I don't think that is a good idea.
E.g. adding {name} to common.config_flow.title.reauth seems to be the cause for a lot of these error reports.
Maybe we should change the log message to include a link to Lokalise?
quaint moon
E.g. adding `{name}` to `common.config_flow.title.reauth` seems to be the cause ...
Yea that was an interesting change
quaint moon
OK, I see another way this error will continue to happen - it is common (actuall...
hold up what
I don't completely follow
cosmic totem
E.g. it was previously used in the value here: https://app.lokalise.com/project/130246255a974bd3b5e8a1.51616605/?view=single&reference_lang_id=640&single_lang_id=748&search=component::pi_hole::config::step::reauth_confirm::title
The translation was [%key:common::config_flown::title%] PI-Hole
Actually bad example, as here the default string also used this key
My point is that I have seen several translations that uses keys not in the default string
I think this should be discouraged
quaint moon
Oh but we disabled that
As in, you reference the full key or you don't
[%key:common::config_flown::title%] is accepted by hassfest
[%key:common::config_flown::title%] Pi-Hole is not
in strings.json
quaint moon
You mean people use it in the translations
cosmic totem
Yes
A lot
And I think it is a bad idea
Should probably be added to the above rules
quaint moon
Hmm
How can we enforce that?
What did that parse to?
"step": {
"api_key": {
"data": {
"api_key": "Klucz API"
}
},
"reauth_confirm": {
"data": {
"api_key": "Klucz API"
},
"description": "Wprowad\u017a nowy klucz API PI-Hole dla {host}/{location}",
"title": "Autoryzacja wygas\u0142a dla {name} PI-Hole"
},
This was from the things fetched yesterday
cosmic totem
Yup
While the default string is Reauthenticate PI-Hole
Without placeholder
So hassfest has a rule that is not enforced in Lokalise
cosmic totem
Hmm - "a lot" may have been an overstatement
But definitevly done!
Probably not possible to enforce in Lokalise.
From the point of Lokalise, why have a key that only refers to another key, and no other text? The app should just use the same key in all places. (That does not work the way HA builds the keys dynamically of course)
quaint moon
Probably not possible to enforce in Lokalise.
We have contact with them
cosmic totem
Could you ask if it possible to enforce that rule in Lokalise then?
quaint moon
We could try, but I think I need to have a good explanation with good examples
as I am now just about grasping it
hahaha
And we should check why we enforced this in the first place
cosmic totem
It will at least be case the non-fuzzy filter will not catch
Or - what happens with a translation that uses a key that becomes unverified (fuzzy)? Will that translation also become unverified?
I guess figuring out why this is prevented in strings.json is a good first step 🙂
cosmic totem
This placeholder stuff is more troublesome than I expected.
quaint moon
@icy kernel
icy kernel
Seems some keys are on lokalize but not in dev: https://app.lokalise.com/project/130246255a974bd3b5e8a1.51616605/?view=multi&search='{
component::blink::exceptions::invalid_device::message does not exist in dev
quaint moon
Oh but that can happen
Keys are never deleted from lokalise
When you messaged me I thought you meant that the single quotes weren't updated
icy kernel
I thought the same, but it seems there are some orphans left with single quotes
pastel plank
My point is that I have seen several translations that uses keys not in the defa...
This is actually a bad example of a source string.
Concatenation should always be forbidden in software source strings (here 2 strings are concatenated: string with reference key, then string "Pi-Hole").
We never know the grammar rules (such as declension, article, etc) in the target language, so the concatenation doesn't always work.
So I suggest not to allow to use reference key unless it is for the whole string.
quaint moon
Yea that's what I found out as well
also in the company I worked at
Like
yes it has some benefit
but it just haves you end up with a translation string list with identity crisis
cosmic totem
@pastel plank That is enforced in hassfest, but not on Lokalise, so translations may do this
wind musk
🚀