Transifex for Pale Moon core strings

The l10n of Pale Moon. Rawr.
tmtisfree

Re: Transifex for Pale Moon core strings

Unread post by tmtisfree » 2016-01-02, 17:16

Moonchild wrote:I think I did but to be certain I'm refreshing the strings now from the latest langpack.
Thank you.

tmtisfree

Re: Transifex for Pale Moon core strings

Unread post by tmtisfree » 2016-01-04, 14:14

Hi,
Moonchild wrote:Awesome!
I'll have a closer look at it when I have time, and merge the reviewed strings in with the current language pack setup.
The only problem is that I still need to write the tool to split the monolithic files back out ;)
I have completed the translation of the remaining string for French on Transifex.

As I see no obvious or simple way to convert core.dtd + core.properties to a .xpi file myself,I suppose you now have those miraculous tools to get it back for Pale Moon v26, so that we can test our work?

Cheers

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35481
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Transifex for Pale Moon core strings

Unread post by Moonchild » 2016-01-07, 01:01

If it wasn't for Babelzilla not allowing full language packs we wouldn't be in this mess needing tools to combine and then split again after translating for Transifex (their command-line tool has not been cooperative either).
So no, there's no easy way for you to do this yourself, unless you want to use the tools that are posted in the palemoon-localization repo which aren't documented and catered to my workflow ;)
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

tmtisfree

Re: Transifex for Pale Moon core strings

Unread post by tmtisfree » 2016-01-07, 09:08

Moonchild wrote:If it wasn't for Babelzilla not allowing full language packs we wouldn't be in this mess needing tools to combine and then split again after translating for Transifex (their command-line tool has not been cooperative either).
Why have you merge all these files in the first place (a Transifex limitation in the number of allowed uploads perhaps?)? Transifex allows translators to merge all files (virtually) if needed (to bulk edit, fast review, etc).
Moonchild wrote:So no, there's no easy way for you to do this yourself, unless you want to use the tools that are posted in the palemoon-localization repo which aren't documented and catered to my workflow ;)
The tools I see on Github (https://github.com/MoonchildProductions ... /transifex) are just the ones to merge. As there are no helper file generated with start+end flags and a tree to known what comes from where, yes, I can't see how to split them back. ;)

But probably, you have already automated the split process for all languages, meaning the resulting .xpi files are already somewhere on your HDD only waiting for us translators to FTP them? :)

Cheers,
TMTisFree

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35481
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Transifex for Pale Moon core strings

Unread post by Moonchild » 2016-01-07, 11:04

tmtisfree wrote:Why have you merge all these files in the first place (a Transifex limitation in the number of allowed uploads perhaps?)? Transifex allows translators to merge all files (virtually) if needed (to bulk edit, fast review, etc).
Because I didn't want to upload 260 files individually for each language. That is way too much unneeded work.

And I noticed I didn't actually commit the tools to the repo yet. Done so now.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

tmtisfree

Re: Transifex for Pale Moon core strings

Unread post by tmtisfree » 2016-01-07, 20:01

Moonchild wrote:
tmtisfree wrote:Why have you merge all these files in the first place (a Transifex limitation in the number of allowed uploads perhaps?)? Transifex allows translators to merge all files (virtually) if needed (to bulk edit, fast review, etc).
Because I didn't want to upload 260 files individually for each language. That is way too much unneeded work.

And I noticed I didn't actually commit the tools to the repo yet. Done so now.
Thanks, I finally manage to get a working French .xpi file! Some points:

1/ the "split.exe" file does not work on x64 machine (I recompiled the split.pb code to a x64 target with administrative rights to get it working, see attachment)
2/ in the install.rdf file, I have to put 25.9 as a minimum version to have Pale Moon 26.0.0b4 accepting the .xpi as compatible
3/ the new .xpi seems to be missing some files (they are not in core.dtd and core.properties in fact):

Code: Select all

\chrome\fr\locale\fr\global\xpinstall\xpinstall.properties
\browser\chrome\fr\locale\browser\safebrowsing\phishing-afterload-warning-message.dtd
\browser\chrome\fr\locale\browser\safebrowsing\report-phishing.dtd
After installing the .xpi, I see some issues with strange formatted shortcuts and some &brandShortName strings all over the place, but as it is not an official .xpi built I will just test the translation for the moment (also put as attachment anyway).

Cheers,
TMTisFree
Attachments
fr_v26.0.xpi
fr_v26.0.xpi French translation file for Pale Moon 26
(331.2 KiB) Downloaded 57 times

tmtisfree

Re: Transifex for Pale Moon core strings

Unread post by tmtisfree » 2016-01-07, 20:03

Here is the split.exe in a Zip file for Windows x64.
Attachments
split.zip
split.exe for Windows x64 machines
(8.47 KiB) Downloaded 40 times

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35481
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Transifex for Pale Moon core strings

Unread post by Moonchild » 2016-01-07, 21:54

xpinstall.properties is not in use. I guess the fr langpack as grabbed from Mozilla never cleaned that up.

As for the safebrowsing... that's an interesting issue. Since we don't use or build Google safebrowsing, those locale files never end up in the language pack, so they aren't part of the source strings... Kind of a special case.
You don't really have to worry about those strings -- we'll not use them ourselves so they will be static. As-is you'll have to merge the split-out files with existing files, anyway, since the transifex files only include .dtd and .properties, and none of the other files in the langpack.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

tmtisfree

Re: Transifex for Pale Moon core strings

Unread post by tmtisfree » 2016-01-08, 01:04

Moonchild wrote:xpinstall.properties is not in use. I guess the fr langpack as grabbed from Mozilla never cleaned that up.

As for the safebrowsing... that's an interesting issue. Since we don't use or build Google safebrowsing, those locale files never end up in the language pack, so they aren't part of the source strings... Kind of a special case.
You don't really have to worry about those strings -- we'll not use them ourselves so they will be static. As-is you'll have to merge the split-out files with existing files, anyway, since the transifex files only include .dtd and .properties, and none of the other files in the langpack.
OK.


Please note 2 problems with the merged files downloaded from Transifex (all languages):

1/ in all "core.dtd" files (that is, all languages) on Transifex, 2 strings from the original \AB-CD\browser\chrome\AB-CD\locale\browser\preferences\preferences.dtd were mangled when you combined and uploaded the dtd files to Transifex. Specifically (for French for example), the following lines does not exist anymore in core.dtd file:

Code: Select all

<!ENTITY  buttonForward.tooltip   "Avancer d'une page">
<!ENTITY  buttonBack.tooltip      "Reculer d'une page">
Example:
single French preferences.dtd from Pale Moon v25.8.1

Code: Select all

<!-- This Source Code Form is subject to the terms of the Mozilla Public
   - License, v. 2.0. If a copy of the MPL was not distributed with this
   - file, You can obtain one at http://mozilla.org/MPL/2.0/. -->


<!ENTITY  prefWindow.titleWin "Options">
<!ENTITY  prefWindow.titleGNOME "Préférences de &brandShortName;">
<!-- When making changes to prefWindow.styleWin test both Windows Classic and
     Luna since widget heights are different based on the OS theme -->
<!ENTITY  prefWinMinSize.styleWin2 "width: 49em; min-height: 38em;">
<!ENTITY  prefWinMinSize.styleMac "width: 52.5em; min-height: 50em;">
<!ENTITY  prefWinMinSize.styleGNOME "width: 52.5em; min-height: 40.5em;">

<!ENTITY  paneGeneral.title "Général">
<!ENTITY  paneTabs.title "Onglets">
<!ENTITY  paneContent.title "Contenu">
<!ENTITY  paneApplications.title "Applications">
<!ENTITY  panePrivacy.title "Vie privée">
<!ENTITY  paneSecurity.title "Sécurité">
<!ENTITY  paneAdvanced.title "Avancé">

<!-- LOCALIZATION NOTE (paneSync.title): This should match syncBrand.shortName.label in ../syncBrand.dtd -->
<!ENTITY  paneSync.title          "Sync">

<!ENTITY  buttonForward.tooltip   "Avancer d'une page">
<!ENTITY  buttonBack.tooltip      "Reculer d'une page">
single French preferences.dtd for Pale Moon v26.0.0.3 after split from core.dtd

Code: Select all

<!-- This Source Code Form is subject to the terms of the Mozilla Public
   - License, v. 2.0. If a copy of the MPL was not distributed with this
   - file, You can obtain one at http://mozilla.org/MPL/2.0/. -->


<!ENTITY  prefWindow.titleWin     "Options">
<!ENTITY  prefWindow.titleGNOME   "Préférences de &brandShortName;">
<!-- When making changes to prefWindow.styleWin test both Windows Classic and
     Luna since widget heights are different based on the OS theme -->
<!ENTITY  prefWinMinSize.styleWin2      "width: 49em; min-height: 38em;">
<!ENTITY  prefWinMinSize.styleMac       "width: 52.5em; min-height: 50em;">
<!ENTITY  prefWinMinSize.styleGNOME     "width: 52.5em; min-height: 40.5em;">

<!ENTITY  paneGeneral.title       "Général">
<!ENTITY  paneTabs.title          "Onglets">
<!ENTITY  paneContent.title       "Contenu">
<!ENTITY  paneApplications.title  "Applications">
<!ENTITY  panePrivacy.title       "Vie privée">
<!ENTITY  paneSecurity.title      "Sécurité">
<!ENTITY  paneAdvanced.title      "Avancé">

<!-- LOCALIZATION NOTE (paneSync.title): This should match syncBrand.shortName.label in ../syncBrand.dtd -->
<!ENTITY  paneSync.title          "Sync"><!--##@@\workdir\en-US\browser\chrome\AB-CD\locale\browser\preferences\privacy.dtd@@##-->
Note the last line. As the combine script removed the 2 lines, it also put the flag line of the next dtd file (privacy.dtd) not on a new line but after the last string of the last line of preferences.dtd, as follow:

Code: Select all

<!ENTITY  paneSync.title          "Sync"><!--##@@\workdir\en-US\browser\chrome\AB-CD\locale\browser\preferences\privacy.dtd@@##-->
So, when I split the core.dtd file, the privacy.dtd file was not generated: it was kept at the EOF of preferences.dtd resulting in a XML parsing error in Pale Moon when I tried to open the main Settings.
The issue arose probably because there is no empty line at the end of the original preferences.dtd file, which fooled the combine script.

To see the problem in whatever language core.dtd file on Transifex, download one and search for the string <!--##@@\workdir\en-US\browser\chrome\AB-CD\locale\browser\preferences\privacy.dtd@@##-->.


2/ The second point. If you compare the 2 codes above from preferences.dtd in Pale Moon v25.8.1 and from preferences.dtd in Pale Moon v26.0.0.3, you will see that the v26 version (from Transifex) has the &brandShortName; changed as &brandShortName;. Unfortunately all ampersand, < and > (and perhaps other) in core.dtd (not in core.properties strangely) have been expanded as "&" "<" and ">", respectively.

When I manually correct for all these issues (together with adding the 2 lost strings), all is fine and dandy (except the Sync feature is broken).


I'm not sure if what I wrote makes sense, so feel free to ask for details if needed.


Edit: it seems core.properties is also slightly affected in the xul.properties file:

after split in \chrome\fr\locale\fr\global\xul.properties:

Code: Select all

# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.

MissingOverlay=Le chargement d'un overlay depuis %1$S a échoué.
PINotInProlog=L'instruction de traitement <?%1$S?> n'a désormais plus d'effet en dehors du prologue (consulter le bogue 360119).
NeededToWrapXUL=La boîte XUL pour l'élément %1$S contient un enfant %2$S intégré, forçant tous ses enfants à être englobés dans un bloc.
NeededToWrapXULInlineBox=La boîte XUL pour l'élément %1$S contient un enfant %2$S intégré, forçant tous ses enfants à être englobés dans un bloc. Ceci peut être corrigé en remplaçant "display: -moz-inline-box" par "display: -moz-inline-box; display: inline-block".##@@\\workdir\\fr\\chrome\\AB-CD\\locale\\AB-CD\\global\\devtools\\debugger.properties@@##
but the debugger.properties file is correctly generated at least.
Again, there is no empty line at EOF in the v25 xul.properties file.


Edit2:
The file \chrome\AB-CD\locale\AB-CD\services\sync.properties coming from the split gets a right arrow symbol at EOF which should not be there. The right arrow symbol is situated at the EOF of core.properties in fact (sync.properties appears to be the last one appended).

Edit3:
Tn the Sync setup window, the string "Fill in the details and click next." is not translated.

Cheers,
TMTisFree

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35481
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Transifex for Pale Moon core strings

Unread post by Moonchild » 2016-01-08, 17:15

Thanks. Just what I don't need, files missing newlines at the end of files.

Also, I'm surprised that ampersands were converted to their html element version. That's bad. I'll poke the Transifex people about this.

re: edit3: that's not part of the browser, it's a string returned by the Pale Moon Sync server.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

tmtisfree

Re: Transifex for Pale Moon core strings

Unread post by tmtisfree » 2016-01-08, 21:10

Moonchild wrote:re: edit3: that's not part of the browser, it's a string returned by the Pale Moon Sync server.
OK
Moonchild wrote:Thanks. Just what I don't need, files missing newlines at the end of files.
Also, I'm surprised that ampersands were converted to their html element version. That's bad. I'll poke the Transifex people about this.
I just download your ~9.30pm update from Transifex:
1/ the preferences.dtd file still misses 2 strings at EOF after splitting (see my post above)
2/ the xul.properties file still has a flag line appended at the end of the last non empty line after splitting (see my post above)
3/ the sync.properties file still has a right arrow at the end after splitting (see my post above)
4/ the & are all gone, & and both < and &rt; are still there in core.dtd: beware that bulk replacing those with < and > in core.dtd screws the Sync window setup (still investigating why)

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35481
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Transifex for Pale Moon core strings

Unread post by Moonchild » 2016-01-08, 21:44

1) those strings shouldn't be in the langpack. Please understand that a number of components have been removed and the old langpacks may still have strings in there that are no longer used. Preferences doesn't have "back" and "forward" buttons. I assume this was culled with the in-content preferences.
2) I just tested this with a different language, and it works fine.
3) If you're talking about the 0x1A character at theend, that's of no consequence.
4) I'm sure transifex will be working on this over time. I hope so anyway
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

tmtisfree

Re: Transifex for Pale Moon core strings

Unread post by tmtisfree » 2016-01-08, 22:02

Moonchild wrote:1) those strings shouldn't be in the langpack. Please understand that a number of components have been removed and the old langpacks may still have strings in there that are no longer used. Preferences doesn't have "back" and "forward" buttons. I assume this was culled with the in-content preferences.
2) I just tested this with a different language, and it works fine.
3) If you're talking about the 0x1A character at theend, that's of no consequence.
4) I'm sure transifex will be working on this over time. I hope so anyway
1/ OK, was'nt sure if it was needed.
2/ for some reason, the flag string was in the translation on Transifex. I have removed it so all good now.
3/ 4/ OK

KNTRO

Re: Transifex for Pale Moon core strings

Unread post by KNTRO » 2016-01-08, 22:58

I got 2 notifications from Transifex:
The source file of the resource core.properties of the project Pale Moon has been updated.
The source file of the resource core.dtd of the project Pale Moon has been updated.
I can't tell what's the news however as I've found no new string or something. What's new?

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35481
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Transifex for Pale Moon core strings

Unread post by Moonchild » 2016-01-09, 00:55

I added some extra newlines, and a number of strings were removed.

See: https://github.com/MoonchildProductions ... its/master
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35481
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Transifex for Pale Moon core strings

Unread post by Moonchild » 2016-01-13, 14:44

OK, Everyone please STOP if you are translating or proofreading in Transifex.

The people there obviously have a wrong parser for DTD files, that mangles the files both when inserting into their db and when saving it back out, and they have no intention of fixing it. This escaping/unescaping regardless of location will also likely bite them in the ass when they offer more of their website translation services because, plainly put, they are converting special characters to entities regardless of where they occur in the files. Both inside tag parameters and between tags (only the latter should be done).
Transifex support wrote: I completely understand your concerns.

However, It seems that these conversions were done intentionally when the format handler was originally developed.

So when your source file contains "<", ">" "&" characters, you should convert them into "<" ">" "&" before you upload it on Transifex. Transifex normally anticipates this and un-escapes them before inserting it into the database so that when translators see them in the editor, they can see the original character.

There is a problem when a source language file comes into Transifex with some characters not escaped. In this case, our parser doesn't find anything to un-escape, but the compiler will escape it nevertheless, hence the behaviour you are getting. We think that Transifex's behaviour is correct, but even if we were to change it, it would create massive backwards compatibility issues with files that have already been uploaded to Transifex. Thus, we will not be able to help you with this, sorry.
We are effectively back to square one. I can grab what languages have been completed or almost-completed, manually repair the files, and offer those for Pale Moon 26. For the rest and future updates I will be forced to find another solution.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

KNTRO

Re: Transifex for Pale Moon core strings

Unread post by KNTRO » 2016-01-13, 15:12

Dammit! :x It's 2016 and I can't believe there's no acceptable parser for DTDs! :x

dark_moon

Re: Transifex for Pale Moon core strings

Unread post by dark_moon » 2016-01-13, 15:13

Bad. Transifex was nice for translating. :(

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35481
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Transifex for Pale Moon core strings

Unread post by Moonchild » 2016-01-13, 18:01

KNTRO wrote:Dammit! :x It's 2016 and I can't believe there's no acceptable parser for DTDs! :x
I know, right?

Trados probably does it, but that's very much $$$ to get and I don't expect volunteer translators to have access to it.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

Locked