Web-based translations

[robn]

We've had an idea that it would be good to have a web-based translation system available to reduce the overhead for translators. The topic has come up again in IRC in the last few days and this time I was motivated to do something about it.

What I'd like to encourage is "drive-by" translations - random user loading it up and translating a single string. If we can make that easy then we stand a good chance of having good-quality translations that are kept up to date.

Transifex have a web-based translation system with some really nice facilities. They offer their service free to open-source projects, so I've signed us up and imported our core translations (ie the files under data/lang): https://www.transifex.com/projects/p/pioneer/

Nice things they provide:

• Notification emails when new strings are added to the source file
• Machine translation via Google or Microsoft translation services (I've hooked our project up to Microsoft's because you have to pay for Google's API these days)
• API so we can hook it up to the build system
• Web hook when translations are updated so we can automatically update the repository

So far I'm really liking the look of this. I've started integrating it on robn/transifex, though it won't be working properly until tonight or tomorrow when I change to JSON language files rather than our own format. But that should be pretty easy.

I'm not doing anything with Lua translations yet. Those are complicated by the flavour system, which allows multiple alternative translations for strings and grouping of translations, giving all sorts of possibilities for adverts and missions. We probably need to either drop flavours entirely or do them differently, not sure yet. We need to do something though, the Lua translations are unwieldy and not at all mod-friendly. So later.

Feel free to sign up to Transifex and poke around, change things, whatever. I need to know if this makes translations easier for you before I go ahead with it. Please comment!

[bszlrd]

Good idea, it's easier for sure this way. (I didn't even know there are web systems like this)

I've tried it out and filled some missing strings and corrected some others (not mistranslations, but some stuff sounded strane) in the hungarian translation.

[walterar]

Excellent idea. I'm already there.

Stupid question: How many zeros has "a billion" in your language? :)

[jpab]

Stupid question: How many zeros has "a billion" in your language? :)

Not a stupid question at all. It has changed in the past, and was different in America and Britain. In modern use it is the "short scale" billion in most English speaking countries: 1,000,000,000.

(Wikipedia: Long and short scales)

John B

[robn]

This work is done, so translations are now being managed through Transifex. From now on all translations should be done there, and we won't be accepting direct changes to the non-English translation files.

Details here: http://pioneerwiki.com/wiki/Translations

[FluffyFreak]

This is really cool, great work robn!
I also had no idea that systems like this even existed!!! :D

[walterar]

If missing some translation in other language, takes the English by default?

[robn]

If missing some translation in other language, takes the English by default?

Yes.

[walterar]

For some reason not working in my mod.
I have to Achernar (and others) in the startup options. The add in mods/lang/ui-core/en.json, es.json and ru.json
In those languages​​, all perfect. But if I change to French or German or Italian, says that lack Achernar
"Error: ui / MainMenu.lua: 95: unknown token translation: START_AT_ACHERNAR"

[robn]

Oh, I see what you mean. No, there's no fallback built into Pioneer anymore. All languages need to provide all tokens. Transifex will fill in the English strings for any that are missing, but of course that won't work for strings that aren't managed that way.

Your mod shouldn't try to change the language files that ship with Pioneer. Create your own language resource and use it. There's no reason you can't load multiple translation resources (though you will have to track changes to that resource because it belongs to someone else). Just make extra calls to Lang.GetResource:

Code: Select all

local l = Lang.GetResource("ui-core")
local mystrings = Lang.GetResource("mod-whatever")

That will look for the current language code. If you want to force it to use a specific language every time, pass it as a second arg to GetResource:

Code: Select all

local mystrings = Lang.GetResource("mod-whatever", "de")

Or, just use strings as-is, and don't make them translatable.

I think what I'd like is to let mods arrange their own fallbacks. Its a bit crap right now because GetResource returns an empty table if the requested resource is not found, so fallbacks need to be done by testing for a known string and trapping the exception:

Code: Select all

local mystrings = Lang.GetResource("mod-whatever")
if not pcall(function () return mystrings.SOME_STRING end) then
    mystrings = Lang.GetResource("mod-whatever", "en")
end

I think I'd prefer this:

Code: Select all

local mystrings = Lang.GetResource("mod-whatever") or Lang.GetResource("mod-whatever", "en")

That is, GetResource returns nil if the resource wasn't found.

That still won't work for partial translations, but I think that's fine. We're not really allowing partial translations in core anymore either. A resource needs to provide all strings, or not exist. That's what Transifex provides and I don't think its unreasonable to expect modders to do the same thing. Most mods shouldn't have too many extra strings so managing it manually shouldn't be too hard, but its also not hard to write a couple of scripts to fill in the gaps, or to use Transifex or similar yourself if that's appropriate for the situation.

Feedback on this welcome. Especially on the proposed syntax above, which I'll implement today.