Hi there, now I sorted the languages as mentioned in the last i18n thread. It looks realy tidy now.
regards movimax
Markus Hahn wrote:
Hi there, now I sorted the languages as mentioned in the last i18n thread. It looks realy tidy now.
Well, if you want this to go into the official VDR source then you should do this based on the latest version ;-)
Also, this will break the OSD language selection for many users, but then again this should have been stored a a three letter code, anyway, so this might be a good opportunity tu justify this break...
Klaus
Hi,
Klaus Schmidinger wrote:
Markus Hahn wrote:
Hi there, now I sorted the languages as mentioned in the last i18n thread. It looks realy tidy now.
Well, if you want this to go into the official VDR source then you should do this based on the latest version ;-)
Also, this will break the OSD language selection for many users, but then again this should have been stored a a three letter code, anyway, so this might be a good opportunity tu justify this break...
Attached is patch that basically does 2 things: 1. stores OSD language selection as a 3-letter language code; 2. unties the present hard-wired number of OSD languages to be the same as the number of *content* languages defined in vdr.
Let me detail a bit... - Backwards compatibility: As the order of languages (and thus, the mapping to the indices used until now) hasn't changed, and is dictated by the Phrases array, OSD language setting still operates internally with the language index, plugins are not affected by this change (apart from those reading setup.conf directly, like vdr-burn for it's external scripts for example, but that can be easily adapted). The only thing that is visible after using this patch is that on the first start of VDR, the default language is used (English). - At present, this patch uses the 3-letter language code used for EPG and audio languages (and the number of codes and languages also is the same defined for audio and epg), but if for example, in the future, external language files would be used, this code could be something else, and the change to that new scheme would also just have VDR present itself in the default language in *that* new language system. For example, if gettext would be used, then the code needed for setlocale (LC_ALL, "de_DE@euro") would be stored as the OSD language selection.
Lucian
P.S. regarding gettext i18n, yes, I worked on it and because an i18n system with external language files needs to be flexible with respect to number of languages, the separation of OSD languages from content languages, as well as storing the OSD language selection as a string is benefic to this. BTW, gettext i18n is working well (just the non-cached implementation ready yet), and I'll soon post a patch for those interested.
diff -Naur vdr-1.3.37_orig/config.c vdr-1.3.37/config.c --- vdr-1.3.37_orig/config.c 2005-09-09 17:08:59.000000000 +0200 +++ vdr-1.3.37/config.c 2005-11-28 10:18:25.520000000 +0100 @@ -401,9 +401,41 @@ return true; }
+void cSetup::StoreOsdLanguage(const char *Name, int & IdxVal, const char * const *OsdLangVals) +{ + char * valuestring = NULL; + if (OsdLangVals[IdxVal][3] == ',') { + // if it's the 3-letter code which might be followed by + // a comma and another 3-letter code, just store the first code + char buff[4]; + strncpy(buff, OsdLangVals[IdxVal], 3); + buff[3] = '\0'; + valuestring = strdup(buff); + } + else { + // it's either a single code, or in a possible future gettext version, + // the locale, which could be longer, like 'de_DE@euro' + valuestring = strdup(OsdLangVals[IdxVal]); + } + Store(Name, valuestring); +} + +bool cSetup::ParseOsdLanguage(const char *Value, int & IdxVal, int NumOsdLangs, const char * const *OsdLangVals) +{ + for(int i = 0; i < NumOsdLangs; i++) { + if (strcmp(Value, OsdLangVals[i]) == 0 || // first compare the strings on their whole length + (OsdLangVals[i][3] == ',' && // one more chance, maybe we're still using the 3-letter codes + strncmp(Value, OsdLangVals[i], strlen(Value)) == 0) ) { + IdxVal = i; + return true; + } + } + return false; +} + bool cSetup::Parse(const char *Name, const char *Value) { - if (!strcasecmp(Name, "OSDLanguage")) OSDLanguage = atoi(Value); + if (!strcasecmp(Name, "OSDLanguage")) return ParseOsdLanguage(Value, OSDLanguage, I18nNumOsdLangs, I18nOsdLangCodes()); else if (!strcasecmp(Name, "OSDSkin")) strn0cpy(OSDSkin, Value, MaxSkinName); else if (!strcasecmp(Name, "OSDTheme")) strn0cpy(OSDTheme, Value, MaxThemeName); else if (!strcasecmp(Name, "PrimaryDVB")) PrimaryDVB = atoi(Value); @@ -468,7 +500,7 @@
bool cSetup::Save(void) { - Store("OSDLanguage", OSDLanguage); + StoreOsdLanguage("OSDLanguage", OSDLanguage, I18nOsdLangCodes()); Store("OSDSkin", OSDSkin); Store("OSDTheme", OSDTheme); Store("PrimaryDVB", PrimaryDVB); diff -Naur vdr-1.3.37_orig/config.h vdr-1.3.37/config.h --- vdr-1.3.37_orig/config.h 2005-11-11 14:22:02.000000000 +0100 +++ vdr-1.3.37/config.h 2005-11-28 10:14:26.330000000 +0100 @@ -202,6 +202,8 @@ cSetupLine *Get(const char *Name, const char *Plugin = NULL); void Store(const char *Name, const char *Value, const char *Plugin = NULL, bool AllowMultiple = false); void Store(const char *Name, int Value, const char *Plugin = NULL); + void StoreOsdLanguage(const char *Name, int & IdxVal, const char * const *OsdLangVals); + bool ParseOsdLanguage(const char *Value, int & IdxVal, int NumOsdLangs, const char * const *OsdLangVals); public: // Also adjust cMenuSetup (menu.c) when adding parameters here! int __BeginData__; diff -Naur vdr-1.3.37_orig/i18n.c vdr-1.3.37/i18n.c --- vdr-1.3.37_orig/i18n.c 2005-11-04 15:36:27.000000000 +0100 +++ vdr-1.3.37/i18n.c 2005-11-28 10:22:19.060000000 +0100 @@ -5520,6 +5520,12 @@
cI18nList I18nList;
+// number of OSD languages, for now identical with I18nNumLanguages, +// but with external OSD language files, the number could become different +// from the number of defined content languages +int I18nNumOsdLangs = I18nNumLanguages; + + // ---
void I18nRegister(const tI18nPhrase * const Phrases, const char *Plugin) @@ -5563,6 +5569,27 @@ return &Phrases[1][0]; }
+const char * const * I18nOsdLangCodes(void) +{ + // for now, similar to I18nLanguageCode(int Index), but could use different identifiers, + //like 'de_DE@euro' in the future + return &Phrases[2][0]; +} + +const char * const * I18nOsdLanguages(void) +{ + // for now, identical with I18nLanguages(), but the size of this array could be different + // in the future, with external OSD languages + return &Phrases[0][0]; +} + +int I18nOsdLanguageIndex(const char *Code) +{ + // for now, identical with I18nLanguageIndex(const char *Code), but the size and + // identifiers in this array could be different in the future, with external OSD languages + return I18nLanguageIndex(Code); +} + const char *I18nLanguageCode(int Index) { return 0 <= Index && Index < I18nNumLanguages ? Phrases[2][Index] : NULL; diff -Naur vdr-1.3.37_orig/i18n.h vdr-1.3.37/i18n.h --- vdr-1.3.37_orig/i18n.h 2005-09-09 16:50:35.000000000 +0200 +++ vdr-1.3.37/i18n.h 2005-11-28 10:14:26.350000000 +0100 @@ -13,6 +13,7 @@ #include <stdio.h>
const int I18nNumLanguages = 20; +extern int I18nNumOsdLangs;
typedef const char *tI18nPhrase[I18nNumLanguages];
@@ -22,6 +23,9 @@
const char * const * I18nLanguages(void); const char * const * I18nCharSets(void); +const char * const * I18nOsdLangCodes(void); +const char * const * I18nOsdLanguages(void); +int I18nOsdLanguageIndex(const char *Code); const char *I18nLanguageCode(int Index); int I18nLanguageIndex(const char *Code); const char *I18nNormalizeLanguageCode(const char *Code); diff -Naur vdr-1.3.37_orig/menu.c vdr-1.3.37/menu.c --- vdr-1.3.37_orig/menu.c 2005-11-05 18:29:22.000000000 +0100 +++ vdr-1.3.37/menu.c 2005-11-28 10:14:26.350000000 +0100 @@ -1863,7 +1863,7 @@ useSmallFontTexts[2] = tr("always"); Clear(); SetSection(tr("OSD")); - Add(new cMenuEditStraItem(tr("Setup.OSD$Language"), &data.OSDLanguage, I18nNumLanguages, I18nLanguages())); + Add(new cMenuEditStraItem(tr("Setup.OSD$Language"), &data.OSDLanguage, I18nNumOsdLangs, I18nOsdLanguages())); Add(new cMenuEditStraItem(tr("Setup.OSD$Skin"), &skinIndex, numSkins, skinDescriptions)); if (themes.NumThemes()) Add(new cMenuEditStraItem(tr("Setup.OSD$Theme"), &themeIndex, themes.NumThemes(), themes.Descriptions())); diff -Naur vdr-1.3.37_orig/themes.c vdr-1.3.37/themes.c --- vdr-1.3.37_orig/themes.c 2005-11-04 15:19:54.000000000 +0100 +++ vdr-1.3.37/themes.c 2005-11-28 10:14:26.350000000 +0100 @@ -114,7 +114,7 @@ int lang = 0; char *l = strchr(n, '.'); if (l) - lang = I18nLanguageIndex(++l); + lang = I18nOsdLanguageIndex(++l); if (lang >= 0) { free(descriptions[lang]); descriptions[lang] = strdup(v); @@ -167,9 +167,9 @@ bool result = true; cSafeFile f(FileName); if (f.Open()) { - for (int i = 0; i < I18nNumLanguages; i++) { + for (int i = 0; i < I18nNumOsdLangs; i++) { if (descriptions[i]) - fprintf(f, "Description%s%.*s = %s\n", i ? "." : "", 3, i ? I18nLanguageCode(i) : "", descriptions[i]); + fprintf(f, "Description%s%.*s = %s\n", i ? "." : "", 3, i ? I18nOsdLangCodes()[i] : "", descriptions[i]); } for (int i = 0; i < MaxThemeColors; i++) { if (colorNames[i])
Lucian Muresan wrote:
Hi,
Klaus Schmidinger wrote:
Markus Hahn wrote:
Hi there, now I sorted the languages as mentioned in the last i18n thread. It looks realy tidy now.
Well, if you want this to go into the official VDR source then you should do this based on the latest version ;-)
Also, this will break the OSD language selection for many users, but then again this should have been stored a a three letter code, anyway, so this might be a good opportunity tu justify this break...
Attached is patch that basically does 2 things:
- stores OSD language selection as a 3-letter language code;
Ok, I'll adopt this for version 1.4.
- unties the present hard-wired number of OSD languages to be the same
as the number of *content* languages defined in vdr.
This will have to wait until after version 1.4, I'm not going to change that now.
Klaus
Klaus Schmidinger wrote:
Lucian Muresan wrote:
Hi,
Klaus Schmidinger wrote:
Markus Hahn wrote:
Hi there, now I sorted the languages as mentioned in the last i18n thread. It looks realy tidy now.
Well, if you want this to go into the official VDR source then you should do this based on the latest version ;-)
Also, this will break the OSD language selection for many users, but then again this should have been stored a a three letter code, anyway, so this might be a good opportunity tu justify this break...
Attached is patch that basically does 2 things:
- stores OSD language selection as a 3-letter language code;
Ok, I'll adopt this for version 1.4.
- unties the present hard-wired number of OSD languages to be the same
as the number of *content* languages defined in vdr.
This will have to wait until after version 1.4, I'm not going to change that now.
Actually, this separation is done in a way that leaves everything unchanged (or better, transparent), it rather prepares the code for external language files due to the flexibility I tried to put in. But you're right, as it's not necessary only for the 3-letter code storing/parsing, that can wait. Anyway, thanks for considering :-)
Lucian
Klaus Schmidinger wrote:
Lucian Muresan wrote:
Hi,
Klaus Schmidinger wrote:
Markus Hahn wrote:
Hi there, now I sorted the languages as mentioned in the last i18n thread. It looks realy tidy now.
Well, if you want this to go into the official VDR source then you should do this based on the latest version ;-)
Also, this will break the OSD language selection for many users, but then again this should have been stored a a three letter code, anyway, so this might be a good opportunity tu justify this break...
Attached is patch that basically does 2 things:
- stores OSD language selection as a 3-letter language code;
Ok, I'll adopt this for version 1.4.
Well, after further consideration I decided against this. This would currently only break compatibility, without any real advantage.
This whole thing will have to wait until 1.5.x.
Klaus
Markus Hahn wrote:
Hi there, now I sorted the languages as mentioned in the last i18n thread. It looks realy tidy now.
Hmmm. Imho the order of the languages is the same for VDR's i18n.c and for all the plugins, so shouldn't this break the i18n's of *all* plugins? Or do I miss something?
On long term, its probably best to move i18n out of the c source and into editable language definition files that can be loaded on runtime. (I think thats what Klaus meant with 'language files'.)
Cheers,
Udo
Udo Richter wrote:
Markus Hahn wrote:
Hi there, now I sorted the languages as mentioned in the last i18n thread. It looks realy tidy now.
Hmmm. Imho the order of the languages is the same for VDR's i18n.c and for all the plugins, so shouldn't this break the i18n's of *all* plugins? Or do I miss something?
You're of course absolutely right here. I should have thought of this, too.
On long term, its probably best to move i18n out of the c source and into editable language definition files that can be loaded on runtime. (I think thats what Klaus meant with 'language files'.)
Yes, that's what I meant. But let's do this in version 1.5.x.
Klaus
Am Mittwoch, 16. November 2005 14:57 schrieb Klaus Schmidinger:
Udo Richter wrote:
Hmmm. Imho the order of the languages is the same for VDR's i18n.c and for all the plugins, so shouldn't this break the i18n's of *all* plugins? Or do I miss something?
Thats seems to be a real problem. but most oft the plugins are real bad translated. Could be a chance to force the maintainers to do something. It doesn`t make a lot of work.
Beside the new i18n.c (vdr-1.3.36), I fetched wordlists out of i18n.c and rest of the sources. (piped through sort). There are some diffrences.
regards movimax
You're of course absolutely right here. I should have thought of this, too.
On long term, its probably best to move i18n out of the c source and into editable language definition files that can be loaded on runtime. (I think thats what Klaus meant with 'language files'.)
Yes, that's what I meant. But let's do this in version 1.5.x.
Klaus
vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Markus Hahn wrote:
Am Mittwoch, 16. November 2005 14:57 schrieb Klaus Schmidinger:
Udo Richter wrote:
Hmmm. Imho the order of the languages is the same for VDR's i18n.c and for all the plugins, so shouldn't this break the i18n's of *all* plugins? Or do I miss something?
Thats seems to be a real problem. but most oft the plugins are real bad translated. Could be a chance to force the maintainers to do something. It doesn`t make a lot of work.
If Klaus said he plans to implement language files in a later version, why breaking things now only to brake them later again?
Beside the new i18n.c (vdr-1.3.36), I fetched wordlists out of i18n.c and rest of the sources. (piped through sort). There are some diffrences.
This sorting thing can also be done, guess what, with "xgettext -ktr *.c -s -o vdr.po" and then I'll have a gettext template file (and when using gettext for example, the sorting doesn't matter too much anymore, but of course it can be done, like in the above example, editing language files can be done with PO-editors). I was silent in this matter since the original thread which caused this discussion, and animated by actually Thiemo's idea about language files I gave the whole matter a thought and even played a bit with the code. So, I already have a rough concept of quickly turning i18n.h and i18n.c into using gettext (which is available in glibc anyway, and is most commonly used in the linux world) for external language files. It can be done with minimal code changes in the rest of the program, as the "tr" macro can be used further on, and only some things in the functions implemented in i18n.c require some changes. If someone says reading translations from external files is slow on every call to "tr", this can also be solved by reading their whole contents into memory, only it would be dynamical (not bound to a hard-coded number of UI languages, as theese can be added bthen by distributors or even users whithout having to compile the program). And the whole stuff is really easily extendable to plugins, as their maintainers now can entirely throw away their i18n files and just let the xgettext utility parse their sources for "tr"-enclosed strings to be translated and generate po-files in which again, the translated strings can be entered by non-programmers too. And the same technique can be applied even to commands.conf and reccmds.conf with some littel helper function in VDR, which would first generate a template file out of the command.conf files, as their command labels themselves are not enclosed by the "tr" macro (that, if the syntax remains the same). But, this also can be left optional to distributors/packagers, VDR would only look for some languagefiles if they're available at the expected location, according to the configured language, if the#re not there, everything remains as before. And this won't be the same as in my latest patch, having the commands actually in multiple files.
regards movimax
You're of course absolutely right here. I should have thought of this, too.
On long term, its probably best to move i18n out of the c source and into editable language definition files that can be loaded on runtime. (I think thats what Klaus meant with 'language files'.)
Yes, that's what I meant. But let's do this in version 1.5.x.
Klaus
That beeing said, Klaus & everyone, what would you think about letting me give it a try? I think and I hope I'll be able to provide a patch in few days, with all existing translations migrated to the new aproach and adapted makefiles (maybe until this weekend, depending on spare time), in the meantime we could discuss the concept, maybe I'll have questioons for few details (maybe there are issues other than spare time or priorities which made you plan this for 1.5.x). Well, in the end we'll see if it's good or not, but we might have the chance to workout the breakage in this i18n matter only once and already now (as I said, I think I'll be able to do it myself, so it's up to me to find the time, and I'll promise I'll yell or let it be if I happen to get stuck, you won't have to wait for this if you already plan a feature-freeze for releasing 1.4, I only think we could give it a shot, it might turn out good enough for having it merged). But please tell me if there is ablsolutely no interest for this now.
Lucian
Lucian Muresan wrote:
...
On long term, its probably best to move i18n out of the c source and into editable language definition files that can be loaded on runtime. (I think thats what Klaus meant with 'language files'.)
Yes, that's what I meant. But let's do this in version 1.5.x.
Klaus
That beeing said, Klaus & everyone, what would you think about letting me give it a try? I think and I hope I'll be able to provide a patch in few days, with all existing translations migrated to the new aproach and adapted makefiles (maybe until this weekend, depending on spare time), in the meantime we could discuss the concept, maybe I'll have questioons for few details (maybe there are issues other than spare time or priorities which made you plan this for 1.5.x). Well, in the end we'll see if it's good or not, but we might have the chance to workout the breakage in this i18n matter only once and already now (as I said, I think I'll be able to do it myself, so it's up to me to find the time, and I'll promise I'll yell or let it be if I happen to get stuck, you won't have to wait for this if you already plan a feature-freeze for releasing 1.4, I only think we could give it a shot, it might turn out good enough for having it merged). But please tell me if there is ablsolutely no interest for this now.
Lucian
I definitely will not change the i18n methods for version 1.4. You are, of course, free to do whatever you like, but none of this will go into version 1.4. I also won't adopt the sorting patch, since it would break plugins. I18n in version 1.4 will be exactly as it is right now.
I have a few things that I still want to finish for 1.4, and i18n changes is none of them.
Sorry, gotta draw the line somewhere...
Klaus
Klaus Schmidinger wrote: [...]
I definitely will not change the i18n methods for version 1.4. You are, of course, free to do whatever you like, but none of this will go into version 1.4. I also won't adopt the sorting patch, since it would break plugins. I18n in version 1.4 will be exactly as it is right now.
I have a few things that I still want to finish for 1.4, and i18n changes is none of them.
Sorry, gotta draw the line somewhere...
Klaus
Thanks, that was quick. Sorry for annoying you (at least, that's what your last two sentences in this reply just tell).
Lucian
Lucian Muresan wrote:
Klaus Schmidinger wrote: [...]
I definitely will not change the i18n methods for version 1.4. You are, of course, free to do whatever you like, but none of this will go into version 1.4. I also won't adopt the sorting patch, since it would break plugins. I18n in version 1.4 will be exactly as it is right now.
I have a few things that I still want to finish for 1.4, and i18n changes is none of them.
Sorry, gotta draw the line somewhere...
Klaus
Thanks, that was quick. Sorry for annoying you (at least, that's what your last two sentences in this reply just tell).
Lucian
I didn't mean to sound annoyed - I really am not.
I just wanted to make a clear statement, so that nobody is disappointed later.
Klaus
Am Mittwoch, 16. November 2005 17:42 schrieb Lucian Muresan:
That beeing said, Klaus & everyone, what would you think about letting me give it a try? I think and I hope I'll be able to provide a patch in few days, with all existing translations migrated to the new aproach and adapted makefiles (maybe until this weekend, depending on spare time), in the meantime we could discuss the concept, maybe I'll have questioons for few details (maybe there are issues other than spare time or priorities which made you plan this for 1.5.x). Well, in the end we'll see if it's good or not, but we might have the chance to workout the breakage in this i18n matter only once and already now (as I said, I think I'll be able to do it myself, so it's up to me to find the time, and I'll promise I'll yell or let it be if I happen to get stuck, you won't have to wait for this if you already plan a feature-freeze for releasing 1.4, I only think we could give it a shot, it might turn out good enough for having it merged). But please tell me if there is ablsolutely no interest for this now.
Seems that you already made some serious thoughts about how this could be done. I already thought that Klaus does not want to make major concept changes until 1.4 is out but if you still want to go for it i wouldn't mind ;)
@Klaus did you already make some thoughts about the message format? would gettext be your choice? do you have other ideas?
Just a few remarks and thoughts of mine: - it would be nice if we would not "lose" the already existing translations - i would like to have only _one_ language source file for everything to let the translators deal with. how could we archive that not every plugin has to be translated on it's own? (we would have some resellers and even users/customers that could deal with the translations, but only if its easy to handle for them. i already realized that it's not practicable to send them
25 files)
- i also like your idea of making translation files (.mo) hashable in memory to speed up things.
What do others, especially the distribution packagers and translators think?
Thiemo
rollercoaster@reel-multimedia.com wrote:
Am Mittwoch, 16. November 2005 17:42 schrieb Lucian Muresan:
That beeing said, Klaus & everyone, what would you think about letting me give it a try? I think and I hope I'll be able to provide a patch in few days, with all existing translations migrated to the new aproach and adapted makefiles (maybe until this weekend, depending on spare time), in the meantime we could discuss the concept, maybe I'll have questioons for few details (maybe there are issues other than spare time or priorities which made you plan this for 1.5.x). Well, in the end we'll see if it's good or not, but we might have the chance to workout the breakage in this i18n matter only once and already now (as I said, I think I'll be able to do it myself, so it's up to me to find the time, and I'll promise I'll yell or let it be if I happen to get stuck, you won't have to wait for this if you already plan a feature-freeze for releasing 1.4, I only think we could give it a shot, it might turn out good enough for having it merged). But please tell me if there is ablsolutely no interest for this now.
Seems that you already made some serious thoughts about how this could be done. I already thought that Klaus does not want to make major concept changes until 1.4 is out but if you still want to go for it i wouldn't mind ;)
@Klaus did you already make some thoughts about the message format? would gettext be your choice? do you have other ideas?
AFAICS ATM gettext() is the way this is normally done, so I guess I would do it that way, too.
There would be one file for each language in VDR itself, and each plugin would also have its own set of language files. There will also be a way of converting the exiting i18n files into the new format.
But this is as far as I'll go here now. I'm having my mind full of other things and just don't want to elaborate on this right now.
If you want to implement something here now, that's your choice. But please don't be mad at me if I choose to do things differently in version 1.5.
Klaus
Klaus Schmidinger wrote:
rollercoaster@reel-multimedia.com wrote:
Am Mittwoch, 16. November 2005 17:42 schrieb Lucian Muresan:
That beeing said, Klaus & everyone, what would you think about letting me give it a try? I think and I hope I'll be able to provide a patch in few days, with all existing translations migrated to the new aproach and adapted makefiles (maybe until this weekend, depending on spare time), in the meantime we could discuss the concept, maybe I'll have questioons for few details (maybe there are issues other than spare time or priorities which made you plan this for 1.5.x). Well, in the end we'll see if it's good or not, but we might have the chance to workout the breakage in this i18n matter only once and already now (as I said, I think I'll be able to do it myself, so it's up to me to find the time, and I'll promise I'll yell or let it be if I happen to get stuck, you won't have to wait for this if you already plan a feature-freeze for releasing 1.4, I only think we could give it a shot, it might turn out good enough for having it merged). But please tell me if there is ablsolutely no interest for this now.
Seems that you already made some serious thoughts about how this could be done. I already thought that Klaus does not want to make major concept changes until 1.4 is out but if you still want to go for it i wouldn't mind ;)
@Klaus did you already make some thoughts about the message format? would gettext be your choice? do you have other ideas?
AFAICS ATM gettext() is the way this is normally done, so I guess I would do it that way, too.
There would be one file for each language in VDR itself, and each plugin would also have its own set of language files. There will also be a way of converting the exiting i18n files into the new format.
Yes, all of this is part of my plan, too.
But this is as far as I'll go here now. I'm having my mind full of other things and just don't want to elaborate on this right now.
Ok, we wouldn't want to bother you in that case. I hope we can, however keep the discussion on this list, we'll keep the *gettext* subject so you can go over to other things as long as you don't have time for this now, is this ok? But if it's not ok just say so :-)
If you want to implement something here now, that's your choice. But please don't be mad at me if I choose to do things differently in version 1.5.
We could live with that, I think. but I also think there is some time then, for things to sediment and get tested, and prove to be good or bad, so it's ok in the end if they get what they deserve: adopted, even if only as idea, or dropped...
Lucian
rollercoaster@reel-multimedia.com wrote:
- i would like to have only _one_ language source file for everything to let
the translators deal with. how could we archive that not every plugin has to be translated on it's own? (we would have some resellers and even users/customers that could deal with the translations, but only if its easy to handle for them. i already realized that it's not practicable to send them
25 files)
I don't think that this is possible. Whats the point of separate plugins if all i18n strings must be in core VDR?
Distributions wont like that either: They like to have everything for a plugin in one package, and no dependencies to the core VDR package. Imagine that each plugin update would require a core VDR update to add the missing i18n strings...
Maybe there's a way to merge all .po files into one for translators, and split the results back to separate files for re-integration.
Cheers,
Udo
Udo Richter wrote:
rollercoaster@reel-multimedia.com wrote:
- i would like to have only _one_ language source file for everything to let
the translators deal with. how could we archive that not every plugin has to be translated on it's own? (we would have some resellers and even users/customers that could deal with the translations, but only if its easy to handle for them. i already realized that it's not practicable to send them
25 files)
I don't think that this is possible. Whats the point of separate plugins if all i18n strings must be in core VDR?
Distributions wont like that either: They like to have everything for a plugin in one package, and no dependencies to the core VDR package. Imagine that each plugin update would require a core VDR update to add the missing i18n strings...
Maybe there's a way to merge all .po files into one for translators, and split the results back to separate files for re-integration.
Actually, I don't understand the point of having all languages in a file, as a translator seldom does all those languages (at least I think so, maybe I'm missing something Thiemo is thinking of).
Lucian
rollercoaster@reel-multimedia.com wrote:
Am Mittwoch, 16. November 2005 17:42 schrieb Lucian Muresan:
That beeing said, Klaus & everyone, what would you think about letting me give it a try? I think and I hope I'll be able to provide a patch in few days, with all existing translations migrated to the new aproach and adapted makefiles (maybe until this weekend, depending on spare time), in the meantime we could discuss the concept, maybe I'll have questioons for few details (maybe there are issues other than spare time or priorities which made you plan this for 1.5.x). Well, in the end we'll see if it's good or not, but we might have the chance to workout the breakage in this i18n matter only once and already now (as I said, I think I'll be able to do it myself, so it's up to me to find the time, and I'll promise I'll yell or let it be if I happen to get stuck, you won't have to wait for this if you already plan a feature-freeze for releasing 1.4, I only think we could give it a shot, it might turn out good enough for having it merged). But please tell me if there is ablsolutely no interest for this now.
Seems that you already made some serious thoughts about how this could be done. I already thought that Klaus does not want to make major concept changes until 1.4 is out but if you still want to go for it i wouldn't mind ;)
@Klaus did you already make some thoughts about the message format? would gettext be your choice? do you have other ideas?
Just a few remarks and thoughts of mine:
- it would be nice if we would not "lose" the already existing translations
They wouldn't get lost.
- i would like to have only _one_ language source file for everything to let
the translators deal with. how could we archive that not every plugin has to be translated on it's own? (we would have some resellers and even users/customers that could deal with the translations, but only if its easy to handle for them. i already realized that it's not practicable to send them
25 files)
I don't see why they should be all sent to one translator.
- i also like your idea of making translation files (.mo) hashable in memory
to speed up things.
I would basicly load them to be accessible in a similar way as they are now.
Lucian
Am Mittwoch, 16. November 2005 19:55 schrieb Lucian Muresan:
- i would like to have only _one_ language source file for everything to
let the translators deal with. how could we archive that not every plugin has to be translated on it's own? (we would have some resellers and even users/customers that could deal with the translations, but only if its easy to handle for them. i already realized that it's not practicable to send them > 25 files)
I don't see why they should be all sent to one translator.
they do not need all language files, but all files from vdr and all the plugins. Thats also what Udo meant (i beleave) with "merging them together". I would want to make it as easy as possible for translators, but i see that having every "language set" in one file would be hard (if not impossible) to maintain...
But on the other hand I do not see the need of having different namespaces (domains) for every plugin.
Thiemo
Thiemo Gehrke wrote:
But on the other hand I do not see the need of having different namespaces (domains) for every plugin.
To avoid collisions. A string may be differently translated in different contexts. Within one plugin, this can be avoided. But with different plugins, its surely fun to watch two plugin writers arguing about the 'right' translation. ;)
Cheers,
Udo
On Thu, 17 Nov 2005 02:19:30 +0100, in gmane.linux.vdr you wrote:
Thiemo Gehrke wrote:
But on the other hand I do not see the need of having different namespaces (domains) for every plugin.
To avoid collisions. A string may be differently translated in different contexts. Within one plugin, this can be avoided. But with different plugins, its surely fun to watch two plugin writers arguing about the 'right' translation. ;)
You could still get collisions within a single plugin, unless you use unique string IDs (assuming this is how gettext works?). If the plugin i18n rules specify that the string ID has to contain the name of the plugin (e.g. FEMON_MENU_SETUP_BGROUND_TRANSP = "Background transparency") then inter-plugin string ID collisions can be avoided completely.
Sharing strings even within a plugin is /a bad idea/ - you never know where the string might be reused next time, and the translator may need to inflect, conjugate or decline the term differently in each context. If you're worried translators having to translate the same term twice, how about persuading translators to use translation memory (such as OmegaT, see http://www.omegat.org/ - doesn't handle PO natively, but can AIUI a filter is available). This gives the translator the freedom to decide whether to translate the string in the same way twice or whether to change his translation on the basis of the context.
When it comes to preloading language support and that delaying VDR's start-up times, how about allowing the user to select which languages he wants to compile in? A compile flag UILANGS=/path/to/list_of_ui_langs would mean that each user could, if they wanted to, define their own list of languages to compile in. For example, I only speak English, German and French, so my list_of_ui_langs would contain: en de fr
Given most reseller versions are for specific markets (with tailored channels.conf, etc.) this would allow resellers to tailor their packages too.
Iwan
Iwan Davies wrote:
On Thu, 17 Nov 2005 02:19:30 +0100, in gmane.linux.vdr you wrote:
Thiemo Gehrke wrote:
But on the other hand I do not see the need of having different namespaces (domains) for every plugin.
To avoid collisions. A string may be differently translated in different contexts. Within one plugin, this can be avoided. But with different plugins, its surely fun to watch two plugin writers arguing about the 'right' translation. ;)
You could still get collisions within a single plugin, unless you use unique string IDs (assuming this is how gettext works?). If the plugin i18n rules specify that the string ID has to contain the name of the plugin (e.g. FEMON_MENU_SETUP_BGROUND_TRANSP = "Background transparency") then inter-plugin string ID collisions can be avoided completely.
Sharing strings even within a plugin is /a bad idea/ - you never know where the string might be reused next time, and the translator may need to inflect, conjugate or decline the term differently in each context. If you're worried translators having to translate the same term twice, how about persuading translators to use translation memory (such as OmegaT, see http://www.omegat.org/ - doesn't handle PO natively, but can AIUI a filter is available). This gives the translator the freedom to decide whether to translate the string in the same way twice or whether to change his translation on the basis of the context.
I also think it's good not to mix the strings, both among vdr main module / plugins, or languages. I think gettext allows multiple domains in one file, but I think multiple languages is impossible whithout degrading the use of gettext just as a storage.
When it comes to preloading language support and that delaying VDR's start-up times, how about allowing the user to select which languages he wants to compile in? A compile flag UILANGS=/path/to/list_of_ui_langs would mean that each user could, if they wanted to, define their own list of languages to compile in. For example, I only speak English, German and French, so my list_of_ui_langs would contain: en de fr
Given most reseller versions are for specific markets (with tailored channels.conf, etc.) this would allow resellers to tailor their packages too.
Sure, this should be build options.
Lucian
Iwan Davies wrote:
You could still get collisions within a single plugin, unless you use unique string IDs (assuming this is how gettext works?).
The current implementation of VDR - and the typical use of gettext - use the English translation as a key, with the option to pass through this text if no translation is found. (gettext: or no language file found)
There may be a way to 'tag' in gettext, so that in case of pass-through at least the tag gets removed, don't know. VDR's i18n allows such tags in form "tag$english text", where everything up to $ will always be deleted on output.
Sharing strings even within a plugin is /a bad idea/ - you never know where the string might be reused next time, and the translator may need to inflect, conjugate or decline the term differently in each context.
Plugins are usually maintained by one person, so this shouldn't be as problematic as with all plugins in one context.
Cheers,
Udo
On Thu, 17 Nov 2005 15:47:20 +0100, in gmane.linux.vdr you wrote:
Iwan Davies wrote:
You could still get collisions within a single plugin, unless you use unique string IDs (assuming this is how gettext works?).
The current implementation of VDR - and the typical use of gettext - use the English translation as a key, with the option to pass through this text if no translation is found. (gettext: or no language file found)
There may be a way to 'tag' in gettext, so that in case of pass-through at least the tag gets removed, don't know. VDR's i18n allows such tags in form "tag$english text", where everything up to $ will always be deleted on output.
Sharing strings even within a plugin is /a bad idea/ - you never know where the string might be reused next time, and the translator may need to inflect, conjugate or decline the term differently in each context.
Plugins are usually maintained by one person, so this shouldn't be as problematic as with all plugins in one context.
I'm thinking particularly of problems I've experienced in the past in various localisation projects, where, for example, the phrases "von" and "bis" have been reused in different contexts wth represent different meanings ("von Datum"/"bis Datum" and "von Uhrzeit"/"bis Uhrzeit"). In French, for example, the first "von"/"bis" would be translated "du"/"jusqu'au" and the second "de"/"jusqu'à".
In English there is no problem in this particular example, but another situation could see the German use the same noun as a command an an option setting, whereas English would require the command to be translated as an imperative and the setting as a noun. The point is, the plugin developer has no way of knowing how reuse of strings (and duplicate use of string IDs) will affect the quality of the translation, so the safest approach is to assign each string a (G)UID. Incorporating the plugin name in this UID ensures it is globally unique for vdr. On the other hand, judicious use of the tag system in vdr would have the same effect, as long as you could be sure that each tag is unique and none are reused.
As I say, redundancy is dealt with during the translation process by using translation memory, which at least gives the translator the option of deciding whether to reuse an existing identical term or not.
Cheers
Iwan
Iwan Davies wrote: [...]
I'm thinking particularly of problems I've experienced in the past in various localisation projects, where, for example, the phrases "von" and "bis" have been reused in different contexts wth represent different meanings ("von Datum"/"bis Datum" and "von Uhrzeit"/"bis Uhrzeit"). In French, for example, the first "von"/"bis" would be translated "du"/"jusqu'au" and the second "de"/"jusqu'à".
In English there is no problem in this particular example, but another situation could see the German use the same noun as a command an an option setting, whereas English would require the command to be translated as an imperative and the setting as a noun. The point is, the plugin developer has no way of knowing how reuse of strings (and duplicate use of string IDs) will affect the quality of the translation, so the safest approach is to assign each string a (G)UID. Incorporating the plugin name in this UID ensures it is globally unique for vdr. On the other hand, judicious use of the tag system in vdr would have the same effect, as long as you could be sure that each tag is unique and none are reused.
This kind of problem is taken care of at present in VDR's i18n.c by preceding such pharases by an arbitrary string describing the context in which it is used, separated by a character chosen by Klaus, '$' from the actual english string. In that case, translations may be different and are also identified properly, and everything before the '$' character is ignored. But as you said, a developer may anticipate such situations, but cannot do this for all cases and all languages that may join the list, so there will be feedback from users who notice some bad sounding translation in a certain context, and it will be fixed by splitting that english strings by the scheme already used by Klaus. And this shouldn't change if gettext will ever be used.
BTW, while writing a plugin only meant for developers, which extracts all phrases known at present in VDR[*] and generates po-Files for all the known languages at present, after loading the po-files in poEdit, I noticed that there are 5 duplicate strings, which even in the classical concept are redundant, one of them never gets used, but that will be another posting.
Basicly I'm close to trying gettext i18n, will keep you updated, I hope this weekend.
Lucian
[*] ...and as a next step, from any loaded plugin, so the old translations are already in the gettext world. Anyway, if plugin devs will ever follow this, and they have to test their plugins anyway, I'm seeing no problem with doing this extraction at runtime instead of parsing the source files). The po-files are gnerated with all the message IDs and their translations and even with the "note leading/trailing blanks, singular/plural" comments, header information with translation authors for VDR (for plugins I think I will assume for the respective languages the same translators as in VDR, if a translation is available). This way we will have up-to-date po-files, in sync with the "classic" VDR phrases anytime (and if) the switch to gettext will be ever done, as in the mean time updates to the "classic" translations may still come.
Lucian Muresan wrote:
I gave the whole matter a thought and even played a bit with the code. So, I already have a rough concept of quickly turning i18n.h and i18n.c into using gettext (which is available in glibc anyway, and is most commonly used in the linux world) for external language files. It can be done with minimal code changes in the rest of the program, as the "tr" macro can be used further on, and only some things in the functions implemented in i18n.c require some changes. And the whole stuff is really easily extendable to plugins, as their maintainers now can entirely throw away their i18n files and just let the xgettext utility parse their sources for "tr"-enclosed strings to be translated and generate po-files in which again, the translated strings can be entered by non-programmers too.
All in all, this is worth some thoughts. And maybe its a good idea to let this cook as a patch until the 1.5 cycle starts.
Some early thoughts about this:
Migrating VDR core to gettext seems to be possible in an acceptable effort. Where should these language files be stored? /usr/share/locale or in the VDR conf folders?
One thing that is critical is the integration of plugins: VDR plugins use their own plugin tr() namespace, based on the PLUGIN_NAME_I18N set in the Makefile. To make things worse, i18n will fall back to the VDR namespace if nothing was found in the plugin namespace, giving plugins access to all VDR strings.
How does this translate into gettext? Does each plugin use its own textdomain(), and if, how does the domain switching happen? Will plugins based on the old i18n.c concept continue to work, or do they have to migrate to gettext? Will a converted plugin work under a non-gettext VDR version too?
For the existing plugins, we imho need a smooth upgrade path, preferably some scripting-fu that turns i18n.c into .po files. Maybe this is also an option for compatibility: For backwards compatibility, some script turns .po files to old-style i18n.c files.
Cheers,
Udo
Udo Richter wrote:
Lucian Muresan wrote:
I gave the whole matter a thought and even played a bit with the code. So, I already have a rough concept of quickly turning i18n.h and i18n.c into using gettext (which is available in glibc anyway, and is most commonly used in the linux world) for external language files. It can be done with minimal code changes in the rest of the program, as the "tr" macro can be used further on, and only some things in the functions implemented in i18n.c require some changes. And the whole stuff is really easily extendable to plugins, as their maintainers now can entirely throw away their i18n files and just let the xgettext utility parse their sources for "tr"-enclosed strings to be translated and generate po-files in which again, the translated strings can be entered by non-programmers too.
All in all, this is worth some thoughts. And maybe its a good idea to let this cook as a patch until the 1.5 cycle starts.
I might start something tonight.
Some early thoughts about this:
Migrating VDR core to gettext seems to be possible in an acceptable effort. Where should these language files be stored? /usr/share/locale or in the VDR conf folders?
Well, this is a question I was about to ask opinions, too. In the first phase, until I have something up and running, It wouldn't bother me much, but I'd welcome an argumentation on this one, too.
One thing that is critical is the integration of plugins: VDR plugins use their own plugin tr() namespace, based on the PLUGIN_NAME_I18N set in the Makefile. To make things worse, i18n will fall back to the VDR namespace if nothing was found in the plugin namespace, giving plugins access to all VDR strings.
It is possible to keep both concepts for a while.
How does this translate into gettext? Does each plugin use its own textdomain(), and if, how does the domain switching happen?
I think so, every plugin should have it's own domain (therefore, po -> mo file). The switching would be done in the tr() macro (it's underlying function) when invoked with the plugin name argument like it is now invoking actually the same I18nTranslate, which is only looking up the current Phrases eiter in VDR 'domain', or in the plugin's:
#ifdef PLUGIN_NAME_I18N #define tr(s) I18nTranslate(s, PLUGIN_NAME_I18N) #else #define tr(s) I18nTranslate(s) #endif
If the translations are to be cached in memory, they would be read once at startup of VDR and each plugin, and a dynamically allocated Phrases[] array would be filled with their contents, thus having the possibility of keeping most of the functions in the i18n.c (including I18nTranslate) almost as they are now.
Will plugins based on the old i18n.c concept continue to work, or do they have to migrate to gettext? Will a converted plugin work under a non-gettext VDR version too?
If both concepts ar kept for a while, then I think it should be even possible that a converted plugin works on a non-gettext VDR, but only if the plugin itself kept the original concept too.
For the existing plugins, we imho need a smooth upgrade path, preferably some scripting-fu that turns i18n.c into .po files. Maybe this is also an option for compatibility: For backwards compatibility, some script turns .po files to old-style i18n.c files.
Well, even that should be possible, but I wonder if that's not effort in the wrong direction if we would really push the new gettext "way".
Lucian
Lucian Muresan wrote:
How does this translate into gettext? Does each plugin use its own textdomain(), and if, how does the domain switching happen?
I think so, every plugin should have it's own domain (therefore, po -> mo file). The switching would be done in the tr() macro (it's underlying function) when invoked with the plugin name argument
Just realized: There's a dgettext() that can be used instead of temporarily switching the domain.
If the translations are to be cached in memory, they would be read once at startup of VDR and each plugin, and a dynamically allocated Phrases[] array would be filled with their contents
Probably worth to investigate whether that is really better. If gettext has been implemented in a smart way, its probably much faster than the current VDR i18n implementation. And pre-loading all strings may also add to the VDR startup time.
Cheers,
Udo
Udo Richter wrote:
Lucian Muresan wrote:
[...]
If the translations are to be cached in memory, they would be read once at startup of VDR and each plugin, and a dynamically allocated Phrases[] array would be filled with their contents
Probably worth to investigate whether that is really better. If gettext has been implemented in a smart way, its probably much faster than the current VDR i18n implementation. And pre-loading all strings may also add to the VDR startup time.
Yes, that's sure worth to be investigated. We'll see what's actually better.
Lucian
I demand that Udo Richter may or may not have written...
[snip]
How does this translate into gettext? Does each plugin use its own textdomain(), and if, how does the domain switching happen?
It should; see dgettext(3).
Will plugins based on the old i18n.c concept continue to work, or do they have to migrate to gettext?
I'd give them some time (and output warnings).
Will a converted plugin work under a non-gettext VDR version too?
I see no reason why it shouldn't.
For the existing plugins, we imho need a smooth upgrade path, preferably some scripting-fu that turns i18n.c into .po files. [...]
I don't think so. Translatable strings should be marked up for extraction in the standard way: let the existing gettext tools do the work.
Am Mittwoch, 16. November 2005 14:54 schrieb Udo Richter:
On long term, its probably best to move i18n out of the c source and into editable language definition files that can be loaded on runtime. (I think thats what Klaus meant with 'language files'.)
I will improve Rangas Setup-Plugin to this feature. maybe we get some usefull experience for the vdr.
regards movimax
-
Darren Salt -
Iwan Davies -
Klaus Schmidinger -
Lucian Muresan -
Markus Hahn -
rollercoaster＠reel-multimedia.com -
Thiemo Gehrke
-
Udo Richter