xoops forums

Mamba

Moderator
Posted on: 2013/1/7 16:10
Mamba
Mamba (Show more)
Moderator
Posts: 10960
Since: 2004/4/23
#11

Re: Proposal for a "universal language" system.

Quote:
This would NOT add complexity at the module level. The beauty is that in those places you CAN and DO use generic phrases as quoted above it would save a lot of work.

Extra table to have, extra query to run, that adds complexity to me, and it creates an extra point for potential.

The AltSys module offers a translation part, where you can modify things, if you don't like the current translation. Maybe we can look into it as well.

"Language Packs" are good. I hope that we'll have in near future a "Download/Update Module" in the Admin, which will:

- download the right language during installation
- allows user to download extra languages for each module or all modules
- checks and updates System module, and all other modules, incl. Language Packs.

Quote:
If in fact this looks like something that might be of interest I would be willing to try putting something together to show what I mean.

Let's wait for Trabis what he comes up with. He already has some good starting points, so I am looking forward to his work.

I think, it's better if you continue with the Installer and PDO stuff, because nobody else is working on it. We need to stay focused!

Speaking of which - I need to go back to the Basic Module Pack
Support XOOPS => DONATE
Use 2.5.10 | Docs | Modules | Bugs

redheadedrod

Home away from home
Posted on: 2013/1/7 17:48
redheadedrod
redheadedrod (Show more)
Home away from home
Posts: 1296
Since: 2008/2/26
#12

Re: Proposal for a "universal language" system.

No worries Mamba, I wouldn't consider working on anything else until I am comfortable with where the PDO and Install stuff is.

I hope to finish up the PDO stuff this week. A little behind but keep having family stuff come up. I do start classes next week but I want to be totally done with the PDO stuff (Other than bug testing) well before then. I will be working on the Install script after that until it is done. Which is why I mentioned the delay in working on anything else.

Outside of that.. The complexity would be contained within the core system and would not apply to the module. At this point it is worth considering and maybe Trabis can have it in the back of his mind when he is working on the implementation of the new system. The extra Queries and such would ONLY be used when the module would be installed/removed and if the language constants were changed. Assuming I understand where Trabis is headed, this proposed system could be added on afterwards and potentially could use whatever system Trabis comes up with and only effect the core system. May be able to add on a module/extension/addon to accomplish this. To incorporate it during install/removal of modules there may need to be some tweaking to the module install system as well but shouldn't be a big deal.

The important thing here is sharing ideas and trying to make things simpler..

Rodney

Mamba

Moderator
Posted on: 2013/1/7 18:04
Mamba
Mamba (Show more)
Moderator
Posts: 10960
Since: 2004/4/23
#13

Re: Proposal for a "universal language" system.

Quote:
The important thing here is sharing ideas and trying to make things simpler..

Absolutely! That's the whole idea behind a community discussion!
The progress can only happen if we look at issues from various perspectives and share our ideas!

Our goal is that we'll be the easiest CMS for internationalization/localization! So we need to keep things simple, efficient, and elegant

But we also need to look at ways to minimize memory consumption, and number of queries generated.
Support XOOPS => DONATE
Use 2.5.10 | Docs | Modules | Bugs

trabis

Core Developer
Posted on: 2013/1/8 0:36
trabis
trabis (Show more)
Core Developer
Posts: 2268
Since: 2006/9/1 1
#14

Re: Proposal for a "universal language" system.

What about using classes instead of ini files? Here is an example of a class:
include_once dirname(dirname(__FILE__)) . '/en/en.php';
class 
XoopsLocalePt extends XoopsLocaleEn {
    const 
EDIT 'Editar';
    const 
DELETE 'Apagar';
    const 
HOME 'Início';
    const 
SAVE 'Salvar';
}
And
<?php
include_once dirname(__FILE__) . '/pt.php';
class 
XoopsLocale extends XoopsLocalePt{
}
Example of usage:
include_once dirname(__FILE__) . '/global/pt/locale.php';

$x = new XoopsLocale();

echo 
$x::DELETE;
echo 
'<br>';
echo 
$x::EDIT;
echo 
'<br>';
echo 
$x::SAVE;
echo 
'<br>';
This is 3 times faster then defines and just a little bit slower then ini files. On the upper side, we gain autocompletion in file editors. Autocompletion is something we do have with defines and that we will miss if using ini. Since it is class based, languages can extend other languages. pt-PT can extend pt-BR, etc If a constant is missing, the parent class will provide the constant. This requires 2 files per module. One containing the translations, and other just for type casting(just extends the class that has the translations). Translations can be accessed by XoopsLocale::STRING_TO_TRANSLATE; PublisherLocale::STRING_TO_TRANSLATE; NewsLocale::STRING_TO_TRANSLATE; No need to prefix strings with _MI_MODULENAME_(STRING_TO_TRANSLATE) :) We can also use this classes to hold the current XoopsLocal methods. The same class provides translations(via const) and methods for localization like substr, date, formattimestamp, etc Updating translations is simple. Using the editor 'Structure view' you can see what constants are in grey and just update them. Even if a constant is missing for your language, it will fallback to the english version delivered by the module/core(or whatever class it extends). You can also use a diff tool and merge easily. If we need to parse the classes to insert the translations in db we can use php Reflection. One thing you cannot do: Force a string to be translated in another language. If the PT classes are loaded, you cannot force a module/core to display a string in EN. You cannot have two XoopsLocale loaded or two PublisherLocale loaded at the same time! A 'class already exists' will pop out. Anyway, even if using ini files would allow it, it would not be advised for performance reasons. I have prepared a package with some examples. There is a folder called memusage where you can compare performace of ini vs array vd define vs class.

irmtfan

Module Developer
Posted on: 2013/1/8 5:15
irmtfan
irmtfan (Show more)
Module Developer
Posts: 3419
Since: 2003/12/7
#15

Re: Proposal for a "universal language" system.

@Mamba:
I think you misunderstood me.
I dont want to move translations from "modules/MODULE/language" folder to "themes" folder. I want to provide extra functionality for customization

It means you can have your old method for translations as a module developer or translator but also you will have the chance to create different translations for each theme.
IMO Xoops should provide a mechanism that all "front side" customizations (that currently we are doing in different places like language folder of every module) would be implemented inside "themes" folder.

for example for xhelp module the priority for parsing translations would be:

1- IF EXISTS themes/YOUR_THEME/modules/xhelp/language/, TAKE IT;
2- ELSEIF EXISTS themes/default/modules/xhelp/language/, TAKE IT;
3- ELSE TAKE XOOPS_ROOT/modules/xhelp/language/

In current system in 2.5.5/2.6 we just have third option.

IMO it is beauty, very easy and simple with legacy support while open new doors for theme designers.

As i said in the last post it can be done for all "front side" files like images/icons/js/css

It has been done in newbb recent versions for images/icons and i enhanced it in "irmtfan branch" for css and js too.
My designer is really happy with it because he can build a "newbb independent template set" for each theme.

in current newbb version all these files are inside "modules/newbb/templates" folder (eg: modules/newbb/templates/css) but if you dont like that they can be inside the MODULE root as before for legacy support. (I see in 2.6 core developers put the media files in module root)

eg: newbb/language, newbb/images, newbb/icons, newbb/css and newbb/js

one can change that place easily.

the place of these files inside the MODULE is not important at all. Just we need to decide about a standard location. Only provide that customization functionality in "themes" folder is important.

@ Trabis:

Any system can provide those features in my last post is OK in my point of view.
Only at a first glance it seems it is hard for beginners to do that class method for their local languages unless they could do it with a GUI language tool.

I will test that class.

Edit:
Ok i test it.
I still cannot understand this:
Quote:

If a constant is missing, the parent class will provide the constant.


You said it cannot load 2 XoopsLocale class. then how it can parse missing constants in another language?

eg in pt that you mentioned lets say one constant is missing:
<?php
class ModuleLocaleEn {
    const 
MODULE_NAME 'My module';
    const 
MODULE_DESC 'My module description'// irmtfan
}

and in pt:
<?php
class ModuleLocalePt {
    const 
MODULE_NAME 'O meu módulo';
}


can MODULE_DESC be parsed in pt?

Edit2:
maybe you could do this? in module/pt/pt.php
<?php
include_once '../en/en.php';
class 
ModuleLocalePt extends ModuleLocaleEn{
    const 
MODULE_NAME 'O meu módulo';
}

irmtfan

Module Developer
Posted on: 2013/1/8 7:01
irmtfan
irmtfan (Show more)
Module Developer
Posts: 3419
Since: 2003/12/7
#16

Re: Proposal for a "universal language" system.

@ Trabis:
Ok i understand. you did it just for XoopsLocalePt so for ModuleLocalePt it would be:

<?php
include_once dirname(dirname(__FILE__)) . '/en/en.php';

class 
ModuleLocalePt extends ModuleLocaleEn{
    const 
MODULE_NAME 'O meu módulo';
}


IMO it is good method. Only simplicity should be concerned. ( is it hard for translators?)

About performance I cannot say much. It is certainly faster than current php define method.

trabis

Core Developer
Posted on: 2013/1/8 13:46
trabis
trabis (Show more)
Core Developer
Posts: 2268
Since: 2006/9/1 1
#17

Re: Proposal for a "universal language" system.

Quote:
irmtfan wrote: @ Trabis: Ok i understand. you did it just for XoopsLocalePt so for ModuleLocalePt it would be:
<?php
include_once dirname(dirname(__FILE__)) . '/en/en.php';

class 
ModuleLocalePt extends ModuleLocaleEn{
    const 
MODULE_NAME 'O meu módulo';
}
Correct. Most language should extend the english version. Having pt-PT extending pt-BR is possible but not advised because you force the user to have both the locales available. It is better to just copy all pt-BR class and then do the changes. As for methods, they will be only in the global class (XoopsLocale). The core languages are often delivered by translation team and they also have to create/maintain XoopsLocal(current) class, so I don't see a problem. If a user can edit defines, he can edit constants also. As for the performance, I know it will be better then defines so I'm not concerned. INI are faster and they use less memory because they are compact. My problem is mainly with the auto completion. I think you can imagine yourself opening your translation file(again and again) to see what was the key you where looking for. This happens to me with module configs and I want to avoid having the same problem with languages.

Mamba

Moderator
Posted on: 2013/1/8 16:01
Mamba
Mamba (Show more)
Moderator
Posts: 10960
Since: 2004/4/23
#18

Re: Proposal for a "universal language" system.

@ Trabis:
INI files are really fast, on your benchmark something like 30% faster than the Class solution, and over 200% when compared to Defines.

The Class solution is about 160% faster than Defines, so this is good news.

But the memory consumption surprised me. Defines consume around 20% of what all the others consume.

I like the fact that there will be only one language file for the whole module, instead of multiple files needed right now. So this will be a major improvement.

The question is: how will it work for solutions like the x-Center module and its method:
https://xoops.org/modules/news/article.php?storyid=6287

I would love to see each XOOPS module in the future to be "multilingual" out of the box. What I like about the Wishcraft's solution above is that there are no constraints for length of a string. In comparison, if we use language tags, e.g. [en]for English[/en] and [fr]for French[/fr], you can only have as many languages as it fits into a field, and that's normally is enough for 2-3 languages.

The above solution is perfect to have a true multilingual solution, regardless of whether we deal with 2 or 50 languages. I hope, we can come up with a seamless design, that would allow this for every future XOOPS module.

@ Irmtfan:
I totally understand your need for "design" customization at the theme level, and it makes perfect sense.
I think, we're in agreement that all the "default" stuff (i.e. translations, basic images) should be at the language level, so the user needs to download the "language pack" and be done with it.

If there is a new theme that he wants, then he downloads it, and if the designer decides to provide different images, CSS files, then it will be a "theme pack". I am still not sure why would we have a different translation for a theme, but from Core perspective, I don't see a problem, as long as there is no too much time penalty for checking all the different folders
Support XOOPS => DONATE
Use 2.5.10 | Docs | Modules | Bugs

redheadedrod

Home away from home
Posted on: 2013/1/8 17:22
redheadedrod
redheadedrod (Show more)
Home away from home
Posts: 1296
Since: 2008/2/26
#19

Re: Proposal for a "universal language" system.

I have NO experience with INI files but defines are very limited. The Class idea sounds the best to me for flexibility.

The only issue I see with classes is that in order to use one class you will need a class that contains all of the constants you want to use which could use alot of memory. The fact you can extend a class and replace the old constant with a new one is a very good thing. You can not do that with a define. Again I don't know how INI files work so I can only compare to defines.

With the define system didn't they split them up to save memory or was it solely to reduce page load speed? As to memory it should be compared to the % of the total over all used and see if it is a significant chunk. As I said, I think the only issue I can see with the classes is if you have one class that contains all of the language constants in it that class could be considerable in size. But again, maybe it is not a concern if PHP is efficient when storing strings.

Now I think I am going to go look at what the heck a INI file is... ;)

Rodney

trabis

Core Developer
Posted on: 2013/1/8 21:12
trabis
trabis (Show more)
Core Developer
Posts: 2268
Since: 2006/9/1 1
#20

Re: Proposal for a "universal language" system.

Quote:

Mamba wrote:

But the memory consumption surprised me. Defines consume around 20% of what all the others consume.


It bugs me too.
In fact, if I declare the class inside the test script(instead of using a separate file), the memory usage is also low (equal to define) - I guess it has to do on how php stores the constants and how it reports the memory usage.

I've also notice that, when I add more strings, the array method takes more memory per string than the class const. If I have just few lines, array takes less memory. If I add more lines, then the class const wins. I think the class makes use of memory just for the declaration, but that the strings take less memory then arrays.

I did some investigation and I found 2 very interesting articles:
http://imrannazar.com/Memory-Usage-of-Constants-in-PHP

and

http://nikic.github.com/2011/12/12/Ho ... rays-really-Hint-BIG.html