Multilanguality seems to be one of the most important things to put in the core as soon as possible. The longer it's postponed, the harder it will be to do correct - without breaking alot of things.

To make it possible to search and sort contens from all supported languages, requires intimate knowledge of many writing systems, as some will write all or part of their contens "backwards".

A propper sort-algorithm will also have to though up, in order to sort latin and non-latin leters correctly. (It's allready a problem with the memberlist, if one uses multi-byte chars)

Best regards,

Also to enable easy multi-language support, all graphical elements containing text should be removed, and replaced by text or symbols.

About 18 hours ago I was told about The Open Group, and their technical standard on Context-Dependent and Directional Text.

I've only had time to read the first 25 pages, but I've realized it's a very important document for XOOPS core developers (and others) to read.

The technical standard (ISBN 1-85912-142-X January 1997), is a very detailed standard which in 90 pages describe most problems with multilanguage systems supporting the so-called bidirectional text. It's available online for free in both html and pfd.
To download, go here: http://www.opengroup.org/products/publications/catalog/c616.htm.

Supporting a multitude of languages is much more complex then I initially thought. The above mentioned work, introduces the most problematic written systems, which is Arabic, Urdu, Farsi, Hebrew, Yiddish, Thai and Korean - several is not directly described, but covered by the others - they include Devanagari, Gurmukhi, Gujarati, Oriya, Bengali, Tamil, Telugu, Malayalam, Sinhaese, Khmer (Cambodian), Lao, Mongolian, Chinese and Tibetan.
It reveals many important things - most of which will have to go into the XOOPS core to give it it's best impact on the overall system.

The pdf document is only about 90 pages, and introduces many c-functions dealing with the problems faced by applications in order to store data in a search and display-able manner.

Best regards.

Quote:

This is great news. As soon as I have time for it, I'll collect all the multilangual hacks I've made, for you to include.

Skalpa,

Your description sounds good.
You obviously has alot more expierence with databases then I do.


There's one part where I disagree:

Quote:

We have allready seen the speed old modules (from 1.2/1.3) are converted into 2.0 - which is not very impressive.
I'm afraide we will end up with most modules beeing single language, and users would then have to hack the modules to make them work on a multilanguage site. It's not far from like it is today.

IMHO, most multilanguage things should happen per default in the core, but module writers should be able to take control if needed.

Quote:

This is pretty off-topic, so I'll answer quickly and then go away

I tried it shortly on two machines, and was not impressed.
A 1.2GHz machine could barely maintain a problem free voice communication to a 2.4GHz machine on the same ethernet hub (with low networking trafic). Typical telephones require 8kbyte/s of data or less transmitted, but SkyPe went way beyond 90kbytes/s, and had a worse sound quality.
I was also not very impressed with the GUI.

I may take a look at it again, once it's more mature.

Best regards,

Quote:

I agree.

Some thoughts on how it could be implemented:

Xoops has core functions to help developers display just about anything on the screen. If we want a heading, wi'll call the heading routine in the core. If we want to display a string, we'll call another core routine to display the string. But if we want to edit the string, then we will call the same core routine but with another set of parameters.

I belive (but I haven't checked) it's the same for database access, call a routine to fetch information and another routine to store it. (If it's not there, then it's probably about time to have it implemented!)

This is something I belive can be expolited.

If every user gets the opertunity to make a prioritized list of languages which the user would like to see aswell as not to see, then the routine which is responsible for fetching data in the database could always return the first language in the list which had any text entered.
It would probably be relatively easy to implement.

Entering the texts would be abit more tricky, but by pathcing the right core functions - such as the ones to edit a string, described above - it would be possible to make a loop inside the core function, which would return eg. four strings with four different languages, instead of the one string that was origially ordered by the programmer.
The user entering the text would then enter text into the strings he wants to, and the core function responsible for storing the text will automatically distribute the text into the correct language threads in the database.

I don't know if it's realistic approach to addmultilagual support in this fasion, but it seems to me to be a relative simple way to reach most of the goals a multilangual site could want. (Although, it will not allow you to link to a specific languageversion of a certan article!)

Please think it over.

Quote:

I like this idear - even if the amount of languages supported by those translators are relative limited.

Best regards.

Quote:

First of all, this approach will not remove the square brackets for the common user.
Second, we would have to introduce a function a disassemble the line, and sort it into various threads in the database.
And third, if somebody desided to edit the text, we would have to assemble the line with square brackets again from the various threads in the database.

I fear it will be very troublesome.

Just a quick note. (I'll be back with more later).

joelg wrote:
Quote:

I agree. For most people it's not simple, and thus a better solution should be developed!
However, we should also keep in mind that for some systems the square bracket approach would be alot easier to use. We should also keep in mind that there are several other hacks where square brackets in combination with the text-sanitizer are beeing used.
Therefore I think the multilangual hack should be integrated into the system, and system and module designers should be taught to parse all non-static text (text not included in the language files) though the text-sanitizer unless the text is to be edited.

Based on your message, mvabdam has started this very interesting thread.

Everybody using or thinking about making a multilangual site is encuraged to participate in the discussion.

Best regards,

I'm sorry for beeing quiet for so long.

Beeing one of the people hacking away to implement the square brackets system to create a multilangual system, I've slowly started to form an opinion about how to implement a multilangual system.

I'll start by going though the square brackets approach, then some idears I have to implement the square brackets approach in an easier manner then currently done and finaly an idear on how to make a overall better multilangual system.

The square brackets approach does work, but it's not easy to maintain and it's a huge database "bandwidth" waster. It requires ALL text not part of the language files to be routed though the textsanitizer. A few exceptions exixts, such as the places where you edit the lines containing the square brackets, typically in admin.

The good thing about the square bracket approach, is that everything not included in square brackets will be treated as common language information.
The bad thing is that you need to remember to include the language specific in square brackets containing the language name, and that you will edit all languages at the same time.


The square bracket technics does indeed deal with all parts of xoops, including content, titles, administration, and block titles. BUT many of these still need to be hacked.
I have a number of hacks still not published on this site, but they will be as soon as time allows.


I have an idear on how to implement the hack in a global fasion. The idear is based on my lowlevel programming skills - I have no idear if it's possible within php, XOOPS or in another fasion. (The only php I know is from hacking Xoops).

I'll base this description on a C alike syntax.

Any kind of text comming out of a C program will come out one letter at a time, usualy though a routine called putchar(). The putchar rutine is the one one will patch if you need to convert from eg. Mac line-terminaltion to PC/Windows line-termination.
If XOOPS or php implements a function like that, then it would be possible to add the textsanitizer there, and thus make the system and all existing modules support the square brackets approact. The maximum linelength in both the system/modules and database will still have to be ajusted, though.
A person with intimate php knowledge may be able to make this hack.


Now for the overall system to become multilangual will require alot more the the square brackets approach.
The multilangual system I'm currently working on, uses danish, japanese and english. The common language is english, but the danes are learning japanese, and the japanese are learning danish.
This means that if a message is written in one language only, then people reading one or the other languages should still see that message, even if it's not their selected language. To combat problems with corrupted text, all contens is using multibytes chars.

One way I can think of to implement this, is to have (in this case) three databases running, one for each language.
At any time a message is entered, it should be saved only in the database for the language selected when typing the message.
If the message is then displayed in another language, then it should automatically select the language it was originally written in. If I choose to edit the message when using another language then the one it was written in, then it should ask for a translation, and save the new translation in the current language database. If two or more translations exists, and you select to view or edit the message using another language, then it should use a prioritized list to select which translation to display.
Many aspects of this still need to be ironed out, but I belive the basis to implement this approach is available.
It could be made as a transparent layer, where the database access is controlled by some core functions, which are matched to the language selector.
A special translator module may need to be made, too, where one can select one of many source languages, and a destination language - possibly with a personal selectable preferences setting. I imagine two senarios for this translator module - one is to edit existing contents, the other is to list and edit any untranslated contents.

I'm looking forward to a discussion about this important subject.

Best regards,