11
bjuti
Re: upgrade problem with Croatian characters
  • 2009/11/24 22:07

  • bjuti

  • Just can't stay away

  • Posts: 871

  • Since: 2009/1/7 2


Quote:
bjuti koja je kod tebe prije verzija xoopsa bila(pre 2.4) 2.33?


2.33 tj po?eo je sajt da radi na 2.3.2

---

I'm still using 2.3.3, but the site was builded from scratch on 2.3.3, so I had no conversion problems. Just letter "š" is "scaron"-something.

12
wishcraft
Re: upgrade problem with Croatian characters

sacron is utf-8 and there is a couple other versions of it with utf-16.. there is an object type for handling this with modules if you check object.php you will find the following class intialisation types:

define('XOBJ_DTYPE_UNICODE_TXTBOX'16);
define('XOBJ_DTYPE_UNICODE_TXTAREA'17);
define('XOBJ_DTYPE_UNICODE_URL'18);
define('XOBJ_DTYPE_UNICODE_EMAIL'19);
define('XOBJ_DTYPE_UNICODE_ARRAY'20);
define('XOBJ_DTYPE_UNICODE_OTHER'21);


These will allow forums and any form of module to handle any form of unicode that is utf-8, utf-16, utf-32 all you have to do for example is change the handler type in the class and 2.4.0 will allow your user to free type into any field and have it displayed as is.

As URL's will soon support korean, chinese, greek, most asia pacific and eastern countries characters define XOBJ_DTYPE_UNICODE_URL is very important for the handling of this so there is no corruption due to database character support or character sets. This will for xample store arabic characters in a latin character set MySQL database.

13
ghia
Re: upgrade problem with Croatian characters
  • 2009/11/25 0:51

  • ghia

  • Community Support Member

  • Posts: 4953

  • Since: 2008/7/3 1


This is relying on the MySQL ability of converting the database tables and all to easy.
The big problem is that the definition of the character set of the table and the characters that are effectively stored in, does not match.
For example the Latin 1 set contains 191 characters, while on the byte level 256 posibilities exists. Mosttimes the data comes from the use of the character set of the users computer, eg MS Windows and set Win-1252, which contains 219 characters.
That makes that the conversion from Latin-1 to UTF-8 is undefined for some 28 characters, which are now shown as the diamonds with question marks.

I think, a possible way to correct this is that you need to get your saved tables from the latin1 backup in SQL mode and then convert this to UTF-8 before restoring it to the server.


14
wishcraft
Re: upgrade problem with Croatian characters

Yeah I Know ghai that is why I made those options in the object.php so there was no matter what was used to store the information on it still would.

It uses the XOOPS encoding options to encode the field for storage and uses around 64 characters plus punction to store it. Currently it utilises URLENCODE and URLDECODE to convert characters from an escaped unicode (UTF-8 etc) format to a flat file like storage of it..

This is because in 3.0 a PDO is included which not all database support UTF-8 and have only limited storage. It is convert on the fly by XOOPS not MySQL with these fields, so it is stored in a flat file format like arrays are in the database and then encoded on the way back out to the console or screen.

15
bumciach
Re: upgrade problem with Croatian characters
  • 2009/11/25 9:55

  • bumciach

  • Not too shy to talk

  • Posts: 153

  • Since: 2007/6/25


Quote:

ghia wrote:
Your database seems to be latin1. The collation is not reported.
Change mainfile.php and global.php to use latin1 and ISO-....


Not exactly. In mainfile.php and global.php set what we want to have the output. So we set the same encoding in which the files are encoded (and what is set into section in theme.html).
Setting the database has no meaning there. Provided that the characters in the database are not corrupted.

On my example. In the datebase I have latin2 and collation latin2_general_ci. But my pages are utf-8 so the mainfile.php is set to utf-8 and utf8_general_ci. It works without problems, although the encoding of the database and XOOPS is different.
It is simple only difficult to understand (a very long time I could not deal with it).

16
ghia
Re: upgrade problem with Croatian characters
  • 2009/11/25 10:45

  • ghia

  • Community Support Member

  • Posts: 4953

  • Since: 2008/7/3 1


This strategy of further using an existing latin database and a UTF-8 HTML, works only for the strict character set of the database and for so far these tables have not been corrupted by other users' character sets as explained in my previous post.

You can not using it for storing other characters (from different language character sets) in this database, except with the use of encoding/decoding techniques as outlined by wishcraft, which complicates information retrieval (by eg search) in the database.

17
bjuti
Re: upgrade problem with Croatian characters
  • 2010/1/28 23:46

  • bjuti

  • Just can't stay away

  • Posts: 871

  • Since: 2009/1/7 2


Any idea how to resolve that š issue (instead of Š)? All other characters are ok, only one letter is the problem?!

Plz help!

18
ghia
Re: upgrade problem with Croatian characters
  • 2010/1/29 0:39

  • ghia

  • Community Support Member

  • Posts: 4953

  • Since: 2008/7/3 1


And if you use latin2 in mainfile and ISO-8859-2 in global?

19
bjuti
Re: upgrade problem with Croatian characters
  • 2010/1/29 9:45

  • bjuti

  • Just can't stay away

  • Posts: 871

  • Since: 2009/1/7 2


My database is in utf-8 from the begining. Therefore I say that i have problem with only one letter, eg. character. In other CMSes there is no such bug (or whatever it is). :)

My language files are in utf-8, and utf-8 is defined in mainfile.php. :)

20
ghia
Re: upgrade problem with Croatian characters
  • 2010/1/29 10:48

  • ghia

  • Community Support Member

  • Posts: 4953

  • Since: 2008/7/3 1


Sorry, I was confused by the answer of bumciach
Quote:
On my example. In the datebase I have latin2 and collation latin2_general_ci. But my pages are utf-8 so the mainfile.php is set to utf-8 and utf8_general_ci. It works without problems, although the encoding of the database and XOOPS is different.
Tested here with XOOPS Version - XOOPS 2.4.3, PHP Version - 5.2.8, MySQL Version - 5.1.30-community and Article 2.0 with DHTML editor without problems using the pangram.

Login

Who's Online

320 user(s) are online (222 user(s) are browsing Support Forums)


Members: 0


Guests: 320


more...

Donat-O-Meter

Stats
Goal: $100.00
Due Date: Nov 30
Gross Amount: $0.00
Net Balance: $0.00
Left to go: $100.00
Make donations with PayPal!

Latest GitHub Commits