xoops forums

wizanda

Home away from home
Posted on: 7/2 21:23
wizanda
wizanda (Show more)
Home away from home
Posts: 1588
Since: 2004/3/21
#1

Question About Unicode In PHP 5.3 to 5.6+

My site still runs an optimized form of Xoops, and my server is upgrading to only allow PHP 5.6+...

When I post on the forum, PM, comments, etc... With 5.3 it would convert Hebrew, Smilies to unicode; yet when trying 5.6 it puts question marks.

What in Xoops would be processing the code before it is posted, and where would need editing to fix it please?

wizanda

Home away from home
Posted on: 7/3 5:36
wizanda
wizanda (Show more)
Home away from home
Posts: 1588
Since: 2004/3/21
#2

Re: Question About Unicode In PHP 5.3 to 5.6+

Basically it is because since PHP 5.6+ the default value for the default_charset configuration setting is UTF-8, before that it was an empty string.

This can be fixed by adding this to the common.php file:

ini_set( 'default_charset', "" );

Which then makes sure the character set is the same as it was before, an empty string across the site.

Thanks for the help; I've learned to code because of Xoops.

wizanda

Home away from home
Posted on: 7/3 12:31
wizanda
wizanda (Show more)
Home away from home
Posts: 1588
Since: 2004/3/21
#3

Re: Question About Unicode In PHP 5.3 to 5.6+

The only issue with that solution of setting the same character set across the site, is any characters posted that don't fit into that system, then don't encode properly.

Could edit the files that post, to have an undefined character set; which then partially works, yet still doesn't encode properly in PHP 5.6+.

So guess I'm stuck, where I don't know how to solve this easily.

Mamba

Moderator
Posted on: 7/3 22:29
Mamba
Mamba (Show more)
Moderator
Posts: 11155
Since: 2004/4/23
#4

Re: Question About Unicode In PHP 5.3 to 5.6+

Quote:
With 5.3 it would convert Hebrew, Smilies to unicode; yet when trying 5.6 it puts question marks

Are they stored in the DB? If yes, couldn't you just convert the DB to UTF-8? This way the existing text would be displayed correctly, and new text would be stored as UTF-8.

Or am I missing something?
Support XOOPS => DONATE
Use 2.5.10 | Docs | Modules | Bugs

wizanda

Home away from home
Posted on: 7/5 9:11
wizanda
wizanda (Show more)
Home away from home
Posts: 1588
Since: 2004/3/21
#5

Re: Question About Unicode In PHP 5.3 to 5.6+

Quote:

Mamba wrote:

Are they stored in the DB? If yes, couldn't you just convert the DB to UTF-8? This way the existing text would be displayed correctly, and new text would be stored as UTF-8.

Or am I missing something?

It is on posting on the forum, articles, PM, comments, etc, when the system has an empty string for the character set in PHP 5.3, it translates characters to UTF8 into the database... Wiwimod doesn't translate them, and works on posting.

On PHP 5.6+ where it has UTF8 set as the default character set , it doesn't translate the characters, and instead puts them as question marks into the database.

The database is already UTF8.

Don't worry if you don't know what causes it, as got that many issues that need attending to, might have to archive the site, and start a new one.

Think really this is just to say thank you Xoops for teaching me to code; as I recoded the whole system of our site based on what Skalpa did, and then learned loads of ways to be smarter.

Mamba

Moderator
Posted on: 7/5 11:57
Mamba
Mamba (Show more)
Moderator
Posts: 11155
Since: 2004/4/23
#6

Re: Question About Unicode In PHP 5.3 to 5.6+

If you would start a new site, you might consider the upcoming XOOPS 2.5.11, which is very stable and is running on PHP 8

Reg. your UTF-8 issues, you might take a look at the TextSanitizer's Wiki extension, with the encode and decode functions. Maybe it will help...
Support XOOPS => DONATE
Use 2.5.10 | Docs | Modules | Bugs

geekwright

Quite a regular
Posted on: 7/5 21:18
geekwright
geekwright (Show more)
Quite a regular
Posts: 315
Since: 2010/10/15
#7

Re: Question About Unicode In PHP 5.3 to 5.6+

This is odd, because the problems switching to PHP 5.6 are usually solved by making sure the PHP configuration specifies
default_charset "utf-8"


You mentioned you just see question marks. Are those question marks or unicode replacement characters - question marks in diamonds?

What is the define for _CHARSET in language/*/global.php ?

Πρέπει να λειτουργεί;

wizanda

Home away from home
Posted on: 7/5 22:56
wizanda
wizanda (Show more)
Home away from home
Posts: 1588
Since: 2004/3/21
#8

Re: Question About Unicode In PHP 5.3 to 5.6+

Quote:

geekwright wrote:
This is odd, because the problems switching to PHP 5.6 are usually solved by making sure the PHP configuration specifies
default_charset "utf-8"


You mentioned you just see question marks. Are those question marks or unicode replacement characters - question marks in diamonds?

What is the define for _CHARSET in language/*/global.php ?

Πρέπει να λειτουργεί;

They're plain question marks: ????

The_CHARSET in language/*/global.php was: ISO-8859-1.

Can set that as UTF-8, yet when posting it still makes question marks.

With global.php as ISO-8859-1, and ini_set("default_charset", ""), it then encodes it... Yet then has odd character errors across the site.

Thank you for the help.