This is just to warn people of a possible problem - some recent discussion on the Perl 5 Porters list about Red Hat 8.0 and Perl 5.8.0 (the default Perl on this distribution) indicates that this Linux distribution sets the locale variables (LANG, LC_ALL, etc) to *.UTF8. This makes Perl open all files in UTF8 mode (i.e. expecting them to be Unicode in an 8-bit encoding). The result would be that TWiki code may well have problems with uploading binary attachments. However, it's possible that CGI.pm and existing fixes for TWiki to run on Windows take care of these issues.

If anyone is using this combination for TWiki, it would be useful to know.

-- RichardDonkin - 29 Nov 2002

I had lots of other problems with RedHat 8.0 related to UTF as well (not just perl, java as well). I fixed all (at least the ones I knew about) by simply setting "export LANG=en_US" in my .profile

-- JohnCavanaugh - 30 Nov 2002

Red Hat 7.2 seems to have UTF8 locales as well, but I think that the en_US ones use ISO-8859-1 whereas the en_US.UTF8 ones use UTF8. You would need to set the TWiki $siteLocale to en_US.ISO-8859-1 since the last part is used by TWiki I18N as the character set in the HTTP headers and HTML meta tags.

-- RichardDonkin - 20 Dec 2002

I tried Twiki on RH7.1 and it worked beautiful. I upgraded my box to RH8.0 and nothing works anymore. testenv does not even read the TWiki.cfg file. I'll have to search a solution.

-- LucBos - 13 Jan 2003

Later builds of Perl 5.8, e.g. 5.8.1, may fix this problem - this patch requires a command line switch to use 'UTF8 everywhere' mode. Not sure if later builds of 5.8.0 will have this patch, but 5.8.1 will do when it arrives.

-- RichardDonkin - 27 Jan 2003

I found a possible solution, but I'm not sure of it's full ramifications, so use at your own risk: If you're on RH8, you can change the LANG variable named in /etc/sysconfig/i18n from LANG="en_US.UTF-8" to LANG="en_US.iso885915". That has seemed to fix most of my UTF-8 nightmares. This should also fix things such as the Konsole in KDE from displaying boxes where a dash is suppoed to be (e.g. on man pages).

A more detailed description is as follows: The Bash executable reads /etc/profile first, before ~/.bash_profile, ~/.bash_login, or ~/.profile. It is in this file (/etc/profile) that it executes all scripts in the /etc/profile.d directory. One file in particular called lang.sh is responsible for setting up your LANG variable. If you read this file, you will see that the /etc/sysconfig/i18n file is sourced to get your LANG setting. Simply change /etc/sysconfig/i18n to have the appropriate language in LANG, and you should be set. AGAIN, I don't know the full effects of this change, so DO SO AT YOUR OWN RISK.

Hope this helps.

-- ToddLaWall - 28 Feb 2003

Setting lang.sh would be useful for many applications, but I'm not sure why it is affecting TWiki - perhaps the Perl interpreter is picking up this value, but bash is not normally involved in CGI applications such as TWiki. You can set the TWiki.cfg file's $siteLocale variable to the value you would use in the LANG variable in TWikiRelease01Feb2003 or later - see InternationalisationEnhancements.

-- RichardDonkin - 02 Mar 2003

Red Hat 9 has similar issues with Perl 5.8 - they've shipped a maintenance snapshot (post-5.8.0) that has some UTF8 bugs, although apparently less than. As with Red Hat 8, setting LANG=C or at least something other than *.UTF-8 is recommended in order to use MakeMaker (i.e. installing CPAN modules using perl Makefile.PL; make; make install). TWiki doesn't normally require such modules on Linux, but it's worth knowing just in case.

-- RichardDonkin - 13 May 2003

Perl 5.8.1 (or later) doesn't use Unicode by default, and provides the -C switch and the PERL_UNICODE environment variable, which control use of Unicode globally by a script. See perl581delta.

-- RichardDonkin - 13 Feb 2004

---

Related Pages:

• InternationalisationEnhancements for links on Unicode etc
• InternationalisationUTF8 for discussion on (future) UTF-8 support in TWiki
• FailedAuthenticationWithApache2OnRedHat8, TWikiOnLinux, TWikiOnRedHat
• IssuesWithPerl5dot8 - links to generic 5.8 issues