Question

I'm having a problem with TWiki 4.2 and mod_perl. After I start or restart Apache, things (continue to) work normally for a few seconds, then certain links start failing with 500 errors. This persists for a minute or so, then things return to normal. The links are Search, Changes, Preferences, and sometimes RSS. The problem did not happen when I ran TWiki 4.2 on Ubuntu 6.06, and it happens regardless of client OS/Browser.

I asked about this on IRC; scroll down to 18:02:

http://koala.ilog.fr/twikiirc/bin/irclogger_log/twiki?date=2008-07-10,Thu

Typically, this is the error I see in the browser:

TWiki detected an internal error - please check your TWiki logs and webserver logs for more information.

Can't locate TWikipath in @INC (@INC contains: path path path path path path path path . path)

Here is the corresponding TWiki log entry:

| 10 Jul 2008 - 11:04 | Can't locate TWiki/Plurals.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.8.8 /usr/local/share/perl/5.8.8 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8 /usr/local/lib/site_perl . /etc/apache2) at /var/data/twiki/lib/TWiki/Render.pm line 542.
 at /var/data/twiki/lib/TWiki/Render.pm line 542
        TWiki::Render::_renderWikiWord('TWiki::Render=HASH(0x8edaf48)', 'Main', 'TWikiUsers', 'Users', '', 1, 'undef') called at /var/data/twiki/lib/TWiki/Render.pm line 530
        TWiki::Render::internalLink('TWiki::Render=HASH(0x8edaf48)', 'Main', 'TWikiUsers', 'Users', '', 1, 'undef', 1) called at /var/data/twiki/lib/TWiki/Render.pm line 723
        TWiki::Render::_handleSquareBracketedLink('TWiki::Render=HASH(0x8edaf48)', 'Main', 'WebPreferences', 'T', 'undef') called at /var/data/twiki/lib/TWiki/Render.pm line 1125
        TWiki::Render::getRenderedVersion('TWiki::Render=HASH(0x8edaf48)', '---+!! <nop>Main Web Preferences\x{a}\x{a}The following settings are ...', 'Main', 'WebPreferences') called at /var/data/twiki/lib/TWiki/UI/View.pm line 397
        TWiki::UI::View::_prepare('---+!! <nop>%WEB% Web Preferences\x{a}\x{a}The following settings are...', 'TWiki=HASH(0x8edb3e0)', 'Main', 'WebPreferences', 'TWiki::Meta=HASH(0x8dc2d58)', 0) called at /var/data/twiki/lib/TWiki/UI/View.pm line 377
        TWiki::UI::View::view('TWiki=HASH(0x8edb3e0)') called at /var/data/twiki/lib/TWiki/UI.pm line 159
        TWiki::UI::__ANON__() called at /var/data/twiki/lib/CPAN/lib//Error.pm line 379
        eval {...} called at /var/data/twiki/lib/CPAN/lib//Error.pm line 371
        Error::subs::try('CODE(0x8c7e040)', 'HASH(0x8ec7220)') called at /var/data/twiki/lib/TWiki/UI.pm line 197
        TWiki::UI::run('CODE(0x86204f0)', 'view', 1) called at /var/data/twiki/bin/view line 32
        ModPerl::ROOT::ModPerl::Registry::var_data_twiki_bin_view::handler('Apache2::RequestRec=SCALAR(0x8c73eac)') called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 204
        eval {...} called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 204
        ModPerl::RegistryCooker::run('ModPerl::Registry=HASH(0x8b03110)') called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 170
        ModPerl::RegistryCooker::default_handler('ModPerl::Registry=HASH(0x8b03110)') called at /usr/lib/perl5/ModPerl/Registry.pm line 31
        ModPerl::Registry::handler('ModPerl::Registry', 'Apache2::RequestRec=SCALAR(0x8c73eac)') called at TWiki/Plurals.pm line 0
        eval {...} called

In addition to the above, I sometimes get the following output when I click Search:

Could not perform search. Error was: Can't locate TWiki/Search.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.8.8 /usr/local/share/perl/5.8.8 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8 /usr/local/lib/site_perl . /etc/apache2) at /var/data/twiki/lib/TWiki.pm line 1549. at /var/data/twiki/lib/TWiki.pm line 1549 TWiki::search('TWiki=HASH(0x8289468)') called at /var/data/twiki/lib/TWiki.pm line 3402 TWiki::__ANON__() called at /var/data/twiki/lib/CPAN/lib//Error.pm line 379 eval {...} called at /var/data/twiki/lib/CPAN/lib//Error.pm line 371 Error::subs::try('CODE(0x8e4a13c)', 'HASH(0x8b37d58)') called at /var/data/twiki/lib/TWiki.pm line 3409 TWiki::SEARCH('TWiki=HASH(0x8289468)', 'TWiki::Attrs=HASH(0x8b37e60)', 'WebSearch', 'TWiki', 'TWiki::Meta=HASH(0x8e4ad60)') called at /var/data/twiki/lib/TWiki.pm line 2659 TWiki::_expandTagOnTopicRendering('TWiki=HASH(0x8289468)', 'SEARCH', ' \x{a}"" \x{a}type="word"\x{a}scope="" \x{a}web="" \x{a}nosearch="" \x{a}zeroresults=...', 'WebSearch', 'TWiki', 'TWiki::Meta=HASH(0x8e4ad60)') called at /var/data/twiki/lib/TWiki.pm line 2581 TWiki::_processTags('TWiki=HASH(0x8289468)', '---+ %MAKETEXT{"Web Search"}%\x{a}\x{a}%IF{"defined search" then=\'%I...', 'CODE(0x85a82e0)', 16, 'WebSearch', 'TWiki', 'TWiki::Meta=HASH(0x8e4ad60)') called at /var/data/twiki/lib/TWiki.pm line 2505 TWiki::expandAllTags('TWiki=HASH(0x8289468)', 'SCALAR(0x85c6444)', 'WebSearch', 'TWiki', 'TWiki::Meta=HASH(0x8e4ad60)') called at /var/data/twiki/lib/TWiki.pm line 3233 TWiki::INCLUDE('TWiki=HASH(0x8289468)', 'TWiki::Attrs=HASH(0x8e4a508)', 'WebSearch', 'Main', 'TWiki::Meta=HASH(0x8b06ccc)') called at /var/data/twiki/lib/TWiki.pm line 2659 TWiki::_expandTagOnTopicRendering('TWiki=HASH(0x8289468)', 'INCLUDE', '"TWiki.WebSearch"', 'WebSearch', 'Main', 'TWiki::Meta=HASH(0x8b06ccc)') called at /var/data/twiki/lib/TWiki.pm line 2581 TWiki::_processTags('TWiki=HASH(0x8289468)', '%INCLUDE{"%SYSTEMWEB%.%TOPIC%"}%\x{a}', 'CODE(0x85a82e0)', 16, 'WebSearch', 'Main', 'TWiki::Meta=HASH(0x8b06ccc)') called at /var/data/twiki/lib/TWiki.pm line 2505 TWiki::expandAllTags('TWiki=HASH(0x8289468)', 'SCALAR(0x85ada94)', 'WebSearch', 'Main', 'TWiki::Meta=HASH(0x8b06ccc)') called at /var/data/twiki/lib/TWiki.pm line 2857 TWiki::handleCommonTags('TWiki=HASH(0x8289468)', '%INCLUDE{"%SYSTEMWEB%.%TOPIC%"}%\x{a}', 'Main', 'WebSearch', 'TWiki::Meta=HASH(0x8b06ccc)') called at /var/data/twiki/lib/TWiki/UI/View.pm line 396 TWiki::UI::View::_prepare('%INCLUDE{"%SYSTEMWEB%.%TOPIC%"}%\x{a}', 'TWiki=HASH(0x8289468)', 'Main', 'WebSearch', 'TWiki::Meta=HASH(0x8b06ccc)', 0) called at /var/data/twiki/lib/TWiki/UI/View.pm line 377 TWiki::UI::View::view('TWiki=HASH(0x8289468)') called at /var/data/twiki/lib/TWiki/UI.pm line 159 TWiki::UI::__ANON__() called at /var/data/twiki/lib/CPAN/lib//Error.pm line 379 eval {...} called at /var/data/twiki/lib/CPAN/lib//Error.pm line 371 Error::subs::try('CODE(0x8388140)', 'HASH(0x8b0765c)') called at /var/data/twiki/lib/TWiki/UI.pm line 197 TWiki::UI::run('CODE(0x861a568)', 'view', 1) called at /var/data/twiki/bin/view line 32 ModPerl? ::ROOT::ModPerl::Registry::var_data_twiki_bin_view::handler('Apache2::RequestRec=SCALAR(0x8ddd2e8)') called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 204 eval {...} called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 204 ModPerl? ::RegistryCooker::run('ModPerl::Registry=HASH(0x82f473c)') called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 170 ModPerl? ::RegistryCooker::default_handler('ModPerl::Registry=HASH(0x82f473c)') called at /usr/lib/perl5/ModPerl/Registry.pm line 31 ModPerl? ::Registry::handler('ModPerl::Registry', 'Apache2::RequestRec=SCALAR(0x8ddd2e8)') called at TWiki/Search.pm line 0 eval {...} called

Here is my Apache config, essentially the same as given by http://twiki.org/cgi-bin/view/TWiki/ApacheConfigGenerator:

BrowserMatchNoCase ^Accoona blockAccess
BrowserMatchNoCase ^ActiveAgent blockAccess
BrowserMatchNoCase ^Attache blockAccess
BrowserMatchNoCase BecomeBot blockAccess
BrowserMatchNoCase ^bot blockAccess
BrowserMatchNoCase Charlotte/ blockAccess
BrowserMatchNoCase ^ConveraCrawler blockAccess
BrowserMatchNoCase ^CrownPeak-HttpAgent blockAccess
BrowserMatchNoCase ^EmailCollector blockAccess
BrowserMatchNoCase ^EmailSiphon blockAccess
BrowserMatchNoCase ^e-SocietyRobot blockAccess
BrowserMatchNoCase ^Exabot blockAccess
BrowserMatchNoCase ^FAST blockAccess
BrowserMatchNoCase ^FDM blockAccess
BrowserMatchNoCase ^GetRight/6.0a blockAccess
BrowserMatchNoCase ^GetWebPics blockAccess
BrowserMatchNoCase ^Gigabot blockAccess
BrowserMatchNoCase ^gonzo1 blockAccess
BrowserMatchNoCase ^Google\sSpider blockAccess
BrowserMatchNoCase ^ichiro blockAccess
BrowserMatchNoCase ^ie_crawler blockAccess
BrowserMatchNoCase ^iGetter blockAccess
BrowserMatchNoCase ^IRLbot blockAccess
BrowserMatchNoCase Jakarta blockAccess
BrowserMatchNoCase ^Java blockAccess
BrowserMatchNoCase ^KrakSpider blockAccess
BrowserMatchNoCase ^larbin blockAccess
BrowserMatchNoCase ^LeechGet blockAccess
BrowserMatchNoCase ^LinkWalker blockAccess
BrowserMatchNoCase ^Lsearch blockAccess
BrowserMatchNoCase ^Microsoft blockAccess
BrowserMatchNoCase ^MJ12bot blockAccess
BrowserMatchNoCase MSIECrawler blockAccess
BrowserMatchNoCase ^MSRBOT blockAccess
BrowserMatchNoCase ^noxtrumbot blockAccess
BrowserMatchNoCase ^NutchCVS blockAccess
BrowserMatchNoCase ^RealDownload blockAccess
BrowserMatchNoCase ^Rome blockAccess
BrowserMatchNoCase ^Roverbot blockAccess
BrowserMatchNoCase ^schibstedsokbot blockAccess
BrowserMatchNoCase ^Seekbot blockAccess
BrowserMatchNoCase ^SiteSnagger blockAccess
BrowserMatchNoCase ^SiteSucker blockAccess
BrowserMatchNoCase ^Snapbot blockAccess
BrowserMatchNoCase ^sogou blockAccess
BrowserMatchNoCase ^SpiderKU blockAccess
BrowserMatchNoCase ^SpiderMan blockAccess
BrowserMatchNoCase ^Squid blockAccess
BrowserMatchNoCase ^Teleport blockAccess
BrowserMatchNoCase ^User-Agent\: blockAccess
BrowserMatchNoCase VoilaBot blockAccess
BrowserMatchNoCase ^voyager blockAccess
BrowserMatchNoCase ^W3C blockAccess
BrowserMatchNoCase ^w3search blockAccess
BrowserMatchNoCase ^Web\sDownloader blockAccess
BrowserMatchNoCase ^WebCopier blockAccess
BrowserMatchNoCase ^WebDevil blockAccess
BrowserMatchNoCase ^WebSec blockAccess
BrowserMatchNoCase ^WebVac blockAccess
BrowserMatchNoCase ^Webwhacker blockAccess
BrowserMatchNoCase ^Webzip blockAccess
BrowserMatchNoCase ^Wells blockAccess
BrowserMatchNoCase ^WhoWhere blockAccess
BrowserMatchNoCase www\.netforex\.org blockAccess
BrowserMatchNoCase ^WX_mail blockAccess
BrowserMatchNoCase ^yacybot blockAccess
BrowserMatchNoCase ^ZIBB blockAccess
BrowserMatchNoCase ^$ blockAccess


PerlSwitches -T 
PerlRequire /var/data/twiki/tools/mod_perl_startup.pl

ScriptAlias /twiki/bin "/var/data/twiki/bin"
Alias /twiki "/var/data/twiki"

<Directory "/var/data/twiki/bin">
    AllowOverride None
    Order Allow,Deny
    Allow from all
    Deny from env=blockAccess

    Options ExecCGI FollowSymLinks
    SetHandler cgi-script

    AuthUserFile /var/data/twiki/data/.htpasswd
    AuthName 'Enter your WikiName'
    AuthType Basic

    ErrorDocument 401 /twiki/bin/view/TWiki/TWikiRegistration

    <FilesMatch "^(configure)$">
        SetHandler cgi-script
        Order Deny,Allow
        Allow from all
        Require user admin
    </FilesMatch>

    <FilesMatch "(attach|edit|manage|rename|save|upload|view|rest|.*auth).*">
        SetHandler perl-script
        PerlResponseHandler ModPerl::RegistryPrefork
        PerlSendHeader On
        PerlOptions +ParseHeaders
    </FilesMatch>
</Directory>

<Directory "/var/data/twiki/pub">
    Options None
    AllowOverride None
    Order Allow,Deny
    Allow from all
    Deny from env=blockAccess

    # This line will redefine the mime type for the most common types of scripts
    AddType text/plain .shtml .php .php3 .phtml .phtm .pl .py .cgi
</Directory>

<Directory "/var/data/twiki/pub/Trash">
   Deny from all
</Directory>

<Directory "/var/data/twiki/data">
    Deny from all
</Directory>

<Directory "/var/data/twiki/templates">
    Deny from all
</Directory>

<Directory "/var/data/twiki/lib">
    Deny from all
</Directory>

<Directory "/var/data/twiki/locale">
    Deny from all
</Directory>

<Directory "/var/data/twiki/tools">
    Deny from all
</Directory>

<Directory "/var/data/twiki/working">
    Deny from all
</Directory>

And here's my mod_perl startup script:

#!/usr/bin/perl -w
use strict;
use warnings;

$ENV{MOD_PERL} =~ /mod_perl/ or die "mod_perl_startup called, but mod_perl not used!";

use ModPerl::RegistryLoader;
use File::Spec;

my $binurlbase = '/twiki/bin'; # must be set by the user
my $binbase = '/var/data/html/twiki/bin'; # must be set by the user
my $registrymodule = 'ModPerl::RegistryPrefork'; # must be set by the user

my $rl = ModPerl::RegistryLoader->new(
   package => $registrymodule, # must be set by the user
);

chdir $binbase;

foreach my $binscript (<$binbase/*>) {
   my (undef,undef,$scriptname) = File::Spec->splitpath($binscript);
   $scriptname =~ m|^([^/]+)$|;
   my $script = $1;
   if ($script !~ m/configure|register|resetpasswd/) { # don't precompile uncommon commands especially configure which has a ton of unnecessary s***
            $binscript =~ m|^(.+)$|;
            my $bin = $1;
            open BINSCRIPT,'<',$binscript;
            if (<BINSCRIPT> =~ m|^\#\!/usr/bin/perl|) {
                    $rl->handler("$binurlbase/$script", $bin);
            }
            close BINSCRIPT;
   }
}

1;

These two config files incorporate a couple of improvements from my IRC conversation, but don't solve the issue. I tried switching to the worker MPM and the corresponding mod_perl registry, but that just broke the scripts permanently, rather than transiently.

I'm out of ideas. I'd be interested in talking to anyone who has managed to get 4.2 working on Ubuntu 8.04.

Thanks.

Environment

-- BruceNourish - 10 Jul 2008

Answer

If you answer a question - or have a question you asked answered by someone - please remember to edit the page and set the status to answered. The status is in a drop-down list below the edit box.

The log entry =Can't locate TWiki/Plurals.pm in @INC = together with the provided value of @INC give some hints. There is no TWiki path in that list, though TWiki's scripts try hard to get /var/data/twiki/lib/ into @INC. And, as we see, TWiki/Render.pm has been found there.

One guess, and one suggestion for a workaround:

• Do you have other mod_perl scripts which fiddle with @INC? The observation of things going wierd and then "back to normal" often indicates that things are messed up by other scripts, and "repair" happens when mod_perl creates a fresh, unpolluted process.
• You could try to "harden" your TWiki by including all modules you need in your mod_perl startup script. The error log entry points to Plurals.pm, which is a module which is maybe not required on every TWiki request. If you use it in advance, the problem should disappear regardless of other processes which fiddle with @INC.

-- HaraldJoerg - 10 Jul 2008

I have no other mod_perl scripts in this Apache instance. TWiki is the only webapp running. I'll try the "use" thing.

-- BruceNourish - 14 Jul 2008

I had the same problem today but there was a simple solution: $binbase in the startup script was set wrong.

So, I think you should set it to my $binbase = '/var/data/twiki/bin'; # must be set by the user

-- LarsKanis - 18 Jul 2008

That fixed it... no idea how that typo got there. Thanks.

-- BruceNourish - 22 Jul 2008