By default TWiki searches its database by forking the standard grep program in a sub-process to search through the text files in the database cache. This is usually pretty fast, but unfortunately in some environments (specifically with mod_perl) forking a sub-process is very expensive, and another approach is needed.
NativeSearch is a c-code module that can be plugged into TWiki to replace the default searching algorithm. It publishes an XS interface that allows it to be called from Perl, without having to fork a sub-process. It has comparable performance to the standard forking search in most environments, but is often significantly faster than forked search when TWiki is run under a CGI accelerator. The effect is most pronounced with mod_perl, but is also there for SpeedyCGI.
You require shell access to the server to install this module. Refer to TWiki:Codev.NativeSearch for a list of systems where the module has been successfully compiled and installed (and please update it if you find a new one).
If you are installing on a TWiki version before 4.1.2, please refer to TWiki:Codev.NativeSearch for instructions. Note that an old version of the package was pre-installed with 4.1.2; that pre-installation will be overwritten by this package.
Before you start, there are some prerequisites.
you must have ExtUtils::MakeMaker installed (available from CPAN)
you must have a C compiler (gcc has been tested). Most Unix/Linux systems will have this already. On Win32 MinGW is recommended.
you will need make (a standard development tool). Unix/Linux users will probably have this already; others may have to search. You are recommended to use nmake on Win32.
finally you will need the pcre library (required for perl compatible regular expressions). This is a standard Linux library and will be available in an RPM or Debian package or whatever for your system. It is also available from GnuWin32 for Windows. You need both the runtime library (binary) and also the developer header files, which are usually shipped in a seperate package.
You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server where TWiki is running.
Like many other TWiki extensions, this module is shipped with a fully
automatic installer script written using the BuildContrib.
If you have TWiki 4.2 or later, you can install from the configure interface (Go to Plugins->Find More Extensions)
If you have any problems, then you can still install manually from the command-line:
Download one of the .zip or .tgz archives
Unpack the archive in the root directory of your TWiki installation.
Run the installer script ( perl <module>_installer )
Run configure and enable the module, if it is a plugin.
Repeat for any missing dependencies.
If you are still having problems, then instead of running the installer script:
Make sure that the file permissions allow the webserver user to access all files.
Check in any installed files that have existing ,v files in your existing install (take care not to lock the files when you check in)
Manually edit LocalSite.cfg to set any configuration variables.
Building the c-code
cd to the root directory in your TWiki installation. Now, assuming you have administrator (root) access to the server:
$ cd tools/native_search
$ perl Makefile.PL
$ make install
$ perl test.pl -i -l NativeTWikiSearch test.pl Makefile.PL NativeTWikiSearch.xs
(on Win32 you should use nmake)
Make sure that perl test.pl returns something sane, and NOT a
(test.pl is simply a stub interface to the native search, so it behaves like grep. The test above simply greps for the string "NativeSearch" in the files Makefile.PL and NativeTWikiSearch.xs)
If you do not have administrator (root) access to the server, you can still
install. Follow the steps above, but use perl Makefile.PL INSTALLBASE=/your/dir to install to a different directory. Exactly where you install depends on your system configuration.
Now, in configure, under "Store Settings", you should be able to select the search algorithm TWiki::Store::SearchAlgorithms::Native (click the button first if you have TWiki 4.2 or later).
Run a TWiki search, and watch the web server error log for any problems.
Because performance can differ widely between two apparently similar webserver configurations, you are recommended to do your own benchmarking and select the search algorithm that performs best for you.
Many thanks to the following sponsors for supporting this work: