NOTE: This is a SupplementalDocument topic which is not included with the official TWiki distribution. Please help maintain high quality documentation by fixing any errors or incomplete content. Put questions and suggestions concerning the documentation of this topic in the comments section below! Use the Support web for problems you are having using TWiki.

Auto-generated twiki.conf to be included from httpd.conf

twiki.conf created from your data

The text field below now contains a complete twiki.conf file to be included from httpd.conf. In many distributions the twiki.conf file is automatically included by placing it in the directory that contains the other included Apache config files. Typically in /etc/httpd/conf.d or /etc/apache2/conf.d

You can also place it anywhere else add the following line to the end of your main httpd.conf file: include "/path/to/twiki.conf"

By pressing the button below you select all the text in the textarea. Then you just need to copy the text to the pastebuffer and paste it into the twiki.conf file.

# Autogenerated httpd.conf file for TWiki.
# Generated at http://twiki.org/cgi-bin/view/TWiki/ApacheConfigGenerator
# For TWiki version 4.1.X

# We set an environment variable called blockAccess.
#
# Setting a BrowserMatchNoCase to ^$ is important. It prevents TWiki from
# including its own topics as URLs and also prevents other TWikis from
# doing the same. This is important to prevent the most obvious
# Denial of Service attacks.
#
# You can expand this by adding more BrowserMatchNoCase statements to
# block evil browser agents trying the impossible task of mirroring a twiki
#
# Example:
# BrowserMatchNoCase ^SiteSucker blockAccess
# BrowserMatchNoCase ^$ blockAccess

BrowserMatchNoCase ^Accoona blockAccess
BrowserMatchNoCase ^ActiveAgent blockAccess
BrowserMatchNoCase ^Attache blockAccess
BrowserMatchNoCase BecomeBot blockAccess
BrowserMatchNoCase ^bot blockAccess
BrowserMatchNoCase Charlotte/ blockAccess
BrowserMatchNoCase ^ConveraCrawler blockAccess
BrowserMatchNoCase ^CrownPeak-HttpAgent blockAccess
BrowserMatchNoCase ^EmailCollector blockAccess
BrowserMatchNoCase ^EmailSiphon blockAccess
BrowserMatchNoCase ^e-SocietyRobot blockAccess
BrowserMatchNoCase ^Exabot blockAccess
BrowserMatchNoCase ^FAST blockAccess
BrowserMatchNoCase ^FDM blockAccess
BrowserMatchNoCase ^GetRight/6.0a blockAccess
BrowserMatchNoCase ^GetWebPics blockAccess
BrowserMatchNoCase ^Gigabot blockAccess
BrowserMatchNoCase ^gonzo1 blockAccess
BrowserMatchNoCase ^Google\sSpider blockAccess
BrowserMatchNoCase ^ichiro blockAccess
BrowserMatchNoCase ^ie_crawler blockAccess
BrowserMatchNoCase ^iGetter blockAccess
BrowserMatchNoCase ^IRLbot blockAccess
BrowserMatchNoCase Jakarta blockAccess
BrowserMatchNoCase ^Java blockAccess
BrowserMatchNoCase ^KrakSpider blockAccess
BrowserMatchNoCase ^larbin blockAccess
BrowserMatchNoCase ^LeechGet blockAccess
BrowserMatchNoCase ^LinkWalker blockAccess
BrowserMatchNoCase ^Lsearch blockAccess
BrowserMatchNoCase ^Microsoft blockAccess
BrowserMatchNoCase ^MJ12bot blockAccess
BrowserMatchNoCase MSIECrawler blockAccess
BrowserMatchNoCase ^MSRBOT blockAccess
BrowserMatchNoCase ^noxtrumbot blockAccess
BrowserMatchNoCase ^NutchCVS blockAccess
BrowserMatchNoCase ^RealDownload blockAccess
BrowserMatchNoCase ^Rome blockAccess
BrowserMatchNoCase ^Roverbot blockAccess
BrowserMatchNoCase ^schibstedsokbot blockAccess
BrowserMatchNoCase ^Seekbot blockAccess
BrowserMatchNoCase ^SiteSnagger blockAccess
BrowserMatchNoCase ^SiteSucker blockAccess
BrowserMatchNoCase ^Snapbot blockAccess
BrowserMatchNoCase ^sogou blockAccess
BrowserMatchNoCase ^SpiderKU blockAccess
BrowserMatchNoCase ^SpiderMan blockAccess
BrowserMatchNoCase ^Squid blockAccess
BrowserMatchNoCase ^Teleport blockAccess
BrowserMatchNoCase ^User-Agent\: blockAccess
BrowserMatchNoCase VoilaBot blockAccess
BrowserMatchNoCase ^voyager blockAccess
BrowserMatchNoCase ^W3C blockAccess
BrowserMatchNoCase ^w3search blockAccess
BrowserMatchNoCase ^Web\sDownloader blockAccess
BrowserMatchNoCase ^WebCopier blockAccess
BrowserMatchNoCase ^WebDevil blockAccess
BrowserMatchNoCase ^WebSec blockAccess
BrowserMatchNoCase ^WebVac blockAccess
BrowserMatchNoCase ^Webwhacker blockAccess
BrowserMatchNoCase ^Webzip blockAccess
BrowserMatchNoCase ^Wells blockAccess
BrowserMatchNoCase ^WhoWhere blockAccess
BrowserMatchNoCase www\.netforex\.org blockAccess
BrowserMatchNoCase ^WX_mail blockAccess
BrowserMatchNoCase ^yacybot blockAccess
BrowserMatchNoCase ^ZIBB blockAccess
BrowserMatchNoCase ^$ blockAccess


# The ScriptAlias defines the bin directory as a directory where CGI
# scripts are allowed.
# The first parameter will be part of the URL to your installation e.g.
# http://my.co.uk/twiki/bin/view/...
# The second parameter must point to the physical path on your disc.
ScriptAlias /twiki/bin "/var/www/twiki/bin"

# The Alias defines a url that points to the root of the twiki installation.
# It is used to access files in the pub directory (attachments etc)
# It must come _after_ the ScriptAlias.
Alias /twiki "/var/www/twiki"

# This specifies the options on the TWiki scripts directory. The ExecCGI
# and SetHandler tell apache that it contains scripts. "Allow from all"
# lets any IP address access this URL.
<Directory "/var/www/twiki/bin">
    AllowOverride None
    Order Allow,Deny
    Allow from all
    Deny from env=blockAccess

    Options ExecCGI FollowSymLinks
    SetHandler cgi-script

    # Password file for TWiki users
    AuthUserFile /var/www/twiki/data/.htpasswd
    AuthName 'Enter your WikiName: (First name and last name, no space, no dots, capitalized, e.g. JohnSmith). Cancel to register if you do not have one.'
    AuthType Basic

    # File to return on access control error (e.g. wrong password)
    ErrorDocument 401 /twiki/bin/view/TWiki/TWikiRegistration

# Limit access to configure to specific IP addresses and or users.
# Make sure configure is not open to the general public.
# It exposes system details that can help attackers.
<FilesMatch "^(configure)$">
    SetHandler cgi-script
    Order Deny,Allow
    Allow from all
</FilesMatch>

</Directory>

# This sets the options on the pub directory, which contains attachments and
# other files like CSS stylesheets and icons. AllowOverride None stops a
# user installing a .htaccess file that overrides these options.
# Note that files in pub are *not* protected by TWiki Access Controls,
# so if you want to control access to files attached to topics you need to
# block access to the specific directories same way as the ApacheConfigGenerator
# blocks access to the pub directory of the Trash web
<Directory "/var/www/twiki/pub">
    Options None
    AllowOverride None
    Order Allow,Deny
    Allow from all
    Deny from env=blockAccess

    # Disable execusion of PHP scripts
    php_admin_flag engine off

    # This line will redefine the mime type for the most common types of scripts
    AddType text/plain .shtml .php .php3 .phtml .phtm .pl .py .cgi
#
#add an Expires header that is sufficiently in the future that the browser does not even ask if its uptodate
# reducing the load on the server significantly
#IF you can, you should enable this - it _will_ improve your twiki experience, even if you set it to under one day.
# you may need to enable expires_module in your main apache config
#LoadModule expires_module libexec/httpd/mod_expires.so
#AddModule mod_expires.c
#<ifmodule mod_expires.c>
#  <filesmatch "\.(jpg|gif|png|css|js)$">
#       ExpiresActive on
#       ExpiresDefault "access plus 11 days"
#   </filesmatch>
#</ifmodule>
#

</Directory>

# Security note: All other directories should be set so
# that they are *not* visible as URLs, so we set them as =deny from all=.
<Directory "/var/www/twiki/data">
    deny from all
</Directory>

<Directory "/var/www/twiki/templates">
    deny from all
</Directory>

<Directory "/var/www/twiki/lib">
    deny from all
</Directory>

<Directory "/var/www/twiki/locale">
    deny from all
</Directory>

<Directory "/var/www/twiki/tools">
    deny from all
</Directory>

<Directory "/var/www/twiki/pub/_work_areas">
    deny from all
</Directory>

mod_perl startup script

[error] Content-type: text/plain\n\nPerl error when reading LocalSite.cfg: \nPlease inform the site admin.\nBEGIN failed--

If you choose to run mod_perl you can create a file /var/www/twiki/tools/mod_perl_startup.pl and copy all the text from the text area below and paste into this file.

#!/usr/bin/perl -w
use strict;
use warnings;

$ENV{MOD_PERL} =~ /mod_perl/ or die "mod_perl_startup called, but mod_perl not used!";

use ModPerl::RegistryLoader;
use File::Spec;

my $binurlbase = '/twiki/bin'; # must be set by the user
my $binbase = '/var/www/twiki/bin'; # must be set by the user
my $registrymodule = 'ModPerl::RegistryPrefork'; # must be set by the user

my $rl = ModPerl::RegistryLoader->new(
   package => $registrymodule, # must be set by the user
);

chdir $binbase;

foreach my $binscript (<$binbase/*>) {
   my (undef,undef,$scriptname) = File::Spec->splitpath($binscript);
   $scriptname =~ m|^([^/]+)$|;
   my $script = $1;
   if ($script !~ m/configure|register|resetpasswd/) { # don't precompile uncommon commands especially configure which has a ton of unnecessary s***
            $binscript =~ m|^(.+)$|;
            my $bin = $1;
            open BINSCRIPT,'<',$binscript;
            if (<BINSCRIPT> =~ m|^\#\!/usr/bin/perl|) {
                    $rl->handler("$binurlbase/$script", $bin);
            }
            close BINSCRIPT;
   }
}

1;

Contributors: KennethLavrsen, JoshuaCharlesCampbell

Comments & Questions about this Supplemental Document Topic

Thank you very much Kenneth for sharing this with the TWikiCommunity!

-- PeterThoeny - 03 Jul 2006

I'm not sure if I need this or not. Frankly, it confuses me a bit. Here's my comments. 1) The first field asks for the full path... I think (but am not sure) it means the file path, not the URL path. This is a common issue with most TWiki docs. 2) Are all fields required to create a valid file? For example, do I have to lock this down to specific IP addresses? What if I leave it blank? 3) Do I need mod perl? I dunno... 4) This sentence is confusing: "The text field below now contains a complete twiki.conf file to be included from httpd.conf" Does that mean that I should take the text from the text box and paste it inside of my httpd.conf file?

For now I am just going to steer clear. Thx anyway!

-- GeoffreyMack - 06 Oct 2006

Geoffrey.

• On 1). Yes this is the file path. Not the URL path. I hoped the default example path /var/www/twiki would tell you that since this is a common path on a Linux machine for the Apache folders.
• On 2). You must give a path name to the twiki root folder. Like /var/www/twiki. You do not need to give a list of IP addresses or names. It is recommended to protect your configure. If you read the INSTALL.html in the twiki root you get the right context.
• On 3) No you do not need mod_perl. mod_perl needs to be installed on your Apache installation for you to be able to use it and you need a lot of RAM. mod_perl compiles all the perl code ONCE and keep it in RAM. This speeds up the execution of TWiki by at least factor 2. But mod_perl also keeps all variables in RAM so the programs must be written with great care. The TWiki core and the default plugins are pretty mod_perl safe now. But some other plugins do not work with mod_perl. I recommend to start a new TWiki installation without. And once it works you can try and activate it. Read the topic ModPerlUnix for more information. This ApacheConfigGenerator is a helper that generates a working apache config also for mod_perl but you need to know more about it to use it. As a beginner, run without.
• On 4) I do not recommend editing the default httpd.conf file. Normally distributions have an include statement in httpd.conf that includes all files that ends with .conf in a specific directory often called something like /etc/httpd/conf.d. You simply create a file called twiki.conf in this directory and paste in all the text from the text area. And then restart Apache. The topic TWikiOnRedHat has a very detailed step by step guide how to install TWiki including how to setup the config files for Apache using this topic.

-- KennethLavrsen - 11 Oct 2006

When I use the settings generated by your tool with my Apache/Mod_PERL installation, the WysiwygPlugin gets disabled. Is this a known bug?

• Please ask support questions in the Support web, thanks. -- PeterThoeny - 24 Jan 2007

-- AlokNarula - 23 Nov 2006

The script should quote the value for AuthUserFile. I as installing Twiki under Windows and it was using c:\program files\...\twiki\... and therefore the value for AuthUserFile had a space, which was not quoted by the generator in this page. This caused Apache2 to fail to start. If spaces are not supported, this script should warn the user. If, on the other hand, paes are supported, then the script should quote the value. Note how it quotes pretty much every other value. It took my a while to isolate the problem and figure out why apache2 would no longer start.

-- MarcioMarchini - 15 Jan 2007

The previous user was probably adding notes using the WYSIWYG editor, which totally messed up the form fields (not user error, is one of the many WYSIWYG editor bugs). I restored the previous topic version and merged the changes in.

-- PeterThoeny - 30 Jan 2007

I added new many more spiders to the blockAccess list; this is from twiki.org's Apache configuration.

-- PeterThoeny - 06 Feb 2007

Thks, I was starting to think to add mod_perl to my apache server, but knew nothing, reading this it'was 10min to get the twiki running twice fast than before.

-- MarcoSilva - 04 Mar 2007

Peter, it should be nice if we could rank these tips and howtos, like in vim.org, that has simple three buttons to rank its tips. Anyway, always in gratitude with you for twiki.

-- MarcoSilva - 04 Mar 2007

The tags have a vote function. Please add your vote to existing tags or add additional tags to topics you find useful. Topics with the highest tag votes show up first in the tag search page.

-- PeterThoeny - 05 Mar 2007

With all the "BrowserMatchNoCase" lines, the generated document no longer fits into a text box without scrolling. Would it be possible to save the plain text for the generated twiki.conf to another page, which we can click on and download?

-- KeithLofstrom - 21 Mar 2007

I think for security we should avoid exposing the TWiki root as an htdoc enabled directory. Sure, it can be secured as done by explicitely denying access to sub directories, but what if an admin creates a new sub directory and does not realize it is exposed?

So, this setup is more safe:

ScriptAlias /twiki/bin "/var/www/twiki/bin"
Alias /twiki/pub "/var/www/twiki/pub"

These are the only two directories that need to be exposed.

-- PeterThoeny - 27 Aug 2007

Only problem with this is that then the root html files are not accessible. But my guess is that not many have them there anyway.

-- KennethLavrsen - 03 Sep 2007

I think for security we better not have the twiki root exposed as htdoc root. An admin has file access during installation, thus can point a browser to the file URL to see INSTALL.html. Also, the http URL is not accessible before Apache config is done, so TWiki root exposure is a moot point.

-- PeterThoeny - 04 Sep 2007

added expires_module section - whatever time limit you set, will be how long the browser will wait until it tests to see if the static files have changed - thus reducing your round trip times dramatically, and reducing the number of connections needed to your server.

-- SvenDowideit - 27 Sep 2007

It would be nice to enhance this generator for an extranet setup where the whole twiki/bin and twki/pub directories are under authentication.

Example for twiki/bin:

AuthUserFile /path/to/twiki/data/.htpasswd
AuthName 'Enter your WikiName'
AuthType Basic
ErrorDocument 401 /error.html
Options -Indexes
<FilesMatch "configure.*">
  require user PeterThoeny
</FilesMatch>
<FilesMatch ".*">
    require valid-user
</FilesMatch>

Example for twiki/pub:

AuthUserFile /path/to/twiki/data/.htpasswd
AuthName 'Enter your WikiName'
AuthType Basic
ErrorDocument 401 /error.html
<FilesMatch ".*">
  require valid-user
</FilesMatch>

-- PeterThoeny - 02 Jan 2008

"AddType text/plain .shtml .php .php3 .phtml .phtm .pl .py .cgi"

Be aware that this directive introduces an issue if for example you have the associated html page for the wysiwyg editor. as the text/html document will have text/plain mime type. Maybe wise to limit the coverage of the directory or location.

-- AlexandreDulaunoy - 11 Feb 2008

I really don't think that the W3 rules should be there; they prevent using the W3Cs? validator. PatternSkin goes to a lot of trouble to validate, and people should be proud of that.

-- ChrisFLewis - 13 Feb 2008

I can't start apache on my Mac when the line php_admin_flag engine off is in the config file.

-- ArthurClemens - 19 Feb 2008

If you're using mod_perl, the documentation should read that you must have that Perl script in place, otherwise Apache won't start properly.

-- JimRizzo - 22 May 2008

It's really important to use the right handlers - in reality, to use RegistryPrefork. I spent a couple of maddening days chasing my tail trying to get mod_ssl, mod_rewrite, mod_perl and twiki's template login system playing happily together. I kept getting periodic episodes in which you would have to pass through the login template multiple times.

Once I've had some sleep and gotten through finals, I'll (at the least) get the fully documented setup out there for folks to peruse. Don't know that I'll deal with adding it to the generator though...

-- HilaryHolz - 04 Jun 2008

Thank you Hilary for the feedback. Looking forward seeing details on how to improve docs.

-- PeterThoeny - 04 Jun 2008

Small correction: the regexp for blocking direct access to attachments that end with .htm or .html was missing an [lL]

-- MarkusUeberall - 15 Jul 2008

The generated mod_perl startup script contained the twiki base path. my $binbase = '/var/www/html/twiki/bin'; # must be set by the user This path was apparently not substituted to the path entered in the form. This causes weird errors when using mod_perl. See ModPerlWierdness.

So I corrected this topic accordingly. Hope that was right.

-- LarsKanis - 18 Jul 2008

Added a warning message for mod_perl: You should first configure TWiki before trying to enable it, otherwise Apache won't start (no LocalSite? .cfg)

-- OlivierRaginel - 15 Aug 2008