TWiki Installation based on Subversion

Once you have followed the instructions in SubversionReadme you can run a TWiki from this checkout area simply by pointing Apache at it in the normal way and running configure.

Note these instructions describe how to install TWiki from the subversion trunk, which we started using in March 2008. The instructions are slightly different for checking out release branches made before this date. For instructions appropriate to those older branches, see version 18 of this topic.

By default the checked-out TWiki core won't include any of the extensions (plugins or contribs). To enable extensions in a checkout area, you can either install them the normal way, or use the pseudo-install.pl script to install them. On Unix/Linux this script generates soft-links from the core tree to the extensions, so you can work on your code in situ and see the impact of the changes on your live twiki without needing to do an install step. Windows doesn't support soft links, so the script can also be run in -copy mode (the default on Windows), but in this case you will have to re-run it each time you change your extension. Remember you have to enable any plugins you want to test in configure.

Use pseudo-install.pl default to enable the default contribs and plugins (e.g. TwistyContrib which is relied on by PatternSkin) and pseudo-install.pl all to install the whole works.

See the header comment of the pseudo-install.pl script (top level of checkout) for options and more information. Note that pseudo-install.pl only works with extensions that have a MANIFEST file, as required by the BuildContrib.

Script examples below are for bash shell.

See HowToStartExtensionDevelopmentInSubversion if you work on extensions.

Example of running pseudo-install

The typical situation is that you want to run a pseudo-installed TWiki checked out from the trunk. And if you develop plugins, you want to be able to activate your plugin in this installation. This is the entire sequence for checking out the trunk from subversion and doing the pseudo-install. We assume that you want to run your Subversion based install in /var/www/twiki

The following commands check out an absolutely minimal TWiki (the core + default user mapping only). This is the smallest checkout that will run. The steps are:

1. create the root directory called twiki
2. checkout the core from the subversion trunk
3. checkout TWikiUserMappingContrib
4. pseudo-install TWikiUserMappingContrib
5. change the ownership so the entire tree is owned by the user running the Apache. In this example the user name is "apache".
6. point Apache at the checkout

Change the commands to fit your actual file locations and Apache user. Some commands may have to be run as root.

cd /var/www
mkdir twiki
cd twiki
svn co http://svn.twiki.org/svn/twiki/trunk/core core
svn co http://svn.twiki.org/svn/twiki/trunk/TWikiUserMappingContrib TWikiUserMappingContrib
cd core
perl pseudo-install.pl TWikiUserMappingContrib
cd ..
# If necessary, change ownership of all files to the webserver user.
# In this case that is 'apache:apache', though it may also be 'www-data:www-data'
# e.g. on Debian and Ubuntu systems, or something else entirely. Check first.
chown -R apache:apache .
# Now configure Apache to use the TWiki in /var/www/twiki/core

Note: If the apache error log has lots of Symbolic link not allowed or link target not accessible type messages, then you probably need to add +FollowSymLinks to the Options for the /var/www/twikidev/core/pub directory in your apache configuration.

This is the most basic TWiki install possible. Normally you will want to checkout and pseudo-install the default skins and plugins (PatternSkin, SpreadSheetPlugin, TablePlugin, CommentPlugin etc.

cd /var/www/twiki
for module in `grep '!include' core/lib/MANIFEST | sed -e 's/^.*\///g'`
do
    svn co http://svn.twiki.org/svn/twiki/trunk/$module
done

(This is bash shell script. The script-challenged can get the list of modules to check out by inspecting the !include lines in core/lib/MANIFEST)

Once you have checked these modules out you can install them all in one go using the special default parameter to pseudo-install.pl

cd /var/www/twiki/core
perl pseudo-install.pl default

Now and then you will want to keep your installation in sync with the subversion server. The pseudo-install script is not intelligent enough to cope with changes to files (MANIFESTs), so this is the idiot proof way to update. It first removes all the links, then does the svn update. And finally does a new pseudo-install.

cd /var/www/twiki
perl pseudo-install.pl -uninstall all
svn update
perl pseudo-install.pl all
cd ..
chown -R apache:apache .

Normally just doing the svn update will be enough, unless someone has removed files (and even then you can usually ignore it).

Tips, hints, and useful commands

Installing everything

The example commands above describe how to install a minimalist TWiki, which requires you to check out and install each additional extension by hand. You can also take a maximalist approach and check out the entire trunk. This may be appropriate if you are going to be working on a lot of different extensions, or simply want to have everything to hand and don't care how long (or how much space) it takes. This is how http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/WebHome work (you can see the scripts that maintain that site in core/tools/develop). Here's how:

cd /var/www
mkdir twiki
cd twiki
# checkout *everything*
svn co http://svn.twiki.org/svn/twiki/trunk .
cd core
perl pseudo-install.pl default
# or use 'all' in place of 'default' to install all plugins and extensions (not recommended!)
cd ..
chown -R apache:apache .
# Now configure Apache to use the TWiki in /var/www/twiki/core

Note: If the apache error log has lots of Symbolic link not allowed or link target not accessible type messages, then you probably need to add +FollowSymLinks to the Options for the /var/www/twiki/core/pub directory in your apache configuration.

Now you can install additional extensions using perl pseudo-install.pl extension_name and uninstall with perl pseudo-install.pl -uninstall extension_name

Delete all soft links in a subversion-based install

This is handy if you have changed a lot of MANIFESTS or have manually soft-linked any files, and want to reset the checkout to a known state. Assume your trunk checkout is at /var/www/twiki

find /var/www/twiki/core -type l -exec rm \{\} \;

Create a new extension

You can quickly and easily create a new extension using the create_new_extension.pl script that is installed in core when you pseudo-install the BuildContrib.

Set up the unit test framework

If you are developing new code you will want to set up the development and test environment. For this you will need to pseudo-install BuildContrib, UnitTestContrib and TestFixturePlugin.

pseudo-install.pl BuildContrib UnitTestContrib TestFixturePlugin

Then:

cd test/unit
export TWIKI_LIBS=/var/www/twiki/core/lib
perl ../bin/TestRunner.pl TWikiSuite

(or equivalent on Windows)

Developer Experiences (use cases)

I work "as if" I were going to download a tarball. In effect I do.

• I have a machine with a SVN checkout area ~twikidev/twiki/branches/DEVELOP. Actually I have a few 'custom' and 'scratch' areas along side where I can make custom builds, but that's beside the point.
• I use the normal svn up to update it.
• I use the tools/build.pl to build a tarball. Or 'tarzball' if you prefer that nomeclature.
• The tarzball gets moved (scp) to the target machine and un-tarz'ed
• The target machine is configured with Apache
• In my case there is ~apache/twiki/datar1, ~apache/twiki/datar2, ~apache/twiki/datar3 and so on.
• I also have a version of Dakar from July, before Icons, sessions and many orher thigns were migrated to the core.
  • This allows me to do side-by-side comparisons and tests
• The machines support NFS, so I can easly run diff between not only version but also the SVN reference.
  • I also have tools that can run directory level diff, three-way diff and selectively/interactively merge two differnt revision to produce a third.

I realise this is way beyond what is needed to "just" produce a release, but it does ensure a number of things.

• Since the check-out area never 'runs' I don't get problems with TOPICINFO when I do update a topic and submit it.
• I can see more comprehensively what changes are going on and compare their impact as well as the lexical change.
• I can run different revisions side by side. I could use this for benchmarking, but I don't want to get into that argumentative area.
• I can work on different approaches.

The SVN checkout is a reference for me. It never gets 'scribbled' on so I never have to worry about subversion conflicts. I don't get G or C notifications.

My motivation at the moment is:

1. Testing. Not all the bugs go straight into the Bugs DB. I usually check with the developers first in case there is a misinterpretation.
2. Limited release for the few sites that are willing to try the betas

No doubt this will change, but I suspect that my model will now. The individual test locations will become client baseline locations.

-- AntonAylward - 08 Nov 2005

as a developer counter point to Anton, i use an SVK based local branch that I then rsync to my web host. this way http://www.home.org.au is backed up, and in a versioning system, allowig me to tweak the code in a managed way, while still being about to svn update to the latest DEVELOP code. I don't recommend this for everyone, but if I were running from the released version, I would still be rysncing from my svk repository, so that I get the trivial change management and backup facilities. But then I am quite fond of having my /etc dir (and my $HOME, when i get back to Sydney) in my svk repos...

-- SvenDowideit - 08 Nov 2005

See also: SoYouWantToBeATWikiDeveloper, DeveloperResponsibilities, TWikiRelease, TWikiProductionRelease

-- Contributors: CrawfordCurrie, KennethLavrsen, MartinCleaver, PeterThoeny, WillNorris, AntonAylward

Discussion

Which pseudo-install do you use? There are two of them, "pseudo-install.pl" and "pseudo-install-twiki.pl".

Also, the above directions may not be appropriate for the (new?) directory structure in the SVN. After checkout, my pseudo-install.pl is in the {checkout dir}/core not in {checkout dir} as implied above.

-- PeterStephens - 13 May 2008

pseudo-install.pl

These instructions work well on the "release" branch, but not in the "MAIN" branch...

-- RafaelAlvarez - 13 May 2008

Thanks for the heads-up, I updated the instructions. Sorry, I don't know what pseudo-install-twiki.pl is, and have never used it myself (which kinda suggests it is unmaintained).

-- CrawfordCurrie - 14 May 2008

Thanks Crawford. After using the apache config generator I also added "+FollowSymLinks" to the "Options" for the "/var/www/twikidev/core/pub" Directory section to allow the site to access the pseudo-link'd files. It was obvious that this would not work without it as the error_log had lots of, "Symbolic link not allowed or link target not accessible: /var/www/twikidev/core/pub/TWiki/PatternSkin, referer: http://www.planetzetox.com/twikidev/bin/view" messages.

-- PeterStephens - 14 May 2008

Just a little observation: I believe that in the Example of running pseudo-install, where it says:

perl pseudo-install.pl TWikiUserMappingContrib
cd ..
chown -R apache:apache twiki

it should say cd ../.. as you are in the core directory and have to go above the twiki directory so that the chown works. Maybe a hint for newbies, that apache should be substituted by e.g. www-data, depending on each configuration, would be helpful.

-- SebastianKlus - 16 Jul 2008

Integrated. Next time, Sebastian, just make the changes yourself.

-- CrawfordCurrie - 17 Jul 2008

Thanks, I will do so. The thing is that sometimes I do not want to mess around in a text someone wrote who has about 100 times more experience in all that stuff than me

-- SebastianKlus - 18 Jul 2008

Repeated same note in "Example of running pseudo-install" heading about "add +FollowSymLinks to the Options for the /var/www/twikidev/core/pub directory in your apache configuration" to the "Installing everything" heading

-- GeorgeTrubisky - 2010-12-21