create new tag
, view all tags


TWiki on Git Overview

The goal of this plugin is to enable a distributed operation of TWiki. Multiple TWiki installations may share some or all of their webs such that these webs are synchronized with each other.

Synchronization of webs is achieved by leveraging git as the repository for TWiki topics.

Git is a distributed version control system. This plugin uses Git as the Store module, replacing the standard RCS module. The store module controls the storage and revision control of topics and attachments. Git will then also distribute any topic and attachment under its control to all other synchronized TWiki installations.

Connection between each site is through ssh, a secure mode of accessing remote sites.

In the current version, two modes of operations are supported: Distributed Mode and Centralized Mode.

Distributed Mode


All sites are symmetrical in this mode. Each site has to fetch the content of all other and merge with its own content.

Centralized Mode


A central repository is used to store a copy of each site. Each site then fetches the content of the other sites from the central repository and merges with its own content. Each site pushes its content to the central repository.

The centralized mode is advantageous if the sites cannot connect directly to each other, e.g., if they maintain firewalls that do not allow ssh connections. In this mode, the individual sites only need to be able to connect to the central repository.


The installation of the GitPlugin primarily involves setup of the git repositories and the communication environment, which is described in this section.

The following terminology is used throughout these instructions.

  • <localserver> refers to the local TWiki server. A user account <localuser> is assumed to exist.
  • <remoteserver> refers to any remote TWiki server. A user account <remoteuser> is assumed to exist.
  • <centralserver> refers to the central repository server. A user account <centraluser> is assumed to exist.
  • twiki refers to the parent directory of the data and pub directories of each TWiki installation.
  • <localname> refers to an arbitrary name given to the local repository.
  • <remotename> refers to an arbitrary name given to a remote repository.
  • <centralname> refers to an arbitrary name given to the central repository.
  • <centralrepo> refers to the path to the central repository.
  • <remoterepo> refers to the path to the remote repository.
  • <localrepo> refers to the path to the local repository.

Note that the local and remote repositories must be the the parent directory of the data and pub directories of each TWiki installation. The central repository need not be a TWiki installation.

To begin, TWiki should be installed as normal. Make sure it works well before attempting to install GitPlugin. It is assumed that at each site, the user indicated above, such as <localuser> will execute TWiki.

These instructions assume a linux distribution for the TWiki server.

SSH Installation

ssh public and private key should be prepared for each server. For example, you can log in to <remoteserver> as <remoteuser> from <localuser>@<localserver> without a password. Keep the private key in a directory where <localuser> has the right to access it, but ensure that the permissions satisfy the requirements of ssh.

<localuser>@<localserver>$ssh-keygen -t rsa
~/.ssh/id_rsa is your private key, ~/.ssh/id_rsa.pub is your public key
<localuser>@<localserver>$ chmod 700 ~/.ssh
<localuser>@<localserver>$ chmod 600 ~/.ssh/id_rsa
Transfer <localpublickey> to <remoteserver>.
<remoteuser>@<remoteserver>$cat <localpublickey> >> ~/.ssh/authorized_keys
<remoteuser>@<remoteserver>$ chmod 700 ~/.ssh
<remoteuser>@<remoteserver>$ chmod 600 ~/.ssh/authorized_keys
Now you should be able to login in to <remoteserver> as <remoteuser> from <localuser>@<localserver>. Perform these steps for each pair of servers that need to synchronize with each other.

Git Installation

Install git on each server. For some linux distributions, git is not in the standard repository. For example, for CentOS, one has to install the EPEL packages first.

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm
To install Git from the source: Install git from binary distribution:
  • Fedora/CentOS: yum -y install git
  • Unbuntu/Debian: apt-get install git

After git has been installed, check the version using git --version. Ensure the version is at least 1.7

Distributed Mode

Perform these steps for all remote sites. <remotename> should be different for each remote server.

cd ./twiki
git init
git config user.email “<localusermail>”
git config user.name “<localusername>”
git remote add <remotename><remoteuser>@<remoteserver>:<remoterepo>

Centralized Mode

Perform the following steps on the central server. <repository> is an arbitrary directory where the central repository is located.

cd <repository>
git init --bare --shared

Perform the followikng steps in each (local and remote) TWiki installation. <localname> should be different for each remote server.

cd ./twiki
git init
git config user.email “<localusermail>”
git config user.name “<localusername>”
git remote add <centralname> <centraluser>@<centralserver>:<centralrepo>

Check installation

After these steps were done in all sites, you can test that these configurations are working by executing

cd ./twiki
git fetch <centralname>
for centralized installation; use <centralname> for distributed installation. If no warning or error surfaces, continue. Otherwise, check the configuration, as indicated by the warning or error. You can get more information how to install git on http://git-scm.com/. If you have difficulties authenticating through git and ssh, see below.

Repository initialization

Create a file .gitignore to ignore files that should not be synchronized.

git status
will show all untracked files. Now create the .gitignore file from these. A sample file has been provided in lib/TWiki/Plugins/GitPlugin/dot.ignore.

If you want to remove some file <file name> or directory <directory name> from the repository, append their path to the .gitignore file and then use the following to remove them from the git repository:

git rm --cache <file name>
git rm -rf --cache <directory name>
git commit -m<comments>

Configuration Editor

  • Open http://<localserver>/twiki/bin/configure in your browser.
  • Select Git as the StoreImpl mode in Store settings. If you can't select it, please view the notes below.
  • Some items should be filled in the “General git settings” section.
    • {Git}{mergeMode}: Select centralized or distributed correspoding to the intended mode of operation, matching the git installation.
    • {Git}{Path}: The path of the git executable. Usually, the path is /usr/bin/git.
    • {Git}{SSHKey}: Specify the path where the private key stored.
    • {Git}{remoteName}: Comma-separated list of remote sites names (referred to as <nickname> during setup above.
    • {Git}{localName}: Name of the local repository.
    • {Git}{backupReposname}: Name of the central repository.
    • {Git}{outSync}: Comma-separated list of webs which should not be synchronized and have not been added to the git repository (see the section "Selection of Webs not to be synchronized" below).
    • {Git}{root}: The path to the git repository; must be the parent directory of the data/ and pub/ directories.
    • {Git}{LogPath}: The path where the merge log file should be placed. (A good location may be ~apache/log or {Git}{root}/gitLog.
    • {Git}{MasterEmail}: if there are errors or warnings during a merge, GitPlugin will send emails to those listed.

Selection of Webs not to be synchronized

Any web listed in the {Git}{outSync} configuration parameter will not be synchronized. These webs use RCS as the repository as in standard TWiki. The list of webs excluded from synchronization should be consistent between sites.

When you want to create an out-of-sync web
  1. Add the name of the web to .gitignore
  2. Add the name of the web to {Git}{outSync}
  3. Create new web

When you want to change a synchronized web to an out-of-sync web

  1. Add the web name to .gitignore file and {Git}{outSync}
  2. git rm -rf --cache <directory> removes the web from the git repository. (remember to remove the web both in data/ and pub/)

When you want to change an out-of-sync web to be synchronized

  1. Remove the web name from .gitignore file and {Git}{outSync}
  2. git add .
  3. git commit -m"<comments>"


  1. Topics and attachments in out-of-sync webs can only be moved to the out-sync webs and topics.
  2. Topics and attachments in synchronized webs can only be moved to the synchronized webs and topics.
  3. When deleting a topic or attachment from a synchronized web, it will be permanently deleted and not moved to the Trash web.

Repository preparation

To guarantee the local initial repository is based on the current revision (to avoid conflicts which maybe not easy to be resolved) be sure to follow these steps.

If this is the first TWiki installation to push to the server

  1. Use git add . and git commit -m"<comments>" to initialize the local repository.
  2. If distributed mode is used, do nothing.
  3. If centralized mode is used, execute git push <centralname> master:<localname> to upload repository.

If this is a TWiki installation that is to be added to an existing setup

  1. The directories which will be synchronized should be moved out of the twiki directory.
  2. If distributed mode is used, for each remote site
    1. execute git fetch <remotename> to fetch the remote repository.
    2. execute git merge master/<remotename>
  3. If centralized mode is used
    1. execute git fetch <centralname> to fetch the remote repository
    2. execute git merge <centralname>/<remotename> for each remote site
  4. Move local webs to by synchronized into the twiki directory

Synchronization Setup

For web-based synchronization, create a topic with content %GITMERGE{""}%. Synchronization can be triggered either manually, by viewing this topic, or automatically, by executing a cron job.

When sychronization encounters conflicts, several buttons will be displayed with each file to allow remeding the situation:

Button Function File has been
Add Adds the conflicting file to the repository modified, added
Delete Deletes the file deleted, added
Retrieve Checks out the latest version of this file from the repository modfied, deleted
Ignore Adds the name of this file to .gitignore file added

To invoke synchronization via cron, edit the crontab file for the <localuser> and append

*/1 * * * * cd /path/to/twiki/bin; ./view /TWiki.AutoMerge
where /path/to/twiki/ is the path to the local TWiki installation, and AutoMerge is the topic with the GITMERGE tag. The above entry will perform synchronization every minute. Adjust the interval time as needed.

Converting webs between out-of-sync and sync.

GitPlugin supports the conversion of a traditional TWiki web to a synchronized web while preserving the revision history. Refer to GitPluginConvert for more details.

Synchronization of plugin work areas

If a plugin work area needs to be synchronized between sites, these can be selected in the configure script. For example, the repositories for the ActionTrackerPlugin or TagMePlugin benefit from being shared across sites.

Synchronized work areas will be stored in gitWorking/ and symbolic links will be created to this area from working/ to allow restoration of the original work area, if needed.

It is best to set this up ahead of time, but if the plugin has already been in use, the work area will be automatically moved. It may be necessary to remove the original working area from the git repository as git considers both the linked to file and the link different in the repository.


  1. Ensure that the TWiki user can access the ssh private key.
  2. Because the Plugin needs to modify the LocalSite.cfg, if this file has already been created, $TWiki::cfg{StoreImpl} needs to be modified in the file lib/LocalSite.cfg directly. Alternatively, delete this file and revisit configure.
  3. Conflicts caused by renaming or moving webs are not resolved. Consequentially, and ordinary user is not allowed to rename webs. Such must be performed by an administrator who should then update local repository with the latest content from the other sites and then perform the renaming.
  4. After merging, the revision history may be changed.
  5. The web configured as $TWiki::cfg{Git}{LocalTrashWebName} will serve as the Trash web for out-of-sync webs. The put back operation is not supported. This web should be added to .gitignore.


Warning, important When synchronizing very large sets of changes the process may take larger than is permitted by the webserver and the update may be interrupted. Therefore, it is better to rely on cron for updates, or, when a large sychronization is anticipated, to perform this from the command line.

Warning, important git is somewhat sensitive to transfer interruptions, which might require manual completion of the push/fetch cycle.

GitPlugin Global Settings

Configuration for GitPlugin are performed via configure. Ensure that all variables defined for the GitPlugin are given appropriate values, per their description.

GitPlugin Installation Instructions

  • This plugin works only with TWiki 4 and beyond.

There are two ways to install the Plugin.

  • Download the ZIP file from the Plugin web (see below)
  • Unzip GitPlugin.zip in your twiki installation directory. Content:
File: Description:
data/TWiki/GitPlugin.txt Plugin topic
data/TWiki/GitPluginDirectories.txt relevant topic
data/TWiki/GitPluginConvert.txt relevant topic, convert outSync web to sync web
lib/TWiki/Plugins/GitPlugin.pm Plugin Perl module
bin/gitop Invokes the plugin. Must be executable by the TWiki user
bin/viewgitconflictfile Invokes the plugin. Must be executable by the TWiki user
lib/TWiki/Store/Git.pm Realizes git as the store module; interfaces with Store.pm
lib/TWiki/Plugins/GitPlugin/git.pl Runs the git commands.
lib/TWiki/Plugins/GitPlugin/GitAction.pm Realizes the confirmation button in a topic with conflicts and a selection button that enables determination which revision to keep.
lib/TWiki/Plugins/GitPlugin/GitRun.pm Similar to Sandbox::sysCommand; formats a command and submits for execution.
lib/TWiki/Plugins/GitPlugin/SyncWorkingDir.pm Realizes merging the work_area of plugin which need to be synced
lib/TWiki/Plugins/GitPlugin/View.pm Similar to UI::View; displays different versions of conflicting attachments

The other is as follows.

  • Download the tgz file and installer file from the Plugin web (see below)
  • Put the the two files with your bin, lib, data and pub directories together.
  • run perl GitPlugin _installer to install GitPlugin automatically.

Some tips for using git

If you have trouble authenticating to the central repository, try the following:

  • Create a shell script as follows:
ssh -i /path/to/private/key "$@"
  • set the GIT_SSH environment variable to point to this script.

Plugin Info

Plugin Author:
Proposed and initially developed as class project for TWiki:Main/ThomasWeigert 's CS397 "Software Systems Development" at Missouri University of Science and Technology, Fall 2009 by Joshua Bohde, Afik Cohen, Peter Hoeft, Jeff Arneson, Jessica Egler, Kenneth Perry, Steven Wallace, and further enhanced by Yao Zhou and Brute Ma of UniqueSoft LLC.
License: GPL (GNU General Public License)
Plugin Version: 10711
Change History:
01 Nov 2011 Various performance improvements to avoid obtaining full git log, which is extremely slow in very large directories.
11 Jul 2011 Add ssh port option
30 May 2011 Display untracked files correctly.
Modify the format of error message when there are errors/warnings in AutoMerge
16 May 2011 Support to convert outSync web to sync web
Add a switch to control error email
29 Apr 2011 Fix the bug that When a file is found to conflict or be added which has a file name that includes a # sign, then this character is omitted in the displayed file name.
The comment of attachment which commit in AutoMerge will be retained
21 Apr 2011 Add $VERSION and $RELEASE information.
Enable file name can use non-word.
Enhancement that work area of ActionTrackerPlugin can be synced.
22 Mar 2011 Fix the bug that if creat a web based on a no attachment web, it will prompt error when attach
18 Mar 2011 For untracked files, Git Plugin needs to recognize other files than just the files under data/ and pub/
25 Feb 2011 Enhancement that while there is warnings/errors during sync, emails will be sent to who care it.
Sync log will be recorded.
When topic has merge conflict, email will be sent to the author who last modified it.
Enhancement that work area of TagMePlugin can be synced.
01 Feb 2011: Fix the bug that When choosing a particular item to merge ("Select" button) where the conflict is in a table, the replacement breaks the table by inserting empty lines before and after the merged text.
Enhancement AutoMerge page that have a checkbox that selects all checkable items when there are files detected not under revision control.
Enhancement conflict topic that allow users to select conflict attachement and meta data.( 7 types: parent, allowtopicview, denytopicview, allowtopicrename, denytopicrenmae, allowtopicchange, denytopicchange ) and add anchor of conflict content.
11 Jan 2011: fix the bug that pressing "add" button of files in pub which edited out of twiki is not correct.
22 Nov 2010: Added support for synchronizing edits performed outside of TWiki.
leverage RcsFile and RcsWrap/RcsLite to manage non-sychronized webs.
01 May 2010: Initial release
TWiki Dependency: $TWiki::Plugins::VERSION 1.1
CPAN Dependencies: none
Other Dependencies: Git 1.7
Perl Version: 5.005
Plugin Home: http://TWiki.org/cgi-bin/view/Plugins/GitPlugin
Feedback: http://TWiki.org/cgi-bin/view/Plugins/GitPluginDev
Appraisal: http://TWiki.org/cgi-bin/view/Plugins/GitPluginAppraisal

Related Topics: TWikiPlugins, DeveloperDocumentationCategory, AdminDocumentationCategory, TWikiPreferences, GitPluginDirectories

Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatmd5 GitPlugin.md5 r5 r4 r3 r2 r1 manage 0.1 K 2011-11-12 - 05:42 ThomasWeigert  
Compressed Zip archivetgz GitPlugin.tgz r5 r4 r3 r2 r1 manage 205.5 K 2011-11-12 - 05:42 ThomasWeigert  
Texttxt GitPlugin.txt r1 manage 21.5 K 2011-11-12 - 05:42 ThomasWeigert  
Compressed Zip archivezip GitPlugin.zip r5 r4 r3 r2 r1 manage 210.5 K 2011-11-12 - 05:42 ThomasWeigert  
Unknown file formatEXT GitPlugin_installer r3 r2 r1 manage 3.9 K 2011-11-12 - 05:42 ThomasWeigert  
Texttxt GitPlugin_installer.pl.txt r3 r2 r1 manage 3.9 K 2011-11-12 - 05:42 ThomasWeigert  
Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r8 - 2011-11-12 - ThomasWeigert
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2016 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.