import1Add my vote for this tag transformation1Add my vote for this tag create new tag
, view all tags

Generic Wiki to TWiki Topic Migrator

This script migrates a Generic Wiki's text files to a new TWiki site. The script is for people who switch over from some other Wiki to TWiki and can't find a conversion specific to their situation. It is intended to be used once to migrate all existing topics into an empty TWiki web. Thus far, it has only been tested in migrating a Wiki using Ward Cunningham's original Wiki code. It is written, however, to be customizable to other specific situations. Knowledge of Perl would be required to customize it.

Syntax rules can be converted by modifying the regex array. The default conversions are:

use constant FILTER_REGEXES => [
   [qr|^\*|   =>  '   *' ],
   [qr|\t|   =>  '   '  ],
   [qr|'''|  =>  '*'    ],
   [qr|''|   =>  '_'    ],
   [qr|1\. | =>  '1 '   ],
   [qr|^\s+(.)\$| =>  '<verbatim>\n$1</verbatim>'],      


  • The conversion only migrates the text of the topics. All previous versions and all user information is lost. The pages will appear to have been added by the user specified in the script (if one has been specified).
  • Essentially, the script is just automating something you might do manually, which is opening a topic page in the original wiki, creating a topic of the same name in TWiki, and then pasting the text from the old page to the new page. It does this for every topic that can be traced up to your top level Wiki topic (even if through a topics listing page).


Modify the script's constants and execute it. Watch the output for status messages.

Attention: The destination wiki is assumed to be empty or unimportant. Existing topics will be overwritten if they have the same names as topics in the migrated web!

Addon Installation Instructions

  • Download the ZIP file from the Plugin web (see below)
  • Unzip generic2twiki.zip and copy script to twiki/tools or any location you like. ZIP Content:
    File: Description:
    generic2twiki migration script
  • Make the generic2twiki script executable (Windows users may wish to place a ".pl" or ".plx" at the end of the script's filename to make it immediately linked to their perl executable.)
  • The script can be run from any computer that has web access to the original Wiki and the TWiki site. It's essentially a Wiki crawler.
  • Change the constants in the script to provide the values specific to your migration. For example:

# Set the following values if your Wikis require authentication
# in order to edit topics.
# Leave the WIKI_PORT and WIKI_REALM empty unless you use 
#  Windows-based NT authentication on your wiki.  In that case,
#  for WIKI_PORT, you must use domainname:port, such as www.mywiki.com:80
#  and leave WIKI_REALM blank (don't forget to put your NT domain name in 
#  front of your user id like this:  DOMAIN\UserID)
use constant DEST_WIKI_UID => '';
use constant DEST_WIKI_PWD => '';
use constant DEST_WIKI_PORT => '';
use constant DEST_WIKI_REALM => '';
use constant SRC_WIKI_UID => '';
use constant SRC_WIKI_PWD => '';
use constant SRC_WIKI_PORT => '';
use constant SRC_WIKI_REALM => '';

# These two urls should point to the wiki server you're migrating to.  Be sure to 
# change mynewwiki.com and WikiName to the correct values for your environment.
# The default Wiki name in TWiki is "Main"
use constant DEST_SAVE_URL
   => 'http://mynewwiki.com/wiki/bin/save/WikiName/$TOPIC$';
use constant TOPIC_SRC_URL
   => 'http://mynewwiki.com/wiki/bin/view/WikiName/$TOPIC$';

# The server name in this url should be changed to match the URL you would use 
#  to access the topics on the wiki you're migrating away from.  In this example,
#  the wiki script is named wiki.pl and is in the cgi-bin directory.  This is 
#  probably not the same as your configuration.  The text after the ? in the url
#  should probably not have to change.
use constant SRC_EDIT_URL 
   => 'http://myoldwiki.com/cgi-bin/wiki.pl?action=edit&id=$TOPIC$';

# change SRC_HOME_TOPIC to match the starting topic for your old wiki.  On 
# the original wiki.pl script, this is usually "HomePage"
use constant SRC_HOME_TOPIC => 'HomePage';

# Change this topic if you wish for your migrated wiki's home page to be placed somewhere
# other than the beginning page of your new wiki (WebHome is TWiki's default start topic)
use constant DEST_HOME_TOPIC => 'WebHome';

use constant TEXT_EXTRACT_REGEX => qr|wrap="virtual">(.+)</textarea>|is;

# Replace "WikiName" with the name of your destination wiki (as you did above in
use constant TOPIC_EXTRACT_REGEX => qr|<a href="/wiki/bin/edit/WikiName/(\w+)\?topicparent=([a-zA-Z0-9.]+)">\?</a>|i;

Add-On Info

  • Set SHORTDESCRIPTION = Script to convert any standard Wiki content to TWiki format using HTTP

Addon Author: JamesTillman
Addon Version: 21 Feb 2003
Addon License: GPL
CPAN Dependencies: LWP
Other Dependencies: None
Perl Version: 5
Addon Home: http://TWiki.org/cgi-bin/view/Plugins/GenericWikiToTWikiAddOn
Feedback: http://TWiki.org/cgi-bin/view/Plugins/GenericWikiToTWikiAddOnDev

Related Topics: TextFormattingRules

-- JamesTillman - 21 Feb 2003

Argh, I hate Perl (this little gem took me 2 hours to figure out). Here's a replacement for filter_text so that replacement rules can include $1, etc. Uses String::Interpolate, which you'll have to get from CPAN.

sub filter_text {
        my($text_ref) = @_;
        my $filter_regexes = FILTER_REGEXES;
        foreach my $substitution (@$filter_regexes) {
                my ($regex, $replacement) = @$substitution;
                my ($rrr) = new String::Interpolate $replacement;
                $$text_ref =~ s/$regex/$rrr/mge;

-- BryanLarsen - 05 Sep 2003

In order to migrate from MoinMoin, I had to modify the filter_text sub... since I had not seen the above tweak :

sub filter_text {                                                                                                              
        my($text_ref) = @_;                                                                                                    
        my $filter_regexes = FILTER_REGEXES;                                                                                   
        foreach my $substitution (@$filter_regexes) {                                                                          
                my ($regex, $replacement) = @$substitution;                                                                    
                $$text_ref =~ s/$regex/$replacement/mg;                                                                        
        $$text_ref =~ s/^\= (.+) \=$/---++ $1/mg;                                                                              
        $$text_ref =~ s/^\=\= (.+) \=\=$/---+++ $1/mg;                                                                         
        $$text_ref =~ s/^\=\=\= (.+) \=\=\=$/---++++ $1/mg;                                                                    

For TWiki 20040901, I also had to change the following constants :

use constant TEXT_EXTRACT_REGEX => qr|style="width:100%">(.+)</textarea>|is;
use constant TOPIC_EXTRACT_REGEX => qr|<a href="/twiki/bin/edit/Main/(\w+)\?topicparent=([a-zA-Z0-9.]+)"><sup>\?</sup></a>|i;

-- BenVoui - 07 Oct 2004

Topic attachments
I Attachment History Action Size Date Who Comment
Compressed Zip archivezip generic2twiki.zip r1 manage 2.6 K 2003-03-17 - 15:53 JamesTillman Zip file containing the generic2twiki.pl script
Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r8 - 2006-10-07 - PeterThoeny
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.