create new tag
, view all tags

Replication Using Rsync

One of a set of ReplicationTechnologiesForTWiki


Rsync was used to great effect at Inktomi many years back for a significant time period (>12 months of active use) to replicate data between sites, and worked very well. One limitation of the approach taken was that you could only have one master site for each web , but that was resolved by simply making the edit and attach links point at the appropriate master site for that web. Rsync is designed to be lightweight and replication between 5 systems on 2 continents worked very well and transparently. (To the extent that the preferred mechanism of shipping content between sites for certain tasks became attaching the content to a TWiki page, and letting the "TWiki Net" as people referred to it perform the replication. Furthermore this was despite the fact that Inktomi also sold content distribution & synchronisation software.)

-- MichaelSparks - 14 Jul 2005

-- MartinCleaver - 19 Jul 2005

Our TWiki installation uses a slightly different approach. We are using the September 2004 release, which had some left-over mirror code within it. We made a few modifications to this code as needed, only amounted to a few lines in three or four files, and changed the pattern skin templates. TWiki.cfg on each server now defines its location, and one web is set to be "master". Each mirror web has all editing commands (Edit, Attach, etc.) struck-out and unavailable. We added a "Select your mirror" pull-down to the top bar so that people could view their local web, or access the master for edits. A 15 minute rsync scheme where the slaves pull from the master keeps everything in sync. We're managing 10G+ of data, three mirrors and a master site using this scheme. Contact me if you would like any more detail.

-- IanHarrison - 03 Jan 2006

yes please Ian smile while we may not end up using your changes, it is always interesting to see what changes people make, and how code (and stubs) are used. I'm not sure if the mirror code has been tested in DakarRelease, as very few people know about it, and fewer use it...

could you post a diff, and any other interesting tid-bits?

-- SvenDowideit - 03 Jan 2006

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r4 - 2006-01-03 - SvenDowideit
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.