create new tag
, view all tags

Ensure Data Integrity - when changing store formats

This topic explains an important item in the TWikiMission: Protect user investment

To our customers the real value of TWiki lies in the content in the many 10000s of pages they create. Being a product that targets the enterprise market, the community is committed to protect this huge investment by ensuring that our customers can upgrade to new versions of TWiki without losing, or otherwise compromising, their content.

The general rule is that all non-compatible updates to the store format should happen seamlessly, on the fly when you edit and save a page.

However there are rare exceptions where a change to the store format is required to be implemented on all pages (or attachments) in one go. For example, such a change may be required in order to implement performance improvements.

These guidelines were decided at the TWikiCommunitySummitRome2007, and apply to all stored data (at this time, page text, meta-data and attachments)

Upgrade non-compatible changes

  1. You should not make non-compatible changes unless really, really needed.
  2. TWiki should upgrade on the fly from all prior store format versions whenever possible.
  3. If an improvement can only be achieved when all content is updated at once, a configure - based upgrade step must be provided (do not require command line access)
  4. Downgrade must always be possible, so you can revert to previous a TWiki version.
    • The downgrade may require a configure step, but must not require command-line access.
  5. Upgrade and downgrade steps must require a target TWiki version number, and must convert from any supported version to that target version.

-- Contributors: TWikiCommunitySummitRome2007, KennethLavrsen, SvenDowideit, CrawfordCurrie


  • 3&4 together mean that IF there is an upgrade script, there must be a downgrade script.
  • 4 actually also means that for any topic format change, there must be a downgrade script.

Which really plays further into the idea that there needs to be a 2 way transform between all formatting changes.

I would suggest that format upgrades must be written, such that all conversions can be called adhoc (on the fly) or batched from configure. Thinking about how this works in the context of the DatabaseStore code i was working on, suggests to me that we should have a different TWiki::Store implementation for each format, and conversions could be done by transfering the topic / topic sets.

-- SvenDowideit - 21 Aug 2007

I reworded to reduce the number of points, clarify a couple, and reflect Sven's points. I also went further down the "no scripts from the command-line" road. configure should support bidiriectional transformation between sequentially numbered store generations.

-- CrawfordCurrie - 21 Aug 2007

Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r3 - 2007-08-21 - CrawfordCurrie
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.