Tags:
compatibility1Add my vote for this tag dakar1Add my vote for this tag upgrade1Add my vote for this tag create new tag
, view all tags

How to Handle Cairo/Dakar Plugin Differences

Introduction

There are differences in Plugins between TWiki Release 01-Sep-2004 (code name CairoRelease) and TWikiRelease04x00x00 (code name DakarRelease). While existing Plugins that only use the published APIs should still work with Dakar, you may find that some plugins don't work any more.

This topic describes how plugin authors can port existing Plugins to Dakar while still maintaining Cairo compatibility.

In Dakar, the Plugins API has also been extended in a number of different ways, exposing some previously unpublished core functionality and improving the efficiency of several handlers. Plugin authors may wish to revisit their plugins to leverage these improvements.

Constraints we are facing

  1. TWiki extensions are one of the biggest assets of TWiki, they must be supported well
  2. File-format change in Dakar making it difficult to maintain Plugins Cairo compatible (tab vs space issue)
  3. API changes in Dakar that require code changes to maintain Plugins Cairo compatible
    • Example: Handler postRenderingHandler is deprecated in favor of a new endRenderingHandler (it is possible to change the code as described in the compatibility sections below, but the code needs to be touched, and there is an issue once the handler is retired)
    • Example: Deprecated API functions getScriptUrlPath, getPublicWebList, formatGmTime, getDataDir, getPubDir, checkDependencies still work in Dakar, but Plugins need to be updated before those functions get retired
  4. Organizations typically stick with an installation for a long time, e.g. do not upgrade to the latest TWiki after a new release is out. These arganizations still would like to install the latest versions of EssentialPluginsAtDakarRelease and less essential Plugins
  5. Not many Plugin authors are actively maintaining their Plugins in SVN
  6. Some Plugins cannot be made available in Cairo, due to Dakar improvements

Requirements

  1. Cairo compatible versions of existing Plugins should be made available for the next 12 months
    • "Please upgrade TWiki" is not always a feasible answer due to other constraints (e.g. "I need the latest ActionTrackerPlugin, but can't upgrade to TWiki 4.0 because we use WebDAV Plugin / because it breaks our existing wiki apps")
  2. Downloading and installing Plugins should be intuitive and easy, both for Cairo and Dakar installations
  3. Keep the repository as easy to use as possible
    • Avoid "to download XyzPlugin for Cairo go here, for Dakar go there" (which was OK for beta testing in Plugins.TWiki)
  4. Plugin download & installation needs to be possible on isolated intranets
    • Internet connection to TWiki.org cannot be assumed (for example A&D companies)

The goal is to make it as easy as possible for the users by keeping Plugins compatible, so that:

  • There is only one download
  • They can be deployed on Cairo and Dakar installations

Choices for Maintaining Plugins

If you called core functions in Cairo, the chances are you will have to do some work to map to Dakar. Much if that work may already have been done for you in the API. Read the new TWikiFuncDotPm API documentation very carefully. Both Func and the handlers have changed. Also read the TWikiReleaseNotes04x00x00.

Now, if your plugin needs work, you have three choices:

  1. Do nothing, and let your Cairo release stand or die
  2. Port to Dakar, and forget about Cairo
  3. Maintain compatibility code interleaved in the Dakar implementation.

1 is a bad idea, unless you have finally realised that you hate everyone, and don't want them to use your plugin any more wink . 2 is probably the easiest route, but we strongly suggest to consider 3, which isn't as hard as it sounds for most simple plugins.

Here's a handy matrix that helps you trade off work, efficiency, and total addressable market (TAM, the number of people who can quickly and easily pick up and install your plugin) and determine which route you should follow:

  minimum work maximum efficency max TAM
minimum work 1 2 2
maximum efficiency 2 2 3
max TAM 2 3 3

Port to Dakar, and forget about Cairo (NOT recommended)

OK, so how do you port a plugin to Dakar? Well, simply examine your code, and for each Func call and each handler check the status in the Dakar API docs. The docs will advise how to upgrade, if necessary. When you are done, add a comment in your plugin topic to describe what the last version that was 100% compatible with TWiki20040904, and a link to the zip/tgz (you may have to link to an older version in the version history of the attachment on twiki.org).

Maintain Plugin Compatibility for Cairo and Dakar (recommended)

If you want to follow route 3, things get a bit more complicated. You need to care for Plugin topics and Plugin code.

Once you are done, upload your Plugin as before. Please remember to test against both Cairo and Dakar.

Compatibility of Plugin Topics

The topic file format changed from Cairo's 1.0 format to Dakar's 1.1 format. The primary difference is that Cairo stored topics text with tab instead of three spaces (used in bullets etc), and Dakar stores topic text as seen in the browser. The encoding of meta data values also changed. In short, to keep your Plugin topic compatible with Cairo and Dakar you should edit the Plugin topics only on a Cairo installation. Alternatively, you can edit topics on a Dakar installation if you edit the topics on the shell level and if you keep 1.0 file format.

Compatibility of Plugin Code

For code you can use conditional code or compatibility code.

Here is an example of conditional code that outputs log messages, taken from the BlackListPlugin:

    $TWiki::Plugins::SESSION
      ? $TWiki::Plugins::SESSION->writeLog( "blacklist", "$web.$topic", $text )
      : TWiki::Store::writeLog( "blacklist", "$web.$topic", $ext );

The same can be done with an if statement, checking the Plugins API version:

    if( $TWiki::Plugins::VERSION < 1.1 ) {
        # Cairo codebase
        TWiki::Store::writeLog( "blacklist", "$web.$topic", $text );
    } else {
        # Dakar codebase
        $TWiki::Plugins::SESSION->writeLog( "blacklist", "$web.$topic", $text );
    }

Dakar offers some compatibility support, where different handlers are called for different API version. This is used where an existing handler has been deprecated in favour of a better implementation:

use vars qw( %TWikiCompatibility );

# This handler is called by TWiki with TWiki::Plugins 1.1 or later
$TWikiCompatibility{endRenderingHandler} = 1.1;
sub endRenderingHandler
{
   # ....
}

# This handler is only called by TWiki before TWiki::Plugins 1.1
sub postRenderingHandler {
   # ....
}

This causes Dakar to skip calling the endRenderingHandler.

Notes:

  • if you don't" define $TWikiCompatibility{endRenderingHandler} then *both handlers will be called, which is probably not what you want.
  • in $TWikiCompatibility{endRenderingHandler} = 1.1 the 1.1 is the version of the Plugins API that the handler was deprecated in. In this example the endRenderingHandler will not be caled if =$TWiki::Plugins::VERSION is 1.1 or greater.

-- Contributors: CrawfordCurrie, PeterThoeny

Discussion

The purpose of a Plugin API is to give Plugin developers a stable and safe environment to write extensions. In a way it is like the API of an OS or windowing system (X11 etc). For most part if you run a Windows program that uses just the published APIs and does not have other dependencies you can be sure that it runs, even if it is several generations back.

TWiki's Plugin API is unfortunately not like that. I find it unfortunate that it has become a moving target, breaking the promise of a stable API.

  • It has not become a moving target. There is a known bug in the API, but other than that Cairo plugins should be fully compatible with Dakar, as long as they restrict themselves to the published API. CC
    • There is work involved in fixing Plugins so that they work in Cairo and Dakar due to the file-format change and deprecated API functions & handlers. Deprecated functions and handlers require code changes because they are scheduled to be retired post Dakar. So, it is a moving target IMO. -- PTh
      • Even if we continue with this discussion about moving targets, there is something I want to point out: I bet that not every plugin works on a Beigin or Athens release out-of-the-box, mostly because several new functions and handlers where added to the API. So, the API has always being a moving target. I'm interested to know why suddenly this is such a big issue, given that most of the plugins that work in Cairo also work in Dakar -- RA.

I find it also offensive that developers make fun of people who have upgrade pain due to Plugin API changes. I hope Sven takes that back.

* No response from Sven so far -- PTh

To address this issue we need either to go back to the real purpose of an API (keep it stable and compatible), or define a process to change the Plugin API (by alerting stakeholders way ahead of time, and by giving customers more voice than core developers.)

  • AFAIK, by deprecating we're saying to stakeholders "in two releases? (two years) we're going to remove these handlers an functions". I think it is time enough. Also, I consider that most core developers are customers too, with the "advantage" that they put their code there they needs are. -- RA

And this is also a reason why we need a ProcessForMarketDrivenInnovation.

-- PeterThoeny - 09 Feb 2006

Length discussion on compatibility issue and bug is removed and can be viewed in rev 1.24.

-- PeterThoeny - 26 Feb 2006

I was wondering, what is the best way to handle differences in regular expressions? ($regex{wikiWordRegex} in Cairo, TWiki::Func::getRegularExpression('wikiWordRegex') in Dakar)?

-- SteffenPoulsen - 13 Feb 2006

Steffen, with conditional code at initialization time. Feel free to enhance above document with an example.

-- PeterThoeny - 13 Feb 2006

Hmm still can't get it to work, really - if I do something like

use TWiki qw(%regex); # Cairo compatibility for regexps
..
    if( $TWiki::Plugins::VERSION < 1.1 ) {
        # Cairo codebase
        $wnre   = $regex{webNameRegex};
        $wwre   = $regex{wikiWordRegex};
        $manre  = $regex{mixedAlphaNumRegex};
        $smanre = $regex{singleMixedAlphaNumRegex};
        $abbre  = $regex{abbrevRegex};
    } else {
        # Dakar codebase
        $wnre   = TWiki::Func::getRegularExpression('webNameRegex');
        $wwre   = TWiki::Func::getRegularExpression('wikiWordRegex');
        $manre  = TWiki::Func::getRegularExpression('mixedAlphaNum');
        $smanre = TWiki::Func::getRegularExpression('singleMixedAlphaNum');
        $abbre  = TWiki::Func::getRegularExpression('abbrevRegex');
    }

then I seem to get Unmatched ) in regex errors when running in Cairo. Tried various back and forth conversions (i.e. using $manre in []'s and without), but can't seem to find out just what will work in both cases.

-- SteffenPoulsen - 13 Feb 2006

Does this work? (without use TWiki qw(...))

$wnre   = $TWiki::regex{webNameRegex};

-- PeterThoeny - 13 Feb 2006

Steffen, getRegularExpression was a Cairo function. Despite the documentation to the effect that only a subset of the REs was available through it, in fact it could be used to access all the REs. You should be using the published function, not direct access to $regex. The only difference between Cairo and Dakar is the fact that Dakar actually documents all the REs.

   # Dakar and Cairo codebases
   $wnre   = TWiki::Func::getRegularExpression('webNameRegex');
   $wwre   = TWiki::Func::getRegularExpression('wikiWordRegex');
   $manre  = TWiki::Func::getRegularExpression('mixedAlphaNum');
   $smanre = TWiki::Func::getRegularExpression('singleMixedAlphaNum');
   $abbre  = TWiki::Func::getRegularExpression('abbrevRegex');

-- CC

Crawford, clearly we have different opinions here. I do not think we can hide the fact that there is work involved in fixing Plugins (see my note above.) I restored the text in a neutral way.

-- PeterThoeny - 14 Feb 2006

There is compatibility bug in the Metadata management for beforeSaveHandler, reported by PeterThoeny. For more information, check Bugs:Item1621.

On "Please upgrade TWiki is not always a feasible answer", we must keep in mind that his is an Open Source project, where people work on what interest them most. If a plugin author want to have compatibility with Cairo, is his choice. If someone needs a Dakar-specific plugin to be Cairo-compatible, and the author don't want to waste time on that (is a waste if he's not interested), then whoever has the need must put the code.

On "avoiding to download XyzPlugin for Cairo go here, for Dakar go there", I would say that if a TWiki Admin finds having multiple download links (one for each version) too cumbersome, he shoulnd't be a TWiki Admin. Period. Nearly every OSS I have found have multiple download links for different versions and branches.

-- RafaelAlvarez - 14 Feb 2006

Peter, there is something very very importan that is missing here: What is your proposal? Not deprecating anything? Not changing the TWiki::Func API? What's the difference between what you would like and what was done?

Also, I did some edits, sumarizing the discussion about the bug in beforeSavehandler

-- RafaelAlvarez - 14 Feb 2006

I thought I expressed my point clearly: What I would like to see is a stable API e.g. less depreciation of APIs and callback handlers, and if possible no removal of those. If you take tar for example, the latest versions still supports the ancient key characters so that scripts using tar do not break (old tar rfb /dev/nrst8 20 dir1 vs. new tar -r -f /dev/nrst8 -b 20 dir1 format)

For upcoming releases I would like to reduce the upgrade pains, such as described by JohnnyKwei in his home page, "Plugin support is the main headache. We use the ImmediateNotifyPlugin heavily and I had to modify it a whole lot to get it to work. There are a few I still can't get to work after making tons of changes. I mean, who is going to update the SectionalEditPlugin and ProjectPlannerPlugin?"

-- PeterThoeny - 27 Feb 2006

Those plugins are bad examples, as they bypass the Func API. As has been said several times in several places, we just can't be responsible for plugins that bypass the Func API.

Besides not changing the rendering semantic (I know this was an upgrade pain for some people) what we can do for Edin* is to finally provide a fully feature Plugin API, with well-defined points of access to Core functions. I mean, create an API layer for each module that is used by both Plugins and other core modules and make this API stable.

Of course, to achieve this we need to greatly decouple each module and define clearly the interface for each module. But then, we have at least a year for that, and that should be enough.

-- RafaelAlvarez - 27 Feb 2006

Can we collate a table of CommonlyAbusedCairoEntryPoints so to aid people adjusting plugins. I'm thinking:

Cairo non-publised methods Purpose Dakar equivalent
   

-- MartinCleaver - 04 Mar 2006

Refactored out SingleBranchPluginDevelopment

-- PeterThoeny - 30 Apr 2006

So it seems like it might be good to provide more resources on migrating from T3 to T4 (Peter mentioned a "Dakar API docs", but I did not see anything, unless that the TWikiFuncDotPm, which isn't an obvious name and it doesn't have migration help).

-- EricHanson - 30 Jun 2006

Eric, there are no additional migration docs. To compare Cairo and Dakar differences:

TWiki 01-Sep-2004 TWiki 4
Plugin API TWiki03.TWikiFuncModule TWiki04.TWikiFuncDotPm
Recommended storage of Plugin data TWiki03.TWikiPlugins TWiki04.TWikiPlugins
Plugin handlers Compare lib/TWiki/Plugins/EmptyPlugin.pm of both versions

-- PeterThoeny - 24 Jul 2006

Edit | Attach | Watch | Print version | History: r35 < r34 < r33 < r32 < r31 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r35 - 2006-07-24 - PeterThoeny
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.