Tags:
create new tag
, view all tags

Developing Extensions with BuildContrib and TWikiShellContrib

Note: This documentation only applies to the latest SVN version of TWikiShellContrib.

BuildContrib and TWikiShellContrib are two very useful TWikiExtensions that can simplify the life of TWiki developers (both Core and Plugins) by automating the boring stuff (copying file around, running tests, packaging and uploading).

For the sake of simplicity, in this guide the term TWikiExtensions refers collectively to Plugins, Add-Ons, Code Contributions and Skins

Summary

Combining TWikiShellContrib and BuildContrib ease the development of TWikiExtensions. The lifecycle becomes:

perl twikishell plugin create TWikiExtension Create a new plugin/addon/contrib
perl twikishell plugin develop TWikiExtension Check out the extension
perl twikishell runtest TWikiExtension or perl twikishell build TWikiExtension test Run the tests for the extension
perl twikishell plugin putback TWikiExtension Check in the extension
(optionally) perl twikishell build release TWikiExtension Build a release zip/tgz for the extension
(optionally) perl twikishell package TWikiExtension ???
perl twikishell build upload TWikiExtension Upload the extension to TWiki.org

Overview

Before detailing how these Contrib complement each other, let's recap them individually.

BuildContrib

The Build.pm module can be used by developers to create a really simple build process for your plugin, addon or contrib module. The advantage of using this module is that it dictates a standard build procedure for plugins, so you won't get file names wrong or get the list of files in the plugin topic wrong.

The module also automatically generates an installer script that manages dependencies in the user installation.

The whole process is manifest-driven, that is, the operations are performed over a set of files as defined in a manifest file that must reside in a known place.

For a complete documentation, check TWiki:Plugins.BuildContrib.

TWikiShellContrib

TWikiShellContrib is a shell (Command line interface wrapper) that can be use to replace all twiki command line scripts with a single interface. Each command available in the twikishell is also available as a command line option using the -e switch.

Intrinsically, it does nothing, from a programmatic point of view it simply exposes perl methods to the command line. However, it comes bundled with some utility TWiki:Plugins.CommandSets, and it is one of these command sets, CommandSet::Plugin that is the subject of this article.

For complete documentation on extending the twikishell with your own commands, please read TWiki:Plugins.TWikiShellContrib

BuildContrib Quick guide to developing TWikiExtensions

BuildContrib performs many functions but it's purpose is pretty simple: By having three files (MANIFEST, DEPENDENCIES and build.pl) in a known place under the build directory, it can automate the build, test, package and upload processes of a package.

BuildContrib works on the basis that your develop your plugins outside of your live environment and that you are happy to 'install' them to try them out.

The Plugin command set turns this on its head. It assumes you want to develop on a live box and once you are finished you will publish them back to the development area before you use build contrib to package and upload them to TWiki.org.

  • Do we need this next section, Raf? Is it too much duplication of the BuildContrib docs?

Preparing to work

BuildContrib makes some basic assumptions about the enviroment:

  • The TWikiExtensions is not being developed your plugin in a live TWiki installation (which is usually a really bad idea), but are instead are developed in a separate directory tree, usually - but not always - a CVS checkout area.
  • The TWikiExtensions has a directory for itself outside the twiki root
    • (ok, this is not an assumption, but if you try to generate a manifest for the TWikiExtensions from a twiki installation it'll end up with a lot of useless entries. Also, it's a good habit to acquire ;))
    • Please explain this - MC
  • The enviroment variable TWIKI_LIBS has a colon-separated path list of Perl libraries needed to satisfy any of the TWikiExtension external dependencies. These may point to any directory.
  • The enviroment variable TWIKI_HOME point to the root of your test TWiki installation.
  • Your module follows the standards for plugins and contribs i.e. it
    1. Has a topic in the data/TWiki directory
    2. Has a perl module in the lib/TWiki/Plugins or lib/TWiki/Contrib directory
    3. Has a sub-directory alongside the perl module that contains the extra .pm files specific to the module

Normal Development Cycle

Let's illustrate how BuildContrib fits in the development lifecycle, step by step.

Create the Directory Structure

The first step to create a TWikiExtension is create the new directory structure in the checkout area. In this you will maintain your local changes.

The recommended directory layout & content for TWikiExtension development is:

|- twikiroot
|  |- twikiplugins
|  |  |- TWikiExtensionName
|  |  |  |- bin
|  |  |  |- lib
|  |  |  |  |- TWiki
|  |  |  |  |  |- (Contrib|Plugins)
|  |  |  |  |  |  |- TWikiExtensionName
|  |  |  |  |  |  |  |- build.pl
|  |  |  |  |  |  |  |- MANIFEST
|  |  |  |  |  |  |  |- DEPENDENCIES 
|  |  |  |  |  |  |- TWikiExtensionName.pm
|  |  |  |- data
|  |  |  |  |- TWiki
|  |  |  |  |   |- TWikiExtensionName.txt
|  |  |  |- pub
|  |  |  |- tests
|  |  |  |  |- unit
|  |  |  |  |  |- TWikiExtensionNameSuite.pm
|  |  |  |- templates

Note: twikiplugins is the checkout area from the plugins repository (either from CVS or SVN)

  • Raf - we're going to deprecate CVS - can you rework these out?

To build the initial MANIFEST file, invoke:

perl build.pl manifest

A "tentative" MANIFEST file will be sent to STDOUT, so it can be copy/pasted or redirected to the MANIFEST file and edited as needed. (This uses BuildContrib)

The MANIFEST file should minimally contain:

lib/TWiki/(Contrib|Plugins)/TWikiExtensionName.pm
data/TWiki/TWikiExtensionName.txt
tests/unit/TWikiExtensionNameSuite.pm

Note that none of the special files used by BuildContrib is in the MANIFEST, because they are usually not distributed to end-users.

For a complete description of the format of the MANIFEST and DEPENDENCIES files, check Here.

"Installing" the TWikiExtension

To test the TWikiExtension, it must be "installed" into the twikiroot. There are two options to do it:

  1. Make the change in the plugins checkout area, and deploy them to the test TWiki installation OR
  2. Deploy the plugin to the test TWiki installation, make the changes "live", and then copy back the modifications to the checkout area

BuildContrib can help in the first case. Invoking

perl build.pl install

will "install" the plugin (along any unsatisfied dependency) in the twiki installation pointed by the enviroment variable TWIKI_HOME.

Adding or Removing files

Often, files are added or removed from the distribution. For example, some plugins have utility methods not in the main .pm file but in separate module. Sometimes the plugin topic is not enough so additional topics are shipped with the plugin.

In those cases, the MANIFEST file must be updated with the adds/deletions or BuildContrib will not work properly.

Running tests

If Test::Unit is installed in the system, it's possible to run the unit test with BuildContrib. Just Invoke:

perl build.pl test

And it will try to run automatically the tests/unit/SamplePluginSuite.pm module.

Preparing the release package

To package the extension for a release, invoke

perl build.pl release

and it will create a .zip and a .tar.gz files based in the content of the MANIFEST file and an installer called TWikiExtensionName_installer (ie. SamplePlugin_installer). All these files will be created in the "root" of the plugin directory. (ie: under twikiplugins/SamplePlugin). As part of the release process, the TWikiExtension topic is processed an the following tags will be replaced:

  • %$MANIFEST% - TWiki table of files in MANIFEST
  • %$DEPENDENCIES% - list of dependencies from DEPENDENCIES
  • %$VERSION% version from $VERSION in main .pm
  • %$DATE% - local date
  • %$POD% - expands to the POD documentation for the package, excluding test modules.

Uploading the changes to TWiki.org

As the last step of the development lifecycle, the released version must be uploaded to the proper topic in TWiki.org. To do this, invoke:

perl build.pl upload

and BuildContrib will create the release package, and upload it automatically to the proper place in TWiki.org.

TWikiShellContrib Quick guide to develop TWikiExtensions

The default CommandSets bundled with TWikiShellContrib can be used to simplify even more the development process, by complementing the operations already performed by BuildContrib.

Preparing to work

TWikiShellContrib ask the user about the enviroment where it's running (where the twiki root is located, how the directories are configure, etc), but it makes two assuptions:

  • The checkout area is called "twikiplugins" and sits under the twiki root
  • MANIFEST and DEPENDENCIES are in the plugin "root"

Note that no enviroment variables need to be set for TWikiShellContrib to work.

Normal Development Cycle

To illustrate how TWikiShellContrib fits in the development lifecycle, let's follow the same process as with BuildContrib.

Create the Directory Structure

The first step to create a TWikiExtension is create the new directory structure in the checkout area, to maintain the repository up-to-date with local changes.

The directory structure is as follows:

A typical directory layout & content for TWikiExtension development is:

|- twikiroot
|  |- twikiplugins
|  |  |- TWikiExtensionName
|  |  |  |- MANIFEST
|  |  |  |- DEPENDENCIES 
|  |  |  |- bin
|  |  |  |- lib
|  |  |  |  |- TWiki
|  |  |  |  |  |- (Contrib|Plugins)
|  |  |  |  |  |  |- TWikiExtensionName
|  |  |  |  |  |  |  |- build.pl
|  |  |  |  |  |  |- TWikiExtensionName.pm
|  |  |  |- data
|  |  |  |  |- TWiki
|  |  |  |  |   |- TWikiExtensionName.txt
|  |  |  |- pub
|  |  |  |- tests
|  |  |  |  |- unit
|  |  |  |  |  |- TWikiExtensionNameSuite.pm
|  |  |  |- templates

Note: twikiplugins is the checkout area from the plugins repository (either from CVS or SVN)

To create this structure, invoke:

perl twikishell plugin create TWikiExtensionName

and TWikiShellContrib will create the directory structure with empty skeletons for all the files. The MANIFEST file is created automatically.

  • Raf - are these files placed under twikiplugins or in the users' live site?

The MANIFEST file should read:

lib/TWiki/Plugins/TWikiExtensionName.pm
data/TWiki/TWikiExtensionName.txt
tests/unit/TWikiExtensionNameSuite.pm

Note that none of the special files used by BuildContrib is in the MANIFEST, because they are usually not distributed to end-users.

Live editing of a TWikiExtension

The invocation:

perl twikishell plugin develop TWikiExtension

Deploys the plugin to the working TWiki installation, in which the programmer can make the changes "live". Another command helps pushes the modifications back to the checkout area.

plugin develop copies all content named in the MANIFEST of the plugin directory into the twiki installation. Also, two files called TWikiExtensionName.MF and TWikiExtensionName.DEP are created in the twiki root. TWikiExtension.MF will have a list of all the copied files, and should be identical to the MANIFEST file. Similary, the TWikiExtension.DEP should be identical to the DEPENDENCIES file.

Now, all the changes can be made "live".

After all the changes, invoke:

perl twikishell plugin putback TWikiExtension

and it will copy all the files listed in TWikiExtension.MF back to the checkout area, updating the MANIFEST and DEPENDENCIES with the content of TWikiExtension.MF and TWikiExtension.DEP respectively.

Adding or Removing files

When adding or removing files from the package, the TWikiExtension.MF must be updated so the proper MANIFEST file can be generated later.

Running tests

If Test::Unit is installed in the system, invoking

perl twikishell runtest TWikiExtensionSuite

will try to run automatically the tests/unit/TWikiExtensionSuite.pm module.

Preparing the release package

To release the package, TWikiShellContrib interfaces with BuildContrib, which handled the release process.

To create the release file, invoke

perl twikishell build TWikiExtension release

This will call automatically the proper build.pl script with the release target

Uploading the changes to TWiki.org

In the same way as releasing the package, invoking

perl twikishell build TWikiExtension upload

will call automatically the proper build.pl script with the upload target.

Additional Operation: Packaging the development version

If for some reason you need to package the whole development version of a TWikiExtension (including the MANIFEST, DEPENDENCIES,etc), invoke:

perl twikishell package TWikiExtension

This will create a tar.gz file in your current directory containing all the files for the plugin.

Building shortcuts

perl twikishell shorcuts

To reduce the amount of typing needed, use twikishell shortcuts to create a set of shellscripts for your path. Then you can do:

  • plugin create TWikiExtension
  • plugin develop TWikiExtension
  • runtest TWikiExtension or build TWikiExtension test
  • plugin putback TWikiExtension
  • (optionally) build release TWikiExtension
  • (optionally) package TWikiExtension
  • build upload TWikiExtension
Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r3 - 2005-11-07 - WillNorris
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.