Installer script for plugins

The current process to install plugins is quite simple: Download plugin, unzip it in the twiki directory, and that's it!... Well, not quite. This process supposes that the plugin author has correctly package the plugin, that no Core package is present in the zip, and that the user make no modification to the default plugin properties. Also, if the plugin contains the ,v file for the topic, then the history in the user installation will be lost.

Much (if not all) of these problems also appeared in the old way to upgrade Twiki with a new release, most of the time the user had to manually merge the topics and configuration files. These problems were resolved in the Twiki installation process by the UpgradeTWiki script, but they remain for the plugins.

The proposal is to have a InstallPlugin script that perform those tasks. It can assume the following:

• It's located in the root directory of the Twiki installation
• Plugins packages (.zip, tar.gz or whatever) are placed in a well-known directory (like... pluginpackages or incoming)
• Plugin topics are in the TWiki or Plugins webs unless specified otherwise
• Plugin packages are named after the plugin (DefaultPlugin.zip for DefaultPlugin)

Then the script could be invoked like:

   perl InstallPlugin <plugin-name> [<web>]

where:

• <plugin-name> : is the name of the plugin to be installed (i.e: DefaultPlugin)
• <web> (optional) : is the name of the web where this plugin will be installed

The installation process should be:

1. Decompress the package to identify those files to be installed
2. Backup all the files that will be modified
3. If there is a file in the lib directory outside the ::Plugin package, give a warning to the user and ask for confirmation.
4. If there is a file in the bin directory to be overwritten, warn the user and ask for confirmation
5. If some topic that is not the plugin topic is to be overwritten, warn the user and ask for confirmation
6. Merge the changes in the plugin topic.
7. (this is a wishlist) Create an uninstall log (just in case)

(steps with () are already implemented) With a script like this, we could "register" plugins into twiki and avoid the discovery of plugins and handlers at runtime, thus increasing performance.

We could get more ambitious (and why not?) and we could have a complete "PluginManager" that download, install, uninstall, enable, disable and order the plugins

-- RafaelAlvarez - 10 Sep 2004

Rafael, glad to see someone taking this on - it's long overdue!

I'm assuming you read BuildContrib and recognise that the post-installer script generated by that package handles dependencies. However it doesn't handle any of your list above, of course, except the uninstall script. I did it the way I did because I didn't want to tackle the job you describe - which I agree needs doing.

Your installer script would have to be able to invoke the post-installs if a package installation requires them. Running the post-install may result in other packages being downloaded and installed as well.

Perhaps it would be better to ship the plugin dependencies in some different way and have your install script resolve them?

Some other things:

• I don;t understand the point of the "web" parameter.
• TWiki.cfg might point all over the shop - templates may not be where you expect them, the TWiki web might not be called TWiki, etc.
• ,v files should be autmatically versioned, which requires handling the locks correctly
• Must be clear about the assumptions made about the installing user. They may not be able to write to all the places they need to.
• Not all installations are pure perl; some require elevated installer privilege (e.g. WebDAVPlugin)
• Refer to MartinGregory for his experiences with TWikiUpgradeScript.

-- CrawfordCurrie - 10 Sep 2004

To be true, I haven't check BuildContrib. I drifted away from the plugin development into the plugin handling and plugin api area.

No, for the points you made:

• The point of the "web" parameter:

As I understood from the PluginsDotPm code, a plugin topic can be in any of three places (in order):

• • TWiki web
  • Plugins web (hardwired in the code)
  • The web of the topic being viewed

In the later case, the plugin will affect only the web where its topic resides. That's the point of the web parameter.

• TWiki.cfg might point all over the shop

Oops, haven't thought of that :). Perhaps having the config dir somewhere in the script or passed as a parameter in the command line or in a config file.

• Must be clear about the assumptions made about the installing user.

The script should check that the user has enough permission to perform the task.

• Not all installations are pure perl

Let's take care of those after the most common case is resolved

As for the plugin dependencies, they must be thought better... I can't decide if it's better to put them in the plugin main .pm file or if a separate file would be better.

-- RafaelAlvarez - 10 Sep 2004

Rafael - you should definitely check BuildContrib - especially the CVS version and the classes TWiki::Contrib::BuildContrib::TWikiCLI::Extension::Dev - this has a bunch of tools that I have almost working to do installs, uploads, etc.

They are in the same package as BuildContrib as there is functionality overlap with Crawford's work.

I plan to talk this through at Fontainbleu with Crawford a week from now but if you want to get into the game then please catch me on TWikiIRC.

Thanks, M.

-- MartinCleaver - 11 Sep 2004

Another example that can bite people who don't read instructions: DiscussionForumAddOn templates zip should not be just unzipped at the top of the hierarchy - it should be unzipped inside a specific data directory.

-- MartinCleaver - 15 Sep 2004

I just uploaded in the Plugins CVS (under the PluginInstaller module) what I have done so far. Please, check it out.

I checked the work in the TWiki::Contrib::BuildContrib::TWikiCLI package, and it seems to me that we're doing two different things. Well, it's more like we're building the same thing starting from opposite points. I think that the PluginInstaller can be modified to be an extention, so it can be invoked from the CLI. For now, thought, I prefer to focus on doing the installation from the already downloaded package.

There are few points I would like to highlight in the code:

1. The installer can configure itself, and if it detects a change in the enviroment it can reconfigure itself. If it fail to detect the comfiguration, then it will ask the user the information it needs.
2. The PluginsInstaller script is a bootstrap, the work is done by the Installer module. That because I didn't want the installer to depend on the location of any directory so I can't include the lib directory in the @INC path before knowing the configuration. The only "fixed" directory is the /incoming directory, and even that can change in a very near future.
3. Each "expected" directory in the plugin package (bin, data, pub, template, lib) is manage by either a Handle module or a handle sub in the Installer module. The last will be removed as soon as I finissh the Handlers for all directories.
4. (Updated) To handle the data directory, it should use the Twiki::UI::Save module to save the topic and create a new revision because that way the installer is independant on the Store implementation used. Saddly, it fails on me with the message "ci: diff failed from unknown reasons" or something like that, so i'm using "ci" directly .
5. The following files and dirs will not be processed:
   1. Any directory that don't have a corresponding Handler or a sub.
   2. Any file in the lib directory outside the TWiki::Plugins package
   3. Any ,v , .lock or ~ file
6. If an existing file is overwriten, a backup is created in the /incoming/ dir.

Few TODOs:

1. To handle plugins with uncommon installation steps, a post-install script can be run.
2. Managing diffs between the old and new topic is still pending.
3. It could detect the presense of wellformed plugins by searching for the existence of some file (like the dependancy file) or something like that.
4. For plugins that are not wellformed, it could unzip them in the install directory and tell the user to continue with a manual installation.

Take a look and tell me what you think.

-- RafaelAlvarez - 16 Sep 2004

Another requirement suggestion for this:

i.e. on installing a plugin, the installer checks to ensure that topics extracted into the TWiki web are prefixed with the name of the plugin.

This would make it impossible for plugins to introduce topics to the TWiki web with bad names.

-- MartinCleaver - 26 Sep 2004

I note that KupuEditorAddOn has its own installer kupu4twiki/install.pl ...

-- MartinCleaver - 03 Oct 2004

Another check: a plugin should not be installed on an incompatible version of TWiki: for instance I just dropped Kupu onto my Beijing install: no warning until I tried to run it. Now I have a manual clean up job to do.

-- MartinCleaver - 03 Oct 2004

If the plugin uses BuildContrib and if there is a Plugins.pm version cited in the dependencies, the BuildContrib installer will warn. However you can't overcome the old "oh shit I already overwrote it" syndrome, because the version check is done at runtime. There are a variety of ways to move that check into the installer script, that's something I'd like to see done. Really, someone needs to turn that basic script in BuildContrib into a real solution.

Is anyone really going to do this, or is it yet another stillborn project? Raf, you started it....?

-- CrawfordCurrie - 03 Oct 2004

Sorry for the long silence in this matter. I had litte coding time these last two weeks.

The basic installer is mostly done. I manage to put the versioning stuff there but I'm not really proud of it for few reasongs:

• Basically, I call the rcs command directly, so this will break with any other Store implementation (or with RCSLite).
• It's not managing the corresponding META tags for author and version and stuff.
• It's not merging the diff on topics

I got a little sidetracked by the work in twikicli and twikishell. I kind of merge both of them (the shell feature using Term::Shell with the discovery mechanism in twikicli) and use that as the launcher for the installer. That spawned a completely different project (called locally TWikiShell).

Now that the shell is working, I will proceed to solve the versioning issues and then merge the install/installer/dependency/download stuff from twikicli to the installer.

-- RafaelAlvarez - 04 Oct 2004

Refactored some discussion to AlternatePluginManagement

-- RafaelAlvarez - 04 Oct 2004

Well, I am glad you like the TWikiShell work Rafael! You would have been better to let me know you'd picked up that ball: for one thing I would have checked in my latest changes!!

I've now done so and will wait for you before doing any more on it.

-- MartinCleaver - 04 Oct 2004

Ok. Now that the TWikiShellContrib is up, time to go back to the PluginsInstaller.

-- RafaelAlvarez - 22 Oct 2004

This topic needs to be refactored with the configure-script-based plugin installer. In PluginInstaller was a question on security of using web based installer. This should be covered here as well.

-- PeterThoeny - 16 Jan 2007

This feature is already covered (and enhanced) by both the configure "extension installer" feature and the installer generated by BuildContrib

-- RafaelAlvarez - 08 Aug 2008