Feature Proposal: TWiki Engines distributed as Contribs

Motivation

Recently, TWikiStandAlone project was accepted and is about to be merged. However I thought about the possibility to include in core only CGI and CLI engines, since they are fully compatible with actual mechanism and have no additional dependencies.

Other reason are:

• More engines can be developed, besides the ones that already exist
• People that doesn't have checkin rights to core svn can contribute with engines
• Development of engines can be faster (no need to go through core new feature/refactor process)

Description and Documentation

This proposal involves some code that is not yet in core, from TWikiStandAlone project. That project consists of an abstraction layer, that makes TWiki independent of execution mechanism (TWiki::Request, TWiki::Request::Upload, TWiki::Response and TWiki::LoginManager::Session), an engine infra-structure, that defines a common interface to the abstraction layer iterate with engines (TWiki::Engine), and some engines (CGI, CLI, ModPerl, FastCGI and HTTP standalone).

TWikiStandAlone proposal was accepted and when I was making the merge plan, I realized that only the abstraction layer, engine infra-structure and CGI+CLI engines really needs to go into core. With them, TWiki will have the same behavior and dependencies that have now. The other engines (FastCGI, ModPerl and HTTP) can be distributed as contribs and this proposal is about it: instead of merging all TWikiStandAlone project into core, merge only what really needs to go into core. (abstraction layer, engine infra-structure and CGI+CLI engines).

In summary:

Code	Role	Target
TWiki::Request TWiki::Request::Upload TWiki::Response TWiki::LoginManager::Session	Abstraction Layer	Core
TWiki::Engine	Engine Infra-Structure	Core
TWiki::Engine::CGI TWiki::Engine::CLI bin/twiki_cgi.pl	CGI+CLI Engines	Core
TWiki::Engine::FastCGI bin/twiki_fastcgi.pl	FastCGI Engine	Contrib
lib/TWiki/Engine/Apache.pm lib/TWiki/Engine/Apache/MP13.pm lib/TWiki/Engine/Apache2.pm lib/TWiki/Engine/Apache/MP19.pm lib/TWiki/Engine/Apache/MP20.pm	ModPerl Engine	Contrib
lib/TWiki/Engine/HTTP.pm lib/TWiki/Engine/HTTP/Restarter.pm tools/twiki_http.pl	HTTP Engine	Contrib

Each engine consists of a TWiki::Engine subclass, maybe some helper scripts and documentation. They should be named as "FooEngineContrib", currently it would be something like:

• FastCGIEngineContrib
  • lib/TWiki/Engine/FastCGI.pm
  • bin/twiki_fastcgi.pl
  • data/TWiki/FastCGIEngine.txt
• ModPerlEngineContrib
  • lib/TWiki/Engine/Apache.pm
  • lib/TWiki/Engine/Apache/MP13.pm
  • lib/TWiki/Engine/Apache2.pm
  • lib/TWiki/Engine/Apache/MP19.pm
  • lib/TWiki/Engine/Apache/MP20.pm
  • data/TWiki/ModPerlEngine.txt
• HTTPEngineContrib
  • lib/TWiki/Engine/HTTP.pm
  • lib/TWiki/Engine/HTTP/Restarter.pm
  • tools/twiki_http.pl
  • data/TWiki/HTTPEngine.txt

And each one would have within its documentation: description, dependencies, how to install, configure and use.

Impact

Discussion

On principle I'm very much in favour of modularisation of the core to support different target environments, but I'm not sure exactly what this proposal involves. What parts of the core are you proposing to move into the Engine classes? Can you be more specific? Perhaps there is some code we should be reading?

-- CrawfordCurrie - 28 Apr 2008

I added more details at description above and the contents of contrib package that each engine already developed would have. All the referenced code is at TWikiStandAlone branch.

-- GilmarSantosJr - 28 Apr 2008

IIUC, what you're proposing is basically to merge the refactoring that extracted the abstractions and the current engines (CGI+CLI), and leave all the engines you implemented as contribs. If that's so, I completely agree with that approach

-- RafaelAlvarez - 28 Apr 2008

Added TWiki::Request::Upload to the the abstraction layer's description above, for completeness.

You got the idea, Rafael

-- GilmarSantosJr - 28 Apr 2008

yes, that sounds like a useful separation - It will make it simpler to do the debian packages too....

What's the timetable for merging it all into trunk?

-- SvenDowideit - 28 Apr 2008

Agreed, that all sounds very sensible. I looked over the code in the branch, and it makes a great deal of sense. A couple of requests, though:

1. Please make sure there are unit tests wherever possible
2. Some more high level overview POD documentation would be nice, especially to help new contributors who may not be familiar with the old code.

Excellent work, Gilmar

-- CrawfordCurrie - 29 Apr 2008

Sven, I'm very busy now and I'm planning to dedicate 6-8 hours/week to play with TWiki. Since it seems to me that we have a consensus about this issue, I'll start to file detailed bug reports and commit changes. I'll try to write very clear reports and to commit all changes in a single turn, since in the middle-of-work trunk will probably be broken.

Thanks, Crawford! I'll consider to write the high-level documentation you suggested. It's very important to anyone that isn't familiar with the code. Unit tests are at my TODO list since a while. I'll work on them as soon as I learn how and have the time for it

-- GilmarSantosJr - 30 Apr 2008

Excellent news Gilmar - Sounds like I should devote a little time to setting up the nightly build and unit tests for the trunk branch, so that we can track your progress I seriously look forward to your work, as it removes alot of the remaining legacies that have held TWiki back.

-- SvenDowideit - 30 Apr 2008

Overall a very nice refactoring work Gilmar!

I am wondering what the performance of standard CGI mode is of TWiki before and after the code refactor. (That is what we ship out of the box.)

-- PeterThoeny - 01 May 2008