TWikiOO

This topic discusses whether having an OO TWiki is a good thing, and if so, what need to be considered. It does not stand as an action plan.

Motivation

According to Damian Conway, you should consider using OO (http://search.cpan.org/dist/perl/pod/perlmodstyle.pod):

• When the system is large or likely to become so
• When the data is aggregated in obvious structures that will become objects
• When the types of data form a natural hierarchy that can make use of inheritance
• When operations on data vary according to data type (making polymorphic invocation of methods feasible) (forms)
• When it is likely that new data types may be later introduced into the system, and will need to be handled by existing code (forms)
• When interactions between data are best represented by overloaded operators
• When the implementation of system components is likely to change over time (and hence should be encapsulated)
• When the system design is itself object-oriented
• When large amounts of client code will use the software (and should be insulated from changes in its implementation) (plugins)
• When many separate operations will need to be applied to the same set of data

-- MartinCleaver - 19 Oct 2004

red are the phrases I think are particularly applicable to TWiki -- CrawfordCurrie - 19 Oct 2004

Existing Notes

Some notes on converting the current TWiki towards TWikiOO. This conversion is now done locally. (See also: HowShouldTWikiBeModularized)

"Going OO requires another rewrite of TWiki. However, it should be possible to do the change into OO gradually, reusing most of the available libraries. ... Using a TWikiOO lib besides the provided TWiki lib, ...parts ... now directly call the original functions (thus now only providing a TWikiOO shell). This approach could be used to develop and test a TWikiOO without making the base code unstable again." -- HowShouldTWikiBeModularized

Design "specs"

Note: old notes removed...

General

I'll try to describe the design I have in mind. I'm not using any modelling tool right now, just words. It's also possible that some of the things I have in mind would not complete fit with the TWiki mission statement, but if you know a better way...

Rough application flow

99.999% of the TWikiOO functionality should be invisible from the webuser. the 0.001% should be a very small script invoking the (hidden) gateway, where the rest of the action is done (see the article at http://www.xml.com/pub/a/2001/12/12/cgi-xml.html where I got the idea from, although I won't be using CGI::AMLApplication since I can't get it installed (make'd) at my WinNT environment).

Gateway

The gateway will act as, well, a gateway between the public kickstart script and the real TWikiOO functionality. It will control the application flow at large. In case of a view: retrieve the (specified) template, retrieve the (specified) content, merge and show.

All communication is done via the Core object.

Core

The core object is a keeper of all the used TWikiOO objects, and is the entry point for the usage of these objects

TWikiOO Objects

I've broken the various components into functional groups (still not the final list):

The idea is that when one wants to store the data in an RDBMS, only the backend objects have to be "replaced", and the rest of the system will keep functioning like it did before etc. The replacement should be done via parameters, so that the switch can be done very easily for non-perl guru's.

Moving data around

Various bits of the data needs to be grouped together while moving it around in TWikiOO (like the meta data of a document and the contents itself)

Currently I'm (partly) making use of XML for internal data movements, mainly beceause I like to work with XML more (personal reason). The downside of this is of course that a lot of XML parsing needs to be done...

There's an option to make seperate objects of this info, but this would also involve the creation of lots of objects and new childs for new added functionality. I've not made up my mind about this yet...

New functionality

New functionality should be easily to add. There are 2 ways to do this:

• OOPlugIns
• Derived objects with added functionality

The PlugIns should be just as easy (or even more?) as they're now and are intended for little extra functionality.

Derived objects are intented for the big work, like replacing how the data is accessed (via a RDBMS instead of filebased).

Security

Just like TWiki now there should be ways to make the usage more secure.

However, this should be done by a -replaceable- object. Instead of splitting the scripts and making use of hand coded htaccess-files, this should be handled by this security object (if this kind of security is wanted).

I've have found some snippets of code that would make this very easy to implement.

In this case, very easily could be switched over to the use of cookies or any otehr form of session management.

Coding and testing

I'm using Test::Unit (http://perlunit.sf.net) for testing my code. I want to test every object and every method in this way in an automated manner. Why? See also the extreme programming sites on testing (like http://www.c2.com), and it keeps one sharp.

Large parts of the current code of TWiki now consists of big functions doing al the work. For TWikiOO these functions need to be split up. Reasons for this are: Various parts are reused, so why write it more than once? Why make it hard for yourself to analyse these functions? And, how would you test the behaviour of such a function that several things - how do you know where it fails?

Beware, this is mend to be constructive. The guys that have made TWiki as it is now have done a great job! But to modify their code is a real challenge if you don't have written it yourself - at least, that's my personnel experience.

Questions & Updates

Has this moved forward?

-- MikeMannix - 27 Dec 2001

Sorry for not updating this, but my internet access is currently very poor (combined with the difficulties sf provides). Work is in progress locally.

However, I stopped with the OO-wrapping. It lead to too many difficulties. What is in progress now is a new build in OO, trying to incorporate the current features of TWiki. I find that I rewrite much of the original code (code reafctoring), so much time is now spend doing testing. For the testing of the individual sub's I make use of PerlUnit - http://perlunit.sf.net - resulting in about 90% of the code I've written so far is testing code. Reason for the rewrite is that the original code is not always logical to follow, and I want the Objects being easily extendible.

Once I've done the view part, I'll upload it here.

-- HansDonner - 31 Dec 2001

Hans, thanks for the reply - it's excellent to hear from you, that you're still at it. In Sep, you started what seemed to me to be really interesting stuff, both with your "code audit", and then your OO work. So it would've been disappointing to've had all that then vanish!

I know I emailed you and we exchanged a note or two that way a while back. If that's easier in your Net situation, and if you're already in email touch with other developers, cc'ing me, or just dropping me an update here and there would be great. If there's any way you think I could help out, lemme know!

Sadly, really, I doubt that SF can last much longer, on this end or even on the other. New for '002...!

-- MikeMannix - 31 Dec 2001

Hans, sounds interesting. Make sure to share also the spec with the community, so that you get feedback. We could switch the TWiki core code base to your rewrite if you want and in case it is compatible and does not have a too big perfomance impact. Realistically it will run in parallel to BeijingRelease (and possibly its successor) until we could do the switch. One thing I would like to avoid is a code fork, jointly we are a stronger community.

-- PeterThoeny - 31 Dec 2001

Hans, I am very very interested in what you are doing ... I would like to help you in testing it!

-- AndreaSterbini - 1 Jan 2002

Hans and company, this is very exiting. I think it would be an interesting study at least, for TWiki's Object Oriented nature to be analized against Booch and Rambaugh style constructs. The Object Management Group has some nice standards in place that i think would be adaptable to TWikiOO.

Of particular interest is the Unified Modeling Language. GentalWare offers the Community edition of Poseidon free of charge. Poseidon is a UML Modeling tool. It could be used to model TWikiOO's architecture into a graphical form and help to organize the core components and the PlugIns.

Visual Paradigm for UML is a easy-to-use UML Modeling tool, which supports UML 2.0 notations.

SDE for Eclipse is also a UML Modelling Plugin for Eclipse. It could be used to generate code from UML diagrams and reverse code to UML diagrams.

ArgoUml is also available as an open source alternative and the base code for Poseidon.

That aside the TwikiDraw could be modified to include UML artifacts with eventual rule referencing.-- DennisDaniels - 16 Mar 2002

I think it would be interesting to incorporate UML modeling and the OMG specs into TWikiOO. Also w3c's Document Object Model should be considered for 'objectifying' TWiki, IMHO.

I think these standards and methods could advance the TWikiOO project a lot. Let me know what ya'll think.

-- CharleyQuinton - 13 Feb 2002

I'd be really interested in reading the architecture documents for your proposals Hans.

Somewhat separately - I was thinking - it might be interesting to make TWiki pages objects in that you could ask a page to perform an operation on itself or on other pages.

Hence a Photo Gallery page could inherit behaviour from the NavPlugin and collate thumbnails from a list of pages. When someone reads the gallery page, code in the NavPlugin page gets run to do the collation.

-- MartinCleaver - 24 Feb 2002

See my new notes above. I don't really work with some written or drawn specifactions, I just mess around and then come up with some new point (I'm not a programmer by profession, just do it for a hobby). Maybe better to first specify some basic requirements for the specs and then build it from there as I expand things along the way....

-- HansDonner - 24 Feb 2002

InterfaceThread - FeaturesThread

^^ Why is this list explicitly included in the page text? Surely we can alter the template to show it from information in the ProjectGroup ? - MC

I am also extremely interested in helping out with this, from a programming perspective, and can make the time to take on some of the load. I have had the itch to do an Perl-OO rewrite ever since I set eyes on the code two years ago.

I have a lot of experience in writing dynamic object architectures for my job at Sun as an asic tools development engineer, and have been writing internal extensions for TWiki over the last year (sorry, haven't had time to polish them up for general consumption).

TWiki is gaining general acceptance in our organization (Volume Server Products), and has proven to be the most effective collaboration tool we have used to date. Before I deploy it on a wider scale, I'd like to spend a couple of months on enhancements and moving over to an OO codebase. My schedule is rather tight, so maybe that could serve as motivation to help get an initial rev out sooner rather than later.

I already have a number of ideas in mind, but they are entirely codebase oriented, rather than document object model based. I figure if the core object/class design is done correctly, adding features and virtual object models for document/data structure should be as simple as defining a couple of new objects. The important thing is getting the base object support framework defined, and a few basic TWiki handler objects defined, which may be swapped out at a later time for something more sophistocated (natural extensibility).

I would also like to suggest that the codebase be defined as a class or set of classes which may be inherited, and whose methods may be overriden (as simple as using an @ISA = qw(TWiki) call) in cases where heavy customization is required. The TWiki object should also just inherit from CGI, and become the query object. In my view script, all I should see is something like this:

   $query=TWiki->new();
   print $query->handleRequest("view");
   

Hans, if you have something else in mind, feel free to stop me before I go willy nilly coding it up.

Some things I have in mind:

• Subclass CGI and replace the CGI query object with a TWiki query object.
• I'm developing an object architecture which, for example, could handle user authentication/identification in such a way that multiple methods could be used (e.g. an object that uses TWikiUsers for identification could be swapped out for one that uses LDAP to construct the user's TWikiName)
• I'd apply the above object architecture to each of the basic functions of TWiki. For example, the installer could select between RcsLite, DBM, or other methods of storage, if they have the correct classes installed.
• The TWiki.cfg file is being separated into a global configuration file and an instance-level configuration file (a non-perl text file which is read in for every invocation of TWiki->new(), containing overrides to the contents of TWiki.cfg). This should allow any number of TWiki systems to be installed on a single mod_perl apache environment.
• I'll be adding direct support for mod_perl callbacks (garbage collection, etc).
• The plugin interface will have to change somewhat (haven't thought that far).
• I'd like to give the user the option to install TWiki as a standard perl module (using perl's Makefile.PL interface for module dependency checking, etc), or use it locally (in ../lib).
• I'm adding hierarchical data storage/search/rename (already added it to my existing installation)

Like I said, I'm extremely motivated and willing to review/develop/test/refactor any available or non-existant next-generation OO code to make this happen.

-- PeterNixon - 17 May 2002

Changes to work with RcsLite mean that two parts of TWiki now use simple objects. One being the file under Store, the other meta variable handling.

-- JohnTalintyre - 17 May 2002

Maybe is time to look also at template language, and use something widely accepted, instead of homegrown, like HTML::Template. May save some time, too.

-- PeterMasiar - 18 May 2002

Might be a good idea to break the templating discussion out into a separate TWikiOOTemplatingSystem topic that covers the pros/cons of each templating system. It might be good enough to make the TWikiTemplatingSystem it's own perl package that could be swapped out for another system. Other systems could be integrated by subclassing them with an interface package that implements the same interface as the TWikiTemplatingSystem.

At the moment, I think the TWiki templating system, plus the use of plugins, provides the required functionality and performance, but it would be good to "productize" it by making it a legitimate installable perl module for others to use outside of TWiki. IMO, the same goes for most of the functionality of TWiki.

-- PeterNixon - 19 May 2002

I agree totally about splitting out and modularising the functionality.

I mentioned this in both CpanPerlModulesRequirement (that we should use standard modules to stop reinventing the wheel and repackage each bit of functionality so that others can improve on the wheels we build) and in ApplicationServer (in which I noted that we are building both an application and an application server and we should seek to split these and possibly use a standard Perl ApplicationServer).

However, I have been told that generally not the TWiki philosophy to depend on external modules. I sincerely hope this is not the case as this can surely not be sustainable.

-- MartinCleaver - 20 May 2002

On the question of CPAN modules the line we've been taking is that TWiki should be easy to install. Many intranet users have problems installing a large number of CPAN modules, especially those that need to be compiled. So at present we'll add in CPAN modules that give good value and that are small enough to include with the TWiki distribution.

OO will give much stronger benefits in some parts of TWiki than others, I suggest a gradual refactoring along these lines. If you look at the Store work I did you will see that I added lots of unit tests (under tools\test) as I went OO. In this case OO gives benefits as polymorphism is used for different storage drivers. Incidentally I also used one CPAN module.

-- JohnTalintyre - 20 May 2002

I would suggest that we still allow the user to install the TWiki modules where he is most likely to have write permission (the current install mode), while also allowing TWiki to be installed via the standard CPAN type install (perl Makefile.PL ; make install). As long as there are no compiled modules, this should work well. There may be a few special cases where compiled modules might be required (LDAP support for company intranets), but folks using those features will most likely have control over their own servers, and these will most likely be non-TWiki modules.

Here's how to install supporting CPAN modules in a nonstandard location:

gtar zxvf some-perl-package.version.tar.gz
cd some-perl-package
perl Makefile.PL INSTALLDIRS=site INSTALLSITELIB=/home/{user}/public_html/twiki/lib
make
make test
make install

You'll get some complaints about not being able to install pod documentation, but otherwise you should have no real problems.

As long as /home/{user}/public_html/twiki/lib == ../lib, these perl modules will be found by all the TWiki cgi scripts.

-- PeterNixon - 20 May 2002

The above highlight the real problems. Not everyone will have make or gtar. I'm suggesting getting modules from CPAN is that hard, but it's hard enough to put some people off. Many people trying out TWiki don't know Perl (I was once one of them), so they won't know about CPAN. Hence the idea of using modules, but only if they are small enough so we can include a known working version with TWiki. If there is a killing module that is big with lots of dependencies, it might be time to re-think. Note that the reason TWiki has its own template system is because we couldn't find anything else that was fully compatible, this is largely because of the overlap between TWiki and templating systems.

-- JohnTalintyre - 20 May 2002

I agree with that suggestion, so I'd like to modify my previous statement by proposing/asking that any plugin provider package a known working version of any CPAN modules that may be required for proper operation of said plugin in their distribution tarball/zipfile. For example, if I need MIME::Types for my plugin to work, here's what my tarball would contain:

bin/myscript
data/TWiki/MyPlugin.txt
lib/TWiki/Plugins/MyPlugin.pm
lib/MIME/Types.pm

At first glance, this doesn't look so bad, but if we have plugin providers go nuts and provide lots of differend CPAN packages with their distributions, the lib directory could get quite messy; It could be a little cleaner if we do this instead:

bin/myscript
data/TWiki/MyPlugin.txt
lib/TWiki/Plugins/MyPlugin.pm
lib/site-lib/MIME/Types.pm

and include a "use lib qw(../lib/site-lib);" in the CGI scripts.

The only problem I forsee is multiple plugins requiring the same CPAN module, but different versions. This might call for a plugin installation management script (so much for simplicity).

-- PeterNixon - 20 May 2002

If they did require particular versions of particular libs then there is something wrong. For a start, most good CPAN authors will not break backward compatibility. Additionally, if a TWiki plugin requires a particular old version then that plugin should be updated.

There would be little point in trying to account for multiple versions of the same plugin in the directory structure simply because Perl would keep a reference to only one of them.

Furthermore, the list of CPAN modules bundled with Perl is now very large. Is there any reason to not make use of those that come with say, Perl 5.6 upwards?

I agree with and support PeterNixon's lib/site-lib proposal.

• With regard to make and gtar (or tar + gzip).
  • There may be merit in providing a howto for these, possibly even providing binaries for the various platforms.

-- MartinCleaver - 25 May 2002

Would it be a good idea to add a "require 5.6.0;" to TWiki.pm, as insurance to both core and plugin developers that the 5.6.0 modules are available for use? It might be safe to assume that the majority of TWiki installations are using perl 5.6.0 or higher.

I don't know if this is helping, or if I'm just making noise, but I'm in the process of coming up a more detailed object architecture and interfaces to support the kinds of transactions that need to occur between the objects Hans is describing. I'm leaning towards a sort of TWikiService fabric to which requests for services can be made via a master TWiki object. TWikiService objects responsible for specific services would respond to such requests. In this scenario, the functionality of TWiki could be entirely composed of pluggable TWikiService objects of various types (Render, Config, Auth, Data, Template, Store, Plugin, etc...). Extensions could be developed by inheriting functionality from an existing Store class, for example, and overriding its methods with the desired functionality (as described by Hans). Basic Inter-Object TWikiService communications methods would also be inherited. I'll post more as it gels.

-- PeterNixon - 26 May 2002

Currently, TWiki targets Perl 5.005_03 (see ReadmeFirst - this is what is bundled with many Linux/Unix distributions. Upgrading to 5.6.1 would be useful for internationalisation, but I'm not sure when that is likely to happen, and until then we shouldn't depend on 5.6.

-- RichardDonkin - 29 May 2002

Given that 5.005_03 is a 1998 release (http://www.perldoc.com/perl5.6/pod/perlhist.html), I am surprised that that's what many (most?) Linux/Unix distributions bundle. But, fine, if that's true, good point.

I note that 5.005 includes LWP. (http://www.perldoc.com/perl5.005_03/lib/LWP.html). So, can we now use that instead of the bit of code in TWiki::Net that fetches pages using the 1.0 HTTP protocol?TWiki::Net causes the bug IncludeDoesNotSendHostHeader.

-- MartinCleaver - 30 May 2002

TWiki::Net also cannot use a proxy server, which most companies still enforce if they're behind a firewall.

-- PeterNixon - 07 Jul 2002

I'd like to start the following discussions, in the following order:

1. TWikiOOUseCases
2. TWikiObjectResponsibilities
3. TWikiObjectInteractions

-- PeterNixon - 15 Jul 2002

It's a good idea! Was forgotten?

-- AurelioAHeckert - 16 Sep 2004

No, not forgotten. But there's a huge amount of resistance (any OO ideas tend to get ignored or rejected). But the code is slowly - oh so slowly - moving towards an OO implementation.

-- CrawfordCurrie - 18 Sep 2004

In ModPerlize you can dowload an Object Oriented TWiki version.

-- MarziaMastrogiacomo - 05 Nov 2004