documentation1Add my vote for this tag create new tag
, view all tags

Inspired by PluggableIsHarmful I started thinking about the documentation, and why it is so hard to use, both for users and for developers.

I have heard many times that the documentation is full of circular references, wordy without being informative, and in places downright misleading. Part of the reason for this is that the documentation really started out as a snapshot of a monolithic system, written by PeterThoeny and MikeMannix, that documented TWiki pretty well at the point in time where it was written. Since then TWiki has evolved, but the documentation architecture has not kept pace with the code architecture. The documentation is regularly the top reason for people to criticise.

As an illustration of this point, consider an installation where a range of plugins has been installed. Each plugin comes with it's own documentation package, which documents the plugin. However no plugin document relates to any other plugin document, core documentation doesn't refer to plugins, and the documentation for that installation is, to put it crudely, a complete mess. All you have to do is visit TWikiVariables on that installation to realise how bad it is; not a single plugin variable is even mentioned. User's don't care if a variable is plugged in or not; all they care about is finding the documentation of it.

Dakar injects a new level of complexity into this as well. People have started to see the advantages of a pluggable code architecture, where different applications can be built from a common code base (the TWikiFor concept). The plugins architecture does not, and cannot ever, address this requirement, as just about any class in the system might be swapped out and replaced. Every time you swap out a module, you need to swap out the documentation for that module as well. The current monolithic documentation makes that staggeringly difficult.

Ideally the documentation needs to be user focused, and good user focused documentation rarely maps to a code architecture. Unfortunately this implies that the documentation has to be heavily refactored (if not completely rewritten) for every release.

We needs some sort of compromise.

As with so many things the devil is in the detail, so here's a specific proposal for refactoring the documentation set to be more flexible and internally self-consistent.

  1. abstract the following code modules out of the core and into Contribs:
    1. Preferences
    2. Access controls
    3. Store/*
    4. Client/*
    5. Render
  2. The core documentation that remains must be re-organised such that any installed contrib can add:
    1. A description of the contrib and what it does, and how it relates to any documented abstraction
    2. Variable descriptions
    3. TML descriptions
    4. Installation notes
  3. Move all plugin configuration into LocalSite.cfg
    1. This implies stronger support for config vars in configure and the installers
    2. It increases security, and allows plugins to install to the TWiki web without breaking the readonly model
Each contrib must take the documentation of all TML and variables that module provides with it, leaving the core documentation usable even if that module is not present in the core. Each removed module may still be represented in the core docs, but only as a abstraction, and only if that module is absolutely required for a minimal working TWiki.

The core documentation should refer to the abstracted modules only in terms of the abstraction e.g. it must not refer to "files on disk" (the Store may be implemented in a DB) or "htpasswd" (the login manager may not use them). Note the action of moving code into contribs is purely symbolic; it is simply being used in this case as a way to make the separation in the documentation, and does not imply any functionality changes.

It should now be possible to write a MANIFEST with no include statements, run it, and end up with a minimal TWiki that works and has consistent, readable documentation.

The lack of hard links between abstractions in the documentation can be addressed by intelligent use of searches, and other wiki concepts such as WikiBadges.

-- Contributors: CrawfordCurrie


-- CrawfordCurrie - 21 Feb 2006

Yeah - this is the constructive way of continuing the scaring "pluggable is harmful". Many thanks smile

As one of the first steps we should brainstorm on the requirements - and then, as soon as possible, derive a "guideline for writing pluggable TWiki features" before people start checkin in on DEVELOP like mad.... Some unsorted thoughts on that:

  • Replacing the current monolithic documentation by something based on intelligent searches and WikiBadges amounts to making the core documentation dynamically "pull in" plugin docs. I don't see any alternative, however I feel a bit uneasy regarding the performance. TWiki documentation for a particular site shouldn't change very often. Readers may get annoyed if they have to wait for the manual to appear to long, as a result of TWiki grepping a whole directory each time they read it.
    • I'm not proposing making it totally dynamic, and never was. That is far too complicated. I forsee that all the dynamism comes about in the installer script for the plugin and configure, and that the running system is (almost) entirely static. Searches would just be used to make things a bit easier for the core docs authore CC
  • The pluggable documentation needs to be able to detect whether the corresponding pluggable code is "active" (i.e. not only "installed") at the moment of reading. It should done in a standardized way, so that other plugins can point to relevant documentation in the case of interactions between plugins (cf. the various table related plugins).
    • %IF{"context WysiwygPlugin_enabled" will do the job for plugins. No good for contribs, though. CC
  • Moving plugin configuration from plugin topics to LocalSite.cfg implies a change of the role model, but only in the (probably rare) case where /bin/configure is not being run by a member of the TWikiAdminGroup.
    • Correct. I have never encountered a site where non-admins are able to edit plugin topics. CC
  • If everything is pluggable, then sooner or later we'll going to have competing implementations for the same feature.
    • I hope so! CC
  • Pluggable documentation adds to the problem of DakarDocumentationModelIsBroken: twiki.org needs to document both how it works as a TWiki in itself, and what alternate options the twiki community as a whole offers to its customers.
    • I was wondering when someone would mention that. This is another reason why the model of editing documentation in a TWiki web is too restrictive. CC
  • Suggestion: Create a variable %PLUGININFO% to collect, in one place, information about that plugin: Is it installed, is it enabled, what's the version.... This is intended to be used in %IF{}% statements, rather than in documentation text.
  • Suggestion: For each of the functions where pluggable alternatives exist, create a variable showing which of them is in place (e.g. %LOGINMANAGER%) so that the core docs know where to point to.
  • SvenDowideit has announced some insights in CleanUpTWikiVariables which would come in handy at this point in time.

-- HaraldJoerg - 21 Feb 2006

I have it in mind that the configuration would all be done at plugin/contrib install time, same as now. That means the installers have to be able to add/remove stuff from documentation topics. Ideally they also need to be able to configure templates; since these are all text files, perhaps the same approach could be adopted for both?

Taking the LOGINMANAGER example, the doc has to say something like this:

TWiki can support logins using a range of different "login managers", any of which
can be plugged in to your installation.

If there are any login managers installed on your system, they will be listed below.
Click on the link to get more information about the individual login managers,
including installation and setup details:
<!-- LoginManagers -->
  * ApacheLoginManager
   * TemplateLoginManager
<!-- /LoginManagers -->

If there are no login managers installed, TWiki will not be able to support logins on your site.
The installer for "NewLoginManager" would search for the <!-- LoginManager --> block and add (or remove if uninstalling) an appropriate link above it.


May I suggest an "intelligent use of search" (© CC) instead (linebreaks added for legibility)?

If there are any login managers installed on your system, they will be listed below.
Click on the link to get more information about the individual login managers,
including installation and setup details:

%SEARCH{"Implements: LoginManager" 
         format="   * $web.$topic:
                      Version $formfield(Version)
                      $percntPLUGININFO{\"$topic\" format=\"$dollarenabled\"}$nop%}%

This would require new, compliant login managers to:

  1. Call their topic *LoginManager
  2. A standardized string to search for (not even necessary if the first requirement is strictly adhered to)
  3. a couple of (optional) form fields
...all of which should be easily achievable.

In the case of a LoginManager, unlike with documentation of variables, I wouldn't think performance is a problem - there'll be always only a few topics to search.

I'd really love if the core docs would be dead stable. Just consider an upgrade: With your installation approach, you'd have to merge the pristine doc for the new TWiki release with the enumeration of login managers which some installation procedure may have added to the old release's documentation.

-- HaraldJoerg - 21 Feb 2006

I've added info to CleanUpTWikiVariables, unfortunatly, i've not progressed further on that issue, I got sidetracked by the release itself.

-- SvenDowideit - 21 Feb 2006

Yes, dead stable core docs is the goal. But remember not everything is a plugin, Contribs have to be handled as well.

The SEARCH approach is OK, but the reason I described an alternative was that I'm not convinced that a search can cut it. Consider a complex module - say, a contrib. That module may declare documentation for several different sections in the documentation framework - for example, it might declare a couple of variables, need to be added to a couple of lists, add paragraphs to a couple more documents. The only way to use SEARCH to do that would be to instrument just about every point in every document. With the section proposal above, I was working towards something a lot more flexible; a "documentation patcher" for want of a better term.

I'd also like to do all this without requiring any code changes - i.e. the "documentation patcher" would be en entirely separate development, in the same way as the BuildContrib stands apart from the core code.

But the key question here is whether this is at all feasible. I don't mean feasible technically - I'm sure it is - but feasible literally - i.e. is it possible to generate a useable documentation set this way, or are the problems so great that it's not even worth trying?

BTW I just tried the experiment of splitting the monolithic TWikiVariables into a set of topics, one per variable, and a search. It took about 5 minutes to do, loads about as fast, and it's a lot more flexible, because plugins and contribs can simply add new VarXXX topics to the documentation set when they declare a variable.

-- CrawfordCurrie - 22 Feb 2006

Splitting that TWikiVariables: Excellent! Will need some fine-tuning, though... (see BUGS). I like the idea to accelerate the search collecting variable documentation by a convention for the topic names: This word has to be spread!

Re feasibility: I think it is worth trying. The key is that the architecture must allow a smooth migration from the current documentation set to a modularized one, so that every step is a step into the right direction. So I suggest we look for a consensus on that direction.

Most probably you are right in that a simple "pull" approach with SEARCH is not sufficient and we need documentation patchers. But, please, only as a last resort: Today admins can simply change either their plugin settings, or even core settings with /bin/configure, and don't care about documentation. A documentation patcher would call for a callback when saving LocalSite.cfg ... shudder ...

Should the concept be tested on one of the existing "pluggable" features, e.g. the often-quoted login managers? I'd volunteer to contribute, since I have an interest in getting my own login manager pluggable without patching TWiki.cfg (and a couple of other modules)... It even should have variables: %USERNAME% should belong to the login manager...

-- HaraldJoerg - 22 Feb 2006

  • only as a last resort - OK, that is a sensible standpoint.
  • %USERNAME% should belong to the login manager .... we are definitely on the same page, here.

We need PeterThoeny to comment here; he knows more about the documentation set than anyone else, and is well placed to assess feasibility of some of these ideas. I'd also like to hear from people like MeredithLesly, who has made noises about contributing to the doc more.

Clearly we need another pass over configuration; see SimplerConfiguration.

-- CrawfordCurrie - 23 Feb 2006

Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r8 - 2006-02-23 - CrawfordCurrie
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.