create new tag
, view all tags
Why think small!

This idea was prompted by several things:

  • Ongoing discussions about the need to refactor TWiki and the state of the current code base.
  • Related difficulty for relatively new programmer and TWiki deployers to easily modify or deploy it.
  • A remark by one poster about how his son would not touch TWiki because of it's proprietary tags and tag scheme.
  • Projects that have been in development for a long time tend to have their code bases become very disorganized.


  • Re-conceptualize TWiki.
    • Use standards like XML and XHTML exclusively. Use other standards like XSLT where they make sense. Eschew the use of "invented here" idioms unless there is no other way.
    • Think in terms of an engine that can support a variety of web based expressions such as web logs, wikis, workflow products, email, RSS feeds, etc.
    • Make page processing more data driven and less code driven to facilitate extensions (see Python Maki project for one example).
  • Re-architect TWiki.
    • Design a new pattern that supports the new TWiki concept. Lay out module archtecture for core, renderer(s), store(s), plugins etc. Architect for per module simplicity and extensibility.
  • Redesign TWiki.
    • Think OO. Think decoupling content/template/scripts/code.

I imagine at least two of the above points are controversial: XML and the engine concept.

XML has been (repeatedly, at least by me ;)) suggested as a representational system for TWiki and one objection has been about performance hits. I find it hard to imagine that being a valid argument in this day of multi-gigabyte servers and 3+GHZ "hyper-threaded" (virtual SMP) cpus.

As to the engine notion, it might be seen as diluting the primary purpose of TWiki as a wiki. I see wiki pages as just one means of expressing interactive content. Wiki pages can be the focus of the development, but I don't think that precludes thinking about a larger scope and making provisions for alternatives.

I have visions of plugins using namespaces to segregate themselves from each other, user extensible tag sets based on plugins that declare the ability to handle those tags and templates that declare their handlers. Yum smile

-- DavidLeBlanc - 13 Aug 2003

Sounds like another plea for a complete re-write to me. But why? Some responses to the points above:

  • Ongoing discussions about the need to refactor TWiki and the state of the current code base.
    • Where there is a need then fine, let's refactor. I've done that, there's nothing to stop others doing the same. Yes, the code base can always be improved - to be honest, especially some of my code :).
  • Related difficulty for relatively new programmer and TWiki deployers to easily modify or deploy it.
    • Not so sure this is such an issue, mind you there's always room for improvement.
  • A remark by one poster about how his son would not touch TWiki because of it's proprietary tags and tag scheme.
    • You can't please everyone ...
  • Projects that have been in development for a long time tend to have their code bases become very disorganized.
    • That's true, but if you track changes to TWiki code in CVS you'll see a lot of tidyup goes on.

So in summary, I still believe in evolutionary change. I'd like to see even more Plugin development, with some parts of the core going into defaut plugins.

-- JohnTalintyre - 13 Aug 2003

If there are many pleas for a rewrite, perhaps there's some smoke behind that fire.

I've been waiting for at least 2 years to see TWiki start moving towards being OO, but it's not happening and, IMO, it should. I don't think I'm alone in wanting TWiki to be OO. That implies a rewrite to me.

WRT to using standards for TWiki notation in lieu of the percent tag format, that has the immediate effect of opening TWiki to a wider audience of developers and content creators who are otherwise not interested in investing time to learn a relatively obscure application's notation. There's a reason why XML has become so popular and pervasive! There's a reason why hacky little languages and notations tend to fade away ("hacky" in this context means "invented here" and is not meant to imply that care and skill did not go into the development). If you change TWiki to use standard (XML) notation, then you please a lot more people than you do by having a proprietary scheme! You also open up the use of a lot more 3rd party tools and libraries and the ability to leverage those to both offload the development load and make TWiki more powerful.

Reconceptualizing, rearchitecting and redesign do not imply that all code gets thrown completely out and one starts from scratch in every case. It's likely that many bits of code would be integrated into a new framework and reused.

-- DavidLeBlanc - 13 Aug 2003

Some of TWiki is OO - I wrote those parts. I'm not convinced it's appropriate to a lot of TWiki, which uses the very efficiet regular expression substitution of Perl in a lot of the renderig pipeline. You say "perhaps there's some smoke behind that fire". What fire? Why is OO needed? I guess the general answer is to make the code base more manageable. I would say TWiki plugins have done a lot in this area - let's take that approach further. Many people have asked for CVS as a backend. With the OO nature of the storage backend a lot of progress could be made quite quickly; but despite much interest in this, no-one has produced any code.

Now XML. A very useful technology and certainly a potential storage format. But, not I think a useful input format. The use of a text area for TWiki (and most - all? wikis) input appears to be me to be both its greatest strength and weakness. Strength as it works in any browser, over any connection, is easy to implement and quick to use. But, it never going to be very easy to use. At present any very easy to use solution has many drawbacks. I hope we'll eventually manage something better, but for now I'll stick to text input and the JavascriptBasedEditor.

-- JohnTalintyre - 13 Aug 2003

Excuse me if I was unclear - with respect to XML, I was thinking in terms of template notation (thus the mention of "percent tags"), not user input. I do think it would be a good idea to convert wiki notation to xml for storage though.

-- DavidLeBlanc - 13 Aug 2003

We've started having some conversations about this on TWikiIRC as well David. Please feel welcome to join us.

-- MartinCleaver - 14 Aug 2003

If I may speak up as a mere end-user: I'd advise against integrating OO or XML techniques just because they are state of the art; only if they serve a tangible user-related purpose, they should of course be considered within the potential array of solutions.

(Aside: I do agree about the "standards" bit of David's plea. For example, trying to synchronize TWiki syntax with other Wiki dialects would make it much easier for others to migrate to TWiki.)

I'm also in favor of improving TWiki evolutionary rather than trying to rewrite it in large parts. I prefer small incremental steps which I can deploy easily; that's why plugins are so popular, IMHO.

'nuff rambling for now. So yes, I do think small, but I don't see anything wrong with it big grin

-- ClausBrod - 14 Aug 2003

Issues to be considered:

  • Why use XML ?
    • What use cases will it solve that the existing system doesn't handle? (API based use cases for treating as an app server here are of course valid)
    • Are there constraints that it imposes on content ?
      • Are these acceptable ?
      • Do they naturally (ie without extra meta data providing extra structure) prevent any useful use cases due to the lack of overlapping tags?
        • This can be solved using a separate tree decribing discrete structure addressed using NiDs
    • If XML is used as a backend storage - why does this require a rewrite of TWiki?
      • It would "just" be another storage backend.
      • How would this benefit the system over (say) the ideas laid out here?
        • Is it mutually exclusive? How would you do that efficiently with XML?
        • Would you want to [[][optimise this sort of behaviour/use]] however?
    • Why not use XML? see XhtmlConsideredHarmful
  • Where to use XSL ?
  • What's the user interface ?
    • The Wiki concept is a user interface in itself - one that tends to be more popular with engineers I've found than non-engineers.
      • LogicallyNestedWebs for example can be implemented using a physically nested structure or by metadata - allowing subwebs inside a single web.
      • This means even separate physical webs is just a storage optimisation.
      • A single XML file could therefore represent a collection of TWiki pages, rather than just one.
        • Are there interesting use cases resulting from, essentially, a book view that contains machine editable structure?
    • XML isn't designed as a human interface - it's designed as a human readable machine to machine interface (for want of a better phrasing)
    • Do we want a machine based interface though? XML is a "good" fit for some cases of that.
      • XML-RPC & SOAP are current vogue.
      • REST seems more pragmatic.
  • Where to use OO, where to use dataflow & transform?
    • These two are not mutually exclusive.

Lots of questions that need answering. This starts to look very much like BigDesignUpFront...

I can see advantages and disadvantages on both sides (Yes! NO!)

-- MichaelSparks - 14 Aug 2003

>... not touch TWiki because of it's proprietary tags and tag scheme.

this is an important issue, or rather it points to one. The wiki syntax of *strong* and _emphasized_ is extremely intuitive, easy to remember, reads as well in edit mode as view mode and nobody I know has expressed dissatisfaction with it.

Headings as ---+ and ---++ make logical sense and are remembered easily. They are still easy to read but have the unfortunate effect of reversed visual impact when editing (---++++++ has more visual weight than ---+ but denotes a less important heading). This can be remedied by adding dashes like this -------------------+ (thanks to PeterKlausner for pointing this out). I have heard people express minor dissatisfaction with the heading syntax, then forget about it and move on.

Bullets as    * is logical, remembered, reads well when editing, but a real pain to use when editing with more than one or two levels. People complain, get used to it, and are (eventually) quiet about it.

Denoting tables with | makes logical sense and is easy to remember while being a real mess to edit. Complaints are loud and never completely die out though they do quiet down.

See the pattern yet? At first we are using the way people naturally communicate almost unaltered and by the end we are re-inventing a new and incompatible markup language. The farther we go from the source point the harder it is to get people to even try TWiki, let alone use it. We need to back away from creating more markup language and focus on the natural language extensions which are the source of wiki power. There have been some very good plugin starts in this arena but they remain on the fringes, mostly because of the large developer and user base which are reluctant to give up the calluses from learning the existing language.

And yes I know David was talking more about twiki's back end code, TMPL:DEF{"athing"} as opposed to <template type="define" value="athing"> .However it serves as a good jumping off point for something I've been chewing on for awhile so I'm using it! If there are developers who prefer to work with that syntax a parser/converter should be pretty straightforward. The mapping looks pretty even. I'm not a programmer though, so what do I know? ;- )

-- MattWilkie - 14 Aug 2003

Matt, your observations are dead right: the very basic markup of TWiki is nice: _this_ instead of Cunningham's ''this'' was a good choice. But the more advanced mark-up choices are either not so nice or not so practical -- or both frown My personal favorite: reStructured Text by the Pythonistas. Headings, bullet lists, description lists... almost everything pleases my eyes more when reading and my fingers when typing!
I tried to rewrite some of TWiki's rules via the DefaultPlugin, but this tends to have odd side effects. I would really like to have pluggable syntax definitions and converters from/to other WikiMLs.

As to the back-end templates: using all % or all > < should be no big deal either way. The problems stem from far too much intertwingling of code, templates, variables, web preferences, site preferences, you name it -- XML is no silver bullet here... Oh, and don't forget Reisner's Rule of Conceptual Inertia: "If you think big enough, you'll never have to do it."

-- PeterKlausner - 14 Aug 2003

Conceptual Inertia! So, that's what I've been suffering from!

Ok, my notion about XML has everything to do with templating and etc. and nothing at all to do with user-level TWiki markup (I hate using other wikis because TWiki has set the standard for me). There is a class of people, many of them admins, who would be reluctant to work on TWiki meta content (templates etc.) if they have to learn a proprietary tag notation IMO.

BTW, does TWiki have overlapping template tags? If so, bad TWiki! bad! wink

WRT user level notation, in a well architected system, that too could be just another user configuration option, one that is facilitated by having everything stored in the back in XML. Being able to offer a "foreign" wiki notation could be a powerful incentive for people to migrate to TWiki.

-- DavidLeBlanc - 15 Aug 2003

Added comment pointing to XhtmlConsideredHarmful -- ColasNahaboo - 15 Aug 2003

I like fully OO pluggable formatter of KwikiWiki. And also bullets are more intuitive IMHO: http://www.kwiki.org/index.cgi?KwikiFormattingRules

* level 1
** level2

-- PeterMasiar - 15 Aug 2003

> We need to back away from creating more markup language and focus on the natural
> language extensions which are the source of wiki power.

I would second this - when I rolled out the NamedIncludeSections at work, I had some people complaining that the syntax was too complex - why can't it pick the sections automatically like everything else? In order to make it acceptable I had to create an app that fills in the contents of the section tags automatically. I actually agree with the sentiment - % based tags are a) ugly b) reinventing a wheel c) go againt the WikiWay - that of replacing machine friendly tags with human friendly syntax.

I would hardly argue that the following (single search tag) is user friendly and follows the values of the WikiWay:

%SEARCH{ "[S]ECTION{.*title;" limit="10" scope="text"  reverse="on"
order="modified" nosearch="on" regex="on" nototal="on"
$n$percntINCLUDE{$quot$topic$quot section=$quotsynopsis$quot}$percnt"}%

(NB, I'm using Peter's syntax for this example) I suspect that's tricky to follow, even for the most hardened TWiki user. (Let alone trying to pull out parts using patterns)

Examples which work well though for users : the SlideShowPlugin, the InterwikiPlugin, CommentPlugin etc. (as well as a number of other features)

>BTW, does TWiki have overlapping template tags? If so, bad TWiki! bad! wink

Not template tags. Unless you count using NamedIncludeSections - and actually that follows a good principle that of Wiki:DontRepeatYourself .

-- MichaelSparks - 15 Aug 2003

Edit | Attach | Watch | Print version | History: r16 < r15 < r14 < r13 < r12 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r16 - 2005-02-18 - SamHasler
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.