Discussion of TWikiWhatWillYouBeWhenYouGrowUp

This isn't the first time this has been discussed, as I'm sure you're aware. The last brainstorming that the community had on this was VisionsOfTWiki . Also jumping out as related are a database called TWiki.

Incidentally, TWikiForms didn't come first - WikiButtons came first and are heavily used on most other wikis. Then came TWikiCategoryTables .

The advantage of WikiButtons over WebForms is the fact it leaves the power to communicate what's important in the hands of the entire wiki userbase rather than concentrating power in a small cabal. The recent suggestion of eliminating interpetation of META tags in topic tags by things like METASEARCH and so on is disturbing and continues the process of trusting users less and less - very anti-wiki. (Increases in access control, limiting of preferences, denial of access to namespace creation (trivial for all users in other wikis - like UseModWiki, MoinMoin, PmWiki).

Flexwiki's use of Key: Value is bothvery wiki (and was suggested as an alternative syntax for metadata), and has a long history of effectiveness. (MIME, RFC:822, RFC:2822, HTTP, RTSP, NNTP all use that approach for metadata, with the approach continuing to scale and cope with clients/servers having very different versions. What they're doing is almost enough to make me consider using windows as a server

Other things worth looking at would be logically nested webs, since essentially the issues there are about data organisation, linking and extraction.

Also fine grained addressing, named include sections, and better - implicit named sections are worth considering. PeterKlausner's reaction against Plugins.SectionalEditPlugin - which led to his comments on his work in Plugins.IncludeIndexPlugin.

Based on heavy use of Plugins.SectionalEditPlugin and named include sections, I suspect that what will actually happen longer term on most wikis is a convergence of the two such that internal wiki anchors can pull in topics of the same name, or be referenced as independent topics.

That way the dichotomy referenced by PeterKlausner is eliminated, and named sections become much more lightweight in terms of syntax. (And hence more wiki like)

You probably should also reference sites like WikiFutures, WikiFeatures, and various other wikis to see what they're doing.

Other notes:

Bear in mind though, many people actually want a wiki. They don't necessarily want more. Out of the dozen wiki installs inside the corporation I work for none of the ones which I had installed were TWiki based. The reasons given? 1) it wasn't wiki like enough 2) difficult to install and upgrade 3) bloated in terms of features.

Sometimes people really do want the simplest database that could possibly work - ie something quick - without all the faff.

When users do want to migrate to something with more structure, as people often do, they want to be empowered to do this. (cf all the bunfights last year about a new namespace/webform - on other wikis, users would just do this, and be empowered to do this)

TWiki's default and "model TWiki culture" (TWiki.org) of not trusting the user prevents this feeling of empowerment (locking of TWikiPreferences, WebPreferences,EnforcedRegistration to edit with a username, HardSecurity rather than SoftSecurity - ie locking things to TWikiAdminGroup, rather than having a TrollGroup FINALPREFERNCES etc)

All those changes in TWiki as a content management system move TWiki more and more, slowly, slowly, bit, by bit, back towards a OneWebMasterSyndrome that made wikis (especially other ones) popular in the first place.

If that process continues, TWiki will end up diverging so far from what makes a wiki a wiki that it really shouldn't call itself one anymore. That would be bad, from everyone's perspective.

-- MS - 08 Feb 2004

Thomas, I know the environment you are contending with in Motorola, and the kinds of questions you probably get asked (why not Compass etc). In a tool-rich enviornment like this, why do people turn to wiki? What does it bring that the other CMS's (such as LiveLink) don't? You hit the nail on the head with your "whiteboard" description; this is the major advantage twiki offers over the others.

• Successfuly wiki salesmanship depends on identifying, and focusing on, the customers requirements.
• In our case they were (at the time):
  1. Basic wiki whiteboard
  2. Easy install
  3. No bugs

Casting my mind back, triggered by your comment regarding forms and slippery slopes, I recall now the days when we were trying to pick a wiki to use. It's interesting that Forms was one of the things that TWiki added early, and was the one feature that nearly decided me against adopting TWiki. Why? Well, at the time I just couldn't see that they benefited us in any way, and they seemed to add an unneccessary layer of complexity to our otherwise nice and simple whiteboard. I elected to go with TWiki anyway, because I thought we could just ignore forms, or maybe find a use for them later. Interestingly enough, we had almost 2 years of experience with TWiki before we found a use for forms. As with forms, another feature we could ignore was security, and for quite a while we did exactly that. In the end, we integrated with our LDAP server because the punters were fed up with having to remember their wikinames; but the important thing is, security was a feature that was there if we needed it, but we could choose to ignore.

• The ability to ignore features was an important advantage to us in getting started with twiki.

Wikis are usually installed, and generally initially used, by geeks. Once we were up and running, the next secondary advantage kicked in; GeekAppeal. The ability to add our own customisations. Initially these customisations were things like custom templates, template topics and categories ( not form based, WikiButton based). As time went on we customised further using plugins (most of which were shared back to the community).

• Simple (codeless) user customisation is an important feature of wiki over other corporate CMS's.

So, anything that locks out any part of the installer/user base from any of these features instantly reduces the appeal of twiki, and the liklihood of adoption. Overcomplexity, inflexible features, too-many-features-out-of-the-box, all reduce the initial appeal.

I think I'm basically agreeing with Michael here.

A quick rant on meta-data.

The use of meta-data above and beyond the absolutely necessary is a continuing source of complexity. Every time some piece of "structured" data is added to a topic, a new feature is required to enable editing of that datum. Unfortunately it often takes several tries to get that editing feature right - witness attachment management, which is still complex and confusing, and continues to get worse - and the very fact that a "special feature" is required limits the rate at which new ways to use the system can be found.

Something which has been annoying me for quite a while now is the way that TWiki arbitrarily adds meta-data (esp. "topic parent") to my topics. I wouldn't mind if it was invisible, but it shows up in searches, and has to be filtered in plugins, and I don't like that.

I'm not saying meta-data isn't necessary - absolutely it is - but it is taking over. Meta-data should only be used where it clearly enhances the TWiki experience, in a way that just can't be done otherwise. Meta-data should not be in the same space as the topic text. After all, RCS meta-data - the most important kind, in my view - isn't.

Here's my list of meta-data that I think should be purged from topics:

1. META:PARENT is unwelcome fluff. Generate topics from a template and add it there, if you insist.
2. META:TOPICINFO author and date fields duplicate RCS information.
3. META:TOPICMOVED can easily be inserted in the topic text.
4. META:FILEATTACHMENT - why oh why is this in the topic? The most the topic should need to know about attachments is %ATTACHURL% (Followup in GetRidOfMetaFileAttachment -- MartinCleaver)

I'd actually quite like to replace META:FORM and META:FIELD with in-line data, as suggested above, so I can see it and easily manipulate it when I do a "non-form" edit.

BTW, having the confidence to remove a feature is much more impressive than the confidence to add one.

end of rant

-- CrawfordCurrie - 09 Feb 2004

As one of the proponents of the WorkFlow system, I think i need to be the counter point.

TWikiForms is a major addition to Wiki'ness for me (professionally). I don't like the syntax that we have, nor do i like the seperation of topictext and the hidden bits. nor do i like the fact that when you create a topic with a template you get both a META version of the info and a table based one (i want the data to be in the file once and only once).

But....

the major selling point for the long term use of our TWiki has been the integration of not only the documents, but also the ability to automate our processes (and for all employee's to edit the processes).

Versioning is also very important.

I would like us to make the TwikiForms & MetaData system more wiki-ish, while adding to its funtionality (in the same way as i think we need to leverage the versioning information (and configuration management aspect) beter.

-- SvenDowideit - 09 Feb 2004

Sven, regarding workflow, have you looked at the WorkFlowAddOn? I have not finished uploading the code yet, but from the example pictures you probably could give some feedback whether this is what you had in mind? I'd appreciate your input. I read through WorkFlow and think what is described as use case there is covered by this addon.

-- ThomasWeigert - 09 Feb 2004

Don't get the wrong end of the stick, Sven, I'm not saying the forms meta-data is worthless; it definitely isn't. I'm saying it shouldn't be embedded in the topic text the way it is. I'm not averse to a parallel meta-database, in fact I've long been a proponent for more efficient storage of structured data (which is how I characterise most metadata). Workflows, forms, parents - all are meta-data associated with topics that should be abstracted out of the topic text and into a more appropriate medium (e.g. - here's a radical thought - a serialisable perl hash. The FormQueryPlugin already has a cache implementation that caches all the meta-data in a web this way).

BTW Thomas, the FormQueryPlugin is a very heavily based on and around forms. It actually takes twiki much further in the structured direction than any other plugin.

-- CrawfordCurrie - 09 Feb 2004

When talking about TWiki being locked down and going towards one Webmaster, it should be remembered that this is optional. Most of the above discussion is about meta data and it is really only form data that people are saying is too locked down. Now it's only the definition of the form that's locked down and then only if WebPreferences is locked down. In reality most people wouldn't create a form as they wouldn't understand how to.

I tried to write the meta data interfaces such that such data could easily move to a database i.e. the meta data is stripped from the topic early and put back late. But, in the interests of KISS, keeping the meta data in the topic worked well.

The use of Key: Value is mentioned above by MS as being more Wikiish. I fair point, the meta data structure never fealt that Wikish to me. How about adding support of this? Something like:

• Parsing topic.
• Picking out key value data and adding to the structure in the Meta object that stores form data
• Making this available in APIs, Include directives etc so that all keys can be accessed the same way

Questions and issues:

• Could later replace the current meta format
• Would you want to render this into the forms table?
• What about the current "Set" format used in preferences? We don't really want two ways of doing this
• Recents Alphas have ability to change variables using Forms, any duplciate Sets are updated. However, the variable has to be added to the form definition.

-- JohnTalintyre - 09 Feb 2004

No! I say again, move metadata out of the topic text, don't shove more in there!

The problem here is "who sees what". "wikiness" (I prefer WikiNature) is only truly relevant when seen through a users' eyes. A user does not care what format a topic is stored in, they only care what it looks like when it is presented to them. If you still preprocess topics with embedded forms and show the forms as HTML tables on the screen, you haven't changed the WikiNature at all, even if you completely reinvented the store format. On the other hand, if you change diffs to display word-by-word, you have radically improved the WikiNature, probably without any store format changes at all! Always, but always, try to think from the users' perspective.

In general the drive towards common meta-data stored in the text is a good one, see WikiCanonicalForm. But you have to know where to stop. Think about it from a types perspective. Wiki text is one type (freeform text), forms content is another (structured data). If you try to load them all into one place, you will encounter the following scenarios:

1. Form data interleaved with text is massively inefficient to search/extract
2. Meta-data that the user didn't type suddenly appears in their topic text
3. An innocent manual edit of form data accidentally destroys a carefully-wrought application

If you can identify data that is currently in meta-data, that shouldn't be there, by all means flatten it into the topic text - it must be data. But just farting about with the syntax of meta-data embedded in topics is dangerous and will result in ruin!

What Thomas says feels right; separation of the meta-data manipulating application from the wiki-freetext manipulating application is key to twiki's success as an application platform.

-- CrawfordCurrie - 09 Feb 2004

Moved my comments to MakingMetaDataMoreWikiData.

There's a simpler way of reducing this whole discussion - especially the whiteboard viewpoint:

• Wiki - n-way dialogue
• CMS - 1 to many monologue (or small group to many monologue)

-- MS - 10 Feb 2004

I'm not ready to move our metadata (like TWikiFormFields ) Out of band. I like the fact that I have a simple versioned database. I would like to increase its wiki-ness, but also see usefulness in a more collaboative CMS model. Traditional CMS may suffer from 1 to many, but i think we can do better (the versioning can save us

Thomas - i've looked at the WorkFlowAddOn a few times, but i really want to see more than images... i want to see what it takes to create and modify workflows - and the code otherwise its easy to say, it looks good.....

-- SvenDowideit - 10 Feb 2004

I am so sorry, Sven, for being tardy. I am just so swamped. I realize the WorkFlowAddOn topic is just a teaser right now. Your interest provides additional motiviation for finishing packaging it up.

-- ThomasWeigert - 10 Feb 2004

Regarding the meta data...

As I said earlier, I believe that twiki meta data today serves two purposes:

1. Represent "real" meta data, such as "parent", "topicinfo", "topicmoved"
2. Represent data for the other twiki applications besides the white board, such as form, attachments, SimpleTableEntryUsingForms, and a number of internal applications people have developed.

I don't think that (1) should be editable by the user. (2) belongs to the applications that created that data and operate on it and, therefore, should not be accessible to the white board application. At least, I don't want somebody putting text on the white board to mess with my application data.

Thus, given how twiki has evolved, it is inappropriate, in my opinion, to expose its meta data to the white board application.

-- ThomasWeigert - 10 Feb 2004

Whilst I can see some issues with TWiki meta data I don't understand why removal of some existing meta data is a critical issue. In response (or at least after my last contribution) Crawford states "No! I say again, move metadata out of the topic text, don't shove more in there!". I was certainly not suggesting adding more meta data. I'm afraid I'm failing to understand people concerns.

-- JohnTalintyre - 10 Feb 2004

It was your last two points I was reacting to, John.

Piecemeal removal of metadata is not what I'm after, though I do find TOPICPARENT somewhat objectionable. No, the discussion is around the removal of ALL metadata from the text topic, to a happier home. There are just so many topics running on this issue that I can't remember all the places it's been discussed, but there seems (to me) to be a growing concensus that cohabitation with the topic text is NOT the right way to handle meta.

-- CrawfordCurrie - 10 Feb 2004