Need a policy that guides inclusion of new plugins

There has recently been a flurry of proposed new plugins to be shipped with the distro. While it may be nice for a plugin author to have the importance of a plugin raised by being included in the distro, there are also disadvantages of having plugins included, as users end up getting more bulk.

In my situation, there are a number of plugins that come already with the standard distribution that we do not use, and I would prefer would not be laying around, e.g., WysiwygPlugin, RenderListPlugin, SmiliesPlugin, EditTablePlugin, CommentPlugin. On the other hand, there are plugins I always install that are not with the distribution, e.g., ControlsPlugin, EditTablerowPlugin, FormQueryPlugin (my mods), GenerateSearchPlugin, MultiEditPlugin, SectionalEditPlugin, SaydoPlugin (not published yet), SignaturePlugin, XpTrackerPlugin (my mods).

However, I do understand the marketing value of having lots of functionality available out of the box. I also understand the additional peer pressure to keep a plugin up-to-date it creates on a plugin author when her plugin is part of the distro.

How can we balance shipping everything, as in the SVN, vs. shipping no plugins?

Before we add any more, we should have a clear policy statement that helps us select the candidates, and maybe remove some of the existing plugins.

I personally would prefer to focus on having a flawless install procedure for plugins, rather than adding plugins to the distro. Maybe also a clear separation into plugins that are shown to work and plugins that are not on the web site?

-- Contributors: ThomasWeigert - 02 Oct 2006

Discussion

Hm, the cleanest, simplest and most logical way of distributing TWiki in a backward compatible way is IMHO: provide two (ZIP)-files: one containing the core code only, the other containing the coreplugins (for backwards compatibility). Downloading and installing Plugins isn't difficult at all. The challenge is to find the right combination of Plugins suiting ones needs.

-- FranzJosefSilli - 02 Oct 2006

Debian uses a similar approach: You can either download the whole package with each and every program in the distro (several CD's worth of data), or you can download the slimer "basic core" distro (180mb) and then use it to install the packages you'll need.

The first time I installed Debian, I downloaded the whole distro. Now that I'm familiar with that I need, I always install the core, and use a script to download and install the required packages.

I bet that something like that could be done with the current infrastructure (BuildContrib + Configure).

-- RafaelAlvarez - 03 Oct 2006

Currently it is a community decision what is included in the distribution. We never all agree. But by majority and as a result of history we have a set of plugins that are "default" which receives more attention than the rest.

These default plugins MUST work when TWiki is released. If the original author is not available, someone else will need to fix things. Otherwise a release is held back.

The rest of the non-default plugins are kept up to date as there are programmers willing to update them. Currently almost half of all plugins still do not work in TWiki4.

With Cairo I used EditTablerowPlugin on my Motion TWiki. I had only used it on 2-3 topics. From February till mid year EditTablerowPlugin did not work with TWiki4. So I had to change my topics and never changed them back. The plugin is maintained by the submitter of this proposal: Thomas. I am not at all trying to criticize anyone. TWiki is an open source project and generally people are not paid to contribute. And sometimes in our lifes and careers there is simply not time to work on TWiki. That we have to accept and respect. And be thankful for every contribution.

At Motorola Copenhagen I only have the default plugins plus those plugins that are maintained by someone that I personally believe to be very committed to keeping them running or maintained by more than one person. Or I evaluate that the plugin is very simple and clean using only official API.

We cannot live with the risk of being stuck and not being able to upgrade TWiki for maybe 6 months because a vital plugin has not been upgraded to be compatible with a new TWiki version - often because the plugin author has not yet upgraded TWiki himself.

So I cannot in any way support the idea of shipping TWiki without a default set of Plugins that the users and admins can trust long term, because I know it will be a lottery which non-default plugins that will work after each upgrade. It has nothing to do with how easy they are to install. It was never really difficult to install plugins.

The current way - with community decisions - based on TWikiMission - committed ownership - API cleanness - broad usage - is already there - etc - seems to be the only practical way to decide which are the default plugins. Like we also decide which features to add and which API to extend or change.

I would not know what to write a policy for it. Why is RenderList in? Why is ActionTracker not? Both good plugins.

The current set of default plugins is most of all the set of plugins that the current size of our regular development community is able to support as such. And we will have to take them on a case by case basic. And the bar will be set high.

-- KennethLavrsen - 03 Oct 2006

My suggestion was to have a policy or decision procedure. I was not proposing a particular policy.

As I understand your suggestion, the policy is to have

• community process
• demonstrated committment to long term support
• willingness of others to agree to maintain if author drops of the radar screen

That is a reasonable policy. By that measure, many of the newly suggested plugins should not be included due to lack of long-term demonstrated commitment.

I would argue that by your criteria a plugin should have gone through one TWiki release cycle before inclusion is considered.

I would support such policy.

-- ThomasWeigert - 04 Oct 2006

That's not quite the way it works. Yes, there is a community process for the decision, but once a plugin is in the core set, it is maintained as if it were core code. The original author will maintain it, but so will any other core developer. This is different to needing a commitment to long-term support from the original author.

While a release cycle is likely before a plugin is included, I don't see it as essential. Some plugins - for example, DateFieldPlugin - are so trivial that insisting on a release cycle is overkill.

IMHO the issues surrounding the community process far outweigh any technical issues. The choice of which plugins to include depends very much on what the TWiki will be used for, which is the origin of the TWikiFor proposals (that never took off due to lack of participation). My strong preference is to build and release a TWiki core that works well without any extensions (core, plugins or contribs) and then subsequently build TWikiFor's that enhance that core with plugins appropriate to a target audience. For example, public internet TWiki admins will want a different default config setup, and will want BlackListPlugin. An admin behind a firewall has rather different expectations. Our failure to distinguish these users has in the past lead to FUD.

-- CrawfordCurrie - 04 Oct 2006

The essential thing for the users is trust. And it is the fact that a small number of plugins are treated as if they were core code that make us users able to trust that they will always work - also after an upgrade. THAT is the essential reason for having them there in the first place.

The team that maintains the core on a regular basis is small and this is why the "bar" for including new default plugin is high.

The criteria: "Twiki releases cannot be relealed unless the default plugins work" is essential. But if the number of default plugins gets too high the core development maintenance grows out of proportions. If the number of default plugins grows the team of committed developers maintaining the TWiki core will have to grow with it.

We cannot just push in more plugins and think: "Ah. now Crawford will maintain it for me".

For the non-default plugins to key element to keep on working is to use only official API.

You cannot create a policy for something that you cannot commit resources for afterwards. It has to be a community decision. And inclusion of a lot of plugins in the core is something that for sure can trigger core team vetos.

-- KennethLavrsen - 04 Oct 2006

Ken, don't understand why you object to a clear statement of a decision criterion (a policy) whether a plugin should be included or not.

I am responding to the flurry of requests that I am seeing. I am fine with no additional plugins being included. I even more would prefer some plugins being pulled out that maybe great, but are not essential to an installation.

I am worried about adding too many plugins. A policy would aid in dealing with these requests.

-- ThomasWeigert - 04 Oct 2006

There are at least 3 reasons why I object.

• We will never agree in the community. You do not propose a policy yourself. And if one of us try to write one, a major negative discussion will follow. We will never agree on such a generic policy. We will have to decide on a case by case basis when a plugin is proposed because not two plugins are the same.
• I do personally not like to replace common sense with policy.
• If we have a policy that defines criteria for new plugins entering core, plugin writers will try to alter their plugins so that fulfill the criteria. And then we will be forced to take them in and we will end up with too many. And if we define a policy that disqualify our current plugins I would not want to see plugins that people have trusted for 4-5 years or more suddenly be abbandonned.

Note that for me it is not a matter of the plugins being distributed in the final ZIP file. It is a matter of having a list of plugins that MUST WORK when TWiki is released and are threated as if they are part of core. That is what is essential. And now that we have configure to enable and disable plugins it does not matter at all that the plugins are there. You can just turn them off if you do not like them. When we were about to release TWiki4 I wrote EssentialPluginsAtDakarRelease trying to get developer focus on the most popular plugins making sure they worked when TWiki4 was released. (The topic is not updated anymore with status). It was an attempt to list the most essential based on a mix of popularity, quality, and scope. I was only partly succesful. Many of them took months before they were upgraded. And there may still be one or two that do not work properly. The current scheme of including a set of plugins seems to have the best effect in getting the needed commitment from the developers to keep them running. And I am sure it is because of this people push for getting their favorite plugin included in the list of default plugins.

There is currently a number of plugins being proposed for inclusion. Nothing is new. People have proposed plugins to be included for years. Let us look at this "flurry of requests".

• EditTablerowPluginAsDefaultPlugin - Basically rejected but if the function is merged with EditTablePlugin it seems to have gained acceptance because we then keep the number of plugins unchanged and the features in EditTablerow would be a natural enhancement of EditTable. Additionally EditTable is an old plugin that needs some maintenance.
• FindElsewherePluginAsDefaultPlugin - The trend is that there is not the needed community support to vote it in. The vote is currently 3 to 1.
• MergeExtendedSelectPluginToCore - This was a few lines of code taken from a plugin and put in the core code. Again no new plugin.
• DateFieldPluginAsDefaultPlugin - The current trend leans towards including the few lines of code in this plugin in the core.

So until now no big danger of adding lots of new plugins to TWiki. I think the community is handling things well without a policy. Just using our brains. And we have a very fine collection of the best brains in our community.

-- KennethLavrsen - 04 Oct 2006

Some good discussions here. One point I am missing though: We need to focus, our TWiki project should be guided by the TWikiMission. This applies also to vetting plugins proposed for inclusion in the default distribution. Overall I think Kenneth layed out a good criteria.

-- PeterThoeny - 04 Oct 2006

I still like having more guidance. But anyway, I withdraw the suggestion to add EditTablerowPlugin due to that a merge is not appropriate.

However, I still suggest that we reduce the number of plugins in the distro.

-- ThomasWeigert - 05 Oct 2006

Kenneth paints an ideal picture; that somehow the team building a release is in a position to ensure all "official plugins work with it. Some observations:

• Most plugins included in the release lack even the most basic test fixtures, so are shipped effectively "on a wing and a prayer"
• Plugins are not adequately manually tested in the release process.
• Support for packages in the release is actually no better (and in some cases worse) compared to packages not in the release. For example, TablePlugin has had an open bug against it since February; hardly the sign of a well-supported extension.
  • I am not sure what that statement is supposed to mean, especially if you compare that to the many long living and open bugs of the TWiki core. -- PTh

I also paint an ideal above; of being able to build and release a core package with now plugins. If we were really serious about the TWikiMission, then this would be the right approach. Ship a minimal core, make sure a well tested and well publicised set of extensions is available and trivially easy to install/remove. Clearly mark extensions in plugins web that are "officially" tested against the release. Use the appraisals system to rate other extensions as well.

I don't seem to be getting any support for this view, though, so in the interests of finding a middle ground, here's a list of the extensions I'd like to drop from the standard distro, with my reasons:

• PatternSkin
  • I know you will all react strongly to this one. While I like this skin a lot, I don't think there should be any skins in the default release. We should focus on a clean, simple set of templates, and on ensuring that the process of installing extra skins is as painless as possible. Pattern skin is huge, and carries a whole bunch of other baggage with it - BehaviourContrib, TwistyContrib etc.
  • This is an absolutely essential plugin, TWikiMission: "Give broad corporate appeal, without sacrificing guidelines" -- PTh
• TwistyContrib
  • As PatternSkin
• BehaviourContrib
  • As PatternSkin
• TwistyPlugin
  • As PatternSkin
    • Arthur could have put the twisty and behavour contribs in the PatternSkin package. I guess he chose to split them out so other skins can take advantage of them which I think is a good thing. TwistyContrib and BehavourContrib are just a few Javascript files. KJL
• NatSkin
  • Same reasons.
    • Actually not a default skin! KJL
• ClassicSkin
  • Same reasons.
    • I would not be crying for long if this was suddenly gone. But I jumped on the TWiki train when Cairo was released so I have not developed any relationship with this old ugly thing. KJL
    • I never understood why we need to make the extra work of creating a separate ClassicSkin; the default skin (when no skin is enabled) looks like the classic skin. We should remove the ClassicSkin template files, but keep the ClassicSkin topic to document the default template files. -- PTh
• SpreadSheetPlugin
  • Violates the KISS principle from the TWikiMission. There is nothing simple about this plugin.
    • May not be simple but we use it all the time because it is one of those plugins that turn TWiki into more than just a text editor. It is heavily used. May the plugin author's soul be blessed for creating this versatile thing. And may his sock shrink in his laundry for the horrible syntax. KJL
    • This is an absolutely essential plugin for the typical corporate deployment. Also, the TWiki doc depends on this plugin in a few locations. It is KISS because WikiChampions can use it to make it easier for users to work on content. -- PTh
• CommentPlugin
  • Nice thing to have, but not used on all sites
    • I think 95%+ uses this plugin in some form or shape. It looks like nothing on the surface but you quickly learn how fantastic it is in TWiki applications thanks to its user defineable templates. KJL
    • This is an essential plugin for usability and productivity in a typical corporate deployment. -- PTh
• EditTablePlugin
  • Again, a nice thing to have, but definitely non-essential and violates the KISS principle. Horrendous piece of code, dreadful maintenance problem.
    • Crawford, it can be more effective to bring forward constructive comments in the EditTablePluginDev topic. -- PTh
    • From a customer perspective this is what makes editing tables doable. Editing large tables in TML is near impossible.KJL
    • This is an essential plugin for many simple TWiki applications. We need to fix the performance problem though. This plugin is KISS in a sense that it makes life much easier for certain applications, such as a call center status board. -- PTh
• RenderListPlugin
  • never understood how this one got included in the first place.
    • If I was to throw out a plugin from the default list - this would be my personal candidate. But that is my personal view. KJL
    • I use this plugin in every corporate deployment I am involved in, mainly as an organizational tool to make it easier to understand the organizational structure, to improve navigation, and to make it easier to find content in a large wiki (as described in HomePageNavigation). This plugin is less useful for smaller organizations with less than a few 1000 topics. With improved plugin installer we could consider removing this plugin from the core distribution. -- PTh
• SlideShowPlugin
  • Definitiely non-essential and violates the KISS principle. IME not easy to use.
    • Managers loves power point. This plugin is a TWiki seller. It has a lot of signal value. And we actually use it heavily in Motorola. We save hours each week by having auto generated presentations that we can just show. Before I spent half my professionel life copying information from emails and Word to Powerpoint. Now most of my presentations are just a TWiki page with SlideShowPlugin which includes misc other TWiki topics. I must admit that it could be easier to use. KJL
    • This is an essential plugin for educational purposes. It also helps sell the wiki behind corporate firewall. I do all my TWiki presentations and tutorials with this plugin. If we remove it we'd need to remove Crawford's ATasteOfTWiki presentation from the distribution, not a good idea since this is essential to get started. -- PTh
• TipsContrib
  • If the tips were maintained e.g. automatically derived from t.o, fine. But the content is pretty poor, and it takes a chunk of time to serve. On balance, I don't think this should be included.
    • This is the first thing I remove when I install TWiki4 KJL
    • This is an important component for educational purposes. TWikis has many bells and whistles, a tip of the day is a good way to teach the power of TWiki. We need more user centric tips. -- PTh
• WysiwygPlugin
  • I have always been against including this. I don't think it's a good advert for TWiki.
    • Sad that this was abbandonned. It looked so promissing. But it may warm your heart that it is heavily used in Motorola - even with its bugs and flaws and popular among a group of users that would otherwise not use TWiki. The hardcore users use TML. KJL
    • Many bugs, but a useful plugin. Should be actively maintained / can be replaced if we have a viable alernative. With the competitive situation, the TWiki project simply cannot afford not to have a WYSIWYG editor. Essential. -- PTh

-- CrawfordCurrie - 05 Oct 2006

I also feel that creating a 'core' releaseable that needs no plugins, and allows us to focus on making the tightest, fastest core engine to build plugins on. Then there can be a set of TWikiFor's that focus on TWikiMission.

-- SvenDowideit - 05 Oct 2006

Agreed. However, I would suggest that if we do this, we release the Core TWiki and a TWikiFor that is close to the current release (with some of the unnecessary plugins stripped out) at the same time.

P.S. Are you saying that BehaviourContrib must also be included to run pattern skin? This is getting out of control. There must be a fall back for JS being turned off. Twisties and any other stuff relying on JS should be pushed into a contrib, which probably applies also to my suggestion of dates in forms and JSCalendarContrib?

-- ThomasWeigert - 05 Oct 2006

The current set of default plugins is a compromise. You cannot find two people that will agree which should be default and which should not. I would personally remove a couple and add a couple if I was to decide alone.

I think this topic has now grown to a very clear proof that the community can never agree on a default plugin policy. We are so far apart that there is no point trying.

-- KennethLavrsen - 05 Oct 2006

Dunno about that; there are at least three of us who agree that a core that runs with no plugins is a good starting point. And I agree with Thomas - and with you, Kenneth - that a TWikiFor that is close to the current release is a good idea.

Note that the "releaseable core" concept is not as alien as it might sound. I regularly run TWiki4 with no extensions installed - especially when testing.

Let me paint a picture of a possible future.

The TWiki core is built using the MANIFEST in tools, which has no include lines. Then, twikiplugins/TWikiForGeneralRelease is built. That extension has a DEPENDENCIES file that lists the core package and all the default plugins. The upshot would be that installing that TWikiForGeneralRelease would be as simple as downloading and running TWikiForGeneralRelease_installer. If you have already installed the core package, then the remaining dependencies on the default plugins would be automatically resolved. You can link specific versions together that you know work.

The approach also encourages other people to develop TWikiFor packages that tailor TWiki for other application domains.

And it cuts that terrible dependency between the core and the default plugins that has been plaguing me for years!

-- CrawfordCurrie - 06 Oct 2006

I like the TWikiFor idea, plus some kind of "Core plugin" set. Actually, http://www.nagios.org is an example of an other project that is doing something like this, they have an "official distribution", an "official plugins" package and a "core addons" package - and it seems to work well.

Currently the plugins I would like to see closer to the core is the plugins that works well in a hierarchical web setting, i.e.:

• TagMePlugin - site-wide index, excellent to cross-link stuff
• FindElsewherePlugin - site-wide reference, vocabolary
• SearchEnginePluceneAddOn - site-wide rated search, mixed results (not listed "pr. web", but listed "pr. relevance")

I would say these plugins are the most used plugins globally in our setting now, and have helped us in scaling the number of webs tremendeously.

At the same path I'm starting to miss stuff such as a SiteRSS, a SiteNotify and so on; it could be part of the same "TWikiFor".

I shouldn't be discussing this here; but baseline is simply that I also have my own idea of what plugins should go where and why - it is not really easy seeing a concensus established here.

-- SteffenPoulsen - 06 Oct 2006

No, I do not think it is a good idea to remove the current essential plugins such as the PatternSkin, SpreadSheetPlugin, CommentPlugin, etc.

The current set of plugins is a nice balance of what many corporate users need, e.g. is in line with the TWikiMission. Well, as we have seen, there are actually many opinions on what a nice balance is; and we will never agree on all points as Kenneth pointed out.

I understand the value and flexibility of the TWikiFor concept: One lean base package and packages for several vertial applications is a compelling idea. I am very much concerned however that we:

• Lose the focus on the TWikiMission.
• We may lose the commitment to test and maintain the current core plugins to be always fit for the primary corporate release, since developers will naturally focus on making the base package stable.
• With no plugins in the base package it will be easier to enhance the plugin API in an incompatible way. There are millions of hours invested in TWiki content by our user base. I hope I get more support from the developers to "protect corporate investment from data corruption and incompatible changes". I would like to avoid repeating the hickups we recently had.
• We are streching our scarse resources even thinner.

-- PeterThoeny - 06 Oct 2006

OK, Peter, can you give us some idea of what we would need to do, to enable those of us that want or need a TWikiCore package (like Bruce suggests in RickMachWouldLikeToCheckIn ) to create the simplest TWiki that works? It seems to me that there are quite a number of developers that would like to try it (like we tried a more community based development system for Dakar). Thus far you've listed concerns, but whats needed is a way to address those, and other (Bruce's and the others that worry about core bloat) concerns - just indicating issues seems to often have the side effect of stagnation.

-- SvenDowideit - 07 Oct 2006

Sven, I hope by "simplest TWiki that works" you are not talking about taking things out of Core? Just Core without the plugins?

I can quite imagine that there are objections to that and putting together a TWikiFor packaging mechanism. We just need to keep the TWikiMission in mind...

-- ThomasWeigert - 07 Oct 2006

I think that my comment on the way Debian works was "Lost in the Noise".

It's quite easy, a no brainer and nearly zero effort to maintain TWO TWiki distributions:

• "Classic TWiki" o simply "TWiki": with the Core and the Core plugins.
• "Core TWiki": No plugins.

(names subject to change based on the comunity taste)

Using the current infrastructure (the way Crawford described), it's quite siple to create the TWiki Core distribution just after building and testing the main distribution.

-- RafaelAlvarez - 07 Oct 2006

Yep, Rafael, the bare bone package could be even created by a script from the TWiki package; as well as other packages. The point is to make sure that there are very stable releases of the standard package (e.g. less bugs than the current 4.0.x releases.)

-- PeterThoeny - 07 Oct 2006

OK, this is the quiet sound of me backing away slowly. Yes, I did mean removing things from the core, as my experience as a SoftwareEngineer and TechnicalArchitect tells me that I can do more by creating a smaller tighter and more generic core system.

-- SvenDowideit - 08 Oct 2006