Feature Proposal: Automated Testing Framework

Motivation

This could help to improve the quality of TWiki base releases, plugins, and user pages

Description and Documentation

An automated testing framework could work in the background, testing all new pages for HTML and other errors. Think of it as "continuous integration" for TWiki sites.

It could keep track of the pages it has checked (including revision dates), so as not to recheck pages unnecessarily. However, there should be an override mode for use when (say) a new plugin has been added.

There are two basic (and complementary) testing modes to consider:

file-based testing

In this mode, the program would walk through the "data" tree, looking at the TWiki source files. This could look for problems in TWiki markup, missing enclosures, etc.

The (Perl-based) SWISH-E library (http://swish-e.org/) might be useful for some of this, as it already understands file-based web site crawling.

page-based testing

In this mode, the program would walk through the walk through the TWiki URLs, looking at rendered pages. There are a number of available tools for checking the validity of HTML (eg, from W3C). Link checking would be another valuable feature.

The Selenium suite (http://selenium.openqa.org/) might be a good starting point, as it allows the automated use of Firefox to simulate user interactions, etc. Given that Firefox is able to perform assorted checks on pages, this combination might be able to run a wide variety of tests.

Examples

Impact

Implementation

-- Contributors: RichMorin - 13 Aug 2008

Discussion

Not quite sure what you are proposing here. What exactly is it testing for? And how does it know when it finds an error?

Note that there are already two testing frameworks in place; unit tests, and the Test Cases web. Both suffer from a lack of maintenance, and there's a risk that the framework you are proposing here would suffer in the same way.

-- CrawfordCurrie - 13 Aug 2008

Is this proposal only to check that the generated HTML is correct?

-- RafaelAlvarez - 13 Aug 2008

and to add to that, the unit tests and the builds are done nightly - see

• trunk
• 4.2.0 branch

the build scripts do some checking, and I was adding perlcritic and other checking tools - it seems to me that the is 'already implemented' but we would love to have more people writing unit tests - which we use for integration style testing of markup rendering too.

I did add a HTML validator to the testing system at one stage, I'm not sure where it is at now though.

-- SvenDowideit - 13 Aug 2008

I wrote up a fairly detailed weblog entry on this topic: Continuous Integration Support for Wikis

-- RichMorin - 2009-11-09