I am just trying to get all the Wiki features working on my 'Pure' WinNT port, using VSS ect and realised that having a series of unit test would ensure that I got every thing working...

For the markup tests all we would need would be a perl script that creates a page with all variations of wiki markup, and then views it and compares to a benchmark... While for the rest we would need omre scripts to check diffs, get versions etc..

(famous last words - I don't have time right now - I am doing this as a rapid hack job to transition from JOSWiki )

-- SvenDowideit - 25 Oct 2000

I fully support your idea. Some automated testing would increase the quality of a release; it also would speed up the time for testing.

-- PeterThoeny - 27 Oct 2000

mmm i think i need this even more now - I don't know all of twiki's features, and don't expect ever to... so i won't know if my changes break anything unless there I have a TestSuite...

How's this for a spec? (I am playing with the idea on my twiki..)

• a set of source texts
  • i think one per feature - with possible exclusion markers to allow docco for that feature
  • OR we could use UnitTest markers on a feature's docco page, and only those bit would be used in the test?
• a corresponding set of correct answers to compare to (easiest to use a copy of the html stream?)
• an intelligent diff to pass or fail
• the output page.

eg. to test this Wiki goto https://twiki.org/cgi-bin/unittest

and then you get an output of complience to the tests...

behind the scenes:

• the source texts can be in the UnitTest wiki and the answers can be ...?? uploaded to the pub area as attachments either to the test page, or to one AnswersPage (i think the second is better otherwise having a single test for attachments becomes a SpecialCase )

-- SvenDowideit - 30 Dec 2000

I am currently writing a UnitTestScript, but am missing one thing.. a good Diff. I don't want to use a module that is not part of the Perl distribution (unless if i have to...) and seeing as I am developing under win95 ....

any ideas (i am going to get a command line diff initially)

---

Try http://search.cpan.org/search?dist=Algorithm-Diff, also see http://language.perl.com/ppt/src/diff/index.html and http://language.perl.com/ppt/src/patch/index.html.

Do you intend to use something like http://search.cpan.org/doc/MSCHWERN/Test-1.16/Test.pm? Would be useful addition too a CPAN package end product.

-- NicholasLee - 27 Feb 2001

more news - I am fixing up what i have done so that it can be integrated into the modular TWiki source (I did a cvs get last night). I will have a look at the CPAN packages later, and I also need to consider if this should be a Plugin, rather thaan a standard component - comments please..

-- SvenDowideit - 27 Feb 2001

Initially I suggest a plugin, that way it is easier to "test the test suite" It can be incorporated later into the core. (Which it should)

Just a brainstorming idea, I don't know the spec of your test suite. How about testing the TextFormattingRules by using a TWikiTestTextFormattingRules topic? It could be structured i.e. sets like this: (as entered)

   * CMD: Italic
   * EDIT: _test test_, _test_; _test_: _test_.
   * VIEW: <EM>test test</EM>, <EM>test</EM>; <EM>test</EM>: <EM>test</EM>.

Do that for each command you want to test. CMD is the command to test, EDIT is what you enter, VIEW is what you should get. The test suite could use read the topic, render it and compare the result. If failed, CMD goes into the report.

-- PeterThoeny - 01 Mar 2001

Its obviously time to spill somedetails of what i have so far.. I was initially wanting to test the rendering engine. For that all I did was to provide ready made pages and canned result pages that the script would compare. I agree that we will need an action test method too, but i wanted to seperate them a bit.

• now I have to find out how the plugins work

--Main.SvenDowideit -01 March 2001

I've just attached a few unit tests for Store.pm. I found them useful when first writing it and in getting them going again a discovered a few more problems. The idea of this type of test would be to exercise at the library level. Still good to have http based tests as well. I suggest these go in CVS under \tools\test or similar.

-- JohnTalintyre - 27 Mar 2001

I've commited an updated version of unittests for Store.pm to CVS. See http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/twiki/twiki/tools/test/. Run be executing unittest, failures are clearly displayed. Do unittest -trace to see individual tests being run.

A small beginning, but, a useful way of doing tests at the "back-end" level i.e. tests Store.pm, not save/rename etc. Tests at http level would also be good.

-- JohnTalintyre - 22 May 2001

I saw the complaints about the acceptance of unit testing in TestsDoNotCompile, and I agree with them. I also agree with CrawfordCurrie that "stubbing" the interfaces between modules will let us better test them in isolation, and encourage decoupling. Here's my idea: have a phantom TWiki::Test::Harness package that will define the stubs in testing. When a non-trivial call between modules needs to be made (e.g. to read a topic), first a check for the appropriate override in TWiki::Test::Harness is made. So this:

   ($meta, $text) = TWiki::Store::readTopic( $theWeb, $theTopic, 1);

becomes this:

   if (defined &TWiki::Test::Harness::readTopic) {
      ($meta, $text) = TWiki::Test::Harness::readTopic($theWeb, $theTopic, 1);
   } else {
      ($meta, $text) = TWiki::Store::readTopic( $theWeb, $theTopic, 1);
   }

In a running TWiki environment the TWiki::Test::Harness doesn't exist. (Well, by Perl semantics, it does exist but is empty, like all packages with valid names that aren't filled in somewhere.) When a unit test needs to override something, it defines the test harness handler with sub TWiki::Test::Harness::<handler> {} syntax.

This is rather cumbersome; we could put the harness check inside the interface calls, but then we'd probably want internal mirror functions without the check so that intramodular calls don't get shunted into the test harness.

-- WalterMundt - 05 Mar 2004

Too cumbersome, Walter. It'll never get acceptance.

I managed stubbing by defining Fixture modules (built using Test::Unit) that could be selectively 'use'd in the test code, thus (example from CommentPlugin):

package CommentTest;

use TWiki::Plugins::CommentPlugin::Comment;
require 'FuncFixture.pm';
require 'StoreFixture.pm';
import TWiki::Func;
import TWiki::Store;

FuncFixture.pm contains:

package TWiki::Func;

use base qw(Test::Unit::TestCase); #to get access to assert, no other reason

Even though the CUT (code under test) use's TWiki::Store.pm and TWiki::Func.pm, and testrunner.pl is called with -I../../../../../lib, the symbols are already defined by the require and import in the test case. Seems to work OK for my limited needs (plugin using Func and Store interfaces)

-- CrawfordCurrie - 05 Mar 2004

I see. And upon consideration, it occurs to me that making use of CPAN modules might be a good idea. Since running the unit tests is certainly not required to operate a TWiki, we shouldn't hesitate to use anything that might be useful to set up the testing. And your method just might do the trick, although a perhaps more selective method would be to 'use' all the code, turn off warnings, and redefine any 'fixture' subs you wish to. E.g. use "*TWiki::Store::readTopic = \&myReadTopicStub" to redirect readTopic calls to a stub after everything is read in.

-- WalterMundt - 05 Mar 2004

The problem with the "load it all" approach is that a lot of TWiki is side-effecting, so you get unwanted effects if you do that. By selectively loading only those modules you are using, you control the splat a bit.

-- CrawfordCurrie - 05 Mar 2004

See TWikiTestInfrastructure for the continuation of this topic. There is a unit test suite in place, in the DEVELOP branch.