create new tag
, view all tags

Feature Proposals » Automation add-on


Current State: Developer: Reason: Date: Concerns By: Bug Tracking: Proposed For:
ImplementedAsExtension HideyoImazu None 2014-05-08     KampalaRelease

Edit Form

DateOfCommitment:   Format: YYYY-MM-DD


There are cases where you want to do some TWiki operation automatically. This is about user level operations usually conducted manually through a web browser such as:

  • rename a topic within the same web or to a different web
  • create a new topic
  • set TWikiForm field values

This is an idea for an add-on, not a core enhancement. So it's not required to be discussed before implementation, I suppose. Still, getting feedback at an early stage should be valuable. Hence I'm writing this.

Description and Documentation

The add-on allows TWiki users to register automated tasks.

This is somewhat related to TasksFramework and CronCentre (implementation in progress: SchedulerContrib), but quite different in scope. This add-on can do the same thing as RunTopicsAddOn. So in a sense, this is a generalized and liberalized (in a sense that not requiring command line access) version of RunTopicsAddOn.

Aspect AutomationAddOn TasksFramework and CronCentre RunTopicsAddOn
Who uses it? TWikiAdminGroup members TWiki admin having access to the server command line
It is similar to bin/statistics cron bin/view
What does it do? rename, save, and view operations (as of now. may do more in the future) as specified by users calls specified Perl functions specified by TWiki admin views specified topics to refresh their cached versions

Use case 1: archiving topics

You have a bulletin board web, on which each article is a topic, and a list of articles is shown by %SEARCH{...}%. It's a high traffic bulletin board, so it accumulates a lot of articles and gets sluggish.

To keep its responsiveness at a certain level, you want to archive old articles to a subweb. Specifically, articles having no update for the past 180 days will be archived. And you want to do it automatically rather than manually.

Use case 2: creating topics

You have a web of your project team, in which there are weekly report topics. A team weekly report %INCLUDE{...}% members' weekly reports so that all members' reports can be seen at a glance. Every week, a new team weekly report page needs to be created based on the member list. You want to automate it.

Use case 3: refresh cached topics

You are using VarCachePlugin and you have topics taking long to be rendered. So you want to have those pages "view"ed periodically. RunTopicsAddOn can do it, but this add-on can it too.


This add-on introduces a script named automation. It's somewhat like the statistics script in the sense that automation and statistics can be used both interactively and in a batch job. When they are invoked from a web browser, they do their jobs for the current web, and when they are invoked from a command line, they traverses all webs and do their jobs for each web.

The automation script is intended to be invoked daily from cron or other similar mechanism. It walks through all webs including subwebs and see if a WebAutomation topic exists. If it does, automation executes automation tasks described on WebAutomation.

If bin/automation is invoked interactively from a web browser, it refers to WebAutomation of the current web and execute tasks in the topic.


A WebAutomation topic has one or more tables written in TWiki markup. On the bulletin board web mentioned above, it would look like:

Operation Topic Parameter
rename Article2014x05x08x00 newweb=BulletinBoard/Archive
rename Article2014x05x08x01 newweb=BulletinBoard/Archive

On the team web mentioned above, it would look like:

Operation Topic Parameter
save Report2014x05x08 templatetopic=ReportTemplate

On a web having heavy-to-render topics, it would look like:

Operation Topic
view HeavyTopic1
view HeavyTopic2

You may think that if tables need to be edited manually, it's not automation. That's true. So those tables tend to be dynamic.

The first one would be the result of:

%SEARCH{"d2n(info.date) < d2n('%DATETIME{"$year-$mo-$day" incdays="-180"}%')"
 header="| *Operation* | *Topic* | *Parameter* |"
 format="| rename | $topic | newweb=%WEB%/Archive |"

The second one would be the result of:

| *Operation* | *Topic* | *Parameter* |
%IF{"'%GMTIME{$wday}%' = 'Fri'" then="| save | Report%GMTIME{$yearx$mox$day}% | templatetopic=ReportTemplate |"}%

The automation script expands TWiki variables on WebAutomation before picking up tasks from tables.

As you see, the third and following columns are for parameters -- one column for one parameter.

As of this writing, only operations intended to be implemented are "rename", "save", and "view". There parameters are same the corresponding scripts.

Not to step on others' toes

Before executing an operation, the topic name and the parameters are sanitized to prevent you from bothering others. Specifically:

  • The topic must not contain slash (/) or dot (.). If it does, the first character of the topic name to the last occurrence of slash or dot are removed
  • The templatetopic and newtopic parameter values are the same as the topic value
  • The newweb parameter value must share the top level web as the current web or Trash. UserSubwebs are exceptions. They are technically second level webs, but they are treated as if they were top level. For example, you cannot move a topic from Main/JoeSchmoe to Main/JaneDoe even though they share the top level web "Main".

Access control implication

When executed in batch, the automation script runs with the administrator privilege. There is no other practical ways.

  • If it runs as the guest user, it cannot do anything interesting -- new topic creation, topic renaming, both needs a real user
  • If it runs as the user who last edited, an automated task can stop working either by
    • WebAutomation is edited by somebody else
    • The permission of the last edit user changes
    • The last edit user is removed from the TWiki's user registration

This leads to that WebAutomation needs to be readable and writable only by privileged users. For this, an enhancement to TWiki::Access::checkAccessPermission() such as this is required.



WhatDoesItAffect: Plugins


-- Contributors: Hideyo Imazu - 2014-05-08


Scheduled tasks that are configurable in TWiki topics is definitely a feature that helps strengthen TWiki as an app platform.

We had discussions and alpha code in the past, see TasksFramework, CronCentre, SchedulerContrib.

I'll provide more feedback...

-- Peter Thoeny - 2014-05-09

Thank you for the feedback, Peter. I clarified the initial part of the proposal. So please take a look.

-- Hideyo Imazu - 2014-05-10

I see a useful case to archive reports in regular intervals, such as a weekly status report that takes a snapshot of a dynamic report that pulls data from multiple status report topics.

-- Peter Thoeny - 2014-05-15

I've realized that this add-on can provide the feature provided by RunTopicsAddOn. So I added it.

-- Hideyo Imazu - 2014-07-04

Also related is the relatively new ExecutePlugin.

-- Peter Thoeny - 2016-01-28

Edit | Attach | Watch | Print version | History: r11 < r10 < r9 < r8 < r7 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r11 - 2016-01-28 - PeterThoeny
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.