create new tag
, view all tags

Recommended Storage of Plugin Data


Plugins sometimes need to store data. This can be plugin internal data such as cache data, or data generated for browser consumption such as images. Plugins should store data using TWikiFuncDotPm functions that support saving and loading of topics and attachments.

Plugin Internal Data

You can create a plugin "work area" using the TWiki::Func::getWorkArea() function, which gives you a persistent directory where you can store data files. By default they will not be web accessible. The directory is guaranteed to exist, and to be writable by the webserver user. For convenience, TWiki::Func::storeFile() and TWiki::Func::readFile() are provided to persistently store and retrieve simple data in this area.

Web Accessible Data

Topic-specific data such as generated images can be stored in the topic's attachment area, which is web accessible. Use the TWiki::Func::saveAttachment() function to store the data.

Recommendation for file name:

  • Prefix the filename with an underscore (the leading underscore avoids a name clash with files attached to the same topic)
  • Identify where the attachment originated from, typically by including the plugin name in the file name
  • Use only alphanumeric characters, underscores, dashes and periods to avoid platform dependency issues and URL issues
  • Example: _GaugePlugin_img123.gif

Web specific data can be stored in the plugin's attachment area, which is web accessible. Use the TWiki::Func::saveAttachment() function to store the data.

Recommendation for file names in plugin attachment area:

  • Prefix the filename with an underscore
  • Include the name of the web in the filename
  • Use only alphanumeric characters, underscores, dashes and periods to avoid platform dependency issues and URL issues
  • Example: _Main_roundedge-ul.gif

Integrating with configure

Some TWiki extensions have setup requirements that are best integrated into configure rather than trying to use TWiki preferences variables. These extensions use Config.spec files to publish their configuration requirements.

Config.spec files are read during TWiki configuration. Once a Config.spec has defined a configuration item, it is available for edit through the standard configure interface. Config.spec files are stored in the 'plugin directory' e.g. lib/TWiki/Plugins/BathPlugin/Config.spec.

Structure of a Config.spec file

The Config.spec file for an extension starts with the extension announcing what it is:
# ---+ Extensions
# ---++ BathPlugin
# This plugin senses the level of water in your bath, and ensures the plug
# is not removed while the water is still warm.
This is followed by one or more configuration items. Each configuration item has a type, a description and a default. For example:
# **SELECT Plastic,Rubber,Metal**
# Select the plug type
$TWiki::cfg{BathPlugin}{PlugType} = 'Plastic';

# **NUMBER**
# Enter the chain length in cm
$TWiki::cfg{BathPlugin}{ChainLength} = 30;

# Set this option to 0 to disable the water temperature alarm
$TWiki::cfg{BathPlugin}{TempSensorEnabled} = 1;
The type (e.g. **SELECT** ) tells configure to how to prompt for the value. It also tells configure how to do some basic checking on the value you actually enter. All the comments between the type and the configuration item are taken as part of the description. The configuration item itself defines the default value for the configuration item. The above spec defines the configuration items $TWiki::cfg{BathPlugin}{PlugType}, $TWiki::cfg{BathPlugin}{ChainLength}, and $TWiki::cfg{BathPlugin}{TempSensorEnabled} for use in your plugin. For example,
if( $TWiki::cfg{BathPlugin}{TempSensorEnabled} && $curTemperature > 50 ) {
    die "The bathwater is too hot for comfort";

The Config.spec file is read by configure, which then writes LocalSite.cfg with the values chosen by the local site admin.

A range of types are available for use in Config.spec files:

BOOLEAN A true/false value, represented as a checkbox
COMMAND length A shell command
LANGUAGE A language (selected from {LocalesDir}
NUMBER A number
OCTAL An octal number
PASSWORD length A password (input is hidden)
PATH length A file path
PERL A perl structure, consisting of arrays and hashes
REGEX length A perl regular expression
SELECT choices Pick one of a range of choices
SELECTCLASS root Select a perl package (class)
STRING length A string
URL length A url
URLPATH length A relative URL path

All types can be followed by a comma-separated list of attributes.

EXPERT means this an expert option
M means the setting is mandatory (may not be empty)
H means the option is not visible in configure

See lib/TWiki.spec for many more examples.

Config.spec files for non-plugin extensions are stored under the Contrib directory instead of the Plugins directory.

Note that from TWiki 5.0 onwards, CGI scripts (in the TWiki bin directory) provided by extensions must also have an entry in the Config.spec file. This entry looks like this (example taken from PublishContrib)

# **PERL H**
# Bin script registration - do not modify
$TWiki::cfg{SwitchBoard}{publish} = [ "TWiki::Contrib::Publish", "publish", { publishing => 1 } ];
PERL specifies a perl data structure, and H a hidden setting (it won't appear in configure). The first field of the data value specifies the class where the function that implements the script can be found. The second field specifies the name of the function, which must be the same as the name of the script. The third parameter is a hash of initial context settings for the script.

TWiki:TWiki/SpecifyingConfigurationItemsForExtensions has supplemental documentation on configure settings.

Maintaining Plugins

Discussions and Feedback on Plugins

Each published plugin has a plugin development topic on TWiki.org. Plugin development topics are named after your plugin and end in Dev, such as MyFirstPluginDev. The plugin development topic is a great resource to discuss feature enhancements and to get feedback from the TWiki community.

Maintaining Compatibility with Earlier TWiki Versions

The plugin interface (TWikiFuncDotPm functions and plugin handlers) evolve over time. TWiki introduces new API functions to address the needs of plugin authors. Plugins using unofficial TWiki internal functions may no longer work on a TWiki upgrade.

Organizations typically do not upgrade to the latest TWiki for many months. However, many administrators still would like to install the latest versions of a plugin on their older TWiki installation. This need is fulfilled if plugins are maintained in a compatible manner.

TIP Tip: Plugins can be written to be compatible with older and newer TWiki releases. This can be done also for plugins using unofficial TWiki internal functions of an earlier release that no longer work on the latest TWiki codebase. Here is an example; the TWiki:TWiki.TWikiPluginsSupplement#MaintainPlugins has more details.

    if( $TWiki::Plugins::VERSION >= 1.1 ) {
        @webs = TWiki::Func::getListOfWebs( 'user,public' );
    } else {
        @webs = TWiki::Func::getPublicWebList( );


(above text included from TWikiPlugins#RecommendedStorageOfPluginData)

-- PeterThoeny - 16 Oct 2003


Background: At my workplace we have several inhouse-use Plugins where data needed to be stored. Some data is cache data, some generated data needs to be accessed by the browser. We came up with the Plugins directory and topic attachment directory solution since it is automatically recognized by the system. For example, if you rename a topic, the Plugin data gets moved correctly.

Two Plugins are already using this convention: GaugePlugin and ChartPlugin. The Gauge Plugin stores image files in the attachment directory of topics that have %GAUGE{...}% variables, e.g. URL %PUBURL%/Web/TopicName/_GaugePlugin_tambar_g1.gif

The Plugin API could/should be enhanced to support data storage of Plugins.

Above recommendation should me moved into the TWikiPlugins documentation once we finalize it. (and INCLUDE it here)

Feedback is appreciated.

-- PeterThoeny - 16 Oct 2003

  • In regards to The leading underscore avoids a nameclash with files attached to the Plugin topic
    • avoid, yes, prevent no.

-- MartinCleaver - 16 Oct 2003

Seems nice. One clarification: is the term Installweb you use the actual name of this "web for generated data", or Plugins ? Shouldn't we use rather %PUBURL/_plugins/FooBarPlugin to show that this web is outside the normal attachements scope? Otherwise this could wreck havoc on efforts to have repair scripts to rebuild/generate attachements metadata, that could prove useful in the future (I toyed with this idea in order to provide access via a wiki frontend to external dirs).

-- ColasNahaboo - 16 Oct 2003

The term Installweb refers to the name of the web where the Plugin is installed. The Plugin data lives in the attachment space, e.g. the Plugin can output "%PUBURL%/$installWeb/FooBarPlugin/_somedata" as the URL for its data. This ensures that the Plugin can be installed in the TWiki web, Plugins web or any other web.

I moved the doc to the TWikiPlugins topic and include it here at the beginning of this topic.

-- PeterThoeny - 15 Dec 2003

The recommendations are good, but ..... in the FormQueryPlugin I create a cache; but it's a web-specific cache, not topic specific. I currently store it in the web's data directory, as <data>/<web>/FormQueryCache. I could move it to <pub>/<web>/_FormQueryCache.bin, I suppose, to be consistent with the recommendations. Can we cover this case in the doc as well, please? Something like:

Where to Store Data for Webs using the Plugin

In the case where the Plugin generates data which is specific to a web, store it in the web's attachment directory.
  • The web's attachment directory is pubdir/Webname
  • Data files should be named as for topic-specific data

-- CrawfordCurrie - 15 Dec 2003

We are headed for trouble if extensions can put stuff in pub/web/_anyname/ - what if there are two plugins that perform blogging and they both choose pub/web/_blog/?

The convention should be pub/web/_PluginName/ - the plugin is deemed to exclusively own that space.

Further, this is an plugin package archive convention only - once the code is installed it should use an API to discover where TWiki actually keeps this information. This will allow TWiki's designers to move the stuff according to the conventions of the store into which the extension is unpacked.

-- MartinCleaver - 21 Nov 2005

That's quite true, and is one of the reasons why the Func interface now (Dakar) supports getWorkArea:

getWorkArea( $pluginName ) -> $directorypath

Gets a private directory for the plugin. The plugin is entirely responsible for managing this directory; TWiki will not read from it, or write to it.

The directory is guaranteed to exist, and to be writable by the webserver user. By default it will not be web accessible.

The directory and its contents are permanent, so plugins must be careful to keep their areas tidy.

-- CrawfordCurrie - 21 Nov 2005

Edit | Attach | Watch | Print version | History: r10 < r9 < r8 < r7 < r6 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r10 - 2005-11-21 - WillNorris
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2015 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.