Recommended Storage of Plugin Data
Introduction
Plugins sometimes need to store data. This can be plugin internal data such as cache data, or data generated for browser consumption such as images. Plugins should store data using
TWikiFuncDotPm functions that support saving and loading of topics and attachments.
Plugin Internal Data
You can create a plugin "work area" using the
TWiki::Func::getWorkArea()
function, which gives you a persistent directory where you can store data files. By default they will not be web accessible. The directory is guaranteed to exist, and to be writable by the webserver user. For convenience,
TWiki::Func::storeFile()
and
TWiki::Func::readFile()
are provided to persistently store and retrieve simple data in this area.
Web Accessible Data
Topic-specific data such as generated images can be stored in the topic's attachment area, which is web accessible. Use the
TWiki::Func::saveAttachment()
function to store the data.
Recommendation for file name:
- Prefix the filename with an underscore (the leading underscore avoids a name clash with files attached to the same topic)
- Identify where the attachment originated from, typically by including the plugin name in the file name
- Use only alphanumeric characters, underscores, dashes and periods to avoid platform dependency issues and URL issues
- Example:
_GaugePlugin_img123.gif
Web specific data can be stored in the plugin's attachment area, which is web accessible. Use the
TWiki::Func::saveAttachment()
function to store the data.
Recommendation for file names in plugin attachment area:
- Prefix the filename with an underscore
- Include the name of the web in the filename
- Use only alphanumeric characters, underscores, dashes and periods to avoid platform dependency issues and URL issues
- Example:
_Main_roundedge-ul.gif
Integrating with configure
Some TWiki extensions have setup requirements that are best integrated into
configure
rather than trying to use TWiki preferences variables. These extensions use
Config.spec
files to publish their configuration requirements.
Config.spec
files are read during TWiki configuration. Once a
Config.spec
has defined a configuration item, it is available for edit through the standard
configure
interface.
Config.spec
files are stored in the 'plugin directory' e.g.
lib/TWiki/Plugins/BathPlugin/Config.spec
.
Structure of a Config.spec
file
The
Config.spec
file for an extension starts with the extension announcing what it is:
# ---+ Extensions
# ---++ BathPlugin
# This plugin senses the level of water in your bath, and ensures the plug
# is not removed while the water is still warm.
This is followed by one or more configuration items. Each configuration item has a
type, a
description and a
default. For example:
# **SELECT Plastic,Rubber,Metal**
# Select the plug type
$TWiki::cfg{BathPlugin}{PlugType} = 'Plastic';
# **NUMBER**
# Enter the chain length in cm
$TWiki::cfg{BathPlugin}{ChainLength} = 30;
# **BOOLEAN EXPERT**
# Set this option to 0 to disable the water temperature alarm
$TWiki::cfg{BathPlugin}{TempSensorEnabled} = 1;
The type (e.g.
**SELECT**
) tells
configure
to how to prompt for the value. It also tells
configure
how to do some basic checking on the value you actually enter. All the comments between the type and the configuration item are taken as part of the description. The configuration item itself defines the default value for the configuration item. The above spec defines the configuration items
$TWiki::cfg{BathPlugin}{PlugType}
,
$TWiki::cfg{BathPlugin}{ChainLength}
, and
$TWiki::cfg{BathPlugin}{TempSensorEnabled}
for use in your plugin. For example,
if( $TWiki::cfg{BathPlugin}{TempSensorEnabled} && $curTemperature > 50 ) {
die "The bathwater is too hot for comfort";
}
The Config.spec file is read by
configure
, which then writes
LocalSite.cfg
with the values chosen by the local site admin.
A range of types are available for use in
Config.spec
files:
BOOLEAN |
A true/false value, represented as a checkbox |
COMMAND length |
A shell command |
LANGUAGE |
A language (selected from {LocalesDir} |
NUMBER |
A number |
OCTAL |
An octal number |
PASSWORD length |
A password (input is hidden) |
PATH length |
A file path |
PERL |
A perl structure, consisting of arrays and hashes |
REGEX length |
A perl regular expression |
SELECT choices |
Pick one of a range of choices |
SELECTCLASS root |
Select a perl package (class) |
STRING length |
A string |
URL length |
A url |
URLPATH length |
A relative URL path |
All types can be followed by a comma-separated list of
attributes.
EXPERT |
means this an expert option |
M |
means the setting is mandatory (may not be empty) |
H |
means the option is not visible in configure |
See
lib/TWiki.spec
for many more examples.
Config.spec
files for non-plugin extensions are stored under the
Contrib
directory instead of the
Plugins
directory.
Note that from TWiki 5.0 onwards,
CGI scripts (in the TWiki
bin
directory) provided by extensions must also have an entry in the
Config.spec
file. This entry looks like this (example taken from PublishContrib)
# **PERL H**
# Bin script registration - do not modify
$TWiki::cfg{SwitchBoard}{publish} = [ "TWiki::Contrib::Publish", "publish", { publishing => 1 } ];
PERL
specifies a perl data structure, and
H
a hidden setting (it won't appear in
configure
). The first field of the data value specifies the class where the function that implements the script can be found. The second field specifies the name of the function, which must be the same as the name of the script. The third parameter is a hash of initial context settings for the script.
TWiki:TWiki/SpecifyingConfigurationItemsForExtensions has supplemental documentation on configure settings.
Maintaining Plugins
Discussions and Feedback on Plugins
Each published plugin has a plugin development topic on TWiki.org. Plugin development topics are named after your plugin and end in
Dev
, such as
MyFirstPluginDev
. The plugin development topic is a great resource to discuss feature enhancements and to get feedback from the TWiki community.
Maintaining Compatibility with Earlier TWiki Versions
The plugin interface (
TWikiFuncDotPm functions and plugin handlers) evolve over time. TWiki introduces new API functions to address the needs of plugin authors. Plugins using unofficial TWiki internal functions may no longer work on a TWiki upgrade.
Organizations typically do not upgrade to the latest TWiki for many months. However, many administrators still would like to install the latest versions of a plugin on their older TWiki installation. This need is fulfilled if plugins are maintained in a compatible manner.
Tip: Plugins can be written to be compatible with older and newer TWiki releases. This can be done also for plugins using unofficial TWiki internal functions of an earlier release that no longer work on the latest TWiki codebase.
Here is an example; the
TWiki:TWiki.TWikiPluginsSupplement#MaintainPlugins has more details.
if( $TWiki::Plugins::VERSION >= 1.1 ) {
@webs = TWiki::Func::getListOfWebs( 'user,public' );
} else {
@webs = TWiki::Func::getPublicWebList( );
}
-
(above text included from
TWikiPlugins
--
PeterThoeny - 16 Oct 2003
Discussions
Background: At my workplace we have several inhouse-use Plugins where data needed to be stored. Some data is cache data, some generated data needs to be accessed by the browser. We came up with the Plugins directory and topic attachment directory solution since it is automatically recognized by the system. For example, if you rename a topic, the Plugin data gets moved correctly.
Two Plugins are already using this convention:
GaugePlugin and
ChartPlugin. The Gauge Plugin stores image files in the attachment directory of topics that have
%GAUGE{...}%
variables, e.g. URL
%PUBURL%/Web/TopicName/_GaugePlugin_tambar_g1.gif
The Plugin API could/should be enhanced to support data storage of Plugins.
Above recommendation should me moved into the
TWikiPlugins documentation once we finalize it. (and INCLUDE it here)
Feedback is appreciated.
--
PeterThoeny - 16 Oct 2003
- In regards to The leading underscore avoids a nameclash with files attached to the Plugin topic
--
MartinCleaver - 16 Oct 2003
Seems nice. One clarification: is the term
Installweb
you use the actual name of this "web for generated data", or
Plugins
?
Shouldn't we use rather
%PUBURL/_plugins/FooBarPlugin
to show that this web is outside the normal attachements scope?
Otherwise this could wreck havoc on efforts to have repair scripts to rebuild/generate attachements metadata, that could prove useful in the future (I toyed with this idea in order to provide access via a wiki frontend to external dirs).
--
ColasNahaboo - 16 Oct 2003
The term
Installweb
refers to the name of the web where the Plugin is installed. The Plugin data lives in the attachment space, e.g. the Plugin can output
"%PUBURL%/$installWeb/FooBarPlugin/_somedata"
as the URL for its data. This ensures that the Plugin can be installed in the TWiki web, Plugins web or any other web.
I moved the doc to the
TWikiPlugins topic and include it here at the beginning of this topic.
--
PeterThoeny - 15 Dec 2003
The recommendations are good, but ..... in the
FormQueryPlugin I create a cache; but it's a
web-specific cache, not
topic specific. I currently store it in the web's data directory, as
<data>/<web>/FormQueryCache
. I
could move it to
<pub>/<web>/_FormQueryCache.bin
, I suppose, to be consistent with the recommendations. Can we cover this case in the doc as well, please? Something like:
Where to Store Data for Webs using the Plugin
In the case where the Plugin generates data which is specific to a web, store it in the web's attachment directory.
- The web's attachment directory is pubdir/Webname
- Data files should be named as for topic-specific data
--
CrawfordCurrie - 15 Dec 2003
We are headed for trouble if extensions can put stuff in
pub/web/_anyname/
- what if there are two plugins that perform blogging and they both choose
pub/web/_blog/
?
The convention should be
pub/web/_PluginName/
- the plugin is deemed to exclusively own that space.
Further, this is an plugin package archive convention only - once the code is installed it should use an API to discover where TWiki actually keeps this information. This will allow TWiki's designers to move the stuff according to the conventions of the store into which the extension is unpacked.
--
MartinCleaver - 21 Nov 2005
That's quite true, and is one of the reasons why the Func interface now (Dakar) supports getWorkArea:
getWorkArea( $pluginName ) -> $directorypath
Gets a private directory for the plugin. The plugin is entirely responsible
for managing this directory; TWiki will not read from it, or write to it.
The directory is guaranteed to exist, and to be writable by the webserver
user. By default it will
not be web accessible.
The directory and its contents are permanent, so plugins must be careful
to keep their areas tidy.
--
CrawfordCurrie - 21 Nov 2005