create new tag
, view all tags

Feature Proposal: TWiki Doc Graphics: Add search path, icon size defined in topics, add cache


Sites typically have some customized icons, especially those with TWikiApplications. Upgrading TWiki is easier if the user supplied images are separated from the TWiki system images.

This is consistent with TWiki.TWikiPreferences / Main.TWikiPreferences handling. It could be applied to other things as well, such as the custom templates of the CommentPlugin.

Description and Documentation

The Main.SiteDocGraphics (site topic) complements the existing TWiki.TWikiDocGraphics (system topic.)

  • Site admins can attach doc graphics to the site topic in the Main web
  • Calling %ICON{}% searches the icon first in the site topic, then in the system topic (that way it is possible to overload system images)
  • The ICONTOPIC preferences setting lists doc graphic topics, by default there is: TWiki.TWikiDocGraphics, Main.SiteDocGraphics
  • Caching is implemented for speed; cache invalidation is done at topic save time of preferences topics and doc graphic topics listed in ICONTOPIC

The tables listing the doc graphics define the images with description, type, size and aliases. Example:

  Variable Description Type Size Alias
Airplane %ICON{airplane}% Airplane gif 16x16  
Award %ICON{award}% Award gif 16x16  
Barcode %ICON{barcode}% Barcode gif 16x16  
JPEG %ICON{jpg}% JPEG gif 16x16 jfif, jpe, jpeg
Compressed Zip archive %ICON{zip}% Compressed Zip archive gif 16x16 arc, arj, b64, bhx, cab, hqx, lzh, mim, tar, taz, tgz, tz, z

For each icon, the name, description, type, size, and aliases are cached for performance. Using this cache, the ICON variable generates the proper img tag including the size. That is, it is no longer needed to manually construct an img tag if the size is not 16x16 pixel.

A _filetypes.txt file was earlier used to map from alternate file types to an existing image file. This is replaced by an "Alias" field in the icon table that lists aliases of icons (if any).

TWiki already handles an undocumented ICONTOPIC setting to point to the %SYSTEMWEB%.TWikiDocGraphics. This setting is extended to support multiple topic names. Topics may overload icons defined in previous topic(s). Default new setting is as follows:

   * Set ICONTOPIC = %SYSTEMWEB%.TWikiDocGraphics, %USERSWEB%.SiteDocGraphics

TWiki does not ship with a Main.SiteDocGraphics topic. The TWiki06x00.TWikiDocGraphics documents how to create a topic in the Main web. A conditional link is provided to create the SiteDocGraphics topic based on a TWikiDocGraphicsTemplate topic in case it does not exist.

The ICON variable handles a new format="" parameter. It understands these variables (with example):

  • $name: Name of icon ('home')
  • $type: Type of icon ('gif')
  • $file: Icon filename ('home.gif')
  • $web: Web where icon is located ('TWiki')
  • $topic: Topic where icon is located ('TWikiDocGraphics')
  • $description: Icon description ('Home')
  • $width: Width of icon ('16')
  • $height: Height of icon ('16')
  • $img: Full img tag of icon ('<img src="/pub/TWiki/TWikiDocGraphics/home.gif" ... />')
  • $info: Img tag with usage info in title
  • $url: URL of icon ('http://example.com/pub/TWiki/TWikiDocGraphics/home.gif')
  • $urlpath: URL path of icon ('/pub/TWiki/TWikiDocGraphics/home.gif')

The ICON, ICONURL and ICONURLPATH variable handle a new default="" parameter to define a default icon name in case the icon is not found. This is optional; if not specified, the icon is assumed to exist in the default location, which is the second entry in the ICONTOPIC (if it exists, else the first), e.g. Main.SiteDocGraphics.

A list of icons or icon names is shown with special token %ICON{list:...}%:

Format: List all icons,
including aliases:
List unique icons only: Example:
Name list,
comma-space delimited:
%ICON{list:all-names}% %ICON{list:unique-names}% addon, air, airplane, ...
Icon list: %ICON{list:all-icons}% %ICON{list:unique-icons}% Add-on Adobe Air Airplane...
Info list,
icons with usage info:
%ICON{list:all-info}% %ICON{list:unique-info}% addon air airplane...
Detailed list,
table format:
%ICON{list:all-table}% %ICON{list:unique-table}% (similar to table format above)

Since there is more than one icon topic we have an "All icons at a glance" section at the top. Screenshot:



Caching is implemented to deliver the icons quickly. LetICONCalculateImageSize proposes to calculate the size. It is much easier to let the icon authors document the size in the icon table, and use that size information instead.

The cache contains records; each record contains info of an image:

  • Name of image without suffix (e.g. rss-feed)
  • Type of image (e.g. gif)
  • Name of web where image file is (e.g. TWiki)
  • Name of topic where image file is (e.g. TWikiDocGraphics)
  • Image width (e.g. 36)
  • Image height (e.g. 14)
  • Image description (e.g. RSS feed)
  • Aliases point to the defining icon

For performance, cache invalidation is done at topic save time:

Cache is created and used at topic rendering time:

  • For each ICON rendering:
    • On first ICON encountered on a topic view / topic rdiff:
      • Rebuild cache file in case it does not exist (e.g. if invalidated)
        • For each topic listed in ICONTOPIC:
          • Parse table for valid rows starting with | %ICON{...
            • Add icon record to cache (replace if already exist)
      • Read cache into memory
    • Render ICON based on cache in memory
      • If specified name is missing in cache, create a default icon assuming 16x16 size and the location of the second topic in ICONTOPIC. For example, %ICON{qwerty}% creates an img tag pointing to qwerty.gif in Main.SiteDocGraphics, description "Qwerty", size 16x16.

Cache file format

The cache is text based. There are three types of lines:

  • Comments: Starting with # hash
  • Icon definition, format: hash key, name, web, topic, file type, width, height, description
  • Alias, format: from name => to hash key

File path: twiki/working/work_areas/VarICON/icon_cache.txt


# Cached icon data; do not edit. See TWiki.TWikiDocGraphics
_default: _default, Main, SiteDocGraphics, gif, 16, 16, Default
abs: wav, TWiki, TWikiDocGraphics, gif, 16, 16, Waveform sound file
addon: addon, TWiki, TWikiDocGraphics, gif, 16, 16, Add-on
air: air, TWiki, TWikiDocGraphics, gif, 16, 16, Adobe Air
airplane: airplane, TWiki, TWikiDocGraphics, gif, 16, 16, Airplane
xla: xls, TWiki, TWikiDocGraphics, gif, 16, 16, Microsoft Excel Spreadsheet
xml: xml, TWiki, TWikiDocGraphics, gif, 16, 16, XML
xml-feed: xml-feed, TWiki, TWikiDocGraphics, gif, 36, 14, XML feed
xml-small: xml-small, TWiki, TWikiDocGraphics, gif, 16, 16, XML feed
xsl: xsl, TWiki, TWikiDocGraphics, gif, 16, 16, XSL (XML style sheet)
ying-yang: ying-yang, TWiki, TWikiDocGraphics, gif, 16, 16, Ying Yang
aif => abs
aiff => abs
arj => arc
au => abs
xlt => xla
xlw => xla
z => arc
zip => arc


I did some performance tests using CPAN:Time::HiRes. Numbers are average times of a few runs:

Action Old spec Using Data::Dumper-
based cache
Using homegrown
Cache rebuild N/A 42.0 ms 21.0 ms
Read cache (or _filetypes.txt) 1.8 ms 19.6 ms 8.8 ms
Read from memory (after 1st icon) 0.34 ms 0.12 ms 0.13 ms
Time to render 10 icons on a page 5.2 ms 20.8 ms 10.1 ms

The first cache implementation did not perform so well. We have now acceptable performance after implementing a homegrown cache format.

How performance is measured:

    use Time::HiRes;
    my $t1 = Time::HiRes::gettimeofday();
    # ...statements under scrutiny here...
    my $t2 = Time::HiRes::gettimeofday();
    my $diff = sprintf( '%.6f', ($t2 - $t1) * 1000 );
    print STDERR "HiRes time diff: $diff ms\n";

-- Contributors: PeterThoeny - 2009-12-23


I had this idea while working on a TWikiApplication for a client.

-- PeterThoeny - 11 Feb 2006

Using an arbitrary topic for icon attachments is already implemented in ICONTOPIC. Somehow that variable is not in the current TWikiVariables (Bugs:Item1631). But overloading is not yet written.

It would also be useful if icons from plugins could be overloaded this way. With a x number of plugins you get very different styles that are hard to control.

See also the related LetICONCalculateImageSize.

-- ArthurClemens - 11 Feb 2006

Yes, LetICONCalculateImageSize is related, and could be implemented as part of this change request.

I think the ICONTOPIC variable is a bit overkill, we removed the use because we merged the pattern skin graphics into the default image library. So, better to remove the ICONTOPIC variable.

Alternatively, we could introduce a new ICONTOPICSEARCHPATH where you can list a number of icon topics to search for icons. But this sounds also like a bit overkill.

-- PeterThoeny - 11 Feb 2006

Thinking this further, for Plugins it is useful to introduce a DOCGRAPHICSSEARCHPATH setting. For example, the RenderListPlugin has icons that could be used elsewhere, and the Plugin could make use of system icons and site icons. The question is performance, a cache expire test needs to be done on several topics instead of just two.

The ICON variable could be enhanced with a format parameter that expands tokens $name, $type, $file, $path, $description, $width, $height, $icon, $iconurl, $iconurlpath. For example, tokens for /path/to/somefile.doc are doc, gif, doc.gif, /p/pub/TWiki/TWikiDocGraphics, Microsoft Word file, 16, 16, full img tag, http://twiki.org/p/pub/TWiki/TWikiDocGraphics/doc.gif, /p/pub/TWiki/TWikiDocGraphics/doc.gif, respectively. Useful to construct a custom format, such as %ICON{ "%URLPARAM{type}%" format="$icon $type is a $description file" }% that renders as: Microsoft Word file doc is a Microsoft Word file.

-- PeterThoeny - 11 Feb 2006

Ping. This came up in Support.AllowICONsInWebPreferences.

-- PeterThoeny - 29 Nov 2006

Good idea. We could easily use the code from the template search path to create other search paths. Whatever we do, it should be consistent, so that users need to learn only one search path scheme. (I assume that this is not a 4.1 action?)

The other idea in this topic is to migrate the Main topic into a site customization web, and have the system configuration in the TWiki web. Let's look through what else is in Main and start moving into that direction.

-- ThomasWeigert - 29 Nov 2006

With yet another thing to search for - how about performance?

It can only be negative. It this worth it? Maybe it would be better to have a static alternative you can define in configure which would not cause further slowdown of an already too slow TWiki.

-- KennethLavrsen - 29 Nov 2006

Yes, it needs to be done with performance in mind, as I noted above. It can be done with intelligent caching (on search path and on icons attached to topics in search path).

Having said that, we can do things step by step: First simply support a TWiki.TWikiDocGraphics and Main.TWikiDocGraphics (with caching), then add a search path later (if inclined).

-- PeterThoeny - 30 Nov 2006

I refined the spec and set a date of commitment to 2009-12-13.

-- PeterThoeny - 2009-12-13

I implemented this on my machine. If no objections I can check this in before the 14 day review period.

-- PeterThoeny - 2009-12-24

This is tracked in TWikibug:Item6380. I changed the topic summary to better reflect the scope of enhancement.

Enhancement is committed into trunk as TWikirev:18259. Unit tests are pending

-- PeterThoeny - 2009-12-31

This is now done.

-- PeterThoeny - 2010-01-19

Edit | Attach | Watch | Print version | History: r15 < r14 < r13 < r12 < r11 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r15 - 2010-01-19 - PeterThoeny
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.