Tags:
caching4Add my vote for this tag performance3Add my vote for this tag search1Add my vote for this tag create new tag
view all tags

Variable Cache Plugin

Cache TWiki variables in selected topics for faster page rendering

Overview

This Plugin caches TWiki variables in selected topics for faster page rendering.

Most topics in a typical TWiki installation are rendered within a small deviation in terms of speed. Exceptions are topics that contain a lot of dynamic content. This is mainly caused by CPU or I/O intensive TWikiVariables, such as %SEARCH{}% and %INCLUDE{}%.

This Plugin caches all %VARIABLES% located in a topic so that the topic can get rendered as quickly as one without variables. An outdated cache gets refreshed when a user is looking at a topic; in which case it will take longer to render the page. Time-based refresh can also be disabled in favor of an on-demand cache update. The cache of affected topics can be invalidated based on parent relationship, backlinks and topic includes.

The rendering of TWiki markup to HTML is not cached. That is, user specific HTML rendering based on skin settings is not affected by the caching.

Syntax Rules and Cache Update Rules

  • Add a %VARCACHE{}% variable to the topic you want to cache
    • It supports these parameters:
      Parameter Description Default
      "..." or
      refresh="..."
      Cache refresh period in hours, e.g. maximum age of cache REFRESH Plugin setting
      cachemsg="..." Message shown when looking at a cached topic
      Note: Use $age to indicate the age of cache, $link to indicate the refresh URL
      CACHEMSG Plugin setting
      updatemsg="..." Message shown after a cache refresh
      Note: Use $link to indicate the refresh URL
      UPDATEMSG Plugin setting
  • The %VARCACHE{}% variable gets:
    • replaced with the cachemsg message when looking at a cached topic
    • replaced with the updatemsg message if the cache has just been updated
    • removed if caching is not active, e.g. when the topic gets rendered dynamically
  • The cache gets updated on topic view if:
    • the cache does not exist (e.g. after adding the %VARCACHE{}% variable)
    • the cache is older than the cache refresh period (unless refresh if set to 0) and the user is allowed to refresh with DENYVARCACHEREFRESH and/or ALLOWVARCACHEREFRESH preferences variables
    • the cache is older than the topic (e.g. the topic has been updated)
    • the topic is accessed with a ?varcache=refresh URL parameter (usually by following the refresh link in the cache message) and the user is allowed to refresh with DENYVARCACHEREFRESH and/or ALLOWVARCACHEREFRESH preferences variables
  • DENYVARCACHEREFRESH and ALLOWVARCACHEREFRESH preferences variables
    • Variables above work with $TWiki::Plugins::VERSION being 6.11 or higher. As of January 2020, that's the case only with the TWiki core in the source repository. No released version of TWiki has that or higher number.
    • There are cases where a user may view a topic but you don't want the user to update the cache. Let's say that the topic T aggregates other topics OTs which have view access restriction. In that case, only the users who have view access to OTs can render T fully. What would you do if you want people to view the fully populated T even if they cannot view OTs? For this situation, DENYVARCACHEREFRESH and ALLOWVARCACHEREFRESH preferences variables are handy. You will set ALLOWVARCACHEREFRESH with the people having view access to OTs. This prevents non-entitled people from updating the cache of T.
    • If those variables are not supported, the best you can do is to put %VARCACHE{0}% and having only entitled people view the topic with ?varcache=refresh when needed.
  • The topic is rendered dynamically without caching when:
    • the topic has URL parameters other than varcache and topic (e.g. to show dynamic content properly, such as table sorting, filtering, searching)
    • the topic is previewed after editing
  • Exclude variables from getting cached:
    • Prefix any variable with VARCACHE_EXCLUDE_ and it will be excluded from caching. For example, to exclude %WIKIUSERNAME% write %VARCACHE_EXCLUDE_WIKIUSERNAME%. Make sure to escape all variables that depend on an escaped variable. For example, escape a FORMFIELD variable if it encloses an escaped WIKIUSERNAME:
      %VARCACHE{"24"}%
      %VARCACHE_EXCLUDE_FORMFIELD{ "FirstName" topic="%VARCACHE_EXCLUDE_WIKIUSERNAME%" }%
    • Or, use <varcache_exclude> ... </varcache_exclude> to exclude sections of a page from caching. Example:
      %VARCACHE{"24"}%
      Cached: %SEARCH{ "." nonoise="on" format="$topic" separator=", " limit="10" }%
      <varcache_exclude>
      Not cached: %SEARCH{ "%WIKINAME%" nonoise="on" format="$topic" separator=", " limit="10" }%
      </varcache_exclude>
      Cached: %WEBLIST{}%

Plugin Settings

This plugin can be configured with preferences settings in Main.TWikiPreferences (site level), a WebPreferences (web level) or in a topic (page level). Do not change the settings here, they are here only to illustrate the default values. To overload, copy the settings to one of the aforementioned topics. On a large site having many webs, it's recommended to set VARCACHEPLUGIN_INVALIDATEONUPDATE none at the site level to prevent save slowness from happening unknowingly. The current site level setting is undefined, which is the same as include.

  • Debug plugin: (See output in data/debug.txt)
    • Set VARCACHEPLUGIN_DEBUG = 0

  • Default refresh period in hours. No automatic refresh: 0, default: 24
    • Set VARCACHEPLUGIN_REFRESH = 24

  • On topic creation and topic update, invalidate cache of affected topics: parent (its parent topic), parent(32) (its parent topics on parent path up to 32 levels), backlinks (its backlink topics in current web), include (its including topics in current web), none (no invalidation), or a comma-list thereof.
    Note: Invalidating affected topics can slow down topic save in very large webs; it is recommended to set this to none for webs that have 10K+ topics.
    • Set VARCACHEPLUGIN_INVALIDATEONCREATE = parent
    • Set VARCACHEPLUGIN_INVALIDATEONUPDATE = include, parent

  • Cache help messages. Use $age to indicate age of cache, $link to indicate the refresh URL:
    • Set VARCACHEPLUGIN_CACHEMSG = This topic was cached $age ago ([[$link][refresh]])
    • Set VARCACHEPLUGIN_UPDATEMSG = This topic is now cached ([[$link][refresh]])
    • #Set VARCACHEPLUGIN_CACHEMSG = Refresh cache
    • #Set VARCACHEPLUGIN_UPDATEMSG = Refresh cache

Limitations

  • Users: The cache is not user specific. For example, all users see the expanded %WIKIUSERNAME% of the person who refreshed the cache (on demand or by looking at a topic with an expired cache) unless the variable is specifically excluded from caching.
    • To exclude a user specific variable such as %WIKIUSERNAME% write %VARCACHE_EXCLUDE_WIKIUSERNAME%.
    • The <varcache_exclude> ... </varcache_exclude> tags can't be used in included topics; use the VARCACHE_EXCLUDE_ variable prefix in included topics.

  • Cache invalidation: Invalidating affected topic caches can slow down topic saves in very large webs.
    • Set VARCACHEPLUGIN_INVALIDATEONUPDATE to none for webs that have 10K+ topics.

Plugin Installation Instructions

Note: You do not need to install anything on the browser to use this plugin. The following instructions are for the administrator who installs the plugin on the TWiki server.

  • For an automated installation, run the configure script and follow "Find More Extensions" in the in the Extensions section.

  • Or, follow these manual installation steps:
    • Download the ZIP file from the Plugins home (see below).
    • Unzip VarCachePlugin.zip in your twiki installation directory. Content:
      File: Description:
      data/TWiki/VarCachePlugin.txt Plugin topic
      data/TWiki/VarCachePluginTest.txt Plugin test topic
      data/TWiki/VarVARCACHE.txt Variable documentation topic
      lib/TWiki/Plugins/VarCachePlugin.pm Plugin Perl module
      pub/TWiki/VarCachePlugin/cachetopic.gif Cache icon 16x16
    • Set the ownership of the extracted directories and files to the webserver user.
    • Make sure the dependencies listed in the table below are resolved. %$DEPENDENCIES%

  • Plugin configuration and testing:
    • Run the configure script, enable the plugin in the Plugins section
    • Configure the plugin: See plugin settings above.
    • Test if the installation was successful:
      • Visit the VarCachePluginTest topic, it contains a nested search.
      • Click on the refresh link or append a ?varcache=refresh parameter to the URL.

Plugin Info

  • One line description, is shown in the TextFormattingRules topic:
    • Set SHORTDESCRIPTION = Cache TWiki variables in selected topics for faster page rendering

Plugin Author: TWiki:Main.PeterThoeny
Copyright: © 2013-2015 Wave Systems Corp.
© 2004-2021 Peter Thoeny TWiki.org
© 2008-2021 TWiki:TWiki.TWikiContributor
Sponsor: Wave Systems Corp. for VARCACHE_EXCLUDE_ variable prefix and cache invalidation logic
License: GPL (GNU General Public License)
Plugin Version: 2021-03-25
2021-03-25: TWikibug:Item7927: Copyright update to 2021
2020-04-14: TWikibug:Item7879: may have caused a topic rendered broken. So now, cache files are written to temporary files and then renamed
2020-01-31: TWikibug:Item7881: _handleVarCache() to avoid direct file access
2020-01-21: TWikibug:Item7877: DENYVARCACHEREFRESH and ALLOWVARCACHEREFRESH variables to be introduced
2018-07-08: TWikibug:Item7841: Copyright update to 2018
2016-12-14: TWikibug:Item7764: May cause an error on a subweb
2016-01-14: TWikibug:Item7708: Copyright update to 2016
2015-03-16: TWikibug:Item7625: Invalidate parent topics on parent path up to indicated level
2015-01-09: TWikibug:Item7568: Document limitation of slow topic saves in very large webs; remove VARCACHE if found in an INCLUDE
2014-02-22: TWikibug:Item7438: Add cache invalidation logic for parents, backlinks and includes
2013-12-29: TWikibug:Item7396: Doc update on limitation of variable excludes
2013-12-09: TWikibug:Item7396: Cache topic also if topic parameter is used
2013-12-04: TWikibug:Item7396: VARCACHE_EXCLUDE_ variable prefix to disable caching of selected variables
2012-11-14: TWikibug:Item7020: Categorize TWiki Variable VARCACHE
2012-09-07: TWikibug:Item6837: Doc fixes
2012-04-27: TWikibug:Item6871: New <varcache_exclude> tag to exclude parts of page from getting cached
2011-05-25: TWikibug:Item6701: Small code refactor; adding $SHORTDESCRIPTION that was missing in previous release
2011-05-24: TWikibug:Item6730: Move cache data from pub directory to working directory; Use proper umask for directories created; Fix to make it better run under mod_perl; Support abbreviated %VARCACHE% syntax; Possible to specify a refresh of 0 (meaning infinite) -- TWiki:Main.TimotheLitt
2011-05-01: TWikibug:Item6701: Doc improvements
2010-04-25: TWikibug:Item6433: Doc fix: Changing TWIKIWEB to SYSTEMWEB
2010-04-06: TWikibug:Item5414: Support for plugins that dynamically add JavaScript and CSS using TWiki::Func::addToHEAD(); added VarVARCACHE variable documentation topic; support nameless parameter for refresh time; fixed typo in UPDATEMSG plugin setting name
2006-01-06: Small fix for TWiki 4.0
2004-07-16: Doc improvements (no code change)
2004-07-15: Initial version
TWiki Dependency: $TWiki::Plugins::VERSION 1.1 (TWiki 4.0, 2006-02-01)
CPAN Dependencies: none
Other Dependencies: none
Perl Version: 5.008
Benchmarks: GoodStyle 99%, FormattedSearch 99%, VarCachePluginTest 208%
Plugin Home: http://TWiki.org/cgi-bin/view/Plugins/VarCachePlugin
Feedback: http://TWiki.org/cgi-bin/view/Plugins/VarCachePluginDev
Appraisal: http://TWiki.org/cgi-bin/view/Plugins/VarCachePluginAppraisal

Related Topics: VarVARCACHE, VarCachePluginTest, TWikiPlugins, DeveloperDocumentationCategory, AdminDocumentationCategory, TWikiPreferences

Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatmd5 VarCachePlugin.md5 r31 r30 r29 r28 r27 manage 0.2 K 2021-03-25 - 07:44 PeterThoeny  
Compressed Zip archivetgz VarCachePlugin.tgz r31 r30 r29 r28 r27 manage 11.5 K 2021-03-25 - 07:44 PeterThoeny  
Compressed Zip archivezip VarCachePlugin.zip r34 r33 r32 r31 r30 manage 14.7 K 2021-03-25 - 07:44 PeterThoeny  
Unknown file formatext VarCachePlugin_installer r7 r6 r5 r4 r3 manage 3.5 K 2021-03-25 - 07:44 PeterThoeny  
Topic revision: r1 - 2021-03-25 - PeterThoeny
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.