Tags:
diagram1Add my vote for this tag create new tag
, view all tags

GaugePluginDev Discussion: Page for developer collaboration, enhancement requests, patches and improved versions on GaugePlugin contributed by the TWikiCommunity.
• Please let us know what you think of this extension.
• For support, check the existing questions, or ask a new support question in the Support web!
• Please report bugs below

Feedback on GaugePlugin

This plugin was needed for my day-time job to build a dashboard application. Spec by Peter and Tait, programming done by TaitCyrus.

  • This plugin is clean, e.g. it only uses published TWiki::Func::* functions.
  • It generates images into the attachment directory of the topic, without any versioning. In order to avoid a nameclash with regular attachments we have these names: %PUBURLPATH%/%WEB%/%TOPIC%/_GaugePlugin_<type>_<name>.png or .gif
  • The plugin generates GIF or PNG depending on the version of the GD library installed. PNG for GD lib version > 1.19, and GIF for older versions.
Possible Improvements:

  • Other gauges like dials, regular bars, LEDs etc. Contributions are welcome!

  • Hash value to enhance performance:
    • Build hash value (checksum) of each gauge based on initial settings and attributes, and store hash value.
    • Regenerate image only of hash value is different.
    • Since there are so few gauge attributes you could use the actual gauge string (expanded with initial values) as the hash value
    • Idea for hash value file name: _GaugePlugin_<type>_<name>.hash, alternatively generate one hash value file per TWiki topic.

  • Create a AttachmentStorePlugin that does the attachment storage handling; offers an API for plugins to use.
-- PeterThoeny - 23 May 2002

Looks cool! When images/attachments are generated? On page save? Or each time page is viewed?

-- PeterMasiar - 24 May 2002

At page view time. This is spec because a GAUGE can be embedded in a SEARCH, that is, the gauge changes even if the topic does not. The IMG tags currently have a time-stamp parameter so that browsers do not cache the images. Performance is not too bad even with this simple setup.

A planned enhancement is to use hash value to create image files only when necessary, and to list this hash value in the URL so that the browser caches the image as long as it is the same.

-- PeterThoeny - 23 May 2002

I got this feedback at my workplace:

I'd like an option where I could get the gauge to show me the endpoints of the data range, and also to show the current value of the gauge.

something like this

         -------------------
       0 |             |   | 100
         -------------------
                      75

         -------------------
       0 |   |             | 100
         -------------------
            25

Would be useful for showing performance numbers and knowing the numerical scale.

I can see that this would be a useful option. Probably better to render the numbers into the graphic image (with some white space on the left and right side) so that the gauges align nicely when stacked on top of each other. This could be enabled with a new parameter switch.

-- PeterThoeny - 30 May 2002

I've tested it on Cygwin where it shows a small bug: the png/gif file must be opened in binary mode. Here is the needed patch:

--- GaugePlugin.pm.orig   2002-05-23 15:23:40.000000000 +0200
+++ GaugePlugin.pm   2002-08-16 10:30:59.000000000 +0200
@@ -249,6 +249,9 @@
     # Write image file.
     umask( 002 );
     open(IMAGE, ">$dir/$filename") || return _make_error "Can't create '$dir/$filename': $!";
+#AS
+    binmode IMAGE;
+#/AS
     if( $GD::VERSION > 1.19 ) {
    print IMAGE $im->png;
     } else {
@@ -457,6 +460,9 @@
     # Create the file.
     umask( 002 );
     open(IMAGE, ">$dir/$filename") || return _make_error "Can't create '$dir/$filename': $!";
+#AS
+    binmode IMAGE;
+#/AS
     if( $GD::VERSION > 1.19 ) {
    print IMAGE $im->png;
     } else {

-- AndreaSterbini - 16 Aug 2002

New version posted. Small fix for binmode (thanks Andrea); passing parameters also to the error image.

-- PeterThoeny - 08 Sep 2002

How about using an html table with coloured backgrounds as a poor man's gauge? The reason I set off this way is that I did not have GD on our server, and only wanted a really basic gauge display. So I did this (warning - newbie-level Perl ahead...):

# display gauge using html table
# pass in int value for percentange done
sub gaugeLite
{
    my $done = $_[0];
    my $todo = 100 - $done;
    my $line="<table border=1 width=100%><tr>\n";
    if ($done > 0) { $line .= "<td width=$done% bgcolor=\"#00ff00\">.</td>\n"; }
    if ($todo > 0) { $line .= "<td width=$todo% bgcolor=\"#ff0000\">.</td>\n"; }
    $line .= "</tr></table>\n";
    return $line;
}

which produces a display something like:

. .

that can be embedded inside a field in a table. The dot in the <td> field is a hack to make the field display, which suggests this may not be a very robust solution, and it is certainly not elegant, but it seems to get the job done for really simple cases.

See Doc Graphics for another example MartyBacke - 15 Sep 2002

-- MartinWatt - 14 Sep 2002

Using a table with colored background is a simple and portable way to build bar charts, I used that at work once in a different context. (BTW, you could use a instead of a dot.)

Yes, the tambar Gauge could have been done just with that. Initially I had plans to add other gauges like a round dial gauge to the GaugePlugin, there you would need a graphics library.

-- PeterThoeny - 15 Sep 2002

non-maintainer update synchronized cvs with released zip version, which was ahead of the cvs version

Internal plugin version: 01 May 2003
Zip version: 1.3

-- MattWilkie - 02 Apr 2004

Updated Plugin with benchmark numbers (no code changes)

-- PeterThoeny - 16 Nov 2004

Note to myself: The GaugePlugin docs could be a bit more explicit.

There are six colors, by default: dark red, light red, dark yellow, light yellow, dark green, and light green. The dark colors are used for the lower thin bar and for the left part of the upper thick bar (color depends on where the value is in relation to the lower bar). The right part of the upper thick bar uses the light color equivalent of the color in the left part.

-- PeterThoeny - 20 Oct 2005

I'm wondering if one could combine the GaugePlugin to make WebStatistics more transparent. Maybe there should be a HotSpotsPlugin that creates graphical representations of topics with much activity (read/write) like shown in the following two screenshots taken from GründerWiki resp. DorfWiki

hotspots2.gif hotspots1.gif

-- FranzJosefSilli - 05 Nov 2005

This Plugin could be extended to SupportSparklines.

Example 1: %GAUGE{ type="sparkline" name="t1" value="88, 84, 82, 92, 82, 86, 66, 82, 44, 64, 66, 88, 96, 80, 24, 26, 14, 0, 0, 26, 8, 6, 6, 24, 52, 66, 36, 6, 10, 14, 30" }%

Produces: sparkline.png

Example 2: %GAUGE{ type="sparkbar" name="t2" value="88, 84, 82, 92, 82, 86, 66, 82, 44, 64, 66, 88, 96, 80, 24, 26, 14, 0, 0, 26, 8, 6, 6, 24, 52, 66, 36, 6, 10, 14, 30" }%

Produces: sparkbar.png

The values would be scaled to fit in a default width="100" height="16", which can be changed. Also color="#880000" could be an additional parameter.

-- PeterThoeny - 28 Apr 2006

Ping! Someone interested in adding sparklines to the GaugePlugin? I think this is a cool & useful feature, relatively easy to implement

-- PeterThoeny - 2010-03-30

Added sparkline, sparkbar, and sparkarea to the ChartPlugin since that was easier than adding this functionality to GaugePlugin or creating a new plugin.

-- TaitCyrus - 2011-05-13

I added a new option scalesize to the GaugePlugin allowing a user to specify the height of the gauge scale as a percentage of the gauge. To support the request above from FranzJosefSilli you can use scalesize="0" which would show the gauge value without showing the gauge scale.

-- TaitCyrus - 2011-05-13

Code reading bug:

Two people (sessions) viewing the same topic can get the wrong image if the gauge data is user-specific. (e.g. from a database, or even a user parameter or environment variable.)

This is because the filename for the generated image is simply pub/web/topic/_GuagePlugiin_type_name.foo. Collisions are inevitable.

Also, cleaning up old images (e.g. a gauge is removed from a topic; renamed; a topic is deleted or moved...) means scanning pub/. In any case, storing transient data in pub/ is deprecated.

Suggestions:

a) Move the temporary images to working/ (getworkarea) This makes it easy to include them in a working/ maintenance script.

b) Come up with a naming scheme that is guaranteed unique - say, an incrementing sequence number - which needs a maintenance script.

c) Better yet, don't generate these files at all. As you can see, they're painful. Why not use a data URI (RFC 2397) and simply inline the data? These images should be small enough for this, and it completely eliminates the file maintenance/collision issues.

I decided not to use this plugin because of these issues.

FYI, a data: uri looks like:

<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg=="  alt="Mystery image" />

Just MIME::Base64::encode_base64 the binary instead of writing it to a file, and prepend the data:image/png;base64,.

Also, NO_PREFS_IN_TOPIC should be set & the doc updated to say that the preferences are set in the user/web/site preferences, not the plugin topic.

And - I don't thing the global variables are a good idea - consider a mod_perl environment. Just fetch the default values for each guage - the preferences code will cache them.

-- TimotheLitt - 2011-05-22

Thank Timothe for the feedback. Tait Cyrus is currently enhancing this plugin. My reply:

a) We can't move the image data from pub to working dir because the images need to be accessible by the browser.

b) Using sequence number means a new image each time, e.g. brings up garbage collection issue. How often are there cases that an image is user dependent? In the rare case that you need to user-dependent you can add the user to the name="" parameter.

c) Interesting, I did not know about the data URI. It makes sense to use it for gauges, the data is small enough, and it avoids multiple http connections. Are data URIs supported by all major browsers? From what version on?

BTW, there might be an issue if the Base64-encoding of data URI requires newlines. In TWiki, HTML tags must not contain newlines, angle brackets are escaped in case there are newlines.

-- PeterThoeny - 2011-05-23

a) You're right, /working doesn't for this application. Sorry about that.

b) As I noted, you have the garbage collection problem now. Topic move/rename/delete, or gauge delete.

Conflicts can be more common than just user names. As I noted, anything based on an environment variable or database or time or form/POST data or SEARCH results or ... Pretty much anything that varies from view to view or session to session.

You can do better than a sequence number; I didn't mean that literally. A MD5 or SHA1 hash of the image data would make a good filename. You don't need web or topic in the name. This will consolidate gauges across all webs/topics (I bet there are a lot of "100%" in the default colors.) It also prevents denial of service by repeated views and avoids taking out a lock on a sequence number file. You check for the filename existing before opening for write to prevent a race (sysopen O_EXCL, of course).

But you still need to garbage collect. With a hashed filename, it's probably easiest to just wipe out anything not accessed within a small number of days. (stat[8]).

Speaking of which, garbage collection for plugins is a sore point. Right now, every site is on its own coming up with some maintenance script - and you pretty much have to read the code of every plugin that you use to find out what you have to do. This is unreasonable Here's an idea:

Extend tick_twiki to plugins. Something like:

# Run plugin garbage collectors

use warnings;
use strict;

foreach my $plugin ( @{$twiki->{plugins}{plugins}} ) {
    next if( $plugin->{disabled} );

    local $TWiki::Plugins::SESSION = $twiki;

    my $cleanup = $plugin->{module} . '::pluginCleanup';

    if( defined( &$cleanup ) ) {
   no strict 'refs';
   &$cleanup( $twiki, $now );
   use strict 'refs';
    }
}

where plugins can have a hander somewhat like this proposed default:

# Garbage collection function called by the tick_twiki.pl cron script
# You should cleanup any old/orphaned files that your plugin creates
# in your working area (or elsewhere).

sub DISABLED_pluginCleanup
{
    my( $twiki, $now ) = @_;

    my $wa = TWiki::Func::getWorkArea($pluginName);
    my $oldest = $now - (24*60*60);  # One day - adjust to suit or use a config variable (e.g. $TWwiki::cfg{$pluginName}{MaxAge})

    # Age-based cleanup

    foreach my $wf ( glob( "$wa/*" ) ) {
      unlink $wf if( (stat( $wf ))[9] < $oldest );
    }

    # Orphan cleanup.  Suppose your working files are "$web_$topic_data" 
   # but .conf files are never to be deleted:

    my $webRE = TWiki::Func::getRegularExpression('webNameRegex');
    my $topicRE = TWiki::Func::getRegularExpression('wikiWordRegex');

    foreach my $wf ( glob( "$wa/*" ) ) {

      my( $web, $topic, $type ) = $wf =~ m!^($webRE)_($topicRE)_.*(?:\.(.*))?$!;

      next unless( defined $web && defined $topic && defined $type && $type ne 'conf' );

      unlink $wf unless( TWiki::Func::TopicExists( $web, $topic ) )    }
    }
} 

Here's a working example for the VarCachePlugin (after my recent patches):

# =========================
sub pluginCleanup
{
    my( $twiki, $now ) = @_;

    my $webRE = TWiki::Func::getRegularExpression('webNameRegex');
    my $topicRE = TWiki::Func::getRegularExpression('wikiWordRegex');

    my $wa = TWiki::Func::getWorkArea($pluginName);

    foreach my $wf ( glob( "$wa/*/*_cache.{head,txt}" ) ) {

      my( $web, $topic, $type ) = $wf =~ m!^$wa/($webRE)/($topicRE)_cache\.(head|txt)$!;

      next unless( defined $web && defined $topic );

      unlink( $wf ) unless( TWiki::Func::topicExists( $web, $topic ) )
    }

}

I didn't cleanup directories in the case of web deletion for this quick prototype, but that's an easy exercise for the reader. A production version would probably also read the topics found & unlink cache files whose topic no longer contains %VARCACHE(?:{(.*?)})?%. I left these out to make the mechanism clear.

For this (Gauge) plugin, of course, you'd have to scan pub, and as I mentioned, use the age (accessed) approach rather than topic references - especially if you go with the hashed filename suggestion.

c) data: URIs were defined in 1998 and were around experimentally before then. Major browsers do support them - to varying degrees. Google "data uri" for info. Basically, IE7+ should be OK; all the other browsers supported earlier. IE7 has a 32K limit; IE9 is supposed to raise it. I've heard iPhone has worked with upto 1MB (but I wouldn't); others are in between. For Gauges, should be OK with any reasonably modern browser.

You could always fall back to files based on known bad browsers and/or file size and/or a preference. But with files the garbage collection problem remains.

Yes, the point of data URIs in general is to reduce connections (or, with keep-alive, requests). But here it has the advantage of also eliminating transient data.

Whitespace - not a problem. Use a null string as the second argument to MIME::Base64::encode_base64.

d) Once again, this plugin is modifying umask without saving/restoring it. This is bad. All plugins need to be scanned for this (and fixed).

e) I'm going to suggest the pluginCleanup API on Foswiki as well - let's try to make this compatible if accepted.

-- TimotheLitt - 2011-05-23

For tracking purposes, the corresponding Foswki work item for the plugin garbage collection API is http://foswiki.org/Tasks/Item10780.

I also created http://twiki.org/cgi-bin/view/Codev/PluginGarbageCollection and pointed it here.

I've prototyped this with TWiki 4.2.3 and the VarCachePlugin (as I recently patched it); the code above is runable, although there may be a better way to iterate over the plugins. It seems to be a reasonable proof of concept.

-- TimotheLitt - 2011-05-23

I just checked in some changes, one of which adds a new parameter of access which will allow users to specify whether the image data is accessed via a file (the original behavior) or embedded within the <img> tag. To embed in the <img> tag add paramter: access="inline".

Another change is to add a new type="simple" which creates a gauge 100% from HTML tables. This would allow users who don't have GD installed to generate gauges.

Also fixed the umask bug mentioned above.

As for generating unique image files, I would agree with Peter that you should use a gauge name that is unique for each user. This is what I do -- I've a generic page with numerous parameters passed into the page and I use some of those parameters when defining name.

-- TaitCyrus - 2011-05-23

Glad to hear that you're making some progress.

A unique name gauge name per user will not solve the problem for the reasons given above; it's not just a user that makes for ambiguity.

Consider a user who logs in via VPN and direct on two machines. Same user. Guage is setup to return a function of ping latency to the remote IP address, from an environment variable. User hits refresh simultaneously. Both are likely to get the last value computed - though other scenarios are possible.

Other scenarios based on environment variables, post data and so on are easy to come up with. And then you get to shared accounts - yes, a really bad idea. But they happen in real life. Then username is is not a sufficient disambiguator.

You can't rely on username - please don't.

Since you're keeping files as an option, I strongly suggest making the filename be '_GuagePlugin_ + a hash of the (binary) data as described above. This will make the filename a function of the data. And use sysopen O_EXCL -- on error, EEXISTS is OK (the file already contains your data), and other error (such as EPERM) gets reported.

My examples aren't contrived - and TWiki wants to be enterprise class software. That means we need to code for 100% of the cases, and make it natural for the average user to do the right thing.

-- TimotheLitt - 2011-05-24

Thanks Tait for working on this plugin! Fast turnaround from discussion to implementation!

Using a hash is almost like a random string, e.g. brings up the garbage collection question. Once this is in place we can use it. For now I believe we can live with a workaround on case by case basis in case the gauge name is not sufficient.

-- PeterThoeny - 2011-05-24

The reason I suggested GaugePlugin as a prefix was precisely to enable garbage collection; all files m/^_GaugePlugin_[[:xdigit:]]{hashlen}\.(?:jpg|png)$/ are one of these temporary files.

The match doesn't have to be that precise; a shell glob of GaugePlugin????????.{jpg,png} (with the right number of ?s) will work just as well.

This can be easily garbage collected using the new scheme. But it can also be collected using a find script (e.g. find pub/ -type f -mtime +1 -name -delete) - which you have to do for the existing gauge name scheme.

I really don't like the idea of 'working around' collisions in topics. This is a burden on our users; is ripe for error (how many will read or follow this discussion), and creates a legacy compatibility issue that we will tiptoe around for years.

We know there's a problem. We shouldn't ignore it. The enterprise-class fix is straightforward.

-- TimotheLitt - 2011-05-24

Correction: Find should be -atime in this application.

-- TimotheLitt - 2011-05-24

FYI, this plugin already names image files _GaugePlugin_tambar_foo.png if a gauge named foo of type tambar is selected.

-- PeterThoeny - 2011-05-24

Yes. You said that a hash made the file name a random string that can't be garbage collected. I'm pointing out that isn't so.

I'm done pointing out that tambar_foo is collision prone and that it doesn't meet any standards for Enterprise class product. Either I'm opaque or you don't care. At least the data uri version shouldn't have the problem.

-- TimotheLitt - 2011-05-25

Yes, there are use cases where the gauge may look different by user name, session ID or the like. Personally I have not seen the need, and I have deployed this plugin in many different web apps.

Should we rush the proposed feature into this plugin? I invite you to take a step back and reflect on this: With TWiki as an enterprise class product I care for:

  • Productization - there is a difference in coding a feature and make it work well and repeatable in production.
  • Documentation - is the new feature documented well? Creating good documentation takes time.
  • Compatibility - is a new feature compatible?
  • Ease of installation - adding a new feature that requires to install additional dependencies or patches makes it harder to install a product, it is not repeatable.
  • Pragmatism - or the 80/20 rule - is a feature really needed by the majority of target users?
  • Usability - is a new feature usable by non technical users?
The PluginGarbageCollection feature is currently discussed, and will likely be implemented in one form or another. Once we have garbage collection we can look into the proposed hash based image name feature. I invite you to take into account the productization, ease of installation and other items I pointed out.

-- PeterThoeny - 2011-05-25

Concerning the issue of two people (sessions) viewing the same topic I would suggest that the solution is to have a gauge name that is unique in all situations. name should include the users/account name as well as the browser IP address (and any other unique identifying information).

I understand the temptation to want the plugin to create unique image names automatically because that would make the users job of creating a topic page easier, but this would just increase the need for PluginGarbageCollection since many many more images would need to be created. In this situation I believe it is not the plugins responsibility to create unique image file names, but rather the developer of the topic page.

Also, if image names were done automatically, then name would not be needed at all. This, though, would then break the ability of any other topic, or within the same topic, to reference a previously created image file since there would be no way to know what the image name is.

With the recent update there are two solutions to the "Enterprise class product" problem:

  1. Set the TAMBAR_ACCESS preference to be inline, or
  2. Set the TYPE preference to be simple
In either case, no update to existing topics is required.and it addresses the "Enterprise class product" problem wonderfully since no image file is created meaning there is no name collision problem.

-- TaitCyrus - 2011-05-25


Edit | Attach | Watch | Print version | History: r36 < r35 < r34 < r33 < r32 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r36 - 2011-05-25 - TaitCyrus
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.