Tags:
create new tag
, view all tags

Proposal

There should be some means of feeding back usage statistics to Plugin authors, so they know whether their plugins are being used or not.

Discussion

In TWikiDrawPluginDev CrawfordCurrie said today: "I've not been quite as active maintaining this plugin for a couple of reasons: .... 3. I'm not particularly convinced there's a broad requirement for this plugin." Considering this, is it worth changing TWikiInstallationForm to record what plugins have been installed so that we can get some idea of the size of user base for each plugin?

-- SamHasler - 15 Dec 2003

One of the reasons for the TWikiInternetDeploymentsSpider was to be able to detect the number of public installations of plugins.

  1. I'll look into making it open so others can continue developing it.
  2. Perhaps there needs to be a way for private installations to post this information.

-- MartinCleaver - 09 Dec 2003

One approach, and much more edgy, would be to use a mailer. For example, one of the functions of the notify script could be to mail the configuration of the TWiki installation to a central point for regular (script) collation and advertisement - say, once a month. This is better than a spider because it would work behind a firewall and help track corporate installations. Of course this would have to be optional for the installer, but I for one wouldn't have had any issue with permitting it behind a corporate firewall. The collation would of course be anonymous, and simply list statistics such as number of installations of the different releases of TWiki (and of course the same of the plugins).

We used this technique extensively in Motorola to track usage of software. Helped immensely in focusing efforts where the greatest customer base was.

-- CrawfordCurrie - 10 Dec 2003

Crawford: Interesting idea of mailing statistics to TWiki.org. There could be concerns on privacy and confidentiality. That is, it should be an optional feature, and a sample e-mail could be shown.

-- PeterThoeny - 12 Dec 2003

Yes, privacy and confidentiality would be an issue, no doubt of it. For that reason, it would have to be very much in-your-face, very obvious to the installer that this functionality was present. I would suggest some words like:

TWiki installation includes an optional mailer that is used by the development team to summarise deployment of TWiki. The mailer runs as part of the mailnotify script, and runs once a month. It generates a mail like the following, which is sent to twiki.org:
Version: Cairo
InstalledPlugins: EmptyPlugin(V1.0) TWikiDrawPlugin(V3.2) SmiliesPlugin(Unknown) SpreadsheetPlugin(V6.2)
Activity: 0.15
This information is used to generate statistics pages on twiki.org, that help the developers focus their efforts and know that what they are doing is worthwhile. See http://twiki.org/cgi-bin/Codev/TWikiActivity. As you can see, no information about the location of the installation, and no mail addresses, are recorded.

If you want your installation to be excluded from the statistics, the mailer can be easily disabled by adding the "NOSTATISTICS" parameter to the mailnotify command line. We would ask you not to, though, as the statistics are invaluable in helping us plan and execute TWiki.

Activity would simply be number of (pages changed since last mail) / (total number of pages). The processing script would cron the mailbox on a daily basis, immediately throw away all header information, and simply collate the statistics to generate some graphs on TWiki pages showing total number of installations of the different releases, with average activity bars. If you have them, statistics on total downloads of the various releases could also be incoporated. Individual plugins could have individual pages showing number of installations.

Of course, there's no way to confirm that "vanilla" versions of anything are being run, or whether they have been locally hacked.

I feel this would be an incredibly useful marketing tool, as well as supporting developers. It would certainly help me!

-- CrawfordCurrie - 12 Dec 2003

Could you mail CRC checksums of the plugin scripts to check if they've been mofified?

-- SamHasler - 12 Dec 2003

Re: e-mailing statistics.

The assumption that deployed sites have smtp mail and have it configured for outgoing is a big assumption! Why not just have it POST the data to twiki.org. It's nearly guaranteed that port 80 will be open, if they're running twiki.

-- JonathanCline - 13 Dec 2003

Only last night was I thinking that we should make available CRC checksums for all the files in the distribution. This way we would know which files are commonly modified.

I like the HTTP POST idea!

-- MartinCleaver - 13 Dec 2003

The POST idea is an interesting one. Some concerns; wouldn't POSTing hang the sending script while the connection was established? And what if the connection fails; do you retry immediately, or wait until next month to send again? Oh, and in a corporate environment it's very common that to get beyond the corporate firewall requires an interactive password entry. And that password is required to be changed every month.

The nice thing about sending mail is that it's a "sling it over the wall" kind of operation, and if twiki.org is down (not an unknown situation!) then the mail still gets there. BTW, how big is the assumption about SMTP mail? This is the 21st century after all. Does anyone have experience of such black-box sites?

-- CrawfordCurrie - 13 Dec 2003

CRC checksums would be good if they can be computed fast and reliably. One mild concern I would have is that not everyone installs Perl in the same place, so a common edit is to the first line: #!/usr/bin/perl -> #!/usr/local/perl5.003/bin/perl or similar. This is likely on sites where multiple Perl versions are extant. Doesn't mean the script has been interestingly modified.

-- CrawfordCurrie - 13 Dec 2003

WRT hanging, we could just give it to LWP.

You are quite right about the CRC issue. Unless we can send diffs (more information) I suppose we could generate CRCs from the second line downwards. Would that work?

-- MartinCleaver - 13 Dec 2003

Yeeeees..... as long as the CRC is only on .pl and .pm, it should work. If it's on all files in the plugin directory, we would run into per-site configurations files (yes, there are a few).

-- CrawfordCurrie - 13 Dec 2003

Re: CRC. Just use md5sum on each file. A good installer (once written) would use likely use md5sum anyway, so it could just send the text file corresponding to the twiki registry.

Re: smtp mail. I installed twiki on three intranets which did not have smtp outgoing configured. That means local root would get the bounced email. I was the only one who ever checked local root's email, and that was hardly a scheduled routine (read: only happened when really bored).

Re: POST as blocking operation. I don't see that it makes any difference, especially if it's a cron job.

Re: POST as firewalled operation. What's that about password auth being required to get out of a firewall? I don't think so. A machine running a web server is already a trusted machine.

-- JonathanCline - 14 Dec 2003

If the first line is edited you would expect it to be on all scripts. Why not store the CRC/md5sum's and just send whether the script has changed since last month. That way you see the initial customisation to all scripts and can ignore it, but then if some plugins are changed frequently you will have a month by month idea of which plugins are changing. So if a useful patch is placed on twiki.org and you see loads of changes to that plugin next month you could assume that many installations have had it applied.

It might also be useful to display this information on the testenv output.

-- SamHasler - 15 Dec 2003

Jonathan: sorry to have to shatter your illusions, but on a large corporate internet most web servers do not have access to the internet, other than through firewalled proxies, which require per-session authentication. Individual staff also have to be individually authorised to access the internet; something to do with minimising surf time versus work time frown

If POST is blocking, then I guess you have a choice; either hang indefinitely on the POST, or time out. If the former, I can envison lots of mailnotify processes cluttering up my machine. If the latter, then statistics get horribly skewed as installations that report one month don't report the next.

md5sum: is available on *nix servers only (or with gnutools). Whatever computes the CRC has to be on all server platforms.

Sam: good idea, but complicated. In the interests of KISS I'd recommend skipping the CRCs in a first version. They can always be added later.

There seems to be a fundamental dichotomy between (1) corporate intranets, where POST outside the corporate subnet is via a firewalled proxy and password protected, but e-mail to the big wide world is easy, and (2) some installations where a POST is possible, but mailing out is not. Of course, we could always use BOTH methods; and get the installer to configure it.

-- CrawfordCurrie - 15 Dec 2003

MD5 CPAN module is available for most platforms: http://testers.cpan.org/show/Digest-MD5.html#Digest-MD5-2.33

-- MartinCleaver - 15 Dec 2003

Ok, skipping the CRCs in a first version makes sense, but I still think that having the installation do the calculation of whether the plugin has changed from the previous report provides much more useful statistics. It shows how frequently plugins are changing over time.

I think it should work something like this:

  • Take CRC's one month after installation and store in a file. For each script store an 'original' CRC and a 'current' CRC. The 'original' CRC will not change once it has been taken for the first time whilst the 'current' will be recalculated each month. The 'original' will be assumed to be on the original scripts plus any changes required to get the plugins working.
  • Next and subsequent months take new CRC's and check if they have changed since the original and previous report. A report should be sent as described above but also stating 'Unmodified', 'Modified', or 'Modified in last month' for each plugin.
  • Then twiki.org just has to keep a running tally of the reports recieved for the current month. At the begining of a new month it starts a new tally and stores the previous months statistics.

The advantages of this to sending out CRC's is that there is no need to do anything with CRC's on the twiki.org server, and no need to store reports after they have been tallied, or correlate reports with previous months. (unless you want to keep them a while to check for 'cheats').

-- SamHasler - 15 Dec 2003

There are definitely some standalone md5 programs available for dos / windows (I used to use them). IIRC, their names were slight variances of md5sum, and of the two that I found at one time, one worked well (i.e., consistently) and one had some problems.

-- RandyKramer - 16 Dec 2003


CategorySurvey
Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r8 - 2006-02-15 - PeterThoeny
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.