Feedback on TWikiShellContrib

Finally, the TWikiShellContrib work is up! Now, let's beef it up with some new commands.

Question: What is the best way to provide extentions to the shell?

1. Upload it to the TWikiShellContrib topic
2. Create a Contrib package for each one

Extension provided by add-ons, plugins and contribs could just be packaged with them. I'm refering to the standalone ones.

-- RafaelAlvarez - 22 Oct 2004

Good work, Rafael!

Some questions/points:

1. Is the twiki home directory really the best place for the uber-command? I would have expected to see it in bin or contrib/bin
2. Can we please have intention revealing interface names in the Ext API? smry says nothing to me, and I can't see why you wouldn't use summary.
3. The "Purpose" needs to say why someone would use this Contrib. What advantage does it bring? Is it for end users or plugin developers?

I assume the "install" subcommand correctly runs the installer script for plugins built with BuildContrib, yes?

-- CrawfordCurrie - 22 Oct 2004

I'm very pleased that Rafael has got this out of the door. I stopped working on it after the lukewarm feedback I got at Font. Kudos to him for seeing it through.

I can answer the first two:

1. I think it was put in /:
   • to be next to the UpgradeTWiki command (yep, that's why - RA)
   • because you would certainly (hmm...) not want this executed over the web (yep, that also - RA)
2. The API was inherited (as is the interactive mechanism) from CPAN:Term::Shell

-- MartinCleaver - 22 Oct 2004

Known issues

1. the directory structure is wrong. Although I would prefer if TWikiPackages each had their own top level, that is not the spec: the top level must contain multiple dirs. i.e. /bin, /lib, etc.
2. If you haven't made setlib.cfg absolute twikishell will be unable to find your lib dir.
3. run_ commands must print \n after their output or it will not get printed with the output. Also they must print, not return as the return value from run (seems to be) ignored.
4. prepareEnviroment should be prepareEnvironment (will change if I start hacking CVS)
5. if you import something and it fails to load because of a syntax error, you cannot reimport once it is fixed. You have to reload the shell.

Feature suggestions

Not asking you to do any of this...

1. Would be useful if import said how many commands it imported, into what namespace and how many it ignored.
2. Also, if it can't find an import, an error would be useful.
3. Would expect it to alias ? to help
4. Would you agree that shells might need to know where the data, pub, template dirs are as well as bin and lib? should we set variables for them in config?
5. need to establish the working directory. the cgi stuff always has bin/ as the cwd - so I suggest we adopt the same for the TWikiShell
6. Need the equivalent of -M to preload a specific class
7. might be useful to catch ctrl-c so a user can interupt without quitting the shell.
8. would be nice if there was an unimport and reset commands
9. We need also:

import TWiki::Plugins::TWikiReleaseTrackerPlugin::Commands

and perhaps:

import TWiki/Plugins/TWikiReleaseTrackerPlugin/Commands

and definitely

import XPlugin
import XContrib
import XAddOn

which should look for XPlugin::Commands. I think a default separation (i.e. to the Commands subclass) is probably a good thing as it: encourages the programmer to create a modular structure for their extension.

1. There should be a way to report to the web.
2. It would be useful to be able to restore history between sessions.

Questions

1. Did you manage to get shell-completion to work?
   1. Haven't tried, because it need yet-another-dependancy.
2. I note that there is no longer the ability to change context (i.e. "use") - we should discuss this, I know it was not documented.
3. Should it still automatically derive help from POD? It doesn't work for me.
4. Rafael, do you think that per-extension implementations should be subclasses of the TWikiShell? - it would make overriding the default mechanisms neater. Also initialisation can then be better componentised into new rather than in BEGIN blocks, and config would be attribute of self rather than having to be passed with every call.

Potential Command sets

1. Might be good to set up a loadable commandset that interprets "!" as a shell escape to the O/S.
2. Also perhaps "&" as an escape to the perl interpreter.
3. Backup the data dir, pub dir, or the whole installation
4. Update the installation with a new distro (Integrate with UpdateTWiki script)
5. Generate Statistics
6. Force Mail Notification.
7. Check and Fix directories permissions
8. Configure TWiki (interface to TWiki.cfg)
9. A set of TWikiFuncCommands: these would expose the TWiki::Func API as a command line interface
10. Make available specific twiki variables, e.g. TWIKIVERSION

A command to set the form on every topic in a web.

-- RafaelAlvarez - 27 Oct 2004

Please, add your ideas. I'm looking for admin-related and developer-related command sets.

-- RafaelAlvarez - 24 Oct 2004

Comments

Well, I know I was one who gave lukewarm feedback; and as I said above "The "Purpose" needs to say why someone would use this Contrib. What advantage does it bring?" I do start to see some advantages for developers (building, running tests) and maybe some for installers, but it seems like a lot of work for relatively little gain. Please, please prove me wrong!

-- CrawfordCurrie - 22 Oct 2004

Well, I just implemented TWikiReleaseTrackerPluginCommands ...

-- MartinCleaver - 22 Oct 2004

Rafael, something like this in the class would be good:

sub printIfVerbose {
  my ($config, $message) = @_;

  if ($config->{verbosity} > 1) {
    print $message."\n";
  }

}

=pod 
---++ message ($config, $level, $message)
if (level > current verbosity) then print message + NL

=cut 

sub printIfTerse {
  my ($config, $message) = @_;

  if ($config->{verbosity} > 0) {
    print $message."\n";
  }

}

sub printIfDebug {
  my ($message) = @_;

  if ($settings{debug} > 0) {
    print $message."\n";
  }

}

-- MartinCleaver - 29 Oct 2004

Which class? TWikiShell?

As every command is receiving the shell and the config, I'll move the print* subs to the shell. I'll add these in the process.

The print* subs in $config will be deprecated.

-- RafaelAlvarez - 29 Oct 2004

Martin and Rafael, I made some enhancements to the NewContribTemplate and created supporting Contrib topics. With this I changed the Contrib topic text:

• Added SHORTDESCRIPTION in "Settings" section
• Added a note to the "Installation Instructions" section
• Added link to GPL in "License" table row
• Added "Appraisal" table row
• Fixed heading levels (use level one only once on top)

Could you take that into the next release of the package?

-- PeterThoeny - 22 Mar 2005

We need an example command set that shows how to add the simplest possible command and that can be used as a template for developing new commands.

Without this it can be too hard to know where to start.

-- MartinCleaver - 03 Aug 2005

Does the twikishell use pod2docs? See tools/gendocs.pl for an example.

-- MartinCleaver - 03 Aug 2005

The example command set is TWiki::Contrib::TWikiShellContrib::Ext::Dump

I'm working on a Command Set to "substitute" the mklinks.sh script. I hope that would be a better example.

-- RafaelAlvarez - 03 Aug 2005

Raf. Finally got a chance to try this version. I'd move all the how-to-develop a twiki CommandSet into a bundled file. Most people will be just users and want to test the functionality. I therefore guess that following should be prominent in your documentation:

> twikishell
twiki > import TWiki::Contrib::CommandSet::Apache
 Absolute path to httpd.conf [/etc/apache/httpd.conf]: --->
twiki > import TWiki::Contrib::CommandSet::Build
twiki > help
Type 'help command' for more detailed help on a command.
  Commands:
    apache add    - Not implemented
    apache delete - Not implemented
    build         - Interfaces with the build.pl script
    debug         - Turn on/off the debugging statements
    exit          - exits the program
    help          - prints this screen, or help on 'command'
    import        - Imports the specified CommandSet
    q             - exits the program
    quit          - exits the program
    verbose       - Sets the verbosity level (1,2)
twiki > import TWiki::Contrib::CommandSet::Package
twiki > import TWiki::Contrib::CommandSet::RunTest
 Path to the unit tests [/test/unit]: --->
twiki > import TWiki::Contrib::CommandSet::TWiki
 Absolute TWiki root path [C:/moreprgs/indigoperl/apache/TWiki]: --->
 Absolute TWiki bin path [C:/moreprgs/indigoperl/apache/TWiki/bin]: --->
 Absolute TWiki lib path [C:/moreprgs/indigoperl/apache/TWiki/lib]: --->
 Absolute TWiki pub path [C:/moreprgs/indigoperl/apache/TWiki/pub]: --->
twiki > import TWiki::Contrib::CommandSet::Plugin
twiki > import TWiki::Contrib::CommandSet::Dump
Type 'help command' for more detailed help on a command.
  Commands:
    apache add     - Not implemented
    apache delete  - Not implemented
    build          - Interfaces with the build.pl script
    debug          - Turn on/off the debugging statements
    dump           - Dumps various debug informations
    dump config    - Dumps the config
    dump handlers  - Dumps the registered handlers
    exit           - exits the program
    help           - prints this screen, or help on 'command'
    import         - Imports the specified CommandSet
    package        - Package a set of files for distribution
    plugin         - Plugin Management
    plugin create  - Creates the suggested directory structure for a new plugin
    plugin develop - Prepares the file of a Plugin/Contrib for development
    plugin putback - Put back Plugin/Contrib files into their checkout area
    q              - exits the program
    quit           - exits the program
    runtest        - Not implemented
    runtest assert - Not implemented
    verbose        - Sets the verbosity level (1,2)

twiki > help plugin create

SYNOPSIS
     plugin create <TWikiExtension> [-contrib] - Creates the directory twikiplug
ins/<TWikiExtension> with the suggested structure

DESCRIPTION
     This command will creates the directory twikiplugins/<Plugin/Contrib>
     with the following structure;
      twikiplugins/
         <TWikiExtension>/
            data/
               TWiki/
                  <TWikiExtension>.txt
            lib/
               TWiki/
                  [Plugin|Contrib]/
                     <TWikiExtension>.pm
                     <TWikiExtension>/
                        build.pl
            pub/
               TWiki/
                  <TWikiExtension>/

      If the -contrib option is present, the directory under lib/TWiki will
      be Contrib instead of Plugin.

i.e. commands 'apache' and 'build' got added to the shell when we did the corresponding input. All these command sets are stored in separate packages allowing TWikiContributors to add commands the same way we add plugins.

Some nice functionality...

twiki > help plugin develop

SYNOPSIS
     plugin develop <Plugin/Contrib> - Copies the files for the Plugin/Contrib i
nto the twiki root for development

DESCRIPTION
     This command will copy all the related files for a Plugin/Contrib
     from the ${TWIKIROOT}/twikiplugins directory to the proper place
     under the ${TWIKIROOT} directory, while creating a manifest file
     with all the files copied.
     This is an alternative to the =mklinks -copy=  command, with the
     added value that it creates a manifest file that can be used by
     the Package CommandSet or the BuildContrib based =build.pl=
     script to create a release version.

EXAMPLE
     twikishell plugin develop TWikiShellContrib

        Will copy all the files from twikiplugins/TWihiShellContrib to
        their proper place and create the TWikiShellContrib.MF file
        under ${TWIKIROOT}.

-- MartinCleaver - 04 Oct 2005

twiki > plugin develop TWikiShellContrib
copying DEPENDENCIES to C:/moreprgs/indigoperl/apache/TWiki/TWikiShellContrib.DEP
copying Conf.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/Apache/Conf.pm
copying Httpd.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/Apache/Httpd.pm
copying Apache.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/Apache.pm
copying Build.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/Build.pm
copying Dump.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/Dump.pm
copying Package.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/Package.pm
copying build.pl_template to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/Plugin/Create/build.pl_template
copying Create.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/Plugin/Create.pm
copying Develop.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/Plugin/Develop.pm
copying PutBack.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/Plugin/PutBack.pm
copying Plugin.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/Plugin.pm
copying Conf.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/RunTest/Conf.pm
copying RunTest.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/RunTest.pm
copying Conf.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/TWiki/Conf.pm
copying TWiki.pm to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/CommandSet/TWiki.pm
copying build.pl to C:/moreprgs/indigoperl/apache/TWiki/lib/TWiki/Contrib/TWikiShellContrib/build.pl
copying DEPENDENCIES to C:/moreprgs/indigoperl/apache/TWiki/TWikiShellContrib.DEP
copying MANIFEST to C:/moreprgs/indigoperl/apache/TWiki/TWikiShellContrib.MF
copying MANIFEST to C:/moreprgs/indigoperl/apache/TWiki/TWikiShellContrib.MF
copying twikishell to C:/moreprgs/indigoperl/apache/TWiki/twikishell
copying TWikiShellContrib.MF to C:/moreprgs/indigoperl/apache/TWiki/TWikiShellContrib.MF

-- MartinCleaver - 04 Oct 2005

By the way, these are all listed in TWiki::Contrib::CommandSet:: from the package 'TWikiShellContrib'...

Raf: *Does build system will import lib files not matching 'CommandSet' ?

twiki$ sh mklinks.sh TWikiShellContrib
twiki$ ls
AUTHORS               data        pub                  test
COPYING               index.html  pub-htaccess.txt     tools
COPYRIGHT             lib         readme.txt           twiki_httpd_conf.txt
LICENSE               locale      robots.txt           twikiplugins
SubversionReadme.txt  logs        root-htaccess.txt
UpgradeTwiki          mklinks.pl  subdir-htaccess.txt
bin                   mklinks.sh  templates
twiki$ more lib/TWiki/Contrib/
.svn               TWikiShellContrib

Note Raf that the Contrib/CommandSet directory is missing, because mklinks.sh didn't use the MANIFEST to bring it in. This is a question of Spec, should a plugin be able to write to a place outside its hierarchy?

-- MartinCleaver - 04 Oct 2005

Martin, I'm completely lost here

If you're proposing to release in a topic the same documentation that appear in the "help" command, I agree and will try to automate that.

twikishell is not using pod2docs, as I don't want to care about the formatting of the pods (and wan't to leave the pods for technical docs). It's using the custom module Help.pm that allows to build man-like help pages.As you see, the result is very nice (for unix people, at least ;))

Another thing is that all the modules in TWiki::Contrib::CommandSet (from anywhere in the @INC) are imported automatically. And the import command should import any module.

-- RafaelAlvarez - 04 Oct 2005

There is no limitation about what a plugin can distribute, as long as the core modules are not modified.

-- RafaelAlvarez - 04 Oct 2005

Thanks Rafael for updating the Plugin. I made some changes to the Plugin topic. Please take this into the next release if you find a chance:

• Escaped many WikiWords
• Small fix in SHORTDESCRIPTION
• Fixed date format in history to international date
• Fixed cell align to right align in info table

Look at the diff to see the changes.

-- PeterThoeny - 28 Feb 2006

Peter, you never clearly documented your TWiki topic formatting guidelines. You always find something, even if new Plugin content is based on the (new) Plugin template. Wouldn't it be more effective to have a clear topic content quality guideline that can be used by community members like Lynnwood, Meredith and myself to do the basic (layout) cleanup while you and others concentrate on content.

For example: Why is it so important to escape all those WikiWords? Isn't that what makes the Wiki part in TWiki?

-- FranzJosefSilli - 01 Mar 2006

The Plugin template gives basically the standard topic layout. The TWikiDocsStyleGuide has guidelines in general (needs to be updated though)

In general, crosslinked content it is good to find related content. But there is no value in linking to the same topic over and over again, linking once or twize in a topic is fine (to give a nice flow in reading).

-- PeterThoeny - 01 Mar 2006

I'm surprised no one has thought of including rename support as a CommandSet. After migration from phpWiki I have a bunch of topics I need to move into new webs and haven't found a good way to do this yet.

-- IanTegebo - 15 Mar 2006

Rafael/Martin: The fixes I did got lost. I re-inserted them to the contrib topic and to the CommandSet topic. Please feel free to take this into the next release.

-- PeterThoeny - 20 May 2006

If the got lost is beause I decided not to include those fixes in the shipped topics.

-- RafaelAlvarez - 20 May 2006

This is certainly up to you. I am just trying to help make TWiki.org and the software more customer focused and more professional. Without my suggested changes:

• Users visiting ContribPackage are left in the dark what the TWikiShellContrib does (missing SHORTDESCRIPTION)
• Using heading 1 more than once is not consistent with other contribs and the doc style guidelines (see NewContribTemplate and TWikiDocsStyleGuide)
• Linking to oneself is confusing to the users (see Jakob Nielsen's Alertbox)
• Code snippets, text to be entered by the user is not highlighted with monospaced text (see TWikiDocsStyleGuide)
• Not using Interwiki links for other packages might break links on sites where the contrib is installed
• Not escaping < and > results in incomplete/confusing documentation
• User ends up in non-existing page: TWiki:Main.TWikiShell
• User is not giving a chance to appraise the contrib

-- PeterThoeny - 20 May 2006

My mistake. I though the only fixes where the "no linking to oneself" ones.

I'll ship the fixed version in a future release.

-- RafaelAlvarez - 25 May 2006

Ian: thanks for the suggestion. There are indeed many such tools that would be well suited to command line access.

Also please do let us know if you have any problems writing a command set.

-- MartinCleaver - 26 May 2006

TWikiShellContrib is not working here. After I run twikishell, every command I try to run (help, quit, import, ...) gives me "Ambiguous command" message. Ex:

twiki > help
Ambiguous command 'help': possible commands:
        help

Any ideas?

-- GuilhermeGarnier - 10 Jan 2008

I never tested it against TWiki 4.x.

I would have loved feedback while I was giving it effort. Nowadays unless someone can pay me to look at it you are pretty much on your own, I'm sorry to say.

-- MartinCleaver - 09 May 2008