Tags:
admin_tool1Add my vote for this tag automation1Add my vote for this tag component1Add my vote for this tag integration1Add my vote for this tag twiki_application1Add my vote for this tag user_interface1Add my vote for this tag workflow1Add my vote for this tag create new tag
, view all tags

Execute Plugin

execute-diagram
Safely execute shell scripts to use TWiki as the user interface for external applications

Introduction

Use this plugin to integrate TWiki with other applications. TWiki serves as the user interface to interact with external applications, such as:

  • QA: Show test results in TWiki topics
  • License management: Generate, list and invalidate licenses

A special purpose plugin could be created for each use case. This plugin offers a generic API, where you configure an integration once, then create TWiki topics that call external scripts using %EXECUTE{ "some_script" some_param="some value" }%. Those topics make up the user interface of the external application. "some_script" is a script that is defined in configure, typically with full path name and some parameters. Additional parameters can be supplied to, and used by the script as needed.

Alternatively, Perl code can be safely executed in the same manner.

Syntax Rules

This plugin handles the %EXECUTE{"..."}% variable. The default "..." parameter determines the script to execute as defined in configure - see Execute Shell Scripts and Execute Perl Code below.

Parameter Explanation Default Example
"name" Name of script to execute, as defined in configure. (none) "test-results"
format="..." Format each line of the script output. The token $text expands to the text of a line. Additional FormatTokens can be used, such as $percnt for the percent sign. "$text" format="| $text |"
newline="..." Change newlines of the script output to something else. FormatTokens can be used, such as $n for newline. "$n" newline=", "
...="..." Any other parameter can be used as parameter to the script. (none) date="2016-05-09"

Example: %EXECUTE{ "qa_results" date="%URLPARAM{date}%" }%

Security Considerations

Special care needs to be taken in a web environment when executing scripts on a server with user supplied input. It is easy to introduce a vulnerability where a server may be compromised if user input is not filtered properly. This plugin is designed with security in mind.

Let's first look into an integration that seems safe, but that can easily be exploited. Say, you have a test environment with nightly builds and want to list test result files below a specific directory called test-results. Example command line to list test results of 2016-05-09:

ls /export/test-results/2016-05-09

Now let's make the test result date a user supplied input, where a TWiki topic contains a form to enter a date. The topic also contains the call to list the files: %EXECUTE{ "test_results" date="%URLPARAM{date}%" }%. Assuming test_results is defined as ls /export/test-results/%date%, we now can list the files in any sub-directory of /export/test-results using date as a user supplied input.

Seems safe? Not really. Two exploits can be done easily:

  • Supplied date is ../../etc - we go up to the top to see the content of the /etc directory
  • Supplied date is 2016-05-09; rm -rf / - we list the directory content, then we delete all files accessible by the webserver user!

To avoid vulnerabilities like this we need to filter all user supplied input.

The scripts are defined in TWiki's configure script. Special care needs to be taken to secure the configure script.

Execute Shell Scripts

To safely execute shell scripts we restrict the execution to a defined list of scripts, and we define the parameters allowed for each script.

Scripts with parameters are defined in TWiki's configure script. In configure, open up the Extensions section, and find the {Plugins}{ExecutePlugin}{Scripts} setting of the ExecutePlugin. The setting is an array of hashes. Each array item defines a script. A script is defined by a hash with these keys:

  • name - name of script, such as 'test_results'
  • type - script type, set to 'script' for shell scripts, 'perl' for Perl scripts
  • command - script to execute, with parameters enclosed in percent signs, such as 'ls /export/test-results/%date%'
  • filter - a RegularExpression filter applied to user supplied parameters, such as '[^0-9\\-]'

Once defined in configure, a script can be executed safely by name in a TWiki topic using %EXECUTE{ "name" }%. Parameters defined in the command can be supplied as EXECUTE parameters.

Note: Shell scripts produce a standard text output (STDOUT), and error messages (STDERR) if any. By default you will get STDOUT, and STDERR will go to the webserver's error log. To control STDOUT and STDOUT in a Linux environment:

  • Capture a command's STDERR and STDOUT together:
    command => 'some-script 2>&1',
  • Capture a command's STDOUT but discard its STDERR:
    command => 'some-script 2>/dev/null',
  • Capture a command's STDERR but discard its STDOUT (ordering is important here):
    command => 'some-script 2>&1 1>/dev/null',

Example

1. Define script:

First we define the script named test_results in configure:

$TWiki::cfg{Plugins}{ExecutePlugin}{Scripts} = [
 {
  name => 'test_results',
  type => 'script',
  command => 'ls /export/test-results/%date%',
  filter => '[^0-9\\-]',
 },
];

2. Execute script:

We can now execute the script in a TWiki topic. Add the EXECUTE variable with these parameters to a TWiki topic:

%EXECUTE{ "test_results" date="%URLPARAM{date}%" }%

Assuming the URL parameter date is 2016-05-09, the following command:

ls /export/test-results/%date%

is parameter-substituted and executed as:

ls /export/test-results/2016-05-09

This command is safe to execute because all characters but '0-9' and '-' are filtered out from the user supplied date parameter.

Execute Perl Code

Executing Perl code is similar to executing shell scripts. Perl code snippets with parameters are defined in TWiki's configure script in the {Plugins}{ExecutePlugin}{Scripts} setting. A code snippet is defined by a hash with these keys:

  • name - name of script, such as 'qa_result'
  • type - script type, set to 'perl'
  • command - Perl code to execute, with parameters enclosed in percent signs, such as 'require qa_tests.pl; my $date = "%date%"; return qa_result($date);'
  • filter - a RegularExpression filter applied to user supplied parameters, such as '[^0-9\\-]'

Once defined in configure, Perl code can be executed safely by name in a TWiki topic using %EXECUTE{ "name" }%. Parameters defined in the command can be supplied as EXECUTE parameters.

Note: In case of an error, a detailed error message is returned starting with EXECUTE ERROR:.

Example

1. Define script:

First we define the script named qa_results in configure. We add a new definition to the existing 'test_results' used in the previous example:

$TWiki::cfg{Plugins}{ExecutePlugin}{Scripts} = [
 {
  name => 'test_results',
  type => 'script',
  command => 'ls /export/test-results/%date%',
  filter => '[^0-9\\-]',
 },
 {
  name => 'qa_results',
  type => 'perl',
  command => 'require qa_tests.pl; my $date = "%date%"; return qa_result($date);',
  filter => '[^0-9\\-]',
 },
];

The command contains the Perl code snippet. The last statement should return the result; don't use print statements.

2. Execute script:

We can now execute the script in a TWiki topic. Add the EXCUTE variable with these parameters to a TWiki topic:

%EXECUTE{ "qa_results" date="%URLPARAM{date}%" }%

Assuming the URL parameter date is 2016-05-09, the following Perl code:

require qa_tests.pl; my $date = "%date%"; return qa_result($date);

is parameter-substituted and executed as:

require qa_tests.pl; my $date = "2016-05-09"; return qa_result($date);

This code is safe to execute because all characters but '0-9' and '-' are filtered out from the user supplied date parameter.

Interactive Example

This interactive example changes a string into UPPERCASE letters. This works if the plugin is enabled and configured with an "uppercase" script as described below.

Input:
Result: %EXECUTE{ "" text="" }%

Configuration and explanation:

First we define a Perl script that does the uppercase conversion. Run the configure script and add the following ExecutePlugin setting in the Extensions section:

$TWiki::cfg{Plugins}{ExecutePlugin}{Scripts} = [
 {
  name => 'uppercase',
  type => 'perl',
  command => 'return uc("%text%");',
  filter => '[^a-zA-Z0-9_\\-\\+ \\.\\,\\:\\!\\?\\(\\)/]',
 },
];

It simply takes a text parameter as the input to the uc() function and returns the result.

Above HTML form is defined as follows:

<form action="%SCRIPTURL{view}%/%WEB%/%TOPIC%#InteractiveExample">
| Input: | <input type="text" name="text" value="%URLPARAM{ "text" encode="html" }%" size="60" class="twikiInputField" />\
 <input type="hidden" name="ucaction" value="uppercase" />\
 <input type="submit" value="Submit" class="twikiSubmit" /> |
| Result: | %EXECUTE{
 "%URLPARAM{ucaction}%"
 text="%URLPARAM{ "text" encode="quote" }%"
}% |
</form>

The form action is the URL of this topic. It also has an #InteractiveExample anchor so that we conveniently end up at the form after a submit. The form has two input elements: A visible text field named text, and a hidden field named ucaction. The field values can be queried with the URLPARAM variable after the form submit.

The EXECUTE variable uses two URLPARAM variables. The first one, "%URLPARAM{ucaction}%", is used for the script name. The URL parameter value is empty before the form submit, e.g. EXECUTE will not do any action. The value will be "uppercase" after form submit, thus triggering the uppercase script. The second URL parameter is used to set the text parameter value. Let's say, the user submits text "abc". We will execute the following after form submit: %EXECUTE{ "uppercase" text="abc" }%, resulting in output ABC.

Disclaimer

An incorrect configuration of a script may introduce a vulnerability. Special care needs to be taken when configuring scripts with user supplied parameters - dangerous characters, which may be OS dependent, need to be filtered out. Configuration errors may result in a vulnerable system, especially if RegularExpression filters are not well understood, or if special characters are not filtered out properly.

This plugin has been designed with security in mind, and it has been tested. However, an unforeseen vulnerability may be found in the future. This plugin does not have ANY WARRANTY, does not even have the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Plugin Installation & Configuration

You do not need to install anything on the browser to use this plugin. These instructions are for the administrator who installs the plugin on the TWiki server.

  • For an automated installation, run the configure script and follow "Find More Extensions" in the in the Extensions section.

  • Or, follow these manual installation steps:
    • Download the ZIP file from the Plugins home (see below).
    • Unzip ExecutePlugin.zip in your twiki installation directory. Content:
      File: Description:
      data/TWiki/ExecutePlugin.txt Plugin topic
      data/TWiki/VarEXECUTE.txt Variable documentation topic
      pub/TWiki/ExecutePlugin/*.gif and *.png Image files
      lib/TWiki/Plugins/ExecutePlugin.pm Plugin Perl module
    • Set the ownership of the extracted directories and files to the webserver user.

  • Plugin configuration:
    • Run the configure script and enable the plugin in the Plugins section.

  • Test if the configuration is successful:
    • Follow the Interactive Example above.
    • Alternatively, do this test:
      • Run the configure script and add the following ExecutePlugin setting in the Extensions section:
        {Plugins}{ExecutePlugin}{Scripts} = [
             {
              name => 'echo',
              type => 'script',
              command => '/bin/echo "%text%"',
              filter => '[^a-zA-Z0-9_\\- ]',
             },
            ]
        • Note: This configuration is Linux specific.
      • Create a topic and add this text:
        %EXECUTE{ "echo" text="Hello world!" }%
      • Expected output:
        Hello world

Plugin Info

  • Set SHORTDESCRIPTION = Safely execute shell scripts to use TWiki as the user interface for external applications

Author: TWiki:Main.PeterThoeny, TWiki.org
Copyright: © 2014 Alba Power Quality Solutions.
© 2014-2016 TWiki:Main.PeterThoeny
© 2014-2016 TWiki:TWiki.TWikiContributor
License: GPL (GNU General Public License)
Sponsor: Alba Power Quality Solutions
Version: 2016-01-07
2016-01-07: TWikibug:Item7580: Fixed documentation from {Commands} to {Scripts} setting
2014-10-22: TWikibug:Item7580: Document STDOUT and STDERR
2014-10-21: TWikibug:Item7580: Show Perl code error; untaint command before execution; add interactive example
2014-10-20: TWikibug:Item7580: Initial version of ExecutePlugin
TWiki Dependency: $TWiki::Plugins::VERSION 1.2
CPAN Dependencies: none
Other Dependencies: none
Perl Version: 5.008
Plugin Benchmark: GoodStyle nn%, FormattedSearch nn%, ExecutePlugin nn%
Home: http://TWiki.org/cgi-bin/view/Plugins/ExecutePlugin
Feedback: http://TWiki.org/cgi-bin/view/Plugins/ExecutePluginDev
Appraisal: http://TWiki.org/cgi-bin/view/Plugins/ExecutePluginAppraisal

Related Topics: VarEXECUTE, TWikiPlugins

Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatmd5 ExecutePlugin.md5 r9 r8 r7 r6 r5 manage 0.2 K 2016-01-08 - 01:51 PeterThoeny  
Compressed Zip archivetgz ExecutePlugin.tgz r9 r8 r7 r6 r5 manage 75.1 K 2016-01-08 - 01:51 PeterThoeny  
Compressed Zip archivezip ExecutePlugin.zip r9 r8 r7 r6 r5 manage 79.0 K 2016-01-08 - 01:51 PeterThoeny  
Unknown file formatEXT ExecutePlugin_installer r2 r1 manage 3.6 K 2016-01-08 - 01:51 PeterThoeny  
Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r5 - 2016-01-08 - PeterThoeny
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2016 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.