(r1) ExecutePlugin < TWiki

TWiki Web>ExecutePlugin (revision 1) (raw view)~~EditAttach~~
---+!! Execute Plugin
<!--
   Contributions to this plugin package are appreciated. Please update this page at
   http://twiki.org/cgi-bin/view/Plugins/ExecutePlugin or provide feedback at
   http://twiki.org/cgi-bin/view/Plugins/ExecutePluginDev.
   If you are a TWiki contributor please update the extension in the SVN repository.
-->
<sticky>
<div style="float:right; background-color:#EBEEF0; margin:0 0 20px 20px; padding: 0 10px 0 10px;">
%TOC{title="Page contents"}%
</div>
<div style="float:right; width:530px; margin:0 0 20px 20px; padding:0 10px 0 10px;">
<img src="%ATTACHURLPATH%/execute-diagram.png" alt="execute-diagram" width="530" height="256" />
</div>
</sticky>
%SHORTDESCRIPTION%

---++ Introduction

Use this plugin to integrate TWiki with other applications. TWiki serves as the user interface to interact with external applications, such as:

   * QA: Show test results in TWiki topics
   * License management: Generate, list and invalidate licenses

A special purpose plugin could be created for each use case. This plugin offers a generic API, where you configure an integration once, then create TWiki topics that call external scripts using =%<nop>EXECUTE{ "some_script" some_param="some value" }<nop>%=. Those topics make up the user interface of the external application. ="some_script"= is a script that is defined in configure, typically with full path name and some parameters. Additional parameters can be supplied to, and used by the script as needed.

Alternatively, Perl code can be safely executed in the same manner.

#SyntaxRules
---++ Syntax Rules

This plugin handles the =%<nop>EXECUTE{"..."}%= variable. The default ="..."= parameter determines the script to execute as defined in configure - see [[#ExecuteShellScripts][Execute Shell Scripts]] and [[#ExecutePerlCode][Execute Perl Code]] below.

%INCLUDE{ "VarEXECUTE" section="parameters" }%

Example: =%<nop>EXECUTE{ "qa_results" date="%<nop>URLPARAM{date}%" }%=

#SecurityConsiderations
---++ Security Considerations

Special care needs to be taken in a web environment when executing scripts on a server with user supplied input. It is easy to introduce a vulnerability where a server may be compromised if user input is not filtered properly. This plugin is designed with security in mind.

Let's first look into an integration that seems safe, but that can easily be exploited. Say, you have a test environment with nightly builds and want to list test result files below a specific directory called test-results. Example command line to list test results of %SERVERTIME{$year-$mo-$day}%:

=ls /export/test-results/%SERVERTIME{$year-$mo-$day}%=

Now let's make the test result date a user supplied input, where a TWiki topic contains a form to enter a date. The topic also contains the call to list the files: =%<nop>EXECUTE{ "test_results" date="%<nop>URLPARAM{date}<nop>%" }<nop>%=. Assuming =test_results= is defined as =ls /export/test-results/%date%=, we now can list the files in any sub-directory of =/export/test-results= using =date= as a user supplied input.

Seems safe? Not really. Two exploits can be done easily:

   * Supplied =date= is =../../etc= - we go up to the top to see the content of the =/etc= directory
   * Supplied =date= is =%SERVERTIME{$year-$mo-$day}%; rm -rf /= - we list the directory content, then we delete all files accessible by the webserver user!

To avoid vulnerabilities like this we need to filter all user supplied input.

The scripts are defined in TWiki's [[%SCRIPTURL{configure}%][configure]] script. Special care needs to be taken to secure the configure script.

#ExecuteShellScripts
---++ Execute Shell Scripts

To safely execute shell scripts we restrict the execution to a defined list of scripts, and we define the parameters allowed for each script.

Scripts with parameters are defined in TWiki's [[%SCRIPTURL{configure}%][configure]] script. In configure, open up the Extensions section, and find the ={Plugins}{ExecutePlugin}{Scripts}= setting of the !ExecutePlugin. The setting is an array of hashes. Each array item defines a script. A script is defined by a hash with these keys:

   * =name= - name of script, such as ='test_results'=
   * =type= - script type, set to ='script'= for shell scripts, ='perl'= for Perl scripts
   * =command= - script to execute, with parameters enclosed in percent signs, such as ='ls /export/test-results/%date%'=
   * =filter= - a %SYSTEMWEB%.RegularExpression filter applied to user supplied parameters, such as ='[^0-9\\-]'=

Once defined in configure, a script can be executed safely by name in a TWiki topic using =%<nop>EXECUTE{ "name" }<nop>%=. Parameters defined in the command can be supplied as EXECUTE parameters.

__Note:__ Shell scripts produce a standard text output (STDOUT), and error messages (STDERR) if any. By default you will get STDOUT, and STDERR will go to the webserver's error log. To control STDOUT and STDOUT in a Linux environment:

   * Capture a command's STDERR and STDOUT together:
     %BR% =command => 'some-script 2>&1',=
   * Capture a command's STDOUT but discard its STDERR:
     %BR% =command => 'some-script 2>/dev/null',=
   * Capture a command's STDERR but discard its STDOUT (ordering is important here):
     %BR% =command => 'some-script 2>&1 1>/dev/null',=

---+++ Example

__1. Define script:__

First we define the script named =test_results= in configure:

<verbatim>
$TWiki::cfg{Plugins}{ExecutePlugin}{Scripts} = [
 {
  name => 'test_results',
  type => 'script',
  command => 'ls /export/test-results/%date%',
  filter => '[^0-9\\-]',
 },
];
</verbatim>

__2. Execute script:__

We can now execute the script in a TWiki topic. Add the EXECUTE variable with these parameters to a TWiki topic:

=%<nop>EXECUTE{ "test_results" date="%<nop>URLPARAM{date}<nop>%" }<nop>%=

Assuming the URL parameter =date= is =%SERVERTIME{$year-$mo-$day}%=, the following command:

=ls /export/test-results/%date%=

is parameter-substituted and executed as:

=ls /export/test-results/%SERVERTIME{$year-$mo-$day}%=

This command is safe to execute because all characters but ='0-9'= and ='-'= are filtered out from the user supplied =date= parameter.

#ExecutePerlCode
---++ Execute Perl Code

Executing Perl code is similar to executing shell scripts. Perl code snippets with parameters are defined in TWiki's [[%SCRIPTURL{configure}%][configure]] script in the ={Plugins}{ExecutePlugin}{Scripts}= setting. A code snippet is defined by a hash with these keys:

   * =name= - name of script, such as ='qa_result'=
   * =type= - script type, set to ='perl'=
   * =command= - Perl code to execute, with parameters enclosed in percent signs, such as ='require qa_tests.pl; my $date = "%date%"; return qa_result($date);'=
   * =filter= - a %SYSTEMWEB%.RegularExpression filter applied to user supplied parameters, such as ='[^0-9\\-]'=

Once defined in configure, Perl code can be executed safely by name in a TWiki topic using =%<nop>EXECUTE{ "name" }<nop>%=. Parameters defined in the command can be supplied as EXECUTE parameters.

__Note:__ In case of an error, a detailed error message is returned starting with =EXECUTE ERROR:=.

---+++ Example

__1. Define script:__

First we define the script named =qa_results= in configure. We add a new definition to the existing ='test_results'= used in the previous example:

<verbatim>
$TWiki::cfg{Plugins}{ExecutePlugin}{Scripts} = [
 {
  name => 'test_results',
  type => 'script',
  command => 'ls /export/test-results/%date%',
  filter => '[^0-9\\-]',
 },
 {
  name => 'qa_results',
  type => 'perl',
  command => 'require qa_tests.pl; my $date = "%date%"; return qa_result($date);',
  filter => '[^0-9\\-]',
 },
];
</verbatim>

The =command= contains the Perl code snippet. The last statement should return the result; don't use print statements.

__2. Execute script:__

We can now execute the script in a TWiki topic. Add the EXCUTE variable with these parameters to a TWiki topic:

=%<nop>EXECUTE{ "qa_results" date="%<nop>URLPARAM{date}<nop>%" }<nop>%=

Assuming the URL parameter =date= is =%SERVERTIME{$year-$mo-$day}%=, the following Perl code:

=require qa_tests.pl; my $date = "%date%"; return qa_result($date);=

is parameter-substituted and executed as:

=require qa_tests.pl; my $date = "%SERVERTIME{$year-$mo-$day}%"; return qa_result($date);=

This code is safe to execute because all characters but ='0-9'= and ='-'= are filtered out from the user supplied =date= parameter.

#InteractiveExample
---++ Interactive Example

This interactive example changes a string into UPPERCASE letters. This works if the plugin is enabled and configured with an "uppercase" script as described below.

<form action="%SCRIPTURL{view}%/%WEB%/%TOPIC%#InteractiveExample">
| Input: | <input type="text" name="text" value="%URLPARAM{ "text" encode="html" }%" size="60" class="twikiInputField" />\
 <input type="hidden" name="uppercase_action" value="uppercase" />\
 <input type="submit" value="Submit" class="twikiSubmit" /> |
| Result: | %EXECUTE{ "%URLPARAM{uppercase_action}%" text="%URLPARAM{ "text" encode="quote" }%" }% |
</form>

__Configuration and explanation:__

First we define a Perl script that does the uppercase conversion. Run the [[%SCRIPTURL{configure}%][configure]] script and add the following !ExecutePlugin setting in the __Extensions__ section:

<verbatim>
$TWiki::cfg{Plugins}{ExecutePlugin}{Scripts} = [
 {
  name => 'uppercase',
  type => 'perl',
  command => 'return uc("%text%");',
  filter => '[^a-zA-Z0-9_\\-\\+ \\.\\,\\:\\!\\?\\(\\)/]',
 },
];
</verbatim>

It simply takes a =text= parameter as the input to the =uc()= function and returns the result.

Above HTML form is defined as follows:

<verbatim>
<form action="%SCRIPTURL{view}%/%WEB%/%TOPIC%#InteractiveExample">
| Input: | <input type="text" name="text" value="%URLPARAM{ "text" encode="html" }%" size="60" class="twikiInputField" />\
 <input type="hidden" name="ucaction" value="uppercase" />\
 <input type="submit" value="Submit" class="twikiSubmit" /> |
| Result: | %EXECUTE{
 "%URLPARAM{ucaction}%"
 text="%URLPARAM{ "text" encode="quote" }%"
}% |
</form>
</verbatim>

The form action is the URL of this topic. It also has an =#InteractiveExample= anchor so that we conveniently end up at the form after a submit. The form has two input elements: A visible text field named =text=, and a hidden field named =ucaction=. The field values can be queried with the URLPARAM variable after the form submit.

The EXECUTE variable uses two URLPARAM variables. The first one, ="%<nop>URLPARAM{ucaction}%"=, is used for the script name. The URL parameter value is empty before the form submit, e.g. EXECUTE will not do any action. The value will be ="uppercase"= after form submit, thus triggering the uppercase script. The second URL parameter is used to set the =text= parameter value. Let's say, the user submits text ="abc"=. We will execute the following after form submit: =%<nop>EXECUTE{ "uppercase" text="abc" }%=, resulting in output =ABC=.

#PluginDisclaimer
---++ Disclaimer

An incorrect configuration of a script may introduce a vulnerability. Special care needs to be taken when configuring scripts with user supplied parameters - dangerous characters, which may be OS dependent, need to be filtered out. Configuration errors may result in a vulnerable system, especially if %SYSTEMWEB%.RegularExpression filters are not well understood, or if special characters are not filtered out properly.

This plugin has been designed with security in mind, and it has been tested. However, an unforeseen vulnerability may be found in the future. This plugin does not have ANY WARRANTY, does not even have the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

---++ Plugin Installation &amp; Configuration

You do not need to install anything on the browser to use this plugin. These instructions are for the administrator who installs the plugin on the TWiki server.

%TWISTY{
 mode="div"
 showlink="Show details %ICONURL{toggleopen}% "
 hidelink="Hide details %ICONURL{toggleclose}% "
}%

   * For an __automated installation__, run the [[%SCRIPTURL{configure}%][configure]] script and follow "Find More Extensions" in the in the __Extensions__ section.
      * See the [[http://twiki.org/cgi-bin/view/Plugins/BuildContribInstallationSupplement][installation supplement]] on TWiki.org for more information.

   * Or, follow these __manual installation__ steps:
      * Download the ZIP file from the Plugins home (see below).
      * Unzip ==%TOPIC%.zip== in your twiki installation directory. Content:
        | *File:* | *Description:* |
        | ==data/TWiki/ExecutePlugin.txt== | Plugin topic |
        | ==data/TWiki/VarEXECUTE.txt== | Variable documentation topic |
        | ==pub/TWiki/ExecutePlugin/*.gif== and ==*.png== | Image files |
        | ==lib/TWiki/Plugins/ExecutePlugin.pm== | Plugin Perl module |
      * Set the ownership of the extracted directories and files to the webserver user.

   * Plugin configuration:
      * Run the [[%SCRIPTURL{configure}%][configure]] script and enable the plugin in the __Plugins__ section.

   * Test if the configuration is successful:
      * Follow the [[#InteractiveExample][Interactive Example]] above.
      * Alternatively, do this test:
         * Run the [[%SCRIPTURL{configure}%][configure]] script and add the following !ExecutePlugin setting in the __Extensions__ section:
        <verbatim>{Plugins}{ExecutePlugin}{Scripts} = [
     {
      name => 'echo',
      type => 'script',
      command => '/bin/echo "%text%"',
      filter => '[^a-zA-Z0-9_\\- ]',
     },
    ]</verbatim>
            * __Note:__ This configuration is Linux specific.
         * Create a topic and add this text:
           %BR% =%<nop>EXECUTE{ "echo" text="Hello world!" }%=
         * Expected output:
           %BR% =Hello world=

%ENDTWISTY%

#PluginInfo
---++ Plugin Info

   * Set SHORTDESCRIPTION = Safely execute shell scripts to use TWiki as the user interface for external applications

%TABLE{ tablewidth="100%" columnwidths="170," }%
|  Author: | TWiki:Main.PeterThoeny, [[http://twiki.org/][TWiki.org]] |
|  Copyright: | &copy; 2014 Alba Power Quality Solutions. %BR% &copy; 2014-2016 TWiki:Main.PeterThoeny %BR% &copy; 2014-2016 TWiki:TWiki.TWikiContributor |
|  License: | GPL ([[http://www.gnu.org/copyleft/gpl.html][GNU General Public License]]) |
|  Sponsor: | Alba Power Quality Solutions |
|  Version: | 2016-01-07 |
%TWISTY{
 mode="div"
 showlink="Show Change History %ICONURL{toggleopen}%"
 hidelink="Hide Change History %ICONURL{toggleclose}% "
}%
%TABLE{ tablewidth="100%" columnwidths="170," }%
|  2016-01-07: | TWikibug:Item7580: Fixed documentation from ={Commands}= to ={Scripts}= setting |
|  2014-10-22: | TWikibug:Item7580: Document STDOUT and STDERR |
|  2014-10-21: | TWikibug:Item7580: Show Perl code error; untaint command before execution; add interactive example |
|  2014-10-20: | TWikibug:Item7580: Initial version of !ExecutePlugin |
%ENDTWISTY%
%TABLE{ tablewidth="100%" columnwidths="170," }%
|  TWiki Dependency: | $TWiki::Plugins::VERSION 1.2 |
|  CPAN Dependencies: | none |
|  Other Dependencies: | none |
|  Perl Version: | 5.008 |
|  [[TWiki:Plugins.Benchmark][Plugin Benchmark]]: | %SYSTEMWEB%.GoodStyle nn%, %SYSTEMWEB%.FormattedSearch nn%, %TOPIC% nn% |
|  Home: | http://TWiki.org/cgi-bin/view/Plugins/ExecutePlugin |
|  Feedback: | http://TWiki.org/cgi-bin/view/Plugins/ExecutePluginDev |
|  Appraisal: | http://TWiki.org/cgi-bin/view/Plugins/ExecutePluginAppraisal |

__Related Topics:__ VarEXECUTE, %SYSTEMWEB%.TWikiPlugins
Topic revision: r1 - 2024-03-02 - TWikiAdminUser
TWiki
Webs
Blog
TWiki
ZMobile
Copyright © 1999-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
Note: Please contribute updates to this topic on TWiki.org at TWiki:TWiki.ExecutePlugin.