create new tag
, view all tags
This page contains the original notes as I start to refactor the LinuxCommandsOnTWiki page.

You can run arbitrary Linux commands on TWiki, basically by creating a script. Here are some resources, later I will summarize in my own words.

I've had difficulty finding this topic, so I'm adding some key words and phrases: Linux, commands, command line, TWiki, CGI-Telnet,

There are security implications to using CGI-Telnet -- make sure you understand them.

See AboutThesePages.



I started creating this page to collect links and notes related to performing operations on a TWiki at SourceForge for which I do not have root access (and cannot modify the Apache configuration files -- although maybe that will become a separate topic).

CGI-Telnet is a program that lets you run arbitrary Linux (host OS) commands and hence is a major security hole.

Another approach is to write a specific purpose script like PeterThoeny does somewhere in the TWikiInstallationNotes to create some directories as "precious spare".

I would want to create scripts to:

  • Create new webs (directories)
  • Set the permissions of files and directories (I have to run as root?? or sudo, or setuid, or something??)
  • Do mass edits, like run sed to convert <H3Contents</H3> (ignore the <nop>s) to *Contents*.

I've seen discussion of what might be a third approach for certain specific instances -- somebody suggested something like uploading a file, copying it, deleting the original, renaming the copy to the original file name in order to get a file with the required permissions. (I'm sure this description is not correct as I'm writing it from a vague recollection. Probably some of the steps have to be done using something like CGI-Telnet??) If I find the description again (or if someone else does) I'll put a link here or correct my description (probably both).)

Script to Replace a Line on All TWiki Pages

A Sample sed/bash Script

Changing the RCS "locker" on TWiki

for f in *,v; do sed 's/nobody\:/www-data\:/' $f > x; mv x $f; done

Revision to replace <H3>Contents</H3> (ignore the <nop>s) with *Contents* -- now tested -- note that I escaped the "/" and "*" characters:

for f in *.txt; do sed 's/<H3>Contents<\/H3>/\*Contents\*/' $f > x; mv x $f; done

Question: Will running a "script" like the above change the owner of the file to the user running the script? (I suspect so.)

If so, I should find a way to run it as the "Apache user" -- IIRC, there is a way to do that, something like running it as a cgi script -- maybe then it would have to be in Perl.

See SedUsage.

A Sample Perl Script

From: http://twiki.org/cgi-bin/view/Codev/WindowsInstallCookbook#Editing_the_CGI_scripts, (refactored?) by RichardDonkin?

4. Editing the Shebang lines

Now to edit the curiously named 'shebang lines' at the top of the TWiki CGI scripts...

  • You must use the Cygwin shell to do this (unless you are a Perl expert) - don't use the Windows command shell, cmd.exe (aka DOS Prompt)
  • Then do the following, which quickly edits the 19 or so files, using Perl - the important lines are in bold.
  • Type the Perl line very carefully
    • If you do mis-type the perl line, you can restore from the .backup directory and re-run the command, as it will only edit the original files, not the backups with '~' suffixes.

$ cd /twiki/bin

$ ls
attach   geturl         oops     rdiff     save        testenv  viewfile
changes  installpasswd  passwd   register  search      upload
edit     mailnotify     preview  rename    statistics  view

$ mkdir .backup 
$ cp * .backup

$ head -1 view
#!/usr/bin/perl -wT

$ perl -pi~ -e 's;#!/usr/bin/perl;#!c:/cygwin/bin/perl;' *[a-z]

$ head -1 view
#!c:/cygwin/bin/perl -wT

$ ls
attach    geturl          oops      rdiff      save         testenv   viewfile~
attach~   geturl~         oops~     rdiff~     save~        testenv~  view~
changes   installpasswd   passwd    register   search       upload
changes~  installpasswd~  passwd~   register~  search~      upload~
edit      mailnotify      preview   rename     statistics   view
edit~     mailnotify~     preview~  rename~    statistics~  viewfile

If for some reason the edit goes wrong, just type cp .backup/* . (while within the bin directory) to restore the original distribution files. Use ls -a to see the .backup directory, and ls -a .backup to view its contents.

Optional step: you can do 'rm *~' to clean out the backups made by Perl, but that's not essential as all the original files cannot be executed. If you do this, type the command very carefully, as a space after the '*' will wipe out all files in this directory!

A Sample CGI Script

The following copied from Codev.TWikiOnSourceForge and possibly edited:

     -----( start of file )-----
# quick and dirty hack to create directories owned by 'nobody'

use CGI;
use wiki;


sub main
    print "Content-type: text/html\n\n";
    print "TWiki test\n<p>";

    my $dataDir = &wiki::getDataDir();
    print "dataDir: |$dataDir|<br>\n";

#(uncomment and customize the next lines
#   mkNobodyDir( "$dataDir/dummy1" );
#   mkNobodyDir( "$dataDir/dummy2" );
#   ...

    print "End Twiki test\n";

# create dummy dir owned by nobody
sub mkNobodyDir
    my( $dir ) = @_;
    print "mkdir $dir <br>\n";
    `mkdir $dir`;
    `chmod 777 $dir`;

     -----( end of file )-----

After running the script I have the spare directories I can use now, i.e. to rename a spare to data , and to move files into it from the original location.

-- PeterThoeny - 25 Jun 2000

If you make the directories world writable (as the above script does), this doesn't give any advantage.

If you don't make them world writable, you'll end up being unable to manipulate files in them from your shell account. In particular, you'll be unable to delete them, so you'd need another script that's the reverse of the above script.

-- JoachimDurchholz - 07 Dec 2000

If the above script (or any Perl script) gives you troubles, you can redirect stderr to stdout. Do so at the beginning of your CGI script:

open(STDERR,'>&STDOUT'); # redirect error to browser
$| = 1;                  # no buffering
print "Content-type: text/html\n\n";

Or, look at the error_log of the web server. SourceForge has two load balanced servers, so you need to look at both of them:

tail -f /usr/local/log/oakenfold/errors_log
tail -f /usr/local/log/nirvana/errors_log

-- PeterThoeny - 15 Aug 2000

Regarding files writable by user nobody : The easiest is to leave them as whatever user they are and to chmod 666 them. (Note that making files writable by everybody can be a security issue.) Text and RCS files are reset to user nobody the next time they are updated. (If you want the files made to be owned by user nobody you could write an other script that reads the files and recreates them one by one.)

-- PeterThoeny - 25 Jun 2000

One more "gotcha" with TWiki on SourceForge is running the statistics script that updates the WebStatistics topics in all webs. It should be run as user "nobody" from a cron job, which is not possible because we don't have root access. The statistics script runs also as a CGI script (it determines how it's run at run-time), and the geturl script can be used in a cron job to grab the bin/statistics page. The problem is that this does not work at SourceForge if done from inside (shell or cron job).

Solution: Use a free anonymizer that grabs the bin/statistics page from outside, e.g. the cron job requests a page from a anonymizer web site, which in turn grabs the page from SourceForge. A crutch, but it works.

Example crontab entry that updates the statistics of the fictious myprj project at 5 minutes passed midnight:

05 00 * * * (cd /home/groups/myprj/cgi-bin; ./geturl anon.free.anonymizer.com /http://myprj.SourceForge.net/cgi-bin/statistics >/dev/null 2>&1)

-- PeterThoeny - 04 Oct 2000

You don't need an anonymizer on SF. The reason why it doesn't work as expected is that sourceforge are using virtual hosting. So you need to specify the Host: header in the request. Here is the crontab entry:

2 1 * * * (cd /home/groups/m/my/myprj/cgi-bin/twiki; ./geturl myprj.sourceforge.net  /cgi-bin/twiki/statistics 80 'Host: myprj.sourceforge.net' >/dev/null 2>&1)

While this looks a bit clumsy, it works well. Might be worth to consider making the Host: header default in geturl, wouldn't hurt ease of use I think.

-- KlausRennecke - 26 Oct 2001

Another Hint

From TopicSaveErrorOnISPServer

In the shorter term, you'll need to look at TWikiOnSourceForge - one of PeterThoeny's comments of 25 Jun 2000 has a Perl CGI script that you can install to reset ownership etc - you may well require some customisation, e.g. to copy file foo to foo.bak, delete foo, then rename foo.bak to foo (thereby changing ownership of file foo to 'nobody' - on some modern Unixes you can't chown from another user to your user, hence the rigmarole). You'll need to do this on all the files installed, or perhaps just the *.txt,v files... There's a similar but more powerful/dangerous script from JoachimDurcholz on the same page that would let you run any command - sort of shell access from your browser's URL bar...

One of the first things you should do with this script is to do a find ... -print | xargs ls -ld and print the output to the web browser - you can then compare this with the TWikiDocumentation to see where the permissions are wrong. The Test directory must be writeable by the Apache userid, as must all the files (.txt and .txt,v).

Another good command to do is uname -a - this will tell you the version of Unix you are using, so you can find the online manual pages on the Web :)... Hopefully it's Linux as that's easy to install on a local PC for practice.

One SecureSetup tip - name the CGI script as if it was a password, i.e. twiki-do-nonobviouspassword - otherwise this is a big security hole (and make sure that nobody can browse your /cgi-bin/ directory to find the name!).

Permissions are 95% of the hassle in a new or moved TWiki installation... The above will require some CGI hacking unfortunately - unless you already know Perl and some Unix, switching web hosts will be much easier. Installing TWiki on a test box running Linux would probably help a lot, as you could test your CGI scripts there before uploading them.

-- RichardDonkin - 10 Feb 2002


I'm sure this mentioned on other TWiki pages, here are the links I found today:


  • RandyKramer - 05 Apr 2002
  • <If you edit this page, add your name here, move this to the next line>

Page Ratings

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r2 - 2002-08-24 - PeterThoeny
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by PerlCopyright 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding WikiLearn? WebBottomBar">Send feedback
See TWiki's New Look