refactoring in progress

There are ways to run arbitrary Linux or other commands on a (remote) web server from your browser by:

• creating a cgi script with the desired commands
• using a tool like CGI-Telnet

If you run a TWiki on a web hosting site where you do not have root access (like SourceForge) it can be very helpful to take advantage of these capabilities. (Even so, some things can be done only (or primarily) by an administrator.)

The possibility of using suid, sgid, and sudo may increase the capabilities.

These approaches do pose security risks -- make sure you understand and deal with them (not discussed on this page so far).

Key words: Linux, commands, command line, TWiki, CGI-Telnet,

See AboutThesePages and LinuxCommandsOnTWikiNotes.

Aside: Another problem running a TWiki on SourceForge is that without root access I cannot modify the Apache configuration files -- that will become a separate topic.

Contents

Objective

Learn to:

• write and use a CGI script to execute arbitrary Linux commands
• use CGI-Telnet
• recognize and deal with the associated security risks

Learn to create scripts to:

• Create new webs (directories)
• Set the permissions and ownership of files and directories (I have to run as root?? or sudo, or setuid, or something??)
• Do mass edits, like run sed to convert <H3Contents</H3> (ignore the <nop>s) to *Contents*.

Aside: One approach to changing the ownership of a file without root access is to run a program as the user that you'd like to be the owner of the file, then:

• copy the file to a new file name
• delete the original file
• rename the copy to the original file name

Script to Replace a Line on All TWiki Pages

sed/bash Command to Make Global Replacements

A sed/bash script to replace <H3>Contents</H3> (ignore the <nop>s) with *Contents*. Tested from the CLI -- note that I escaped the "/" and "*" characters:

for f in *.txt; do sed 's/<H3>Contents<\/H3>/\*Contents\*/' $f > x; mv x $f; done

Question: Will running a "script" like the above change the owner of the file to the user running the script? IIUC, yes, unless the script is suid'd.

But, that is not all bad -- if I run it as a cgi program then it will run as the Apache user and the modified files will be owned by the Apache user. This is not the ideal, IIRC.

I would like to either have the files:

• owned by the Apache user, but also be "owned" by my SourceForge group, and then have permissions like 664 (for .txt and .txt,v files).
• owned by the Apache user (and the Apache group) and then have permissions like 666 (or even 646).

Brainstorming: If I can get the CGI script to be owned by the Apache user and my SourceForge group, and with the suid and sgid bits set, maybe I can get exactly the results I want.

See SedUsage.

perl/bash Command to Make Global Replacements

$ perl -pi~ -e 's;#!/usr/bin/perl;#!c:/cygwin/bin/perl;' *[a-z]

Notes:

• Before starting, make a backup directory (mkdir <backup_dir>), and copy all the original files to that directory (cp * <backup_dir>).
• If for some reason the edit goes wrong, just type cp <backup_dir>/* . (while within the bin directory) to restore the original distribution files. Use ls -a to see the <backup_dir> directory, and ls -a <backup_dir> to view its contents.
• Optional step: you can do 'rm *~' to clean out the backups made by Perl, but that's not essential as all the original files cannot be executed. If you do this, type the command very carefully, as a space after the '*' will wipe out all files in this directory!

A Sample CGI/perl Script

The following copied from Codev.TWikiOnSourceForge and possibly edited:

     -----( start of file )-----
#!/usr/bin/perl
# quick and dirty hack to create directories owned by 'nobody'

use CGI;
use wiki;

&main();

sub main
{
    print "Content-type: text/html\n\n";
    print "TWiki test\n<p>";

    my $dataDir = &wiki::getDataDir();
    print "dataDir: |$dataDir|<br>\n";

#(uncomment and customize the next lines
#   mkNobodyDir( "$dataDir/dummy1" );
#   mkNobodyDir( "$dataDir/dummy2" );
#   ...

    print "End Twiki test\n";
}

# create dummy dir owned by nobody
sub mkNobodyDir
{
    my( $dir ) = @_;
    print "mkdir $dir <br>\n";
    `mkdir $dir`;
    `chmod 777 $dir`;
}

     -----( end of file )-----

Some notes (without trying the program, so far):

• AFAIK, the print statements just print some status information to your browser.
• AFAICT, the use wiki statement is so that the program can access twiki variables like $dataDir -- printing the $dataDir lets you confirm that its value is what you think it is.
• As written it sets the directory permissions to 777 -- I may want to experiment with this. "If you make the directories world writable (as the above script does), this doesn't give any advantage. If you don't make them world writable, you'll end up being unable to manipulate files in them from your shell account. In particular, you'll be unable to delete them, so you'd need another script that's the reverse of the above script. -- JoachimDurchholz - 07 Dec 2000"
• This is a nice sample cgi/perl script -- now I need to think about modifying it appropriately for my purposes.

Debugging

If the above script (or any Perl script) gives you troubles, you can redirect stderr to stdout. Do so at the beginning of your CGI script:

open(STDERR,'>&STDOUT'); # redirect error to browser
$| = 1;                  # no buffering
print "Content-type: text/html\n\n";

Or, look at the error_log of the web server. SourceForge has two load balanced servers, so you need to look at both of them:

tail -f /usr/local/log/oakenfold/errors_log
tail -f /usr/local/log/nirvana/errors_log

refactoring stopped here

---

Regarding files writable by user nobody : The easiest is to leave them as whatever user they are and to chmod 666 them. (Note that making files writable by everybody can be a security issue.) Text and RCS files are reset to user nobody the next time they are updated. (If you want the files made to be owned by user nobody you could write an other script that reads the files and recreates them one by one.)

-- PeterThoeny - 25 Jun 2000

---

One more "gotcha" with TWiki on SourceForge is running the statistics script that updates the WebStatistics topics in all webs. It should be run as user "nobody" from a cron job, which is not possible because we don't have root access. The statistics script runs also as a CGI script (it determines how it's run at run-time), and the geturl script can be used in a cron job to grab the bin/statistics page. The problem is that this does not work at SourceForge if done from inside (shell or cron job).

Solution: Use a free anonymizer that grabs the bin/statistics page from outside, e.g. the cron job requests a page from a anonymizer web site, which in turn grabs the page from SourceForge. A crutch, but it works.

Example crontab entry that updates the statistics of the fictious myprj project at 5 minutes passed midnight:

05 00 * * * (cd /home/groups/myprj/cgi-bin; ./geturl anon.free.anonymizer.com /http://myprj.SourceForge.net/cgi-bin/statistics >/dev/null 2>&1)

-- PeterThoeny - 04 Oct 2000

---

You don't need an anonymizer on SF. The reason why it doesn't work as expected is that sourceforge are using virtual hosting. So you need to specify the Host: header in the request. Here is the crontab entry:

2 1 * * * (cd /home/groups/m/my/myprj/cgi-bin/twiki; ./geturl myprj.sourceforge.net  /cgi-bin/twiki/statistics 80 'Host: myprj.sourceforge.net' >/dev/null 2>&1)

While this looks a bit clumsy, it works well. Might be worth to consider making the Host: header default in geturl, wouldn't hurt ease of use I think.

-- KlausRennecke - 26 Oct 2001

Another Hint

From TopicSaveErrorOnISPServer

In the shorter term, you'll need to look at TWikiOnSourceForge - one of PeterThoeny's comments of 25 Jun 2000 has a Perl CGI script that you can install to reset ownership etc - you may well require some customisation, e.g. to copy file foo to foo.bak, delete foo, then rename foo.bak to foo (thereby changing ownership of file foo to 'nobody' - on some modern Unixes you can't chown from another user to your user, hence the rigmarole). You'll need to do this on all the files installed, or perhaps just the *.txt,v files... There's a similar but more powerful/dangerous script from JoachimDurcholz on the same page that would let you run any command - sort of shell access from your browser's URL bar...

One of the first things you should do with this script is to do a find ... -print | xargs ls -ld and print the output to the web browser - you can then compare this with the TWikiDocumentation to see where the permissions are wrong. The Test directory must be writeable by the Apache userid, as must all the files (.txt and .txt,v).

Another good command to do is uname -a - this will tell you the version of Unix you are using, so you can find the online manual pages on the Web :)... Hopefully it's Linux as that's easy to install on a local PC for practice.

One SecureSetup tip - name the CGI script as if it was a password, i.e. twiki-do-nonobviouspassword - otherwise this is a big security hole (and make sure that nobody can browse your /cgi-bin/ directory to find the name!).

Permissions are 95% of the hassle in a new or moved TWiki installation... The above will require some CGI hacking unfortunately - unless you already know Perl and some Unix, switching web hosts will be much easier. Installing TWiki on a test box running Linux would probably help a lot, as you could test your CGI scripts there before uploading them.

-- RichardDonkin - 10 Feb 2002

Links

I'm sure this mentioned on other TWiki pages, here are the links I found today:

• TWikiDebugging
• Codev.TWikiDebugging#Surviving_without_Telnet_SSH
• http://www.rohitab.com/cgiscripts/cgitelnet.html
• http://twiki.org/cgi-bin/view/Codev/SourceForgeHowTo and search for [precious spare]

Contributors

• RandyKramer - 05 Apr 2002
• Stuff was borrowed from other TWiki pages with material contributed by:
  • RichardDonkin
  • PeterThoeny
• <If you edit this page, add your name here, move this to the next line>

Page Ratings