Tags:
security1Add my vote for this tag create new tag
view all tags

Security Alert: TWiki search function allows arbitrary shell command execution

ALERT! Please join the
twiki-announce list:
To get immediate alerts of high priority security issues, please join the low-volume twiki-announce list - details at TWikiAnnounceMailingList

Update 27 Sep 2005: Please upgrade to TWikiRelease04Sep2004 or apply patchs for SecurityAlertExecuteCommandsWithRev and SecurityAlertExecuteCommandsWithInclude. -- PeterThoeny - 27 Sep 2005

Vulnerable Software Version

Attack Vectors

HTTP GET requests towards the Wiki server (typically port 80/TCP). Usually, no prior authentication is necessary.

Possibly also HTTP POST, but this is untested.

Impact

An attacker is able to execute arbitrary shell commands with the privileges of the web server process, such as user nobody.

Details

The TWiki search function uses a user supplied search string to compose a command line executed by the Perl backtick (``) operator.

The search string is not checked properly for shell metacharacters and is thus vulnerable to search string containing quotes and shell commands.

An example search string would be: "test_vulnerability '; ls -la'"

If access to TWiki is not restricted by other means, attackers can use the search function without prior authentication.

See Also: SecurityAlertExecuteCommandsWithSearchHackReports

Countermeasures

  • Apply hotfix (see patches at end of this e-mail). The hotfix is known to prevent the current attacks, but it might not be a complete fix.
  • Upgrade to the latest patched production release TWikiRelease02Sep2004
  • Filter access to the web server.
  • Use the web server software to restrict access to the web pages served by TWiki.
  • Rewrite the code to use Perl code to open and scan the files instead of running fgrep in the shell. (slow)

Authors and Credits

Markus Goetz, Joerg Hoh, Michael Holzt, Florian Laws, Hans Ulrich Niedermann, Andreas Thienemann, Peter Thoeny, Florian Weimer, Colas Nahaboo contributed to this advisory.

Hotfix

Patch for twiki/lib/TWiki/Search.pm of TWiki Production Release 01-Sep-2004

*** TWiki20040901/Search.pm  2004-11-12 11:54:47.000000000 -0800
--- ./Search.pm 2004-11-12 12:08:29.000000000 -0800
***************
*** 434,439 ****
--- 434,446 ----
      my $tempVal = "";
      my $tmpl = "";
      my $topicCount = 0; # JohnTalintyre
+
+     # fix for Codev.SecurityAlertExecuteCommandsWithSearch
+     # vulnerability, search: "test_vulnerability '; ls -la'"
+     $theSearchVal =~ s/(^|[^\\])([\'\`])/\\$2/g;    # Escape ' and `
+     $theSearchVal =~ s/[\@\$]\(/$1\\\(/g;           # Defuse @( ... ) and $( ... )
+     $theSearchVal = substr($theSearchVal, 0, 1500); # Limit string length
+
      my $originalSearch = $theSearchVal;
      my $renameTopic;
      my $renameWeb = "";

Patch for twiki/lib/TWiki/Search.pm of TWiki Production Release 01-Feb-2003

*** TWiki20030201/Search.pm     2004-11-12 12:11:52.000000000 -0800
--- ./Search.pm 2004-11-12 12:12:20.000000000 -0800
***************
*** 135,140 ****
--- 135,147 ----
      my $tempVal = "";
      my $tmpl = "";
      my $topicCount = 0; # JohnTalintyre
+
+     # fix for Codev.SecurityAlertExecuteCommandsWithSearch
+     # vulnerability, search: "test_vulnerability '; ls -la'"
+     $theSearchVal =~ s/(^|[^\\])([\'\`])/\\$2/g;    # Escape ' and `
+     $theSearchVal =~ s/[\@\$]\(/$1\\\(/g;           # Defuse @( ... ) and $( ... )
+     $theSearchVal = substr($theSearchVal, 0, 1500); # Limit string length
+
      my $originalSearch = $theSearchVal;
      my $renameTopic;
      my $renameWeb = "";

Patch for twiki/lib/TWiki/Search.pm of TWiki Production Release 01-Dec-2001

*** TWiki20011201/Search.pm     2004-11-12 12:15:55.000000000 -0800
--- ./Search.pm 2004-11-12 12:16:45.000000000 -0800
***************
*** 133,138 ****
--- 133,145 ----
      my $tempVal = "";
      my $tmpl = "";
      my $topicCount = 0; # JohnTalintyre
+
+     # fix for Codev.SecurityAlertExecuteCommandsWithSearch
+     # vulnerability, search: "test_vulnerability '; ls -la'"
+     $theSearchVal =~ s/(^|[^\\])([\'\`])/\\$2/g;    # Escape ' and `
+     $theSearchVal =~ s/[\@\$]\(/$1\\\(/g;           # Defuse @( ... ) and $( ... )
+     $theSearchVal = substr($theSearchVal, 0, 1500); # Limit string length
+
      my $originalSearch = $theSearchVal;
      my $renameTopic;
      my $renameWeb = "";

Patch for twiki/bin/wikisearch.pm of TWiki Production Release 01-Dec-2000

*** TWiki20001201/wikisearch.pm 2004-11-12 12:18:55.000000000 -0800
--- ./wikisearch.pm     2004-11-12 12:23:07.000000000 -0800
***************
*** 117,122 ****
--- 117,129 ----

      my $tempVal = "";
      my $tmpl = "";
+
+     # fix for Codev.SecurityAlertExecuteCommandsWithSearch
+     # vulnerability, search: "test_vulnerability '; ls -la'"
+     $theSearchVal =~ s/(^|[^\\])([\'\`])/\\$2/g;    # Escape ' and `
+     $theSearchVal =~ s/[\@\$]\(/$1\\\(/g;           # Defuse @( ... ) and $( ... )
+     $theSearchVal = substr($theSearchVal, 0, 1500); # Limit string length
+
      if( $doBookView ) {
          $tmpl = readTemplate( "searchbookview" );
      } else {

-- PeterThoeny - 12 Nov 2004

Discussions

Refactored out SingleEntryPointForSystemCalls discussion.

-- PeterThoeny - 14 Nov 2004

I applied the Cairo fix and felt safe smile
... and forgot about the update.

Later some of my searches stopped working - very confusing at first. It was
$theSearchVal = substr($theSearchVal, 0, 200); # Limit string length
(a hard and quiet ruler) truncating my search-criteria and thus invalidating the search.

200 seems like a pretty small length for a searchstring - and if it has to be small, users should at least be warned and search stopped.

-- NielsKoldso - 14 Nov 2004

This has also stopped one of my searches from working. The second search on CodevBasicFormSearch has a search term with 272 characters.

I tried to create a search to find others that wouldn't work but SEARCH{\"[^\"]{200}.*\" doesn't appear to work as a search term. I'm guessing that the {200} expands the previous item before the security check. smile

-- SamHasler - 15 Nov 2004

yes, please raise (or eliminate) the limit; this change breaks several of my TWikiApplications. besides, rm -rf / easily fits within 200 characters, so i don't think it's really buying anything, anyway.

-- WillNorris - 15 Nov 2004

See SingleEntryPointForSystemCalls for an alternative patch against the Halloween beta. I'm using this on my site right now.

-- KennethPorter - 16 Nov 2004

A different patch (contributed) for twiki/bin/wikisearch.pm of TWiki Production Release 01-Dec-2000

This patch has not been checked by PeterThoeny, but I prefer this approach on Un*x systems; see the discussion.

*** wikisearch.pm       Wed Nov 15 07:19:48 2000
--- wikisearch.pm       Thu Nov 18 16:48:36 2004
***************
*** 150,155 ****
--- 150,156 ----
          $theSearchVal =~ s/>/>/go;
          $theSearchVal =~ s/^\.\*$/Index/go;
          $tmplSearch =~ s/%SEARCHSTRING%/$theSearchVal/go;
+       $theSearchVal =~ s{'}{'\\''}g;
          if( $doInline ) {
              $searchResult .= $tmplSearch;
          } else {

-- ChapmanFlack - 18 Nov 2004

My different patch for the 01-Dec-2000 release follows the reasoning in SingleEntryPointForSystemCalls. I checked the Perl documentation for `string` and Perl promises to use /bin/sh on Un*x systems, so it is fair to rely on Bourne shell quoting. I have not observed a need to further special-case ` and @( in the string; having the string properly quoted appears to suffice, and I can perform searches containing those characters.

Is there a demonstrated need to limit the string length? Do you suspect a possible failure mode other than E2BIG from exec?

-- ChapmanFlack - 18 Nov 2004

We could use the TWikiInternetDeploymentsSpider to find every public installation. If we could get the administrators to install the TWikiReleaseTrackerPlugin then we can remotely check the MD5s of each file in their installations.

-- MartinCleaver - 18 Nov 2004

a thread on http://freedesktop.org/ about security, and the entry on http://securitytracker.com

-- WillNorris - 18 Nov 2004

I just downloaded TWiki20040901.tar.gz from http://twiki.org/getpackage.html. It does not appear to have a patch for this issue applied to it. I think that any TWiki distributions downloaded from this site should already have had this patch applied. It's probably already the case that a large percentage of public (google-able) TWiki sites have been attacked using this vulnerability. No sense adding new targets.

-- ClaussStrauch - 18 Nov 2004

Chapman, the current fix is not perfect, SingleEntryPointForSystemCalls will be a better one. I copied the string limitation from the person who wrote the advisory. Could you or someone else confirm if it is safe to remove? It breaks some TWikiApplications.

-- PeterThoeny - 19 Nov 2004

I think we should set string length higher, to say 1000 characters. The person who wrote the advisory was probably not aware of TWikiApplications use of searching.

I agree with Clauss that the existing distribution should be patched.

-- RichardDonkin - 19 Nov 2004

I will create a patched version once we have a solid fix. I'd like to be able to search for "O'Hara" and have the string length eliminated or subtantially increased if safe. Perl/shell experts please advise.

-- PeterThoeny - 19 Nov 2004

Hang on Peter! Are you saying that you would rather leave a known-bad version up while we get it right?

IMO, NO VERSION is certainly better than a bad one.

-- MartinCleaver - 19 Nov 2004

Agreed. This is what Subversion branches are for. Make a branch off the 20040901 tag and re-release with an interim patch. (Except that I don't see a tags/20040901 tag in the repository.)

-- KennethPorter - 19 Nov 2004

+1 to the suggestion to lock the downloads until the fix is out. There's even an automated script out there to abuse the exploit!

-- RafaelAlvarez - 19 Nov 2004

please, please update all of the zips or take them offline until they are updated

-- WillNorris - 20 Nov 2004

Wow, big considerate concerns. Good point. I am going to update the releases. (The funny thing is in Oct 2003 with NoShellCharacterEscapingInFileAttachComment nobody seemed to care for 11 month...)

I updated the hot fix a bit: Escaped single quotes (as suggested by Chapman), and increased the string size limit. I kept the @() and backtick escape until verified to be overkill, although most probably not needed. This will be the version for the updated release.

TWikiCodebaseSecurityAudit with SingleEntryPointForSystemCalls will be a clean fix.

-- PeterThoeny - 20 Nov 2004

FWIW, I was a victim of the exploit. For the curious I'm happy to provide relevant Web log entries and a tarball of the files that were uploaded to my server. Some steps to see if you've been compromised:

  1. Examine your access log. This is of course the best method for discovering a break-in, but there are a couple other quicker methods I found based on how my box was compromised:
  2. Look in your error log for wget output
  3. "ls -al" in /tmp and /var/tmp for files/directories owned by your Web server process
  4. Also, look carefully at any running processes owned by Web server process.

-- ToddAGibson - 20 Nov 2004

Todd, I am sorry about your trouble. Thanks for the hints.

OK, I updated the production release with the patch, is now called TWiki Production Release 02-Sep-2004, TWikiRelease02Sep2004. The download page is updated. After bringing my kids to bed I will update related Codev topics.

  • Related Codev topics are now updated.

-- PeterThoeny - 20 Nov 2004

To remind people of the urgency to apply the security patch provided in TWikiRelease02Sep2004: my site was hacked using twiki, my front page defaced (no twiki page).

From Dreamhost support:
This is a hole in twiki, apparently. Please upgrade and / or fix this bug. I have removed the execute bit from "search" in your twiki directory.

This attacker already installed / ran a lot of malicious sofware on our machine.

I have removed most of the stuff s/he installed, however please check your site thoroughly and make sure there aren't any unexpected files there.

-- ArthurClemens - 20 Nov 2004

Arthur, a couple of questions for you:

  • Did Dreamhost detect the intrusion and initiate action?
  • Any worthwhile info on locations where files were added? Or could they be anywhere?

Thanks!

-- LynnwoodBrown - 20 Nov 2004

I detected the front page defacement and contacted support. Support found it was caused by the twiki hole. They installed stuff into /tmp and /var/tmp mostly, so there aren't backups. If you look through your HTTP access logs (in ~/logs/). Any commands that were executed through twiki would be in the logs, like the examples I sent you.

Basically, they installed a backdoor login program, some exploit attempts, etc.

-- ArthurClemens - 21 Nov 2004

I checked these instructions: http://koala.ilog.fr/twikiirc/bin/irclogger_log/twiki?date=2004-11-21,Sun&sel=23#l19

I can confirm that several of my hosts were scanned, though it is not yet clear how to determine if the hack attempts succeeded. Certainly nothing has been damaged yet.

-- MartinCleaver - 21 Nov 2004

Wrt detecting hacks: in the best case, the intruder just gets access as the (non-root) web server, was not able to escalate privileges, and wasn't very clever about leaving backdoors (such as bogus CGI scripts) or doing things like trying to crack .htaccess files. These cases are relatively easy to clean up. In the worse case, the intruder managed to get root access and has installed a rootkit of some sort and some type of sniffer to gain access to other hosts. You can run chkrootkit (http://www.chkrootkit.org) to detect many rootkits, but your alternatives are often either a lot of forensic analysis combined with crossing your fingers, or doing an operating system re-install.

-- ClaussStrauch - 22 Nov 2004

See TWikiSecurityAlertProcess for links to the http://freedesktop.org/ compromise through this hole - has generated some bad publicity for TWiki.

-- RichardDonkin - 24 Nov 2004

Peter, when you copied the length limitation from the person who wrote the advisory, was there any reasoning for it given in that advisory? (Is it linked somewhere I can read it?) Or was it more "length limitations feel safe, let's throw one in"?

The usual reason for limiting length is that you either know or suspect you will be passing the string into code with a preallocated fixed buffer that could be overrun. In the case of an argument being passed to a shell, the shell gets an element in argv that's an array already allocated to the correct length; the shell will pass it on to fgrep which receives it the same way, and all fgrep needs to do is compare against it. None of these operations is likely at all to involve fixed-size buffer copies. The only other manipulation I'd expect is in Perl itself as it prepares the command string, and I'm pretty sure Perl dynamically allocates its strings and has been pretty well vetted for buffer overflows.

So while I'm not in a position to promise there's no possible buffer overflow in there, I would argue in this case that throwing in an arbitrary length limit without having a particular risk in mind is overkill. If you add a length limit, the limit you pick should be based on the size of the suspected fixed-length buffer you're worried about; shorter than that is unnecessary, and longer is useless except for the false sense of security.

Also, did the advisory give a rationale for the backtick and @( escaping? Once I patch to properly quote the string, I can successfully search for things that contain backticks and @(, which leads me to conclude they're not getting special interpretation and so the quoting is sufficient--but I could check more carefully if I had examples that actually failed before.

My reasoning about the sufficiency of quoting applies only to systems where Perl uses a Bourne-family shell for its command parsing, which Perl promises it does on all Un*x systems. On other systems, where the exact syntax of command lines is defined by some other command parser, my bets are off.

-- ChapmanFlack - 24 Nov 2004

In the case of an argument being passed to a shell, the shell gets an element in argv that's an array already allocated to the correct length

... and (I meant to say) the result if the kernel can't pass an argument that long is E2BIG, which should be a safe failure mode.

-- ChapmanFlack - 24 Nov 2004

Added a link to SecurityAlertExecuteCommandsWithSearchHackReports to the report.

-- SamHasler - 25 Nov 2004

For those interested, e-mails made public:

  • sec-alert.txt: Security Altert sent by PTh to known TWiki site administrators on 12 Nov at 5pm PST

  • advisory-facts.txt: Timeline of security alert, my reply to the flamewar going on between the two parties that raced to publish the vulnerability first.

-- PeterThoeny - 25 Nov 2004

You can mount your /tmp rw,noexec,nosuid on linux at last. It will prevent some known toolZ.

-- MartinSteldinger - 28 Nov 2004

It looks to me as if the patches have bugs in them. They excise the character before the ' or ` in a search string. Here is a perl command line demonstrating the code that's a problem: perl -e '$theSearchVal="foo`bar";$theSearchVal =~ s/(^|[^\\])([\`])/\\$2/g;print "$theSearchVal\n"'

The output is "fo`bar".

Here are the bad patch lines for the Feb 01 2003 release:

+     $theSearchVal =~ s/(^|[^\\])([\'\`])/\\$2/g;    # Escape ' and `
+     $theSearchVal =~ s/[\@\$]\(/$1\\\(/g;           # Defuse @( ... ) and $( ... )

and here are the corrected ones:

+     $theSearchVal =~ s/(^|[^\\])([\'\`])/$1\\$2/g;    # Escape ' and `
+     $theSearchVal =~ s/([\@\$])\(/$1\\\(/g;           # Defuse @( ... ) and $( ... )

Can someone tell me how I'm misunderstanding the code, or verify that my corrections are correct?

-- BobbyMartin - 15 Dec 2004

You are correct, the hotfix is not perfect, it eats the character preceeding the escaped character. We will soon release a more solid version.

-- PeterThoeny - 15 Dec 2004

any sign of that solidified patch ?

-- KeithHelfrich - 27 Feb 2005

You may want to add your support for TWikiRelease23Feb2005

-- MartinCleaver - 23 Jun 2005

Edit | Attach | Watch | Print version | History: r51 < r50 < r49 < r48 < r47 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r51 - 2006-10-08 - SvenDowideit
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.