create new tag
, view all tags

Bug: Search does not work with bad TWiki names

Twiki unfortunately does not prevent creating TWiki pages with bad names (one might get a warning, but can create the page anyway). This can cause a fatal failure in the search functionality:

If you create a topic named Test() and then try WebSearch, this will entirely fail due to a shell error.

The apache 2 logfile APACHE_HOME/log/error_log then contains the following lines:

sh: -c: line 1: syntax error near unexpected token `Test()'
sh: -c: line 1: `/bin/fgrep -i -l -- 'something . Very / unLikelyTo + search - for ;-)' Test().txt TestTopic1.txt 
TestTopic9.txt WebChanges.txt WebHome.txt WebIndex.txt WebLeftBar.txt WebNotify.txt WebPreferences.txt 
WebRss.txt WebSearch.txt WebSearchAdvanced.txt WebStatistics.txt WebTopicList.txt'

TWiki should either

  • prevent the creation of such pages or
  • escape problematic characters when assembling the shell command

Test case

Go to Sandbox and create a page named Test() and save it.

Run WebSearch in this web and watch the Apache2 logfiles.

You will not get any search results.


TWiki version: TWikiRelease02Sep2004
TWiki plugins: DefaultPlugin, EmptyPlugin, InterwikiPlugin
Server OS: Linux
Web server: Apache 2
Perl version: 5.8.x
Client OS: Linux
Web Browser: Firefox

-- GuidoOstkamp - 08 Mar 2005

Impact and Available Solutions

Follow up

Fix record


This is not an issue in DevelopBranch, due to the use of the sandbox for all command-line activities.

-- CrawfordCurrie - 09 Mar 2005

The error log output caused a search result on Codev.WebSearch. I've added spaces in the 'something.Very/unLikelyTo...' line.

You see - it's not that very unlikely.

-- ArthurClemens - 15 Apr 2005

I'm afraid I don't understand Crawford's comment about the use of the sandbox, but I'm watching my error logs on my web server and this is one of the ones I see a lot. The problem in my case is that people are creating attachment files whose names contain whitespace; when the grep command is invoked that fails. I'll attach a simple patch that fixes the problem unless your filename contains a single-quote character (').

To me this entire method of invoking the grep command is simply fatally flawed. Not only does it suffer from an extremely difficult quoting problem (especially when you consider portability!!) which is not just a functional issue but also a security issue, but it's slow as well.

The Right Thing To Do here is to write a Perl function that invokes the command directly, the way system() does when you pass it multiple arguments, but that replaces `` (grabs the output). I'd be very surprised if a module like that doesn't already exist but if not I have Perl code I'd be happy to donate that will do the job.

-- PaulSmith - 14 Jun 2005

I think it may be related to SearchedTopicnameCannotContainNonAlphanumChars :

The problem comes from _makeTopicPattern subroutine in lib/TWiki/Search.pm, which does not deal correctly topics with bad Twiki names.

See SearchedTopicnameCannotContainNonAlphanumChars for more details and workaround.

-- NicolasRaibaut - 27 Jan 2006

Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatdiff Search.diff r1 manage 0.8 K 2005-06-14 - 22:35 PaulSmith Patch to TWiki/Search.pm to quote arguments to grep
Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r5 - 2006-01-27 - NicolasRaibaut
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.