Bug: No Search Results returned for ALLOWWEBVIEW webs

If ALLOWWEBVIEW is set for a web then no results are returned for that web in any search. The CGI script "search" is usually registered with .htaccess as "allow from all" and not "require valid-user", so the user is not authenticated at time of search. If you change this to "require valid-user" then all of your public searches will break.

Test case

See Support.SearchFailsInHiddenWeb for original question and my solution. If this is not a permanent solution perhaps it could be documented as a work around.

Environment

TWiki version:	TWikiRelease01Feb2003
TWiki plugins:	DefaultPlugin, EmptyPlugin, InterwikiPlugin
Server OS:	Debian Linux stable/woody
Web server:	Apache 1.3.26
Perl version:	5.6.1
Client OS:
Web Browser:

-- LyleWinton - 01 Jun 2004

Follow up

Fix record

No one seems to have commented, so I will post my fix here and call it a PatchProposal. The problem is that the CGI script "search" is usually registered with .htaccess as "allow from all" and not "require valid-user", so the user is not authenticated at time of search. If you change this to "require valid-user" then all of your public searches will break ... not a good solution. My solution is in 3 parts

1. create another search script that is authenticated
2. create a new WebSearch Template Topic (say WebSearchAuth) that references this script
3. code a new function IF{} so we can do a check if the user is authenticated
4. use the IF{} function to either include the WebSearch or WebSearchAuth depending on whether they are identified as the guest user (ie. not logged in)

This solution will allow searching of any private web topics (ie. those where ALLOWWEBVIEW is set) from within private webs. Alternatively, you could not implement the IF{} function (which I personally think is damn useful) and tell admins in the documentation to modify private webs so their WebSearch topic INCLUDE WebSearchAuth .

Proposed steps for the developers...

1)  Create a new search CGI script called searchauth
   > cd twiki/bin
   > cp search searchauth

2)  Set the new script to require validation
   > vi .htaccess
# Add these lines...
<Files "searchauth">
       require valid-user
</Files>

3)  Go to the web topic %TWIKIWEB%.WebSearch (usually TWiki.WebSearch)

4)  Copy the contents to a new topic TWiki.WebSearchAuth.

5)  Paste the contents from the TWiki.WebSearch into this new topic, changing
    all references to  search%SCRIPTSUFFIX%  to  searchauth%SCRIPTSUFFIX%  .

6)  Modify the perl module lib/TWiki.pm to incorporate the new function %IF{}% .
    The patch for this file is attached.  Should be pretty self explanatory.

• TWiki.pm.patch

7)  Modify all the WebSearch pages especially in _default web so new Webs
    inherit this.  This should be the new contents...
%IF{a="%WIKIUSERNAME%" type="regex" b="\.(TWikiGuest|guest)$" format="$percntINCLUDE{\"%TWIKIWEB%.WebSearch\"}$percnt" else="$percntINCLUDE{\"%TWIKIWEB%.WebSearchAuth\"}$percnt" }%

--OR--

7)  Change the search page in your ALLOWWEBVIEW web to include the newly created
    topic WebSearchAuth.  eg. Edit MyWebName.WebSearch so it now contains...
%INCLUDE{"%TWIKIWEB%.WebSearchAuth"}%

Specifically, authenticated users will be able to search all web pages from a private Web using TWiki.WebSearchAuth, but will only be able to search all public pages from a public Web still using TWiki.WebSearch .

-- LyleWinton - 01 Jul 2004

Hello Lyle, thanks for the patch.. I'm not quite sure why this has not been done before - but I'm guessing that many of us have avoided the issue totally by using one of the SessionPlugins. Am I right in thinking that you are not using a SessionPlugin?

-- SvenDowideit - 04 Jul 2004

Hello Lyle, I use a lot of webs on my wiki so I prefered to replace your step 7 with this :

7)  Go to the web topic %TWIKIWEB%.WebSearch (usually TWiki.WebSearch)

8)  Copy the contents to a new topic TWiki.WebSearchNoAuth.

9)  Modify all the WebSearch pages (%TWIKIWEB%.WebSearch included) especially in _default web so new Webs
    inherit this.  This should be the new contents...
%IF{a="%WIKIUSERNAME%" type="regex" b="\.(TWikiGuest|guest)$" format="$percntINCLUDE{\"%TWIKIWEB%.WebSearchNoAuth\"}$percnt" else="$percntINCLUDE{\"%TWIKIWEB%.WebSearchAuth\"}$percnt" }%

-- AlainBeauvois - 12 Jul 2004

In my experience, it is hard to find any system today offers access-controlled search on content primarily under access control. Most indexing methods (lucene etc.) are for public searches and will require architectural changes to make them recognize access controls (and that too if you work out integration aspects). With this capability, twiki may have an edge over other collaboration products.

Hence request to put put this into the core (but would like a generic version of IF on lines of ConditionalPlugin).

-- VinodKulkarni - 13 Jul 2004

This solution seems over-engineered. See SearchShouldRespectAccessControlsOnWebs for a simpler (and IMHO equally effective) solution.

-- CrawfordCurrie - 21 Feb 2005