create new tag
, view all tags

Redirect is bad for TWiki's health

TWiki uses HTTP redirect (30x) quite heavily. Examples:
  1. redirect is used when an access control violation occurs
  2. redirect is used for oops scripts
  3. redirect is used by some state machines e.g. topic rename
  4. redirect may be used by arbitrary plugins
Historically TWiki uses redirect because it makes coding easier, basically because it lets you treat individual TWiki scripts as procedures. By using a redirect instead of a procedure call, you are "clearing the context" ready to handle the different query.

Unfortunately, there are major technical problems associated with use of 30x redirect. For some background, read http://ppewww.ph.gla.ac.uk/~flavell/www/post-redirect.html. This has resulted in different interpretations of redirect behaviour between browsers, between server implementations, and between scripts.

The flaw that bites TWiki hardest is the handling of parameters. In summary, if a POST request is redirected, then there is a high risk that the parameters to the POST will be lost through the redirect. This doesn't matter in many cases, but there are a couple of scenarios in which it is a killer for TWiki:

  1. A POST request causes an access control violation (Bugs:Item2295)
    1. TWiki redirects to the login script
    2. login redirects to the original URL on successful authentication (login could capture POST parameters in the GET redirect, but these can easily exceed the GET size limit.)
  2. Apache hangs because a GET request resulting from a 30x has pending input on STDIN that hasn't been drained by CGI (Bugs:Item2327)
  3. POST data is lost due to oops redirect and back (Bugs:Item2314)
Plugin behaviour with redirect is rather unpredictable, but the chances are good that it goes wrong somewhere.

There's another scenario in which a redirect is irritating. Try modifying the URL in your browser to point to a non-existant web. You will get an oops redirect. This rewrites the Location in the redirect as an oops url, so your URL bar changes. If you often edit URLs in the URL bar, then this can get really annoying.

So let's consider how we might eliminate redirect.

Fortunately, when I was recoding for TWiki-4, I eliminated most manual redirects, replacing them with oops exceptions. So the first thing would be to eliminate the redirect on oops. This isn't too hard, as during Cairo development I broke the scripts down into re-entrant procedures in UI. The only issue to be solved is what to do when a script has already written HTTP headers (e.g. search, which tries to produce output iteratively to give early feedback on results).

The redirect used by the access control violations is trickier. An access control violation might occur at any point during processing, though we can recode to try and make sure that it will happen before headers are output. However login managers that use webserver authentication require a redirect, because we want to trigger access to an authenticated script (login or logon) to force the server to authenticate.

So maybe recoding to eliminate redirect isn't possible in this case. Sessions might offer another way for us to avoid the POST/GET problem.

  1. On triggering a redirect, make sure we drain CGI STDIN and write the CGI params to the session DB (on disc on the server)
  2. On GET, check if we got there via a 30x and if so, repopulate $CGI::param with the data from the session.
This would probably be simpler to code than trying to eliminate redirect, and would work for plugins as well. However it would require session support to be enabled (currently optional). Of course, if sessions are not enabled, we can try and encode the POST parameters into the GET, but I am concerned that the GET size limit would cause a bunch of mysterious errors if we did that.

Any other ideas? (before anyone says anything about them, cookies are not useful for solving this. See me after class for reasons.)

-- Contributors: CrawfordCurrie


I am afraid that storing the data to be carried over in a session object will not work as you might be browsing the same site in multiple tabs simultaneously and just get to see the session data produced by one tab in the other. An alternative would be to store the data in the html and get it back next time.

-- MichaelDaum - 28 May 2006

Argh, yes, you are right. OK, so sessions are no good. I'm not sure what "store the data in the html" means. The scenario is this:

  1. User POSTs view with humungous form data
  2. view discovers access control violation
  3. view redirects to logon
  4. Redirect does a GET of logon, parameters are lost
You can't eliminate step 3, because your .htaccess has
<Files "logon">
require valid-user
i.e. that's how it triggers authentication.

Actually, there might be another way to do this. Instead of using the session store, TWiki could use a store unique to this request, thus:

  1. User POSTs view with humungous form data
  2. view discovers access control violation
  3. view redirects to logon
    1. TWiki::redirect stores POST data in a file with a unique ID, rewrites the redirect to a GET with a single parameter viz. the unique ID
  4. redirect does a GET of logon
    1. logon sees UID in URL parameters, loads cached POST data
    2. login succeeds, logon POSTs back to original URL.
If logon doesn't see a UID, then the redirect simply uses the remaining parameters specified in the URI.

-- CrawfordCurrie - 28 May 2006

That should work.

-- MichaelDaum - 28 May 2006

Note that in theory, you can redirect to a POST using 302. However CGI.pm is completely SNAFU at handling it :-(. The HTTP spec also requires that the redirect is interactively authorised by the user in this case frown frown

-- CrawfordCurrie - 28 May 2006

It never occured to me that redirects might cause problems. Need / no need to avoid is certainly connected to the architecture used. To keep the extended path and URL parameters, isn't it possible to simply pass the PATH_INFO and QUERY_STRING along to the redirects?

  • No. As discussed above, a redirect is a GET, which is inherently limited in length. You cannot reliably pack POST parameters into a GET. -- CC
    • Isn't that a special case? POST is used for large data (text in save etc) with already authenticated users. Is there ever a need to redirect with the large data? -- PTh
      • Yes. For example, a save after an edit as TWikiGuest to a topic that has since had it's permissions changed. An edit which fires with initial text provided from a textarea in the form submission page. There are many more. -- CC

If we look at avoiding redirects we need to solve a few open questions:

  • Save needs to be authenticated, after that you should end up in non-authenticated view (so that the user can send a URL in an e-mail that does not require login to see the topic)
    • I don't understand this point. -- CC
      • When basic auth is used with not authenticated view script and authenticated save and viewauth script, you want to end up in the not authenticated view script after the save option -- PTh
        • Why? -- [[CC]
          • So that I can send a view URL by e-mail, and the recepient does not need to login to see the topic. -- PTh
  • User looks an the unauthenticated view, clicks on a button and should end up in authenicated view, with URL params retained.
    • Correct, that's the login sequence which currently does not retain URL params -- CC
      • Again, question of switching from not-authenticated view script to authenticated viewauth script -- PTh

-- PeterThoeny - 01 Jun 2006

If TWiki handled authentication rather than sharing responsibility for this task with Apache, would the problem go away?

-- MartinCleaver - 02 Jun 2006

Yes. But the reason for delegating to Apache is to take care of all the gribbly authentication schemes out there, like mod_auth_ldap, without requiring TWiki modules for every one.

-- CrawfordCurrie - 02 Jun 2006

While there may be other reasons for not using redirect, throw E, where E is on OopsException, has the same problem, per my experimentation, with respect to parameter handling.

-- ThomasWeigert - 01 Nov 2006

Edit | Attach | Watch | Print version | History: r14 < r13 < r12 < r11 < r10 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r14 - 2006-11-01 - ThomasWeigert
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.