create new tag
, view all tags

Feature Proposal: Set User Agent for TWiki's Include Requests


TWiki's INCLUDE{} can bring in content from other webservers. But, it does not set the UserAgent in its requests, therefore many servers reject the request as being too spammy.

Description and Documentation

See http://koala.ilog.fr/twikiirc/bin/irclogger_log/twiki?date=2008-07-25,Fri for the conversation that led to this.

It comes in as e.g.

- - [25/Jul/2008:12:14:50 -0500] "GET /~twiki4/cgi-bin/view/Bugs HTTP/1.0" 403 310 "-" "-"

A config option to set the UA should be simple to add. Just define it as a $TWiki::cfg{Net}{UserAgent} option, then include it in external requests (TWiki::Net::getExternalResource)

Here's the Moz standard: http://www.mozilla.org/build/revised-user-agent-strings.html - the RFC is at http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.43

peterthoeny recommended to specify in order of importance, hence: TWiki/4.1.2/twiki.org i.e. "TWiki" + twiki version + host name

I think it's best to put as a config item as it means we can hardcode it on t.o. for now

We elected to have the UA include a site identity string, i.e. if the site knows itself as twiki.org to include that to differentiate it from other twiki sites on the same server

Once this is complete we can show the Bugs web summary on each the Plugin page.




-- Contributors: MartinCleaver - 25 Jul 2008


Can we have this for TWikiRelease04x03?

-- RafaelAlvarez - 25 Jul 2008

It is a good idea.

Note one little thing. Today we recommend setting up TWiki to deny itself to avoid an obvious DOS attack where you let two TWiki's eat themselves up.

If one includes the other that includes the first the including goes on until the TWiki time out. And then it only takes a little fun with ab to get two Twikis to burn each other up.

So if we implement this we also need to make sure we can setup TWiki to not accept itself as user agent.

-- KennethLavrsen - 25 Jul 2008

Note - without a committed developer this proposal cannot be accepted - only discussed.

A bug item was raised but it is too early for that.

Target release would be GeorgeTown which is 5.0. There is no 4.3 on the roadmap. Next release must address the important roadmap items and not just small feature creaping.

-- KennethLavrsen - 28 Jul 2008

Ping! This looks like a small and useful enhancement. Anyone taking it?

-- PeterThoeny - 2010-06-23

Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r6 - 2010-06-23 - PeterThoeny
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.