SID-01185: Find More Extensions gives permission denied

Status:	Answered	TWiki version:	4.1.2	Perl version:	don't know at the moment
Category:	CategoryAddOns	Server OS:	SUSE 10	Last update:	14 years ago

Running TWiki-4.1.2, Sat, 03 Mar 2007, build 13046 on SUSE Linux.

Using 'configure' I attempt to "Find More Extensions", which shows no obvious error, except for the following:

Extension Description Most Recent Version Installed Version Tested On Tested On OS Action 0 extensions out of 0 already installed

I took a packet trace of what was happening on the wire. The request goes out:

GET http://twiki.org:80/cgi-bin/view/Plugins/FastReport?skin=text&contenttype=text/plain HTTP/1.0

but the response that comes back is:

<p>You don't have permission to access /cgi-bin/view/Plugins/FastReport on this server.</p>

Using 'wget' from the Linux command-line, specifying the same proxy and the same URL works fine!

Help please?

thanks, Greg

-- GregChoules - 2011-05-23

Discussion and Answer

Not sure, rare case. Possibly your server IP addresses is banned on twiki.org? Did you do the wget test from your TWiki server or a local machine? If you let me know your server's external IP adddress I can test if it is banned.

-- PeterThoeny - 2011-05-24

The wget is from the same machine, so same source IP. In any case, this machine is behind a firewall (checked that all is ok through that) AND uses a proxy to access the Internet. I don't know the actual outside IP that would have been used at the moment, but I'll find it in case it may be important (I think not though). My thoughts are that there must be something slightly different in the HTTP requests themselves to allow one to work and the other not to, but since my expertise stops at layer 3/4 I don't really know what I'd be looking for. I have packet traces available of both working and non-working cases (from the server itself) if that would help? cheers, Greg

-- GregChoules - 2011-05-24

Google for "my IP address", it will tell you the external IP address. I can look in the twiki.org apache logs to see if there is anything unusual.

-- PeterThoeny - 2011-05-24

ok. My IP will be 217.171.129.236 (outside of the proxy). I have just made 10 attempts to get Extensions at 08:20 UTC

-- GregChoules - 2011-05-24

I checked: Your IP address is not blocked. Here is some info:

• One of your GET's access log and error log:
  • 217.171.129.236 - - [24/May/2011:01:20:16 -0700] "GET /cgi-bin/view/Plugins/FastReport?skin=text&contenttype=text/plain HTTP/1.0" 403 307 "-" "-"
  • [Tue May 24 01:20:16 2011] [error] [client 217.171.129.236] client denied by server configuration: /var/www/twiki-local/bin/view
• Other GET's access log:
  • foo.example.com - - [24/May/2011:08:46:50 -0700] "GET /cgi-bin/view/Plugins/FastReport?skin=text&contenttype=text/plain HTTP/1.0" 200 114255 "-" "TWikiConfigure/1.0 +http://twiki.org/"
  • bar.example.com - - [24/May/2011:10:39:22 -0700] "GET /cgi-bin/view/Plugins/FastReport?skin=text&contenttype=text/plain HTTP/1.1" 200 114255 "-" "TWiki::Net/21156 libwww-perl/5.805"

As you can see, the agent is missing in your request. On twiki.org we block user agents that do not identify themselves. Check with your IT if your proxy strips the user agent.

-- PeterThoeny - 2011-05-24

Ah, that would explain it. Thanks Peter.

It's not the proxy stripping off the UA (since the fetch using 'wget' went through the same path), it is actually Twiki not putting one in to start with. I can't paste the binary packet traces here, but below are the two requests, copy/pasted from Wireshark.

works (using 'wget')

###
GET http://twiki.org/cgi-bin/view/Plugins/FastReport HTTP/1.0
User-Agent: Wget/1.9.1
Host: twiki.org
Accept: */*

doesn't work (using Twiki)

###
GET http://twiki.org:80/cgi-bin/view/Plugins/FastReport?

I didn't miss copying anything in the latter case. There just is no UA string present.

So the question becomes, how do I get our Twiki installation to insert a UA string?

-- GregChoules - 2011-05-24

Something else must be going on. Check sub getUrl in your twiki/lib/TWiki/Configure/UI.pm. This is what we have in TWiki-4.1.2, the version you indicated:

    my $req = "GET $path HTTP/1.0\r\nHost: $host\r\nUser-agent: TWikiConfigure/1.0 +http://twiki.org/\r\n\r\n";

-- PeterThoeny - 2011-05-24

The file "..twiki/lib/TWiki/Configure/UI.pm" contains the following

<pre> my $req = "GET $path HTTP/1.0\r\nHost: $host\r\nUser-agent: TWikiConfigure/1.0 +http://twiki.org/\r\n\r\n"; if ($TWiki::cfg{PROXY}{HOST} && $TWiki::cfg{PROXY}{PORT}) { $req = "GET http://$host:$port$path HTTP/1.0\r\n\r\n"; $host = $TWiki::cfg{PROXY}{HOST}; $port = $TWiki::cfg{PROXY}{PORT}; } </pre>

Since we DO have a proxy host AND port defined I think that the getUrl in the 'if' clause will apply, which contains no information about UA. So i believe that I need to do the following. Change

<pre> $req = "GET http://$host:$port$path HTTP/1.0\r\n\r\n"; </pre>

within the 'if' clause to..

<pre> $req = "GET http://$host:$port$path HTTP/1.0\r\nHost: $host\r\nUser-agent: TWikiConfigure/1.0 +http://twiki.org/\r\n\r\n"; </pre>

Does that sounds sensible?

-- GregChoules - 2011-05-25

sorry about the formatting! Is this any better?

<code> my $req = "GET $path HTTP/1.0\r\nHost: $host\r\nUser-agent: TWikiConfigure/1.0 +http://twiki.org/\r\n\r\n"; if ($TWiki::cfg{PROXY}{HOST} && $TWiki::cfg{PROXY}{PORT}) { $req = "GET http://$host:$port$path HTTP/1.0\r\n\r\n"; $host = $TWiki::cfg{PROXY}{HOST}; $port = $TWiki::cfg{PROXY}{PORT}; } </code>

<code> $req = "GET http://$host:$port$path HTTP/1.0\r\nHost: $host\r\nUser-agent: TWikiConfigure/1.0 +http://twiki.org/\r\n\r\n"; </code>

-- GregChoules - 2011-05-25

Oh, that looks like a bug in TWiki-4.1.2. Yes your fix looks good. This is fixed in TWiki-5.0, which uses TWiki::Net::getExternalResource instead of local http get code in configure.

-- PeterThoeny - 2011-05-25

Did that and network traffic has certainly gone up. From the packets traces it looks like the list is coming back. However, my browser session to our TWiki times out while it is doing the "Consulting TWiki.org..." bit and no files appear to have been updated in the ...../twiki/ branch of the filing system. We can't find anything useful in logs so I'm stuck again! My first thought was that it is a filing system permission thing - TWiki is trying to put the information received from twiki.org somewhere it can't and hence the whole process stalls. Your advice, once again, would be most welcome.

thanks

-- GregChoules - 2011-05-26

If upgrading to TWiki-5.0 is not an option, try downloading and installing plugins manually on the server. Each plugin package has an install script.

-- PeterThoeny - 2011-07-08

If you answer a question - or someone answered one of your questions - please remember to edit the page and set the status to answered. The status selector is below the edit box.