create new tag
, view all tags

SID-02228: How to prevent TWiki from serving mixed content

Status: Asked Asked TWiki version: 6.0.2 Perl version: 5.10.1
Category: CategorySecurity Server OS: CentOS 6.7 Last update: 1 year ago

I have installed TWiki and verified all the system requirements are satisfied. Then, with the help of a Linux Administrator, we setup TWiki/Apache to use Active Directory for authentication. As a result, we want to use SSL for security reasons. The /etc/httpd/conf.d/twiki.conf file has been configured to redirect everything over https.

Now comes the problem. We've got everything configured and authentication is working properly. However, the web browsers (Chrome and IE) are blocking scripts and stylesheets because they are considered mixed content. See the following sample error message:

Mixed Content: The page at 'https://twiki.company.com/do/view' was loaded over HTTPS, but requested an insecure script 'http://twiki.company.com/pub/TWiki/BehaviourContrib/behaviour.compressed.js'. This request has been blocked; the content must be served over HTTPS.

Now, the problem can be resolved temporarily by telling the browser to "Load Unsafe Scripts", but this has to be done every time the pages are loaded (hardly an acceptable solution). I searched TWiki.org and the only suggestion I found was to provide a fully qualified URL for ScriptUrlPath and PubUrlPath (e.g. "https://twiki.company.com/pub"). However, this is a hack at best and the top menu of the pages gets messed up due to embedded quoting (") issues.

How do I configure TWiki to use https for all requests? I would think addition a configuration item to specify (http or https) for all requests would be a good idea.

-- Mark Rutz - 2016-07-19

Discussion and Answer

The suggestions to use absolute URLs where TWiki asks for paths are hacks, indeed. Don't do that.

  • If you redirect everything over https, then you would also redirect pub. I guess you only did it for the bin part. But anyway, that should only be necessary for people who are using http bookmarks, and those usually don't point to pub URLs. So, since you only mention Apache's config file:
  • Have you configured {DefaultUrlHost} to use https in TWiki's configuration?

-- Harald Jörg - 2016-07-19

TWiki configuration:

{DefaultUrlHost} = https://twiki.company.com
{PermittedRedirectHostUrls} = http://twiki.company.com
{ScriptUrlPath} = /do
{PubUrlPath} = /pub
{PubDir} = /path/to/twiki/pub

Apache twiki.conf:

<VirtualHost *:80>
    DocumentRoot "/path/to/twiki"
    ServerName twiki.company.com
    RedirectMatch "/$" https://twiki.company.com/do/view

    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule ^/(.*) https://%{SERVER_NAME}/$1 [R=301,L]
    RewriteLogLevel 5

<VirtualHost _default_:443>
   ... Settings for SSL Host including AD authentication, etc.

-- Mark Rutz - 2016-07-20

Thanks for the config information!

  • TWiki configuration is fine. If you need the {PermittedRedirectHostUrls} setting, then this indicates a situation where TWiki does not respect its own {DefaultUrlHost} settings. I call that a bug.
  • Apache configuration looks fine, too.

With these settings the server should respond with a 301 status code to every single request, whether a topic or its "static" components. I'd guess the logging you've activated confirms that.

There's something weird going on: In TWiki, as far as I know, the JavaScript libaries are all loaded from its own pub section. On http://twiki.org, it looks like this:

<script type="text/javascript" src="/p/pub/TWiki06x00/BehaviourContrib/behaviour.compressed.js"></script>

I have no idea how the browsers expand this to a HTTP URL if the current page has been loaded over HTTPS. Unfortunately there's nothing TWiki can do about it. You could, just for once, accept the "unsafe" content and inspect the source code of your TWiki page whether it contains a relative link to the offending scripts.

I have one wild guess: The scripts, CSS files and the like are usually cached, by the browsers as well as by corporate proxies and so-called "web accelerators". Could you try to clear the caches and re-try?

-- Harald Jörg - 2016-07-20

I don't know that I need the {PermittedRedirectHostUrls} setting. I'll try clearing the entry tomorrow.

I cleared my cache and then went to the main page directly (https://twiki.company.com). I'm still getting mixed content. I accepted the "unsafe" content and looked at the source. There are still a lot of references to "http://twiki.company.com" in the source. However, you are correct that the javascript references just point to /pub (e.g. src="/pub/TWiki/TWikiJavascripts/twikilib.js").

-- Mark Rutz - 2016-07-20

As Harald mentions, I also suspect a caching issue. If a page was cached under http, embedded links will be http, even if the cached page is accessed under https.

While loading a TWiki page watch the twiki logs to verify cache/no cache.

-- Peter Thoeny - 2016-07-21

Mark: I'd still like to sort out these references to http. Could you please give an example for such a link? If the link is created from your body text (and not in TWiki's style and navigation panels), could you post the TWiki markup which causes the "bad" link to be created? I have found a couple of plugins/options where TWiki creates absolute links, but so far none where it doesn't respect DefaultUrlHost.

The HTTP links should be harmless because it is Apache's duty to rewrite them to HTTPS, so no unsafe content is supposed to reach your browser. Unless you have a caching proxy between you and the server: Those so-called "web accelerators" tend to "optimize" traffic by not so strictly adhering to the headers sent by the browsers with regard to caching options.

With regard to log files, I'd like to add: Caching of TWiki pages itself can be spotted in TWiki logs (/data/log201607.txt per default), but to detect caching of CSS, JS and images you need to refer to the web server's log file.

-- Harald Jörg - 2016-07-21

I've cleared cache in Chrome. I've cleared cache in IE. I've tried incognito mode. All without luck. I'm still getting mixed content.

Here's a snip from the Apache access_log: - mark.rutz [21/Jul/2016:10:38:12 -0400] "GET /do/view HTTP/1.1" 200 50491 - - [21/Jul/2016:10:38:14 -0400] "GET /pub/TWiki/JQueryPlugin/jquery-migrate.js HTTP/1.1" 200 7199 - - [21/Jul/2016:10:38:14 -0400] "GET /pub/TWiki/JQueryPlugin/jquery-all.css HTTP/1.1" 200 7722 - - [21/Jul/2016:10:38:14 -0400] "GET /pub/TWiki/JQueryPlugin/jquery-all.js HTTP/1.1" 200 37588

Here's a snip from the TWiki log:

2016-07-21 - 14:38:13 mark.rutz view Main Mozilla
2016-07-21 - 14:38:15 mark.rutz viewfile WebPreferences favicon.ico

Here's a snip from the source of the Main page with the bad links:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">
<script type="text/javascript" src="/pub/TWiki/JQueryPlugin/jquery.js"></script>
<script type="text/javascript" src="/pub/TWiki/JQueryPlugin/jquery-migrate.js"></script>
<link rel="stylesheet" href="/pub/TWiki/JQueryPlugin/jquery-all.css" type="text/css" media="all" />
<script type="text/javascript" src="/pub/TWiki/JQueryPlugin/jquery-all.js"></script>
<title> WebHome &lt; Main &lt; TWiki</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<link rel="icon" href="/pub/Main/WebPreferences/favicon.ico" type="image/x-icon" />
<link rel="shortcut icon" href="/pub/Main/WebPreferences/favicon.ico" type="image/x-icon" />
<link rel="alternate" href="http://twiki.company.com/do/edit/Main/WebHome?t=1469111893" type="application/x-wiki" title="Edit WebHome" />
<link rel="edit" href="http://twiki.company.com/do/edit/Main/WebHome?t=1469111893" title="Edit WebHome" />
<meta name="SCRIPTURLPATH" content="/do" />
<meta name="SCRIPTSUFFIX" content="" />
<meta name="TEXT_JUMP" content="Jump" />
<meta name="TEXT_SEARCH" content="Search" />
<meta name="TEXT_NUM_TOPICS" content="Number of topics:" />
<meta name="TEXT_MODIFY_SEARCH" content="Modify search" />
<link rel="alternate" type="application/rss+xml" title="RSS Feed" href="/do/view/Main/WebRss" />
<base href="http://twiki.company.com/do/view/Main/WebHome"></base>
<!--BEHAVIOURCONTRIB--><script type="text/javascript" src="/pub/TWiki/BehaviourContrib/behaviour.compressed.js"></script>
<script type="text/javascript" src="/pub/TWiki/TWikiJavascripts/twikilib.js"></script>
<script type="text/javascript" src="/pub/TWiki/TWikiJavascripts/twikiWindow.js"></script>
<script type="text/javascript" src="/pub/TWiki/TWikiJavascripts/twikiEvent.js"></script>

-- Mark Rutz - 2016-07-21

Well, I made some progress. I put some debug logging into TWiki.pm and figured something out. The $query->url() in the 'new' subroutine is returning "http://twiki.company.com" even though my browser is explicitly using "https://twiki.company.com". Further logging indicates that 'secure' is set to 0 in the TWiki::Request. This is why http is being used instead of https.

So, you know what did work to fix my site? Forcing $this->{urlHost} = $TWiki::cfg{DefaultUrlHost} in the 'new' subroutine. Of course, this is a serious hack. The real fix is to determine how/why 'secure' is being set to 0 in the TWiki::Request.

-- Mark Rutz - 2016-07-21

Thanks for the reports! But, as always: This rises more questions:

  • Did you verify that the server logs are from an HTTPS request? In the configuration above, both virtual hosts use the same log file. You'd need to place different CustomLog directives within the virtual host sections to see the difference.
  • Do you see any indications for successful rewriting from your rewrite resp. error log? (by the way: for recent mod_rewrite versions, RewriteLogLevel is a no-op: See http://httpd.apache.org/docs/current/mod/mod_rewrite.html). To repeat: With a working rewrite configuration, there's no chance for the client to actually receive http content over HTTP. In particular: Links to the pub directory do never reach TWiki code. If they result in a http response to your browser, then either your web server or mod_rewrite is broken.
  • Please run do/configure for your TWiki, with both http and https, and examine the first section Environment variables. I'm particularly interested in the values for HTTPS and SERVER_PORT. If HTTPS is ON, then TWiki should flag the session as secure.
  • Are you using a proxy (see your browser configuration) or is your TWiki located behind a load balancer?

-- Harald Jörg - 2016-07-21

1) TWiki is trying http even when my browser is explicitly pointed to https. I split the access and error logs for http and https. Without my hack, when my browser points to https://twiki.company.com/do/view, the original "GET /do/view" is in the https log, but then there are several gets recorded in the http log.

2) I enabled the rewrite log and turned logging up to 5. I see everything being properly rewritten to https.

3) Entering https://twiki.company.com/do/configure into the browser (and the password at the first page), I see the following values. There wasn't an HTTPS environment variable in the configuration display.

HTTP_HOST   twiki.company.com
HTTP_ORIGIN   https://twiki.company.com
HTTP_REFERER   https://twiki.company.com/do/configure
SERVER_NAME   twiki.company.com

The results are the same when I enter http and redirection is enable. Without redirect, the request fails since I don't have anything configured for the http side.

4) No, I'm not using a proxy. No, TWiki is not located behind a load balancer.

-- Mark Rutz - 2016-07-22

Thanks again for reporting back!

The critical thing is that you do not see a HTTPS environment variable, and even worse, that SERVER_PORT is 80. There's something strange going on with your web server: The default HTTPS port is 443 and not 80. WIth your settings, TWiki has no information that the request was through a secure connection, or to be precise: The libraries behind $this->{urlHost} have no information.

In http://httpd.apache.org/docs/2.2/mod/mod_ssl.html, there's information about the environment variables, and that they are provided by the server only if SSLOptions StdEnvVars is set. That could be a clue why you don't see the HTTPS env var (and adding the SSLOptions directive could be the fix we've been searching for) - but it is no explanation whatsoever why the SERVER_PORT result is straight wrong.

One final questions befire I give up: you write then there are several gets recorded in the http log. What URLs in the original requests cause these? (Background: If they are from relative URLs, then Chrome should create HTTPS requests in a HTTPS response).

-- Harald Jörg - 2016-07-26

      Change status to:
ALERT! If you answer a question - or someone answered one of your questions - please remember to edit the page and set the status to answered. The status selector is below the edit box.
Status Asked
Title How to prevent TWiki from serving mixed content
SupportCategory CategorySecurity
TWiki version 6.0.2
Server OS CentOS 6.7
Web server Apache 2.2.15
Perl version 5.10.1
Browser & version Chrome 51.0.2704.106 m
Edit | Attach | Watch | Print version | History: r12 < r11 < r10 < r9 < r8 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r12 - 2016-07-26 - HaraldJoerg
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.