Question

The way the TWiki scripts create passwords in the .htpasswd file seems to be incompatible with Apache 2 running on Windows 2000.

Details

I followed the instructions in WindowsInstallCookbook.
I followed the instructions in TWiki.TWikiInstallationGuide#Enabling_Authentication_of_Users.

I copied TWikiRegistrationPub to TWikiRegistration.

Registering a user creates an entry in .htpasswd that looks like this:
WikiCreated:{SHA}1epOQN2/x252PFBQSSqy4Von4B4=

Attempts to authenticate using this user fail (the authentication dialog box re-pops up three times until it finally gives up and goes to the oops page.)

Curious about this, I decided to try to create an entry in .htpasswd directly, to ensure that the rest of my setup is ok.

htpasswd .htpasswd htpasswdMD5Created

This results in an entry that looks like this in the .htpasswd file:
htpasswdMD5Created:$apr1$CE/.....$KZcv5xWuYAhXvqPW.dwtw0

You'll notice that this does not start with {SHA} and is a bit longer.

Results:
Authentication using the htpasswdMD5Created user succeeds. I am able to edit pages when I am logged in like this.

Apache's htpasswd tool also has the ability to create SHA entries in the .htpasswd file by using the -s switch, like this:
htpasswd -s .htpasswd htpasswdSHACreated

Using the same password as I did for the other users results in an entry that looks identical to the one that the TWiki registration script creates. My complete .htpasswd file is shown here for easy comparison:

WikiCreated:{SHA}1epOQN2/x252PFBQSSqy4Von4B4=
htpasswdSHACreated:{SHA}1epOQN2/x252PFBQSSqy4Von4B4=
htpasswdMD5Created:$apr1$CE/.....$KZcv5xWuYAhXvqPW.dwtw0

Trying to authenticate with the htpasswdSHACreated user also fails.

It seems to me that Apache (2.0.40, at least) does not like these {SHA} types of passwords (even though it's htpasswd tool can create them in this format.) This is a problem, since the code in TWiki's register script creates passwords in this format on a Windows box.

• TWiki version: 01 Dec 2001
• Web server: Apache 2.0.40 on Windows 2000
• Server OS: Windows 2000
• Web browser: IE 5
• Client OS: Windows 2000

See Also

• ApachePasswords

-- TimBailen - 13 Sep 2002

Related: FailedAuthenticationWithApache2OnRedHat8

-- PeterThoeny - 17 Oct 2002

I've run into the same problem with the setup as described above. I've applied the patch recommended in ApachePasswords by RalfHandl but still can't get authentication to work. Does anyone know of a solution that will work with this configuration?

-- MichaelBrand - 12 Nov 2002

All current versions of Apache running on Windows now use the MD5 authentification. This is a problem as I don't believe these passwords can be created or validated outside of the Apache server or htpasswd utility. By current I mean both Apache releases 1.3.26+ and 2.0.39+

-- JonLambert - 30 Nov 2002

Here's a quickie hack. I made this against the 112802 alpha version.

In lib/TWiki.cfg add this in the section # flag variables that could change:

#                   Use htpasswd.exe password generation 
$doHtpasswdAuth = "1";

In lib/TWiki.pm add this in the section # TWiki config variables: use vars qw( list

   $doHtpasswdAuth

In bin/register change the following in the main routine

    # generate user entry and add to .htpasswd file
    if( ! $remoteUser ) {
        if ( $TWiki::doHtpasswdAuth ) {
            my $cmd;
            $cmd = "htpasswd -b $TWiki::htpasswdFilename $wikiName $passwordA";
            $cmd =~ /(.*)/;
            $cmd = $1;       # safe, so untaint variable
            `$cmd`;
        } else { 
            htpasswdAddUser( htpasswdGeneratePasswd( $wikiName, $passwordA ) );
        }    
    }

Make sure htpasswd.exe is accessible via $safeEnvPath This does not make bin/installpasswd and bin/passwd useable

I apologize in advance for this code as today is the first time I've ever touched perl. :-P

-- JonLambert - 30 Nov 2002

Do you have a link talking about the change to require MD5 + Basic Authentication in Apache? I couldn't find anything on apache.org, and I'd be surprised if this is the only option now. It seems like MD5 + Basic Authentication is now the default, requiring a config change for other mechanisms, but I'd like to understand when this change happened and why, before requiring it in TWiki.

As long as you have AuthType Basic in your twiki/bin/.htaccess file, you should be using Basic Authentication. I have this working with Cygwin Apache 1.3.24.

Some URLs that would be useful in writing a pure Perl version of the MD5 based user+password creation code are:

• Crypt::PasswdMD5 - CPAN module to do Apache-compatible MD5 encryption - use the apache_md5_crypt function.
• Code for Apache dbmmanage that uses Crypt::PasswdMD5 - Apache licensed
• sample code and explanation

The downside is that this requires two more modules - Crypt::PasswdMD5 and Digest::MD5.

One key point is that the MD5 encryption needs an 8-character salt field - the current TWiki code in the htpasswdGeneratePasswd routine uses 2-character salt (since the normal Perl crypt uses DES crypt on Unix). It would be quite easy to turn this salt generation into a loop, ideally in a separate routine that is passed the number of characters required.

Your code should be OK on Windows, since I don't think Windows lets other users see the command line arguments to htpasswd, but if run on Unix it would let anyone see passwords by running ps -ef in a loop, looking for htpasswd.

-- RichardDonkin - 30 Nov 2002

I just reviewed the latest source codes for Apache (2.0.43 and 1.3.27). There is no SHA1 validation in 2.0.43, but there is in 1.3.27. The 2.0 version checks for the MD5 identifier $apr1$ and if not found defaults to crypt(), if available. On Windows crypt() is replaced with a plain text string comparison.

-- JonLambert - 30 Nov 2002

Here's another quickie hack for plain text password support. Again it's against the 112802 alpha version.

In lib/TWiki.cfg add this in the section # flag variables that could change:

#                   Use plain text password authorization
$doPlainTextAuth = "1";

In lib/TWiki.pm add this in the section # TWiki config variables: use vars qw( list

   $doPlainTextAuth

In bin/register insert the following at the top of the htpasswdGeneratePasswd routine

sub htpasswdGeneratePasswd
{
    my ( $user, $passwd ) = @_;

    # support for plaintext passwords
    if ( $TWiki::doPlainTextAuth ) {
        return "$user\:$passwd";
    }

In bin/passwd insert the following at the top of the htpasswdGeneratePasswd routine

sub htpasswdGeneratePasswd
{
    my ( $user, $passwd ) = @_;

    # support for plaintext passwords
    if ( $TWiki::doPlainTextAuth ) {
        return "$user\:$passwd";
    }

and this at the top of the htpasswdCheckPasswd routine

sub htpasswdCheckPasswd
{
    my ( $old, $oldcrypt ) = @_;
    my $pwd ;

    # support for plaintext passwords
    if ( $TWiki::doPlainTextAuth ) {
        if( $old eq $oldcrypt ) {
            return "1";
        }
        return "";
    }

That seems like a good enough solution for my purposes.

From what I've read of HTTP 1.1 headers is that if you aren't using SSL the passwords are sent across the net in unencrypted base64 format anyways if you're using Basic authentication regardless of how Apache stores them, no?

-- JonLambert - 30 Nov 2002

The issue isn't the cleartext transmission of passwords, though that is important. This problem is that on multiuser systems that use plain text passwords means that other users may be able to see the passwords of all the TWiki users.

-- FrancisLiu 30 Jan 2003

Confusingly, it turns out that there are two entirely separate ways of using MD5 with Apache authentication, in both Apache 1.3 and 2.0:

• Basic Authentication: the password is encoded during transmission in Base64, i.e. basically plain text. It can be stored in plain text, DES crypt format (Unix only), MD5 format (starts with $apr1$) or SHA1 format (only in Apache 1.3, removed in Apache 2.0, starts with {SHA}). Supported by all browsers, and storage format makes no difference at all to the protocol. The MD5 format is the default on Apache 2.0 for Windows.
• Digest Authentication: the authentication is done 'on the wire' through a challenge/response protocol based on MD5 digests. This is nothing to do with the MD5 storage format mentioned above, and in fact the htpasswd file format for this has 3 fields instead of 2 for the above MD5 format. This is created using the htdigest command not htpasswd, and is supported only by relatively recent versions of IE, Netscape/Mozilla and Opera.

I tested creation of passwords in htpasswd for MD5 Basic Authentication on Apache 1.3.24 and it works fine - I also wrote some Perl code to do this but it is currently separate to register (based on the very useful HTTPD:UserAdmin), and needs the new Crypt::PasswdMD5 module to work. However, since SHA1 has gone away in Apache 2.0 (apart from in htpasswd -s!), MD5 basic auth is the best choice IMO to run a reasonably secure TWiki server.

Although plain text storage format makes no difference to the on-the-wire protocol, it does mean that anyone with the rights to run a CGI script as the Apache nobody userid can see the password used. At least with MD5 Basic Authentication the storage format makes it hard to get the password (which could be used on other systems), though it does let someone with scripting knowledge log on as that user via a script. So I'm reluctant to support plain-text storage format in TWiki as it creates a much bigger security hole - people sniffing packets are not as numerous as those who have accounts on web servers, particularly in web hosting environments.

-- RichardDonkin - 01 Dec 2002

Hmmmm. I'm just about to move my trial TWiki from an unauthenticated Netscape Enterprise Server (which as nonroot I have little control over) to an intranet LDAP authenticated Apache server (which I am installing myself). I'd really like to use Apache 2.0 as I don't have to go chasing a non-core mod_auth_ldap module (it's included in 2.0). As a relative ignoramus, would I be right in saying that since I won't be using htpasswd for authentication, then this MD5 issue will not affect me?

Or is there another reason entirely why I shouldn't use Apache 2.0?

-- GarethEdwards - 31 Jan 2003

This issue won't affect you as long as you use LDAP for authentication, as you say. There are some other issues with Apache 2.0, so if you want a quiet life I'd use Apache 1.3 for now - see KnownIssuesOfTWiki01Feb2003. However, patches to fix bugs when using 2.0 are always welcome!

-- RichardDonkin - 11 Apr 2003

I've solved this problem this way:

[Changing twiki/lib/user/htpasswduser.pm] sub _htpasswdGeneratePasswd

...
    } elsif ( 'md5' eq $TWiki::htpasswdEncoding ) {
      my $cmd;
      $cmd = "htpasswd -nbm $user $passwd";
      $cmd =~ /(.*)/;
      $cmd = $1;       # safe, so untaint variable
            
      open(RES, "$cmd |");
      my $result = <RES>;
      chomp($result);
      $result =~ s/^(.*)?://;
      close(RES);
      $encodedPassword = $result;#Digest::MD5::md5_hex( $toEncode );

    }
...

And, of course, change in TWiki.cfg -

$htpasswdEncoding= "md5"

But you should be sure that htpasswd.exe is accessible from your script.

-- AlexanderSorokin - 29 Sept 2004

I found that by upgrading to Apache 2.0.52 (windows) that SHA encrypted passwords in .htpasswd started working

-- LyallPearce - 15 Dec 2004

Thanks for noting this, will update IssuesWithApache2dot0. There is another issue with Apache 2.0 if you use I18N features, see InternationalisationIssues, so it probably shouldn't be recommended in TWikiSystemRequirements yet.

-- RichardDonkin - 16 Dec 2004