GeoLookupPluginDev Discussion: Page for developer collaboration, enhancement requests, patches and improved versions on GeoLookupPlugin contributed by the TWikiCommunity.
• Please let us know what you think of this extension.
• For support, check the existing questions, or ask a new support question in the Support web!
• Please file bug reports in the GeoLookupPlugin bug database.

GeoLookupPlugin Feedback and Development

-- PeterThoeny - 2010-01-23

Peter,

This is an interesting start, but far from complete. It's a harder problem than it appears.

IP addresses in private networks won't work; as many (most?) TWiki deployments are internal, this is a serious limitation. The maxmind data, as you note, has its limits. One also needs to get updates.

Other sources of geolocation data include DNS LOC records (not used as often as one might think, but a good way to handle the private networks); whois, and google maps API.

I have a private standalone CGI script (perl) that handles much of this. It's not suitable for public release as it has several hacks unique to my private network. However, If you're interested in generalizing it and incorporating the logic into this plugin, send me a private email & I'll provide the source.

I can also provide a script that handles the maxmind updates (I run it as a cron job.)

-- TimotheLitt - 2012-07-27

Agreed, this plugin is not suitable for corporate-wide internal use. I had an idea to crowdsource the geolocation with employees, WhereIsPluginDev.

Timothe, I appreciate the offer to provide the source for your DNS LOC records based gelocation. May be at another time, I have too many other things going on.

The script for maxmind updates looks useful. Could you attach it here?

-- PeterThoeny - 2012-07-27

I created a new blog on Free Geolocation Lookup - Where is This Website or IP Address?. It explains in detail how the geolookup works, which is a good learning opportunity for advanced TWiki application programming using SpreadSheetPlugin formulas.

-- PeterThoeny - 2012-08-14

Sorry for the non-reply. I signed up for change notifications on this topic, but didn't get any.

Here is the update process that I use. Don't tweak the crontab frequency too high, as maxmind TOS say you'll be locked out. They seem to only update around the third of each month, but I look every couple of days in case of interim updates. I've also noticed that their servers tend to be down on Sundays recently, so you'll get failure e-mails if you happen to hit that.

If you're not running SELINUX, the chcon can be commented-out.

Enjoy.

Use this crontab entry for updates:

#
# Update GeoIP database every couple of days
#
19 3 * * */2 /var/www/tools/update_geoip
#

The script:

/var/www/tools/update_geoip
#!/bin/bash

#
# Update the geoip databases
#

function updatedb {
    local DBLOC="$1"
    local DBNAME="$2"

    cd /usr/share/GeoIP || exit $?

    # If a zip file is here, restart
    if [ -f $DBNAME.new.gz ] || [ -f $DBNAME.new ]; then
        rm -f $DBNAME.new.gz $DBNAME.new
    fi

    # Download

    if ! wget -q -O $DBNAME.new.gz $DBLOC ; then
        R=$?
        echo "Unable to download $DBNAME database update"
        return $R
    fi

    # Test the .zip file

    if ! gunzip -t $DBNAME.new.gz ; then
        R=$?
        echo "Downloaded $DBNAME.dat.gz failed integrity test"
        return $R
    fi

    # Unzip, compare, update if different

    if gunzip -f $DBNAME.new.gz ; then
        if diff -q $DBNAME.dat $DBNAME.new 2>&1 >/dev/null ; then
#       echo "No Change to $DBNAME.dat"
            rm $DBNAME.new
        else
            cp --preserve=all -f $DBNAME.dat $DBNAME.dat.bak
            mv -f $DBNAME.new $DBNAME.dat
            chcon system_u:object_r:httpd_sys_script_ro_t:s0 $DBNAME.dat 2>&1 > /dev/null
            echo "Updated $DBNAME.dat"
            ls -l $DBNAME.dat.bak $DBNAME.dat
            if [ -x /usr/sbin/apache ]; then
                /usr/sbin/apache graceful
            else
                if [ -x /usr/sbin/apachectl ]; then
                   /usr/sbin/apachectl graceful
               else 
                  service httpd restart
              fi
            fi
        fi
        return 0
    else
        echo "Failed to unzip $DBNAME.dat"

        rm -f $DBNAME.new $DBNAME.new.gz
        return 1
    fi
}

# Country database
updatedb "http://www.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz" "GeoIP"

# City database
updatedb "http://www.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz" "GeoIPCity"

Thank you Timothe! I'll update the docs on next release. I have to look into the e-mail notification issue on twiki.org.

-- PeterThoeny - 2012-09-01