Geo programming with python and javascript

Aug. 27, 2010

comments

As mentioned in a previous post, I built a widget using Google maps that tracks a friends journey using GPS logs. This post is about how to build such a widget.

I have given Andi the GPS CS1KASP tracker I had laying around.

Download
Logfile format
GPSBabel
XML parsing and cleanup
Google Maps Widget

Below you see the widget for reference.

Download

You can download the code for this howto.

Logfile format

Andi sent me his logfiles via email

bash

ls -l *.log
   10002 2010-08-26 10:46 WG20100815025810.log
 1424810 2010-08-26 10:46 WG20100815162202.log
  779561 2010-08-26 10:46 WG20100816162336.log
  700241 2010-08-26 10:47 WG20100817202507.log
   84433 2010-08-26 10:47 WG20100818033625.log
 1004726 2010-08-26 10:47 WG20100818180009.log
    1800 2010-08-26 10:47 WG20100819204401.log
  789287 2010-08-26 10:47 WG20100819204533.log
  795328 2010-08-26 10:47 WG20100820193232.log

And they look like this inside

@Sonygps/ver1.0/wgs-84
$GPGGA,025810,6018.7672,N,13415.5247,W,1,03,02
$GPGSA,A,2,10,30,31,,,,,,,,,,02.3,02.0,01.0*01
$GPGSV,2,1,07,02,46,071,50,05,27,126,00,30,60,
$GPGSV,2,2,07,31,33,286,31,04,,,28,10,34,081,4
$GPRMC,025810,A,6018.7672,N,13415.5247,W,000.0
$GPVTG,279.2,T,,M,000.0,N,000.0,K,A*03
$GPGGA,025835,6019.1549,N,13416.0216,W,1,03,08

Since google maps can't deal with this directly, we'll have to convert it to KML (short for Keyhole Markup Language).

GPSBabel

GPSBabel is up to that job nicely. First you need to install it

bash

sudo apt-get install gps-babel

I am picking one of the larger logs and convert it to KML

bash

gpsbabel -i nmea -f WG20100815162202.log -o kml -F path.kml

In order to test if it worked, you can upload it to Google maps

Go to My Maps and click on "Create New Map"

Click on "Import"

Pick a file

The result looks like this:

So it basically worked, but it's not very nice. Adding a few options to GPSBabel the result can be improved.

bash

simplify="-x simplify,count=100,crosstrack"
infile="-i nmea -f WG20100815162202.log"
kmlopts="line_color=ff2a7fff,lines=1,trackdirection=1"
outfile="-o kml,$kmlopts -F path.kml"
gpsbabel $infile $simplify $outfile

That's better, but way to many arrows. So let's get rid of some of them.

XML parsing and cleanup

The following function (adapted from geolocator) calculates the distance between a pair latitude/longitude coordinates into kilometers.

python

from math import pi, sin, cos, sqrt, atan2

earthradius = 6371.0

def distance(loc1, loc2):
    '''Haversine formula
        give coordinates as (lat_decimal,lon_decimal) tuples
    '''

    lat1, lon1 = loc1
    lat2, lon2 = loc2

    # convert to radians
    lon1 = lon1 * pi / 180.0
    lon2 = lon2 * pi / 180.0
    lat1 = lat1 * pi / 180.0
    lat2 = lat2 * pi / 180.0

    # haversine formula
    dlon = lon2 - lon1
    dlat = lat2 - lat1
    a = (
        (sin(dlat/2))**2 +
        cos(lat1) * cos(lat2) * (sin(dlon/2.0))**2
    )
    c = 2.0 * atan2(sqrt(a), sqrt(1.0-a))
    km = earthradius * c
    return km

On to parsing the KML file. The module xml.etree from the pythons stdlib is good for such jobs. There's also a couple of helpers that come in handy:

python

from xml.etree import ElementTree as et
import sys

#some helpers
def search(element, parent=None):
    yield element, parent
    for child in element:
        for node in search(child, element):
            yield node

def search_tag(element, name):
    for node, parent in search(element):
        if node.tag == name:
            yield node, parent

def search_folder(element, name):
    for node, parent in search_tag(element, 'Folder'):
        name_node = node.find('name')
        if name_node is not None and name_node.text == name:
            yield node

The document needs to be opened and cleaned up because ElementTree leaves useless namespace information on tags, and GPSBabel outputs a lot of descriptions that only make the file larger:

python

def open_document(name):
    doc = et.parse(name)
    root = doc.getroot()

    # get rid of the namespaces
    for node, parent in search(root):
        node.tag = node.tag.split('}')[-1]

    #delete descriptions
    for node, parent in search_tag(root, 'description'):
        parent.remove(node)

    return doc, root

Then we remove all arrow placemarks that are closer then 50km to any of the placemarks we have let trough

python

if __name__ == '__main__':
    doc, root = open_document(sys.argv[1])

    # remove all waypoints close to previous
    # waypoints
    for folder in search_folder(root, 'Points'):
        places = []
        for placemark in folder.findall('Placemark'):
            placemark.find('name').text = ''
            coords = placemark.find('Point/coordinates')
            latlng = map(float, coords.text.split(',')[:-1])
            
            for previous in places:
                if distance(previous, latlng) < 50: #km
                    folder.remove(placemark)
                    break
            else:
                places.append(latlng)
    
    # output the file
    print  '<?xml version="1.0" encoding="UTF-8"?>'
    root.attrib['xmlns'] = "http://earth.google.com/kml/2.1"
    doc.write(sys.stdout)

Running this program over the first file and putting it into google maps

bash

python cleanup.py path.kml > path_final.kml

And how it looks

I collate all the tracking files into one now, for this GPSBabel supports batch files:

bash

ls *.log > batch.txt

batch.txt now looks like this

WG20100815025810.log
WG20100815162202.log
WG20100816162336.log
WG20100817202507.log
WG20100818033625.log
WG20100818180009.log
WG20100819204401.log
WG20100819204533.log
WG20100820193232.log

Quickly inserting the appropriate options with the vim command :%s^-i nmea\r-f /g

-i nmea
-f WG20100815025810.log
-i nmea
-f WG20100815162202.log
-i nmea
-f WG20100816162336.log
-i nmea
-f WG20100817202507.log
-i nmea
-f WG20100818033625.log
-i nmea
-f WG20100818180009.log
-i nmea
-f WG20100819204401.log
-i nmea
-f WG20100819204533.log
-i nmea
-f WG20100820193232.log

And run it with

bash

simplify="-x simplify,count=600,crosstrack"
kmlopts="line_color=ff2a7fff,lines=1,trackdirection=1"
outfile="-o kml,$kmlopts -F path.kml"
gpsbabel -b batch.txt $simplify $outfile
python cleanup.py path.kml > final_path.kml

Google Maps Widget

The first thing you need for a maps element on your page is a div that can contain it

html

<div id="map" style="width:500px; height:500px"></div>

Then you need to include the Google maps API

html

<script type="text/javascript"
    src="http://maps.google.com/maps/api/js?sensor=false">
</script>

Finally the code that makes the map and points to the KML file has to be added

html

<script type="text/javascript">
    $(document).ready(function(){
        var map = new google.maps.Map($('#map')[0],{
            mapTypeId   : google.maps.MapTypeId.SATELLITE
        });
        var path = new google.maps.KmlLayer(
            'http://the.path.to.the.kml'
        );
        path.setMap(map);
    });
</script>

The jQuery method $(document).ready is executed when the document has loaded. $('#map') is also jQuery syntax to fetch the map element. The path points to the KML file we generated.

That is all you need for an easy and useful little google maps embeding. I hope the explanations where useful, now go and make some mad maps!