Over the last few weeks I have been teaching second year students a number of digital tools. This week it was Google maps and I thought I would share the process I went through to create a map/mash up example to show them.
So here is the map.
Not too interesting I know. It’s a map of post office closures in areas of the UK shown on a website called public servant daily last October. I just did a search around for the list based on a quick think about a geographically relevant story and this was the first site that popped up with data that looked copy and pasteable.
What I figured I needed was a list of postcodes that I could some how convert in to a feed that Google maps would understand. Now you can point the spreadsheet directly at a map via Google maps spreadsheet importer thingy but I didn’t want to do a lot of hand converting post codes in to lat/long information. There are some tools that will do it, like this one, but they all seem to need some kind of preformatting. So I needed something that would do that ‘more’ automatically. That’s where yahoo pipes came in.
A quick search around pipes found me several examples of postcode conversion pipes to play with.
Setting up a spreadsheet
So to start I cut and paste the data in to a Google spreadsheet. If you look at the sheet you can see it is actually a bit of a mess in places but it’s raw data. Truth be told I went through excel to do some column splitting and then combining to get the postcode out of the address.
Next step was to publish the spreadsheet so I could use the data. You can publish in a number of formats but the pipes examples I had seen used CSV (comma separated values) and that seemed easier than trying to deliver the RSS feed.
publish your spreadsheet as a csv file
The publish option gives you a URL for the CSV feed. By setting the publish options to automatically update I knew it would be the most up to date.
Yahoo pipes
The postcode to lat/long pipe
Now that I had the feed I needed to get it in to pipes. So the first block is the Fetch CSV (found in Sources) option. This essentially gets the csv feed published from Google docs and spits it out in a list based on the columns in the sheet.
Then a loop block (found in operators) takes each item in the spreadsheet and runs a Location builder operation. To combine the two, you add the loop operator and then drop the Location Builder in to the space.
Drop the location builder pipe on the loop
The location builder “converts a description of a place into geographical data.” You can see that is set to use column 4 as the location- that’s the postcode – so we can generate the lat and long information that google needs to plot the points on a map.
The output of that loop are packaged up in the item.loop.locationbuilder results. So the last thing to do is make sure that the results are in a format that Google maps can understand. As far as I understand it Google plays with the Geocode RSS format and fitting with that convention seemed to be the norm on other pipes. So the next box takes each element and renames it in to something more Geocode friendly.
I’ve used column 1, the post office name and column 5, the post office address (or a truncated version) as a description. The lat and long results from the location builder are then renamed to match the requirement for geocode RSS.
The result of that is then plugged in to the pipe output.
Get a feed
When you save the pipe you can then run it and it spits out a fairly plain list. You can subscribe to the list of results in a number of ways including the RSS with all the geocode content. If you right-click over the link you can copy the url.
If you take that link over to Google maps and paste the RSS url in to the maps search box. Hey presto! the results are plotted.
You can then click the Link to this page option and gather the link data to embed on your site. Result!
Conclusion
This may be old hat to some. New to others. What pleased me was how easy this was made by the way pipes allows you to share and edit other peoples work. It feeds my approach of ‘oh, that’s good you could use that to do this..” hacking.
I’m also thinking that the new Google forms thing will make interesting addition to this mix. All it needs is a bit more data and more flexible pipe to make for richer content and things could get very mapalicious. Hope there is something there to play with.
Raw material
17 Responses
nigel barlow
February 29th, 2008 at 10:48 am
1I don’t believe it Andy.This was going to be my idea for the digital for the Digital Newsroom presentation.Really I had already started working on this yesterday afternoon.
I hope that I can still use it and not get accused of plagarism.
That’s two things now and you have moved the predentation to my wife’s birthday and she is not very happy either
Craig McGinty
February 29th, 2008 at 11:17 am
2I’ve been playing with this as well, is very exciting. Trying to get the new form option set up to allow people to submit event details to a calendar that’s hooked up to a map.
Anyway, here are a couple of interesting article that might prove useful:
http://blogs.open.ac.uk/Maths/ajh59/012594.html
http://blog.wired.com/monkeybites/2008/02/pipes-mashup-tu.html
Happy mapping
Paul Daniel
February 29th, 2008 at 1:04 pm
3How about using a Fetch Page module in Pipes instead of the spreadsheet?
http://pipes.yahoo.com/pipes/pipe.info?_id=mpxQ9L7m3BGH_0PcjknRlg
Andy
February 29th, 2008 at 5:10 pm
4Paul,
Thanks, neat solution. My Regex is nowhere near up to hacking around like that but now i have an example I can work through. Cheers
alexander katzeff
March 1st, 2008 at 4:26 pm
5Here’s a great video on the folks behind the pipes team @ yahoo!: http://next.yahoo.net/archives/36/under-the-sink-with-pipes
Tom Whitwell
March 6th, 2008 at 9:58 pm
6Thanks for a fantastic tutorial. I just used this to scan the results from this survey I’ve been running on Times Online using Google Spreadsheets – 1,000 results so far. http://business.timesonline.co.uk/tol/business/money/article3484700.ece
Thing I love/hate about Google apps is that once you’ve got one thing working, you start yearning for the next bit of functionality. I want colour-coded tiny icons (or even postcode-shaped polygons), and I want to be able to do them as easily as I can do this.
Oh well, time to read the API properly and learn javascript.
Andy
March 6th, 2008 at 10:05 pm
7Tom
Glad it was of some use. Thanks for an example I can show the students to prove that I don’t just make this stuff up.
Dan N. Moldovan
March 11th, 2008 at 10:22 am
8If you like Yahoo Pipes take a look at my MacrosReader and play with more than 25 pipes.
http://reader.macrostandard.com/
Thank you!
Times online and google maps by andydickinson.net
March 11th, 2008 at 11:14 am
9[...] map was built using Yahoo pipes and google maps, based on a post of mine with a little google forms magic built [...]
Times Online experiments with crowdsourcing | Daily EM
March 12th, 2008 at 8:41 am
10[...] the Times staff got inspired by a detailed post that Andy Dickinson, a popular journalism blogger, wrote on this blog about how to create data [...]
James
August 1st, 2008 at 2:18 pm
11I’ve followed the steps and the link works (very interesting and useful, thanks) but when others try to load the page using the google maps link, none of the data shows – any ideas?
pop
November 3rd, 2008 at 3:52 am
12Help ME PLZ, I do not known why map not shown , Please recommend Plz !!!!
Thx so lots
LINKS | Micropayments don’t work, but everyone has a better idea | byJoeyBaker
February 22nd, 2009 at 4:33 am
13[...] Maps, spreadsheets, yahoo pipes and post offices by andydickinson.net: How to use a gdoc to create a google map of locations [...]
RichardCoe
March 6th, 2009 at 10:21 am
14Hi andy, this is great but unfortunately I’ve been frustrated at the last hurdle – I can generate a pipe which spits out my list of data with the geo: lat and geo: lon associated with a title. I get the RSS feed URL but then when I paste that into the google maps search box it tells me it’s not a valid KML or KMZ file. Any thoughts? thanks in advance….
Andy
March 6th, 2009 at 2:06 pm
15Hi Richard
Can you point me in the direction of your pipe and I’ll take a look
Andy
Alex
July 9th, 2009 at 8:56 am
16Hi guys,
how do i change the spreadsheet and my pipe to create different icons or colors?
i would like to have informations in my kml-file, so that gmaps will show automaticly different icons.
hope you can help me.
Alex
Data Mashups with Google Spreadsheets and Yahoo Pipes | BryanPovlinski.com
March 21st, 2010 at 4:35 am
17[...] I found a couple of fascinating blog posts on how to mashup data from anywhere on the web with Google spreadsheets and Yahoo pipes. To see the exact steps on how to do this please refer to those 2 articles. It’s [...]
RSS feed for comments on this post · TrackBack URI
Leave a reply
Of interest
Stuff you may be looking for
Categories
Archives
Widget
JOURNALISMDAILY.COM
Disclaimer
Andy would like to point out that the views expressed in this blog are his own and do not reflect the views of the University or Department of Journalism.
RSS Feed
A word from our sponsors
What I'm twittering.
journalism
Mac
Online
Tools
video
Recent Posts
Recent Comments
my del.icio.us
andydickinson.net is proudly powered by WordPress - BloggingPro theme by: Design Disease