Stefan Wehrmeyer

On the things I do.

It’s Mapnificent

This blog post is wildly outdated in most parts. A description of the latest Mapnificent version can be found here: A Mapnificent World.

Mapnificent visualizes interesting data sets with interactive layers on a map of Berlin. It was inspired by Mapumental, a project by MySociety. Mapumental looks awesome and I wanted to build something like that. I am going to talk about how I implemented Mapnificent, specifically the Public Transport layer and how it compares to Mapumental.

But let me shortly describe how Mapumental seems to work (did not try it myself, it’s still in private beta). The project page of Mapumental on mysociety.org explains that Mapumental is basically a flash application that loads images from a tile server. As I understand it, when you set a point on the Mapumental map covering whole UK, Mapumental will calculate the quickest route from every station in UK (there are 300.000) to this point with an optimized routing algorithm written in C++ based on the actual timetables combined with heavy caching. It’s kind of hardcore and requires a cloud setup to scale.

Before Mapumental there was Travel Maps by mysociety, a program that calculated journeys and displayed them in one image. However, you have to fill in a form to request a quote for a custom map. So it takes a lot more time and is not for free. I am just guessing here, but they probably just made a lot of requests to a travel planner site and scraped the result for their custom maps.

I started similarly. I build an interactive map that shows how long a journey to the Hasso Plattner Institute in Potsdam, Germany would take when you start from any point in a grid over the general area of Berlin. For this I actually queried the VBB Fahrinfo site about 30.000 times and it took a day to gather all the data (their service is not really fast).

When I had a minute value for every point in the grid, I wondered how to visualize this efficiently without falling back to proprietary technologies like flash and heavy stuff like tile servers. My first approach was using Google Maps API Overlay Classes. To keep it simple: doesn’t scale. Too many datapoints each one represented in the DOM did not work for me.

Then I experimented with the Canvas element. I placed a canvas outside but still over the Google Map. Obviously that sucks: you can’t click on the map anymore. With the help of the really handy ELabel (crazy homepage) I placed the canvas inside the map: now you can click on the map and move it, great. But wait, how about zooming in? That doesn’t work. My fix was making the canvas bigger so it would cover the same geographical area. However, apparently no browser I know can handle a canvas with over 50 million pixels. They all just crash and it’s kind of understandable. To counter that I restricted zooming so a user could only zoom to a certain level. But that sucks, too. What I wanted was a canvas that should not grow too big, but the user should still be able to drag the map around without seeing edges of the canvas where there shouldn’t be any. So some not so pretty calculations later and the dimensions of the canvas are now three times bigger than the respective dimensions of the map and the position of the canvas will adapt when a user drags the viewport of the map over a certain threshold.

So that was the visualization part, no flash, no tile-server. What about the Public Transport map that works for every point? Any C++ written routing in the cloud? No, it’s all done in JavaScript. And takes less than two seconds to calculate. So, is this really possible? No, of course not, I fake a lot: I don’t use actual timetables, but only estimates, don’t care about buildings when walking from/to a station and sometimes I am really just guessing a number. But it turns out that this approach still works good enough.

Details? Mapnificent basically knows the location of every station in Berlin, the lines that connect them, the time between two adjacent stations on a line and the time interval of the line. When a user clicks on the map, Mapnificent finds the next couple of stations, calculates the time to walk to them and then takes all the lines from these stations and every other reached station. When switching lines it “waits” half the interval of the next line (or currently six minutes if the line interval is unknown). Mapnificent remembers the time it took to get to a station and if you reach that station with a time less than before, it overrides it and keeps going. If it reaches a station where another line got faster, it stops. Really simple recursion. The outcome is a list of minutes it takes to get to every station. Mapnificent then simply draws a growing circle from every station as soon as this station is reached.

I can’t prove it, but it looks good and by checking some known routes I would say the margin of error is generally at most +-4 minutes which I think is still acceptable if you just want to get the big picture. And Mapnificent is just for the big picture anyways. If you want the real thing, you still need to ask an online trip planner. And if there is S-Bahn chaos in Berlin again, you can’t even trust the trip planner.

Since I have not used Mapumental myself, I can’t really say how it compares to Mapnificent. Mapnificent just works for Berlin whereas Mapumental apparently works for the whole UK. Mapumental is probably really accurate with actual timetables whereas Mapnificent uses a good basis of public transport information and some guesstimates. Mapumental looks really awesome, it needs some time to calculate a route if not cached (10 to 30 seconds according to their website), but the interface is really fast (flash + rendered tiles). Mapnificent looks awesome too (with some flaws here and there), calculates routes faster (but less correct) and the interface is fast enough, but the speed depends on the browser, JavaScript engine, implementation of canvas, number of active layers and zoom level. Currenlty the Mapnificent interface won’t really scale with a lot of active layers (that’s why you can deactivate them).

However, the point I want to make is that the setup of Mapumental is apparently quite complex, expensive (server side routing, rendering, caching) and generally hardcore while Mapnificent is just JavaScript and Canvas running in the browser and performs the same task with comparable results.

JavaScript also makes it really easy to open up the framework for addional layers. Users can add their own layer by just writing a couple of lines of JavaScript and providing a JSONP data source. Have a look at the Mapnificent documentation for details.

I want to thank the creators of Mapumental for their great and really inspiring work.

Update:

I got an invite to Mapumental’s private beta! First hurdle for a non-UK citizen, how the hell do UK postcodes work? You need to enter one of the location you want to arrive at. They give examples, but I wanted to arrive in London and it’s really hard to guess a UK postcode right. First impression when I finally found one on the net: Looks nice, tile server is slow (obviously they can’t compare to Google Maps, but I guess they are working on it) and the Flash overlay lags a bit when dragged.

But all that aside, it is really impressive: you can actually travel the whole UK! I knew that before, but when I saw it, I understood the amount of work behind it. Mapnificent has only 2213 stations for Berlin, that’s 3% of what Mapumental is providing. It’d be really interesting to scale Mapnificent up to this magnitude (or is it mapnitude?). It’s relieving that they are actually also just drawing circles from stations and that their travel patterns look similar: it means I got it quite right. Well, let’s see where we go from here.

  1. laphotos reblogged this from stefanwehrmeyer
  2. compare-prices-uk reblogged this from stefanwehrmeyer
  3. nikon-d7000-reviews reblogged this from stefanwehrmeyer
  4. johntempestblog reblogged this from stefanwehrmeyer
  5. zoik reblogged this from stefanwehrmeyer
  6. stefanwehrmeyer posted this