Stefan Wehrmeyer

On the things I do.

A Mapnificent World

I just released Mapnificent for 17 cities in the US and some other cities world wide (before Mapnificent was only available for Berlin and somehow for London night buses). You can watch a short video about what Mapnificent is and what it can do here. This post will explain how Mapnificent works and how I scaled it up to more than 20 cities with hopefully more cities to come.

Summary (tl;dr)

I get the data from the awesome GTFS Data Exchange, extract and compress all information necessary to calculate all journeys from any point to every stop in the system in the browser with JavaScript and visualize the result with an HTML5 Canvas overlay on a Google map. I hope people use it to find the best place to live, work and meet and that they discover new things about their public transport system. To make Mapnificent truly useful, there is an API to embed a Mapnificent overlay in your Google Maps application.

Data

Google specified the GTFS format in order to be able to use public transport information in Google Maps. The public transport providers publish a file with all necessary information inside and the magnificent GTFS Data Exchange collects these files and keeps them up-to-date in one place. So getting the data for the cities is easy. The hard part was that I have to read and adhere to the license agreement of every single provider. Some providers like New York’s MTA do it right by not having any complicated license text at all, some like San Francisco require me to put a text of 98(!) words on the site. I don’t know what they hope to accomplish, but I can work with that. However, I have a problem at the moment with the Terms of Use of Los Angeles Metro and San Diego Metro where they say that I’m only authorized to reproduce the data  in the form provided by them. What? Shall I show users the raw GTFS files? I guess not, but Mapnificent aggregates and converts data and I’m unsure if this is covered by the license. I have tried to contact Metro LA on two different channels, but did not get a response (their sites also make it extremely difficult to contact them by email). If you can change anything about that, please feel free to do it.

Routing

From the GTFS data Mapnificent knows all routes and their stops, how long a route needs between two stops and the average headway of each route in a given time range (e.g. in the morning). This is compressed into JSON and delivered to your browser. When you set a point, Mapnificent looks for the closest couple of stations, then starts to travel to every station in the network it can reach and records the minimum amount of time it takes to a station. The calculation process runs in a Web Worker thread. For some smaller public transport systems it is so fast that I can enable the “Calculate on Drag” feature: check it out for Sacramento and Berlin.

There is a major heuristic involved: how can Mapnificent know how long a trip takes when it doesn’t even know the exact time the next bus/train/helicopter departs from a stop? It really can’t know for sure. To compensate for this lack of knowledge Mapnificent makes some assumptions.

It assumes that you will time your journey in a way that you don’t have to wait for your first transport option. Of course this does not always hold in real life, but it’s a reasonable assumption to work with.

It assumes further that if you change transportation, you will probably not immediatley catch the next bus/train/spaceship. The average waiting time could be thought of as headway divided by two. However, I found that the waiting time (for Berlin) is actually lower. That can be explained by the fact that the public transport agency supposedly designed the timetables in a smart way that allows connecting trains/busses/submarines to wait for each other in order to let riders change more quickly. Also people plan their trips in a way that reduces waiting time. So the waiting time can be considered to be less than the mathematical average. I tried around a bit with a logarithmic formula which worked ok for Berlin, but for the whole world I settled for a simpler approach: headway divided by three.

How accurate is it?

Difficult to say. I have actually never been to any of the cities except Berlin and London, so I don’t know if any important transportation options are missing (because they are not available as a GTFS feed) or if anything is totally incorrect. I created Mapnificent for Berlin first (because I usually live there) and then used it as the basis for a seminar at Hasso Plattner Institute (where I’m a Master student). There I actually had the chance to compare the times calculated by Mapnificent Berlin (starting point was Berlin main station) to times from an actual trip planner command line interface. Here is a graph that shows how many errors in minutes there were for trips to every station in Berlin:

You can see that the majority of trips (53 %) have an error less than or equal to 5 minutes and 96 % have an error of less than or equal to 20 minutes. I traced the outliers to stops only reachable by very infrequent buses with headways of around an hour. This gave me some confidence about the relative accuracy of Berlin, but at the moment I can’t do the same analysis for all the other cities I added to Mapnificent. I asked friends to check out some routes in cities they know and it seemed to be sufficiently correct to them. If you find anything totally odd, please contact me.

Performance

Well, to sum it up, I can’t wait for browsers to get true graphics hardware acceleration. JavaScript performance is ok for latest browsers, but drawing a couple of thousand circles noticeably blocks the UI thread in browsers for some cities. Also a really annoying bug in Webkit’s Canvas Compositing prevents the awesome intersection feature from working in Chrome and Safari. If you can change anything about that, I urge you to do it.

Mapnificent requires Web Worker and Canvas to work. I tried the latest official versions of Opera (fast), Chrome (fast), Safari (ok) and Firefox (slow) as well as the latest beta versions of Webkit, Chromium (both seem to have a bug that cuts of the lower part of the canvas) and Firefox 4. I didn’t try Internet Explorer 9 (not sure if they already have Web Worker support), but I don’t have Vista or Windows 7, so I won’t be able to test it there anyways.

If I was a research department in a big company I would say that the product is ready for the browsers of the average web user in 2-3 years and would wait accordingly. But since that is not the case, I release it now, even though it pains me to see people using Mapnificent in suboptimal environments. Please browser developers, make that pain go away soon.

What is it for?

Mapnificent ist not a trip planning service and you should in fact use the trip planning services of the public transport providers that are linked to in the bottom left corner of the site.

Mapnificent is rather a tool to explore many fuzzy trips at once to see more possibilities where to live, to work or to meet up. With the integrated Google Local Search you may discover that even though some place is a few miles away, it is still reachable with public transport in a couple of minutes. Mapnificent on its own is fun to play with and looks awesome, but to uncover the real use cases, it has a JavaScript API that lets you integrate a Mapnificent overlay into your Google Maps application.

The old Mapnificent

I ported Berlin and London to the new Mapnificent, but they both rely on hand scraped data sources which are difficult to maintain. The old blog post about Mapnificent is wildly outdated, but might be fun read. The source of the new Mapnificent is not published as Open Source at the moment (I still have to sort some stuff out), but you can still find the old Mapnificent source on my GitHub account.

Closing Words

I want to thank all the beta testers that gave awesome feedback, Mapnificent would be uglier and less usable without you. I am excited to bring this project to the next level and I can’t even begin to imagine where Mapnificent is going from here.

  1. copywriter-copywriting reblogged this from stefanwehrmeyer
  2. laphotos reblogged this from stefanwehrmeyer
  3. ampyourworlddim reblogged this from stefanwehrmeyer
  4. poipoipoipoile reblogged this from stefanwehrmeyer
  5. relentlessfuchia reblogged this from stefanwehrmeyer
  6. swag-too-dopenah reblogged this from stefanwehrmeyer
  7. thatofmen reblogged this from stefanwehrmeyer
  8. local-ceopro-3-ceo reblogged this from stefanwehrmeyer
  9. mortg-15-payments reblogged this from stefanwehrmeyer
  10. moscato-wines reblogged this from stefanwehrmeyer
  11. cnbcnews reblogged this from stefanwehrmeyer
  12. tumbltumbtumbtumb reblogged this from stefanwehrmeyer
  13. day-spa-and-waxing-san-antonio reblogged this from stefanwehrmeyer
  14. buy-pinterest-repins-followers reblogged this from stefanwehrmeyer
  15. cuildmyranksi37 reblogged this from stefanwehrmeyer
  16. boydoingseo11 reblogged this from stefanwehrmeyer
  17. website-searchs-eng reblogged this from stefanwehrmeyer
  18. stefanwehrmeyer posted this