OpenStreetMap Haiku:
Using OSM and Overpass
for generative poetry
Some corner of London, last Monday, around 11am:
Wet to the boneLooking at you from the 4th floor
Is it too early for a beer?
Same place, more random poetry:
A couple getting a loanYawns in the classroom
A train underneath
Or maybe, a few miles from there:
On the water’s surfaceGoing to work
Flipping burgers
Here's what's happening: we automated making haikus about places. Looking at every aspect of the surroundings of a point, we can generate a poem about any place in the world. The result is sometimes fun, often weird, most of the time pretty terrible. Also probably horrifying for haiku purists (sorry). Go ahead and give it a try.
But, why ?
That's a good question. Because we can ?
The original inspiration for this is a project called every thing every time by Naho Matsuda. Her work aims at creating "impractical poetry" from a variety of data streams and sensors across the city: air quality, traffic, shift schedules, mosque prayers, etc. The result is then displayed in real time in the city streets. We've always been totally fascinated by the project, which at the time seemed a refreshing take, slightly impertinent, on the whole "smart cities" schtick.
We thought that creating a global version of the same idea would be interesting, not the least because it would allow us to get our hands dirty with OpenStreetMap data.
So in OpenStreetMap Haiku, verses are randomly assembled collecting information about a place, taken from OpenStreetMap. This post goes through the steps involved, which hopefully should help understand how OSM data works.
As they say: “OpenStreetMap is not a map®”. One can rather think about OSM as a gigantic database of all the things in the world. In fact, there is such a thing as a list of all the things in the world, check it out: OSM map features.
Hey, this is nothing less than an attempt at methodically classifying our reality. So yes, it is a pretty damn long wiki page. You were warned.
In OpenStreetMap Haiku, we use that crazy amount of data by matching OSM tags with random verses.
For example, if a supermarket is close to the map center (an object has a tag shop=supermarket
), it would randomly yield one of: "Salad cabbage and carrots", "The cashier’s bored" or "A lonely aisle" (etc).
A swimming pool (tag leisure=swimming_pool
): ‘Smells of chlorine’.
A coffee shop named ‘Grey House’ (tags amenity=cafe
and name=‘Grey House’
: “Fresh coffee from Grey House”.
And so on.
That set of verses is easily extensible - have a look and add yours :). We also take into account the weather and local time of the day, but for now let's focus on the geographical aspect of things: OpenStreetMap and the mighty Overpass. We'll see how Overpass is our gateway to tap into the incredible richness of OpenStreetMap.
An OpenStreetMap refresher
OpenStreetMap can be a bit overwhelming at first. Not only it's a lot of data, but the software ecosystem that surrounds it is diverse and complex. Might seem obvious, but the best place to start is there: openstreetmap.org.
Counter-intuitively, openstreetmap.org is not great at being a useful map and doesn't really try to be. It is excellent though, as an entry point and a exploration tool for OpenStreetMap data.
Using that 'Query features' functionality is pretty much what we'll be programmatically doing to generate haikus. So far, what you see is a bunch of named objects. Looking further, you'll start noticing that these objects have relationships: a metro station belongs to a portion of a railway, which itself belongs to an entire metro line:
And so goes the holy trinity of OpenStreetMap: Nodes, Ways, Relations. Typically an object (a metro line, a golf course, a ski resort) can be represented by a relation. This relation does not have its own geometry, but rather links to either a bunch of points aka nodes (a golf hole) or a bunch of lines aka ways (a ski piste), which themselves link to point nodes.
Those three basic primitives can each hold an unlimited amount of data. Any number of tags, of any form, can be added to any of those three types of objects. See for yourself:
Those tags are always composed of a key=value
tuple, and are completely free form: anyone can add any tag they deem useful, there is no schema or spec of any sort. There are conventions, though. In general by way of consensus - our carpet hanger tag for example, was approved by the community at some point. In fact, there is a dedicated tagging mailing-list, which is a fascinating place, with questions ranging from tagging fraction house numbers? and choosing between bbq=yes or barbecue_grill=yes to much more complicated geopolitical issues such as tagging disputed boundaries in Crimea.
This is very nice, but how do we programatically access this data? There is such a thing as an OpenStreetMap API. But we should avoid using it for simple read-only scenarios. The OSM API is mostly designed to be used by editing software, a lot of its endpoints are meant for writing, and reading operations are awkward for simple use cases. So here comes our joker:
Overpass, and how to tame it
Overpass is a read-only API to OpenStreetMap, which uses its own query language, Overpass QL, and has its own handy sandbox tool located at overpass-turbo.eu.
Overpass is fantastic and weird and powerful and scary. Its query language is very terse, very OSM-specific, very powerful, and can be slightly off-putting, especially for those of us not entirely fluent in Malbolge.
So, young and full of ideals, you set out for overpass-turbo.eu, and, oooh look: it has a friendly "wizard" function. Give it a go with the suggested params. It will generate a query on the left panel, but it's quite hard to tell what's going on. So, not quite the friendly wizard - more like black magic:
Let's try to decipher Wizard-generated query. Go to Overpass Turbo and find the Eiffel Tower, in Paris, and copy paste the following:
What this means is 'retrieve all the nodes that represent trees ([natural=tree]
) in the currently visible map area (({{bbox}})
). Then give us the result of that query. Check out the nodes appearing in the map when you click 'Run'. They have extra information such as circumference, height, and even the species of the trees! Now let's look at output options by looking at kebab places:
A nicer JSON output now replaced the default XML format ([out:json];
), and we've asked for only the geometries, without the tags (out skel;
). We also specified a specific area filter, a radius around
a point, so the query should look for all kebab places in a 3000 meters radius around the Eiffel Tower (around:3000,48.8605,2.2957)
, instead of the `bbox` specified by the frontend. You should see in the map various kebab restaurants around the Eiffel tower such as "Kebab Élysée" (click on the node id to access the full data on osm.org).
Now let's explore hierarchical relations with a bus stop/line:
This does not yield anything visible on the map, but have a look at the data tab, and you'll see an object representing the whole bus line:
This happened because we added that instruction: <;
in our query, which means we want to retrieve all the objects "owning" the selected nodes: in that case, the relationship object representing the bus line, that this bus stop belongs to (recursing up). The next step is to recurse down from that:
What happened? From a single bus stop, queried around a point, we queried the parent bus line relationship. Then we queried all the bus line "children", which resulted in visible geometries on the map: all the bus line stop nodes/points, as well as the actual itinerary of the bus line as a way/line.
Hopefully this should have given you some clues to untangle that wizard-generated query, and more. Here's a useful breakdown:
APIs for Haikus
For OpenStreetMap Haiku we built an Overpass query that tries to answer the question "what's around that point?" (adapted from the query used in osm.org's "Query features" functionality):
A few things to note:
- it's a template where the clicked point latitude and longitude, and a hardcoded radius are injected;
nwr
is a shortcut to query all 3 of node, ways, and relations;[~"."~"."]
will fetch all object that have any key and any value for it's tag, effectively discarding geometries without a tag;- there's a second query for
landuse|landcover|natural
tags with 4 times the set radius - the idea being that we want to "look further" for landscape features
Check out the Overpass QL reference for specific language elements.
It's not only about where
Things get a bit more interesting when we not only look at "here", but also at "now". We're using two additional APIs for that:
- TimeZoneDB to get local time from a lat/lon pair
- OpenWeatherMap for local weather.
This allows some really cool variations on the base verses, by adding conditions to some of them or injecting extra information in the templates. See for instance:
But really, why ?
We think we have created a little window into, maybe not the world, but into the world of OpenStreetMap. OpenStreetMap is a revolutionary project, it's the most complete map of the world ever made. An idea that is usually conveyed by enumerating grand statistics: 5 million users, 5 billion nodes, 3 million changes a day, 100 million distinct tags, etc. We think this can and should also be expressed in a more sensible manner and with a pinch of randomness. Doing what a contributor to OSM does: stop, and look around.
It's also always fascinating to see how, with just a few hundreds of very basic building blocks, meaning emerges from chaos, the imagination unravels, narratives materialize...
In the garage where I belongLeaving home
Two children
Let us know what you think: contact@satellitestud.io or create an issue on Github.
The day is youngThe world is big
Above the freeway noise