maps – Школа за податоци – Македонија

A Weekend of Data, Hacks and Maps in Nigeria

olubabayemi — Tue, 16 Sep 2014 19:11:44 +0000

It was another weekend of hacking for good all around the world, and Abuja, Nigeria was not left out of the weekend of good, as 30 participants gathered at the Indigo Trust funded space of Connected Development [CODE] on 12 – 14 September, scraping datasets, brainstorming creating technology for good, and not leaving one thing out – talking soccer (because it was a weekend, and Nigeria “techies” love soccer especially the English premiership).

Participants at the Hack4Good 2014 in Nigeria

Leading the team, was Dimgba Kalu (Software Architect with Integrated Business Network and founder TechNigeria), who kick started the 3 day event that was built around 12 coders with other 18 participants that worked on the Climate Change adaptation stream of this year #Hack4Good. So what data did we explore and what was hacked over the weekend in Nigeria? Three streams were worked :

Creating a satellite imagery tagging/tasking system that can help the National Space Research Development Agency deploy micromappers to tag satellite imageries from the NigeriaSat1 and NigeriaSat2
Creating an i-reporting system that allows citizen reporting during disasters to Nigeria Emergency Management Agency
Creating an application that allows citizens know the next water point and its quality within their community and using the newly released dataset from the Nigeria Millennium Development Goal Information System on water points in the country.

Looking at the three systems that was proposed to be developed by the 12 coders, one thing stands out, that in Nigeria application developers still find it difficult to produce apps that can engage citizens – a particular reason being that Nigerians communicate easily through the radio, followed by SMS as it was confirmed while I did a survey during the data exploration session.

Coders Hackspace

Going forward, all participants agreed that incorporating the above medium (Radio and SMS) and making games out of these application could arouse the interest of users in Nigeria. “It doesn’t mean that Nigerian users are not interested in mobile apps, what we as developers need is to make our apps more interesting” confirmed Jeremiah Ageni, a participant.

The three days event started with the cleaning of the water points data, while going through the data pipelines, allowing the participants to understand how these pipelines relates to mapping and hacking. While the 12 hackers were drawn into groups, the second day saw thorough hacking – into datasets and maps! Some hours into the second day, it became clear that the first task wouldn’t be achievable; so much energy should be channelled towards the second and third task.

SchoolofData Fellow – Oludotun Babayemi taking on the Data Exploration session

Hacking could be fun at times, when some other side attractions and talks come up – Manchester United winning big (there was a coder, that was checking every minutes and announcing scores) , old laptops breaking (seems coders in Abuja have old ones), coffee and tea running out (seems we ran out of coffee, like it was a sprint), failing operating systems (interestingly, no coders in the house had a Mac operating system), fear of power outage (all thanks to the power authority – we had 70 hours of uninterrupted power supply) , and no encouragement from the opposite sex (there was only two ladies that strolled into the hack space).

Bring on the energy to the hackspace

As the weekend drew to a close, coders were finalizing and preparing to show their great works. A demo and prototype of streams 2 and 3 were produced. The first team (working on stream 2), that won the hackathon developed EMERGY, an application that allows citizens to send geo-referenced reports disasters such as floods, oil spills, deforestation to the National Emergency Management Agency of Nigeria, and also create a situation awareness on disaster tagged/prone communities, while the second team, working on stream 3, developed KNOW YOUR WATER POINT an application that gives a geo-referenced position of water points in the country. It allows communities; emergency managers and international aid organizations know the next community where there is a water source, the type, and the condition of the water source.

(The winning team of the Hack4Good Nigeria) From Left -Ben; Manga; SchoolofData Fellow -Oludotun Babayemi; Habib; Chief Executive, CODE – Hamzat

Living with coders all through the weekend, was mind blowing, and these results and outputs would not be scaled without its challenges. “Bringing our EMERGY application live as an application that cuts across several platforms such as java that allows it to work on feature phones can be time consuming and needs financial and ideology support” said Manga, leader of the first team. Perhaps, if you want to code, do endeavour to code for good!

Dispatch: School of Data Summer Camp

Heather Leson — Tue, 29 Jul 2014 18:29:51 +0000

Close your eyes and imagine an inviting space that you can connect, learn and share with new colleagues from around the world. What would that look like? Well, at School of Data we dreamed big to deliver an amazing event in collaboration with so many people. We are still reflecting but would consider it a successful first School of Data Summer Camp.

The School of Data Summer Camp was held on July 18 – 21, 2014 at Villa Adlon in Potsdam, Germany. There we collaborated to build School of Data with all the participants: partners, network leaders, local instances, fellows, senior fellows, funders, special guests and staff. The 4-day event with 47 people focused on individual and community growth with participatory activities in a programme consisting of conversations, documenting, skillshares, brainstorming, networking, and gamestorming.

Here is a small window into the spirit of Summer Camp:

(Video by Social Tic’s Juan Manual Casanueva)

Thank you

The house, lake, gardens and complete environment provided a perfect setting to unite people for School of Data Summer Camp. We would like to extend a very special thanks to our host Mathilda Huss of Villa Adlon. She generously provided the beautiful location.

Special thanks goes to our funders who made all of this possible: Partnership for Open Data (World Bank, Open Data Institute, and Open Knowledge), Hewlett Foundation, Open Society Foundations, IndigoTrust, the British Embassy Skopje, Hivos, and SEATTI.

We are all taking a much needed rest and then will begin to share more about the next steps for School of Data. Thanks again to everyone who made School of Data Summer camp so special.

How to: Choropleth Maps with D3

Heather Leson — Fri, 06 Jun 2014 19:50:11 +0000

[Guest Cross-post from Jonathon Morgan of Crisis.net. CrisisNET finds, formats and exposes crisis data in a simple, intuitive structure that’s accessible anywhere. Now developers, journalists and analysts can skip the days of tedious data processing and get to work in minutes with only a few lines of code. See the Original post]

D3 is quickly become the de facto library for browser-based data visualizations. However while it’s widely used for line graphs and bar charts, its mapping features are still fairly underutilized — particularly in relation to more established tools like CartoDB, and of course Google Maps. Those tools have their place, but when you need fine-grained control over the presentation and interactivity of your geospatial data, D3 can be a powerful alternative.

Today we’ll walk through how to create a popular visualization; the choropleth map. These are used to show the relative concentration of data points within a given region. For example this might be the number of people within a particular age range in every county in a state, or the number of reported cases of the flu in each state in a country. The information we’ll be mapping is a little more exotic. I recently collaborated with Eliot Higgins, an arms transfer analyst focused on the ongoing conflict in Syria, to retrieve data from 1,700 Facebook pages and YouTube accounts associated with militant groups and humanitarian organizations working in Syria. We ingested that data into CrisisNET, which then made it possible for us to generate a “heat map” showing which parts of Syria are experiencing the most intense fighting.

In order to do this we’ll need to:

Work with projections to transform latitude, longitude pairs to x, y browser coordinates
Render city boundaries as SVG paths using D3 drawing tools
Shade each city relative to its reported level of violence

Let’s get started.

Before we can do anything we’ll need some data. A geospatial “feature” (like a city, state, etc), is defined as a polygon, which is represented as a list of latitude/longitude pairs. For example:

[
[ 36.712428478000049, 35.83274311200006 ],
[ 36.704171874000053, 35.830347390000043 ],
…
]

Each pair is a corner of the polygon, so if you plotted them on a map and connected the dots, you would get the outline of the feature. Awesome!

Geospatial data comes in a variety of formats, like shapefiles, and KML. However the emerging standard, particularly for use in web applications, is GeoJSON. Not surprisingly, this is the format supported by D3 and the one we’ll be using.

Depending on the region you’re trying to map, GeoJSON polygons defining features in that region may be easy to find — like these GeoJSON files for all counties in the United States. On the other hand, particularly if you’re interested in the developing world, you’ll probably need to be more creative. To map cities in Syria, I tracked down a shapefile from an NGO called Humanitarian Response, and then converted that shapefile to GeoJSON using a tool called ogr2ogr. Fortunately for you, I’ve made the GeoJSON file available, so just download that and you’ll be ready to go.

Let’s Talk Projections

With our polygons in hand, we can start mapping.

Remember that latitude and longitude coordinates denote positions on the surface of the Earth, which is not flat (it is an ellipsoid). Your computer screen is a plane (which means it’s flat), so we need some way to translate the position of a point on a curved surface to its corresponding point on a flat surface. The algorithms for doing this are called “projections.” If, like me, you’ve forgotten most of your high school geometry, you’ll be pleased to learn that D3 comes included with a number of popular projections, so we won’t need to write one. Our only job is to choose the correct projection for our visualization.

The Albers and Azimuthal Equal Area projections are recommended for choropleth maps, but I found both rendered my cities in a way that didn’t connect all the points in the polygons from our shapefile, so some of the city outlines didn’t form an enclosed shape. This made it impossible to shade each city without the color overflowing into other parts of the map. Although this is probably due more to my lack of familiarity with the specifics the Albers and Azimutha projections, I found that the Conic Conformal projection worked out of the box, so that’s the one I chose.

Drawing the Map

Now that you understand the background, we can start coding. First, attach an element to the DOM that will serve as our canvas.

Next create an SVG element and append it to the map DOM node we just created. We’ll be drawing on this SVG element in just a second.

// Size of the canvas on which the map will be rendered
var width = 1000,
height = 1100,
// SVG element as a JavaScript object that we can manipulate later
svg = d3.select(“#map”).append(“svg”)
.attr(“width”, width)
.attr(“height”, height);

Despite the rather lengthy explanation, defining the projection in our application is actually fairly straightforward.

// Normally you’d look this up. This point is in the middle of Syria
var center = [38.996815, 34.802075];

// Instantiate the projection object
var projection = d3.geo.conicConformal()
.center(center)
.clipAngle(180)
// Size of the map itself, you may want to play around with this in
// relation to your canvas size
.scale(10000)
// Center the map in the middle of the canvas
.translate([width / 2, height / 2])
.precision(.1);

With a projection ready to go, we’re ready to instantiate a path. This is the path across your browser window D3 will take as it draws the edges of all our city polygons.

// Assign the projection to a path
var path = d3.geo.path().projection(projection);

Finally, let’s give some geospatial data to our path object. This data will be projected to x, y pairs, representing pixel locations on our SVG element. When D3 connects these dots, we’ll see the outlines of all the cities in Syria.

Let’s use d3’s json method to retrieve the GeoJSON file I referenced earlier.

d3.json(“cities.json”, function(err, data) {
$.each(data.features, function(i, feature) {
svg.append(“path”)
.datum(feature.geometry)
.attr(“class”, “border”);
});
});

That’s it!

Most of the heavy lifting is taken care of by D3, but in case you’re curious about what’s happening, here’s a little more detail. Our GeoJSON file contains an array of features, each of which is a polygon (which is represented as an array of longitude, latitude coordinate pairs). We pass the polygon to our path using the datum method, and the polygon is then converted by our projection to a linestring of pixel positions which is used by the browser to render a path DOM node inside our svg element. Phew.

With a working map of the country, we can now change its appearence and add interactivity just like any other DOM node. Next week we’ll use the CrisisNET API to count reports of violent incidents for each city in Syria, and shade each city on the map with CSS based on those report counts.

In the meantime you can checkout the full, working map on our Syria project page.