World Bank Open API 2.0 Launched
API - What?
Fact: World Bank has been collecting massive amounts of data, for the past 50+ years, and now possesses one of the richest repository of information about economic development in the world.
World Bank Open API is an initiative of the World Bank that opens the wealth of the World Bank's global economic data to the outside world, in a standard, easily accessible way. Open API allows third parties to develop mash-ups and applications with the World Bank data and easily create different kinds of interesting visualizations and insightful reports.
A pilot version of the Open API system was released last year. Now the World Bank is releasing version 2.0 of the API that addresses many structural, technical and interface improvements. We were honored to be the architects and developers of this version.
Divide et Impera
One of the first things you notice when you look into World Bank data is the large volume. Currently, the API exposes a variety of data for approximately 200 countries for more than 50 years. Looking at the volume of data and hearing the feedback of API 1.0's users, it became obvious that being able to easily browse/explore data was as important as the interface to access a specific piece of information.
In addition to the large volume, the Bank had plans to expose more data through the same interface, so the architecture had to be scalable. To achieve the level of browseability and scalability required, we decided to dissect data into an extensible collection of well-defined entities, exposed through the API as query "end-points" in a REST architecture.
Endpoints currently exposed by the API are: Countries, Photos, LendingTypes, IncomeLevels, Indicators[$type], Authors, Topics, Keywords, Sources. A simple API call like: http://open.worldbank.org/v2/countries?api_key=XXXX returns the list of countries in the system, with some vital details about each country like: the name, iso2 code, region, income level, capital city, the geo data of the location of the capital etc. To get specifics about a particular country, you can use the iso2 code captured from the country listing to construct your next query: http://open.worldbank.org/v2/countries/ge?api_key=XXXX gives data regarding a particular country.
Getting Started
To get started with the World Bank API, you need to obtain a developer key from the API website: http://developer.worldbank.org. After free registration, you will be granted an API KEY that you need to add to all your calls as a query argument "api_key".
Aside from the list of the countries, another easy and interesting call explores the economic indicators in the system:
http://open.worldbank.org/v2/indicators?api_key=XXXX
which gives a list of all indicators in the system, a short description, and the unique ID and the source of the indicator values. Source usually indicates which section or research group of World Bank has prepared the data. For instance, the enormously popular Doing Business Report (http://doingbusiness.org/) is indeed one of the sources exposed by the WB OpenAPI.
Again, you can use the ID to limit your query to the specific indicator type:
http://open.worldbank.org/v2/indicators/IT.TEL.TOTL.P3?api_key=XXXX
Since the API is now implemented as a well structured RESTful architecture, you can chain entities and filter an end-point by some other entity. For instance, you can easily look-up a list of indicators available for a specific country:
http://open.worldbank.org/v2/countries/ge/indicators?api_key=XXXX
And, checking an indicator value across multiple years for a specific country is equally easy:
http://open.worldbank.org/v2/countries/ge/indicators/NY.GDP.MKTP.CD?api_key=XXXX
You can limit the result-set to a range of years:
http://open.worldbank.org/v2/countries/ge/indicators/NY.GDP.MKTP.CD?date=2002:2004&api_key=XXXX
(Note: to understand the unit of measurement of an indicator type, refer to the description of the indicator type provided by indicator queries).
What's Next?
There are many other types of useful queries you can use to explore World Bank data. Our goal here is not to re-create the official documentation, but to give you a sense of the powerful lookups you can issue, with just a web-browser and some URL tweaking, and no specialized tools. What you can do with actual code and integration with other tools is probably only limited by imagination.
We have no doubt that soon after the new API is released, different companies and individuals will come up with numerous creative ways to use the Bank's rich data in different mash-ups and visualizations and we will see many cool widgets and sites driven by the API.
We are very excited to be part of one more big step toward open data sharing and integration at the World Bank and would like to express our gratitude to the visionaries at the World Bank who are making their rich data freely accessible on the Web to the public.
Comments
wow!
that's great news. I've used the WDI database many times for my economics education but I also find it to be a great resource for business research. Looking forward to see some cool web 2.0 interfaces on this massive database.
API != "open data sharing"
Using any of the resulting cool Web 2.0 interfaces for business research would be very brave, since going by a conservative reading of the World Bank's API Terms of Use, any use of the data beside non-commercially serving it straight to your web users (with attribution) is forbidden:
"[...] You may not retain any copies of the World Bank Data. Your license is limited to making direct server calls to http://open.worldbank.org for purposes of distributing World Bank Data to your end users on your Web site(s) and/or Application(s) immediately upon receipt by your servers."
"Freely accessible", but not "freely usable", which rather defeats the nominal purpose of having an API. In fact any significant use of this data could be considered copyright infringement. Sure, you can pass on some statistics to your users, but as soon as one of them does so much a post a number from those statistics to their blog, they're criminals in the eyes of the World Bank. You can (and should) argue that facts about the world are not subject to copyright, but unless you've got a lot of money, you're not going to be able to out-argue the World Bank if it comes to court.
Until the World Bank surrenders all copyright claims to the data, or at least licenses the data under reasonable terms this data should be considered legally tainted and unusable. The API is not an invitation to use the World Bank's "open" data, it's an invitation to stick an ad on your website for data which remains as proprietary as if it were locked in a vault somewhere, and leave yourself vulnerable to future legal action. The World Bank can wait a couple of years for people to build non-trivial applications, then start sending the threatening letters demanding payment - you sent them your address with your API key registration after all.
I'm not singling out the World Bank here; it's a general rule. Read the fine print before using an API. It's likely that what the API enables and what the Terms of Use permits are two entirely different things.
This is just a first step
Mathew,
thank you for your thoughtful feedback.
I am not a World Bank employee, neither do I represent the World Bank. We will pass on your opinion to them, but meantime, I would like to chime my two cents in, if you don't mind.
Personally, I generally would also love to see all useful data/knowledge to be available and free - free as in "freedom" (not beer). This said, there're some serious considerations in play here that need mentioning:
Thank you
Great response regarding Terms of Service
Your points with regards to the terms of service are well stated.
Depending upon what the core business model of the company or non-profit might be, the API terms of service needs to reflect that value. Netflix lets you use their data for non commercial and commercial, as long as you don't directly compete. Their business is video rental, not the catalog data. Companies like the NYTimes don't allow for commercial use, as their business is the stories that you can access from the APIs. The World Bank wouldn't be any different - if you want to use the content for commercial purposes, then you would pay for commercial use.
I am on the World Bank team
I am on the World Bank team that worked to make the API available, and I wanted to make a couple points. First, as Irakli said, this is a work in progress. As background, we made the first version of the API available several months ago and heard from developers that they wanted a more URL-based call structure, so we listened to that and were able to deliver a new version. So it's good to hear what people's concerns are and that feedback can help us as we move forward.
Regarding some of the specific points raised by Matthew, some to be honest would be difficult to change. For example, the requirement to not store data locally was important in ensuring that any Bank data would be accurate and up to date, and I can't see that changing.
Others points we have thought about ourselves, and nothing is set in stone. For example, and this might happen outside the API, we are interested in exploring different licensing options for some of our content, such as the Creative Commons license that Matthew linked to. However, again as Irakli said, these are changes that can't be made quickly, at least in an institution the size of the World Bank. They require time, legal review, etc. So I see this version of the API as a first step. We're interested in making the API to the available and seeing how people use it.
World Bank API & Linked Data
Are you interested in exposing the data behind the API in Linked Data form. This will enable the following:
1. Richer output from your API 2. Alternative modes of interaction
Links:
1. http://linkeddata.org
Kingsley Idehen http://twitter.com/kidehen
Yes, this is something we'd
Yes, this is something we'd definitely be interested in exploring. I'll take a look at the links you've provided.
Jeff, World Bank
1000 Calls / Day?
The legal terms do give me pause, but I'm not surprised. Seems like standard fare for an organization where strategy/technology and legal sit in different offices.
What I am surprised by is the speed limit. The API is limited to 1000 calls a day. What, besides a cute toy for one user, can you do with 1000 calls a day? Any interesting application is going to issue many tens of calls per user interaction.
Even if you want to call for more info than you need, and display it piecemeal as the user browses - isn't that starting to get grey on the legalese?
How can anyone build even a moderate prototype on 1000 calls a day?
1000 Calls / Day
I totally agree with the issue of the number of API calls. 1000 calls per day is not even close to being sufficient. It would be fairly easy even for a single developer by themselves to exceed this limit in a typical day of development while creating the mashup. That's going to be very frustrating and is not going to endear yourself to developers. Once the mashup is publicly available, it *will* go down in short order, even if only one or two sites link to it. To put this in perspective, 100,000 calls is considered small by some, but really why would you put a limit in place at all? It's only going to limit API adoption. Worst case, do what Google does and throttle the requests. They put a limit on the free maps API access so that every ten seconds or so, if more than the ten second threshold has been exceeded, the excess is dropped until the next ten seconds. That way bursts are controlled, but access is maintained over the course of the day. Then again, really the only issue you should be worried about is sustained high traffic which may indicate commercial or improper use. It would be better just to monitor the API keys looking for that kind of activity and warning the account holders (and closing accounts) as appropriate.
API Calls - World Bank perspective
It's Jeff from the World Bank again. We picked the original limit of 1000 calls/per day based on a comparison of similar services - just checking Mashery customers right now, services like Compete, Zoominfo, and White pages are between 1000 and 2000 calls per day per key. Larger players like Calais have much higher limits, in the tens of thousands, but starting out, we didn't ourselves in that league. We have now upped the calls to 5000 per day per key. We'd love to see what kind of use we're getting, and we're ready to continue upping the limit if needed. Thanks very much for oyur interest and we appreciate the feedback.
Jeff
Channel for Proposals?
Jeff,
if I may suggest, maybe a channel can be created on the API website (could be something as simple as a contact form or even a special e-mail address) where developers can quickly submit the links to working prototypes, they have built with the API, and requests for higher limit. Then the Bank could grant higher limits for projects that really need it?
Thanks for the quick response.
It's nice to see that there's someone on the other end of the line for this one.
Support?
I didn't find an address for support, so I'm busy filing my bug reports on the documentation pages. E.g. http://developer.worldbank.org/docs/Documentation_Version2/Country_Queries
Post new comment