Open Agricultural Data:
Summary of the D8 Open Data Event at IFPRI (5/1/2013)
Last week, I attended a fabulous Open Data event with my colleague Shawn Mole at the International Food Policy Research Institute (IFPRI). This was a 2-hour "side event" on Open Data for agriculture as well as a "bolt-on" event to the G8 Open Data Conference for Agriculture at the World Bank that happened two days prior.
If you're a fan of Open Data, APIs, or agriculture, this was the place for you. The event clearly conveyed the impressive amount of work being done at the intersection of data and agriculture. At it's core, the conversation boiled down to this: how do we lower the barrier of access to this data to new communities in order to open the huge potential for new and innovative services?
For those of you unable to make it, here's my best attempt at capturing the most salient takeaways, specifically from IFPRI and the World Bank.
, IFPRI’s Head of Knowledge Management, opened the event stressing two key points:
1. Not only does this research data need to be available and open, it also needs appropriate structure in order for it to be digestible by both humans and machines.
2. By moving forward in this direction, the decision making capabilities of those in charge of agricultural development will improve greatly and, in turn, improve the well being of the people they lead.
She then handed the floor over to Neil Fantom who heads up the World Bank’s Open Data initiative and shared a fitting anecdote for the problem at hand. In 1998, Neil worked in the statistics office and met Todd Benson who ran a joint project called the 'Social Statistics of Malawi'. In order to collect data at that time, Todd had to go to various government ministries and agencies, take any data sets he could find (crop production, population density, policy mapping etc.), mash them up, and put them into publication. While this information is incredibly helpful, that sort of process is tedious and difficult to apply to a large number of locations.
World Bank Open Data Lessons Learned:
Neil gave a short but incredibly thoughtful presentation about how Open Data can greatly expedite and improve this process. He then followed with the top 10 things that the World Bank has learned as part of its Open Data initiative, which are:
1. Give them Fast and Easy Access: Before they launched Open Data, they had good, nicely-designed, but heavy and complicated tools. These tools were "good for the geeks", but they weren't so good for the general public. Always key to keep the user in mind.
2. Don't Start with the Hard Stuff: By starting with the "World Development Indicators" data set, which had good metadata and support around it, they were able to engage users with this simple, but familiar and proven information. (Over 20-30 years worth!)
3. Open Data is Unreal and Hot: 'Nuff said. I agree! Neil cited this 2010 tweet which is the intersection of unreal hotness and Open Data - something you don’t hear about too often ;)
4. Data Can Turbocharge the Fight Against Poverty: Neil noted that Bono was asked what it would take to turbocharge the fight against poverty. The first words out of his mouth? Open Data. He was interested in it from 2 angles:
1. Helping support the design of better programs and policies
2. Increasing accountability through transparency, which helps government to better engage citizens and deliver services
5. Free Data is not Free: By making the World Development Indicators data set publicly available, the World Bank lost out on $3M worth of revenue that helped to employ the team that created it. Even though investment costs have gone up, they've been working to replace the income generated by that data set through investments.
6. But it's Good for Business: Regardless of the revenue loss, the World Bank's Open Data program has been great for business. Not only has it become much easier to argue the benefits of data since the program's inception, but it's helped to drive an audience to their cause.
This chart shows the trend of weekly visits to http://data.worldbank.org since they launched in April 2010. Shooting from 100,000 visitors per week in 2010 to 400,000 visitors per week in 2013 shows the power and usefulness of the data, especially when 10,000-15,000 weekly visitors through a paid database would be considered high. Not to mention that this site accounts for 25% of traffic that comes to the World Bank website. This explosive growth of traffic reflects a very successful model of improving organizational reach with Open Data.
7(a). Be Technically Open: If you want to make your data truly open and easily usable, it's critical that you make it available in a standards-based way. Don't use proprietary or non-parsable formats.
Good = json, txt, csv, xml, html, API, rdf, etc.Bad = pdf, jpg, gif,png, & other proprietary formats.
Good = CC-BY attribution license, opendefinition.org.Bad = Subscriptions, restrictions on commercial
8. Label Your Stuff: When you're looking for the right can of soup, labels are critical. So too with data. Metadata is good. Try to make your data clear, by making it clear what's in it.
9(a). Be Open to New Engagement: This has been one of the more difficult things for the World Bank in terms of having to change their model. In the past, they would put their data out and *hope* that people would use it. Now, they're trying to be much more proactive about reaching out to new communities and the way that they use data.
9(b). Because Others Can Do it Better: This can be a tough lesson for an organization to learn, let alone recognize. The World Bank spent a lot of money on their own search engine, but it's nothing like Google. Not surprisingly, Google does search better than the World Bank. But if you do a Google search for "external debt Tanzania", you'll get a link that comes from the World Bank database. Why? Because the World Bank has made their data available in an open way. Neil also noted that Google has translated that same data set into Bahasa, because they're a multi-national company.
10. Open Data is a Public Good for the Public Good: Who is the World Bank's open data for? People. In the end, it’s all about reaching people like this:
Following Neil’s Top 10 list, the other panelists provided a good summary of their various open data projects, to which the links of each recording done by IFPRI can be found here:
- Opening remarks
- Neil Fantom of World Bank
- Johannes Keizer of the Food Agricultural Organization (FAO)
- What the FAO has done with AGRIS in the last 2 years
- Daniel Mason-D'Croz, IFPRI
- International Model for Policy Analysis of Agricultural Commodities and Trade (IMPACT)
- Kathleen Flaherty, IFPRI
- Agricultural Science and Technology Indicators (ASTI)
- Perrihan Al-Riffai, IFPRI
- Opening Spatial Data in the Middle East
- Indira Yerramarredy, IFPRI
- IFPRI's Open Dataset Catalog at Dataverse
- Soonho Kim, IFPRI
- Open Data Sets for Markets & Trade Analysis
- Melanie Bacou, IFPRI
- Spatially-disaggregated Indicators - Harvest Choice
From working with IFPRI, we know their commitment to improving the connectivity between content, data, and users and we applaud their sponsorship of this D8 event. Phase2 shares this commitment. One recent example is a project we began with a technology partner Ontotext to integrate enterprise linked open data stores (i.e., RDF triple stores) with Drupal. More to follow on that, in a future blog post. And to learn more about our efforts in Open Data, follow us @phase2!