16 Jan 2010 08:26 am

Today is the first day of “Mixing and Remixing Information 2010:  Open Government and the Web” (MRI 2010) — welcome!

(The instructor is Raymond Yee (yee@berkeley.edu, twitter: @rdhyeeBio sketch)


  1. General welcome
  2. Conceptual overview of the course:  APIs, mashups, and open government
  3. Hands-on activity
  4. Walk-through of the course syllabus & schedule
  5. Assignments for Day 2
  6. Closing Q&A

Conceptual overview of the course

The purpose of this section is to Introduce people to three main topics of the course:  APIs, mashups, and open government.

Basic terms

  • APIs are “application programming interfaces”
  • A mashup, in the words of the Wikipedia, is a web site or web application “that seamlessly combines content from more than one source into an integrated experience.”

How to Understand Mashups

You can understand a specific mashup by answering a number of basic questions:

  • What is being combined?
  • Why are these elements being combined?
  • Where is the remixing or recombination happening?
  • How are various elements being combined (that is, first in the interface but also behind the scenes in the technical machinery)?
  • How can the mashup be extended?

One pattern you will see repeated among mashups that link two web sites is the combination of three actions:

  1. Data is extracted from a source web site.
  2. This data is translated into a form meaningful to the destination web site.
  3. The repackaged data is sent to the destination site.

Example Mashups

How APIs are being used today

  • Google:  The Maps API is responsible for the proliferation of Google’s maps to tens of thousands of websites.  With the popularity of the Maps API, Google began offering APIs to many more of its offerings, including YouTube, Google Search, Documents, AdSense, Calendar, and App Engine.
  • Facebook:  Facebook Apps, driven by the website’s API, enhances Facebook’s standing as one of the central online social networking platforms.
  • Amazon:  The online retailer, which started its API offering with a product advertising API that mirrors its retail store, has expanded its API to encompass its utility computing platform
  • Yahoo!:  APIs, which are central to the Yahoo! Open Strategy (YOS), to “deliver open, industry-leading platforms that attract the most publishers and developers“, provide access to such services as Flickr, Yahoo! Search, Maps, Local listings.  Yahoo! also created Pipes and YQL to let developers program more easily with APIs.
  • Microsoft:  Not to be outdone by Google, Microsoft is building Windows Azure, a large set of APIs to services hosted on its servers.
  • Salesforce parlayed its customer relationship management system into a major business platform via its APIs.
  • The New York Times provides an API to give access to millions of its articles dating back about three decades.
  • Etsy, a site for artists to sell handmade goods, released an API to officially bless work by third party developers who had been driving useful traffic to Etsy.
  • Twitter has built an empire of third-party services that uses its API; the traffic to the API is an order of magnitude greater than that to its own regular website.
  • OCLC, a major library consortium, provides access to the records of its WorldCat catalog via an API.
  • WordPress: Every installation of the popular open source weblogging software, ships with a built-in API, resulting in thousands of websites with APIs that can be used right away.

Sample API calls


Let’s geocode 700 14th St, Denver, CO 80202, using geocoder.us (HTML) or geocoder.us (REST), whose output is

<?xml version=”1.0″?>
<rdf:RDF xmlns:dc=”http://purl.org/dc/elements/1.1/”
<geo:Point rdf:nodeID=”aid52644862″>
<dc:description>700 14th St, Denver CO 80202</dc:description>

Library of Congress SRU

MODS record for Milosz’s Collected Poems (See SRU is Simple – SRU: Search/Retrieval via URL, Standards, Library of Congress and ModsFromLibraryOfCongressQuery)

Open Government

This is the fifth time I’m teaching MRI.  This year, I’m introducing a thematic element to the course to bring some unity  in the choice of content area and problems to address.  Open government is a fascinating field, a good match for the methodologies of information remix.   (Let’s use a definition from the Wikipedia as a starting point for open government:

Open government is the political doctrine which holds that the business of government and state administration should be opened at all levels to effective public scrutiny and oversight. In its broadest construction it opposes reason of state and national security considerations, which have tended to legitimize extensive state secrecy. The origins of open government arguments can be dated to the time of the European Enlightenment: to debates about the proper construction of a then nascent civil society.

How did I get into open government and the Stimulus in particular?

Sunlight Foundation

The Sunlight Foundation is a major player in the area of US government transparency.  It is doing great work, which is worth following and understanding.

Sunlight APIs

Obama and Open Government

A lot of excitement in this field in the US because of the Obama administration’s pledges of open government.

See also:

Local efforts emerging — not all federal

BART feeds: BART – For Developers and BART – Developer App Center

How about Pittsburgh (my other city)? I see Connecting Citizens, Government and Political Professionals – MyGov365.com and Sunlight Labs: Projects – OpenGov.

Citycamp (Jan 22-23, 2010): BarCamp / CityCamp

Think about crime data, transit data, etc. See City-Go-Round – Bus schedules, train schedules, trip planners, transit maps, and public transit apps.

Other backgound links

Stimulus Notes

I have another page for on the Stimulus that I compiled for a Pittsburgh Hackathon

Non-Stimulus Projects

We can brainstorm project ideas but also look at the list of ideas at unlight wiki list: Project Ideas – Sunlabs wiki

Supplementary References for my talk

For a highly readable article on how companies get into creating APIs to enable mashups, read How to Manage Volunteer Software Developers, Managing Technology Article – Inc. Article — it’s the story of how cutting-edge usage of Etsy prompted Etsy to hire Mashery to build an API.

Recently, I attended a Workshop on Application Programming Interfaces for the Digital Humanities (Oct 2009)

You might enjoy a very accessible CBC segment on mashups at Online activism, lurking, mashups, and APIs (Episode 76 – May 6 & 9, 2009 | Spark | CBC Radio)

Hands-on activity

Get into groups of 2-3 and look through the entrants and winners of the Sunlight Foundation Apps for America and Apps for America 2.  Look also at  DataSF.  Discuss what you think are good and useful and what is missing from what you saw.  Brainstorm 3 apps that you think would be useful to be  created.

Course Syllabus and Schedule


Class Projects

MRI 2010 is  a project-driven course; students will be proposing and creating projects related to open government.  It’s useful to look at what students in previous installments of MRI have created:


1. Write me an email (yee@berkeley.edu) telling me the following:

1) What you hope to learn and accomplish in the course?

2) What programming and other technical background do you have? (e.g., programming languages you know, what you have programmed, how much experience do you have with web programming)

3) What concerns you might have?

4)  Create a flickr and twitter account and send me the userID for your accounts.

I’ll have more of the computer infrastructure set up when we meet next on Monday.


Read the Intro, Chaps 1, 2, and 3 and be prepared to discuss them on Monday, Jan 25, 2010.

Don’t hesitate to contact me!

Trackback this Post | Feed on comments to this Post

Leave a Reply

You must be logged in to post a comment.