Mixing and Remixing Information

See Mixing and Remixing Information » s09-Day 10 Ajax/JavaScript Widgets and Their APIs

Remember — project proposals are due tonight.

http://blog.mixingandremixing.info/s09/day09/:

Agenda for Today

1. Welcome to our visitors (12:40pm:  10 minutes):  Finnish graduate students, here in the Bay Area  as part of a one-week field trip for a graduate course “Bit Bang: Rays to the Future”, organized and led by Professor Yrjö Neuvo—Nokia´s former Chief Technology Officer (CTO).  The objective of this year-long course is to study the broad long-term impact of information and communications technologies on lifestyles, society and business. The students come from Helsinki University of Technology, Helsinki School of Economics (HSE) and University of Art and Design Helsinki (UIAH) and have various backgrounds including engineering, arts and design, social sciences, and information technology.
2. Some background on the project (12:50pm: 10 minutes)
3. Lightning presentations of project ideas from InfoRemix2009 students (1:00pm: 10 min)
4. Small group thinking about project proposal (1:10pm — 15min)
5. Large group presentations of projects (1:25pm: 30 min) — 3 minutes each
6. Conclusions (1:55pm — 5 min)

Background

In today’s class, we will  be discussing their project
proposal.   (See  http://blog.mixingandremixing.info/2009/02/20/day-8-notes-and-day-9-mon-feb-23-preparations/).

The project proposals students need to submit are outlined here:

http://blog.mixingandremixing.info/s09/project-proposal-format/

Here’s a note I sent to my students to help them model their proposal
thinking/writing:

http://blog.mixingandremixing.info/2009/02/18/project-brainstorming-a-rough-take/

Project proposals dues on Wednesday, Feb 25 at 11pm.

Readings for Next Class (Day 10):  Chapter 8 on Ajax and JavaScript Widgets.

For those of you who missed class last Wednesday, Feb 18, you missed a great talk by Bernt Wahl on the business case behind mashups.

On Monday, Feb 23, we will be discussing your project proposals.  The proposals were originally due on Monday, but I’ve decided to extend the due date to Wednesday, Feb 25 at 11pm to give you some time to improve your proposal.   To facilitate discussion, please bring 25 paper copies of your draft proposals to circulate in class.  We will be hosting some graduate student visitors from Finland, who will be very interesting in learning about your ideas.

This weekend, I will respond to any draft ideas that you have sent me.  I encourage you all to contact me to arrange a time to talk to me.

To help my students develop their projects, I’d like to model the project development process by writing about two projects that I’m pursuing over the next two months:

• entering an application for the Sunlight Foundation’s Apps For America competition
• writing a grant proposal on the application of semantic web technologies to art history for the NEH Digital Humanities start-up grant competition

In this post, I will tackle the first topic only.

I will write some of my background thinking leading to more formal statements of my project proposals, produced in a format I outline in Mixing and Remixing Information » Project Proposal Format.

Government Transparency / Apps For America

A place to start is of course to read the rules of the Apps for America competition. To enter, you need to produce a working application to submit by March 31.

Note the criteria for apps for America [emphasis mine]:

1. Usefulness to constituents for watching over and communicating with their members of Congress
2. Potential impact of ethical standards on Congress
3. Originality of the application
4. Potential usability of the application
5. Code quality of application

To get a feel for how to do to stay in the competition, you should look at the APIs, scan them for their capacities, and  also start brainstorming ideas:

• Sunlight Labs API
• OpenSecrets.org API
• FollowTheMoney.org API
• Capitol Words API
• Sunlight Labs API Lawmaker Dataset
• Sunlight Labs API Lobbyist Dataset
• Sunlight Partytime Database Dump

As you do so, reflect on their own interests questions and passions.  As I thought about my own interests in this area, the following came to my mind:

• I would like to influence government (to the small or large degree that I can)  to go in a “good direction”. There are things I care about to varying degrees.  My friends’ interests affect my own.  For some issues I can and would like to put in a lot of work.  For other areas,  I might just monitor the issues.  For yet other topics, I would want to take intermediate levels of action like write my congressional representative, and notify others about what I know etc.
• I want to help people learn and do research; I have an educational and teaching and journalistic angle.  I think of serving intermediaries, building tools to help people collaborate, learn, teach, etc.
• I’ve been recently focused on learning about senators and was astounded about how little I know about such a powerful and relatively small group of people.  Hence I have in mind developing a tool to help people to get to know the players and the ecosystem of Congress.  Do people really know their representatives? Would it help both the politicians and the people you somehow they knew each other better?
• I like to make a more regular habit of writing my government officials.  I know of many issue-specific websites that get you fired up about this is an issue and then urge you to write your congressperson and provide you boilerplate letters.  How can that process be better integrated into my own workflow and that of my friends and ultimately of all concerned people and organizations?
• One learning goal for myself:  learn how to use Freebase and explore semantic web technologies and approaches. I’d also like to learn Django and one of the JavaScript libraries plus some good database skills.  I know the theory pretty well, but I’ve been longing to build a real world application.
• My educational focus and focus on tools for the intelligent layperson and for those who mediate between specialists and laypeople (educators, journalists).
• My bias:  some visualizations are good first applications of an API — but I (personally) don’t find that most of them yield that much insight  (Since some do, I need to be more precise about what I really trying to say here) …I want to build tools to let people find answers and then let them incorporate those answers in an interpretive narrative.  I also lean towards stuff that connects insight to action.  Tools that let people network with friends and lobby congress, write their newspapers, etc. (tradition models of activism….the one that seems to be implicit in lots of issue-oriented websites (e.g., EFF Take Action, Save Darfur’s model for lobbying Congress)

While brainstorming ideas, I looked for some inspiration at Project Ideas – Sunlabs wiki and the list of Big Issues

Some brainstorms:

• a website (or desktop application) thing that makes it easier and more fun to communicate with your congressional representatives and representatives key to your causes….[I've heard it said that we often underestimate the impact of some well-placed communication with congressional reps....]
• alternative dashboards for tracking bills and members of congress.  I’ve tried  Govtrack.us  but I’ve not been able to integrate it well into my own workflow.  No excuse, I suppose, since govtrack.us has an API.
• tools to let people more easily crowdsource the quick reading and analysis of bills, both of fast-moving ones and slow moving ones. I didn’t think that attempts like that of the Huffington Post to solicit comments from readers on the stimulus bill is the way to go…but what is?  I was wondering whether anyone attempted to use Amazon’s Mechanical Turk to partition out pieces and then aggregate the results in a hierarchical editorial process, like the one  used to look for Jim Gray (though, alas unsuccessfully).
• an educational application to learn more about your senators and congressional representatives, the committees.  Maybe a quiz game?  I found that I had never heard the name of most senators until I started doing computation on their names.  Is it useful to know their names and what they do?  For whom?  And if so, how to make a game out of it?
• how to implement some of the ideas in  Blaser, Britt, David Weinberger, and Joe Trippi. “Digital Government through Social Networks: A Natural Alliance?.” Puebla, Mexico. http://dotorgware.com/documents/DGSNA-SocialNetworks435VCDs1.pdf.
• any way to improve on TARP tracking or the stimulus spending documented at http://recovery.gov — or elsewhere?  (I have to say that this is a very fast moving area, one that is seems to be too fast moving for me because I can’t devote full time on this topic.)
• Data-mining the flow of money, the voting records of congress (which I don’t think is in the Sunlight Fdn Apis–but might be in govtrack.us apis), and other data about the congress to identify “statistical anomalies” that might indicate some sign of untoward behavior.   My guess is that you would need some serious statistical knowledge +   some theoretical and empirical knowledge of how political influence really works,  which is beyond what I can do unless I line up some expert help…. (Being a Numb3rs fan, I was wondering whether one could use math used to model how jurors influence one another to members of Congress.

What I’m settling on for the purposes of writing this model, knowing that I might change if there’s a good reason to….but also keeping in mind the March 31 deadline.

A Specific Project:  Using Freebase to Prototype/Build a PolDB

Though I’m certain yet that this is the project I plan to submit, I’m leaning towards building a prototype for PolDB, which is described on the Project pages as:

Build an open IMDB-like service for All 300,000+ elected officials in the country and build biographies on all of them much like the style of IMDB. Allow for community participation, submission and vetting.

Accomplishing this task is beyond what I can do by March 31 — however, I will explore how much progress one can make towards building PolDB by leveraging Freebase, the Wikipedia, and other sources of data, including the Sunlight Labs API.

Let me now answer the questions (in point form) listed in the project proposal.  What I’d like in the actual proposals is a more integrated narrative.

What problem is your project aimed at solving? Alternatively, why would someone want to use what you are making in your project?

According to Metaweb Technologies, Inc. : About:

Freebase, created by Metaweb Technologies, is an open database of the world’s information. It’s built by the community and for the community – free for anyone to query, contribute to, build applications on top of, or integrate into their websites.

My project aims to seed a PolDB using freebase.org as the database and possibly application hoster (using the Acre platform). One model I have in mind of an existing application built on top of Freebase APIs is bioventurist.com (and its companion Freebase base http://bioventurist.freebase.com/) (You can read more about it on the Freebase developers’ list.

Is your project doable given the constraints of time, our starting knowledge, etc?

Yes, if I don’t aim to build the entire PolDB project.  I plan to start by making sure that congressional data already available in APIs such as the Sunlight Labs API gets into Freebase properly.  I will see to what degree the native Freebase interface for a base will accomodate editing and vetting by the community.

What interface are you imagining? Is it a web, desktop, or mobile application? What platform are you running? Often a rough sketch of the interface can help clarify a lot of issues

I’m imagining a web application such as bioventurist.com using Freebase as a back-end.  I will also prototype a data extractor so that the data for PolDB will not be tied to the Freebase indefinitely

What data or services are planning to bring together? Be specific.

• Freebase
• Sunlight APIs
• NYT Congress API
• govtrack.us APIs — to look at name reconciliation of congress folks.

What’s your plan for getting the data. Often the data you might want might not be easily available.

Beyond the APIs for congress…..that becomes trickier [need to think through this]

Do the APIs you plan to use actually support the functionality that you need in your application? Show how it does so.
What programming language do you plan to use?

Action Plan

Break down the project into steps. You can end up changing the steps later, but I want to make sure you have a clear conception on what the steps are.

• learn Freebase 101 — the UI, metadata schemes, APIs
• implement a small base completely — e.g, US Senators first — a sample that is small enough that I could conceivably enter most of it by hand.
• evaluate what I’ve done to figure out how to scale up.

[More details would be good....]

Highlight what you are currently working on.

Right now, I’m learning Freebase.  On Saturday, Feb 28 I will attend an event at Freebase headquarters on how to build a base.

Identify areas of “high risk,” areas that you are uncertain about and/or things that might undermine the entire project. Write about how you are planning to deal with these potential problem areas.

Some risks:

• how are we to keep the freebase system up to date after I am no longer working on it?  Will it be worse to have a pile of old data?
• Are there barriers in the APIs that prevent me from taking data there and pushing them into Freebase?
• Will the issues be so complicated at the federal level that I won’t be able to get to the state and local levels?
• Is it too risky to use Freebase?  What kind of lock-in?

[I have to write about what I plan to do....]

If you are unclear about what you might want to do, I’d suggest starting with Flickr-centric ideas if you don’t have any other ideas in mind.

I’ve brought up the topic of trying to track TARP (Troubled Assets Relief Program). What can we do to improve our oversight of TARP?  Can we build off of:

• Can we build better presentations/visualizations of Tracking the $700 Billion Bailout – The New York Times?  That is, is the NYT just a simple visualization of the original data from the feds?
• What tools would help journalists/bloggers do investigations such as TPMMuckraker | Talking Points Memo | Who’s Running TARP? You Might Not Wanna Know?
• How can we work with the government data librarians to do the best work possible?

Keeping in mind that the project proposals are due on Feb 23. (See http://blog.mixingandremixing.info/s09/project-proposal-format/) and that tomorrow’s class (Feb 11 Day 7 is devoted to an introduction to enterprise mashups with a guest speaker Justo Hidalgo of Denodo Technologies), that next Monday is President’s Day, and that Wednesday Feb 18, we have Bernt Wahl to talk to us about business perspectives around mashups — we won’t have much class time before your project proposals are due.  Consquently, I encourage you to

• talk to me outside of class.  I’m willing to meet with you during office hours (MW 2-3pm) as well as other times.
• email me
• most importantly, I think, engage your fellow classmates and me in the electronic forums we have available to us.

Let me elaborate on that last point.  We have a number of places to communicate available:

• the bSpace forums
• the bSpace chatroom
• the infomix mailing list

I especially commend to you the forums — they provide a place for structured conversations.   To support your project proposal writing, I’ve added an assignment to update me by Feb 16 at 5pm on your progress towards writing a proposal.

Day 6

For those of you who missed the Google Geo-APIs workshop yesterday, take a look at the links to the tutorials provided at the workshop:

• http://tinyurl.com/sh09geo
• http://tinyurl.com/maps-api-codelab
• http://tinyurl.com/earthapi-codelab

Today’s class is the Mixing and Remixing Information » s09-Day 06 Google Geo-APIs workshop

See Mixing and Remixing Information » s09-Day 05 Intro to APIs via Flickr API

Here are some exercise to help one get use to programming on the ischool clusters.

Exercise

get warmed up on ischool.berkeley.edu

• login with your ischool account — you all have a ~/public_html directory — e.g., /home/rdhyee/public_html

• add test.php to your directory

~/public_html
<?php
echo “hello — how are you? This year is 2009″;
?>

See
http://people.ischool.berkeley.edu/~rdhyee/test.php

• download source code from Apress

in ~/public_html type

mkdir Book
cd Book

curl -o Source.zip http://www.apress.com/resource/bookfile/3901
unzip Source.zip
get the permissions right: chmod -R o+rX Ch* R*

get flickrsearch.php working in Chapter 6

http://people.ischool.berkeley.edu/~rdhyee/Book/Chapter06/flickrsearch.php

setfacl -m u:apache:r–,o:—,m:rwx flickrsearch.php

e.g., http://people.ischool.berkeley.edu/~rdhyee/Book/Chapter06/flickrsearch.php

See

• s09-Day 03 Working with Feeds, RSS, and Atom
• s09-Day 04 Creating Mashup Projects from APIs

See also the Project Proposal Format