Spring 2010

Project Proposals and Spring 201023 Feb 2010 10:40 pm

Hi all, I am Julian and this is my project proposal.

What problem is your project aimed at solving? Alternatively, why would someone want to use what you are making in your project?

I am thinking about extracting real-time air quality data from the California Air Resources Board and create an air quality map along with real-time traffic data (meshed with Gmap). Combined with various meteorological data such as wind speed, temperature, and precipitation, one could look at the various impacts traffic has on the air quality.

What interface are you imagining? Is it a web, desktop, or mobile application? What platform are you running? Often a rough sketch of the interface can help clarify a lot of issues

The interface would be google map. Since a lot of computation is involved, I would expect the platform to be strictly desktop and web.

What data or services are planning to bring together? Be specific.

I am planning to bring together the data from California Air Resources Board and the Mobile Millennium Project. The Mobile Millennium project “will design, test and implement a state-of-the-art system to collect traffic data from GPS-equipped mobile phones and estimate traffic conditions in real-time. It is a partnership between government, academia, and industry.” [http://traffic.berkeley.edu/theproject.html]

What’s your plan for getting the data. Often the data you might want might not be easily available.

I am already in contact with the Professor responsible for this project in my department and we are sorting things out.

Do the APIs you plan to use actually support the functionality that you need in your application? Show how it does so.

Google map will be able to plot the traffic data as color coded paths. This would show the level of congestion on the road. Air quality data could also be easily shown on the map by pinpointing the sampling station and color coding the pin.

What programming language do you plan to use?


Provide an action plan:

Break down the project into steps. You can end up changing the steps later, but I want to make sure you have a clear conception on what the steps are.

1. Obtain traffic data in a workable format.
2. Analyze and devise a scheme to automate extraction of air quality data.
3. Plot the traffic and air quality data on the same map (trial, using a particular set of data).
4. Plot the traffic and air quality data on the same map, using different sets of data.
5. Create a user interface where date and time could be selected for a specific set of data.
6. Animation of changes in traffic and air quality data over time.
7. Integrate the system for future data extraction.

Highlight what you are currently working on.

Currently I am discussing with the Professor on how to getting the traffic data in a workable format. The data is there, it is just difficult to have it in a google-map-friendly format.

Identify areas of “high risk,” areas that you are uncertain about and/or things that might undermine the entire project. Write about how you are planning to deal with these potential problem areas.

One of the high risk areas is that the traffic data might not even be available in a workable format for me to integrate into the map. In that case, I would either have to look for alternate traffic data or use a different approach in showcasing the air quality data, such as comparing the changes in air quality with the changes in Federal and State air quality standards over the years.


Writing a javascript to extract csv data files from the California Air Resources Board and put them into a database.

Spring 201023 Feb 2010 09:43 pm

This is more or less the proposal I sent to the class list last week, but it takes into account a few of the things we’ve talked about since then.

What problem is your project aimed at solving? Alternatively, why would someone want to use what you are making in your project?

My project is aimed at solving one of the fundamental problems of street parking in San Francisco neighborhoods: knowing when and where to move your car. Most blocks in neighborhoods with residential permit parking have a two-hour window each week where all cars must be moved off the street for cleaning. The problem arises because the street layout suggests nothing about the pattern of days (Monday cleaning vs. Tuesday vs. Friday) nor the times (8-10 a.m. vs. 6-8 a.m. vs. 9-11 a.m.) when the streets have cleaning.

This is partly about tickets — the cost of even one permit parking violation nearly equals the cost of the street permit for the whole year — but it’s also about efficient use of time. There’s a good amount of strategy involved in parking on city streets: In general, you want to find the space that will allow you the maximum number of days before moving again (i.e. if you’re moving your car on a Thursday morning, a Wednesday space is a gem, while a Friday is just an annoyance). But there are exceptions: Say you know you have to get up early to drive somewhere on Monday morning, so you’ll purposely seek out a Monday space so you don’t have to wake up early twice in the same week. Knowing where, say, all of the Tuesday parking blocks are would help me figure out my parking plan before I even leave the house.

Additionally, I imagine a crowdsourcing element of the project. In many San Francisco neighborhoods, this is a time of heavy construction (partly due to funding from ARRA!) and streets that are normally open for parking have been occasionally blocked for weeks if not months at a time. I imagine pulling in Twitter content (either using geolocation information on tweets or specific hashtags relating to the different parking zones in the city, i.e. #SFparkingS, #SFparkingZ) where residents could share information about which blocks are open and which are closed.

Is your project doable given the constraints of time, our starting knowledge, etc?

After discussing this project in class, I do believe it’s doable, though I’ll probably attack it first on a smaller scale, perhaps just using one of the city-provided KML files to attempt to create this kind of map for a single parking zone at first.

What interface are you imagining? Is it a web, desktop, or mobile
application? What platform are you running?

In the best-case scenario, this would be a website/mobile app combo. However, I believe a lot could be accomplished by an interactive map on the web to start with, especially if any of that information could also be accessed via a web browser on a smartphone. Having it be web-only solves some of the problems I’m trying to fix (I could pull up the list of all the nearest Thursday parking spaces before leaving the house, for example), but it introduces new ones (needing to write down those streets/blocks to have them handy in the car). Still, it’s better than what I have now!

What data or services are you planning to bring together? What’s your plan for getting the data?

The sfdata.org site offers several datasets (mostly KML files) that will likely be of use to me, and I’ve already downloaded most of them. One set offers maps of the street sweeping routes in the city and the days and times each block
(or, more specifically, each side of each block) has cleaning, though at this point I’m not sure how that data can actually be used. Another set lists all the temporary permits that could block off parking spaces, which may be a little tougher to integrate (not all blocked parking would necessarily be residential parking, so I’d need some pretty precise location data, and I’m not sure at this point what exactly these files offer in that respect). With Twitter, I’ve already mentioned the hashtags I could potentially use to collect data.

Do the APIs you plan to use actually support the functionality that you need in your application? Show how it does so.

I now know a little bit more about how to work with the KML files and the Google Maps API, so I do believe it’s possible to integrate the two — and, more importantly, that it’s possible for me to learn enough to integrate the two! That said, I still need to learn how to parse the text information that’s included with the KML files and figure out how to best display it on the map. Is it possible I’d actually end up redrawing the map, using a different color line for each day that there’s street sweeping? Or is there an easier way to mess with the information I already have?

The Twitter piece would clearly be the easiest here, since I could just use the RSS feeds of the hashtags.

What programming language do you plan to use?

Not sure – but I hope that whatever I have in mind can be done in Python + whatever we learn in class, because that’s what I know!

Tentative Action Plan
Step 1: Gather my data from city sources (mostly done)
Step 2: Figure out what that data actually includes and how it can be used
(Step 2a: Understand what information is included in a KML file and how to use it)
Step 3: Learn how to use a mapping API and find out what happens if I try to use a KML file with it.
Step 4: Determine the scope of my project (i.e., does it turn out that these files are so easy to work with that I can cover all the parking zones in the whole city? Or should I narrow my focus to one specific, geographic area?)
Step 5: Attempt to integrate the KML street sweeping data with the map.
Step 6: Attempt to integrate the KML parking permit data with the map.
Step 7: Work on various methods for searching (see “high-risk areas” below)
Step 8: Attempt to integrate Twitter with the map

High-Risk Areas/Questions
1. The KML files might not provide exactly the information I’m looking for, or they need to be parsed in a way I don’t understand yet. For example, now that I’ve looked at one of the KML files in Google Earth, I realize that I have no idea how they’re broken up — the files neither correspond to a parking permit zone nor to a government district. That makes me worry about my plan to just cover one zone, because I’m not sure the KML file I would be using would correspond to anything useful.

2. For this to really work well, I’d need to develop a couple of different ways of searching the information — perhaps pulling up a map of the streets immediately around an address as well as allowing someone to enter an address and pull up the nearest Tuesday or Friday spaces. That means dealing with outputs in different formats – I might want to show a route on a map for the first but pull up a text list for the second.

3. Is the programming required for this going to be totally over my head? From our class discussions, it sounds like figuring out how to do spatial search will be the hardest part hands-down.

Spring 201022 Feb 2010 10:42 am

See Day 9 notes on Ajax and an intro to the Google Maps API.

Spring 201017 Feb 2010 04:18 pm

Day 8 notes have been posted.

Spring 201003 Feb 2010 11:32 am

We had such a great discussion on Monday that we didn’t get to cover a lot of what I set out for that day — so we’re going to pick up from two days ago.

Exercises from that day are now due on Sunday Feb 7 at 11pm.

Spring 201001 Feb 2010 10:34 am

Today’s notes have been posted: Mixing and Remixing Information » s10-Day 04 Tagging and Data Feed

Spring 201027 Jan 2010 09:49 am

I’ve decided to post some slides to guide our work today:

Spring 201025 Jan 2010 10:04 am

I’ve posted the notes to Day 2.

Spring 201022 Jan 2010 10:14 am

We’re very fortunate to have Laurel Ruma as a guest speaker (for the course Mixing and Remixing Information: Open Government and the Web) to lead us on “An Overview of Gov 2.0”  on Monday, Jan 25 (12:30-1:15pm, 110 South Hall, UC Berkeley).

Laurel is Gov 2.0/open government evangelist at O’Reilly Media and Co-chair of Gov 2.0 Expo 2010.  She is also co-editor of the forthcoming book Open Government: Collaboration, Transparency, and Participation in Practice. 1st ed. O’Reilly Media, 2010.   Laurel knows a lot about open government and many of the key players in the field and has done a tremendous amount to raise awareness and understanding of this field.

Everyone is invited.

Spring 201020 Jan 2010 10:36 am

I’ve posted the class notes for Day 01 — Course Introduction.

« Previous PageNext Page »