Cities are finding useful ways of handling a torrent of data

Cities are finding useful ways of handling a torrent of data

Apr 27th 2013 | CHICAGO |From the print edition

THANKS to Brett Goldstein, Chicago’s chief information officer, it is easy to discover a great deal about his city. In the past three months 5,973 vehicles were moved; since the start of 2011, 72,687 complaints about faulty lights in alleyways have been reported; and in the first half of 2012 the tourist-information website was apparently unavailable for 5,870 minutes. (The city says this was caused by a fault in the monitoring software.)

Needless to say, Mr Goldstein will want to get this fixed if he is to retain his annual salary of $154,992. Yet the nugget of data is a tiny detail in a vastly larger enterprise: to make Chicago’s data openly accessible and useful to the millions of people who live and work there.

Many cities around the country find themselves in a similar position: they are accumulating data faster than they know what to do with. One approach is to give them to the public. For example, San Francisco, New York, Philadelphia, Boston and Chicago are or soon will be sharing the grades that health inspectors give to restaurants with an online restaurant directory.Another way of doing it is simply to publish the raw data and hope that others will figure out how to use them. This has been particularly successful in Chicago, where computer nerds have used open data to create many entirely new services. Applications are now available that show which streets have been cleared after a snowfall, what time a bus or train will arrive and how requests to fix potholes are progressing.

New York and Chicago are bringing together data from departments across their respective cities in order to improve decision-making. When a city holds a parade it can combine data on street closures, bus routes, weather patterns, rubbish trucks and emergency calls in real time.

As cities also start to look back at historical data, fascinating discoveries are being made. Mike Flowers, the chief analytics officer in New York, says that if a property has a tax lien on it there is a ninefold increase in the chance of a catastrophic fire there. And businesses that have broken licensing rules are far more likely to be selling cigarettes smuggled into the city in order to avoid paying local taxes. Over in Chicago, the city knows with mathematical precision that when it gets calls complaining about rubbish bins in certain areas, a rat problem will follow a week later.

The next step is to use these predictions to inform policymaking. New York is already doing this, for example by deciding where to send its cigarette-tax inspectors. Chicago is not quite at this point yet, but is ambitiously trying to build an “open-source predictive analytics platform”. This means that it will publish as many data as it can, as close to real time as possible, in a way that will allow anyone to mine them for useful insights into the city.

Moreover, the software Chicago plans to create will be made public, allowing other cities to use it to set up similar systems of their own. (New York keeps its analysis behind closed doors and uses proprietary technology.) It is a big job and means cleaning up 10 billion lines of unstructured data. The hope is that entirely new services will emerge, as well as a great deal of new intelligence about how the city works.

It is still unclear whether Rahm Emanuel, the mayor of Chicago, is ready to let data run his city. If he is not, then all these efforts will result in little more than a City Hall think-tank. Mr Emanuel seems committed. One obstacle is clear, though. All these data will also allow the public to scrutinise the mayor and his officials more closely than ever before.