Moving Cities Beyond Performance Measurement

Article February 4, 2015 Read time:

Projects: American Cities Project

Overview

The recent explosion in the availability of data is changing the way Americans make decisions and do business in fields as diverse as sports, public health, shopping, and politics. The business of government is no exception. At the local level, new methods of collecting and analyzing information have varied and far-reaching effects on the ability of leaders to understand and work within their fiscal constraints and meet residents’ needs.

Local governments have used performance measurement—collecting and studying data with the aim of improving operating efficiency and effectiveness—for decades, but today’s cities have access to a wealth of other data. Those on the cutting edge are using these data with new analytical tools in innovative ways that often reach beyond the conventional definition of performance measurement. For example, the New York City Fire Department compiles information from various city departments about building characteristics—such as construction material, fireproofing, height, date of construction, and last inspection date—to prioritize buildings for inspections. Boston uses a cellphone app, called Street Bump, to help detect potholes using the accelerometers built into cellphones.

What’s new, beyond the sheer volume of data, to help governments improve?

Local governments previously examined statistics only within individual departments. Today, they are gleaning new insights by combining data across agencies.
Government officials typically reviewed performance statistics only periodically—annually, semiannually, or quarterly. Now they often have access to usable data in real time, allowing them to be more responsive and efficient.
In the past, cities primarily used analytics to understand past events. Today, some are exploring predictive analytics, using data to anticipate occurrences and outcomes.

As local governments continue to operate under fiscal restraints after the Great Recession (as recent Pew Charitable Trusts research found), data and analytics offer cities a critically important way to stretch limited dollars and improve services. Cities today, in the words of one Boston official, need to be ambidextrous organizations that can collect trash, teach kids, and enforce laws today but also innovate and learn to do better tomorrow.¹

This analysis looks at some of the innovative ways in which cities are using new tools and technologies and considers some of the challenges they face in using data effectively.

Using data in new ways

The huge quantities of data now available to governments present opportunities for cities seeking to improve services while cutting costs.

In an interview with Pew researchers, Jeff Tryens, a former New York City deputy director for performance management, noted that “performance measures are only one place to look for the data that you need to improve whatever it is you’re doing.”² He pointed to New York’s efforts to figure out which restaurants were dumping cooking oil into sewers and clogging pipes across the city. Instead of sending inspectors out to try to catch perpetrators in the act, the Mayor’s Office of Data Analytics compared a list of restaurants that have grease-hauling contracts with the locations of sewer blockages—information from unrelated city departments that had not been connected before—to determine which restaurants were most likely to be dumping grease.

Tryens said these data sources didn’t shed much light until city personnel figured out how to effectively cross-reference them. “The rest of the fun stuff was doing lots of analytics to try and figure out what was going on which caused that performance measure to underperform,” he said. City inspectors eventually issued violations on 95 percent of the targets on the suspect list, according to the Mayor’s Office of Data Analytics. The increased enforcement led to a decrease in sewer blockages and to savings on inspection and remediation.

Other cities around the country are also using data and analysis in innovative ways:

Boston’s Problem Properties Task Force identifies and responds to “problem properties,” which are nuisance buildings and/or vacant lots with persistent criminal activities and code violations. Four substantiated complaints within a 12-month period can land a property on the city’s list of such properties and result in fines and other enforcement actions. The task force analyzes trends using data points from various city departments—including 25-month crime statistics by neighborhood or police district and top-10 address lists for code violations—to predict which properties are at risk for further problems. It then works with landlords to address complaints and violations promptly. In the first two years of the task force’s operation, 275 properties were identified as potential problems. Of those, 58 were listed as problem properties, 44 of which have been remedied and removed from the list. An additional 39 cases were resolved before they were officially classified as problems. Calls to 911 about properties designated for crime-related issues fell by about 70 percent after the cases were deemed resolved.³
Detroit collects information about response times, medical emergencies, calls for assistance, and other matters from the Fire Department, computer-aided dispatch, 911 dispatch, geographic information system, and other records through FireView Dashboard, a real-time tracking system. City officials use the information to allocate resources for the Fire Department, estimate response times, and plan community outreach. Budget cuts have forced the department to temporarily shut down some fire companies on a rolling basis to save on overtime costs, but the city had little information about how the brownouts would affect response times. One city official told Pew, “To get a response time would be to get two light-duty personnel to go through boxes of written reports and get a calculator to average out the response times.”⁴ The new system has helped the department determine which fire companies to brown out at what times to minimize the impact on response times, improving services for residents while maximizing city resources.
Las Vegas is using a system known as the Park Asset Data Collection and Data Conversion Program (ParkPAD) to track and measure the city’s park system, cutting costs while improving services for residents. The system stores quantitative data and maps for all park amenities, including benches, restrooms, trees, soil, sod, and hundreds of other components. Previously, the city had to pay for staff time to assess the needs before work could begin. For example, if a park needed new sod, the city would send an employee to the park with a surveyor’s wheel to measure the area needing replacement. Today, using ParkPAD’s digital maps and measurements, the city can determine how much sod it needs within minutes, saving hours of manual labor and improving accuracy.

Using data effectively

For local officials hoping to make real and ongoing improvements to government operations, collecting and analyzing data are just the beginning.

Speaking at a recent National League of Cities conference, Rick Cole, deputy mayor for budget and innovation in Los Angeles, said cities should use data to identify potential problems, understand why they are happening, and find solutions. “It’s not the numbers. It’s what you do with the numbers,” he told the audience in Austin, Texas.

Cole advised city officials to check data frequently and make adjustments to operations as necessary to improve performance. In addition, he encouraged leaders to foster a culture among municipal employees that prioritizes innovation and enhancement rather than placing blame, noting that punitive environments inevitably lead to a temptation to “cook the books.”

Whether deliberate or accidental, inaccurate information can lead to flawed decision-making. In New York City, numbers showing a dramatic drop in violence at Rikers Island were found to be faulty, omitting hundreds of inmate fights. Two officials at the facility were promoted based on the erroneous figures. City investigators subsequently concluded in a confidential report that the numbers were inaccurate and recommended that the officials be demoted—one has since retired—according to The New York Times.

Determining which measures are meaningful in assessing government performance can also pose challenges. A recent report on performance management from the National League of Cities explored the issues through interviews with staff from cities across the country. Two frequently offered suggestions: that officials work with employees in city departments to identify which performance metrics to use and that cities measure both outcomes (long-term impact) and outputs (actions taken or completed). The NLC report notes that selection of the most appropriate metrics is often an iterative process, requiring adjustments over time to ensure the best results.

When done correctly, performance metric selection leads municipal leaders to think about the broader questions of whom they are trying to serve and how. Cole gave the example of libraries: Twenty years ago, libraries might have been judged on how many books were checked out. Today, they serve many other purposes, such as providing a safe place for children to go after school and serving as a resource for adults looking for jobs. Because of this evolution, the performance of a modern library requires new metrics.

Conclusion

New data and analytics offer local leaders the opportunity to provide better, more efficient services even as budgets remain tight.

Stephen Goldsmith, a professor of the practice of government at Harvard University’s Kennedy School of Government and a former two-term mayor of Indianapolis, called the potential for cities to improve performance using data and analytics “enormous and unlimited.”⁵

“We are at a point in time where the tools that allow us to drive performance exceed the application of those tools,” Goldsmith said. “It’s not technology that’s holding us back; it’s the conceptualization of how you use the tools in a practical way.”

Endnotes

Chris Osgood, co-chair of Boston Mayor’s Office of New Urban Mechanics, interview with Pew staff, April 22, 2014.
Jeff Tryens, interview with Pew staff, Nov. 11, 2014.
Chris Dwelley, Boston citywide performance manager, interview with Pew staff, Nov. 20, 2014; Christopher English, city policy analyst who helps to lead the Problem Properties Task Force, pers. comm., Jan. 29, 2015.
Orlando Watkins, now-retired captain with the Detroit Fire Department, interview with Pew staff, April 11, 2014.
Stephen Goldsmith, interview with Pew staff, Dec. 2, 2014.