The big deal about big data

The world generates a gargantuan amount of data every day — around 2.5 quintillion bytes per day, according to a 2017 IBM estimate.

As digital devices of all shapes and sizes perform countless operations and transactions, they create data footprints that contribute to that daily total. Local governments possess and continually deploy many connected devices, such as utility meters, card readers, surveillance cameras and telematics-enabled vehicles.

The data these devices yield can provide governments with invaluable insights. Governments that have leveraged this sort of data well have gained new abilities and have seen great results in their communities across their operations and budgets. But to be properly utilized, that data must be organized, analyzed and converted into actionable insights.

That can be a hurdle, as the data we’re collecting today isn’t the same data we collected in the past.

The new normal

The data that was generated about 10 to 20 years ago was straightforward. “It was very structured, it had a very specific format that you could then store, process and analyze,” says Vijay Raja, IoT solutions marketing lead at data management company Cloudera.

In contrast, the data that today’s digital devices generate is massive, rapidly transmitted and is highly varied in form. Raja gives the example of a single connected turbine, which can transmit 1 to 2 terabytes of data in one day. Connected cars, he adds, can generate new data every millisecond.

Today, turbine and car data is more likely to be analyzed together, too. Big data implies a “composite of various data inputs” as opposed to data in silos, explains Alan Shark, executive director of the Public Technology Institute (PTI) and an associate professor at George Mason University. Since any sort of information in a digital format can be considered part of big data, it entails a variety of information ranging from structured data like geospatial coordinates or phone numbers to unstructured data like video clips or text messages.

“How do you make sense of this data? How do you drive intelligence, how do you drive actionable intelligence? That really, is what big data and big data analytics is all about,” Raja explains.

Shark says that the departments making most effective use of the data they’ve generated are police and transportation. Big data-related transportation initiatives of two large cities exemplify how working with big data can lead to insightful intelligence on transportation.

The first step in making sense of that data though is actually gathering it.

The art of data collection

When Kansas City, Mo., officials broke ground on a streetcar system in 2012, they decided to make it “the world’s most advanced streetcar,” as Kansas City Chief Innovation Officer Bob Bennett puts it. To that end, the city worked with several technology companies and local contractors to install 328 Wi-Fi access points and 178 traffic video sensors across a 54-block corridor of the streetcar route between mid-2014 and mid-2016.

While the city paid about $1.4 million for the installation contract, these connected devices have given Kansas City a stockpile of ever-increasing data. Coupled with a video analytics platform from Xaqt, the traffic sensors’ data provides real-time detection for traffic, parking, the streetcar line, pedestrian traffic and live events, Xaqt CEO Chris Crosby explains.

“Think of it as a video camera that isn’t storing face recognition or license plates,” Crosby says. “But what it is doing is it’s analyzing what’s in the frame.”

Sensors are a useful and relatively inexpensive addition to a new project, Shark says.  But governments don’t always bear the entire burden of collecting data. Sometimes, municipalities can receive data without installing infrastructure, as Louisville, Ky. has demonstrated with navigation smartphone app Waze.

By participating in Waze’s Connected Citizens program, Louisville publishes construction and road closure data on its open platform, which Waze incorporates into its app for Louisville motorists, Louisville Metro Government Data Officer Michael Schnuerle explains. In exchange, Waze gives the city access to a feed of anonymous data its users collect that shows traffic jams and alerts. That feed is then transmitted to an internal city server.

“It’s really a treasure trove of valuable real-time and historic data that we can analyze to get some insights that we otherwise don’t have any way of knowing,” Schnuerle says.

To test the data’s accuracy, Louisville commissioned a $50,000 traffic study and found that its Waze-generated data was about 90 percent comparable to the study’s findings.  “We didn’t save money in that case, but it helps us validate that we could use this viable method going forward,” Schnuerle explains.

Raja is quick to point out, however, that data obtained from sensors alone is meaningless in a big data context. “The real value from IoT comes when you can combine that sensor data with other internal and external data sources. This is all about how you add context to sensor data,” he says.

Kansas City combines its video feed data with about 4,200 existing city data sets along with third-party data that Xaqt has access to, Bennett says. Louisville meanwhile, has traffic and collision data being fed into its server alongside its Waze data.

But like raw ore obtained in mining, only so much can be done with raw data, even if sourced from different locations. The real value is obtained through refining those materials into valuable assets.

“Data unto itself is meaningless,” Shark explains. “Data has to be formed into information, information leads to better knowledge, knowledge leads to better decision-making and improved quality of life of citizens.”

The question then becomes, whether to use internal staff to turn that big data into knowledge or to contract that work.

In-house vs. contracting: The analysis conundrum

Upon assuming his post as city manager of Cincinnati in September 2014, Harry Black began spearheading the city’s adoption of data analytics and performance management initiatives.  By May 2015, Cincinnati had a dedicated Office of Performance Data Analytics (OPDA) business in its own building.

The OPDA has data-driven performance agreements with each city department, and uses its dedicated statistics facility to analyze data, Black explains.

In addition, Cincinnati’s OPDA has an internal IT staff, and its data team has gotten its own tools to work with the data in the ways it needs to. Securing its own resources has helped to sustain the OPDA’s operation internally while allowing the data team to work directly with experts at other city departments.

“We’ve now been able to harvest their data — working with the subject matter experts, understanding what we’re trying to pull and then [automating] those processes,” Cincinnati Chief Data Officer Brandon Crowley explains. “We solely live and sustain our success through automation, so we buy tools that will help us facilitate that purpose, automation, and that’s how we’ve been successful.”

By building its own data warehouse infrastructure, the city estimates it’s avoided $500,000 in consultant and data management services, city documents show. Additionally, the city estimates it avoided $860,000 due to “purchasing developing, centralizing and automating IT services.”

“We utilize outside, external resources, but to the extent that we don’t have to, we try not to. We want to try to become as independent as possible,” Black explains, citing manageability and dependability as major reasons for this approach.

Like Cincinnati, Louisville does all of its data analysis internally. Schnuerle and another staff data scientist handle the bulk of the analysis using different tools, but they’ll sometimes work with data professionals within other departments. Louisville is also using Amazon Web Services (AWS) tools to build its own data warehouse in the cloud. For the Kentucky municipality though, this approach offers more than just dependability and manageability.
“That’s great because it has really in a way forced us and our IT department to become familiar with the cloud and AWS and get the skills and training needed to do that,” Schnuerle says.

Xaqt built the analytics platform that combines the Kansas City smart streetcar corridor infrastructure data alongside existing city data sets and third-party data.  The company’s data scientists also analyze the data and make it operational, with open communication between the company and city hall throughout the process. Xaqt’s work with the city is done as part of a public-private partnership, Bennett explains.

Kansas City has also used Xaqt for other data analysis throughout the city as well. But not all of Kansas City’s data-related work is outsourced. The city has an office of performance management, an open data officer on staff and performs some of its own data analysis.Bennett predicts the city will do more performance measurements assessment in-house.
“They’re used to making data-driven decisions, and when [Xaqt] came aboard, we were able to expand that capacity and deliver more things for them,” Crosby notes.

The results come in

Concerning transportation, Louisville and Kansas City’s big data initiatives have yielded new abilities and knowledge for internal operations that they hadn’t possessed before.

In addition to proving the needlessness for traffic studies, Louisville’s traffic data analysis shows it faulty equipment at intersections that staff can then fix, Schnuerle says. Waze reports of car accidents —replete with location data — are usually transmitted before 911 calls are made, and Louisville staff can point traffic cameras towards the accident so that professionals in the city’s real-time crime center can determine the appropriate response in advance. Louisville employees are working on automating the cameras’ movement towards an accident.

Meanwhile, Xaqt has built several dashboards using data from the streetcar corridor project to better help Kansas City manage parking in real-time through insights in trends of parking violations, compliance, arrivals and occupancy, Crosby explains.

The platform is also giving Kansas City the ability to predict the future. The city is undertaking a pilot project with 10 of its major streets in which data analysis techniques called predictive analytics will be used to predict potholes and forecast street maintenance.

“[The combined data] will allow us to start changing city operations to the point where they become predictive in nature or proactive in nature based on predictive analytics, instead of reactive to a crisis and maybe get in front of that crisis,” Bennett explains.

Taken as a whole, OPDA’s work in Cincinnati proves that effective data analysis and effective use of gained insights can positively affect multiple departments within a city.

As part of the city’s data mapping performed with the Cincinnati Department of Public Services, city green spaces were cleaned and maintained 300 percent more by May 2017 than they were in May 2016, according to city documents. OPDA’s work has led to blighted property abatements to increase from 250 per year in 2015o to 1,000 per year in 2017. Tall grass, weeds and litter-related customer service complaints dropped 59 percent between fiscal year 2016 and 2017.

Fiscal savings is one major benefit felt across multiple Cincinnati departments as a result of OPDA’s work. Monitoring firefighters’ overtime has resulted in a reduction of $972,491 since the OPDA was introduced, according to city documents. Working with collections departments, citywide debt collection went up $523,840 between 2015 and 2016 as a result of meetings with various city departments.

Many of the meetings OPDA held with other city departments were fostered through the OPDA’s Innovation Lab— a space in the OPDA facility that allows OPDA employees to “break processes out, to rebuild them, to identify problems,” as Black explains. There, the city employs its CincyStat leadership strategy in which the chief performance officer frequently meets with the city manager and each department’s leadership and uses data to examine past performance and set new performance goals, according to city documents. Lean Six Sigma concepts are also applied to better city processes in a collaborative manner.

The existence of this lab and the benefits it’s brought to the city underscore an important part of any big data operation: the need for collaboration and education across departments.

Creating big data culture

Because big data consists of a composite of different data types, the data of multiple departments is often used when generating insights. With insights that can in turn, help multiple departments, educating other departments on big data’s utility and getting them to buy into collaboration is key.

The ability to predict potholes with its data wasn’t an original goal of Kansas City’s when it began its streetcar corridor analytics project. It came from conversations with the city’s public works director when Crosby and Bennett were showing the platforms ability to combine data with analytics, Crosby recalls.

Bennett had encountered challenges in convincing other city employees of the analytics’ relevance. The exchange with the public works director proved to be a breakthrough moment in getting department directors to relate the project’s abilities to relevant usage scenarios.

OPDA’s Innovation Lab and performance agreements with other Cincinnati departments exemplify how collaboration can be employed when working with big data. But a major step in creating a collaborative culture is creating a data governance policy, which Kansas City, Cincinnati and Louisville have in place. Key aspects of such a policy are including every department, accounting for internal and public data storage, and covering data standards, privacy, classification, security and retention, Schnuerle advises.

Educating the public to promote buy-in to big data initiatives is also key. Kansas City, Cincinnati and Louisville post lots of their analyzed data online for citizens of their communities to view. For Cincinnati, there’s evidence that the public has been receptive to this open data—their online interactive data dashboard, CincyInsights, received 58,000 page views between its December 2016 launch and May 2017, according to city documents.

“We share as much as possible,” says Schnuerle of Louisville. “Since I’m in charge of the open data website, I want to put everything out there that I can.”

As a preliminary step towards using big data in government, Shark recommends convening a small working group of professionals across departments to identify how existing data can be used to make better decisions across multiple departments.

Even with such a foot forward in pursuing big data, cultures aren’t created overnight. “We have 19 city departments, and to make the smart cities piece relevant to them means 19 sets of education processes. So that takes a good bit of time,” Bennett says with a laugh.

That may be partly why Cincinnati’s chief data officer is proudest not necessarily of a particular project or set of OPDA projects, but of a more abstract accomplishment.

“We now have a culture where we rely on data,” Crowley says. “… We pat ourselves on the back if you will, about [our] particular projects. But I think what we’re most proud of is the culture that we’ve been able to change here in the city of Cincinnati around data and the need for data.”

p.p1 {margin: 0.0px 0.0px 15.0px 0.0px; line-height: 22.2px; font: 15.0px Georgia; color: #323333}
p.p2 {margin: 0.0px 0.0px 15.0px 0.0px; line-height: 22.2px; font: 12.0px Georgia; color: #323333}
span.s1 {font-kerning: none}
span.s2 {font: 12.0px Georgia; text-decoration: underline ; font-kerning: none; color: #0e5f8b}
span.s3 {font: 15.0px Georgia; font-kerning: none}

_____________

_{To get connected and stay up-to-date with similar content from American City & County:
Like us on Facebook
Follow us on Twitter
Watch us on YouTube}