Commentary: the Big Data breakthrough
The public sector is under pressure to make better use of their data at the same time it’s being asked to “do more with less.” As agencies collect greater volumes of data, administrators are finding that the old methods of analysis of this data are inadequate and that new ideas for handling Big Data are needed to yield breakthrough insights.
In the past, people have constructed hypotheses to query their data. But queries are based on human assumptions, bias and “gut feel,” and the resulting analysis can fail to show important features and relationships within the data. While queries can uncover clues, it is a game of chance that usually results in long periods of iterative guesswork, often requiring specialized skills in math and computer science. This approach simply isn’t scalable for today’s data challenges.
Recently, an exciting new approach to data analysis has been applied to the Big Data challenge. Topological Data Analysis (TDA) is a new technology that enables people to visualize and understand — for the first time — datasets of any size or type including structured and unstructured data by using geometrically inspired algorithms.
Ayasdi is the first company to use TDA to help government agencies and other organizations find breakthrough insights in large and complex datasets quickly and without queries. Ayasdi’s platform explores datasets with hundreds of machine-learning algorithms in minutes to automatically discover insights that could not be determined through query-based or ad hoc approaches. Ayasdi is designed for any type of user, and does not require coding or model building. The cloud-based platform scales to meet the most demanding processing requirements and is secured using industry-standard encryption.
Ayasdi’s software is already being used by federal government agencies to gain actionable insights across multiple and diverse application domains. At the state and local level, Ayasdi can be used to discover hidden insights and patterns in data from traffic, planning and development, census and population, crime and fraud intelligence, and energy consumption and smart grid optimization, as well as other applications.
Visit www.ayasdi.com to learn more or contact me directly at [email protected].
Ben Mann is VP Federal Operations at Palo Alto, Calif.-based Ayasdi. The company has created a scalable solution that allows for analysis of large quantities of complex data. Its Ayasdi Iris solution uses TDA to highlight the underlying geometric shapes of data, and allows for real-time interaction to produce immediate insights.
