About the research
Iowa DOT consumes data from multiple streams, including probe data from INRIX and weather data from Mesonet, which is stored to assist in smart decision making. The cumulative data size for the past 5 years of data can easily be in the range of 15–20 terabytes.
Despite access to unprecedented amount of data, decision makers are often restricted in their ability to explore these data sets. Under the present set up, a simple query, such as how many crashes happen during congested conditions, can’t be answered easily and requires a dedicated research project.
This research proposes to demonstrate a simple proof of concept of cyberinfrastructure that addresses constraints on decision makers. The three data sets that will be used for this demonstration will be crash, INRIX mobility, and weather for the past three years.
The demonstration will include the following benefits over the existing system:
- Data from multiple sources will be saved in a format that leads to extended ability to query across these data sources. So, queries like how many crashes happened during snow and congested conditions should be easy to perform.
- The high-performance compute cluster will be used to store and manipulate the data, and hence, the data processing time will be significantly reduced.
- The data set will be available for visual queries. The decision maker can visually filter and explore the data using dashboard developed for the project.
- Automatic data mining and clustering techniques will be deployed to identify typical trends and anomalies that depart from these trends.