Distributed Traffic Flow Prediction with Label Proportions: From in-Network towards High Performance Computation with MPI

Thomas Liebig, Marco Stolpe, Katharina Morik

Abstract:

Modern traffic management should benefit from the diverse sensors, smart phones, and social networks data that offer the potential of enhanced services. In disaster scenarios, it is no longer guaranteed that a central server and reliable communication is always available. This motivates a distributed computing setting with restricted communication. Also in distributed High Performance Computing communication costs have to be reduced to the minimum and costly broadcast to all compute nodes hould be avoided. We want to learn local models with high communication efficiency. They still require the exchange of label information in a setting of supervised learning. The transmission of all labels among the nodes can be as costly as communicating all observations. Sophisticated methods are required to trade-off prediction performance against communication costs.

We hereby present an in-network algorithm based on local models that only sends label counts to neighboring nodes. Therefore the method is a novel approach that transfers no data about individual observations, but just aggregated label information. We outline its MPI implementation. And evaluate our approach on real world data in a traffic monitoring scenario. Tests reveal that in comparison to sending all labels, the algorithm is scalable.