Reproducibility study of "Data-Driven Methods for Balancing Fairness and Efficiency in Ride-Pooling"
Keywords: ride-pooling, fairness, LSTM, ILP, resource allocation, matching algorithms, reinforcement learning
TL;DR: This is a reproducibility study of the paper "Data-Driven Methods for Balancing Fairness and Efficiency in Ride-Pooling”.
Abstract: Reproducibility Summary
Scope of reproducibility
Our work attempts to verify two methods to mitigate forms of inequality in ride-pooling platforms proposed in the paper Data-Driven Methods for Balancing Fairness and Efficiency in Ride-Pooling: (1) integrating fairness constraints into the objective functions and (2) redistributing income of drivers. We extend this paper by testing for robustness to a change in the neighbourhood selection process by using actual Manhattan neighbourhoods and we use corresponding demographic data to examine differences in service based on ethnicity.
Methodology
The authors of the paper provide preprocessed data and code implemented in TensorFlow, which we transform into PyTorch. Experiments in this reproducibility study can be divided into 3 parts: (1-2) we reproduce the results regarding objective functions and income redistribution using data and settings provided in the paper and code; (3) we apply this approach to the same data grouped into Manhattan neighbourhoods. Further, we examine discrepancies between service rates of different ethnicities using neighbourhood-specific demographic data as a proxy for this protected information.
Results
The results in the original paper regarding different objective functions were reproduced within a margin of error. Also, income redistribution is able to reduce wage inequality, albeit to a lesser degree. The objective functions appear to be sensitive to the neighbourhood selection mechanism. While the results of the rider-fairness objective functions are maintained, performance of the driver-fairness objective functions declines. There appear to be only small differences in service rates between ethnicities, while rider-side fairness seems to mitigate inequalities the most. However, this is only achieved by worsening the service for well-served neighbourhoods instead of improving it for underserved ones.
What was easy
The simulation logic as well as the training and testing procedures in the provided code were straightforward to execute.
What was difficult
To be able to run the authors' code we needed to make several changes to it. Moreover, specific parts of the original research were not explicitly mentioned in the paper. Another point of difficulty was the absence of preprocessing code which was not detailed properly and could not be fully reproduced. The reproducibility of the paper relied on the provided code, communication with the authors as well as previous works.
Communication with original authors
We contacted the authors about the preprocessed data that was not hosted online due to licensing issues. They supplied it as well as responded very quickly and provided clarifications on the parameters and their values in the code.
Paper Url: https://www.ijcai.org/proceedings/2021/51
Paper Venue: IJCAI 2021
4 Replies
Loading