The Ethical Evaluation Method of Algorithmic Behavior Based on Computational Experiments

Fangyi Chen, Xiao Xue, Xiao Wang

Abstract: The widespread application of artificial intelligence algorithms has brought about various ethical concerns, such as algorithm discrimination. While there have been some efforts focused on enhancing the ethical performance of algorithms, the evaluation of their ethical behavior has been largely neglected. However, conducting practical evaluations is often infeasible due to factors such as cost, legal, and other constraints. In this context, computational experiments have emerged as novel and powerful computational theories and tools for quantitative analysis in complex social systems. This paper proposes an ethical evaluation method of algorithmic behavior, called EMAB, which leverages the computational experiment and simulation to construct an AI-driven artificial society, providing a dynamic and feedback-based environment for evaluating the ethics of algorithms. EMAB includes users, algorithms, and a dynamic data circulation mechanism between them. Taking the recommendation algorithm as an example, we design test scenarios to verify the superiority of the fair recommendation algorithm over the unfair recommendation algorithm. The experimental results illustrate the effectiveness and necessity of EMAB. The proposed method provides a novel perspective for algorithm evaluation involving ethics.