Causal Analysis of Intellectual Complexity in Software Engineering

Chunhao Li, Mengdi Zhang, Sijia Meng, Xuning Gao, Jialin Yu, Jianyuan Hu, Shuo Wang, Ricardo Silva, Bin Chen

Abstract: In modern software engineering, Research and Development (R&D) Efficiency is pivotal for project success and competitive advantage. Intellectual Complexity, encompassing problem complexity and cognitive load, significantly impacts R&D Efficiency. However, the causal relationships between cognitive load factors and duration of development remain inadequately explored. This study addresses this gap by applying advanced causal discovery and inference techniques to elucidate these relationships within the software development process. Utilizing a comprehensive dataset from the company’s internal development systems, we employ Graphical Lasso (Glasso) and Feature Selection using MMD and Generative Neural Networks (FSGNN) to construct an initial causal skeleton, which is further refined using ARACNE and expert domain knowledge. Causal Graph Neural Networks (CGNNs) are then leveraged to orient the edges, resulting in a coherent Directed Acyclic Graph (DAG). Through causal inference, we quantify the Average Treatment Effects (ATE) of key cognitive load factors, including story points, repository count, and cyclomatic complexity, on development duration. Our findings reveal that reducing task complexity and code complexity can significantly decrease development time, by up to one day per unit change, providing actionable insights for optimizing R&D processes. This study advances the application of causal models in software engineering and provides practical strategies to enhance R&D Efficiency by mitigating intellectual complexity.

External IDs:dblp:conf/ijcnn/LiZMGYHWSC25