Which API is Faster: Mining Fine-grained Performance Opinion from Online Discussions

Yuekai Huang, Junjie Wang, Song Wang, Rupeng Zhang, Qing Wang

Abstract: Inefficient API usage is one of the main reasons for software performance issues. Current practice of API documentation mainly provides its functionalities, while the performance related information are seldom covered in the official documentation. Meanwhile, the online discussions brings various pieces of information about the efficiency of API, yet buried in massive messages. Existing approaches would derive API opinion with pattern-based techniques, and typically result in inaccurate and coarse-grained result. This paper proposes a relation-aware approach RAMiner for the fine-grained API-related performance opinion mining from online discussions. It leverages pre-trained Large Language Model (LLM), thus can better capture the semantics of the text and API tokens. Besides, it disentangles the task into subtasks to cope with the situation of limited labeled data for fine-tuning the model, and incorporates relation-aware design for capturing the fine-grained opinion of each mentioned API. The experimental results show that, RAMiner can correctly predict 70% opinions, which largely outperforms the baselines. We also demonstrate its potential usage in promoting the code generation models in recommending more efficient code snippets. This approach can also be utilized to extract other non-functional opinions, e.g., security, compatibility.