A Workflow-Based Methodological Framework for Hybrid Human-AI Enabled Scientometrics

António Correia, Shoaib Jameel, Daniel S. Schneider, Hugo Paredes, Benjamim Fonseca

Abstract: With cutting edge scientific breakthroughs, human-centred algorithmic approaches have proliferated in recent years and information technology (IT) has begun to redesign socio-technical systems in the context of human-AI collaboration. As a result, distinct forms of interaction have emerged in tandem with the proliferation of infrastructures aiding interdisciplinary work practices and research teams. Concomitantly, large volumes of heterogeneous datasets are produced and consumed at a rapid pace across many scientific domains. This results in difficulties in the reliable analysis of scientific production since current tools and algorithms are not necessarily able to provide acceptable levels of accuracy when analyzing the content and impact of publication records from large continuous scientific data streams. On the other hand, humans cannot consider all the information available and may be adversely influenced by extraneous factors. Using this rationale, we propose an initial design of a human-AI enabled pipeline for performing scientometric analyses that exploits the intersection between human behavior and machine intelligence. The contribution is a model for incorporating central principles of human-machine symbiosis (HMS) into scientometric workflows, demonstrating how hybrid intelligence systems can drive and encapsulate the future of research evaluation.