hmOS: An Extensible Platform for Task-Oriented Human-Machine Computing

Hui Wang, Zhiwen Yu, Yao Zhang, Yanfei Wang, Fan Yang, Liang Wang, Jiaqi Liu, Bin Guo

Abstract: With rapid advancements in artificial intelligence (AI) technologies, AI-powered machines are increasingly capable of collaborating with humans to enhance decision-making in various human–machine collaboration scenarios, e.g., medical diagnosis, criminal justice, and autonomous driving. As a result, human–machine computing (HMC) has emerged as a promising computing paradigm that integrates the expertise of humans with the reliable data processing capabilities of machines. Using HMC to facilitate the processing of domain-specific tasks has a lot of potential, but is limited in system-level scalability, i.e., there is no one common easy-to-use interface. In this article, we present human-machine operating system (hmOS), an open extensible platform for researchers to experiment with HMC for investigating system-centric human–machine collaboration problems. hmOS supports flexible human–machine collaboration on the strength of the quality-aware task decomposition and allocation. To achieve that, the underlying system architecture and runtime environment are first developed to build a foundational abstraction for the kernel of hmOS. Second, hmOS facilitates flexible human–machine collaboration through a suitability-based task allocation mechanism, quality estimation guided by fuzzy rules, and iterative feedback on result tuning. We implement the newly proposed hmOS in a prototype featuring interactive interfaces. Finally, we conduct extensive and realistic experiments to validate the effectiveness of our platform across diverse tasks, showcasing the broad feasibility of hmOS.