A safety-critical decision support system evaluation using situation awareness and workload measures

Mohsen Naderpour, Jie Lu, Guangquan Zhang

Abstract: Highlights • The suitability of a cognitive decision support system is investigated. • An evaluation approach considering situation awareness and workload measures is proposed. • A computerized system based on the proposed approach is implemented. • The implemented system is used in a safety-critical environment. Abstract To ensure the safety of operations in safety-critical systems, it is necessary to maintain operators׳ situation awareness (SA) at a high level. A situation awareness support system (SASS) has therefore been developed to handle uncertain situations [1]. This paper aims to systematically evaluate the enhancement of SA in SASS by applying a multi-perspective approach. The approach consists of two SA metrics, SAGAT and SART, and one workload metric, NASA-TLX. The first two metrics are used for the direct objective and subjective measurement of SA, while the third is used to estimate operator workload. The approach is applied in a safety-critical environment called residue treater, located at a chemical plant in which a poor human-system interface reduced the operator׳s SA and caused one of the worst accidents in US history. A counterbalanced within-subjects experiment is performed using a virtual environment interface with and without the support of SASS. The results indicate that SASS improves operators׳ SA, and specifically has benefits for SA levels 2 and 3. In addition, it is concluded that SASS reduces operator workload, although further investigations in different environments with a larger number of participants have been suggested.