An important characteristic of cyber-physical systems is their capability to respond, in-time, to events from their physical environment. However, to the best of our knowledge there exists no benchmark for assessing and comparing the interrupt handling performance of different software stacks. Hence, we present a flexible evaluation method for measuring the interrupt latency and throughput on ARMv8-A based platforms. We define and validate seven test-cases that stress individual parts of the overall process and combine them to three benchmark functions that provoke the minimal and maximal interrupt latency, and maximal interrupt throughput.

Quantifying the Latency and Possible Throughput of External Interrupts on Cyber-Physical Systems

To rigorously benchmark the performance of low-power wireless protocols, it is essential to monitor and quantify the RF activity in a given testing environment. Indeed, unwanted radio interference in the surroundings of wireless nodes may worsen their communication performance. Similarly, an inconsistent RF noise across multiple test runs may prevent the ability to fairly compare their results. Unfortunately, to date, this aspect is often neglected by the community, especially due to the lack of monitoring tools enabling a quantitative assessment of RF activity in large testing facilities. In this paper, we move the first steps towards the creation of a low-cost tool automating the distributed monitoring of RF usage in a low-power wireless testbed. Specifically, we first instrument modern Raspberry Pi devices to sense any ongoing activity on the RF channel, enabling a functionality that is typically not available on off-the-shelf Wi-Fi hardware. We then show that one can synchronize the RF measurements of multiple Raspberry Pi connected to a common Ethernet backbone with an average error below 200 us. We further devise exemplary strategies to quantify the difference in RF activity across test runs, and enable the real-time detection of deviations in the current RF channel usage compared to what was measured in an earlier run. We finally showcase the ability to compare the RF activity during several test runs and detect when additional radio interference was present in the environment, as well as when diverse interference patterns were artificially generated. 

Towards an Automated Monitoring of RF Activity in Low-Power Wireless Testbeds

The development, evaluation, and comparison of wireless IoT and cyber-physical systems requires testbeds supporting inspection of logical states and accurate observations of physical performance metrics. We present FlockLab 2, a second generation testbed supporting multi-modal, high-accuracy and high-dynamic range measurements of power and logic timing and at the same time in-situ debug and trace infrastructure of modern microcontrollers allowing for reproducible evaluation and benchmarking. We detail the architecture, provide a characterization and demonstrate the interface, the supported services and the tools of the FlockLab 2 testbed. 

FlockLab 2: Multi-Modal Testing and Validation for Wireless IoT

Synchronous transmissions (ST ) is a wireless communication technique that has been shown to be particularly efficient in low-power multi-hop networks. Since 2011, research on ST mainly focused on the physical layer defined by the IEEE 802.15.4 standard. Nowadays, Bluetooth is another pervasive technology embedded by default in almost all connected objects; researchers recently started to investigate whether the benefits of ST also apply to Bluetooth.

This paper presents the results of a replication study of ST using the popular and low-cost nRF52840 Dongle, which supports all modes of the Bluetooth 5 standard as well as IEEE 802.15.4. We measure the packet reception rate for different parameters known to affect ST for all physical layers supported by the platform. We use a data exploration application that allows to extract useful information from the measurements and uncover new insights. We confirm that ST is viable on Bluetooth, as previously shown. Moreover, our data show that successful ST on Bluetooth cannot be explained by “constructive interference” or capture effect alone: multiple effects interplay in a way that is not yet fully understood.

MobiCom 2020 Workshop CPS-IoTBench Submissions

Quantifying the Latency and Possible Throughput of External Interrupts on Cyber-Physical Systems

Towards an Automated Monitoring of RF Activity in Low-Power Wireless Testbeds

FlockLab 2: Multi-Modal Testing and Validation for Wireless IoT

Synchronous transmissions on Bluetooth 5 and IEEE 802.15.4 – A replication study