
## Shared Autonomy Systems with Stochastic Operator Models
Clarissa Costen, Marc Rigter, Bruno Lacerda, Nick Hawes
Keywords: Planning and Scheduling: Planning under Uncertainty, Agent-based and Multi-agent Systems: Human-Agent Interaction, Humans and AI: Human-AI Collaboration, Planning and Scheduling: Markov Decisions Processes, Robotics: Human Robot Interaction
IJCAI/2022/Proceedings/0640 - Shared Autonomy Systems with Stochastic Operator Models.pdf

### Implementation
_Given the documentation given by the authors on the method, how much time investment would it be to re-implement the method from scratch?_

[10]

The authors do not provide a link to their implementation. A diagram of their framework is presented in figure 1. They present citations on the first domain (simulator) used. No practical details can be found.

### Data
_Given the data description in the documentation, how much effort take to either: Find the same dataset the authors used, or similar datasets and defend the comparability, or acquire one from scratch?_

[7] 

(0/2)

The authors use two simulated environments in their method, on the fist of which they provide a citation. They provide clear descriptions on each environment but no direct links to the software implementation used. Thus these implementations will have to be traced or comparable ones have to be found and explained.

### Configuration 
_Given the (hyper)parameters, including semantic parameters, of the method: How much effort would it take to acquire the algorithm configurations used for their results, and compare against their budgetary constraints?_

[10]

There is no mention of parameters in the method. The absence of the implementation makes it hard to verify whether parameters are applicable or not. There are various values stated in each experiment, but they are seemingly environment parameters.

### Experimental Procedure
_Given the experimental set-up of the work, how difficult is it to set up a new experiment, similar to those presented in the original work, with the same procedure?_

[4]

The authors evaluate both environments with task specific metrics, which are pretty straightforward. They evaluate this by varying the simulated success rate pd. They state they ran 'over 2000 simulations' in the captions of figure 2 and 3, but not if this is per data point nor how this is aggregated. For the second the amount of simulations is not stated.

### Expertise
_How much effort would it take to acquire the expertise required to reproduce the work independently relying on the available documentation?_

[8]

Requires experience in human interaction system with AI, and a lot of practical experience in the domain due to missing implementation and specifications.
