
## Deep Neural Networks Learn Meta-Structures from Noisy Labels in Semantic Segmentation
Yaoru Luo, Guole Liu, Yuanhao Guo, Ge Yang
Keywords: Computer Vision (CV)
AAAI/2022/Proceedings/20085 - Deep Neural Networks Learn Meta-Structures from Noisy Labels in Semantic Segmentation.pdf

### Implementation
_Given the documentation given by the authors on the method, how much time investment would it be to re-implement the method from scratch?_

[10]

The authors do not provide a source for their implementation. They pseudo code on their method, some examples but no details on their implementation.

### Data
_Given the data description in the documentation, how much effort take to either: Find the same dataset the authors used, or similar datasets and defend the comparability, or acquire one from scratch?_

[6]

(4/4)

The authors use four data sets over two tasks and provide citations on them. They refer for detailed information on them to the Appendix which is missing due to AAAI limitations. Were this present this cost would be significantly lower. The references do seem to indicate these are publicly available data sets, but this could have been stated by the authors.

### Configuration 
_Given the (hyper)parameters, including semantic parameters, of the method: How much effort would it take to acquire the algorithm configurations used for their results, and compare against their budgetary constraints?_

[5]

The authors presented method only has a single threshold parameter t according to the pseudo code. The configurations of the methods used from previous works are not stated, but could be present in the missing appendix. However these configurations are not on the method itself, yet required to acquire the results.  

### Experimental Procedure
_Given the experimental set-up of the work, how difficult is it to set up a new experiment, similar to those presented in the original work, with the same procedure?_

[7]

The authors state they provide experimental configurations in the appendix. They state they present the testing scores in figure 2. There is no clear explanation on training/testing divison of the data. The metrics are briefly stated and common. 

### Expertise
_How much effort would it take to acquire the expertise required to reproduce the work independently relying on the available documentation?_

[6]

Requires a good understanding on image segmantation and meta structures.
