## Select and Augment: Enhanced Dense Retrieval Knowledge Graph Augmentation
Micheal Abaho, Yousef H. Alfaifi
Keywords: 
JAIR/2023/Proceedings/14365 - Select and Augment: Enhanced Dense Retrieval Knowledge Graph Augmentation.pdf
Project URL: 

### Implementation
_Given the documentation given by the authors on the method, how much time investment would it be to re-implement the method from scratch?_

[10]

No implementation provided by authors +4. No explicit mention of libraries, frameworks, or extensive details +4. Authors mention that they use a pre-trained model, which they site, but no link is included. No pseudocode, but good use of framework overview figure +1. 

### Data
_Given the data description in the documentation, how much effort take to either: Find the same dataset the authors used, or similar datasets and defend the comparability, or acquire one from scratch?_

[3]

No direct link, but data sources are cited +1. Datasets are not described +1. Dataset statistics are included. Dataset is not private. 

### Configuration 
_Given the (hyper)parameters, including semantic parameters, of the method: How much effort would it take to acquire the algorithm configurations used for their results, and compare against their budgetary constraints?_

[5]

Method hyperparameters are included, but not all summarised (Table 2 does not include k or alpha) +1. Not all hyperparameter values are clearly stated +1. Authors state that hyperparameters were tuned on validation data, but not using which method and with which budget +2. 

### Experimental Procedure
_Given the experimental set-up of the work, how difficult is it to set up a new experiment, similar to those presented in the original work, with the same procedure?_

[1]

Metrics are explicitly listed, but though the authors state they follow previous work, there are no concrete citations when the metrics are introduced +1. Train/val/test split is clearly stated. Evaluation acquisition strategy clearly stated. Aggregation is clear, no statistical measures were used.

### Expertise
_How much effort would it take to acquire the expertise required to reproduce the work independently relying on the available documentation?_

[2]

The paper contains concrete equations where necessary, and contains a good framework overview image to clarify. Some knowledge of common LLMs may help, but is not required, as these models can be treated as a black-box within the framework. 
