
## What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration
Libo Qin, Qiguang Chen, Hao Fei, Zhi Chen, Min Li, Wanxiang Che
Keywords: 
NeurIPS/2024/Proceedings/95185 - What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration.pdf
Project URL: 

### Implementation
_Given the documentation given by the authors on the method, how much time investment would it be to re-implement the method from scratch?_

[10]

Overview in figure 1, 2, 3. The code is not released. 

### Data
_Given the data description in the documentation, how much effort take to either: Find the same dataset the authors used, or similar datasets and defend the comparability, or acquire one from scratch?_

[4]

(19/19)

The authors list the datasets in table 2 with the task category and citations. No links, or statistics given and only task descriptions.

### Configuration 
_Given the (hyper)parameters, including semantic parameters, of the method: How much effort would it take to acquire the algorithm configurations used for their results, and compare against their budgetary constraints?_

[1]

The authors state the parameter space for the experiments in section 4. However these parameters are relating to the baselines and not their own method. Based on their description, the method seems HP free.

### Experimental Procedure
_Given the experimental set-up of the work, how difficult is it to set up a new experiment, similar to those presented in the original work, with the same procedure?_

[3]

The authors describe the metrics in section 4 with citation. Many are standard. The authors state in caption of table 2 that they 'evenly' sampled 500 items of each data set of table 2, but what that means is not clear. No train/test split applicable or strategy either.

### Expertise
_How much effort would it take to acquire the expertise required to reproduce the work independently relying on the available documentation?_

[6]

Requires expertise on multi-modal context learning, and since the code is missing the entire evaluation will have to be grasped thoroughly for reimplemnting.
