
## ExpeL: LLM Agents Are Experiential Learners
Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, Gao Huang
Keywords: NLP: (Large) Language Models, CMS: Agent Architectures, ML: Reinforcement Learning, PRS: Planning with Language Models
AAAI/2024/Proceedings/31635 - ExpeL: LLM Agents Are Experiential Learners.pdf

### Implementation
_Given the documentation given by the authors on the method, how much time investment would it be to re-implement the method from scratch?_

[2]

The authors provide the source code in a GitHub repository (https://github.com/LeapLabTHU/ExpeL), which is documented by including references to the paper and installation notes but does lack some comments or has very short comments for code explanation. The authors also provide prompts/demos and three examples of pseudo code. Furthermore, graphic visualisations of the method are presented in the documentation.

### Data
_Given the data description in the documentation, how much effort take to either: Find the same dataset the authors used, or similar datasets and defend the comparability, or acquire one from scratch?_

[3]

(4/4)

The method uses four publicly available datasets (benchmarks), and documents them with references and a brief description. No statistics can be found. No direct links available.

### Configuration 
_Given the (hyper)parameters, including semantic parameters, of the method: How much effort would it take to acquire the algorithm configurations used for their results, and compare against their budgetary constraints?_

[4]

The authors introduce various parameters to their algorithm as can be seen in their pseudo code. They do not seem to specify under which values of these parameters the results were acquired, although they do seem to be available on the GitHub but its a bit unclear. No specification on how this was achieved.

### Experimental Procedure
_Given the experimental set-up of the work, how difficult is it to set up a new experiment, similar to those presented in the original work, with the same procedure?_

[1]

The authors specify what techniques of validation are used and what statistics are reported.

### Expertise
_How much effort would it take to acquire the expertise required to reproduce the work independently relying on the available documentation?_

[3]

The method requires a good understanding of text-based learning and transfer learning to grasp completely, but due to the substantial amount of available documentation, including code, examples, illustrations and pseudo code, the method is relatively accesible.