Hide navigation sidebar
Hide table of contents sidebar
Back to top
Toggle table of contents sidebar

# Benchmarking Training

## Prerequisites

Before you begin, ensure you have done the following:

### Install A2Perf

For detailed instructions, please refer to our Installation Guide.

### Proper Submission folder structure

Your submission should be placed in the `A2Perf/a2perf/a2perf_benchmark_submission/` directory. The structure should look like this:

    A2Perf/
    ├── a2perf/
    │   ├── a2perf_benchmark_submission/
    │   │   ├── init.py
    │   │   ├── train.py
    │   │   ├── inference.py
    │   │   ├── requirements.txt
    │   │   └── [your supporting files and directories]

To get started quickly, you can use our template repository as a starting point. You can find the template at:

[REDACTED]

This template provides the basic structure and files needed for your submission. You can clone this repository and modify it to fit your specific implementation.

#### Explanation of files:

- `__init__.py`: The `__init__.py` file can be an empty file.

- `train.py`: The `train.py` file includes the function `train()`, which A2Perf calls for the training of your algorithm.

- `inference.py` Next, the `inference.py` file is subsequently used for benchmarking the trained agent. This file includes several key functions.
  `load_model(env)`: This function loads and returns the trained agent. A2Perf passes the environment that is being tested via the `env` parameter. This allows the agent loading logic to use any context needed, such as the environment object.
  `preprocess_observation(observation)`: Preprocesses the observation before feeding it to the agent. If no preprocessing is required, simply return the initial observation.
  `infer_once(model, observation)`: Passes a single observation to the loaded agent and returns the predicted action. This function performs a single inference step.

- `requirements.txt`: Specifies any package dependencies required to run the submission code. This file may include version constraints for the dependencies. Having an explicit requirements file ensures a consistent environment for evaluation.

## Update the `a2perf_benchmark_submission` Submodule

For the purposes of this tutorial, we will use the repository of our local baselines as the “submission.” To use this, please update the `a2perf_benchmark_submission` submodule to the following branch: `baselines-local`

### Navigate to the Submodule Directory

    cd a2perf/a2perf_benchmark_submission

### Checkout the branch with code for baselines

    git fetch origin
    git checkout baselines-local

### Pull Latest Changes

    git pull origin baselines-local

### Back to the Main Directory

Return to the root directory of the `A2Perf` repository:

    cd ../..

## Running the Training Benchmark

### Running locally with XManager (Docker)

#### Running the Benchmark

    xmanager launch xm_launch.py -- \
      --experiment-name=test \
      --root-dir=~/gcs/a2perf/experiments/ \
      --domain=QuadrupedLocomotion-DogPace-v0  \
      --submission-gin-config-path=a2perf/submission/configs/quadruped_locomotion/train.gin \
      --user=$USER \
      --participant-module-path=a2perf/a2perf_benchmark_submission \
      --participant-args="gin_config_path=configs/quadruped_locomotion/dog_pace/ppo.gin"

#### Command line arguments

- `root-dir`: Specifies the directory where experiment logs and artifacts will be saved.

- `gin-config`: Points to the Gin configuration file for the Dog Pace environment in the Quadruped Locomotion domain.

- `participant-module-path`: Indicates the path to the directory containing the submission code. Adjust this path to point to your `a2perf_benchmark_submission` directory.

- `participant-args`: Provides additional arguments for the participant’s code, including the path to the algorithm-specific Gin configuration file.

XManager will automatically launch a Docker container with the necessary dependencies installed. It will also create a new experiment directory at `~/gcs/a2perf/experiments/<experiment-id>/test/1/`. The number `1` is included because we are running a single work unit in the experiment. For more details on work units, refer to XManager’s documentation.

Important: Make note of the `<experiment-id>` in your experiment directory path. You will need this ID when running the inference benchmark later. The `<experiment-id>` is a unique identifier for your training run and is typically a long string of numbers.

The experiment directory will contain all logs and artifacts generated during the benchmark. Here is how the directory structure will look at the end of training:

    ~/gcs/a2perf/experiments/1724700456099
    └── test
        └── 1
            ├── collect
            ├── metrics
            ├── policies
            ├── submission_config.gin
            ├── training_complete
            └── train

- `collect/`: Contains TensorBoard summaries for each of the separate collection policies. Each subdirectory (e.g., `actor_0`, `actor_1`, …) corresponds to a different actor’s summary data.

  Example structure:

      collect/
      ├── actor_0
      │   └── summaries
      │       └── 0
      ├── actor_1
      │   └── summaries
      │       └── 1
      └── ...

- `metrics/`: Stores system metrics collected during training, such as `train_emissions.csv`, which logs energy consumption and emissions data.

- `policies/`: Contains the saved policies generated during training. This includes different policies such as `collect_policy`, `greedy_policy`, and the main `policy`. Each subdirectory represents a different policy and contains the necessary files for TensorFlow models, including saved models and variables.

  Example structure:

      policies/
      ├── collect_policy
      ├── greedy_policy
      └── policy

- `train/`: Contains additional checkpoint information and TensorBoard logs from the training process, which are useful for monitoring training progress and debugging.

### Running Locally Without Docker

If you prefer to run the benchmark locally without using Docker, follow these steps:

#### Installing Dependencies

To run the benchmark locally, you need to manually install the required Python dependencies. Run the following command:

    pip install -r A2Perf/a2perf/a2perf_benchmark_submission/requirements.txt

#### Running the Benchmark

Once the dependencies are installed, you can run the benchmark for the Quadruped Locomotion - Dog Pace environment with the following command:

    cd A2perf
    export A2PERF_ROOT=$(pwd)
    a2perf $A2PERF_ROOT/a2perf/a2perf_benchmark_submission \
      --root-dir=~/gcs/a2perf/experiments/test_without_docker \
      --submission-gin-config-path=$A2PERF_ROOT/a2perf/submission/configs/quadruped_locomotion/train.gin \
      --participant-args="gin_config_path=configs/quadruped_locomotion/dog_pace/ppo.gin"

#### Command line arguments

- `root-dir`: Specifies the directory where experiment logs and artifacts will be saved.

- `gin-config`: Points to the Gin configuration file for the Dog Pace environment in the Quadruped Locomotion domain.

- `participant-module-path`: Indicates the path to the directory containing the submission code. Adjust this path to point to your `a2perf_benchmark_submission` directory.

- `participant-args`: Provides additional arguments for the participant’s code, including the path to the algorithm-specific Gin configuration file.

Make sure to adjust the paths according to your setup if they differ from the example provided.