Reward-HCA

The code in this repository supplements the paper "Improving Reward-Based Hindsight Credit Assignment".

Repository Structure

• scripts/train.py — training entry point. Reads its configuration from wandb.config (see config_defaults at the top of the file for all available parameters and their defaults).
• agents/hca.py — the RHCAAlgo implementation (R-HCA / COCOA).
• models.py — actor-critic and hindsight model architectures.
• environments/ — custom Gymnasium environments used in the paper (Catch-v0, ContextualChain-v0), registered with gym.register in environments/__init__.py.
• curr_utils/ — formatting, storage, and other helper utilities (adapted from torch-ac).

Instructions

Run training with:

python3 -m scripts.train

Configuration is read from wandb.config (see config_defaults in scripts/train.py for all parameters and their defaults), so runs are typically launched as part of a W&B sweep, which overrides these defaults per run:

wandb sweep sweep.yaml
wandb agent <sweep-id>

Dependencies

• numpy
• torch
• torch-ac
• gymnasium
• wandb