Auto STS RL

Slay the Spire automation and reinforcement-learning playground built around the Communication Mod bridge.

This repository currently contains:

• a deterministic combat baseline
• an LLM-powered combat agent
• a TCP bridge between the game and local agents
• a Gymnasium wrapper for live-game interaction
• a small headless combat sandbox for fast experiments

Project Layout

Path	Purpose
src/tcp_bridge.py	Bridge process that receives game state from Communication Mod and forwards commands to an agent
src/llm_agent.py	LLM combat agent that plans a full turn from the current combat snapshot
src/rule_agent.py	Deterministic baseline agent
src/sts_gym_env.py	Gymnasium environment for the live game
src/headless_env.py	Tiny local combat environment for quick tests and data generation
src/entities.py	Shared combat data structures
tests/	Unit tests for the bridge and agents
verify_env.py	Dependency and hardware smoke test
config/.env	Local secrets and runtime settings

Requirements

• Slay the Spire
• Communication Mod
• Python 3.10+
• Python packages used by the project:
  • openai
  • python-dotenv
  • gymnasium
  • numpy
  • stable-baselines3
  • torch

Setup

1. Install the Python dependencies.
2. Copy .env.example to config/.env.
3. Set DEEPSEEK_API_KEY.
4. Optional: set DEEPSEEK_MODEL to the model you want to use, for example deepseek-v4-flash or deepseek-chat.

Example .env keys:

DEEPSEEK_API_KEY=sk-your-key-here
DEEPSEEK_MODEL=deepseek-v4-flash
DEEPSEEK_MAX_TOKENS=4096
DEEPSEEK_REPAIR_ATTEMPTS=1
STS_CARD_INDEX_BASE=1
STS_AUTO_CHOOSE_REWARDS=0
STS_LOG_RAW_LLM=1

Run The Live Game

1. Configure Communication Mod to launch python src/tcp_bridge.py.
2. Start Slay the Spire.
3. Run one agent in a separate terminal:

python src/rule_agent.py

or

python src/llm_agent.py

Notes:

• The bridge listens on 127.0.0.1:9339 by default.
• Communication Mod command indexes are 1-based for cards and 0-based for targets.
• llm_agent.py requests a full-turn plan and executes it step by step.

Gymnasium And Headless Environments

• src/sts_gym_env.py exposes the live game as a Gymnasium environment with actions for playing hand cards and ending the turn.
• src/headless_env.py provides a compact local combat sandbox and includes a small smoke test when run directly:

python src/headless_env.py

Verification

Run the core checks with:

python verify_env.py
python -m unittest tests.test_llm_agent tests.test_rule_agent tests.test_tcp_bridge

Current Status

The live agent path is the main focus of the project. The current implementation emphasizes:

• full-turn LLM planning with cached execution
• deterministic fallback behavior
• bridge-level diagnostics
• reusable RL-style state wrappers