Quantest

Quantest is a Hermes Agent skill for testing whether an investment idea, alpha signal, factor, timing rule, screen, or portfolio hypothesis survives point-in-time empirical scrutiny.

It is designed to validate measurable claims produced by stock-screening and fundamental-research workflows such as Alphora and the Financial Alpha Team.

What Quantest Does

• Cleans and aligns market, fundamental, and alternative data point in time.
• Supports cross-sectional, time-series, factor, event, and machine-learning research.
• Constructs, normalizes, and neutralizes signals.
• Forms quantile portfolios and evaluates monotonicity.
• Separates discovery, validation, and final holdout periods.
• Runs purged, embargoed, and walk-forward out-of-sample tests where required.
• Models turnover, fees, spreads, slippage, impact, liquidity, capacity, and borrow constraints.
• Performs performance and exposure attribution.
• Tests robustness across periods, sectors, universes, parameters, and implementation assumptions.
• Uses dependence-aware confidence intervals and proportionate multiple-testing controls.
• Reports failed tests and separates statistical significance from economic significance.

Core Safeguards

A Quantest validation must enforce the following controls when applicable:

1. No look-ahead bias.
2. No survivorship bias where constituent history is available.
3. No use of financial statements before their verified publication time.
4. No revised-data hindsight when historical vintages are decision-relevant.
5. Chronological discovery, validation, and final holdout periods.
6. No final-holdout reuse or post-hoc retuning without a new untouched period.
7. Purging and embargo for overlapping labels or holding periods.
8. Training-window-only preprocessing, neutralization, feature selection, and fitting.
9. Realistic implementation costs and liquidity constraints.
10. Disclosure of all material failures, variants, and parameter sensitivity.

The core information clock is:

observation_time -> available_at -> decision_time -> execution_time -> label_end_time

Recommended Research Flow

Alphora screening
       ↓
Financial Alpha Team deep dive
       ↓
Extract 1–3 decision-relevant falsifiable claims
       ↓
Quantest empirical validation
       ↓
Reject / revise / monitor / advance with constraints
       ↓
Portfolio decision and ongoing monitoring

Quantest should test specific measurable claims rather than attempt to reduce an entire qualitative company thesis to one backtest.

Validation Levels

Level	Purpose
Diagnostic	Early idea triage and data-feasibility checks
Standard validation	Point-in-time testing with chronological holdouts, costs, quantiles, and robustness
Full research review	Walk-forward validation, capacity, attribution, bootstrap intervals, and multiple-testing controls

Repository Structure

quantest/
├── SKILL.md
├── README.md
├── references/
│   └── research-design-and-statistics.md
└── templates/
    └── research-tear-sheet.md

• SKILL.md — operating workflow, safeguards, stage gates, and verification checklist.
• references/research-design-and-statistics.md — detailed point-in-time, portfolio, validation, and statistical methodology.
• templates/research-tear-sheet.md — standardized output template for completed empirical tests.

Installation

Clone the repository into a Hermes skills directory:

git clone https://github.com/Koktongkt/quantest.git

Then place or clone it under an appropriate skills category, for example:

<hermes-profile>/skills/research/quantest/

Start a new Hermes session or reload skills after installation.

Suggested Python Ecosystem

Use the smallest stack needed for the research question:

• pandas or Polars
• NumPy and SciPy
• statsmodels
• scikit-learn
• OpenBB when configured and suitable
• vectorbt when its assumptions match the strategy
• QuantStats for supplementary analytics
• PyPortfolioOpt when optimization is genuinely necessary

The skill does not require every package and favors transparent custom research code when publication lags, changing universes, or execution rules make generic vectorization unsafe.

Output

A full run produces a research tear sheet containing:

• Hypothesis contract and amendments
• Data manifest and bias audit
• Discovery, validation, and final-holdout results
• Gross and net performance
• Statistical intervals and economic significance
• Costs, turnover, liquidity, capacity, and borrow assumptions
• Quantile and attribution diagnostics
• Robustness and parameter sensitivity
• Failed tests and total trial count
• A final Reject, Revise and retest, Monitor / paper trade, or Advance with constraints decision

Important Limitation

A positive backtest does not prove that a security or strategy will perform in the future. Quantest evaluates whether the measurable part of a thesis has historically demonstrated sufficiently robust, implementable, out-of-sample evidence.

License

The skill metadata declares the project under the MIT License.