Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/cla.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,10 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
path-to-signatures: "signatures/cla.json"
path-to-document: "https://github.com/bkd-dotcom/umbra-eval/blob/main/CLA.md"
path-to-document: "https://github.com/Signetry/eval/blob/main/CLA.md"
branch: "cla-signatures"
allowlist: "dependabot[bot],bkd-dotcom"
custom-notsigned-prcomment: "Thanks for your contribution! Before it can be merged, please read the **[Contributor License Agreement](https://github.com/bkd-dotcom/umbra-eval/blob/main/CLA.md)** — Umbra is **All Rights Reserved**, and by signing you assign copyright/ownership of your contribution to the Owner (you may not use, sell, or commercialize it yourself). To agree, reply with exactly:"
custom-notsigned-prcomment: "Thanks for your contribution! Before it can be merged, please read the **[Contributor License Agreement](https://github.com/Signetry/eval/blob/main/CLA.md)** — Umbra is **All Rights Reserved**, and by signing you assign copyright/ownership of your contribution to the Owner (you may not use, sell, or commercialize it yourself). To agree, reply with exactly:"
custom-pr-sign-comment: "I have read the CLA Document and I hereby sign the CLA"
custom-allsigned-prcomment: "All contributors have signed the CLA. ✅"
lock-pullrequest-aftermerge: false
4 changes: 2 additions & 2 deletions .github/workflows/reviewer.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Reviewer
# Advisory PR review (umbra-reviewer): surfaces architecture + security issues
# and posts one recommendation comment. Advisory only — it never merges and never
# fails the PR. Changes to security-sensitive surfaces (workflows, packaging) are
# escalated to a human. See https://github.com/bkd-dotcom/umbra-reviewer
# escalated to a human. See https://github.com/Signetry/reviewer
on:
pull_request:

Expand All @@ -24,7 +24,7 @@ jobs:
python-version: "3.12"
- name: Install umbra-reviewer
# source-available (All Rights Reserved); install from source, not PyPI.
run: pip install "umbra-reviewer @ git+https://github.com/bkd-dotcom/umbra-reviewer@v0.1.1"
run: pip install "umbra-reviewer @ git+https://github.com/Signetry/reviewer@v0.1.1"
- name: Compute the PR diff
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Until `1.0.0` the public API may change between minor versions.

### Changed

- Install `umbra-core` from its **source repository** (`git+https://github.com/bkd-dotcom/umbra-core@v0.5.4`)
- Install `umbra-core` from its **source repository** (`git+https://github.com/Signetry/core@v0.5.4`)
instead of PyPI — umbra-core is All Rights Reserved and no longer distributed on
PyPI. Enables `tool.hatch.metadata.allow-direct-references`.

Expand Down
2 changes: 1 addition & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ an individual is officially representing the community in public spaces.

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the maintainers via
[GitHub private vulnerability reporting](https://github.com/bkd-dotcom/umbra-eval/security/advisories/new)
[GitHub private vulnerability reporting](https://github.com/Signetry/eval/security/advisories/new)
or by opening a confidential issue. All complaints will be reviewed and
investigated promptly and fairly.

Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@
> This repository is strictly for viewing and contributing to the original project. You may not use, copy, modify, distribute, or commercialize this code for your own personal or commercial projects without explicit written permission. Only the original author retains the right to use and monetize this project.


**The public adversarial evaluation suite for [Umbra](https://github.com/bkd-dotcom/umbra-umbrella).**
**The public adversarial evaluation suite for [Umbra](https://github.com/Signetry/signetry).**

Measures **attack success rate (ASR)** and **utility-under-defense** for coding-agent
threats — governed by the same [`umbra-core`](https://github.com/bkd-dotcom/umbra-core)
threats — governed by the same [`umbra-core`](https://github.com/Signetry/core)
pipeline production uses.

[![Source-available](https://img.shields.io/badge/source-available-informational.svg)](CLA.md)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome%20(CLA)-brightgreen.svg)](https://github.com/bkd-dotcom/umbra-umbrella/issues/10)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome%20(CLA)-brightgreen.svg)](https://github.com/Signetry/signetry/issues/10)

</div>

Expand Down Expand Up @@ -125,14 +125,14 @@ print(report.overall()) # asr_ungoverned, asr_governed, utility_governe

## Contributing

**Source-available, PRs welcome** (not open source; All Rights Reserved). Contribute under the [CLA](CLA.md) — you're **credited** ([CONTRIBUTORS.md](CONTRIBUTORS.md)) but gain no ownership or right to use/sell it. Start at the [good-first-issues board](https://github.com/bkd-dotcom/umbra-umbrella/issues/10). The best contribution is **a new test case**. Add a detection
**Source-available, PRs welcome** (not open source; All Rights Reserved). Contribute under the [CLA](CLA.md) — you're **credited** ([CONTRIBUTORS.md](CONTRIBUTORS.md)) but gain no ownership or right to use/sell it. Start at the [good-first-issues board](https://github.com/Signetry/signetry/issues/10). The best contribution is **a new test case**. Add a detection
**corpus case** (`umbra_eval/detection/corpus/`, with a cited `provenance`; SAFE
decoys must stay 0 false positives) or an **adversarial scenario**
(`umbra_eval/scenarios/`). See [CONTRIBUTING.md](CONTRIBUTING.md) and
[CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md). Every case ships with a test; CI gates the
benchmark on 100% recall / 0 FP.

Part of the Umbra platform — see the [umbrella overview](https://github.com/bkd-dotcom/umbra-umbrella).
Part of the Umbra platform — see the [umbrella overview](https://github.com/Signetry/signetry).

## License

Expand Down
10 changes: 5 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,14 @@ classifiers = [
"Topic :: Software Development :: Quality Assurance",
]
dependencies = [
"umbra-core @ git+https://github.com/bkd-dotcom/umbra-core@v0.5.4",
"umbra-core @ git+https://github.com/Signetry/core@v0.5.4",
]

[project.urls]
Homepage = "https://github.com/bkd-dotcom/umbra-eval"
Repository = "https://github.com/bkd-dotcom/umbra-eval"
Issues = "https://github.com/bkd-dotcom/umbra-eval/issues"
Umbrella = "https://github.com/bkd-dotcom/umbra-umbrella"
Homepage = "https://github.com/Signetry/eval"
Repository = "https://github.com/Signetry/eval"
Issues = "https://github.com/Signetry/eval/issues"
Umbrella = "https://github.com/Signetry/signetry"

[project.scripts]
umbra-eval = "umbra_eval.cli:main"
Expand Down