Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
02b6c44
initital setup (Copied over files from FD)
47thomasj Mar 30, 2026
ae0e63a
first draft
47thomasj Mar 30, 2026
f8317f0
more setup stuff
47thomasj Mar 30, 2026
e43af58
stuff
47thomasj Mar 30, 2026
cb84101
fixed path thing
47thomasj Mar 30, 2026
efc436f
set up stuff
47thomasj Mar 30, 2026
436dcb5
first round of tests pass!
47thomasj Mar 30, 2026
9c54fa8
added executor compatibility for python
47thomasj Apr 1, 2026
77a3121
added executor
47thomasj Apr 1, 2026
39c8fc4
fixed implimentations
47thomasj Apr 1, 2026
bcf0cfd
everything passes
47thomasj Apr 1, 2026
c943d5a
significantly simplified runner
47thomasj Apr 1, 2026
535f26b
validator implimentation
47thomasj Apr 1, 2026
96f556a
simplified test bridge add method calls
47thomasj Apr 1, 2026
2b577e0
Final tests fixed
47thomasj Apr 1, 2026
accf3eb
final clreaning
47thomasj Apr 1, 2026
46977a0
yaml
47thomasj Apr 1, 2026
fa34ddf
no dependancies
47thomasj Apr 1, 2026
75164d5
delete things
47thomasj Apr 1, 2026
f07fc26
more
47thomasj Apr 1, 2026
951789e
moved src files and created deploy script
47thomasj Apr 3, 2026
59e78de
deploy scripts
47thomasj Apr 3, 2026
332fc1b
changed node version
47thomasj Apr 3, 2026
60a3010
docs and simplify
47thomasj Apr 3, 2026
b2fdfd5
Merge pull request #1 from byuawsfhtl/setup
epbay01 Apr 3, 2026
d5de07f
outlined test files and made simple classes/functions to test
47thomasj Apr 3, 2026
acc53a7
added tests
47thomasj Apr 3, 2026
0ee4fe3
moved to one node system
47thomasj Apr 3, 2026
84968e0
first tests pass
47thomasj Apr 3, 2026
0d4138a
changed some tests for better coverage
47thomasj Apr 3, 2026
a3f81f1
more tests
47thomasj Apr 3, 2026
0b59764
deleted mocking
47thomasj Apr 6, 2026
2386bf6
coverage ignore
47thomasj Apr 6, 2026
8d7bb07
yaml changes
47thomasj Apr 6, 2026
51344f1
renamed to they actually run.
47thomasj Apr 6, 2026
35068c1
standard stuff
47thomasj Apr 6, 2026
f1b26c5
mor tests
47thomasj Apr 6, 2026
0dd90ae
standard
47thomasj Apr 6, 2026
b88849f
small test fix
47thomasj Apr 6, 2026
2dfe5fb
run build
47thomasj Apr 6, 2026
a28c4f6
Update README.md for improved formatting and clarity in function desc…
47thomasj Apr 6, 2026
662839d
Merge pull request #2 from byuawsfhtl/tests-and-mocks
epbay01 Apr 6, 2026
8d39485
version
47thomasj Apr 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .coverage
Binary file not shown.
3 changes: 3 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[run]
source = source
omit = tests/*
56 changes: 56 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: CI

on:
pull_request:
branches: [prd, stg, dev]
paths-ignore:
- "**/README.md"
- "**/.gitignore"
- "**/docs/*"

jobs:
checkMeds:
name: Check Meds (merge every day)
runs-on: ubuntu-latest
steps:
- name: Check Meds
uses: byuawsfhtl/MedsAction@v1.0.0

standardCheck:
name: Python Standard Check
runs-on: ubuntu-latest
steps:
- name: Follow Python Standard
uses: byuawsfhtl/PythonStandardAction@v1.2.0

checkTests:
name: Test Coverage Check
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v4
with:
token: ${{ secrets.RLL_BOT_PA_TOKEN }}

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.12'

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: npm

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e .
npm ci
npm run build

- name: Check Test Coverage
uses: byuawsfhtl/TestCoverageAction@v1.0.0
with:
exclude_paths: 'tests/*'
133 changes: 133 additions & 0 deletions .github/workflows/deploy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
name: Deploy

on:
push:
branches: [prd, dev]
paths-ignore: # Pushes that include only these changed files won't trigger actions
- "**/README.md"
- "**/.gitignore"
- "**/docs/*"
- "**/.github/*"
- "**/tests/*"
- "**/_version.py"

jobs:
update-version:
name: Update Version
runs-on: ubuntu-latest
permissions: write-all
steps:
- name: Update Version
uses: byuawsfhtl/UpdateVersion@v1.0.9
with:
token: ${{ secrets.RLL_BOT_PA_TOKEN }}
versionPath: ${{ github.workspace }}/_version.py

build-python:
name: Build Python distribution 📦
if: github.ref == 'refs/heads/prd' # Only publish when pushing to prd
needs: update-version
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }} # Explicitly checkout the branch that triggered the workflow
# This ensures we're using the latest commit including version updates
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.x"
- name: Install pypa/build
run: python3 -m pip install build
- name: Build a binary wheel and a source tarball
run: python3 -m build
- name: Store the distribution packages
uses: actions/upload-artifact@v4
with:
name: python-package-distributions
path: dist/

publish-to-pypi:
name: Publish Python 🐍 distribution 📦 to PyPI
if: github.ref == 'refs/heads/prd' # Only publish when pushing to prd
needs: build-python
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/PyScriptTestUtils
permissions:
id-token: write

steps:
- uses: actions/checkout@v4 # Needed to ensure repo context
- name: Download all the dists
uses: actions/download-artifact@v4
with:
name: python-package-distributions
path: dist/
- name: Verify dist exists
run: ls -l dist/
- name: Publish distribution 📦 to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.PYPI_ACCESS_TOKEN }}

build-typescript:
name: Build TypeScript distribution 📦
if: github.ref == 'refs/heads/prd' # Only publish when pushing to prd
needs: update-version
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22.x'
registry-url: 'https://registry.npmjs.org'
- name: Install dependencies
run: npm ci
- name: Sync version from _version.py to package.json
run: |
VERSION=$(python3 -c "exec(open('_version.py').read()); print(__version__)")
npm version "$VERSION" --no-git-tag-version --allow-same-version
- name: Build TypeScript
run: npm run build
- name: Store the TypeScript distribution
uses: actions/upload-artifact@v4
with:
name: typescript-package-distributions
path: |
dist
package.json
package-lock.json
README.md

publish-to-npm:
name: Publish TypeScript 📘 distribution 📦 to npm
if: github.ref == 'refs/heads/prd' # Only publish when pushing to prd
needs: build-typescript
runs-on: ubuntu-latest

steps:
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '18.x'
registry-url: 'https://registry.npmjs.org'
- name: Download TypeScript distribution
uses: actions/download-artifact@v4
with:
name: typescript-package-distributions
path: npm-publish
- name: Verify package layout
working-directory: npm-publish
run: ls -la && ls -l dist/
- name: Publish to npm
working-directory: npm-publish
run: npm publish
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} # Token will expire every 90 days. Will have to update this token periodically.
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
node_modules/
.vscode/
dist/
build/
*.egg-info/
__pycache__/
*.py[cod]
1 change: 1 addition & 0 deletions .standardignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
tests/*
153 changes: 152 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,153 @@
# PyScriptTestUtils
A package that runs pytests concurrently in both python and TypeScript to enforce equal behavior between packages that support both languages

A package that runs pytest-style checks concurrently in Python and TypeScript so dual-language libraries can enforce the same behavior in both implementations.

## Architecture

- **`PyScriptTestRunner`** (Python): register one **named** Python function per operation with the TypeScript RPC name your Node bridge expects, then call `run` with the same `test_data` shape as before.
- **`PyScriptTestBridge`** (TypeScript): register handlers with `addMethod(tsName, (args) => response)`. The compiled **`test_bridge_entry.js`** is invoked by the runner via `node`; it parses one JSON request from argv and prints one JSON response.

## Python: registration and `run`

```python
runner = PyScriptTestRunner(
"Path/to/built/test/bridge.js",
(Optional) serializer = function to serialize objects into consistent Json-like structures,
(Optional) deserializer = function to deserialize objects into an expected custom class.
)

def create_flexible_date(arg1, arg2, ...) -> FlexibleDate:
some code...
return flexible_date_object

runner.add_method(create_flexible_date, "createFlexibleDate", (Optional) executor=lambda args: create_flexible_date(args[0], args[1], ...))
runner.add_method(combine_flexible_dates, "combineFlexibleDates", (Optional) ts_pack_input=True)
# ...

py_result, ts_result = runner.run(
"create_flexible_date",
"createFlexibleDate",
test_data,
)
```

Rules:

- **`add_method(py_callable, ts_method_name, *, ts_pack_input=False)`**
- `path/to/brige` must be the path to the **built** dist of the ts bridge, usually within a dist/ dir. Ex: `Path(__file__).resolve().parent() / "dist" / bridge.js`
- `py_callable` must be a **named** function (not a lambda). The registry key is `py_callable.__name__` (what you pass as the first argument to `run`).
- `ts_method_name` must match `addMethod` on the TS side and the JSON `method` field.
- `executor` is some exeutable that processes the test data if neccesary.
- `ts_pack_input=True`: for this operation the runner sends `args: [input_data]` to Node (single array argument). Use this for TS handlers that expect one aggregate argument (for example `combineFlexibleDates` with `const [datesData] = args`).
- **`run(python_name, ts_name, test_data)`** checks that `ts_name` matches the name registered for `python_name`.
- By default, registered Python callables receive a **single** argument: `test_data["input"]`, and returns the (optionally) serialized result of running the callable on the argument. If more arguments are needed, or if other post-processing on the output is needed, provide a test executor.
- Serializers and deserializers are optional. If a function is meant to return or take in a custom class, the runner must be provided with (de)serialization function(s), otherwise the data will be treated as raw types (bool, int, float, str, etc...). By default, the deserializer is capable of deserializing lists, so the provided deserialization function only needs to support deserialiation into the given class. The serializer does not support this.

## TypeScript: `test_bridge.ts`

The test bridge contains all that information neccessary for the python runner to call the TS functions under test. It is instantiated with optional serializer and deserializer parameters like the runner.

**Example:**

```ts
function serializeFlexibleDate(fd: FlexibleDate): any {
if (fd.constructor.name === "FlexibleDate") {
some code...
return jsonLikeObject;
}
return fd
}

function deserializeFlexibleDate(data: any): FlexibleDate {
if (can serialize to FlexibleDate...) {
return new FlexibleDate(serialization logic...);
}
return data;
}

const bridge = new PyScriptTestBridge(serializeFlexibleDate, deserializeFlexibleDate);

bridge.addMethod("createFlexibleDate", (args) => new FlexibleDate(args[0]));
```

Rules:

**`addMethod("TSFunctionName, exector)`**
- `TSFunctionName` must exactly match an `add_method()` entry on the cooresponding test runner. This is how the runner knows which function to run within the bridge.
- `executor` must be the test executor function which runs the function under test. Unlike the runner, the bridge has no default executor, and so an executor must be provided with each `addMethod()` call.

**Serializer and Deserializer**
- Unlike the runner, the bridge has no automatic serialization handling. This is a result of differences between Python and TypeScript runtime behavior. Thus, the (de)serialization functions must recognize whether the objects being passed into them can be (de)serialized as intended.
- Alternatively, it is possible to have multiple bridge instances/files with and without (de)serializers as needed, though this is not recommended unless neccessary.

## RPC argument list (`args`)

The subprocess request is `{ "method": string, "args": any[], "mocks": object }`.

- For most operations the runner sets **`args`** from `test_data["input"]` as:
- `[input_data]` when `input_data` is not a `list`,
- or **`input_data` as-is** when it is already a `list`.
- When **`ts_pack_input=True`**, the runner always sends **`args: [input_data]`** (one element), so the TS handler uses `const [x] = args` (for example a list of serialized dates).

Align Python `input_data` in tests with this contract so both sides see the same logical inputs.

## Test Examples

```python
class TestIdenticalDates:
"""Test comparison of identical dates returns perfect score of 100."""

test_cases = [
{
"input": [
{"likelyYear": 2020, "likelyMonth": 1, "likelyDay": 15},
{"likelyYear": 2020, "likelyMonth": 1, "likelyDay": 15}
],
"expected": 100,
"description": "identical full dates"
},
{
"input": [
{"likelyYear": 2020, "likelyMonth": 5, "likelyDay": None},
{"likelyYear": 2020, "likelyMonth": 5, "likelyDay": None}
],
"expected": 100,
"description": "identical year-month dates"
},
{
"input": [
{"likelyYear": 1995, "likelyMonth": None, "likelyDay": None},
{"likelyYear": 1995, "likelyMonth": None, "likelyDay": None}
],
"expected": 100,
"description": "identical year-only dates"
}
]

@pytest.mark.parametrize("test_case", test_cases, ids=lambda x: x['description'])
def test_identical_full_dates(self, test_case):
test_data = {"input": test_case["input"], "expected": test_case["expected"], "mocks": {}}
py_result, ts_result = runner.run(
"compare_two_dates",
"compareDates",
test_data
)
assert py_result == test_case["expected"], f"Python failed for {test_case['description']}"
assert ts_result == test_case["expected"], f"TypeScript failed for {test_case['description']}"
runner.assert_strict_parity(py_result, ts_result, test_case['description'])
```

## Notes

- When testing construction of custom classes, the runner will attempt to deserialize the TS result via the provided deserializer function. This means that expected test results **can** be custom classes.
- When testing custom class methods, the runner cannot deserialize those custom classes, meaning that the input test-data must be provided pre-serialized.


## Developing

```bash
npm ci
npm run build
```

Requires Node.js and npm.
Empty file added _version.py
Empty file.
Loading
Loading