Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .agents/skills/e2e-test/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,39 @@ external resource (the same workflow files in a fixture repository, say) therefo
no external locking or concurrency group needed. Verify assumptions like this against the OpenTofu
version in use rather than trusting them.

### The exception: a test whose subject *is* a change

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

d: I am not sure how helpful this text below is. I find its rather hard to read and understand. And basically only tells us that an output can change if a tag value somewhere else changes... I can also remove it if you think its nonesense here?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, remove it, it doesn't add much value. you could move such details as README.md to the e2e/tag-inputs folder or split it up as succinct comments for the tf files?


Some behaviour only exists as a change to something that already exists. A `TAG` input is the
example: meshStack resolves it from a tag on the workspace, project, payment method or landing zone,
and re-resolves it when that tag changes or when the object it reads from is reassigned. A fresh
building block per value proves resolution and nothing else — the follow-the-metadata behaviour needs
one building block and a tag that moves under it. That takes shared state, so it takes one file.

Neither reason above applies to such a test, and saying why is what keeps the exception narrow:

- It never touches `display_name`, so the rename-without-a-run trap is not in play.
- It never touches `building_block_definition_version_ref`, so the released-version-only upgrade path
is not in play either. The building block's own `spec` identity is fixed for the whole file; only
the objects *around* it change.

Two things such a file has to get right, both learned the hard way:

- **Bump an input the provider tracks in every mutating run.** A TAG input is resolved server-side,
so nothing in the configuration tells the provider a tag moved — it plans no change and waits for
nothing. Bumping a `USER_INPUT` makes it issue an update and await a run, and that run resolves the
current tag values. Be explicit in the file about what this does and does not prove: the value a
tag holds now reaches the next run; *not* that the tag edit alone triggered a run. The provider
surfaces no way to attribute a run to an edit.
- **Let meshStack's own run get out of the way first.** meshStack triggers a run when a tag a
building block reads changes. That run competes with the one the test triggers: `awaitRun` polls
the block's aggregate status and stops at the first terminal one, so the wrong run's outputs can
land in state, and meshStack rejects an update issued while a run is in flight. A `time_sleep`
between the tag writes and the building block, replaced per scenario via `triggers`, is enough —
and it belongs in the fixture module, not in a polling script.

`modules/meshstack/noop/e2e/tests/building_block_noop_tag_inputs_hub.tftest.hcl` is the worked
example. If a test does not clear this bar, use separate files.

---

## Running tests
Expand Down Expand Up @@ -484,6 +517,7 @@ source setup-override-provider.sh
- [ ] tftest asserts `status.status == "SUCCEEDED"` and key outputs (references `var.test_context.*` directly — non-null in both modes)
- [ ] Variant flags (sync/async and similar) are **root variables of the `e2e/` module** with a default, not `test_context` fields
- [ ] One `.tftest.hcl` file per variant, pinning the flag in a file-level `variables` block — never several `run` blocks sharing one file's state
- [ ] …unless the subject of the test *is* a change to a live building block's surroundings — then one file with shared state, and see [The exception](#the-exception-a-test-whose-subject-is-a-change) for the two things it must get right
Comment on lines 519 to +520

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f: merge those two into one clear instruction and not into "do this ... unless"
or just relax the requirement of "one tftest.hcl per variant"?

- [ ] Writes into a long-lived shared fixture go to a per-run ephemeral slice named from `name_suffix`, owned by the `e2e/` module and included in the building block's `depends_on`
- [ ] State the live apply cannot reach is covered by a mocked `<cloud>_<service>_unit.tftest.hcl` in `e2e/tests/`, targeting `module { source = "../buildingblock" }` — and only where the two bars are cleared (worth testing, unreachable by the apply); anything the apply *can* reach is an assertion on the apply instead
- [ ] Every mocked run is mutation-checked: break the module, watch the run fail
10 changes: 10 additions & 0 deletions modules/meshstack/noop/buildingblock/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,13 @@ Use it to:
| `multi_select_json` | `MULTI_SELECT` | `USER_INPUT` | Same as above, as a raw JSON string |
| `some-file.yaml` | `FILE` | `STATIC` | Written to working directory; read via `file("some-file.yaml")` |
| `sensitive-file.yaml` | `FILE` | `STATIC` (sensitive) | Like above, encrypted at rest |
| `project_tag` | `CODE` | `TAG` | Values of a meshProject tag, resolved by meshStack — `null` when the tag holds no value |
| `payment_method_tag` | `CODE` | `TAG` | Values of a meshPaymentMethod tag on the project's payment method |
| `landing_zone_tag` | `CODE` | `TAG` | Values of a meshLandingZone tag on the tenant's landing zone |

The three `TAG` inputs are declared by the **tenant-level** definition only. A `TAG` input can
read a workspace, project, payment method or landing zone tag, and a workspace-level definition
has no project — so only a tenant-level one can reach the latter three.

### How FILE Inputs Work

Expand Down Expand Up @@ -69,10 +76,13 @@ No modules.
| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_flag"></a> [flag](#input\_flag) | n/a | `bool` | n/a | yes |
| <a name="input_landing_zone_tag"></a> [landing\_zone\_tag](#input\_landing\_zone\_tag) | n/a | `list(string)` | `null` | no |
| <a name="input_multi_select"></a> [multi\_select](#input\_multi\_select) | n/a | `list(string)` | n/a | yes |
| <a name="input_multi_select_json"></a> [multi\_select\_json](#input\_multi\_select\_json) | n/a | `string` | n/a | yes |
| <a name="input_num"></a> [num](#input\_num) | n/a | `number` | n/a | yes |
| <a name="input_optional_text"></a> [optional\_text](#input\_optional\_text) | n/a | `string` | `"tf-default-value"` | no |
| <a name="input_payment_method_tag"></a> [payment\_method\_tag](#input\_payment\_method\_tag) | n/a | `list(string)` | `null` | no |
| <a name="input_project_tag"></a> [project\_tag](#input\_project\_tag) | n/a | `list(string)` | `null` | no |
| <a name="input_sensitive_text"></a> [sensitive\_text](#input\_sensitive\_text) | n/a | `string` | n/a | yes |
| <a name="input_sensitive_yaml"></a> [sensitive\_yaml](#input\_sensitive\_yaml) | n/a | `any` | n/a | yes |
| <a name="input_single_select"></a> [single\_select](#input\_single\_select) | n/a | `string` | n/a | yes |
Expand Down
3 changes: 3 additions & 0 deletions modules/meshstack/noop/buildingblock/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,9 @@ output "debug_input_variables_json" {
multi_select_json = var.multi_select_json
static = var.static
static_code = var.static_code
project_tag = var.project_tag
payment_method_tag = var.payment_method_tag
landing_zone_tag = var.landing_zone_tag
user_permissions = var.user_permissions
user_permissions_json = var.user_permissions_json
})
Expand Down
19 changes: 19 additions & 0 deletions modules/meshstack/noop/buildingblock/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,22 @@ variable "multi_select" {
variable "multi_select_json" {
type = string
}

# A TAG input resolves to the tag's values, and to null when the tag holds no value on the object it
# is read from — an absent key and an empty one both mean unset. Every TAG input is therefore a
# nullable list, whichever meshObject it reads from.

variable "project_tag" {
type = list(string)
default = null

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default = null is redundant afaik, as nullable = true, or is terraform issuing a warning here?

concerning best practice, and to avoid nullability issues, maybe a default = [] is better here?

}

variable "payment_method_tag" {
type = list(string)
default = null
}

variable "landing_zone_tag" {
type = list(string)
default = null
}
Loading
Loading