Add kind-deployment validation to CI pipeline#1359
Conversation
| ## | ||
|
|
||
| # ── install gh CLI and jq ─────────────────────────────────────────── | ||
| apk add --no-cache --quiet \ |
There was a problem hiding this comment.
CLI tools could be added to the cf-deployment-concourse-tasks image:
https://github.com/cloudfoundry/cf-deployment-concourse-tasks/blob/f5e2b103800315eac9fa6ba3e7b6c6054e17816c/ci/pipeline.yml#L12
| @@ -0,0 +1,17 @@ | |||
| --- | |||
There was a problem hiding this comment.
Special Concourse tasks can be stored in the runtime-ci repository:
https://github.com/cloudfoundry/runtime-ci
| --- | ||
| platform: linux | ||
| image_resource: | ||
| type: docker-image |
There was a problem hiding this comment.
You can re-use the cf-deployment-concourse-tasks image here.
| The existing daily sync in kind-deployment is **batch and delayed**: it only | ||
| picks up versions after they appear in a cf-deployment *release* (tagged), | ||
| which can be days or weeks after they land on `develop`. There is no | ||
| automated path that validates whether the versions currently being tested in | ||
| BOSH environments also work on Kubernetes. |
There was a problem hiding this comment.
Probably I haven't understood it, but was is the gap? The main in kind-deployment uses the versions validated by cf-depoyment. So every user get working version.
For validation, we introduced the parameter fresh validation which then uses the versions from the develop branch in cf-deployment instead.
The helm charts and docker files are automatically created when a new bosh release (capi, diego, ...) is created. Independently if cf-deployment is already using the version or not yet.
If you still see a gap, please let me know that we can work on closing it :)
There was a problem hiding this comment.
You're right — "gap" was misleading. kind-deployment's main already uses validated versions. Updated the doc to remove the "gap" framing.
| - task: trigger-kind-smoke-workflow | ||
| file: runtime-ci/tasks/trigger-github-workflow/task.yml | ||
| params: | ||
| GITHUB_TOKEN: ((ard_wg_gitbot_token)) |
There was a problem hiding this comment.
I created a dedicated token with the exact permissions you need. Please check the secret manager of the wg.
| ### 1. Missing container images | ||
|
|
||
| When cf-deployment `develop` has a new release version, the corresponding | ||
| container image may not yet be built and pushed to `ghcr.io`. The sync |
There was a problem hiding this comment.
It is not really a sync script checking for images and helm charts. What happens is this:
- A new github release is created for a bos release repo (e.g. capi-release)
renovatewill bump the version incf-k8s-releases(once a day)- The PR from renovate will be verified and automarged if successful
- In
main, helm chart and docker images will be created and published
There can be 2 cases in which some artifact is missing:
cf-deploymentis too fast.renovateruns once a day and verification/release might take 1-2 hours.- The artifact could not be created automatically (e.g. verification failed)
|
|
||
| ### 2. Concourse-to-GitHub API reliability | ||
|
|
||
| The trigger-github-workflow task depends on the GitHub API being available |
There was a problem hiding this comment.
At least rate-limit shouldn't be a problem since you use a token.
There was a problem hiding this comment.
Agreed, removed that concern from the doc since we use an authenticated token.
| both durations. If the pipeline triggers frequently, kind validation may | ||
| fall behind BOSH. | ||
|
|
||
| **Mitigation:** Because smoke and CATS create independent kind clusters, |
There was a problem hiding this comment.
An other mitigation would be to only use cats. I think it cannot happen that smoke tests find an error that cats doesn't find, right? It finds it much faster of course.
There was a problem hiding this comment.
Well that depends on the CATs and smoke-tests configuration. smoke-tests could find issues if tests are enabled that are disabled in CATs.
| the in-workflow sync, so the test could validate a slightly newer or older | ||
| set of versions than the exact commit that triggered the job. | ||
|
|
||
| **Mitigation:** Acceptable — the kind check is informational and |
There was a problem hiding this comment.
Another mitigation would be that we add a parameter to receive the exact commit sha that should be used. If that ever turn out to be a problem.
3 participants
Please take a moment to review the questions before submitting the PR
🚫 We only accept PRs to develop branch. If this is an exception, please specify why 🚫
This PR targets develop. ✅
WHAT is this change about?
Adds Cloud Foundry on Kubernetes (kind) validation to the cf-deployment Concourse pipeline. Two new jobs — kind-smoke-tests and kind-cats — dispatch the existing kind-smoke.yaml and kind-cats.yaml GitHub Actions workflows in cloudfoundry/kind-deployment with the fresh-validation: true input. That input (added in kind-deployment#424, already merged) makes the workflow sync its Helm chart versions from cf-deployment's develop branch before deploying, so the Kubernetes deployment is tested with the same release versions currently being validated on BOSH.
The reusable trigger-github-workflow task has been moved to runtime-ci#687 so it can be shared across pipelines. The gh CLI required by the task is provided by the cloudfoundry/cf-deployment-concourse-tasks image — see cf-deployment-concourse-tasks#249 which adds it to the image.
What customer problem is being addressed? Use customer persona to define the problem e.g. Alana is unable to...
Alana, a CF release engineer, is unable to know whether the component versions on develop — already validated on BOSH — also deploy and pass acceptance tests on Kubernetes. Today kind-deployment only syncs from tagged cf-deployment releases, which lag develop by days or weeks, so there is no early signal that a version bump breaks the Kubernetes path. This change gives that signal at the same time BOSH validation runs.
Please provide any contextual information.
The kind jobs trigger on the same gate as every BOSH environment (unit-test-ops-files, lint-cf-deployment-manifest, unit-test-golang-tests, unit-test-update-releases-coverage), so they run in parallel with fresh/upgrade/experimental/windows/lite/FIPS.
They are grouped under a new kind-neville pipeline group (Harry Potter naming convention).
The version sync now happens inside the kind workflows via fresh-validation; no separate sync job or PR branch is needed.
The trigger-github-workflow task lives in runtime-ci#687 and is referenced via the existing runtime-ci resource. It uses the cloudfoundry/cf-deployment-concourse-tasks image (which gains the gh CLI via cf-deployment-concourse-tasks#249 .
The jobs are non-blocking — bless-manifest does not depend on them, so a kind failure will not gate releases while the Kubernetes path matures.
Has a cf-deployment including this change passed cf-acceptance-tests?
Does this PR introduce a breaking change? Please take a moment to read through the examples before answering the question.
How should this change be described in cf-deployment release notes?
Not release-noteworthy — internal CI change with no operator-facing impact. (Suggested, if included: "CI now validates develop versions on Kubernetes via kind-deployment, in parallel with BOSH.")
Does this PR introduce a new BOSH release into the base cf-deployment.yml manifest or any ops-files?
Does this PR make a change to an experimental or GA'd feature/component?
Please provide Acceptance Criteria for this change?
What is the level of urgency for publishing this change?
Tag your pair, your PM, and/or team!
@oliver-heinrich @jochenehret