Evaluate QA GitHub Actions for Cloudbeat handover

[uri-weisman]

Summary

Review the Cloudbeat GitHub Actions that currently support the QA process and map which actions are still needed by the contextual team as part of the handover.

A specific decision point is whether the existing environment-creation flow should remain in the GitHub Action workflow, or whether it should be replaced with a new flow built on currently supported tooling such as the oblt CLI.

Scope

• Inventory the Cloudbeat QA-related GitHub Actions and the steps they perform today.
• Identify which actions should be kept, removed, or replaced.
• If replacement is preferred, outline the recommended approach and supporting tooling.

Acceptance Criteria

• A documented mapping exists for the Cloudbeat QA GitHub Action steps in use today.
• Each action/step is categorized as keep, remove, or replace for the contextual team.
• There is a recommendation on whether to continue using the current environment-creation flow.
• If a new approach is preferred, the issue captures the proposed replacement path and next implementation steps.

[gurevichdmitry]

Workflow overview

#	Workflow file	Display name	Triggers	Primary purpose	Calls other workflows	Handover category
1	test-environment.yml	Create Environment	manual, reusable	Provision an ESS or Serverless stack; install CDR and/or CIS integrations on real cloud/VM/K8s resources (Elastic Agent–based and/or agentless). Optionally run Python integration sanity tests and Kibana UI tests.	—	Split
2	test-e2e-flow.yml	Test E2E Flow	reusable	Parameterized nightly E2E flow: provision an environment with real integration data, run integration and UI sanity tests, then destroy it. Used for both ESS and Serverless.	test-environment.yml, destroy-environment.yml	Deprecate†
3	test-runner.yml	Test Runner	manual, schedule	Nightly orchestrator on main: defines cron schedule, run order, and inputs (ESS first, then Serverless).	test-e2e-flow.yml (×2 jobs)	Deprecate†
4	cdr-infra.yml	Create Environment with Cloud Logs (CDR)	manual	Manual dispatch focused on CDR (cloud logs): provision ESS or Serverless and install CDR integrations with real ingestion on agent-based hosts. Optionally add CIS when cis-infra is enabled.	test-environment.yml	Migrate‡
5	install-integrations.yml	Install Integrations	manual	Attach CDR and/or CIS integrations to an existing stack (no new ELK deploy). Agent-based only (agentless: false).	test-environment.yml	Deprecate
6	upgrade-environment.yml	Test Upgrade Environment	manual	Deploy a base stack with CIS on real agent-based infrastructure → upgrade ELK stack and agents → run post-upgrade integration sanity and UI tests.	test-environment.yml	Deprecate
7	create-env-ea.yml	Create EA Environment	manual	EA / demo path: deploy ELK only and populate with generated demo data (no CDR/CIS on real or agentless collectors).	—	Migrate
8	destroy-environment.yml	Destroy Environment	manual, reusable	Tear down an environment created by Create Environment (manual dispatch or reusable call).	—	Deprecate†
9	destroy-expired-environment.yml	Destroy Expired Environment	schedule	Daily cleanup: find deployments past expiration (S3 config) and destroy each one.	destroy-environment.yml	Deprecate†

Handover legend: Split = different outcome per team · Migrate / Migrate‡ = move ownership or tooling (‡ = tied to Create Environment / CDR decision) · Deprecate = remove when no longer needed · Deprecate† = remove only after upstream decision and cleanup (see dependency graph below).

Handover by team

Categories: Keep · Migrate · Replace (needs POC) · Deprecate · N/A (not used by that team).

Workflow	Cloud Services	Core Analysis	Rationale (summary)
Create Environment (test-environment)	Deprecate manual spin-up; deprecate CIS composites (cis-agentless, cis-agent-based)	Replace CDR path only: provision infra + VMs + CDR integrations	Core reusable workflow. Cloud Services: comprehensive pre-installed integration data was the main value, but CIS testing today is mostly agentless (simpler, faster) — manual full env spin-up is low value. Core Analysis: needs CDR (Terraform infra, agent-based VMs, integration install) — not the CIS stack.
Test E2E Flow	Deprecate† if Create Environment deprecated; Replace test strategy (see notes)	N/A	Runs integration sanity (test_sanity_checks.py, test_sanity_telemetry.py) + Kibana UI FTR. Nightly runs surfaced bugs in Fleet, integrations, and Elastic Agent packaging — rarely Cloudbeat itself.
Test Runner	Deprecate† (same as E2E Flow)	N/A	Scheduled entry point for E2E Flow only.
CDR infra (cdr-infra)	Partial — cloud asset discovery relevance	Migrate‡ — primary CDR entry point	Thin wrapper over Create Environment with infra-type: cdr or all. Follows the same migration/split as Create Environment.
Install Integrations	Deprecate	Deprecate	Edge case: attach integrations to an existing stack. Low usage for both teams.
Upgrade Environment	Deprecate	Deprecate	Used mainly on QA release cycles; high maintenance. Reuses Create Environment + custom upgrade steps.
Create EA Environment	N/A	Migrate	Core Analysis only. Move to dedicated repo; refactor stack provision (e.g. QAF). Generated demo data — no CDR/CIS.
Destroy Environment	Deprecate†	Deprecate†	Required while Create Environment envs exist. After deprecation: clean up all existing deployments, then remove.
Destroy Expired Environment	Deprecate†	Deprecate†	Nightly wrapper over Destroy Environment — remove together.

Additional recommendations (beyond Table Handover by Team)

Area	Suggestion
Post-merge testing (Cloud Services)	Revisit the per–cloud-provider test pipeline. Integration sanity checks (pytest -m sanity) may be faster and sufficient as a post-merge gate vs full multi-CSP nightly provisioning.
UI sanity (Kibana FTR)	Depends on CIS-deployed data (account counts, findings, data relevance). If CIS provisioning is deprecated, UI sanity should be deprecated, moved to Kibana quality-gate pipeline, or replaced with slimmer checks (e.g. ecp-synthetics monitors). Current suite ≈ 15 min (~5 min Kibana bootstrap + ~10 min test run).
Core Analysis — CDR POC options	(1) New repo owning CDR Terraform + integration install. (2) ESTEC-style stack with preinstalled integrations + optional CDR add-on. (3) Hook into Observability-managed clusters (clusters provisioned elsewhere; only add integrations). Each needs a separate POC.
Composite actions (Table 3)	When workflows are removed, orphan review: cis-agentless, cis-agent-based, kibana-ftr likely go with CIS/E2E deprecation; cdr + elk-stack migrate with Core Analysis; slack-notification may remain if other workflows keep it.
Env metadata / S3	Deprecating Create Environment implies retiring S3 env config, expiration tracking (.ci/scripts/get_expired_envs.sh), and related cleanup scripts after one-time env purge.

Handover dependency graph

Several entry-point decisions cascade to downstream workflows. † marks conditional deprecation.

flowchart TD
  subgraph entry["Entry-point decisions"]
    D1["Cloud Services: deprecate Create Environment<br/>(manual spin-up + CIS composites)"]
    D2["Core Analysis: migrate CDR path<br/>(new repo / ESTEC / Observability POC)"]
    D3["Core Analysis: migrate Create EA Environment<br/>(QAF refactor)"]
    D4["Cloud Services: replace nightly test strategy<br/>(integration sanity vs per-CSP pipeline)"]
  end

  TE["test-environment.yml<br/>(Create Environment)"]
  CDR["cdr-infra.yml"]
  II["install-integrations.yml"]
  UP["upgrade-environment.yml"]
  E2E["test-e2e-flow.yml"]
  TR["test-runner.yml"]
  DE["destroy-environment.yml"]
  DEE["destroy-expired-environment.yml"]
  EA["create-env-ea.yml"]

  CIS["cis-agentless / cis-agent-based<br/>(composite actions)"]
  KFTR["kibana-ftr<br/>(UI sanity)"]
  SAN["Integration sanity tests<br/>(test_sanity_checks, test_sanity_telemetry)"]

  D1 --> TE
  D2 --> TE
  D2 --> CDR

  TE --> CIS
  TE --> CDR
  CDR --> TE
  II --> TE
  UP --> TE

  D1 -->|"deprecate CIS"| CIS
  D1 -->|"if CIS gone"| KFTR

  E2E --> TE
  E2E --> DE
  TR --> E2E

  D1 -->|"if TE deprecated"| E2E
  D1 -->|"if TE deprecated"| TR
  D4 --> SAN

  UP -->|"also uses"| KFTR
  UP -->|"also uses"| SAN

  DE --> DEE
  D1 -->|"cleanup envs then"| DE
  D1 -->|"then"| DEE

  D3 --> EA

  classDef entry fill:#e8f4fc,stroke:#0969da
  classDef deprecate fill:#fff1f0,stroke:#cf222e
  classDef migrate fill:#fff8e6,stroke:#bf8700
  classDef replace fill:#f0f6ff,stroke:#6366f1

  class D1,D2,D3,D4 entry
  class E2E,TR,II,UP,DE,DEE,CIS,KFTR deprecate
  class CDR,EA migrate
  class SAN replace

Loading

Cascade summary

If this is decided…	…these become irrelevant or must change
Deprecate Create Environment (Cloud Services path)	cdr-infra (CDR-only manual dispatch), install-integrations, upgrade-environment, test-e2e-flow, test-runner — all call or assume test-environment.
Deprecate CIS composites inside Create Environment	cis-agentless, cis-agent-based; Kibana UI sanity (data-dependent); CIS portions of integration sanity if no alternative env supplies findings.
Deprecate Create Environment entirely	Run one-time cleanup of all deployed envs → then deprecate destroy-environment and destroy-expired-environment.
Migrate CDR (Core Analysis)	Extract cdr composite + Terraform/VM provisioning; cdr-infra and CDR branch of test-environment move together.
Migrate Create EA Environment	Independent of Cloud Services QA stack; only affects Core Analysis demo/EA workflows and elk-stack usage in that path.
Replace nightly E2E with integration sanity only	Keep tests/integration/tests/test_sanity_checks.py + test_sanity_telemetry.py in a lighter CI job (requires a env that still produces findings/telemetry — agentless CIS or shared long-lived stack TBD).

[gurevichdmitry]

FYI, @uri-weisman and @olegsu : I’ve also scheduled a meeting to discuss this topic.