Skip to content

Grafana logs, alerting, and dashboards#130

Open
imol-ai wants to merge 2 commits into
mainfrom
feat/grafana-logs
Open

Grafana logs, alerting, and dashboards#130
imol-ai wants to merge 2 commits into
mainfrom
feat/grafana-logs

Conversation

@imol-ai

@imol-ai imol-ai commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds collection of logs to grafana, some dashboards, and alerting.

Closes #120

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Infrastructure / CI
  • Documentation

API changes

  • This PR does not affect the API
  • This PR changes the API → api/openapi.yaml updated and api/scripts/gen-all.sh re-run

Definition of Done

  • CI passes
  • Pre-commit hooks pass locally
  • Relevant tests added or updated

Summary by CodeRabbit

  • New Features
    • Added Loki and Grafana Alloy to centralize application logs.
    • Added new Grafana dashboards and configured Loki as a Grafana data source.
    • Enabled Grafana alerting for service errors, latency, and availability with Discord notifications.
    • Added HTTP request percentile metrics via Spring Actuator for deeper performance visibility.
  • Improvements
    • Upgraded and refreshed Grafana’s monitoring setup.
    • Updated Grafana access (new host/path with TLS) and the “Try it out!” Grafana link.

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@imol-ai, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 45 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 535edddb-f5a0-43de-8605-0c7ff3fa2fe5

📥 Commits

Reviewing files that changed from the base of the PR and between c76c5d1 and e3cd18c.

📒 Files selected for processing (5)
  • .github/workflows/deploy-k8s.yml
  • infra/k8s/monitoring/grafana-alerting.yaml
  • infra/k8s/monitoring/grafana-dashboards.yaml
  • infra/k8s/monitoring/grafana.yaml
  • services/spring-api/src/main/resources/application.yaml
📝 Walkthrough

Walkthrough

The monitoring stack adds Loki storage, Alloy log collection, Grafana dashboards and Discord alerting, TLS ingress routing, HTTP histogram metrics, and deployment workflow validation for the new monitoring components.

Changes

Monitoring observability

Layer / File(s) Summary
Loki storage and service
infra/k8s/monitoring/loki.yaml
Adds Loki configuration, a 5Gi persistent volume claim, a readiness-checked deployment, and a port 3100 service.
Alloy log collection pipeline
infra/k8s/monitoring/alloy.yaml
Adds namespace-scoped RBAC, pod discovery and log forwarding configuration, and a single-replica Alloy deployment targeting Loki.
Grafana dashboards and alerting
infra/k8s/monitoring/grafana.yaml, infra/k8s/monitoring/grafana-alerting.yaml, infra/k8s/monitoring/grafana-dashboards.yaml, services/spring-api/src/main/resources/application.yaml
Provisions Loki, dashboards, Discord alerting, Grafana plugins and mounts, and HTTP request histograms for Prometheus.
Grafana ingress and access path
infra/k8s/monitoring/grafana-ingress.yaml, README.md
Moves Grafana to the root path on a new TLS-enabled host and updates the documented link.
Monitoring deployment validation
.github/workflows/deploy-k8s.yml, infra/k8s/monitoring/prometheus.yaml
Creates the Discord webhook secret, waits for Loki and Alloy rollouts, and limits Prometheus deployment revision history.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SpringAPI
  participant Prometheus
  participant Alloy
  participant Loki
  participant Grafana
  SpringAPI->>Prometheus: Expose histogram metrics
  Alloy->>Loki: Push labeled pod logs
  Grafana->>Prometheus: Query metrics for dashboards and alerts
  Grafana->>Loki: Query logs for dashboards
Loading

Possibly related PRs

Suggested reviewers: paulwiese, jschoedl

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The PR also adds Grafana dashboards, Discord alerting, and API health alerts, which go beyond the logs-only scope of #120. Move dashboards and alerting into a separate PR or link a broader monitoring issue that explicitly includes them.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: Grafana log collection with supporting dashboards and alerting.
Linked Issues check ✅ Passed The PR adds Alloy/Loki log collection for the monitored namespaces, which satisfies the logs-monitoring goal in #120.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/grafana-logs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@infra/k8s/monitoring/grafana.yaml`:
- Around line 16-22: Add Loki and Alloy Kubernetes manifests under the
monitoring configuration, including the resources required for the `loki`
deployment and `alloy` daemonset referenced by
`.github/workflows/deploy-k8s.yml`, or update the rollout targets and Grafana
datasource configuration to match the resources actually deployed. Ensure the
Loki service resolves as `loki.monitoring.svc.cluster.local:3100`.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 8dd054f8-a31b-4c93-b664-251517f8305f

📥 Commits

Reviewing files that changed from the base of the PR and between d36458b and d3d8200.

📒 Files selected for processing (2)
  • .github/workflows/deploy-k8s.yml
  • infra/k8s/monitoring/grafana.yaml

Comment thread infra/k8s/monitoring/grafana.yaml
@imol-ai imol-ai force-pushed the feat/grafana-logs branch from d3d8200 to bbaca0d Compare July 9, 2026 23:47
@imol-ai imol-ai force-pushed the feat/grafana-logs branch from bbaca0d to a5dde8e Compare July 9, 2026 23:56
@imol-ai imol-ai force-pushed the feat/grafana-logs branch from a5dde8e to ab0e54e Compare July 10, 2026 00:05
@imol-ai imol-ai force-pushed the feat/grafana-logs branch 2 times, most recently from 730e2ae to 59a808f Compare July 10, 2026 00:22

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@infra/k8s/monitoring/alloy.yaml`:
- Around line 135-157: Add a container-level securityContext to the Alloy
container definition, setting runAsNonRoot, runAsUser, readOnlyRootFilesystem,
and allowPrivilegeEscalation: false. Place it alongside the existing image,
args, ports, volumeMounts, and resources fields, using the appropriate non-root
UID supported by the Alloy image.
- Line 137: Update the Grafana Alloy container image under the Kubernetes
monitoring configuration from grafana/alloy:v1.7.1 to the current stable
grafana/alloy:v1.17.1.

In `@infra/k8s/monitoring/loki.yaml`:
- Around line 74-99: Add a container-level securityContext to the loki container
definition, setting runAsNonRoot to true, runAsUser to a non-root UID,
readOnlyRootFilesystem to true, and allowPrivilegeEscalation to false. Keep the
existing volume mounts for writable paths.
- Line 76: The Loki container image is pinned to outdated version 3.4.2 and must
be upgraded. Update the image reference in the Loki deployment to
grafana/loki:v3.6.4 or a newer stable release, preserving the existing image
configuration.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 85a90264-8ac6-4845-bf6c-6e1a8f520894

📥 Commits

Reviewing files that changed from the base of the PR and between d3d8200 and 4a96eb9.

📒 Files selected for processing (7)
  • .github/workflows/deploy-k8s.yml
  • README.md
  • infra/k8s/monitoring/alloy.yaml
  • infra/k8s/monitoring/grafana-ingress.yaml
  • infra/k8s/monitoring/grafana.yaml
  • infra/k8s/monitoring/loki.yaml
  • infra/k8s/monitoring/prometheus.yaml
✅ Files skipped from review due to trivial changes (2)
  • infra/k8s/monitoring/prometheus.yaml
  • README.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/deploy-k8s.yml

Comment on lines +135 to +157
containers:
- name: alloy
image: grafana/alloy:v1.7.1
args:
- run
- /etc/alloy/config.alloy
- --storage.path=/var/lib/alloy/data
- --server.http.listen-addr=0.0.0.0:12345
ports:
- name: http
containerPort: 12345
volumeMounts:
- name: config
mountPath: /etc/alloy
- name: data
mountPath: /var/lib/alloy/data
resources:
requests:
cpu: 30m
memory: 64Mi
limits:
cpu: 100m
memory: 128Mi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Add container-level security context to the Alloy container.

The Alloy container has no securityContext. Add runAsNonRoot, runAsUser, readOnlyRootFilesystem, and allowPrivilegeEscalation: false to follow least-privilege principles.

🔒️ Proposed security hardening
         - name: alloy
           image: grafana/alloy:v1.7.1
+          securityContext:
+            runAsNonRoot: true
+            runAsUser: 10001
+            readOnlyRootFilesystem: true
+            allowPrivilegeEscalation: false
           args:
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
containers:
- name: alloy
image: grafana/alloy:v1.7.1
args:
- run
- /etc/alloy/config.alloy
- --storage.path=/var/lib/alloy/data
- --server.http.listen-addr=0.0.0.0:12345
ports:
- name: http
containerPort: 12345
volumeMounts:
- name: config
mountPath: /etc/alloy
- name: data
mountPath: /var/lib/alloy/data
resources:
requests:
cpu: 30m
memory: 64Mi
limits:
cpu: 100m
memory: 128Mi
containers:
- name: alloy
image: grafana/alloy:v1.7.1
securityContext:
runAsNonRoot: true
runAsUser: 10001
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
args:
- run
- /etc/alloy/config.alloy
- --storage.path=/var/lib/alloy/data
- --server.http.listen-addr=0.0.0.0:12345
ports:
- name: http
containerPort: 12345
volumeMounts:
- name: config
mountPath: /etc/alloy
- name: data
mountPath: /var/lib/alloy/data
resources:
requests:
cpu: 30m
memory: 64Mi
limits:
cpu: 100m
memory: 128Mi
🧰 Tools
🪛 Trivy (0.69.3)

[error] 127-157: Default security context configured

deployment alloy in monitoring namespace is using the default security context, which allows root privileges

Rule: KSV-0118

Learn more

(IaC/Kubernetes)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@infra/k8s/monitoring/alloy.yaml` around lines 135 - 157, Add a
container-level securityContext to the Alloy container definition, setting
runAsNonRoot, runAsUser, readOnlyRootFilesystem, and allowPrivilegeEscalation:
false. Place it alongside the existing image, args, ports, volumeMounts, and
resources fields, using the appropriate non-root UID supported by the Alloy
image.

Source: Linters/SAST tools

serviceAccountName: alloy
containers:
- name: alloy
image: grafana/alloy:v1.7.1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check for Alloy security advisories and latest version
gh api graphql -f query='
{
  securityVulnerabilities(first: 10, ecosystem: DOCKER, package: "grafana/alloy") {
    nodes {
      advisory {
        summary
        severity
        publishedAt
      }
      vulnerableVersionRange
      firstPatchedVersion {
        identifier
      }
    }
  }
}'

Repository: AET-DevOps26/team-devsecops

Length of output: 645


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Show the local context around the Alloy image pin
sed -n '125,145p' infra/k8s/monitoring/alloy.yaml

# Check the latest Grafana Alloy release tag from GitHub
gh api repos/grafana/alloy/releases/latest --jq '{tag_name, published_at, draft, prerelease}'

# List recent Alloy releases to see whether v1.7.1 is still current/stable
gh api repos/grafana/alloy/releases --paginate --jq '.[] | select(.prerelease==false and .draft==false) | {tag_name, published_at} ' | head -n 20

Repository: AET-DevOps26/team-devsecops

Length of output: 1998


🌐 Web query:

grafana alloy v1.7.1 vulnerabilities GitHub advisory stable release v1.17.1

💡 Result:

As of July 10, 2026, the most recent stable release of Grafana Alloy is v1.17.1, which was published on June 29, 2026 [1]. Regarding the version v1.7.1, it is not a recent release; Grafana Alloy has since progressed significantly in versioning [1][2]. There are no specific GitHub security advisories tied to a "v1.7.1" release of Grafana Alloy in the official security database [3][4]. Security vulnerabilities identified in Grafana Alloy are tracked in the GitHub Advisory Database [3] and disclosed via official Grafana channels [5]. For users managing security, it is recommended to monitor the official Grafana Alloy GitHub repository releases and their associated security advisories [6][7], as well as the official documentation for comprehensive release notes [5]. Past vulnerabilities affecting various versions of Grafana Alloy (such as CVE-2024-8975, CVE-2025-22870, and CVE-2025-29786) have been addressed in subsequent updates [3][8][9]. Always ensure you are running the latest stable release (v1.17.1 as of this date) to benefit from the latest security patches and features [1].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# List the most recent non-draft, non-prerelease release tags for grafana/alloy
gh api repos/grafana/alloy/releases --paginate --jq '.[] | select(.draft==false and .prerelease==false) | [.tag_name, .published_at] | `@tsv`' | head -n 15

# Pull the release metadata for the v1.17.1 tag specifically
gh api repos/grafana/alloy/releases/tags/v1.17.1 --jq '{tag_name, published_at, prerelease, draft}'

# Check whether the repo has an advisory or security alert mentioning grafana/alloy or v1.7.1 in public search results via GitHub code search metadata
gh api search/issues -f q='repo:grafana/alloy "v1.7.1" vulnerability OR advisory OR CVE' --jq '.items[] | {title, html_url, created_at}'

Repository: AET-DevOps26/team-devsecops

Length of output: 610


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Query GitHub advisories/discussions through the REST search API for public indicators
gh api search/issues -f q='grafana/alloy v1.7.1 vulnerability OR advisory OR CVE' --jq '.total_count, .items[] | {title, html_url, created_at}'

Repository: AET-DevOps26/team-devsecops

Length of output: 296


Bump Alloy to the current stable release. grafana/alloy:v1.7.1 is far behind the current stable v1.17.1, so update this image to pick up the latest fixes and security patches.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@infra/k8s/monitoring/alloy.yaml` at line 137, Update the Grafana Alloy
container image under the Kubernetes monitoring configuration from
grafana/alloy:v1.7.1 to the current stable grafana/alloy:v1.17.1.

Comment on lines +74 to +99
containers:
- name: loki
image: grafana/loki:3.4.2
args:
- -config.file=/etc/loki/loki.yaml
ports:
- containerPort: 3100
name: http
readinessProbe:
httpGet:
path: /ready
port: http
initialDelaySeconds: 30
periodSeconds: 10
volumeMounts:
- name: config
mountPath: /etc/loki
- name: data
mountPath: /loki
resources:
requests:
cpu: 50m
memory: 128Mi
limits:
cpu: 500m
memory: 384Mi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Add container-level security context to the Loki container.

The Loki container has no securityContext at the container level. It runs as root by default, allows privilege escalation, and has a writable root filesystem. Add runAsNonRoot, runAsUser, readOnlyRootFilesystem, and allowPrivilegeEscalation: false to harden the deployment.

🔒️ Proposed security hardening
       containers:
         - name: loki
           image: grafana/loki:3.4.2
+          securityContext:
+            runAsNonRoot: true
+            runAsUser: 10001
+            readOnlyRootFilesystem: true
+            allowPrivilegeEscalation: false
           args:
             - -config.file=/etc/loki/loki.yaml
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
containers:
- name: loki
image: grafana/loki:3.4.2
args:
- -config.file=/etc/loki/loki.yaml
ports:
- containerPort: 3100
name: http
readinessProbe:
httpGet:
path: /ready
port: http
initialDelaySeconds: 30
periodSeconds: 10
volumeMounts:
- name: config
mountPath: /etc/loki
- name: data
mountPath: /loki
resources:
requests:
cpu: 50m
memory: 128Mi
limits:
cpu: 500m
memory: 384Mi
containers:
- name: loki
image: grafana/loki:3.4.2
securityContext:
runAsNonRoot: true
runAsUser: 10001
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
args:
- -config.file=/etc/loki/loki.yaml
ports:
- containerPort: 3100
name: http
readinessProbe:
httpGet:
path: /ready
port: http
initialDelaySeconds: 30
periodSeconds: 10
volumeMounts:
- name: config
mountPath: /etc/loki
- name: data
mountPath: /loki
resources:
requests:
cpu: 50m
memory: 128Mi
limits:
cpu: 500m
memory: 384Mi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@infra/k8s/monitoring/loki.yaml` around lines 74 - 99, Add a container-level
securityContext to the loki container definition, setting runAsNonRoot to true,
runAsUser to a non-root UID, readOnlyRootFilesystem to true, and
allowPrivilegeEscalation to false. Keep the existing volume mounts for writable
paths.

Source: Linters/SAST tools

fsGroup: 10001
containers:
- name: loki
image: grafana/loki:3.4.2

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check for Loki security advisories and latest version
gh api graphql -f query='
{
  securityVulnerabilities(first: 10, ecosystem: DOCKER, package: "grafana/loki") {
    nodes {
      advisory {
        summary
        severity
        publishedAt
      }
      vulnerableVersionRange
      firstPatchedVersion {
        identifier
      }
    }
  }
}'

Repository: AET-DevOps26/team-devsecops

Length of output: 645


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the Loki manifest around the referenced line and check for the pinned image usage.
sed -n '60,90p' infra/k8s/monitoring/loki.yaml

Repository: AET-DevOps26/team-devsecops

Length of output: 860


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Query the Grafana Loki repo for the latest release tags and confirm whether 3.4.2 is still current.
gh api repos/grafana/loki/releases --paginate --jq '.[].tag_name' | sed -n '1,20p'

Repository: AET-DevOps26/team-devsecops

Length of output: 355


🌐 Web query:

Grafana Loki latest stable release and known security advisories for grafana/loki 3.4.2

💡 Result:

As of July 10, 2026, the latest stable release of Grafana Loki is v3.7.3, released on June 24, 2026 [1]. Regarding Grafana Loki version 3.4.2, it is affected by a known security vulnerability: CVE-2026-21726 (also tracked as GHSA-497x-rrr9-68jp): This is a Path Traversal vulnerability in the Ruler API endpoint (/loki/api/v1/rules/{namespace}) [2][3]. The vulnerability allows an attacker to bypass previous fixes for CVE-2021-36156 by using double encoding to read files [2][4]. This vulnerability affects versions of Loki prior to 3.6.4 [5][4][3]. Since 3.4.2 is older than 3.6.4, it is vulnerable [5][2]. Users are advised to upgrade to a patched version (v3.6.4 or later) to mitigate this issue [2].

Citations:


Upgrade the Loki image pin. infra/k8s/monitoring/loki.yaml:76 still uses grafana/loki:3.4.2, which is behind the current stable release and affected by CVE-2026-21726; bump to at least v3.6.4 (preferably the latest stable).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@infra/k8s/monitoring/loki.yaml` at line 76, The Loki container image is
pinned to outdated version 3.4.2 and must be upgraded. Update the image
reference in the Loki deployment to grafana/loki:v3.6.4 or a newer stable
release, preserving the existing image configuration.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
infra/k8s/monitoring/grafana-alerting.yaml (1)

47-92: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Error rate alert expression lacks zero-traffic safeguard present in dashboard.

The alert expression on line 61 divides by sum(rate(http_server_requests_seconds_count{job="spring-api"}[5m])) without a fallback. When there's no traffic, this produces an empty/no-data result, which noDataState: OK handles correctly. However, the dashboard version of the same query (grafana-dashboards.yaml line 124) adds or on() vector(0) to the numerator for explicit handling. Consider aligning the alert expression for consistency and to avoid relying solely on noDataState for the no-data path.

♻️ Optional: align alert expression with dashboard version
-                  expr: '100 * sum(rate(http_server_requests_seconds_count{job="spring-api",outcome="SERVER_ERROR"}[5m])) / sum(rate(http_server_requests_seconds_count{job="spring-api"}[5m]))'
+                  expr: '100 * (sum(rate(http_server_requests_seconds_count{job="spring-api",outcome="SERVER_ERROR"}[5m])) or on() vector(0)) / sum(rate(http_server_requests_seconds_count{job="spring-api"}[5m]))'
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@infra/k8s/monitoring/grafana-alerting.yaml` around lines 47 - 92, Update the
Prometheus expression in the spring-api-high-error-rate alert’s refId A model to
match the dashboard’s zero-traffic handling by adding `or on() vector(0)` to the
numerator before division. Preserve the existing 5xx rate calculation and
threshold configuration while ensuring the no-traffic result explicitly
evaluates to zero.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@infra/k8s/monitoring/grafana-alerting.yaml`:
- Around line 47-92: Update the Prometheus expression in the
spring-api-high-error-rate alert’s refId A model to match the dashboard’s
zero-traffic handling by adding `or on() vector(0)` to the numerator before
division. Preserve the existing 5xx rate calculation and threshold configuration
while ensuring the no-traffic result explicitly evaluates to zero.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 65cc62bc-a0eb-45ac-8f4a-60a97d73a8bc

📥 Commits

Reviewing files that changed from the base of the PR and between 4a96eb9 and 6c88c47.

📒 Files selected for processing (5)
  • .github/workflows/deploy-k8s.yml
  • infra/k8s/monitoring/grafana-alerting.yaml
  • infra/k8s/monitoring/grafana-dashboards.yaml
  • infra/k8s/monitoring/grafana.yaml
  • services/spring-api/src/main/resources/application.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • infra/k8s/monitoring/grafana.yaml

@imol-ai imol-ai requested review from jschoedl and paulwiese July 10, 2026 13:56
@imol-ai imol-ai force-pushed the feat/grafana-logs branch from 6c88c47 to c76c5d1 Compare July 10, 2026 13:57

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
infra/k8s/monitoring/grafana.yaml (1)

56-58: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Add container-level securityContext.

The pod sets fsGroup: 472 but the container lacks runAsNonRoot, runAsUser, and allowPrivilegeEscalation: false. Adding these hardens the Grafana pod against privilege escalation and ensures it runs as non-root even if the image default changes.

🔒️ Proposed fix
       containers:
         - name: grafana
           image: grafana/grafana:11.6.16
+          securityContext:
+            runAsNonRoot: true
+            runAsUser: 472
+            allowPrivilegeEscalation: false
           env:
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@infra/k8s/monitoring/grafana.yaml` around lines 56 - 58, Add a
container-level securityContext to the Grafana container, setting runAsNonRoot:
true, runAsUser: 472, and allowPrivilegeEscalation: false; retain the existing
pod-level fsGroup setting.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@infra/k8s/monitoring/grafana.yaml`:
- Around line 56-58: Add a container-level securityContext to the Grafana
container, setting runAsNonRoot: true, runAsUser: 472, and
allowPrivilegeEscalation: false; retain the existing pod-level fsGroup setting.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 2b5ea544-2a88-4977-87e6-a18982884603

📥 Commits

Reviewing files that changed from the base of the PR and between 6c88c47 and c76c5d1.

📒 Files selected for processing (5)
  • .github/workflows/deploy-k8s.yml
  • infra/k8s/monitoring/grafana-alerting.yaml
  • infra/k8s/monitoring/grafana-dashboards.yaml
  • infra/k8s/monitoring/grafana.yaml
  • services/spring-api/src/main/resources/application.yaml
🚧 Files skipped from review as they are similar to previous changes (4)
  • services/spring-api/src/main/resources/application.yaml
  • .github/workflows/deploy-k8s.yml
  • infra/k8s/monitoring/grafana-dashboards.yaml
  • infra/k8s/monitoring/grafana-alerting.yaml

@imol-ai imol-ai force-pushed the feat/grafana-logs branch from c76c5d1 to e3cd18c Compare July 10, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Monitoring - logs

1 participant