Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions framework/docs/telemetry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,9 @@ Config section key: `Telemetry` or `Aether:Telemetry`.
"ParseStateValues": true,
"Enrichers": {
"CustomAttributes": { "env": "Production", "team": "platform" },
"Headers": ["x-correlation-id", "x-request-id"]
"Headers": ["x-correlation-id", "x-request-id"],
"RequestHeaderKeyPrefix": "RequestHeader.",
"ResponseHeaderKeyPrefix": "ResponseHeader."
},
"Body": {
"EnableRequestBody": false,
Expand Down Expand Up @@ -278,7 +280,8 @@ public class OrderMetrics
`Telemetry:Logging:Enrichers` (CustomAttributes and Headers) are added as attributes to **every** log record in the application via `EnricherLogProcessor`. This allows querying all logs with the same enrich fields (e.g. `env`, `app`, `RequestHeader.x_correlation_id`) in a single observability query.

- **CustomAttributes**: Key-value pairs added to every log.
- **Headers**: Listed request/response headers are added as `RequestHeader.<name>` and `ResponseHeader.<name>`; values for headers in the sensitive list are redacted. When there is no HTTP context (e.g. background job), only CustomAttributes are added.
- **Headers**: Listed request/response headers are added as `RequestHeader.<name>` and `ResponseHeader.<name>`; values for headers in the sensitive list are redacted. When there is no HTTP context (e.g. background job), only CustomAttributes are added. The header name is normalized (lowercased, `-` replaced by `_`), so `X-Request-Id` becomes `RequestHeader.x_request_id`.
- **RequestHeaderKeyPrefix / ResponseHeaderKeyPrefix**: The prefixes above are configurable (defaults `RequestHeader.` / `ResponseHeader.`). Log backends that cannot store dots in flat field names — OpenObserve, Elasticsearch — lowercase the key and replace `.` with `_` on ingest, so `RequestHeader.act_sub` is queried as `requestheader_act_sub`. Set the prefix to `""` to emit the bare header name (`act_sub`) instead. Request and response keys are distinguished only by these prefixes: giving both the same value (for example both empty) makes a header present on both request and response collapse onto a single key.

### Automatic Enrichment

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
{
private readonly HashSet<string> _sensitiveHeaderNames = BuildSensitiveHeaderNames(options);

public override void OnEnd(LogRecord record)

Check failure on line 22 in framework/src/BBT.Aether.AspNetCore/BBT/Aether/AspNetCore/Telemetry/EnricherLogProcessor.cs

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Refactor this method to reduce its Cognitive Complexity from 29 to the 15 allowed.

See more on https://sonarcloud.io/project/issues?id=burgan-tech_aether&issues=AaAAD7NwCzzMtZyBLy_q&open=AaAAD7NwCzzMtZyBLy_q&pullRequest=92
{
if (record == null)
return;
Expand All @@ -38,15 +38,16 @@
}

var httpContext = httpContextAccessor?.HttpContext;
if (httpContext != null && options.Logging?.Enrichers?.Headers is { Count: > 0 } headerNames)
if (httpContext != null && options.Logging?.Enrichers is { } enrichers
&& enrichers.Headers is { Count: > 0 } headerNames)
{
foreach (var headerName in headerNames)
{
if (string.IsNullOrWhiteSpace(headerName))
continue;
var key = headerName.Trim();
var requestKey = $"RequestHeader.{NormalizeHeaderKey(key)}";
var responseKey = $"ResponseHeader.{NormalizeHeaderKey(key)}";
var requestKey = HeaderEnrichmentKeys.Request(enrichers, key);
var responseKey = HeaderEnrichmentKeys.Response(enrichers, key);
var isSensitive = _sensitiveHeaderNames.Contains(key);

if (httpContext.Request.Headers.TryGetValue(key, out var reqVal))
Expand Down Expand Up @@ -82,14 +83,11 @@
var attrs = record.Attributes;
if (attrs == null)
return false;
foreach (var kv in attrs)

Check warning on line 86 in framework/src/BBT.Aether.AspNetCore/BBT/Aether/AspNetCore/Telemetry/EnricherLogProcessor.cs

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Loop should be simplified by calling Select(kv => kv.Key))

See more on https://sonarcloud.io/project/issues?id=burgan-tech_aether&issues=AaAAD7NwCzzMtZyBLy_r&open=AaAAD7NwCzzMtZyBLy_r&pullRequest=92
{
if (kv.Key == "RequestHeaders" || kv.Key == "ResponseHeaders" || kv.Key == "RequestBody" || kv.Key == "ResponseBody")
return true;
}
return false;
}

private static string NormalizeHeaderKey(string key)
=> key.Replace("-", "_", StringComparison.Ordinal).ToLowerInvariant();
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
using System;

namespace BBT.Aether.AspNetCore.Telemetry;

/// <summary>
/// Builds the attribute keys used to enrich log records with individual HTTP header values.
/// Shared by <see cref="EnricherLogProcessor"/> (all log records) and
/// <see cref="HttpBodyLoggingMiddleware"/> (the HTTP body log scope) so the two paths cannot
/// drift apart and both honour the configured prefixes.
/// </summary>
internal static class HeaderEnrichmentKeys
{
/// <summary>
/// Normalizes a header name for use in an attribute key: lowercase, with '-' replaced by '_'
/// (e.g. <c>X-Request-Id</c> becomes <c>x_request_id</c>).
/// </summary>
internal static string Normalize(string headerName)
=> headerName.Replace("-", "_", StringComparison.Ordinal).ToLowerInvariant();

/// <summary>
/// Builds the enrich key for a header read from the request, honouring
/// <see cref="LoggingEnricherOptions.RequestHeaderKeyPrefix"/>.
/// </summary>
internal static string Request(LoggingEnricherOptions options, string headerName)
=> $"{options.RequestHeaderKeyPrefix ?? string.Empty}{Normalize(headerName)}";

/// <summary>
/// Builds the enrich key for a header read from the response, honouring
/// <see cref="LoggingEnricherOptions.ResponseHeaderKeyPrefix"/>.
/// </summary>
internal static string Response(LoggingEnricherOptions options, string headerName)
=> $"{options.ResponseHeaderKeyPrefix ?? string.Empty}{Normalize(headerName)}";
}
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@
responseBuffer.Position = 0;
await responseBuffer.CopyToAsync(originalResponseBody, context.RequestAborted);
}
catch (Exception ex)

Check warning on line 110 in framework/src/BBT.Aether.AspNetCore/BBT/Aether/AspNetCore/Telemetry/HttpBodyLoggingMiddleware.cs

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Either log this exception and handle it, or rethrow it with some contextual information.

See more on https://sonarcloud.io/project/issues?id=burgan-tech_aether&issues=AaAAD7SCCzzMtZyBLy_t&open=AaAAD7SCCzzMtZyBLy_t&pullRequest=92
{
if (_bodyOptions.EnableResponseBody && CanReadResponseBody(context.Response))
{
Expand Down Expand Up @@ -277,18 +277,18 @@
return (redacted, size, truncated, true);
}

private void LogBodies(
HttpContext context,
string? requestBody,
string? responseBody,
int? requestBodySize,
int? responseBodySize,
bool requestTruncated,
bool responseTruncated,
bool requestCaptured,
bool responseCaptured,
string? requestHeadersJson,
string? responseHeadersJson)

Check warning on line 291 in framework/src/BBT.Aether.AspNetCore/BBT/Aether/AspNetCore/Telemetry/HttpBodyLoggingMiddleware.cs

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Method has 11 parameters, which is greater than the 7 authorized.

See more on https://sonarcloud.io/project/issues?id=burgan-tech_aether&issues=AaAAD7SCCzzMtZyBLy_u&open=AaAAD7SCCzzMtZyBLy_u&pullRequest=92
{
var path = context.Request.Path.Value ?? "";
var statusCode = context.Response.StatusCode;
Expand Down Expand Up @@ -316,18 +316,18 @@

using (_logger.BeginScope(scope))
{
_logger.LogInformation(
"HTTP request/response logged. Path: {Path}, StatusCode: {StatusCode}",
path,
statusCode);

Check warning on line 322 in framework/src/BBT.Aether.AspNetCore/BBT/Aether/AspNetCore/Telemetry/HttpBodyLoggingMiddleware.cs

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Evaluation of this argument may be expensive and unnecessary if logging is disabled

See more on https://sonarcloud.io/project/issues?id=burgan-tech_aether&issues=AaAAD7SCCzzMtZyBLy_0&open=AaAAD7SCCzzMtZyBLy_0&pullRequest=92
}
}

private void AddEnricherProperties(

Check failure on line 326 in framework/src/BBT.Aether.AspNetCore/BBT/Aether/AspNetCore/Telemetry/HttpBodyLoggingMiddleware.cs

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Refactor this method to reduce its Cognitive Complexity from 18 to the 15 allowed.

See more on https://sonarcloud.io/project/issues?id=burgan-tech_aether&issues=AaAAD7SCCzzMtZyBLy_v&open=AaAAD7SCCzzMtZyBLy_v&pullRequest=92
List<KeyValuePair<string, object?>> scope,
HttpContext context,
string? requestHeadersJson,

Check warning on line 329 in framework/src/BBT.Aether.AspNetCore/BBT/Aether/AspNetCore/Telemetry/HttpBodyLoggingMiddleware.cs

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove this unused method parameter 'requestHeadersJson'.

See more on https://sonarcloud.io/project/issues?id=burgan-tech_aether&issues=AaAAD7SCCzzMtZyBLy_w&open=AaAAD7SCCzzMtZyBLy_w&pullRequest=92
string? responseHeadersJson)

Check warning on line 330 in framework/src/BBT.Aether.AspNetCore/BBT/Aether/AspNetCore/Telemetry/HttpBodyLoggingMiddleware.cs

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove this unused method parameter 'responseHeadersJson'.

See more on https://sonarcloud.io/project/issues?id=burgan-tech_aether&issues=AaAAD7SCCzzMtZyBLy_x&open=AaAAD7SCCzzMtZyBLy_x&pullRequest=92
{
if (_enricherOptions.CustomAttributes != null)
{
Expand All @@ -346,12 +346,12 @@
if (string.IsNullOrWhiteSpace(headerName))
continue;
var key = headerName.Trim();
var requestKey = $"RequestHeader.{key.Replace("-", "_", StringComparison.Ordinal).ToLowerInvariant()}";
var responseKey = $"ResponseHeader.{key.Replace("-", "_", StringComparison.Ordinal).ToLowerInvariant()}";
var requestKey = HeaderEnrichmentKeys.Request(_enricherOptions, key);
var responseKey = HeaderEnrichmentKeys.Response(_enricherOptions, key);
var isSensitive = _sensitiveHeaderNames.Contains(key);

if (context.Request.Headers.TryGetValue(key, out var reqVal))
scope.Add(new KeyValuePair<string, object?>(requestKey, isSensitive ? "***REDACTED***" : reqVal.ToString()));

Check warning on line 354 in framework/src/BBT.Aether.AspNetCore/BBT/Aether/AspNetCore/Telemetry/HttpBodyLoggingMiddleware.cs

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Define a constant instead of using this literal '***REDACTED***' 4 times.

See more on https://sonarcloud.io/project/issues?id=burgan-tech_aether&issues=AaAAD7SCCzzMtZyBLy_s&open=AaAAD7SCCzzMtZyBLy_s&pullRequest=92
if (context.Response.Headers.TryGetValue(key, out var resVal))
scope.Add(new KeyValuePair<string, object?>(responseKey, isSensitive ? "***REDACTED***" : resVal.ToString()));
}
Expand Down Expand Up @@ -381,7 +381,7 @@
}

private static List<Regex> CompileRegex(IEnumerable<string> patterns)
=> patterns.Select(p => new Regex(p, RegexOptions.Compiled | RegexOptions.IgnoreCase)).ToList();

Check warning on line 384 in framework/src/BBT.Aether.AspNetCore/BBT/Aether/AspNetCore/Telemetry/HttpBodyLoggingMiddleware.cs

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Pass a timeout to limit the execution time.

See more on https://sonarcloud.io/project/issues?id=burgan-tech_aether&issues=AaAAD7SCCzzMtZyBLy_z&open=AaAAD7SCCzzMtZyBLy_z&pullRequest=92

private static string TruncateUtf8(string value, int maxBytes)
{
Expand Down Expand Up @@ -442,8 +442,8 @@
JsonValueKind.Array => element.EnumerateArray().Select(x => RedactElement(x, sensitiveJsonFields)).ToList(),
JsonValueKind.String => element.GetString(),
JsonValueKind.Number => element.TryGetInt64(out var l) ? l :
element.TryGetDecimal(out var d) ? d :
element.GetDouble(),

Check warning on line 446 in framework/src/BBT.Aether.AspNetCore/BBT/Aether/AspNetCore/Telemetry/HttpBodyLoggingMiddleware.cs

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Extract this nested ternary operation into an independent statement.

See more on https://sonarcloud.io/project/issues?id=burgan-tech_aether&issues=AaAAD7SCCzzMtZyBLy_y&open=AaAAD7SCCzzMtZyBLy_y&pullRequest=92
JsonValueKind.True => true,
JsonValueKind.False => false,
JsonValueKind.Null => null,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,29 @@ public sealed class LoggingEnricherOptions
/// Header names to add as individual enrich properties (e.g. RequestHeader.x_correlation_id). Values for headers in the sensitive list are shown as ***REDACTED***. Full request/response headers remain in RequestHeaders/ResponseHeaders JSON.
/// </summary>
public List<string> Headers { get; set; } = new();

/// <summary>
/// Prefix applied to request-header enrich keys. Defaults to <c>"RequestHeader."</c>, so the
/// header <c>x-correlation-id</c> is emitted as <c>RequestHeader.x_correlation_id</c>.
/// <para>
/// Set to an empty string to emit the bare (normalized) header name instead — useful for log
/// backends such as OpenObserve or Elasticsearch, which cannot store dots in flat field names
/// and therefore lowercase the key and replace <c>.</c> with <c>_</c> on ingest, turning
/// <c>RequestHeader.act_sub</c> into <c>requestheader_act_sub</c>.
/// </para>
/// <para>
/// Note: request and response keys are distinguished only by these prefixes. Setting this and
/// <see cref="ResponseHeaderKeyPrefix"/> to the same value (for example both empty) makes a
/// header present on both the request and the response collapse onto a single key.
/// </para>
/// </summary>
public string RequestHeaderKeyPrefix { get; set; } = "RequestHeader.";

/// <summary>
/// Prefix applied to response-header enrich keys. Defaults to <c>"ResponseHeader."</c>.
/// See <see cref="RequestHeaderKeyPrefix"/> for the prefix semantics and the collision note.
/// </summary>
public string ResponseHeaderKeyPrefix { get; set; } = "ResponseHeader.";
}

public sealed class AetherTracingOptions
Expand Down
Loading