Add OpenTelemetry support via adding the traceparent header from $http_traceparent

[sriemer]

http: Extend send_request() with OTel support

NGINX can be used together with OpenTelemetry (OTel) support
and the nginx-otel module provides it. In the Observability
space it is required that all HTTP request tracers in the line
do trace propagation or trace forwarding at least. This is usually
done by sending out an updated traceparent header. It contains for
example a trace ID and a span ID so that the Observability backend
such as Jaeger can correlate all spans to the same trace.

To be able to trace HTTP requests sent from regular NGINX code and
also the ones sent by LUA HTTP client code with the same NGINX tracer,
the LUA HTTP client code has to be extended to add the traceparent
header from ngx.var.http_traceparent. The value of this
$http_traceparent variable is set by the nginx-otel module. It
updates the span ID inside the traceparent value if NGINX directive
otel_trace_context propagate; is set.

So extend the send_request() function for this, because it already
adds other headers as well and is used when calling request_uri()
as well.

Check if the traceparent header is not set yet but
ngx.var.http_traceparent is set and set it from there.
Also add a code comment describing this as well.

References:

• https://github.com/nginxinc/nginx-otel
• https://nginx.org/en/docs/ngx_otel_module.html
• https://nginx.org/en/docs/http/ngx_http_core_module.html#var_http_

---

t: Add 3 traceparent header tests

The new traceparent header auto-injection from
ngx.var.http_traceparent has to be tested.

So add a new file t/21-traceparent-header.t containing 3 tests
for that:

• TEST 1: No traceparent header is set
• TEST 2: The traceparent header is correctly added when
  ngx.var.http_traceparent is used
• TEST 3: The traceparent header is not modified from
  ngx.var.http_traceparent if it is already set

Use an NGINX directive set $http_traceparent '00-000...-01'; in
front of the LUA block to emulate nginx-otel behavior.

The case that there is a traceparent header added with
request_uri() and ngx.var.http_traceparent is not set, is already
covered by tests 1 and 3.

[sriemer]

Build fails here due to deprecated Luarocks. See #333.
All code has been tested against OpenResty 1.29.2.2. I've built the test environment from a custom Dockerfile.
See #341.

[sriemer]

The nginx.conf parts I've used to test the real thing together with tcpdumping in a docker-compose based demo:

load_module modules/ngx_otel_module.so;
...
http {
...
    upstream backend {
      server server-app:8080;
    }
    otel_exporter {
        endpoint myobservability-agent:4317;
    }
    otel_trace on;
    otel_service_name myname-openresty-otel;
    server {
        ...
        location /openresty-otel-demo {
            otel_trace_context propagate;
            proxy_pass http://backend;
        }
        location /lua-otel-demo {
            otel_trace_context propagate;
            content_by_lua_block {
                local http = require "resty.http"
                local httpc = http.new()
                local res, err = httpc:request_uri("http://server-app:8080", {
                    method = "GET",
                })
                if not res then
                    ngx.status = ngx.HTTP_BAD_GATEWAY
                    ngx.header["Content-Type"] = "text/plain; charset=utf-8"
                    ngx.say("failed to call upstream: ", err)
                    return
                end
                ngx.status = res.status
                if res.headers["Content-Type"] then
                    ngx.header["Content-Type"] = res.headers["Content-Type"]
                end
                ngx.say(res.body)
            }
        }
    }
}

[sriemer]

@pintsized Please review. TIA

Also let me know if you need help from an experienced FOSS maintainer like me in reviewing open PRs.