Skip to content

Add MySQL DBI config translator and pipeline orchestrator #2196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Klarsen04 wants to merge 33 commits into
base: feature/dbi-mysql
Choose a base branch
from
Open

Add MySQL DBI config translator and pipeline orchestrator #2196

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
4fdebbc
Add MySQL receiver translator and register component
Jun 16, 2026
6c32dcd
Add MySQL-specific connector and processor configurations
Jun 16, 2026
137120b
Add MySQL config schema definition
Jun 16, 2026
6b56a34
Add MySQL pipeline translator and orchestrator wiring
Jun 16, 2026
d7e59de
Add MySQL golden YAML test fixtures
Jun 16, 2026
b77324e
Address PR review comments for MySQL DBI
Jun 30, 2026
26f8bb9
feat(translator): Add database_insights merge rule for multi-config s…
Jun 30, 2026
02b6d27
Address PR review comments
Jul 1, 2026
4c84ea5
test: Update .mysql_credentials test fixture to .my.cnf INI format
Jul 2, 2026
15156a6
chore: Update mysqlreceiver replace to .my.cnf INI format commit
Jul 2, 2026
f7367a8
fix(dbi): Count Execute command sessions in MySQL DB Load metrics
Jul 10, 2026
7f4e6b1
feat(translator): add events_statements_summary_by_digest Top SQL met…
Jul 20, 2026
33f7e00
chore: Update mysqlreceiver replace to latest commit c7001ee3110e
Jul 22, 2026
a8253b8
Merge branch 'feature/dbi-mysql' into larkirs/mysql-pipeline-translator
Jul 22, 2026
5c4e9ec
Enable mysql.table.size and replica metrics for DBI
Jul 23, 2026
5f2550a
fix(dbi): parse MySQL server log lines for multiline, timestamp, and …
Jul 23, 2026
b3163f4
Add and enable MySQL Database Insights metrics
Jul 23, 2026
390ae28
Add MySQL to combined v1/v2 EC2 and EKS configs
Jul 24, 2026
0ea2209
chore: update mysqlreceiver to latest commit (bed4560687ee) with PR #…
Jul 28, 2026
f550d8d
fix(test): set MySQL credentials file permissions in combined config …
Jul 29, 2026
d8e127a
chore(dbi): tune MySQL query sample limits
Jul 31, 2026
3660a9e
Merge remote-tracking branch 'origin/main' into larkirs/mysql-pipelin…
Jul 31, 2026
9a48cd1
chore(dbi): update mysqlreceiver to latest dbi-mysql branch (83b5027c…
Aug 6, 2026
10b6295
fix(dbi): Address PR review comments for MySQL translator - Change Qu…
Aug 10, 2026
1aa9733
fix(dbi): Add error logging for MySQL config decode failures
Aug 10, 2026
db72af1
Merge main into larkirs/mysql-pipeline-translator
Aug 11, 2026
d348170
Merge remote-tracking branch 'origin/main' into larkirs/mysql-pipelin…
Aug 11, 2026
7826e2f
Merge remote-tracking branch 'origin/feature/dbi-mysql' into larkirs/…
Aug 11, 2026
879e7cb
fix: Remove extra blank line in mysql translator_test.go
Aug 11, 2026
0409a7f
fix(dbi): Address PR review comments for MySQL translator
Aug 13, 2026
3e0f9f4
fix(databaseinsights): Use per-instance connectors to prevent metric …
Aug 17, 2026
9e67c90
Add explicit MySQL filter handling in excludeMonitorFilter
Aug 18, 2026
e76e0eb
Enable mysql.query.client.count and mysql.query.slow.count metrics
Aug 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ replace (
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsekshyperpodreceiver => github.com/amazon-contributing/opentelemetry-collector-contrib/receiver/awsekshyperpodreceiver v0.0.0-20260806162807-5cc6f180ecb6
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver => github.com/amazon-contributing/opentelemetry-collector-contrib/receiver/awsxrayreceiver v0.0.0-20260806162807-5cc6f180ecb6
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver => github.com/amazon-contributing/opentelemetry-collector-contrib/receiver/jmxreceiver v0.0.0-20260806162807-5cc6f180ecb6
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/mysqlreceiver => github.com/amazon-contributing/opentelemetry-collector-contrib/receiver/mysqlreceiver v0.0.0-20260730161852-83b5027c386b
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/postgresqlreceiver => github.com/amazon-contributing/opentelemetry-collector-contrib/receiver/postgresqlreceiver v0.0.0-20260806162807-5cc6f180ecb6
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver => github.com/amazon-contributing/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.0.0-20260806162807-5cc6f180ecb6
// Cherry-pick of upstream Query XML support for windowseventlogreceiver https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/39055
Expand Down Expand Up @@ -201,6 +202,7 @@ require (
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/journaldreceiver v0.124.1
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kafkareceiver v0.124.1
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver v0.124.1
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/mysqlreceiver v0.124.1
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/postgresqlreceiver v0.124.1
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.124.1
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/statsdreceiver v0.124.1
Expand Down Expand Up @@ -284,6 +286,7 @@ require (
)

require (
filippo.io/edwards25519 v1.1.0 // indirect
github.com/DataDog/datadog-agent/pkg/obfuscate v0.64.3 // indirect
github.com/DataDog/datadog-go/v5 v5.6.0 // indirect
github.com/DataDog/go-sqllexer v0.1.3 // indirect
Expand Down Expand Up @@ -603,7 +606,7 @@ require (
go.opentelemetry.io/collector/component/componentstatus v0.124.0 // indirect
go.opentelemetry.io/collector/config/configcompression v1.30.0
go.opentelemetry.io/collector/config/configgrpc v0.124.0 // indirect
go.opentelemetry.io/collector/config/confignet v1.30.0 // indirect
go.opentelemetry.io/collector/config/confignet v1.30.0
go.opentelemetry.io/collector/config/configretry v1.30.0 // indirect
go.opentelemetry.io/collector/confmap/provider/httpprovider v1.30.0 // indirect
go.opentelemetry.io/collector/confmap/provider/yamlprovider v1.30.0 // indirect
Expand Down Expand Up @@ -671,7 +674,7 @@ require (
google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/ini.v1 v1.67.3 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
k8s.io/kube-openapi v0.0.0-20241105132330-32ad38e42d3f // indirect
k8s.io/kubelet v0.32.3 // indirect
Expand Down
54 changes: 18 additions & 36 deletions go.sum
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions service/defaultcomponents/components.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ import (
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/journaldreceiver"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kafkareceiver"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/mysqlreceiver"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/postgresqlreceiver"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/statsdreceiver"
Expand Down Expand Up @@ -119,6 +120,7 @@ func Factories() (otelcol.Factories, error) {
nopreceiver.NewFactory(),
otlpreceiver.NewFactory(),
prometheusreceiver.NewFactory(),
mysqlreceiver.NewFactory(),
postgresqlreceiver.NewFactory(),
statsdreceiver.NewFactory(),
systemmetricsreceiver.NewFactory(),
Expand Down
1 change: 1 addition & 0 deletions service/defaultcomponents/components_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ func TestComponents(t *testing.T) {
"nop",
"otlp",
"prometheus",
"mysql",
"postgresql",
"statsd",
"systemmetrics",
Expand Down
21 changes: 21 additions & 0 deletions translator/config/schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1852,6 +1852,27 @@
"required": ["endpoint", "instance_name", "username", "password_file"],
"additionalProperties": false
}
},
"mysql": {
"type": "array",
"items": {
"type": "object",
"properties": {
"endpoint": { "type": "string" },
"instance_name": { "type": "string" },
"username": { "type": "string" },
"password_file": { "type": "string" },
"logs": {
"type": "object",
"properties": {
"file_path": { "type": "string" }
},
"additionalProperties": false
}
},
"required": ["endpoint", "instance_name", "username", "password_file"],
"additionalProperties": false
}
}
},
"additionalProperties": false
Expand Down
11 changes: 11 additions & 0 deletions translator/tocwconfig/sampleConfig/opentelemetry/combined_v1_v2_ec2_config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,17 @@
"file_path": "/var/lib/pgsql/data/log/postgresql-*.log"
}
}
],
"mysql": [
{
"endpoint": "localhost:3306",
"instance_name": "my-mysql",
"username": "cw_monitor",
"password_file": "sampleConfig/opentelemetry/testdata/.mysql_credentials",
"logs": {
"file_path": "/var/log/mysql/error.log"
}
}
]
},
"prometheus": {
Expand Down
Loading
Loading