From 07c7efec5e14b7993d55bf1e75babb97fb3b7865 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 13 May 2026 14:24:18 -0600 Subject: [PATCH 1/7] added first draft for targetsource implementation docs --- docs/content/docs/user-guide/targetsource.md | 152 ++++++++----------- 1 file changed, 62 insertions(+), 90 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource.md b/docs/content/docs/user-guide/targetsource.md index d1fee0c..a3a3eb1 100644 --- a/docs/content/docs/user-guide/targetsource.md +++ b/docs/content/docs/user-guide/targetsource.md @@ -10,15 +10,11 @@ The `TargetSource` resource enables dynamic discovery of network devices from ex ## Discovery Sources -TargetSource supports multiple discovery backends: +TargetSource supports the following discovery providers: | Source | Description | |--------|-------------| | `http` | Fetch targets from an HTTP endpoint | -| `consul` | Discover targets from Consul service registry | -| `configMap` | Read targets from a Kubernetes ConfigMap | -| `podSelector` | Create targets from Kubernetes Pods | -| `serviceSelector` | Create targets from Kubernetes Services | ## HTTP Discovery @@ -30,92 +26,55 @@ kind: TargetSource metadata: name: http-discovery spec: - http: - url: http://inventory-service:8080/targets - labels: + provider: + http: + url: http://inventory-service:8080/targets + targetProfile: default + targetLabels: source: inventory ``` -The HTTP endpoint should return a JSON array of target objects. - -## Consul Discovery - -Discover targets from Consul service registry: - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: consul-discovery -spec: - consul: - url: http://consul:8500 - labels: - source: consul - datacenter: dc1 +The HTTP endpoint should return a JSON array of target objects. The following is an example for a valid JSON array: + +```json +[ + { + "address": "spine1:57400", + "name": "spine1", + "labels": { + "role": "spine" + } + }, + { + "address": "leaf1:57400", + "name": "leaf1", + "labels": { + "role": "leaf" + } + }, + { + "address": "leaf2:57400", + "name": "leaf2", + "labels": { + "role": "leaf" + } + } +] ``` -## ConfigMap Discovery +## TargetProfile Inheritance -Read targets from a Kubernetes ConfigMap: +Within the `TargetSource`, the default `TargetProfile` for all targets can be defined using `targetProfile`. Each target discovered inherits the defined value. -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: configmap-targets -spec: - configMap: network-devices - labels: - source: configmap -``` - -The ConfigMap should contain target definitions in a structured format. - -## Kubernetes Pod Discovery - -Create targets from Kubernetes Pods matching a label selector: - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: pod-discovery -spec: - podSelector: - matchLabels: - app: network-simulator - gnmi: enabled - labels: - source: kubernetes - type: simulator -``` - -This is useful for: -- Containerized network simulators -- Virtual network functions (VNFs) -- Development/testing environments - -## Kubernetes Service Discovery +## Label Inheritance -Create targets from Kubernetes Services matching a label selector: +Each discovered target has a label defined to identify the owning `TargetSource`: +- `operator.gnmic.dev/targetsource: datacenter-a` -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: service-discovery -spec: - serviceSelector: - matchLabels: - protocol: gnmi - labels: - source: kubernetes -``` -## Label Inheritance +This label is needed to identify all targets owned by this resource and determine which devices get applied or removed. This label takes precedence over all other labels on the target. -Labels defined in the `TargetSource.spec.labels` field are applied to all discovered targets: +Labels defined in the `TargetSource.spec.targetLabels` field are applied to all discovered targets: ```yaml apiVersion: operator.gnmic.dev/v1alpha1 @@ -123,21 +82,33 @@ kind: TargetSource metadata: name: datacenter-a spec: - consul: - url: http://consul-dc-a:8500 - labels: + provider: + http: + url: http://datacenter-a:8080/targets + targetLabels: datacenter: dc-a environment: production - source: consul ``` All targets discovered from this source will have: - `datacenter: dc-a` - `environment: production` -- `source: consul` This enables using label selectors in Pipelines to select targets by their discovery source. +## Labels from Source of Truth + +Targets can also have labels defined by the external system. These get directly applied to the target with their original key/value pair. + +The gNMIc Operator has a reserved namespace for labels which alter the behavior of the target: +- `gnmic_operator_` + +Following are all supported operator-specific labels: + +| Label | Description | +|--------|-------------| +| `gnmic_operator_target_profile` | Overwrite the `TargetProfile` which is defined in the `TargetSource` | + ## Status The TargetSource status shows discovery state: @@ -155,7 +126,7 @@ status: | `targetsCount` | Number of targets discovered | | `lastSync` | Timestamp of last successful sync | -## Example: Multi-Source Discovery + ## Lifecycle @@ -219,12 +190,13 @@ spec: When a TargetSource discovers a new device: 1. A new `Target` resource is created -2. Labels from `spec.labels` are applied -3. Owner reference is set to the TargetSource +2. The `Profile` gets specified from `spec.targetProfile` +3. Labels from `spec.targetLabels` are applied +4. Owner reference is set to the TargetSource ### Target Updates -When a discovered device's properties change: +Discovered devices get reapplied each time the target gets discovered, overwriting any changes manually made: 1. The corresponding `Target` is updated 2. Clusters using that target are reconciled From 8bab96ce2719903fc42deed051220557ce764bd4 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 13 May 2026 15:46:47 -0600 Subject: [PATCH 2/7] aligning doc with existing pages --- docs/content/docs/user-guide/targetsource.md | 143 +++++++++++++------ 1 file changed, 96 insertions(+), 47 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource.md b/docs/content/docs/user-guide/targetsource.md index a3a3eb1..5895a79 100644 --- a/docs/content/docs/user-guide/targetsource.md +++ b/docs/content/docs/user-guide/targetsource.md @@ -8,53 +8,86 @@ description: > The `TargetSource` resource enables dynamic discovery of network devices from external sources. The operator automatically creates, updates, and deletes `Target` resources based on discovered devices. -## Discovery Sources - -TargetSource supports the following discovery providers: - -| Source | Description | -|--------|-------------| -| `http` | Fetch targets from an HTTP endpoint | - -## HTTP Discovery - -Discover targets from an HTTP endpoint that returns a JSON list of targets: +## Basic Configuration ```yaml apiVersion: operator.gnmic.dev/v1alpha1 kind: TargetSource metadata: - name: http-discovery + name: targetsource-1 spec: provider: - http: - url: http://inventory-service:8080/targets + # see Discovery Providers section targetProfile: default targetLabels: source: inventory ``` -The HTTP endpoint should return a JSON array of target objects. The following is an example for a valid JSON array: +## Spec Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `provider` | object | Yes | Provider-specific discovery configuration. Exactly one provider must be configured | +| `targetProfile` | string | Yes | Reference to `TargetProfile` applied to all targets | +| `targetLabels` | map[string]string | No | Labels added to all discovered targets | + + +## Discovery Providers + +`TargetSource` supports the following discovery providers: + +| Provider | Description | +|----------|-------------| +| `http` | Discover targets from an HTTP JSON endpoint | + +### HTTP Provider + +The HTTP provider discovers targets from an HTTP endpoint returning a JSON array of target definitions. + +```yaml +spec: + provider: + http: + url: http://inventory-service:8080/targets +``` + +#### HTTP Spec Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `url` | string | Yes | URL pointing to the inventory server | + +#### Response Format + +The endpoint must return a JSON array of objects with the following structure: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | string | Yes | Name of the generated `Target` resource | +| `address` | string | Yes | Device address in `host:port` format | +| `labels` | map[string]string | No | Labels added to the generated `Target` resource | + +Example response: ```json [ { - "address": "spine1:57400", "name": "spine1", + "address": "spine1:57400", "labels": { "role": "spine" } }, { - "address": "leaf1:57400", "name": "leaf1", + "address": "leaf1:57400", "labels": { "role": "leaf" } }, { - "address": "leaf2:57400", "name": "leaf2", + "address": "leaf2:57400", "labels": { "role": "leaf" } @@ -62,29 +95,32 @@ The HTTP endpoint should return a JSON array of target objects. The following is ] ``` -## TargetProfile Inheritance - -Within the `TargetSource`, the default `TargetProfile` for all targets can be defined using `targetProfile`. Each target discovered inherits the defined value. - ## Label Inheritance -Each discovered target has a label defined to identify the owning `TargetSource`: -- `operator.gnmic.dev/targetsource: datacenter-a` +Each generated `Target` receives an ownership label identifying the originating `TargetSource`: +```yaml +operator.gnmic.dev/targetsource: targetsource-1 +``` + +This label is automatically managed by the operator and is used to: +- Identify targets owned by a specific `TargetSource` +- Determine which targets should be updated or deleted during reconciliation +The `operator.gnmic.dev/targetsource` label is reserved and always takes precedence over any provider-supplied labels. -This label is needed to identify all targets owned by this resource and determine which devices get applied or removed. This label takes precedence over all other labels on the target. +### TargetSource Labels -Labels defined in the `TargetSource.spec.targetLabels` field are applied to all discovered targets: +Additional labels can be applied to all generated targets using `spec.targetLabels`: ```yaml apiVersion: operator.gnmic.dev/v1alpha1 kind: TargetSource metadata: - name: datacenter-a + name: targetsource-1 spec: provider: http: - url: http://datacenter-a:8080/targets + url: http://targetsource-1:8080/targets targetLabels: datacenter: dc-a environment: production @@ -94,20 +130,27 @@ All targets discovered from this source will have: - `datacenter: dc-a` - `environment: production` -This enables using label selectors in Pipelines to select targets by their discovery source. +This enables Pipelines to select targets using label selectors. -## Labels from Source of Truth +### Labels from Discovery Providers -Targets can also have labels defined by the external system. These get directly applied to the target with their original key/value pair. +Discovery providers may return additional labels for each target. These labels are applied directly to the generated `Target` resource. -The gNMIc Operator has a reserved namespace for labels which alter the behavior of the target: -- `gnmic_operator_` +The `gnmic_operator_` label prefix is reserved for operator-specific behavior. Labels using this prefix are interpreted by the operator and are not applied directly to the generated `Target` resource. -Following are all supported operator-specific labels: +Supported operator labels: | Label | Description | |--------|-------------| -| `gnmic_operator_target_profile` | Overwrite the `TargetProfile` which is defined in the `TargetSource` | +| `gnmic_operator_target_profile` | Overrides the `TargetProfile` configured in the `TargetSource` | + +### Label Precedence + +If the same label key is defined in multiple places, labels are applied in the following order (highest precedence first): + +1. `TargetSource` ownership label (`operator.gnmic.dev/targetsource`) +2. Labels from `TargetSource.spec.targetLabels` +3. Labels returned by the discovery provider ## Status @@ -188,27 +231,33 @@ spec: ### Target Creation -When a TargetSource discovers a new device: +When a `TargetSource` discovers a new device: + 1. A new `Target` resource is created -2. The `Profile` gets specified from `spec.targetProfile` +2. The `TargetProfile` referenced in `spec.targetProfile` is assigned 3. Labels from `spec.targetLabels` are applied -4. Owner reference is set to the TargetSource +4. The `TargetSource` is set as the owner reference ### Target Updates -Discovered devices get reapplied each time the target gets discovered, overwriting any changes manually made: -1. The corresponding `Target` is updated -2. Clusters using that target are reconciled +On each discovery cycle, existing `Target` resources are reconciled with the latest discovered state: + +1. The corresponding `Target` resource is updated and overwritten +2. Clusters consuming the target are reconciled automatically + +> Manual changes to `Target` resources managed by a `TargetSource` are overwritten on every reconciliation cycle. ### Target Deletion -When a device is no longer discovered: -1. The `Target` resource is deleted -2. Clusters stop collecting from that target +When a device is no longer returned by the discovery provider: + +1. The corresponding `Target` resource is deleted +2. Clusters automatically stop using the target ### TargetSource Deletion -When a TargetSource is deleted: -1. All Targets owned by it are deleted (via owner references) -2. Clusters are reconciled to remove those targets +When a `TargetSource` is deleted: + +1. All `Target` resources owned by it are deleted via owner references +2. Clusters are reconciled and remove the deleted targets From 2d1f5fb28a113511781e0c6623f676981b8a6db5 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 20 May 2026 19:31:17 -0600 Subject: [PATCH 3/7] updated targetSource guide based on new CRD --- docs/content/docs/user-guide/targetsource.md | 176 ++++++++++++++++++- 1 file changed, 170 insertions(+), 6 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource.md b/docs/content/docs/user-guide/targetsource.md index 5895a79..896b405 100644 --- a/docs/content/docs/user-guide/targetsource.md +++ b/docs/content/docs/user-guide/targetsource.md @@ -18,6 +18,7 @@ metadata: spec: provider: # see Discovery Providers section + targetPort: 57400 targetProfile: default targetLabels: source: inventory @@ -28,6 +29,7 @@ spec: | Field | Type | Required | Description | |-------|------|----------|-------------| | `provider` | object | Yes | Provider-specific discovery configuration. Exactly one provider must be configured | +| `targetPort` | int32 | No | Default port used if the discovered target does not provide a port. | | `targetProfile` | string | Yes | Reference to `TargetProfile` applied to all targets | | `targetLabels` | map[string]string | No | Labels added to all discovered targets | @@ -49,6 +51,7 @@ spec: provider: http: url: http://inventory-service:8080/targets + ``` #### HTTP Spec Fields @@ -56,16 +59,173 @@ spec: | Field | Type | Required | Description | |-------|------|----------|-------------| | `url` | string | Yes | URL pointing to the inventory server | +| `acceptPush` | bool | No | Enable webhook-based target updates. Defaults to `false`. | +| `authorization` | object | No | Credentials used to access the HTTP endpoint. See _Authorization_ section. | +| `pollInterval` | metav1.Duration | No | Polling interval used to fetch targets from the endpoint. Defaults to `30s`. | +| `timeout` | metav1.Duration | No | Timeout for HTTP requests. Defaults to `10s`. | +| `tls` | object | No | Client TLS configuration for HTTPS endpoints. See _TLS_ section. | +| `pagination` | object | No | Pagination configuration for parsing responses from the HTTP endpoint. See _Pagination_ section. | +| `responseMapping` | object | No | JSON path mapping definitions. See _Response Mapping_ section. | + +##### Authorization + +The HTTP provider supports authenticated requests to the inventory endpoint. + +Exactly one authorization method can be configured. + +###### Basic Authentication + +Credentials can either be defined inline or referenced from a Secret. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/targets + authorization: + basic: + username: admin + password: secret +``` + +Using a Secret reference: + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/targets + authorization: + basic: + credentialsSecretRef: + name: inventory-credentials + key: username +``` + +###### Token Authentication + +Static token authentication can be configured using either an inline token or a Secret reference. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/targets + authorization: + token: + scheme: Bearer + token: eyJhbGciOi... +``` + +Using a Secret reference: + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/targets + authorization: + token: + scheme: Bearer + tokenSecretRef: + name: inventory-token + key: token +``` + +##### TLS + +TLS settings can be configured for HTTPS endpoints. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/targets + tls: + insecureSkipVerify: false +``` + +###### TLS Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `insecureSkipVerify` | bool | No | Skip verification of the server certificate. Defaults to `false`. | +| `caBundle` | []byte | No | Base64-encoded PEM CA bundle used to validate the server certificate. | +| `caBundleSecretRef` | object | No | Reference to a Secret containing a PEM CA bundle. | + +`caBundle` and `caBundleSecretRef` are mutually exclusive. + +##### Pagination + +Pagination can be configured for APIs returning paginated responses. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/devices + pagination: + itemsField: results + nextField: next +``` + +###### Pagination Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `itemsField` | string | No | Top-level JSON field containing the list of target objects. | +| `nextField` | string | No | Top-level JSON field containing the next page reference or pagination token. | + +The `nextField` value may either contain: +- A full URL for the next request +- A pagination token appended as a query parameter to the original URL + +##### Response Mapping + +By default, the HTTP response must follow the structure defined in the _Response Format_ section. + +`responseMapping` allows extracting target fields from arbitrary JSON structures using JSONPath expressions. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/devices + responseMapping: + name: "$.hostname" + ip: "$.management.ip" + port: "$.gnmi.port" + targetProfile: "$.profile" + labels: + role: "$.metadata.role" + site: "$.metadata.site" +``` + +###### Response Mapping Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | string | Yes | JSONPath expression extracting the target name. | +| `ip` | string | Yes | JSONPath expression extracting the target IP address or hostname. | +| `port` | string | No | JSONPath expression extracting the gNMI port. | +| `targetProfile` | string | No | JSONPath expression extracting the `TargetProfile`. | +| `labels` | map[string]string | No | JSONPath expressions extracting target labels. | + +Labels extracted through `responseMapping.labels` are merged with labels from `spec.targetLabels`. + +If the same label key exists in both locations, labels extracted through `responseMapping.labels` take precedence. #### Response Format -The endpoint must return a JSON array of objects with the following structure: +If `responseMapping` is not configured, the endpoint must return a JSON array of objects with the following structure: | Field | Type | Required | Description | |-------|------|----------|-------------| | `name` | string | Yes | Name of the generated `Target` resource | -| `address` | string | Yes | Device address in `host:port` format | +| `address` | string | Yes | Device address (FQDN or IP address) | +| `port` | int32 | No | Port used for gNMI connections. If omitted, `spec.targetPort` is used. | | `labels` | map[string]string | No | Labels added to the generated `Target` resource | +| `targetProfile` | string | No | Reference to a `TargetProfile`. If omitted, `spec.targetProfile` is used. | Example response: @@ -73,21 +233,25 @@ Example response: [ { "name": "spine1", - "address": "spine1:57400", + "address": "spine1", + "port": 57400, "labels": { "role": "spine" - } + }, + "targetProfile": "spine-profile" }, { "name": "leaf1", - "address": "leaf1:57400", + "address": "leaf1", + "port": 57400, "labels": { "role": "leaf" } }, { "name": "leaf2", - "address": "leaf2:57400", + "address": "leaf2", + "port": 57400, "labels": { "role": "leaf" } From abbbee4d33aa4dc5d7c7a5d6098c7bdaafb178fc Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 20 May 2026 20:38:47 -0600 Subject: [PATCH 4/7] renamed response field ip to address --- docs/content/docs/user-guide/targetsource.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/docs/user-guide/targetsource.md b/docs/content/docs/user-guide/targetsource.md index 896b405..9a699ff 100644 --- a/docs/content/docs/user-guide/targetsource.md +++ b/docs/content/docs/user-guide/targetsource.md @@ -193,7 +193,7 @@ spec: url: https://inventory.example.com/devices responseMapping: name: "$.hostname" - ip: "$.management.ip" + address: "$.management.ip" port: "$.gnmi.port" targetProfile: "$.profile" labels: From 21496fc079150a2d74784f65c72533bf70b751b9 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 20 May 2026 20:48:43 -0600 Subject: [PATCH 5/7] moved http into a subfile of TargetSource --- .../docs/user-guide/targetsource/_index.md | 211 ++++++++++++++++ .../{targetsource.md => targetsource/http.md} | 233 ++---------------- 2 files changed, 225 insertions(+), 219 deletions(-) create mode 100644 docs/content/docs/user-guide/targetsource/_index.md rename docs/content/docs/user-guide/{targetsource.md => targetsource/http.md} (50%) diff --git a/docs/content/docs/user-guide/targetsource/_index.md b/docs/content/docs/user-guide/targetsource/_index.md new file mode 100644 index 0000000..16861e7 --- /dev/null +++ b/docs/content/docs/user-guide/targetsource/_index.md @@ -0,0 +1,211 @@ +--- +title: "TargetSource" +linkTitle: "TargetSource" +weight: 4 +description: > + Dynamic target discovery from external sources +--- + +The `TargetSource` resource enables dynamic discovery of network devices from external sources. The operator automatically creates, updates, and deletes `Target` resources based on discovered devices. + +## Basic Configuration + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: targetsource-1 +spec: + provider: + # see Discovery Providers section + targetPort: 57400 + targetProfile: default + targetLabels: + source: inventory +``` + +## Spec Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `provider` | object | Yes | Provider-specific discovery configuration. Exactly one provider must be configured | +| `targetPort` | int32 | No | Default port used if the discovered target does not provide a port. | +| `targetProfile` | string | Yes | Reference to `TargetProfile` applied to all targets | +| `targetLabels` | map[string]string | No | Labels added to all discovered targets | + + +## Discovery Providers + +`TargetSource` supports the following discovery providers: + +| Provider | Description | +|----------|-------------| +| `http` | Discover targets from an HTTP JSON endpoint. [Configuration]({{< relref "http.md" >}}) | + + +## Label Inheritance + +Each generated `Target` receives an ownership label identifying the originating `TargetSource`: +```yaml +operator.gnmic.dev/targetsource: targetsource-1 +``` + +This label is automatically managed by the operator and is used to: +- Identify targets owned by a specific `TargetSource` +- Determine which targets should be updated or deleted during reconciliation + +The `operator.gnmic.dev/targetsource` label is reserved and always takes precedence over any provider-supplied labels. + +### TargetSource Labels + +Additional labels can be applied to all generated targets using `spec.targetLabels`: + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: targetsource-1 +spec: + provider: + http: + url: http://targetsource-1:8080/targets + targetLabels: + datacenter: dc-a + environment: production +``` + +All targets discovered from this source will have: +- `datacenter: dc-a` +- `environment: production` + +This enables Pipelines to select targets using label selectors. + +### Labels from Discovery Providers + +Discovery providers may return additional labels for each target. These labels are applied directly to the generated `Target` resource. + +The `gnmic_operator_` label prefix is reserved for operator-specific behavior. Labels using this prefix are interpreted by the operator and are not applied directly to the generated `Target` resource. + +Supported operator labels: + +| Label | Description | +|--------|-------------| +| `gnmic_operator_target_profile` | Overrides the `TargetProfile` configured in the `TargetSource` | + +### Label Precedence + +If the same label key is defined in multiple places, labels are applied in the following order (highest precedence first): + +1. `TargetSource` ownership label (`operator.gnmic.dev/targetsource`) +2. Labels from `TargetSource.spec.targetLabels` +3. Labels returned by the discovery provider + +## Status + +The TargetSource status shows discovery state: + +```yaml +status: + status: Synced + targetsCount: 42 + lastSync: "2024-01-15T10:30:00Z" +``` + +| Field | Description | +|-------|-------------| +| `status` | Current sync status (Synced, Error, Pending) | +| `targetsCount` | Number of targets discovered | +| `lastSync` | Timestamp of last successful sync | + + + +## Lifecycle + +### Target Creation + +When a `TargetSource` discovers a new device: + +1. A new `Target` resource is created +2. The `TargetProfile` referenced in `spec.targetProfile` is assigned +3. Labels from `spec.targetLabels` are applied +4. The `TargetSource` is set as the owner reference + +### Target Updates + +On each discovery cycle, existing `Target` resources are reconciled with the latest discovered state: + +1. The corresponding `Target` resource is updated and overwritten +2. Clusters consuming the target are reconciled automatically + +> Manual changes to `Target` resources managed by a `TargetSource` are overwritten on every reconciliation cycle. + +### Target Deletion + +When a device is no longer returned by the discovery provider: + +1. The corresponding `Target` resource is deleted +2. Clusters automatically stop using the target + +### TargetSource Deletion + +When a `TargetSource` is deleted: + +1. All `Target` resources owned by it are deleted via owner references +2. Clusters are reconciled and remove the deleted targets + diff --git a/docs/content/docs/user-guide/targetsource.md b/docs/content/docs/user-guide/targetsource/http.md similarity index 50% rename from docs/content/docs/user-guide/targetsource.md rename to docs/content/docs/user-guide/targetsource/http.md index 9a699ff..9fea5d3 100644 --- a/docs/content/docs/user-guide/targetsource.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -1,49 +1,11 @@ --- -title: "TargetSource" -linkTitle: "TargetSource" +title: "HTTP Provider" +linkTitle: "HTTP" weight: 4 description: > - Dynamic target discovery from external sources + HTTP TargetSource Discovery Provider --- -The `TargetSource` resource enables dynamic discovery of network devices from external sources. The operator automatically creates, updates, and deletes `Target` resources based on discovered devices. - -## Basic Configuration - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: targetsource-1 -spec: - provider: - # see Discovery Providers section - targetPort: 57400 - targetProfile: default - targetLabels: - source: inventory -``` - -## Spec Fields - -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| `provider` | object | Yes | Provider-specific discovery configuration. Exactly one provider must be configured | -| `targetPort` | int32 | No | Default port used if the discovered target does not provide a port. | -| `targetProfile` | string | Yes | Reference to `TargetProfile` applied to all targets | -| `targetLabels` | map[string]string | No | Labels added to all discovered targets | - - -## Discovery Providers - -`TargetSource` supports the following discovery providers: - -| Provider | Description | -|----------|-------------| -| `http` | Discover targets from an HTTP JSON endpoint | - -### HTTP Provider - The HTTP provider discovers targets from an HTTP endpoint returning a JSON array of target definitions. ```yaml @@ -54,7 +16,7 @@ spec: ``` -#### HTTP Spec Fields +## HTTP Spec Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -67,13 +29,13 @@ spec: | `pagination` | object | No | Pagination configuration for parsing responses from the HTTP endpoint. See _Pagination_ section. | | `responseMapping` | object | No | JSON path mapping definitions. See _Response Mapping_ section. | -##### Authorization +### Authorization The HTTP provider supports authenticated requests to the inventory endpoint. Exactly one authorization method can be configured. -###### Basic Authentication +#### Basic Authentication Credentials can either be defined inline or referenced from a Secret. @@ -102,7 +64,7 @@ spec: key: username ``` -###### Token Authentication +#### Token Authentication Static token authentication can be configured using either an inline token or a Secret reference. @@ -132,7 +94,7 @@ spec: key: token ``` -##### TLS +### TLS TLS settings can be configured for HTTPS endpoints. @@ -145,7 +107,7 @@ spec: insecureSkipVerify: false ``` -###### TLS Fields +#### TLS Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -155,7 +117,7 @@ spec: `caBundle` and `caBundleSecretRef` are mutually exclusive. -##### Pagination +### Pagination Pagination can be configured for APIs returning paginated responses. @@ -169,7 +131,7 @@ spec: nextField: next ``` -###### Pagination Fields +#### Pagination Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -180,7 +142,7 @@ The `nextField` value may either contain: - A full URL for the next request - A pagination token appended as a query parameter to the original URL -##### Response Mapping +### Response Mapping By default, the HTTP response must follow the structure defined in the _Response Format_ section. @@ -201,7 +163,7 @@ spec: site: "$.metadata.site" ``` -###### Response Mapping Fields +#### Response Mapping Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -215,7 +177,7 @@ Labels extracted through `responseMapping.labels` are merged with labels from `s If the same label key exists in both locations, labels extracted through `responseMapping.labels` take precedence. -#### Response Format +## Response Format If `responseMapping` is not configured, the endpoint must return a JSON array of objects with the following structure: @@ -258,170 +220,3 @@ Example response: } ] ``` - -## Label Inheritance - -Each generated `Target` receives an ownership label identifying the originating `TargetSource`: -```yaml -operator.gnmic.dev/targetsource: targetsource-1 -``` - -This label is automatically managed by the operator and is used to: -- Identify targets owned by a specific `TargetSource` -- Determine which targets should be updated or deleted during reconciliation - -The `operator.gnmic.dev/targetsource` label is reserved and always takes precedence over any provider-supplied labels. - -### TargetSource Labels - -Additional labels can be applied to all generated targets using `spec.targetLabels`: - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: targetsource-1 -spec: - provider: - http: - url: http://targetsource-1:8080/targets - targetLabels: - datacenter: dc-a - environment: production -``` - -All targets discovered from this source will have: -- `datacenter: dc-a` -- `environment: production` - -This enables Pipelines to select targets using label selectors. - -### Labels from Discovery Providers - -Discovery providers may return additional labels for each target. These labels are applied directly to the generated `Target` resource. - -The `gnmic_operator_` label prefix is reserved for operator-specific behavior. Labels using this prefix are interpreted by the operator and are not applied directly to the generated `Target` resource. - -Supported operator labels: - -| Label | Description | -|--------|-------------| -| `gnmic_operator_target_profile` | Overrides the `TargetProfile` configured in the `TargetSource` | - -### Label Precedence - -If the same label key is defined in multiple places, labels are applied in the following order (highest precedence first): - -1. `TargetSource` ownership label (`operator.gnmic.dev/targetsource`) -2. Labels from `TargetSource.spec.targetLabels` -3. Labels returned by the discovery provider - -## Status - -The TargetSource status shows discovery state: - -```yaml -status: - status: Synced - targetsCount: 42 - lastSync: "2024-01-15T10:30:00Z" -``` - -| Field | Description | -|-------|-------------| -| `status` | Current sync status (Synced, Error, Pending) | -| `targetsCount` | Number of targets discovered | -| `lastSync` | Timestamp of last successful sync | - - - -## Lifecycle - -### Target Creation - -When a `TargetSource` discovers a new device: - -1. A new `Target` resource is created -2. The `TargetProfile` referenced in `spec.targetProfile` is assigned -3. Labels from `spec.targetLabels` are applied -4. The `TargetSource` is set as the owner reference - -### Target Updates - -On each discovery cycle, existing `Target` resources are reconciled with the latest discovered state: - -1. The corresponding `Target` resource is updated and overwritten -2. Clusters consuming the target are reconciled automatically - -> Manual changes to `Target` resources managed by a `TargetSource` are overwritten on every reconciliation cycle. - -### Target Deletion - -When a device is no longer returned by the discovery provider: - -1. The corresponding `Target` resource is deleted -2. Clusters automatically stop using the target - -### TargetSource Deletion - -When a `TargetSource` is deleted: - -1. All `Target` resources owned by it are deleted via owner references -2. Clusters are reconciled and remove the deleted targets - From db090b3e6c8f5ff622e0677c5fe2fd0b6e211431 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 20 May 2026 21:00:38 -0600 Subject: [PATCH 6/7] restructured http chapter hierarchy --- .../docs/user-guide/targetsource/http.md | 88 ++++++++++--------- 1 file changed, 45 insertions(+), 43 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index 9fea5d3..b6ff8e4 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -13,7 +13,6 @@ spec: provider: http: url: http://inventory-service:8080/targets - ``` ## HTTP Spec Fields @@ -23,19 +22,19 @@ spec: | `url` | string | Yes | URL pointing to the inventory server | | `acceptPush` | bool | No | Enable webhook-based target updates. Defaults to `false`. | | `authorization` | object | No | Credentials used to access the HTTP endpoint. See _Authorization_ section. | -| `pollInterval` | metav1.Duration | No | Polling interval used to fetch targets from the endpoint. Defaults to `30s`. | -| `timeout` | metav1.Duration | No | Timeout for HTTP requests. Defaults to `10s`. | +| `pollInterval` | duration | No | Polling interval used to fetch targets from the endpoint. Defaults to `30s`. | +| `timeout` | duration | No | Timeout for HTTP requests. Defaults to `10s`. | | `tls` | object | No | Client TLS configuration for HTTPS endpoints. See _TLS_ section. | | `pagination` | object | No | Pagination configuration for parsing responses from the HTTP endpoint. See _Pagination_ section. | -| `responseMapping` | object | No | JSON path mapping definitions. See _Response Mapping_ section. | +| `responseMapping` | object | No | JSONPath mapping definitions. See _Response Processing_ section. | -### Authorization +## Authorization The HTTP provider supports authenticated requests to the inventory endpoint. Exactly one authorization method can be configured. -#### Basic Authentication +### Basic Authentication Credentials can either be defined inline or referenced from a Secret. @@ -64,7 +63,7 @@ spec: key: username ``` -#### Token Authentication +### Token Authentication Static token authentication can be configured using either an inline token or a Secret reference. @@ -94,7 +93,7 @@ spec: key: token ``` -### TLS +## TLS TLS settings can be configured for HTTPS endpoints. @@ -107,7 +106,7 @@ spec: insecureSkipVerify: false ``` -#### TLS Fields +### TLS Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -117,7 +116,7 @@ spec: `caBundle` and `caBundleSecretRef` are mutually exclusive. -### Pagination +## Pagination Pagination can be configured for APIs returning paginated responses. @@ -131,7 +130,7 @@ spec: nextField: next ``` -#### Pagination Fields +### Pagination Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -142,42 +141,16 @@ The `nextField` value may either contain: - A full URL for the next request - A pagination token appended as a query parameter to the original URL -### Response Mapping +## Response Processing -By default, the HTTP response must follow the structure defined in the _Response Format_ section. +The HTTP provider supports two methods for processing responses from the inventory endpoint: -`responseMapping` allows extracting target fields from arbitrary JSON structures using JSONPath expressions. +- **Default Response Format**: The endpoint returns a predefined JSON structure understood directly by the operator. +- **Response Mapping via JSONPath**: Arbitrary JSON structures can be mapped to target fields using JSONPath expressions. -```yaml -spec: - provider: - http: - url: https://inventory.example.com/devices - responseMapping: - name: "$.hostname" - address: "$.management.ip" - port: "$.gnmi.port" - targetProfile: "$.profile" - labels: - role: "$.metadata.role" - site: "$.metadata.site" -``` +If `responseMapping` is configured, the custom mappings are used. Otherwise, the default response format is expected. -#### Response Mapping Fields - -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| `name` | string | Yes | JSONPath expression extracting the target name. | -| `ip` | string | Yes | JSONPath expression extracting the target IP address or hostname. | -| `port` | string | No | JSONPath expression extracting the gNMI port. | -| `targetProfile` | string | No | JSONPath expression extracting the `TargetProfile`. | -| `labels` | map[string]string | No | JSONPath expressions extracting target labels. | - -Labels extracted through `responseMapping.labels` are merged with labels from `spec.targetLabels`. - -If the same label key exists in both locations, labels extracted through `responseMapping.labels` take precedence. - -## Response Format +### Default Response Format If `responseMapping` is not configured, the endpoint must return a JSON array of objects with the following structure: @@ -220,3 +193,32 @@ Example response: } ] ``` + +### Response Mapping via JSONPath + +`responseMapping` allows extracting target fields from arbitrary JSON structures using JSONPath expressions. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/devices + responseMapping: + name: "$.hostname" + address: "$.management.ip" + port: "$.gnmi.port" + targetProfile: "$.profile" + labels: + role: "$.metadata.role" + site: "$.metadata.site" +``` + +#### Response Mapping Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | string | Yes | JSONPath expression extracting the target name | +| `address` | string | Yes | JSONPath expression extracting the target IP address or hostname | +| `port` | string | No | JSONPath expression extracting the gNMI port | +| `targetProfile` | string | No | JSONPath expression extracting the `TargetProfile` | +| `labels` | map[string]string | No | JSONPath expressions extracting target labels | From 69d82bb884225763cb880ad2055a3f12f50326c2 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 20 May 2026 21:03:02 -0600 Subject: [PATCH 7/7] changed example router names --- docs/content/docs/user-guide/targetsource/http.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index b6ff8e4..b5ff3d0 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -168,7 +168,7 @@ Example response: [ { "name": "spine1", - "address": "spine1", + "address": "spine1.local", "port": 57400, "labels": { "role": "spine" @@ -177,7 +177,7 @@ Example response: }, { "name": "leaf1", - "address": "leaf1", + "address": "leaf1.local", "port": 57400, "labels": { "role": "leaf" @@ -185,7 +185,7 @@ Example response: }, { "name": "leaf2", - "address": "leaf2", + "address": "leaf2.local", "port": 57400, "labels": { "role": "leaf"