Skip to content

Get gRPC Max msg size from env variables - #228

Merged
Sreevanich16 merged 4 commits into
Azure:kubesonicfrom
Sreevanich16:configDrivengRPCMaxMsgSize
Jul 14, 2026
Merged

Get gRPC Max msg size from env variables#228
Sreevanich16 merged 4 commits into
Azure:kubesonicfrom
Sreevanich16:configDrivengRPCMaxMsgSize

Conversation

@Sreevanich16

@Sreevanich16 Sreevanich16 commented Jul 7, 2026

Copy link
Copy Markdown

MSFT ADO - 38461436

Why I did it

Few commands are failing with ResourceExhausted error because the gNMI server could not send messages larger than 4 MB due to the default gRPC send message size limit.

root@bjw2-can-4600c-6:/# gnmi_get -xpath_target SHOW -xpath ipv6/route -target_addr 127.0.0.1:50051 -logtostderr -insecure true
== getRequest:
prefix: <
  target: "SHOW"
>
path: <
  elem: <
    name: "ipv6"
  >
  elem: <
    name: "route"
  >
>
encoding: JSON_IETF

F0624 10:15:51.503830 2267008 gnmi_get.go:145] Get failed: rpc error: code = ResourceExhausted desc = grpc: trying to send message larger than max (6785546 vs. 4194304)

How I did it

  1. Default max send/receive msg size is hardcoded as 4 MB. Added the logic of reading the value from env variables which can be added in Deamon set configuration.

How to verify it

Steps followed for testing - these params are needed to be added as env variables

1. docker exec -it telemetry bash
2. supervisorctl stop telemetry
3. export gRPC_MAX_SEND_MSG_SIZE=8388608
4. export gRPC_MAX_RECEIVE_MSG_SIZE=8388608
5. supervisorctl start telemetry
  1. After increasing the MaxSendMsgSize to 8 MB, the original send error is resolved. However, the gnmi_get client now fails with a receive-side limit error, indicating that it cannot handle messages larger than 4 MB:
root@bjw2-can-4600c-6:/# gnmi_get -xpath_target SHOW -xpath ipv6/route -target_addr 127.0.0.1:50051 -logtostderr -insecure true
== getRequest:
prefix: <
  target: "SHOW"
>
path: <
  elem: <
    name: "ipv6"
  >
  elem: <
    name: "route"
  >
>
encoding: JSON_IETF

F0624 10:21:50.736868 2274766 gnmi_get.go:145] Get failed: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (6785546 vs. 4194304)

  1. Error is resolved after Increasing the MaxRecvMsgSize on gnmi_get client end
image

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

@Sreevanich16

Copy link
Copy Markdown
Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@Sreevanich16 Sreevanich16 changed the title Enabling config for gRPC Max msg size Get gRPC Max msg size from env variables Jul 7, 2026
@Sreevanich16

Copy link
Copy Markdown
Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@Sreevanich16

Copy link
Copy Markdown
Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@Sreevanich16

Copy link
Copy Markdown
Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@Sreevanich16

Copy link
Copy Markdown
Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

Comment thread .azure/templates/install-dependencies.yml

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Sreevanich16
Sreevanich16 merged commit 5afaefa into Azure:kubesonic Jul 14, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants