object: add User-Agent to all storage backends - #6925
Closed
heardbefore wants to merge 2 commits into
Closed
Conversation
Set the JuiceFS UserAgent string on HTTP requests for backends that were missing it: S3 (covers B2 S3-compatible endpoint), Qiniu, WebDAV, UFile, TOS, IBM COS, Google Cloud Storage, and Azure Blob Storage. This allows cloud storage providers to identify JuiceFS traffic in access logs, following conventions established by rclone, minio-go, and aws-sdk-go. Uses each SDK's idiomatic mechanism where available (aws middleware, Azure telemetry options, GCS client options, etc.) and falls back to direct header setting for HTTP-based backends. The native B2 backend (kothar/go-backblaze) cannot be covered yet because the library does not expose http.Client injection — tracked with a TODO comment for a future follow-up. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR wires the existing global object.UserAgent string into additional object storage backends so that outbound SDK/HTTP requests consistently include a JuiceFS-identifying User-Agent across providers.
Changes:
- Add User-Agent injection to AWS SDK v2-based backends (S3, Qiniu) via aws-sdk-go-v2 middleware.
- Add User-Agent injection to provider SDK clients (Azure Blob telemetry options, GCS client option, TOS SDK option, IBM COS aws-sdk-go-v1 handler).
- Add
User-Agentheader to HTTP-based backends (WebDAV, UFile bucket creation request, RestfulStorage requests); add a TODO for B2 native backend.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/object/webdav.go | Sets a default User-Agent header on the gowebdav client. |
| pkg/object/ufile.go | Adds User-Agent header to the UFile CreateBucket HTTP request. |
| pkg/object/tos.go | Configures TOS SDK client with UserAgent product name. |
| pkg/object/s3.go | Appends aws-sdk-go-v2 User-Agent middleware to S3 client API options. |
| pkg/object/restful.go | Sets User-Agent header on all requests made through RestfulStorage. |
| pkg/object/qiniu.go | Appends aws-sdk-go-v2 User-Agent middleware to Qiniu’s S3-compatible client options. |
| pkg/object/ibmcos.go | Adds IBM COS (aws-sdk-go-v1) User-Agent handler to the session. |
| pkg/object/gs.go | Creates GCS clients with option.WithUserAgent(UserAgent). |
| pkg/object/b2.go | Documents TODO for native B2 backend pending upstream http.Client injection support. |
| pkg/object/azure.go | Adds shared azblob client options that set telemetry ApplicationID from UserAgent. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+225
to
226
| req.Header.Set("User-Agent", UserAgent) | ||
| now := time.Now().UTC().Format(http.TimeFormat) |
There was a problem hiding this comment.
This change introduces a global User-Agent header on all requests made via RestfulStorage.request, but there are currently no tests asserting the header is actually sent. Consider adding a unit test (e.g., via httptest.Server) that inspects incoming requests and verifies the User-Agent equals object.UserAgent.
Contributor
|
missing some backends like swift\obs |
Wire UserAgent into the Swift (OpenStack) and OBS (Huawei) backends using each SDK's native option: swift.Connection.UserAgent and obs.WithUserAgent respectively. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
|
hi @heardbefore, please resolve conflicts |
Contributor
|
@copilot resolve the merge conflicts in this pull request |
Contributor
|
ref #7393 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
UserAgentvariable into all storage backends that were missing it: S3, Qiniu, WebDAV, UFile, TOS (Volcengine), IBM COS, Google Cloud Storage, and Azure Blob StorageUser-Agentheader setting for HTTP-based backendshttp.Clientinjection — a TODO comment is added, and an upstream PR has been submitted to unblock thisMotivation
Several backends (BOS, COS, OSS, Storj) already set
UserAgent, but the majority — including the S3 backend that covers Backblaze B2's S3-compatible endpoint — did not. This means cloud storage providers cannot identify JuiceFS traffic in access logs.Setting a User-Agent follows conventions established by rclone, minio-go, and aws-sdk-go, and aligns with Backblaze's integration checklist guidance for integrators. It gives operators version visibility for support and issue triage.
Test plan
make juicefsdocker run -p 9000:9000 minio/minio server /data), perform reads/writes, and verifyJuiceFS-<version>appears inmc admin traceoutput's User-Agent fieldpkg/object/*_test.go🤖 Generated with Claude Code