atunnel: broker actor certificates through atelet by EItanya · Pull Request #708 · agent-substrate/substrate

Eitan Yarmush (EItanya) · 2026-08-03T04:24:11Z

Closes #706

Summary

broker short-lived actor certificates from atelet over a same-node mTLS Unix socket
keep the actor private key in atunnel and renew the certificate before expiry
authenticate egress CONNECT using the actor certificate instead of bearer tokens

Testing

make verify
go test -race ./internal/atunnel

Lior Lieberman (LiorLieberman) · 2026-08-03T16:09:39Z

Thanks Eitan Yarmush (@EItanya) !

Quick question before I review more deeply today.

With the goal of "having a JWT available to atunnel to append on a CONNECT request to the egress" -- What parts of the this PR addresses?

Here are the parts we needed (before this pr -- havent reviewed it yet);

Improve the JWT issuance flow as part of MintJWT (properly authorize atelets, having the JWT formats solid, etc)
Implement an in-memory cache with go background routine that refreshes certs for every active actor.
Implement the UDS socket between atunnel and atelet to pass the certs+jwts.

From a very brief glance - I think your PR is attempting to do all three. Is that correct?

Also, For (1), we likely need to sort out two things to make the JWTs useful:

Get the issuer a publicly accessible DNS name that is also oidc-compliant (cc: Taahir Ahmed (@ahmedtd))
Agree on the format for the Subject field on JWTs

See for ref -

substrate/cmd/ateapi/internal/actoridentity/actoridentity.go

Lines 135 to 138 in 3ed6aa0

    
           // TODO: This is currently API but it has to be a globally unique, oidc-compliant and accsible DNS name 
        
           Issuer: "https://api.ate-system.svc", 
        
           // TODO: this format is very likely going to change. 
        
           Subject:    fmt.Sprintf("atespaces:%s:actors:%s", req.GetAtespace(), req.GetActorName()),

Eitan Yarmush (EItanya) · 2026-08-03T17:13:59Z

Yes, with two clarifications: this PR renews JWTs, not actor certificates, and each worker has only one active actor.

JWT issuance/authorization: implemented. Atunnel authenticates to atelet over mTLS using the worker Pod identity. Atelet derives the worker UID from that certificate and resolves its current actor assignment; the request does not supply actor identity. Ateapi then independently revalidates the atelet’s node, worker assignment, actor UID, and permitted audience before minting. The JWT includes the actor UID/resource version and worker Pod UID.
In-memory renewal: implemented for the worker’s single active actor. Atunnel mints before activation, stores the JWT in memory, renews with roughly 10% lifetime remaining, retries until expiry, and fails closed for new tunnels after expiry. Deactivation clears the JWT and stops renewal.
UDS broker: implemented. The UDS carries only JWT requests/responses—not certificates. The Pod certificate is used to mutually authenticate the atunnel↔atelet connection, including verifying both are on the same node.

I agree the issuer and subject contracts remain unresolved. This PR intentionally retains the existing issuer/subject behavior and does not implement PEP verification yet. Before verification lands, we need a stable OIDC issuer with discovery/JWKS reachable by the PEP.

Lior Lieberman (LiorLieberman) · 2026-08-03T22:55:07Z

+			ActorResourceVersion: actor.GetMetadata().GetVersion(),
+			WorkerPodUid:         req.GetWorkerPodUid(),


why do we need these?

Lior Lieberman (LiorLieberman) · 2026-08-04T17:28:31Z

+	clientJWTIssuer      = pflag.String("client-jwt-issuer", "", "The expected issuer URL for client JWTs.")
+	clientJWTAudience    = pflag.String("client-jwt-audience", "", "The expected audience for client JWTs.")
+	actorIDJWTPoolFile   = pflag.String("actor-id-jwt-pool", "", "The file that contains the serialized JWT authority pool for signing actor JWTs")
+	egressGatewayAddress = pflag.String("egress-gateway-address", "", "Address of the egress PEP. Empty disables tunneled egress.")


why do need egressgateway on ateapi?

Lior Lieberman (LiorLieberman) · 2026-08-04T17:36:32Z

+	// control resolves the authenticated worker Pod to its current assignment.
+	control ateapipb.ControlClient
+	// identity revalidates that assignment and signs the actor certificate.
+	identity ateapipb.ActorIdentityClient


identity as "identityService"? feels weird to call a field identity that does not represent an identity

Lior Lieberman (LiorLieberman) · 2026-08-04T17:38:00Z

+	// TODO: Before release, request an atunnel-specific MintCert purpose and
+	// require the egress PEP to reject generic actor certificates.
+	// The request deliberately carries no actor identity. The authenticated Pod
+	// UID is the only input used to select a worker and its current assignment.
+	workerUID, err := authenticatedWorkerUID(ctx)


not sure I am following the TODO comment. I thought we were saying that we need it to be an actor cert thats specifically for atunnel use.

Lior Lieberman (LiorLieberman) · 2026-08-04T17:42:43Z

+	return identity.PodUID, nil
+}
+
+func restrictClientToNode(node *substratex509.PodIdentity) func(tls.ConnectionState) error {


can we get a more descriptive name and/or a comment that tells what you are doing in this func?

Lior Lieberman (LiorLieberman) · 2026-08-04T21:23:14Z

 		containers:   req.GetSpec().GetContainers(),
 		assetPaths:   req.GetRuntimeAssetPaths(),

-		actorVersion:         req.GetActorVersion(),


whats the rationale for removing the actorVersion

Lior Lieberman (LiorLieberman) · 2026-08-04T21:24:00Z

 	// and in every ateom pod (which mounts them as overlay lowerdirs).
-	ImageCacheDir = filepath.Join(BasePath, "image-cache")
+	ImageCacheDir          = filepath.Join(BasePath, "image-cache")
+	CredentialBrokerSocket = filepath.Join(BasePath, "credential-broker.sock")


can we add a comment on top of CredentialBrokerSocket?

Lior Lieberman (LiorLieberman) · 2026-08-04T21:26:48Z

+	"google.golang.org/grpc/credentials"
+)
+
+// BrokerCertificateSource owns atunnel's actor private key and obtains the


I could be wrong but dont we want atelet to be the one that generate the private keys?

Lior Lieberman (LiorLieberman) · 2026-08-04T21:30:54Z

-// Activate allows egress for one actor. There can be only one active actor per
-// worker. bearerToken may be empty until actor JWT issuance is available.
-func (e *Egress) Activate(dialer EgressDialer, atespace, actorName string, actorVersion int64, bearerToken string) error {
+// Activate allows egress with a previously obtained actor certificate and


with a previously obtained actor certificate

thinking out loud - do we care about actorVersion(s)?

Lior Lieberman (LiorLieberman) · 2026-08-04T21:44:12Z

+
+message MintActorCertificateRequest {
+  // DER-encoded PKCS #10 certificate signing request. Atunnel retains the
+  // corresponding private key.


My initial thinking was that atelet is the thing we trust, and hence it generates the private keys and hands them to atunnel. Like kubelet. But your approach may be bettter, though keys would have to be regenerated across suspend/resume?

/cc Taahir Ahmed (@ahmedtd) for thoughts.

Eitan Yarmush (EItanya) marked this pull request as ready for review August 3, 2026 13:39

Eitan Yarmush (EItanya) requested review from Lior Lieberman (LiorLieberman) and Dmitry Berkovich (dberkov) August 3, 2026 13:39

Eitan Yarmush (EItanya) added 7 commits August 4, 2026 01:01

atunnel: broker actor JWTs through atelet

42f3216

atunnel: clarify credential lifecycle

b7a22b6

atunnel: clarify credential trust boundaries

36e58c5

atunnel: keep egress dialer private

cca7142

atunnel: clarify JWT expiry check

25706f7

atunnel: group egress gateway configuration

7137416

atunnel: authenticate egress with actor certificates

facebf3

Eitan Yarmush (EItanya) force-pushed the issue-706-atunnel-identity branch from 3306005 to facebf3 Compare August 4, 2026 01:03

Eitan Yarmush (EItanya) changed the title ~~atunnel: broker actor JWTs through atelet~~ atunnel: broker actor certificates through atelet Aug 4, 2026

Lior Lieberman (LiorLieberman) reviewed Aug 4, 2026

View reviewed changes

Eitan Yarmush (EItanya) mentioned this pull request Aug 4, 2026

Actor Identity Broker can issue an mTLS certificate that identifies a particular atunnel acting on behalf of a particular caller. #734

Open

Navigation Menu

Uh oh!

atunnel: broker actor certificates through atelet - #708

atunnel: broker actor certificates through atelet#708
Eitan Yarmush (EItanya) wants to merge 7 commits into
agent-substrate:mainfrom
kagent-dev:issue-706-atunnel-identity

Eitan Yarmush (EItanya) commented Aug 3, 2026 •

edited

Loading

Uh oh!

Lior Lieberman (LiorLieberman) commented Aug 3, 2026

Uh oh!

Eitan Yarmush (EItanya) commented Aug 3, 2026

Uh oh!

Lior Lieberman (LiorLieberman) Aug 3, 2026

Uh oh!

Lior Lieberman (LiorLieberman) Aug 4, 2026

Uh oh!

Lior Lieberman (LiorLieberman) Aug 4, 2026

Uh oh!

Lior Lieberman (LiorLieberman) Aug 4, 2026

Uh oh!

Lior Lieberman (LiorLieberman) Aug 4, 2026

Uh oh!

Lior Lieberman (LiorLieberman) Aug 4, 2026

Uh oh!

Lior Lieberman (LiorLieberman) Aug 4, 2026

Uh oh!

Lior Lieberman (LiorLieberman) Aug 4, 2026

Uh oh!

Lior Lieberman (LiorLieberman) Aug 4, 2026

Uh oh!

Lior Lieberman (LiorLieberman) Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		ActorResourceVersion: actor.GetMetadata().GetVersion(),
		WorkerPodUid: req.GetWorkerPodUid(),

Uh oh!

Conversation

Eitan Yarmush (EItanya) commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Uh oh!

Lior Lieberman (LiorLieberman) commented Aug 3, 2026

Uh oh!

Eitan Yarmush (EItanya) commented Aug 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Eitan Yarmush (EItanya) commented Aug 3, 2026 •

edited

Loading