From fdf1c123902cca157281d8cb3ef804f2f7e21dc4 Mon Sep 17 00:00:00 2001 From: Ed Sun Date: Thu, 23 Jul 2026 19:55:42 -0700 Subject: [PATCH 1/4] feat(index): scope base-model evictions via CacheEvent adapter presence CacheEvent.adapter_id was a plain proto3 string, so an empty value meant both a legacy producer (sweep every partition) and could not express an adapter-aware producer evicting the base partition. A base-model eviction was thus treated as the cross-partition wildcard and dropped live LoRA hints for the same token hash. Make adapter_id an optional string (proto3 presence) and branch on presence, not emptiness: present (even empty) narrows the removal to that one partition; absent keeps the conservative legacy sweep. The subscriber always sets it; the server carries presence into the index via ev.AdapterId != nil. Wire-compatible (old producers read as absent). New tests at the index and the full PublishEvent wire-to-index seam; grpc-contract.md updated to match. --- docs/design/grpc-contract.md | 4 +- pkg/adapters/engine/mapper.go | 9 +- pkg/adapters/engine/mapper_test.go | 12 +- pkg/index/adapter_partition_test.go | 24 +- pkg/index/index.go | 39 ++- pkg/server/inferencecache_service.go | 12 +- .../v1alpha1/inferencecache.pb.go | 233 +++++++++--------- pkg/server/server_test.go | 35 +++ .../v1alpha1/inferencecache.proto | 22 +- 9 files changed, 234 insertions(+), 156 deletions(-) diff --git a/docs/design/grpc-contract.md b/docs/design/grpc-contract.md index f54ca17b..8ef03ff7 100644 --- a/docs/design/grpc-contract.md +++ b/docs/design/grpc-contract.md @@ -62,7 +62,7 @@ service InferenceCache { - **ReplicaStats** `{ string replica_id = 1; int64 cache_memory_bytes = 2; float hit_rate = 3; float pressure = 4; string client_version = 5; int64 t2_hit_tokens = 6; int64 t2_query_tokens = 7; }` `client_version` is an opaque version string identifying the client-side cache library the reporting replica is linked against (e.g. an LMCache client release). It carries no semver semantics on the wire — producers populate it, the server accepts it, no semver parsing or ordering is performed at this layer. Empty / unset is allowed and means "unknown"; older producers that don't fill the field MUST keep being accepted (additive, v1alpha1-compatible). The field reserves wire space for the producer half of an end-to-end client/server version-skew detection surface; the consumer half (server-side storage, per-CacheBackend exposure, the operator-visible status condition that closes the silent client/server mismatch class of bug) lands in a follow-up change and is intentionally out of scope for this contract update. `t2_hit_tokens` / `t2_query_tokens` are cumulative token counters for the **tier-2 (external offload, e.g. LMCache) cache**, sourced from the engine's `vllm:external_prefix_cache_{hits,queries}_total`. The server derives a presence-aware tier-2 hit-rate per CacheBackend (`status.indexParticipation.t2HitRate`): nil until the tier is exercised (`t2_query_tokens > 0`), then the query-weighted `hits/queries` ratio — so a value of `0` means tier-2 is configured but serving no reloads (a silently-degraded offload tier), distinct from "not yet used". Both default to 0 for producers that don't populate them (additive, v1alpha1-compatible). -- **CacheEvent** `{ Type type; string replica_id; string model_id; string tenant_id; bytes prefix_hash; int64 timestamp_us; string adapter_id = 7; }` +- **CacheEvent** `{ Type type; string replica_id; string model_id; string tenant_id; bytes prefix_hash; int64 timestamp_us; optional string adapter_id = 7; }` Type ∈ `PREFIX_ADDED | PREFIX_EVICTED | REPLICA_UPDATED | ALL_CLEARED` `adapter_id` narrows a `PREFIX_EVICTED` to one adapter partition; empty removes the prefix across **every** partition (the conservative legacy behavior — removal is soft state). Ignored by `ALL_CLEARED` (a flush clears the replica across adapters) and `REPLICA_UPDATED` (liveness is adapter-independent). - **Metric** `{ string name; string type; map labels; double value; int64 timestamp_us; }` (Prometheus `inferencecache_*`, tech spec §4.3) @@ -268,7 +268,7 @@ The fix partitions the **index**, not the hash: - **The fingerprint construction is unchanged.** Adapter identity is never mixed into the hash, so `pkg/fingerprint`, its golden vectors, and every cross-language implementation (Go / Python / Rust) stay byte-for-byte as they were. The hash still answers "what content is this?"; the partition answers "whose KV is it?". - **Matching inside a partition is unchanged.** Exact-match and the block-chain longest-leading-run rule behave identically; the partition only bounds which entries are candidates. - `LookupRouteResponse.adapter_id` (5) echoes the partition the index was consulted in, so a caller can confirm the round trip. It is empty on the fail-open envelopes returned *before* the index is consulted (`TIMEOUT`, tokenizer fail-open, `POLICY_REQUIRES_CHAIN`). -- `CacheEvent.adapter_id` (7) narrows a `PREFIX_EVICTED` to one partition. Without it, one adapter's GPU eviction would drop the identical hash under *every* adapter — discarding hints that are still valid. Empty keeps the original cross-partition sweep. +- `CacheEvent.adapter_id` (7) narrows a `PREFIX_EVICTED` to one partition, keyed on **proto3 presence**. **Present** (even `""`, the base partition) drops the prefix from exactly that one partition, so one adapter's GPU eviction — including a base-model eviction — no longer discards another adapter's still-valid hint for the identical hash. **Absent** keeps the original conservative cross-partition sweep (legacy producers that never set the field). `optional` makes the two distinguishable on the wire without breaking older producers. **Producer / consumer must agree, exactly as they already do for `hash_scheme`.** A lookup that sets `adapter_id` will not match entries ingested without one, and vice versa. Empty/unset is the **default partition**, and this is exactly the pre-adapter behavior **for base-model / non-LoRA traffic**: an engine serving no LoRA emits a nil `lora_id`, so ingest and lookup both land under `""` and are completely unaffected — no regression, no configuration required. **This is not a "single LoRA adapter just works" claim.** Only a *nil* `lora_id` uses `""`; *any* non-nil `lora_id` — including a deployment with exactly one adapter — must resolve to a **configured** `--lora-adapter-names` identity, otherwise it is dropped at ingest (**fail-closed**) rather than landing under `""` or a replica-local alias. So even one LoRA adapter requires the ingest identifier (the subscriber's resolved `adapter_id`) to match the gateway's query `adapter_id` end-to-end, or every lookup silently misses — the same agreement `hash_scheme` already demands. diff --git a/pkg/adapters/engine/mapper.go b/pkg/adapters/engine/mapper.go index c4c6e565..a929f989 100644 --- a/pkg/adapters/engine/mapper.go +++ b/pkg/adapters/engine/mapper.go @@ -89,8 +89,11 @@ func (c Config) ClearedEvent(tsSeconds float64) *icpb.CacheEvent { // EvictedEvent builds one PREFIX_EVICTED CacheEvent for an already-derived prefix // hash (our content fingerprint, the index key to drop) in a specific adapter // partition. The subscriber maps an evicted engine block hash to both via -// positionalIndex.Removed. An empty adapterID is the default partition; the -// server then falls back to its cross-partition (legacy) removal. +// positionalIndex.Removed. adapterID is always set WITH presence (even "" for the +// base-model partition), so the server drops the prefix from exactly that one +// partition — a base-model eviction never sweeps live LoRA hints for the same +// token hash. (A producer that omits the field entirely reads as absent → the +// server's conservative cross-partition legacy sweep.) func (c Config) EvictedEvent(prefixHash []byte, adapterID string, tsSeconds float64) *icpb.CacheEvent { return &icpb.CacheEvent{ Type: icpb.CacheEvent_PREFIX_EVICTED, @@ -98,7 +101,7 @@ func (c Config) EvictedEvent(prefixHash []byte, adapterID string, tsSeconds floa ModelId: c.ModelID, TenantId: c.TenantID, PrefixHash: prefixHash, - AdapterId: adapterID, + AdapterId: &adapterID, TimestampUs: microsFromSeconds(tsSeconds), } } diff --git a/pkg/adapters/engine/mapper_test.go b/pkg/adapters/engine/mapper_test.go index f87faba8..ead98e91 100644 --- a/pkg/adapters/engine/mapper_test.go +++ b/pkg/adapters/engine/mapper_test.go @@ -22,8 +22,12 @@ func TestEvictedEvent(t *testing.T) { if !bytes.Equal(e.PrefixHash, []byte{0x01, 0x02}) { t.Errorf("prefix hash = %x, want 0102", e.PrefixHash) } - if e.AdapterId != "" { - t.Errorf("adapter id = %q, want \"\" (no adapter)", e.AdapterId) + // A base-model eviction carries adapter_id="" WITH presence, so the server + // drops only the base ("") partition instead of sweeping every adapter. + if e.AdapterId == nil { + t.Error("adapter id absent; a base eviction must set it WITH presence to scope removal to the base partition") + } else if got := e.GetAdapterId(); got != "" { + t.Errorf("adapter id = %q, want \"\" (base partition)", got) } } @@ -31,8 +35,8 @@ func TestEvictedEvent(t *testing.T) { // prefix only there — the same hash can be live under another adapter. func TestEvictedEventCarriesAdapter(t *testing.T) { e := testConfig().EvictedEvent([]byte{0x01}, "sql-lora", 1.0) - if e.AdapterId != "sql-lora" { - t.Errorf("adapter id = %q, want sql-lora", e.AdapterId) + if e.AdapterId == nil || e.GetAdapterId() != "sql-lora" { + t.Errorf("adapter id = %v, want sql-lora (present)", e.AdapterId) } } diff --git a/pkg/index/adapter_partition_test.go b/pkg/index/adapter_partition_test.go index bfdb7b51..834c0401 100644 --- a/pkg/index/adapter_partition_test.go +++ b/pkg/index/adapter_partition_test.go @@ -188,7 +188,7 @@ func TestAdapterPartitionEvictionIsScopedToItsAdapter(t *testing.T) { idx.ApplyEvent(Event{ Type: EventPrefixEvicted, ReplicaID: "replica-0", Model: adapterModel, Tenant: adapterTenant, - PrefixHash: hash("same-tokens"), Adapter: "sql-lora", + PrefixHash: hash("same-tokens"), Adapter: "sql-lora", AdapterSet: true, }) if got := lookupUnder(idx, "sql-lora", "same-tokens"); len(got) != 0 { @@ -199,6 +199,28 @@ func TestAdapterPartitionEvictionIsScopedToItsAdapter(t *testing.T) { } } +// A base-model eviction (Adapter "" WITH presence) drops ONLY the base partition +// and must not sweep a live LoRA hint for the same token hash — the over-sweep +// that adapter_id presence exists to fix. +func TestAdapterPartitionBaseEvictionWithPresenceSpareLoRA(t *testing.T) { + idx := New() + ingestUnder(idx, "replica-0", "", "same-tokens") + ingestUnder(idx, "replica-0", "sql-lora", "same-tokens") + + idx.ApplyEvent(Event{ + Type: EventPrefixEvicted, ReplicaID: "replica-0", + Model: adapterModel, Tenant: adapterTenant, + PrefixHash: hash("same-tokens"), Adapter: "", AdapterSet: true, + }) + + if got := lookupUnder(idx, "", "same-tokens"); len(got) != 0 { + t.Errorf("base entry survived its own eviction: %+v", got) + } + if got := lookupUnder(idx, "sql-lora", "same-tokens"); len(got) != 1 { + t.Errorf("sql-lora entry = %+v, want it untouched by the base-model eviction", got) + } +} + // An eviction with NO adapter keeps the original conservative behavior: it // sweeps every partition. That is what a pre-adapter producer emits, and for // such a producer all entries live in the "" partition anyway — so the diff --git a/pkg/index/index.go b/pkg/index/index.go index e227cc7c..1fcddd74 100644 --- a/pkg/index/index.go +++ b/pkg/index/index.go @@ -243,14 +243,17 @@ type Event struct { Model string Tenant string PrefixHash []byte - // Adapter narrows a PREFIX_EVICTED removal to one adapter partition. Empty - // removes the prefix from EVERY adapter partition — the conservative legacy - // behavior, and identical to the pre-adapter code for producers that never - // set it (all of whose entries live in the "" partition anyway). Ignored by - // ALL_CLEARED (a flush clears the replica across adapters) and by - // REPLICA_UPDATED (liveness is adapter-independent). - Adapter string - Timestamp time.Time + // Adapter narrows a PREFIX_EVICTED removal to one adapter partition, but only + // when AdapterSet reports the producer supplied it (proto3 presence). With + // AdapterSet, the removal targets exactly Adapter — including "" for a + // base-model eviction, which then does NOT sweep live LoRA hints for the same + // token hash. Without AdapterSet (a legacy producer that never populated the + // field), the removal sweeps EVERY adapter partition — the conservative legacy + // behavior. Both are ignored by ALL_CLEARED (a flush clears the replica across + // adapters) and by REPLICA_UPDATED (liveness is adapter-independent). + Adapter string + AdapterSet bool + Timestamp time.Time } // LookupRequest asks which replicas hold a given prefix, within a hash scheme. @@ -898,22 +901,18 @@ func (i *Index) ApplyEvent(ev Event) { // adapters at once on the same replica; dropping every partition for one // adapter's GPU eviction would throw away hints that are still valid. // - // An empty ev.Adapter falls back to the original cross-partition sweep. - // That is exact for a pre-adapter producer (all its entries are in the "" - // partition anyway), but an adapter-aware producer ALSO emits "" for a - // genuine base-model eviction — and the sweep then drops live LoRA- - // partition hints for the same prefix hash too. That is conservative soft - // state (at worst a cache miss, re-added by the authoritative - // ReportCacheState path) and a strict non-regression (pre-partition, base - // and LoRA shared one partition, so a base eviction was already total). - // Making "" mean the base partition only, without breaking the legacy - // wildcard, needs explicit adapter_id presence on the event — a tracked - // follow-up. + // Presence (ev.AdapterSet), not emptiness, decides the scope. An + // adapter-aware producer sets ev.Adapter — including "" for a genuine + // base-model eviction — so the removal targets exactly that one partition + // and a base eviction no longer sweeps live LoRA hints for the same hash. + // A legacy producer that never populated the field leaves AdapterSet false, + // and the removal falls back to the original cross-partition sweep (exact + // for such a producer, whose entries all live in the "" partition anyway). for key, replicas := range i.prefixes { if key.tenant != ev.Tenant || key.model != ev.Model || key.prefixHash != hash { continue } - if ev.Adapter != "" && key.adapter != ev.Adapter { + if ev.AdapterSet && key.adapter != ev.Adapter { continue } i.removeReplicaLocked(key, replicas, ev.ReplicaID) diff --git a/pkg/server/inferencecache_service.go b/pkg/server/inferencecache_service.go index a283b49c..389224af 100644 --- a/pkg/server/inferencecache_service.go +++ b/pkg/server/inferencecache_service.go @@ -1034,10 +1034,14 @@ func (s *inferenceCacheService) PublishEvent(_ context.Context, ev *icpb.CacheEv Model: ev.GetModelId(), Tenant: ev.GetTenantId(), PrefixHash: ev.GetPrefixHash(), - // Narrows a PREFIX_EVICTED to one adapter partition; empty keeps the - // conservative cross-partition removal (legacy producers). - Adapter: ev.GetAdapterId(), - Timestamp: microsToTime(ev.GetTimestampUs()), + // Narrows a PREFIX_EVICTED to one adapter partition when the producer + // supplied adapter_id (proto3 presence) — including "" for a base-model + // eviction. Absent keeps the conservative cross-partition removal + // (legacy producers). GetAdapterId() flattens nil to "", so read the + // pointer directly for presence. + Adapter: ev.GetAdapterId(), + AdapterSet: ev.AdapterId != nil, + Timestamp: microsToTime(ev.GetTimestampUs()), }) } return &icpb.Ack{Accepted: true}, nil diff --git a/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go b/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go index c4f330e7..0f54676d 100644 --- a/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go +++ b/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go @@ -1307,15 +1307,19 @@ type CacheEvent struct { TenantId string `protobuf:"bytes,4,opt,name=tenant_id,json=tenantId,proto3" json:"tenant_id,omitempty"` PrefixHash []byte `protobuf:"bytes,5,opt,name=prefix_hash,json=prefixHash,proto3" json:"prefix_hash,omitempty"` TimestampUs int64 `protobuf:"varint,6,opt,name=timestamp_us,json=timestampUs,proto3" json:"timestamp_us,omitempty"` - // Adapter partition a PREFIX_EVICTED removal targets. Set, the server drops - // the prefix only from that partition; empty, it drops the prefix across - // EVERY adapter partition — the conservative legacy behavior (removal is - // soft state: an over-eager drop costs a cache miss, never a wrong answer), - // and identical to today for producers that never populate the field. - // Ignored by ALL_CLEARED (a cache flush clears the replica across every - // adapter) and by REPLICA_UPDATED (liveness is adapter-independent). - // Additive, v1alpha1-compatible. - AdapterId string `protobuf:"bytes,7,opt,name=adapter_id,json=adapterId,proto3" json:"adapter_id,omitempty"` + // Adapter partition a PREFIX_EVICTED removal targets, with explicit presence. + // PRESENT (even "") drops the prefix from exactly that ONE partition ("" is the + // base-model / non-LoRA partition); ABSENT drops it across EVERY adapter + // partition — the conservative legacy sweep, kept for producers that never + // populate the field. Presence is what lets an adapter-aware producer evict the + // base ("") partition WITHOUT also sweeping live LoRA hints for the same + // token-derived hash. Removal is soft state: an over-eager drop costs a cache + // miss, never a wrong answer. Ignored by ALL_CLEARED (a cache flush clears the + // replica across every adapter) and by REPLICA_UPDATED (liveness is + // adapter-independent). Additive, v1alpha1-compatible (proto3 presence uses the + // same wire format; an old producer that never set the field still reads as + // ABSENT → legacy sweep). + AdapterId *string `protobuf:"bytes,7,opt,name=adapter_id,json=adapterId,proto3,oneof" json:"adapter_id,omitempty"` unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache } @@ -1393,8 +1397,8 @@ func (x *CacheEvent) GetTimestampUs() int64 { } func (x *CacheEvent) GetAdapterId() string { - if x != nil { - return x.AdapterId + if x != nil && x.AdapterId != nil { + return *x.AdapterId } return "" } @@ -1824,7 +1828,7 @@ var file_inferencecache_v1alpha1_inferencecache_proto_rawDesc = string([]byte{ 0x20, 0x01, 0x28, 0x0b, 0x32, 0x25, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x75, 0x6d, 0x6d, 0x61, 0x72, 0x79, 0x52, 0x07, 0x73, 0x75, 0x6d, - 0x6d, 0x61, 0x72, 0x79, 0x22, 0xee, 0x02, 0x0a, 0x0a, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, + 0x6d, 0x61, 0x72, 0x79, 0x22, 0x82, 0x03, 0x0a, 0x0a, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x12, 0x3c, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x28, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, @@ -1838,113 +1842,115 @@ var file_inferencecache_v1alpha1_inferencecache_proto_rawDesc = string([]byte{ 0x69, 0x78, 0x5f, 0x68, 0x61, 0x73, 0x68, 0x18, 0x05, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x0a, 0x70, 0x72, 0x65, 0x66, 0x69, 0x78, 0x48, 0x61, 0x73, 0x68, 0x12, 0x21, 0x0a, 0x0c, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x5f, 0x75, 0x73, 0x18, 0x06, 0x20, 0x01, 0x28, 0x03, 0x52, - 0x0b, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x55, 0x73, 0x12, 0x1d, 0x0a, 0x0a, + 0x0b, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x55, 0x73, 0x12, 0x22, 0x0a, 0x0a, 0x61, 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x18, 0x07, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x09, 0x61, 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x49, 0x64, 0x22, 0x68, 0x0a, 0x04, 0x54, - 0x79, 0x70, 0x65, 0x12, 0x14, 0x0a, 0x10, 0x54, 0x59, 0x50, 0x45, 0x5f, 0x55, 0x4e, 0x53, 0x50, - 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12, 0x10, 0x0a, 0x0c, 0x50, 0x52, 0x45, - 0x46, 0x49, 0x58, 0x5f, 0x41, 0x44, 0x44, 0x45, 0x44, 0x10, 0x01, 0x12, 0x12, 0x0a, 0x0e, 0x50, - 0x52, 0x45, 0x46, 0x49, 0x58, 0x5f, 0x45, 0x56, 0x49, 0x43, 0x54, 0x45, 0x44, 0x10, 0x02, 0x12, - 0x13, 0x0a, 0x0f, 0x52, 0x45, 0x50, 0x4c, 0x49, 0x43, 0x41, 0x5f, 0x55, 0x50, 0x44, 0x41, 0x54, - 0x45, 0x44, 0x10, 0x03, 0x12, 0x0f, 0x0a, 0x0b, 0x41, 0x4c, 0x4c, 0x5f, 0x43, 0x4c, 0x45, 0x41, - 0x52, 0x45, 0x44, 0x10, 0x04, 0x22, 0x8d, 0x01, 0x0a, 0x13, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, - 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x19, 0x0a, - 0x08, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, - 0x07, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x49, 0x64, 0x12, 0x1b, 0x0a, 0x09, 0x74, 0x65, 0x6e, 0x61, - 0x6e, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x74, 0x65, 0x6e, - 0x61, 0x6e, 0x74, 0x49, 0x64, 0x12, 0x3e, 0x0a, 0x05, 0x74, 0x79, 0x70, 0x65, 0x73, 0x18, 0x03, - 0x20, 0x03, 0x28, 0x0e, 0x32, 0x28, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, - 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, - 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x2e, 0x54, 0x79, 0x70, 0x65, 0x52, 0x05, - 0x74, 0x79, 0x70, 0x65, 0x73, 0x22, 0x2c, 0x0a, 0x14, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, - 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x14, 0x0a, - 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x09, 0x52, 0x05, 0x6e, 0x61, - 0x6d, 0x65, 0x73, 0x22, 0xe9, 0x01, 0x0a, 0x06, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x12, 0x12, - 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, - 0x6d, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x43, 0x0a, 0x06, 0x6c, 0x61, 0x62, 0x65, 0x6c, 0x73, - 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x2b, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, - 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, - 0x2e, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x2e, 0x4c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x45, 0x6e, - 0x74, 0x72, 0x79, 0x52, 0x06, 0x6c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x12, 0x14, 0x0a, 0x05, 0x76, - 0x61, 0x6c, 0x75, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, 0x01, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, - 0x65, 0x12, 0x21, 0x0a, 0x0c, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x5f, 0x75, - 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x03, 0x52, 0x0b, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, - 0x6d, 0x70, 0x55, 0x73, 0x1a, 0x39, 0x0a, 0x0b, 0x4c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x45, 0x6e, - 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, - 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x22, - 0x42, 0x0a, 0x03, 0x41, 0x63, 0x6b, 0x12, 0x1a, 0x0a, 0x08, 0x61, 0x63, 0x63, 0x65, 0x70, 0x74, - 0x65, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x08, 0x52, 0x08, 0x61, 0x63, 0x63, 0x65, 0x70, 0x74, - 0x65, 0x64, 0x12, 0x1f, 0x0a, 0x0b, 0x72, 0x65, 0x61, 0x73, 0x6f, 0x6e, 0x5f, 0x63, 0x6f, 0x64, - 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, 0x72, 0x65, 0x61, 0x73, 0x6f, 0x6e, 0x43, - 0x6f, 0x64, 0x65, 0x2a, 0x60, 0x0a, 0x09, 0x43, 0x61, 0x63, 0x68, 0x65, 0x54, 0x69, 0x65, 0x72, - 0x12, 0x1a, 0x0a, 0x16, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x55, - 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12, 0x11, 0x0a, 0x0d, - 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, 0x31, 0x10, 0x01, 0x12, - 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, 0x32, - 0x10, 0x02, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, - 0x5f, 0x54, 0x33, 0x10, 0x03, 0x32, 0xcc, 0x06, 0x0a, 0x0e, 0x49, 0x6e, 0x66, 0x65, 0x72, 0x65, - 0x6e, 0x63, 0x65, 0x43, 0x61, 0x63, 0x68, 0x65, 0x12, 0x71, 0x0a, 0x0e, 0x52, 0x65, 0x6e, 0x64, - 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x12, 0x2e, 0x2e, 0x69, 0x6e, 0x66, + 0x48, 0x00, 0x52, 0x09, 0x61, 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x49, 0x64, 0x88, 0x01, 0x01, + 0x22, 0x68, 0x0a, 0x04, 0x54, 0x79, 0x70, 0x65, 0x12, 0x14, 0x0a, 0x10, 0x54, 0x59, 0x50, 0x45, + 0x5f, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12, 0x10, + 0x0a, 0x0c, 0x50, 0x52, 0x45, 0x46, 0x49, 0x58, 0x5f, 0x41, 0x44, 0x44, 0x45, 0x44, 0x10, 0x01, + 0x12, 0x12, 0x0a, 0x0e, 0x50, 0x52, 0x45, 0x46, 0x49, 0x58, 0x5f, 0x45, 0x56, 0x49, 0x43, 0x54, + 0x45, 0x44, 0x10, 0x02, 0x12, 0x13, 0x0a, 0x0f, 0x52, 0x45, 0x50, 0x4c, 0x49, 0x43, 0x41, 0x5f, + 0x55, 0x50, 0x44, 0x41, 0x54, 0x45, 0x44, 0x10, 0x03, 0x12, 0x0f, 0x0a, 0x0b, 0x41, 0x4c, 0x4c, + 0x5f, 0x43, 0x4c, 0x45, 0x41, 0x52, 0x45, 0x44, 0x10, 0x04, 0x42, 0x0d, 0x0a, 0x0b, 0x5f, 0x61, + 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x22, 0x8d, 0x01, 0x0a, 0x13, 0x53, 0x74, + 0x72, 0x65, 0x61, 0x6d, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, + 0x74, 0x12, 0x19, 0x0a, 0x08, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, + 0x01, 0x28, 0x09, 0x52, 0x07, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x49, 0x64, 0x12, 0x1b, 0x0a, 0x09, + 0x74, 0x65, 0x6e, 0x61, 0x6e, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, + 0x08, 0x74, 0x65, 0x6e, 0x61, 0x6e, 0x74, 0x49, 0x64, 0x12, 0x3e, 0x0a, 0x05, 0x74, 0x79, 0x70, + 0x65, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0e, 0x32, 0x28, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, + 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, + 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x2e, 0x54, 0x79, + 0x70, 0x65, 0x52, 0x05, 0x74, 0x79, 0x70, 0x65, 0x73, 0x22, 0x2c, 0x0a, 0x14, 0x53, 0x74, 0x72, + 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, + 0x74, 0x12, 0x14, 0x0a, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x09, + 0x52, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x22, 0xe9, 0x01, 0x0a, 0x06, 0x4d, 0x65, 0x74, 0x72, + 0x69, 0x63, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, + 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x02, + 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x43, 0x0a, 0x06, 0x6c, 0x61, + 0x62, 0x65, 0x6c, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x2b, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, - 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, - 0x61, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2f, 0x2e, 0x69, 0x6e, 0x66, + 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x2e, 0x4c, 0x61, 0x62, 0x65, + 0x6c, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x06, 0x6c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x12, + 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, 0x01, 0x52, 0x05, + 0x76, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x21, 0x0a, 0x0c, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, + 0x6d, 0x70, 0x5f, 0x75, 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x03, 0x52, 0x0b, 0x74, 0x69, 0x6d, + 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x55, 0x73, 0x1a, 0x39, 0x0a, 0x0b, 0x4c, 0x61, 0x62, 0x65, + 0x6c, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, + 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, + 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, + 0x02, 0x38, 0x01, 0x22, 0x42, 0x0a, 0x03, 0x41, 0x63, 0x6b, 0x12, 0x1a, 0x0a, 0x08, 0x61, 0x63, + 0x63, 0x65, 0x70, 0x74, 0x65, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x08, 0x52, 0x08, 0x61, 0x63, + 0x63, 0x65, 0x70, 0x74, 0x65, 0x64, 0x12, 0x1f, 0x0a, 0x0b, 0x72, 0x65, 0x61, 0x73, 0x6f, 0x6e, + 0x5f, 0x63, 0x6f, 0x64, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, 0x72, 0x65, 0x61, + 0x73, 0x6f, 0x6e, 0x43, 0x6f, 0x64, 0x65, 0x2a, 0x60, 0x0a, 0x09, 0x43, 0x61, 0x63, 0x68, 0x65, + 0x54, 0x69, 0x65, 0x72, 0x12, 0x1a, 0x0a, 0x16, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, + 0x45, 0x52, 0x5f, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, + 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, + 0x31, 0x10, 0x01, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, + 0x52, 0x5f, 0x54, 0x32, 0x10, 0x02, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, + 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, 0x33, 0x10, 0x03, 0x32, 0xcc, 0x06, 0x0a, 0x0e, 0x49, 0x6e, + 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x43, 0x61, 0x63, 0x68, 0x65, 0x12, 0x71, 0x0a, 0x0e, + 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x12, 0x2e, + 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, + 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, + 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2f, + 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, + 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, + 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, + 0x68, 0x0a, 0x0b, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x12, 0x2b, + 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, + 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, + 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2c, 0x2e, 0x69, 0x6e, + 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, + 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, + 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, 0x4c, 0x6f, 0x6f, + 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, - 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, - 0x61, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x68, 0x0a, 0x0b, 0x4c, - 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x12, 0x2b, 0x2e, 0x69, 0x6e, 0x66, + 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, + 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, 0x6e, 0x66, 0x65, + 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, + 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, 0x74, + 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, 0x47, 0x65, 0x74, + 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, - 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, 0x65, - 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, - 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, - 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x73, - 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, - 0x44, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, - 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, - 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, - 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, - 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, - 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x73, - 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, - 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, + 0x70, 0x68, 0x61, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, + 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, 0x6e, 0x66, 0x65, + 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, + 0x68, 0x61, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, + 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x5d, 0x0a, 0x10, 0x52, 0x65, 0x70, + 0x6f, 0x72, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x12, 0x29, 0x2e, + 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, + 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, + 0x74, 0x65, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x1a, 0x1c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, + 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, + 0x61, 0x31, 0x2e, 0x41, 0x63, 0x6b, 0x28, 0x01, 0x12, 0x51, 0x0a, 0x0c, 0x50, 0x75, 0x62, 0x6c, + 0x69, 0x73, 0x68, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x12, 0x23, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, + 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, + 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x1a, 0x1c, 0x2e, + 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, + 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x41, 0x63, 0x6b, 0x12, 0x68, 0x0a, 0x11, 0x53, + 0x74, 0x72, 0x65, 0x61, 0x6d, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, + 0x12, 0x2c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, + 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x53, 0x74, 0x72, 0x65, 0x61, + 0x6d, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x23, + 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, + 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, + 0x65, 0x6e, 0x74, 0x30, 0x01, 0x12, 0x61, 0x0a, 0x0d, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, + 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, - 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x52, 0x65, - 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, + 0x2e, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x52, 0x65, + 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, - 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x52, 0x65, 0x73, - 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x5d, 0x0a, 0x10, 0x52, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x43, - 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x12, 0x29, 0x2e, 0x69, 0x6e, 0x66, 0x65, - 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, - 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x55, 0x70, - 0x64, 0x61, 0x74, 0x65, 0x1a, 0x1c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, - 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x41, - 0x63, 0x6b, 0x28, 0x01, 0x12, 0x51, 0x0a, 0x0c, 0x50, 0x75, 0x62, 0x6c, 0x69, 0x73, 0x68, 0x45, - 0x76, 0x65, 0x6e, 0x74, 0x12, 0x23, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, - 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, - 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x1a, 0x1c, 0x2e, 0x69, 0x6e, 0x66, 0x65, - 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, - 0x68, 0x61, 0x31, 0x2e, 0x41, 0x63, 0x6b, 0x12, 0x68, 0x0a, 0x11, 0x53, 0x74, 0x72, 0x65, 0x61, - 0x6d, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x12, 0x2c, 0x2e, 0x69, - 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, - 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x45, 0x76, 0x65, - 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x23, 0x2e, 0x69, 0x6e, 0x66, - 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, - 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x30, - 0x01, 0x12, 0x61, 0x0a, 0x0d, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, - 0x63, 0x73, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, - 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x53, 0x74, 0x72, - 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, - 0x74, 0x1a, 0x1f, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, - 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4d, 0x65, 0x74, 0x72, - 0x69, 0x63, 0x30, 0x01, 0x42, 0x6f, 0x5a, 0x6d, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, - 0x6f, 0x6d, 0x2f, 0x63, 0x61, 0x63, 0x68, 0x65, 0x62, 0x6f, 0x78, 0x2d, 0x70, 0x72, 0x6f, 0x6a, - 0x65, 0x63, 0x74, 0x2f, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x2d, 0x63, 0x61, - 0x63, 0x68, 0x65, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x2f, 0x70, - 0x72, 0x6f, 0x74, 0x6f, 0x2f, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, - 0x63, 0x68, 0x65, 0x2f, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x3b, 0x69, 0x6e, 0x66, - 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x76, 0x31, 0x61, 0x6c, 0x70, - 0x68, 0x61, 0x31, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, + 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x30, 0x01, 0x42, 0x6f, 0x5a, 0x6d, 0x67, 0x69, 0x74, 0x68, + 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x63, 0x61, 0x63, 0x68, 0x65, 0x62, 0x6f, 0x78, 0x2d, + 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x2f, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, + 0x65, 0x2d, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x73, 0x65, 0x72, 0x76, + 0x65, 0x72, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2f, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, + 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2f, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, + 0x3b, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x76, + 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, + 0x33, }) var ( @@ -2027,6 +2033,7 @@ func file_inferencecache_v1alpha1_inferencecache_proto_init() { if File_inferencecache_v1alpha1_inferencecache_proto != nil { return } + file_inferencecache_v1alpha1_inferencecache_proto_msgTypes[14].OneofWrappers = []any{} type x struct{} out := protoimpl.TypeBuilder{ File: protoimpl.DescBuilder{ diff --git a/pkg/server/server_test.go b/pkg/server/server_test.go index af3428b0..84095a17 100644 --- a/pkg/server/server_test.go +++ b/pkg/server/server_test.go @@ -1454,6 +1454,41 @@ func TestPublishEventAppliesToIndex(t *testing.T) { } } +// TestPublishEventBaseEvictionWithPresenceSparesLoRA pins the full wire→index +// seam: a base-model PREFIX_EVICTED carrying adapter_id="" WITH proto3 presence +// drops ONLY the base ("") partition. The service must translate wire presence +// (ev.AdapterId != nil) into index.Event.AdapterSet — reading the VALUE the way +// GetAdapterId does would misclassify a present-but-empty adapter as the legacy +// wildcard and wrongly sweep the co-resident LoRA hint for the same hash. +func TestPublishEventBaseEvictionWithPresenceSparesLoRA(t *testing.T) { + svc := newTestService() + for _, a := range []string{"", "sql-lora"} { + svc.index.Ingest(index.Update{ + ReplicaID: "replica-0", Model: "m", Tenant: "t", HashScheme: "vllm", + Adapter: a, + Prefixes: []index.PrefixRef{{PrefixHash: []byte("p"), TokenCount: 10}}, + }) + } + emptyAdapter := "" // present, empty → base partition only (not the wildcard) + ack, err := svc.PublishEvent(context.Background(), &icpb.CacheEvent{ + Type: icpb.CacheEvent_PREFIX_EVICTED, ReplicaId: "replica-0", + ModelId: "m", TenantId: "t", PrefixHash: []byte("p"), + AdapterId: &emptyAdapter, + }) + if err != nil { + t.Fatalf("PublishEvent: %v", err) + } + if !ack.GetAccepted() { + t.Fatalf("ack.Accepted = false, want true") + } + if got := svc.index.Lookup(index.LookupRequest{Model: "m", Tenant: "t", HashScheme: "vllm", Adapter: "", PrefixHash: []byte("p")}); len(got) != 0 { + t.Errorf("base entry survived its own eviction: %+v", got) + } + if got := svc.index.Lookup(index.LookupRequest{Model: "m", Tenant: "t", HashScheme: "vllm", Adapter: "sql-lora", PrefixHash: []byte("p")}); len(got) != 1 { + t.Errorf("sql-lora entry = %+v, want it untouched by the base-model eviction", got) + } +} + func TestEventTypeFromProto(t *testing.T) { cases := map[icpb.CacheEvent_Type]index.EventType{ icpb.CacheEvent_PREFIX_ADDED: index.EventPrefixAdded, diff --git a/proto/inferencecache/v1alpha1/inferencecache.proto b/proto/inferencecache/v1alpha1/inferencecache.proto index 8e7e5fa5..536a5b8f 100644 --- a/proto/inferencecache/v1alpha1/inferencecache.proto +++ b/proto/inferencecache/v1alpha1/inferencecache.proto @@ -297,15 +297,19 @@ message CacheEvent { string tenant_id = 4; bytes prefix_hash = 5; int64 timestamp_us = 6; - // Adapter partition a PREFIX_EVICTED removal targets. Set, the server drops - // the prefix only from that partition; empty, it drops the prefix across - // EVERY adapter partition — the conservative legacy behavior (removal is - // soft state: an over-eager drop costs a cache miss, never a wrong answer), - // and identical to today for producers that never populate the field. - // Ignored by ALL_CLEARED (a cache flush clears the replica across every - // adapter) and by REPLICA_UPDATED (liveness is adapter-independent). - // Additive, v1alpha1-compatible. - string adapter_id = 7; + // Adapter partition a PREFIX_EVICTED removal targets, with explicit presence. + // PRESENT (even "") drops the prefix from exactly that ONE partition ("" is the + // base-model / non-LoRA partition); ABSENT drops it across EVERY adapter + // partition — the conservative legacy sweep, kept for producers that never + // populate the field. Presence is what lets an adapter-aware producer evict the + // base ("") partition WITHOUT also sweeping live LoRA hints for the same + // token-derived hash. Removal is soft state: an over-eager drop costs a cache + // miss, never a wrong answer. Ignored by ALL_CLEARED (a cache flush clears the + // replica across every adapter) and by REPLICA_UPDATED (liveness is + // adapter-independent). Additive, v1alpha1-compatible (proto3 presence uses the + // same wire format; an old producer that never set the field still reads as + // ABSENT → legacy sweep). + optional string adapter_id = 7; } message StreamEventsRequest { From b848057c96773597135b104f4bfcfdafdaf26c20 Mon Sep 17 00:00:00 2001 From: Ed Sun Date: Thu, 23 Jul 2026 20:17:29 -0700 Subject: [PATCH 2/4] gap-3: use an additive adapter_scoped bool, not optional adapter_id MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit buf breaking rejects changing CacheEvent.adapter_id from proto3 implicit to explicit presence (a cardinality change), and the repo gates on it. Carry the presence signal in a new, purely additive bool adapter_scoped = 8 instead: adapter_scoped=true means adapter_id is authoritative — the removal targets exactly that partition, including "" for a base-model eviction — while false (legacy producers) keeps the conservative cross-partition sweep. Index-side semantics (Event.AdapterSet) and all tests are unchanged; only the wire signal moves from a pointer to the bool. Also fixes the SpareLoRA->SparesLoRA test name and the grpc-contract PREFIX_EVICTED prose. buf breaking/lint, go test ./... pass. --- docs/design/grpc-contract.md | 6 +- pkg/adapters/engine/mapper.go | 25 +- pkg/adapters/engine/mapper_test.go | 16 +- pkg/index/adapter_partition_test.go | 2 +- pkg/server/inferencecache_service.go | 9 +- .../v1alpha1/inferencecache.pb.go | 251 +++++++++--------- pkg/server/server_test.go | 13 +- .../v1alpha1/inferencecache.proto | 30 ++- 8 files changed, 184 insertions(+), 168 deletions(-) diff --git a/docs/design/grpc-contract.md b/docs/design/grpc-contract.md index 8ef03ff7..9ce3874c 100644 --- a/docs/design/grpc-contract.md +++ b/docs/design/grpc-contract.md @@ -62,9 +62,9 @@ service InferenceCache { - **ReplicaStats** `{ string replica_id = 1; int64 cache_memory_bytes = 2; float hit_rate = 3; float pressure = 4; string client_version = 5; int64 t2_hit_tokens = 6; int64 t2_query_tokens = 7; }` `client_version` is an opaque version string identifying the client-side cache library the reporting replica is linked against (e.g. an LMCache client release). It carries no semver semantics on the wire — producers populate it, the server accepts it, no semver parsing or ordering is performed at this layer. Empty / unset is allowed and means "unknown"; older producers that don't fill the field MUST keep being accepted (additive, v1alpha1-compatible). The field reserves wire space for the producer half of an end-to-end client/server version-skew detection surface; the consumer half (server-side storage, per-CacheBackend exposure, the operator-visible status condition that closes the silent client/server mismatch class of bug) lands in a follow-up change and is intentionally out of scope for this contract update. `t2_hit_tokens` / `t2_query_tokens` are cumulative token counters for the **tier-2 (external offload, e.g. LMCache) cache**, sourced from the engine's `vllm:external_prefix_cache_{hits,queries}_total`. The server derives a presence-aware tier-2 hit-rate per CacheBackend (`status.indexParticipation.t2HitRate`): nil until the tier is exercised (`t2_query_tokens > 0`), then the query-weighted `hits/queries` ratio — so a value of `0` means tier-2 is configured but serving no reloads (a silently-degraded offload tier), distinct from "not yet used". Both default to 0 for producers that don't populate them (additive, v1alpha1-compatible). -- **CacheEvent** `{ Type type; string replica_id; string model_id; string tenant_id; bytes prefix_hash; int64 timestamp_us; optional string adapter_id = 7; }` +- **CacheEvent** `{ Type type; string replica_id; string model_id; string tenant_id; bytes prefix_hash; int64 timestamp_us; string adapter_id = 7; bool adapter_scoped = 8; }` Type ∈ `PREFIX_ADDED | PREFIX_EVICTED | REPLICA_UPDATED | ALL_CLEARED` - `adapter_id` narrows a `PREFIX_EVICTED` to one adapter partition; empty removes the prefix across **every** partition (the conservative legacy behavior — removal is soft state). Ignored by `ALL_CLEARED` (a flush clears the replica across adapters) and `REPLICA_UPDATED` (liveness is adapter-independent). + `adapter_id` narrows a `PREFIX_EVICTED` to one adapter partition **when `adapter_scoped` is set** — including `""` (the base-model partition), so a base-model eviction no longer sweeps live LoRA hints for the same hash. Without `adapter_scoped` (legacy producers) the prefix is removed across **every** partition (the conservative sweep — removal is soft state). Ignored by `ALL_CLEARED` (a flush clears the replica across adapters) and `REPLICA_UPDATED` (liveness is adapter-independent). - **Metric** `{ string name; string type; map labels; double value; int64 timestamp_us; }` (Prometheus `inferencecache_*`, tech spec §4.3) - **StreamEventsRequest** `{ string model_id; string tenant_id; repeated CacheEvent.Type types; }` - **StreamMetricsRequest** `{ repeated string names; }` @@ -268,7 +268,7 @@ The fix partitions the **index**, not the hash: - **The fingerprint construction is unchanged.** Adapter identity is never mixed into the hash, so `pkg/fingerprint`, its golden vectors, and every cross-language implementation (Go / Python / Rust) stay byte-for-byte as they were. The hash still answers "what content is this?"; the partition answers "whose KV is it?". - **Matching inside a partition is unchanged.** Exact-match and the block-chain longest-leading-run rule behave identically; the partition only bounds which entries are candidates. - `LookupRouteResponse.adapter_id` (5) echoes the partition the index was consulted in, so a caller can confirm the round trip. It is empty on the fail-open envelopes returned *before* the index is consulted (`TIMEOUT`, tokenizer fail-open, `POLICY_REQUIRES_CHAIN`). -- `CacheEvent.adapter_id` (7) narrows a `PREFIX_EVICTED` to one partition, keyed on **proto3 presence**. **Present** (even `""`, the base partition) drops the prefix from exactly that one partition, so one adapter's GPU eviction — including a base-model eviction — no longer discards another adapter's still-valid hint for the identical hash. **Absent** keeps the original conservative cross-partition sweep (legacy producers that never set the field). `optional` makes the two distinguishable on the wire without breaking older producers. +- `CacheEvent.adapter_id` (7) narrows a `PREFIX_EVICTED` to one partition, gated by the new `adapter_scoped` (8) flag. With `adapter_scoped` set, `adapter_id` — even `""`, the base partition — drops the prefix from exactly that one partition, so one adapter's GPU eviction (including a base-model eviction) no longer discards another adapter's still-valid hint for the identical hash. Without it (legacy producers) the original conservative cross-partition sweep is kept. A separate bool rather than making `adapter_id` `optional`, which the wire-compat gate (`buf breaking`) rejects as a cardinality change — the bool is purely additive. **Producer / consumer must agree, exactly as they already do for `hash_scheme`.** A lookup that sets `adapter_id` will not match entries ingested without one, and vice versa. Empty/unset is the **default partition**, and this is exactly the pre-adapter behavior **for base-model / non-LoRA traffic**: an engine serving no LoRA emits a nil `lora_id`, so ingest and lookup both land under `""` and are completely unaffected — no regression, no configuration required. **This is not a "single LoRA adapter just works" claim.** Only a *nil* `lora_id` uses `""`; *any* non-nil `lora_id` — including a deployment with exactly one adapter — must resolve to a **configured** `--lora-adapter-names` identity, otherwise it is dropped at ingest (**fail-closed**) rather than landing under `""` or a replica-local alias. So even one LoRA adapter requires the ingest identifier (the subscriber's resolved `adapter_id`) to match the gateway's query `adapter_id` end-to-end, or every lookup silently misses — the same agreement `hash_scheme` already demands. diff --git a/pkg/adapters/engine/mapper.go b/pkg/adapters/engine/mapper.go index a929f989..9c0c8b6f 100644 --- a/pkg/adapters/engine/mapper.go +++ b/pkg/adapters/engine/mapper.go @@ -89,19 +89,20 @@ func (c Config) ClearedEvent(tsSeconds float64) *icpb.CacheEvent { // EvictedEvent builds one PREFIX_EVICTED CacheEvent for an already-derived prefix // hash (our content fingerprint, the index key to drop) in a specific adapter // partition. The subscriber maps an evicted engine block hash to both via -// positionalIndex.Removed. adapterID is always set WITH presence (even "" for the -// base-model partition), so the server drops the prefix from exactly that one -// partition — a base-model eviction never sweeps live LoRA hints for the same -// token hash. (A producer that omits the field entirely reads as absent → the -// server's conservative cross-partition legacy sweep.) +// positionalIndex.Removed. adapter_scoped is always set, so the server drops the +// prefix from exactly the adapterID partition (even "" for the base model) — a +// base-model eviction never sweeps live LoRA hints for the same token hash. +// (A producer that leaves adapter_scoped false gets the conservative +// cross-partition legacy sweep.) func (c Config) EvictedEvent(prefixHash []byte, adapterID string, tsSeconds float64) *icpb.CacheEvent { return &icpb.CacheEvent{ - Type: icpb.CacheEvent_PREFIX_EVICTED, - ReplicaId: c.ReplicaID, - ModelId: c.ModelID, - TenantId: c.TenantID, - PrefixHash: prefixHash, - AdapterId: &adapterID, - TimestampUs: microsFromSeconds(tsSeconds), + Type: icpb.CacheEvent_PREFIX_EVICTED, + ReplicaId: c.ReplicaID, + ModelId: c.ModelID, + TenantId: c.TenantID, + PrefixHash: prefixHash, + AdapterId: adapterID, + AdapterScoped: true, + TimestampUs: microsFromSeconds(tsSeconds), } } diff --git a/pkg/adapters/engine/mapper_test.go b/pkg/adapters/engine/mapper_test.go index ead98e91..c3c46581 100644 --- a/pkg/adapters/engine/mapper_test.go +++ b/pkg/adapters/engine/mapper_test.go @@ -22,21 +22,23 @@ func TestEvictedEvent(t *testing.T) { if !bytes.Equal(e.PrefixHash, []byte{0x01, 0x02}) { t.Errorf("prefix hash = %x, want 0102", e.PrefixHash) } - // A base-model eviction carries adapter_id="" WITH presence, so the server - // drops only the base ("") partition instead of sweeping every adapter. - if e.AdapterId == nil { - t.Error("adapter id absent; a base eviction must set it WITH presence to scope removal to the base partition") - } else if got := e.GetAdapterId(); got != "" { + // A base-model eviction carries adapter_id="" but marks it authoritative + // (adapter_scoped), so the server drops only the base ("") partition instead + // of sweeping every adapter. + if got := e.GetAdapterId(); got != "" { t.Errorf("adapter id = %q, want \"\" (base partition)", got) } + if !e.GetAdapterScoped() { + t.Error("adapter_scoped = false; a base eviction must mark adapter_id authoritative to scope removal to the base partition") + } } // An adapter-scoped eviction must name its partition, so the server drops the // prefix only there — the same hash can be live under another adapter. func TestEvictedEventCarriesAdapter(t *testing.T) { e := testConfig().EvictedEvent([]byte{0x01}, "sql-lora", 1.0) - if e.AdapterId == nil || e.GetAdapterId() != "sql-lora" { - t.Errorf("adapter id = %v, want sql-lora (present)", e.AdapterId) + if e.GetAdapterId() != "sql-lora" || !e.GetAdapterScoped() { + t.Errorf("adapter id = %q scoped = %v, want sql-lora / true", e.GetAdapterId(), e.GetAdapterScoped()) } } diff --git a/pkg/index/adapter_partition_test.go b/pkg/index/adapter_partition_test.go index 834c0401..4163dd63 100644 --- a/pkg/index/adapter_partition_test.go +++ b/pkg/index/adapter_partition_test.go @@ -202,7 +202,7 @@ func TestAdapterPartitionEvictionIsScopedToItsAdapter(t *testing.T) { // A base-model eviction (Adapter "" WITH presence) drops ONLY the base partition // and must not sweep a live LoRA hint for the same token hash — the over-sweep // that adapter_id presence exists to fix. -func TestAdapterPartitionBaseEvictionWithPresenceSpareLoRA(t *testing.T) { +func TestAdapterPartitionBaseEvictionWithPresenceSparesLoRA(t *testing.T) { idx := New() ingestUnder(idx, "replica-0", "", "same-tokens") ingestUnder(idx, "replica-0", "sql-lora", "same-tokens") diff --git a/pkg/server/inferencecache_service.go b/pkg/server/inferencecache_service.go index 389224af..02ae043c 100644 --- a/pkg/server/inferencecache_service.go +++ b/pkg/server/inferencecache_service.go @@ -1035,12 +1035,11 @@ func (s *inferenceCacheService) PublishEvent(_ context.Context, ev *icpb.CacheEv Tenant: ev.GetTenantId(), PrefixHash: ev.GetPrefixHash(), // Narrows a PREFIX_EVICTED to one adapter partition when the producer - // supplied adapter_id (proto3 presence) — including "" for a base-model - // eviction. Absent keeps the conservative cross-partition removal - // (legacy producers). GetAdapterId() flattens nil to "", so read the - // pointer directly for presence. + // marked adapter_id authoritative (adapter_scoped) — including "" for a + // base-model eviction. Unset keeps the conservative cross-partition + // removal (legacy producers). Adapter: ev.GetAdapterId(), - AdapterSet: ev.AdapterId != nil, + AdapterSet: ev.GetAdapterScoped(), Timestamp: microsToTime(ev.GetTimestampUs()), }) } diff --git a/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go b/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go index 0f54676d..56ea9565 100644 --- a/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go +++ b/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go @@ -1307,19 +1307,23 @@ type CacheEvent struct { TenantId string `protobuf:"bytes,4,opt,name=tenant_id,json=tenantId,proto3" json:"tenant_id,omitempty"` PrefixHash []byte `protobuf:"bytes,5,opt,name=prefix_hash,json=prefixHash,proto3" json:"prefix_hash,omitempty"` TimestampUs int64 `protobuf:"varint,6,opt,name=timestamp_us,json=timestampUs,proto3" json:"timestamp_us,omitempty"` - // Adapter partition a PREFIX_EVICTED removal targets, with explicit presence. - // PRESENT (even "") drops the prefix from exactly that ONE partition ("" is the - // base-model / non-LoRA partition); ABSENT drops it across EVERY adapter - // partition — the conservative legacy sweep, kept for producers that never - // populate the field. Presence is what lets an adapter-aware producer evict the - // base ("") partition WITHOUT also sweeping live LoRA hints for the same - // token-derived hash. Removal is soft state: an over-eager drop costs a cache - // miss, never a wrong answer. Ignored by ALL_CLEARED (a cache flush clears the - // replica across every adapter) and by REPLICA_UPDATED (liveness is - // adapter-independent). Additive, v1alpha1-compatible (proto3 presence uses the - // same wire format; an old producer that never set the field still reads as - // ABSENT → legacy sweep). - AdapterId *string `protobuf:"bytes,7,opt,name=adapter_id,json=adapterId,proto3,oneof" json:"adapter_id,omitempty"` + // Adapter partition a PREFIX_EVICTED removal targets, honored only when + // adapter_scoped is set. "" is the base-model / non-LoRA partition. Ignored by + // ALL_CLEARED (a cache flush clears the replica across every adapter) and by + // REPLICA_UPDATED (liveness is adapter-independent). Additive, + // v1alpha1-compatible. + AdapterId string `protobuf:"bytes,7,opt,name=adapter_id,json=adapterId,proto3" json:"adapter_id,omitempty"` + // adapter_scoped marks adapter_id as authoritative for a PREFIX_EVICTED: the + // removal targets exactly the adapter_id partition — INCLUDING "" for a + // base-model eviction — instead of sweeping every partition. An adapter-aware + // producer always sets it; false (the default, and what a legacy producer + // sends) keeps the conservative cross-partition sweep. This is the presence + // signal that lets a base-model eviction ("") avoid discarding live LoRA hints + // for the same token-derived hash. Removal is soft state: an over-eager drop + // costs a cache miss, never a wrong answer. A new bool field rather than + // making adapter_id `optional`, which the wire-compat gate rejects as a + // cardinality change. Additive; ignored outside PREFIX_EVICTED. + AdapterScoped bool `protobuf:"varint,8,opt,name=adapter_scoped,json=adapterScoped,proto3" json:"adapter_scoped,omitempty"` unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache } @@ -1397,12 +1401,19 @@ func (x *CacheEvent) GetTimestampUs() int64 { } func (x *CacheEvent) GetAdapterId() string { - if x != nil && x.AdapterId != nil { - return *x.AdapterId + if x != nil { + return x.AdapterId } return "" } +func (x *CacheEvent) GetAdapterScoped() bool { + if x != nil { + return x.AdapterScoped + } + return false +} + type StreamEventsRequest struct { state protoimpl.MessageState `protogen:"open.v1"` ModelId string `protobuf:"bytes,1,opt,name=model_id,json=modelId,proto3" json:"model_id,omitempty"` @@ -1828,7 +1839,7 @@ var file_inferencecache_v1alpha1_inferencecache_proto_rawDesc = string([]byte{ 0x20, 0x01, 0x28, 0x0b, 0x32, 0x25, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x75, 0x6d, 0x6d, 0x61, 0x72, 0x79, 0x52, 0x07, 0x73, 0x75, 0x6d, - 0x6d, 0x61, 0x72, 0x79, 0x22, 0x82, 0x03, 0x0a, 0x0a, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, + 0x6d, 0x61, 0x72, 0x79, 0x22, 0x95, 0x03, 0x0a, 0x0a, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x12, 0x3c, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x28, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, @@ -1842,115 +1853,116 @@ var file_inferencecache_v1alpha1_inferencecache_proto_rawDesc = string([]byte{ 0x69, 0x78, 0x5f, 0x68, 0x61, 0x73, 0x68, 0x18, 0x05, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x0a, 0x70, 0x72, 0x65, 0x66, 0x69, 0x78, 0x48, 0x61, 0x73, 0x68, 0x12, 0x21, 0x0a, 0x0c, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x5f, 0x75, 0x73, 0x18, 0x06, 0x20, 0x01, 0x28, 0x03, 0x52, - 0x0b, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x55, 0x73, 0x12, 0x22, 0x0a, 0x0a, + 0x0b, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x55, 0x73, 0x12, 0x1d, 0x0a, 0x0a, 0x61, 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x18, 0x07, 0x20, 0x01, 0x28, 0x09, - 0x48, 0x00, 0x52, 0x09, 0x61, 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x49, 0x64, 0x88, 0x01, 0x01, - 0x22, 0x68, 0x0a, 0x04, 0x54, 0x79, 0x70, 0x65, 0x12, 0x14, 0x0a, 0x10, 0x54, 0x59, 0x50, 0x45, - 0x5f, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12, 0x10, - 0x0a, 0x0c, 0x50, 0x52, 0x45, 0x46, 0x49, 0x58, 0x5f, 0x41, 0x44, 0x44, 0x45, 0x44, 0x10, 0x01, - 0x12, 0x12, 0x0a, 0x0e, 0x50, 0x52, 0x45, 0x46, 0x49, 0x58, 0x5f, 0x45, 0x56, 0x49, 0x43, 0x54, - 0x45, 0x44, 0x10, 0x02, 0x12, 0x13, 0x0a, 0x0f, 0x52, 0x45, 0x50, 0x4c, 0x49, 0x43, 0x41, 0x5f, - 0x55, 0x50, 0x44, 0x41, 0x54, 0x45, 0x44, 0x10, 0x03, 0x12, 0x0f, 0x0a, 0x0b, 0x41, 0x4c, 0x4c, - 0x5f, 0x43, 0x4c, 0x45, 0x41, 0x52, 0x45, 0x44, 0x10, 0x04, 0x42, 0x0d, 0x0a, 0x0b, 0x5f, 0x61, - 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x22, 0x8d, 0x01, 0x0a, 0x13, 0x53, 0x74, - 0x72, 0x65, 0x61, 0x6d, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, - 0x74, 0x12, 0x19, 0x0a, 0x08, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, - 0x01, 0x28, 0x09, 0x52, 0x07, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x49, 0x64, 0x12, 0x1b, 0x0a, 0x09, - 0x74, 0x65, 0x6e, 0x61, 0x6e, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, - 0x08, 0x74, 0x65, 0x6e, 0x61, 0x6e, 0x74, 0x49, 0x64, 0x12, 0x3e, 0x0a, 0x05, 0x74, 0x79, 0x70, - 0x65, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0e, 0x32, 0x28, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, - 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, - 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x2e, 0x54, 0x79, - 0x70, 0x65, 0x52, 0x05, 0x74, 0x79, 0x70, 0x65, 0x73, 0x22, 0x2c, 0x0a, 0x14, 0x53, 0x74, 0x72, - 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, - 0x74, 0x12, 0x14, 0x0a, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x09, - 0x52, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x22, 0xe9, 0x01, 0x0a, 0x06, 0x4d, 0x65, 0x74, 0x72, - 0x69, 0x63, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x02, - 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x43, 0x0a, 0x06, 0x6c, 0x61, - 0x62, 0x65, 0x6c, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x2b, 0x2e, 0x69, 0x6e, 0x66, - 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, - 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x2e, 0x4c, 0x61, 0x62, 0x65, - 0x6c, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x06, 0x6c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x12, - 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, 0x01, 0x52, 0x05, - 0x76, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x21, 0x0a, 0x0c, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, - 0x6d, 0x70, 0x5f, 0x75, 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x03, 0x52, 0x0b, 0x74, 0x69, 0x6d, - 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x55, 0x73, 0x1a, 0x39, 0x0a, 0x0b, 0x4c, 0x61, 0x62, 0x65, - 0x6c, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, - 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, - 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, - 0x02, 0x38, 0x01, 0x22, 0x42, 0x0a, 0x03, 0x41, 0x63, 0x6b, 0x12, 0x1a, 0x0a, 0x08, 0x61, 0x63, - 0x63, 0x65, 0x70, 0x74, 0x65, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x08, 0x52, 0x08, 0x61, 0x63, - 0x63, 0x65, 0x70, 0x74, 0x65, 0x64, 0x12, 0x1f, 0x0a, 0x0b, 0x72, 0x65, 0x61, 0x73, 0x6f, 0x6e, - 0x5f, 0x63, 0x6f, 0x64, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, 0x72, 0x65, 0x61, - 0x73, 0x6f, 0x6e, 0x43, 0x6f, 0x64, 0x65, 0x2a, 0x60, 0x0a, 0x09, 0x43, 0x61, 0x63, 0x68, 0x65, - 0x54, 0x69, 0x65, 0x72, 0x12, 0x1a, 0x0a, 0x16, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, - 0x45, 0x52, 0x5f, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, - 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, - 0x31, 0x10, 0x01, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, - 0x52, 0x5f, 0x54, 0x32, 0x10, 0x02, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, - 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, 0x33, 0x10, 0x03, 0x32, 0xcc, 0x06, 0x0a, 0x0e, 0x49, 0x6e, - 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x43, 0x61, 0x63, 0x68, 0x65, 0x12, 0x71, 0x0a, 0x0e, - 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x12, 0x2e, - 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, - 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, - 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2f, - 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, - 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, - 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, - 0x68, 0x0a, 0x0b, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x12, 0x2b, + 0x52, 0x09, 0x61, 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x49, 0x64, 0x12, 0x25, 0x0a, 0x0e, 0x61, + 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x5f, 0x73, 0x63, 0x6f, 0x70, 0x65, 0x64, 0x18, 0x08, 0x20, + 0x01, 0x28, 0x08, 0x52, 0x0d, 0x61, 0x64, 0x61, 0x70, 0x74, 0x65, 0x72, 0x53, 0x63, 0x6f, 0x70, + 0x65, 0x64, 0x22, 0x68, 0x0a, 0x04, 0x54, 0x79, 0x70, 0x65, 0x12, 0x14, 0x0a, 0x10, 0x54, 0x59, + 0x50, 0x45, 0x5f, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, + 0x12, 0x10, 0x0a, 0x0c, 0x50, 0x52, 0x45, 0x46, 0x49, 0x58, 0x5f, 0x41, 0x44, 0x44, 0x45, 0x44, + 0x10, 0x01, 0x12, 0x12, 0x0a, 0x0e, 0x50, 0x52, 0x45, 0x46, 0x49, 0x58, 0x5f, 0x45, 0x56, 0x49, + 0x43, 0x54, 0x45, 0x44, 0x10, 0x02, 0x12, 0x13, 0x0a, 0x0f, 0x52, 0x45, 0x50, 0x4c, 0x49, 0x43, + 0x41, 0x5f, 0x55, 0x50, 0x44, 0x41, 0x54, 0x45, 0x44, 0x10, 0x03, 0x12, 0x0f, 0x0a, 0x0b, 0x41, + 0x4c, 0x4c, 0x5f, 0x43, 0x4c, 0x45, 0x41, 0x52, 0x45, 0x44, 0x10, 0x04, 0x22, 0x8d, 0x01, 0x0a, + 0x13, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, + 0x75, 0x65, 0x73, 0x74, 0x12, 0x19, 0x0a, 0x08, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x5f, 0x69, 0x64, + 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x49, 0x64, 0x12, + 0x1b, 0x0a, 0x09, 0x74, 0x65, 0x6e, 0x61, 0x6e, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, + 0x28, 0x09, 0x52, 0x08, 0x74, 0x65, 0x6e, 0x61, 0x6e, 0x74, 0x49, 0x64, 0x12, 0x3e, 0x0a, 0x05, + 0x74, 0x79, 0x70, 0x65, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0e, 0x32, 0x28, 0x2e, 0x69, 0x6e, + 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, + 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, + 0x2e, 0x54, 0x79, 0x70, 0x65, 0x52, 0x05, 0x74, 0x79, 0x70, 0x65, 0x73, 0x22, 0x2c, 0x0a, 0x14, + 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x52, 0x65, 0x71, + 0x75, 0x65, 0x73, 0x74, 0x12, 0x14, 0x0a, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x18, 0x01, 0x20, + 0x03, 0x28, 0x09, 0x52, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x22, 0xe9, 0x01, 0x0a, 0x06, 0x4d, + 0x65, 0x74, 0x72, 0x69, 0x63, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, + 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x74, 0x79, 0x70, + 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x43, 0x0a, + 0x06, 0x6c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x2b, 0x2e, + 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, + 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x2e, 0x4c, + 0x61, 0x62, 0x65, 0x6c, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x06, 0x6c, 0x61, 0x62, 0x65, + 0x6c, 0x73, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, + 0x01, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x21, 0x0a, 0x0c, 0x74, 0x69, 0x6d, 0x65, + 0x73, 0x74, 0x61, 0x6d, 0x70, 0x5f, 0x75, 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x03, 0x52, 0x0b, + 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x55, 0x73, 0x1a, 0x39, 0x0a, 0x0b, 0x4c, + 0x61, 0x62, 0x65, 0x6c, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, + 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, + 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, + 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x22, 0x42, 0x0a, 0x03, 0x41, 0x63, 0x6b, 0x12, 0x1a, 0x0a, + 0x08, 0x61, 0x63, 0x63, 0x65, 0x70, 0x74, 0x65, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x08, 0x52, + 0x08, 0x61, 0x63, 0x63, 0x65, 0x70, 0x74, 0x65, 0x64, 0x12, 0x1f, 0x0a, 0x0b, 0x72, 0x65, 0x61, + 0x73, 0x6f, 0x6e, 0x5f, 0x63, 0x6f, 0x64, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, + 0x72, 0x65, 0x61, 0x73, 0x6f, 0x6e, 0x43, 0x6f, 0x64, 0x65, 0x2a, 0x60, 0x0a, 0x09, 0x43, 0x61, + 0x63, 0x68, 0x65, 0x54, 0x69, 0x65, 0x72, 0x12, 0x1a, 0x0a, 0x16, 0x43, 0x41, 0x43, 0x48, 0x45, + 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, + 0x44, 0x10, 0x00, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, + 0x52, 0x5f, 0x54, 0x31, 0x10, 0x01, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, 0x48, 0x45, 0x5f, + 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, 0x32, 0x10, 0x02, 0x12, 0x11, 0x0a, 0x0d, 0x43, 0x41, 0x43, + 0x48, 0x45, 0x5f, 0x54, 0x49, 0x45, 0x52, 0x5f, 0x54, 0x33, 0x10, 0x03, 0x32, 0xcc, 0x06, 0x0a, + 0x0e, 0x49, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x43, 0x61, 0x63, 0x68, 0x65, 0x12, + 0x71, 0x0a, 0x0e, 0x52, 0x65, 0x6e, 0x64, 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, + 0x65, 0x12, 0x2e, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, + 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, + 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, + 0x74, 0x1a, 0x2f, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, + 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x52, 0x65, 0x6e, 0x64, + 0x65, 0x72, 0x54, 0x65, 0x6d, 0x70, 0x6c, 0x61, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, + 0x73, 0x65, 0x12, 0x68, 0x0a, 0x0b, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, + 0x65, 0x12, 0x2b, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, + 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, + 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, - 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2c, 0x2e, 0x69, 0x6e, + 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, + 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x12, 0x2d, 0x2e, + 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, + 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, + 0x52, 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, + 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, + 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, + 0x6f, 0x75, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, + 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x12, 0x2d, 0x2e, + 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, + 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, + 0x53, 0x74, 0x61, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, + 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, + 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, + 0x74, 0x61, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x5d, 0x0a, 0x10, + 0x52, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, + 0x12, 0x29, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, + 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, + 0x53, 0x74, 0x61, 0x74, 0x65, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x1a, 0x1c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, - 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x52, 0x6f, 0x75, 0x74, - 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, 0x4c, 0x6f, 0x6f, - 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, 0x74, 0x65, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, - 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, - 0x70, 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, - 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, 0x6e, 0x66, 0x65, - 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, - 0x68, 0x61, 0x31, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x50, 0x44, 0x52, 0x6f, 0x75, 0x74, - 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x6e, 0x0a, 0x0d, 0x47, 0x65, 0x74, - 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, - 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, - 0x70, 0x68, 0x61, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, - 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2e, 0x2e, 0x69, 0x6e, 0x66, 0x65, + 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x41, 0x63, 0x6b, 0x28, 0x01, 0x12, 0x51, 0x0a, 0x0c, 0x50, + 0x75, 0x62, 0x6c, 0x69, 0x73, 0x68, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x12, 0x23, 0x2e, 0x69, 0x6e, + 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, + 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, + 0x1a, 0x1c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, + 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x41, 0x63, 0x6b, 0x12, 0x68, + 0x0a, 0x11, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, + 0x6e, 0x74, 0x73, 0x12, 0x2c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, + 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x53, 0x74, + 0x72, 0x65, 0x61, 0x6d, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, + 0x74, 0x1a, 0x23, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, + 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, + 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x30, 0x01, 0x12, 0x61, 0x0a, 0x0d, 0x53, 0x74, 0x72, 0x65, + 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, - 0x68, 0x61, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, - 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x5d, 0x0a, 0x10, 0x52, 0x65, 0x70, - 0x6f, 0x72, 0x74, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, 0x74, 0x65, 0x12, 0x29, 0x2e, - 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, - 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x53, 0x74, 0x61, - 0x74, 0x65, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x1a, 0x1c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, + 0x68, 0x61, 0x31, 0x2e, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, + 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, - 0x61, 0x31, 0x2e, 0x41, 0x63, 0x6b, 0x28, 0x01, 0x12, 0x51, 0x0a, 0x0c, 0x50, 0x75, 0x62, 0x6c, - 0x69, 0x73, 0x68, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x12, 0x23, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, - 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, - 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x1a, 0x1c, 0x2e, - 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, - 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x41, 0x63, 0x6b, 0x12, 0x68, 0x0a, 0x11, 0x53, - 0x74, 0x72, 0x65, 0x61, 0x6d, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, - 0x12, 0x2c, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, - 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x53, 0x74, 0x72, 0x65, 0x61, - 0x6d, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x23, - 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, - 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, 0x43, 0x61, 0x63, 0x68, 0x65, 0x45, 0x76, - 0x65, 0x6e, 0x74, 0x30, 0x01, 0x12, 0x61, 0x0a, 0x0d, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, - 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x12, 0x2d, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, - 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, - 0x2e, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x52, 0x65, - 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, - 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2e, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x2e, - 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x30, 0x01, 0x42, 0x6f, 0x5a, 0x6d, 0x67, 0x69, 0x74, 0x68, - 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x63, 0x61, 0x63, 0x68, 0x65, 0x62, 0x6f, 0x78, 0x2d, - 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x2f, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, - 0x65, 0x2d, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x73, 0x65, 0x72, 0x76, - 0x65, 0x72, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2f, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, - 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2f, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, - 0x3b, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x76, - 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, - 0x33, + 0x61, 0x31, 0x2e, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x30, 0x01, 0x42, 0x6f, 0x5a, 0x6d, 0x67, + 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x63, 0x61, 0x63, 0x68, 0x65, 0x62, + 0x6f, 0x78, 0x2d, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x2f, 0x69, 0x6e, 0x66, 0x65, 0x72, + 0x65, 0x6e, 0x63, 0x65, 0x2d, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x73, + 0x65, 0x72, 0x76, 0x65, 0x72, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2f, 0x69, 0x6e, 0x66, 0x65, + 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, 0x68, 0x65, 0x2f, 0x76, 0x31, 0x61, 0x6c, 0x70, + 0x68, 0x61, 0x31, 0x3b, 0x69, 0x6e, 0x66, 0x65, 0x72, 0x65, 0x6e, 0x63, 0x65, 0x63, 0x61, 0x63, + 0x68, 0x65, 0x76, 0x31, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x31, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, + 0x6f, 0x74, 0x6f, 0x33, }) var ( @@ -2033,7 +2045,6 @@ func file_inferencecache_v1alpha1_inferencecache_proto_init() { if File_inferencecache_v1alpha1_inferencecache_proto != nil { return } - file_inferencecache_v1alpha1_inferencecache_proto_msgTypes[14].OneofWrappers = []any{} type x struct{} out := protoimpl.TypeBuilder{ File: protoimpl.DescBuilder{ diff --git a/pkg/server/server_test.go b/pkg/server/server_test.go index 84095a17..4c7a82e2 100644 --- a/pkg/server/server_test.go +++ b/pkg/server/server_test.go @@ -1455,11 +1455,11 @@ func TestPublishEventAppliesToIndex(t *testing.T) { } // TestPublishEventBaseEvictionWithPresenceSparesLoRA pins the full wire→index -// seam: a base-model PREFIX_EVICTED carrying adapter_id="" WITH proto3 presence -// drops ONLY the base ("") partition. The service must translate wire presence -// (ev.AdapterId != nil) into index.Event.AdapterSet — reading the VALUE the way -// GetAdapterId does would misclassify a present-but-empty adapter as the legacy -// wildcard and wrongly sweep the co-resident LoRA hint for the same hash. +// seam: a base-model PREFIX_EVICTED carrying adapter_id="" with adapter_scoped=true +// drops ONLY the base ("") partition. The service must translate adapter_scoped +// into index.Event.AdapterSet — reading the adapter_id VALUE ("") instead would +// misclassify a base-scoped eviction as the legacy wildcard and wrongly sweep the +// co-resident LoRA hint for the same hash. func TestPublishEventBaseEvictionWithPresenceSparesLoRA(t *testing.T) { svc := newTestService() for _, a := range []string{"", "sql-lora"} { @@ -1469,11 +1469,10 @@ func TestPublishEventBaseEvictionWithPresenceSparesLoRA(t *testing.T) { Prefixes: []index.PrefixRef{{PrefixHash: []byte("p"), TokenCount: 10}}, }) } - emptyAdapter := "" // present, empty → base partition only (not the wildcard) ack, err := svc.PublishEvent(context.Background(), &icpb.CacheEvent{ Type: icpb.CacheEvent_PREFIX_EVICTED, ReplicaId: "replica-0", ModelId: "m", TenantId: "t", PrefixHash: []byte("p"), - AdapterId: &emptyAdapter, + AdapterId: "", AdapterScoped: true, // base partition only, not the wildcard }) if err != nil { t.Fatalf("PublishEvent: %v", err) diff --git a/proto/inferencecache/v1alpha1/inferencecache.proto b/proto/inferencecache/v1alpha1/inferencecache.proto index 536a5b8f..716f616b 100644 --- a/proto/inferencecache/v1alpha1/inferencecache.proto +++ b/proto/inferencecache/v1alpha1/inferencecache.proto @@ -297,19 +297,23 @@ message CacheEvent { string tenant_id = 4; bytes prefix_hash = 5; int64 timestamp_us = 6; - // Adapter partition a PREFIX_EVICTED removal targets, with explicit presence. - // PRESENT (even "") drops the prefix from exactly that ONE partition ("" is the - // base-model / non-LoRA partition); ABSENT drops it across EVERY adapter - // partition — the conservative legacy sweep, kept for producers that never - // populate the field. Presence is what lets an adapter-aware producer evict the - // base ("") partition WITHOUT also sweeping live LoRA hints for the same - // token-derived hash. Removal is soft state: an over-eager drop costs a cache - // miss, never a wrong answer. Ignored by ALL_CLEARED (a cache flush clears the - // replica across every adapter) and by REPLICA_UPDATED (liveness is - // adapter-independent). Additive, v1alpha1-compatible (proto3 presence uses the - // same wire format; an old producer that never set the field still reads as - // ABSENT → legacy sweep). - optional string adapter_id = 7; + // Adapter partition a PREFIX_EVICTED removal targets, honored only when + // adapter_scoped is set. "" is the base-model / non-LoRA partition. Ignored by + // ALL_CLEARED (a cache flush clears the replica across every adapter) and by + // REPLICA_UPDATED (liveness is adapter-independent). Additive, + // v1alpha1-compatible. + string adapter_id = 7; + // adapter_scoped marks adapter_id as authoritative for a PREFIX_EVICTED: the + // removal targets exactly the adapter_id partition — INCLUDING "" for a + // base-model eviction — instead of sweeping every partition. An adapter-aware + // producer always sets it; false (the default, and what a legacy producer + // sends) keeps the conservative cross-partition sweep. This is the presence + // signal that lets a base-model eviction ("") avoid discarding live LoRA hints + // for the same token-derived hash. Removal is soft state: an over-eager drop + // costs a cache miss, never a wrong answer. A new bool field rather than + // making adapter_id `optional`, which the wire-compat gate rejects as a + // cardinality change. Additive; ignored outside PREFIX_EVICTED. + bool adapter_scoped = 8; } message StreamEventsRequest { From c215f21ff807bfa3fb69a9801fe2c4a845d6a112 Mon Sep 17 00:00:00 2001 From: Ed Sun Date: Thu, 23 Jul 2026 20:36:53 -0700 Subject: [PATCH 3/4] gap-3: rename AdapterSet -> AdapterScoped, drop 'presence' wording Codex nits: the index-side field and comments called the eviction-scope signal 'proto3 presence', but the landed wire contract is the independent adapter_scoped bool. Rename index.Event.AdapterSet -> AdapterScoped and reword the comments + the test name so the two mechanisms aren't conflated. No behavior change; go test ./... , buf lint/breaking pass. --- pkg/index/adapter_partition_test.go | 12 ++++----- pkg/index/index.go | 37 ++++++++++++++-------------- pkg/server/inferencecache_service.go | 6 ++--- pkg/server/server_test.go | 6 ++--- 4 files changed, 31 insertions(+), 30 deletions(-) diff --git a/pkg/index/adapter_partition_test.go b/pkg/index/adapter_partition_test.go index 4163dd63..df85d9d5 100644 --- a/pkg/index/adapter_partition_test.go +++ b/pkg/index/adapter_partition_test.go @@ -188,7 +188,7 @@ func TestAdapterPartitionEvictionIsScopedToItsAdapter(t *testing.T) { idx.ApplyEvent(Event{ Type: EventPrefixEvicted, ReplicaID: "replica-0", Model: adapterModel, Tenant: adapterTenant, - PrefixHash: hash("same-tokens"), Adapter: "sql-lora", AdapterSet: true, + PrefixHash: hash("same-tokens"), Adapter: "sql-lora", AdapterScoped: true, }) if got := lookupUnder(idx, "sql-lora", "same-tokens"); len(got) != 0 { @@ -199,10 +199,10 @@ func TestAdapterPartitionEvictionIsScopedToItsAdapter(t *testing.T) { } } -// A base-model eviction (Adapter "" WITH presence) drops ONLY the base partition -// and must not sweep a live LoRA hint for the same token hash — the over-sweep -// that adapter_id presence exists to fix. -func TestAdapterPartitionBaseEvictionWithPresenceSparesLoRA(t *testing.T) { +// A base-model eviction (Adapter "" WITH AdapterScoped) drops ONLY the base +// partition and must not sweep a live LoRA hint for the same token hash — the +// over-sweep that the adapter_scoped flag exists to fix. +func TestAdapterPartitionBaseEvictionScopedSparesLoRA(t *testing.T) { idx := New() ingestUnder(idx, "replica-0", "", "same-tokens") ingestUnder(idx, "replica-0", "sql-lora", "same-tokens") @@ -210,7 +210,7 @@ func TestAdapterPartitionBaseEvictionWithPresenceSparesLoRA(t *testing.T) { idx.ApplyEvent(Event{ Type: EventPrefixEvicted, ReplicaID: "replica-0", Model: adapterModel, Tenant: adapterTenant, - PrefixHash: hash("same-tokens"), Adapter: "", AdapterSet: true, + PrefixHash: hash("same-tokens"), Adapter: "", AdapterScoped: true, }) if got := lookupUnder(idx, "", "same-tokens"); len(got) != 0 { diff --git a/pkg/index/index.go b/pkg/index/index.go index 1fcddd74..84ede0e3 100644 --- a/pkg/index/index.go +++ b/pkg/index/index.go @@ -244,16 +244,17 @@ type Event struct { Tenant string PrefixHash []byte // Adapter narrows a PREFIX_EVICTED removal to one adapter partition, but only - // when AdapterSet reports the producer supplied it (proto3 presence). With - // AdapterSet, the removal targets exactly Adapter — including "" for a - // base-model eviction, which then does NOT sweep live LoRA hints for the same - // token hash. Without AdapterSet (a legacy producer that never populated the - // field), the removal sweeps EVERY adapter partition — the conservative legacy - // behavior. Both are ignored by ALL_CLEARED (a flush clears the replica across - // adapters) and by REPLICA_UPDATED (liveness is adapter-independent). - Adapter string - AdapterSet bool - Timestamp time.Time + // when AdapterScoped says the producer marked it authoritative (the + // adapter_scoped wire flag). With AdapterScoped, the removal targets exactly + // Adapter — including "" for a base-model eviction, which then does NOT sweep + // live LoRA hints for the same token hash. Without AdapterScoped (a legacy + // producer that never set the flag), the removal sweeps EVERY adapter partition + // — the conservative legacy behavior. Both are ignored by ALL_CLEARED (a flush + // clears the replica across adapters) and by REPLICA_UPDATED (liveness is + // adapter-independent). + Adapter string + AdapterScoped bool + Timestamp time.Time } // LookupRequest asks which replicas hold a given prefix, within a hash scheme. @@ -901,18 +902,18 @@ func (i *Index) ApplyEvent(ev Event) { // adapters at once on the same replica; dropping every partition for one // adapter's GPU eviction would throw away hints that are still valid. // - // Presence (ev.AdapterSet), not emptiness, decides the scope. An - // adapter-aware producer sets ev.Adapter — including "" for a genuine - // base-model eviction — so the removal targets exactly that one partition - // and a base eviction no longer sweeps live LoRA hints for the same hash. - // A legacy producer that never populated the field leaves AdapterSet false, - // and the removal falls back to the original cross-partition sweep (exact - // for such a producer, whose entries all live in the "" partition anyway). + // The adapter_scoped flag (ev.AdapterScoped), not emptiness, decides the + // scope. An adapter-aware producer sets it and ev.Adapter — including "" for + // a genuine base-model eviction — so the removal targets exactly that one + // partition and a base eviction no longer sweeps live LoRA hints for the same + // hash. A legacy producer that never sets the flag leaves AdapterScoped false, + // and the removal falls back to the original cross-partition sweep (exact for + // such a producer, whose entries all live in the "" partition anyway). for key, replicas := range i.prefixes { if key.tenant != ev.Tenant || key.model != ev.Model || key.prefixHash != hash { continue } - if ev.AdapterSet && key.adapter != ev.Adapter { + if ev.AdapterScoped && key.adapter != ev.Adapter { continue } i.removeReplicaLocked(key, replicas, ev.ReplicaID) diff --git a/pkg/server/inferencecache_service.go b/pkg/server/inferencecache_service.go index 02ae043c..30487ded 100644 --- a/pkg/server/inferencecache_service.go +++ b/pkg/server/inferencecache_service.go @@ -1038,9 +1038,9 @@ func (s *inferenceCacheService) PublishEvent(_ context.Context, ev *icpb.CacheEv // marked adapter_id authoritative (adapter_scoped) — including "" for a // base-model eviction. Unset keeps the conservative cross-partition // removal (legacy producers). - Adapter: ev.GetAdapterId(), - AdapterSet: ev.GetAdapterScoped(), - Timestamp: microsToTime(ev.GetTimestampUs()), + Adapter: ev.GetAdapterId(), + AdapterScoped: ev.GetAdapterScoped(), + Timestamp: microsToTime(ev.GetTimestampUs()), }) } return &icpb.Ack{Accepted: true}, nil diff --git a/pkg/server/server_test.go b/pkg/server/server_test.go index 4c7a82e2..5aaca438 100644 --- a/pkg/server/server_test.go +++ b/pkg/server/server_test.go @@ -1454,13 +1454,13 @@ func TestPublishEventAppliesToIndex(t *testing.T) { } } -// TestPublishEventBaseEvictionWithPresenceSparesLoRA pins the full wire→index +// TestPublishEventBaseEvictionScopedSparesLoRA pins the full wire→index // seam: a base-model PREFIX_EVICTED carrying adapter_id="" with adapter_scoped=true // drops ONLY the base ("") partition. The service must translate adapter_scoped -// into index.Event.AdapterSet — reading the adapter_id VALUE ("") instead would +// into index.Event.AdapterScoped — reading the adapter_id VALUE ("") instead would // misclassify a base-scoped eviction as the legacy wildcard and wrongly sweep the // co-resident LoRA hint for the same hash. -func TestPublishEventBaseEvictionWithPresenceSparesLoRA(t *testing.T) { +func TestPublishEventBaseEvictionScopedSparesLoRA(t *testing.T) { svc := newTestService() for _, a := range []string{"", "sql-lora"} { svc.index.Ingest(index.Update{ From 8baaa120b5560f362ca8ff6b28010bd81dbd449c Mon Sep 17 00:00:00 2001 From: Ed Sun Date: Thu, 23 Jul 2026 20:55:15 -0700 Subject: [PATCH 4/4] gap-3: also scope eviction on a non-empty adapter_id (upgrade compat) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fix a mixed-version regression: a pre-adapter_scoped producer (new server, not- yet-upgraded subscriber) sends adapter_id="sql-lora" without adapter_scoped, and the new server would wildcard-sweep it — wiping another adapter's live hint. Scope the removal when adapter_scoped is set OR adapter_id is non-empty; only an empty adapter_id with adapter_scoped unset takes the legacy cross-partition sweep. New TestAdapterPartitionEvictionNonEmptyAdapterScopesWithoutFlag; proto/contract comments updated. go test ./..., buf lint/breaking pass. --- docs/design/grpc-contract.md | 4 +- pkg/index/adapter_partition_test.go | 23 +++++++++++ pkg/index/index.go | 39 +++++++++++-------- .../v1alpha1/inferencecache.pb.go | 26 +++++++------ .../v1alpha1/inferencecache.proto | 26 +++++++------ 5 files changed, 75 insertions(+), 43 deletions(-) diff --git a/docs/design/grpc-contract.md b/docs/design/grpc-contract.md index 9ce3874c..4f28b38d 100644 --- a/docs/design/grpc-contract.md +++ b/docs/design/grpc-contract.md @@ -64,7 +64,7 @@ service InferenceCache { `t2_hit_tokens` / `t2_query_tokens` are cumulative token counters for the **tier-2 (external offload, e.g. LMCache) cache**, sourced from the engine's `vllm:external_prefix_cache_{hits,queries}_total`. The server derives a presence-aware tier-2 hit-rate per CacheBackend (`status.indexParticipation.t2HitRate`): nil until the tier is exercised (`t2_query_tokens > 0`), then the query-weighted `hits/queries` ratio — so a value of `0` means tier-2 is configured but serving no reloads (a silently-degraded offload tier), distinct from "not yet used". Both default to 0 for producers that don't populate them (additive, v1alpha1-compatible). - **CacheEvent** `{ Type type; string replica_id; string model_id; string tenant_id; bytes prefix_hash; int64 timestamp_us; string adapter_id = 7; bool adapter_scoped = 8; }` Type ∈ `PREFIX_ADDED | PREFIX_EVICTED | REPLICA_UPDATED | ALL_CLEARED` - `adapter_id` narrows a `PREFIX_EVICTED` to one adapter partition **when `adapter_scoped` is set** — including `""` (the base-model partition), so a base-model eviction no longer sweeps live LoRA hints for the same hash. Without `adapter_scoped` (legacy producers) the prefix is removed across **every** partition (the conservative sweep — removal is soft state). Ignored by `ALL_CLEARED` (a flush clears the replica across adapters) and `REPLICA_UPDATED` (liveness is adapter-independent). + `adapter_id` narrows a `PREFIX_EVICTED` to one adapter partition **when `adapter_scoped` is set** (so `""` targets the base-model partition, not a sweep) **or when `adapter_id` is non-empty** (a pre-`adapter_scoped` producer naming a LoRA adapter — honored so a mixed-version fleet doesn't regress its narrowing). Only an empty `adapter_id` with `adapter_scoped` unset removes the prefix across **every** partition (the conservative legacy sweep — removal is soft state). Ignored by `ALL_CLEARED` (a flush clears the replica across adapters) and `REPLICA_UPDATED` (liveness is adapter-independent). - **Metric** `{ string name; string type; map labels; double value; int64 timestamp_us; }` (Prometheus `inferencecache_*`, tech spec §4.3) - **StreamEventsRequest** `{ string model_id; string tenant_id; repeated CacheEvent.Type types; }` - **StreamMetricsRequest** `{ repeated string names; }` @@ -268,7 +268,7 @@ The fix partitions the **index**, not the hash: - **The fingerprint construction is unchanged.** Adapter identity is never mixed into the hash, so `pkg/fingerprint`, its golden vectors, and every cross-language implementation (Go / Python / Rust) stay byte-for-byte as they were. The hash still answers "what content is this?"; the partition answers "whose KV is it?". - **Matching inside a partition is unchanged.** Exact-match and the block-chain longest-leading-run rule behave identically; the partition only bounds which entries are candidates. - `LookupRouteResponse.adapter_id` (5) echoes the partition the index was consulted in, so a caller can confirm the round trip. It is empty on the fail-open envelopes returned *before* the index is consulted (`TIMEOUT`, tokenizer fail-open, `POLICY_REQUIRES_CHAIN`). -- `CacheEvent.adapter_id` (7) narrows a `PREFIX_EVICTED` to one partition, gated by the new `adapter_scoped` (8) flag. With `adapter_scoped` set, `adapter_id` — even `""`, the base partition — drops the prefix from exactly that one partition, so one adapter's GPU eviction (including a base-model eviction) no longer discards another adapter's still-valid hint for the identical hash. Without it (legacy producers) the original conservative cross-partition sweep is kept. A separate bool rather than making `adapter_id` `optional`, which the wire-compat gate (`buf breaking`) rejects as a cardinality change — the bool is purely additive. +- `CacheEvent.adapter_id` (7) narrows a `PREFIX_EVICTED` to one partition, with the new `adapter_scoped` (8) flag distinguishing a base-model eviction from a legacy sweep. The removal scopes to `adapter_id` when `adapter_scoped` is set (so `adapter_id=""` drops only the base partition, no longer discarding another adapter's still-valid hint for the identical hash) **or** when `adapter_id` is non-empty (a pre-`adapter_scoped` producer that named a LoRA adapter — honored so a mixed-version fleet doesn't regress a `sql-lora` eviction into a cross-partition sweep). Only an empty `adapter_id` with `adapter_scoped` unset keeps the conservative cross-partition sweep. A separate bool rather than making `adapter_id` `optional`, which the wire-compat gate (`buf breaking`) rejects as a cardinality change — the bool is purely additive. **Producer / consumer must agree, exactly as they already do for `hash_scheme`.** A lookup that sets `adapter_id` will not match entries ingested without one, and vice versa. Empty/unset is the **default partition**, and this is exactly the pre-adapter behavior **for base-model / non-LoRA traffic**: an engine serving no LoRA emits a nil `lora_id`, so ingest and lookup both land under `""` and are completely unaffected — no regression, no configuration required. **This is not a "single LoRA adapter just works" claim.** Only a *nil* `lora_id` uses `""`; *any* non-nil `lora_id` — including a deployment with exactly one adapter — must resolve to a **configured** `--lora-adapter-names` identity, otherwise it is dropped at ingest (**fail-closed**) rather than landing under `""` or a replica-local alias. So even one LoRA adapter requires the ingest identifier (the subscriber's resolved `adapter_id`) to match the gateway's query `adapter_id` end-to-end, or every lookup silently misses — the same agreement `hash_scheme` already demands. diff --git a/pkg/index/adapter_partition_test.go b/pkg/index/adapter_partition_test.go index df85d9d5..6d463a3f 100644 --- a/pkg/index/adapter_partition_test.go +++ b/pkg/index/adapter_partition_test.go @@ -221,6 +221,29 @@ func TestAdapterPartitionBaseEvictionScopedSparesLoRA(t *testing.T) { } } +// Upgrade compatibility: a pre-adapter_scoped producer (new server, not-yet- +// upgraded subscriber) sends a non-empty adapter_id but leaves adapter_scoped +// false. Its LoRA eviction must still narrow to that adapter, not regress into a +// cross-partition sweep that wipes another adapter's live hint. +func TestAdapterPartitionEvictionNonEmptyAdapterScopesWithoutFlag(t *testing.T) { + idx := New() + ingestUnder(idx, "replica-0", "sql-lora", "same-tokens") + ingestUnder(idx, "replica-0", "chat-lora", "same-tokens") + + idx.ApplyEvent(Event{ + Type: EventPrefixEvicted, ReplicaID: "replica-0", + Model: adapterModel, Tenant: adapterTenant, + PrefixHash: hash("same-tokens"), Adapter: "sql-lora", // AdapterScoped left false (pre-upgrade producer) + }) + + if got := lookupUnder(idx, "sql-lora", "same-tokens"); len(got) != 0 { + t.Errorf("sql-lora entry survived its own eviction: %+v", got) + } + if got := lookupUnder(idx, "chat-lora", "same-tokens"); len(got) != 1 { + t.Errorf("chat-lora entry = %+v, want it untouched — a non-empty adapter_id must narrow even without adapter_scoped", got) + } +} + // An eviction with NO adapter keeps the original conservative behavior: it // sweeps every partition. That is what a pre-adapter producer emits, and for // such a producer all entries live in the "" partition anyway — so the diff --git a/pkg/index/index.go b/pkg/index/index.go index 84ede0e3..fd5d20ce 100644 --- a/pkg/index/index.go +++ b/pkg/index/index.go @@ -243,15 +243,15 @@ type Event struct { Model string Tenant string PrefixHash []byte - // Adapter narrows a PREFIX_EVICTED removal to one adapter partition, but only - // when AdapterScoped says the producer marked it authoritative (the - // adapter_scoped wire flag). With AdapterScoped, the removal targets exactly - // Adapter — including "" for a base-model eviction, which then does NOT sweep - // live LoRA hints for the same token hash. Without AdapterScoped (a legacy - // producer that never set the flag), the removal sweeps EVERY adapter partition - // — the conservative legacy behavior. Both are ignored by ALL_CLEARED (a flush - // clears the replica across adapters) and by REPLICA_UPDATED (liveness is - // adapter-independent). + // Adapter narrows a PREFIX_EVICTED removal to one adapter partition. The removal + // scopes to Adapter when AdapterScoped is set (the authoritative signal — so + // Adapter="" targets the base partition, not a sweep) OR when Adapter is + // non-empty (a pre-adapter_scoped producer naming a LoRA adapter; honored so a + // mixed-version fleet doesn't regress its narrowing into a sweep). It sweeps + // EVERY adapter partition only when Adapter is empty AND AdapterScoped is unset + // — the conservative legacy behavior for a no-adapter producer. Both are ignored + // by ALL_CLEARED (a flush clears the replica across adapters) and by + // REPLICA_UPDATED (liveness is adapter-independent). Adapter string AdapterScoped bool Timestamp time.Time @@ -902,18 +902,23 @@ func (i *Index) ApplyEvent(ev Event) { // adapters at once on the same replica; dropping every partition for one // adapter's GPU eviction would throw away hints that are still valid. // - // The adapter_scoped flag (ev.AdapterScoped), not emptiness, decides the - // scope. An adapter-aware producer sets it and ev.Adapter — including "" for - // a genuine base-model eviction — so the removal targets exactly that one - // partition and a base eviction no longer sweeps live LoRA hints for the same - // hash. A legacy producer that never sets the flag leaves AdapterScoped false, - // and the removal falls back to the original cross-partition sweep (exact for - // such a producer, whose entries all live in the "" partition anyway). + // The removal narrows to ev.Adapter unless it genuinely cannot tell which + // partition to target. It narrows when EITHER: + // - the producer set adapter_scoped — the new authoritative signal, so + // adapter_id="" means the base partition, not a sweep; OR + // - adapter_id is non-empty — a pre-adapter_scoped producer that named a + // LoRA adapter. Honoring that keeps its narrowing on a mixed-version + // fleet (new server + not-yet-upgraded subscriber) rather than + // regressing a "sql-lora" eviction into a cross-partition sweep. + // It sweeps EVERY partition only when adapter_id is empty AND adapter_scoped + // is unset — a legacy no-adapter producer, whose entries all live in the "" + // partition anyway, so the sweep is exact for it. + scoped := ev.AdapterScoped || ev.Adapter != "" for key, replicas := range i.prefixes { if key.tenant != ev.Tenant || key.model != ev.Model || key.prefixHash != hash { continue } - if ev.AdapterScoped && key.adapter != ev.Adapter { + if scoped && key.adapter != ev.Adapter { continue } i.removeReplicaLocked(key, replicas, ev.ReplicaID) diff --git a/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go b/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go index 56ea9565..9acd6fa7 100644 --- a/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go +++ b/pkg/server/proto/inferencecache/v1alpha1/inferencecache.pb.go @@ -1307,22 +1307,24 @@ type CacheEvent struct { TenantId string `protobuf:"bytes,4,opt,name=tenant_id,json=tenantId,proto3" json:"tenant_id,omitempty"` PrefixHash []byte `protobuf:"bytes,5,opt,name=prefix_hash,json=prefixHash,proto3" json:"prefix_hash,omitempty"` TimestampUs int64 `protobuf:"varint,6,opt,name=timestamp_us,json=timestampUs,proto3" json:"timestamp_us,omitempty"` - // Adapter partition a PREFIX_EVICTED removal targets, honored only when - // adapter_scoped is set. "" is the base-model / non-LoRA partition. Ignored by + // Adapter partition a PREFIX_EVICTED removal targets. Honored when + // adapter_scoped is set, OR when non-empty (a pre-adapter_scoped producer that + // named a LoRA adapter). "" is the base-model / non-LoRA partition. Ignored by // ALL_CLEARED (a cache flush clears the replica across every adapter) and by // REPLICA_UPDATED (liveness is adapter-independent). Additive, // v1alpha1-compatible. AdapterId string `protobuf:"bytes,7,opt,name=adapter_id,json=adapterId,proto3" json:"adapter_id,omitempty"` - // adapter_scoped marks adapter_id as authoritative for a PREFIX_EVICTED: the - // removal targets exactly the adapter_id partition — INCLUDING "" for a - // base-model eviction — instead of sweeping every partition. An adapter-aware - // producer always sets it; false (the default, and what a legacy producer - // sends) keeps the conservative cross-partition sweep. This is the presence - // signal that lets a base-model eviction ("") avoid discarding live LoRA hints - // for the same token-derived hash. Removal is soft state: an over-eager drop - // costs a cache miss, never a wrong answer. A new bool field rather than - // making adapter_id `optional`, which the wire-compat gate rejects as a - // cardinality change. Additive; ignored outside PREFIX_EVICTED. + // adapter_scoped marks adapter_id as authoritative for a PREFIX_EVICTED so that + // adapter_id="" targets the base-model partition instead of sweeping every + // partition — the signal that lets a base-model eviction avoid discarding live + // LoRA hints for the same token-derived hash. An adapter-aware producer always + // sets it. The server ALSO narrows on a non-empty adapter_id when this is false, + // so a pre-adapter_scoped producer's LoRA eviction keeps narrowing on a + // mixed-version fleet; only an empty adapter_id with adapter_scoped=false takes + // the conservative cross-partition sweep. Removal is soft state: an over-eager + // drop costs a cache miss, never a wrong answer. A new bool rather than making + // adapter_id `optional`, which the wire-compat gate rejects as a cardinality + // change. Additive; ignored outside PREFIX_EVICTED. AdapterScoped bool `protobuf:"varint,8,opt,name=adapter_scoped,json=adapterScoped,proto3" json:"adapter_scoped,omitempty"` unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache diff --git a/proto/inferencecache/v1alpha1/inferencecache.proto b/proto/inferencecache/v1alpha1/inferencecache.proto index 716f616b..2718e646 100644 --- a/proto/inferencecache/v1alpha1/inferencecache.proto +++ b/proto/inferencecache/v1alpha1/inferencecache.proto @@ -297,22 +297,24 @@ message CacheEvent { string tenant_id = 4; bytes prefix_hash = 5; int64 timestamp_us = 6; - // Adapter partition a PREFIX_EVICTED removal targets, honored only when - // adapter_scoped is set. "" is the base-model / non-LoRA partition. Ignored by + // Adapter partition a PREFIX_EVICTED removal targets. Honored when + // adapter_scoped is set, OR when non-empty (a pre-adapter_scoped producer that + // named a LoRA adapter). "" is the base-model / non-LoRA partition. Ignored by // ALL_CLEARED (a cache flush clears the replica across every adapter) and by // REPLICA_UPDATED (liveness is adapter-independent). Additive, // v1alpha1-compatible. string adapter_id = 7; - // adapter_scoped marks adapter_id as authoritative for a PREFIX_EVICTED: the - // removal targets exactly the adapter_id partition — INCLUDING "" for a - // base-model eviction — instead of sweeping every partition. An adapter-aware - // producer always sets it; false (the default, and what a legacy producer - // sends) keeps the conservative cross-partition sweep. This is the presence - // signal that lets a base-model eviction ("") avoid discarding live LoRA hints - // for the same token-derived hash. Removal is soft state: an over-eager drop - // costs a cache miss, never a wrong answer. A new bool field rather than - // making adapter_id `optional`, which the wire-compat gate rejects as a - // cardinality change. Additive; ignored outside PREFIX_EVICTED. + // adapter_scoped marks adapter_id as authoritative for a PREFIX_EVICTED so that + // adapter_id="" targets the base-model partition instead of sweeping every + // partition — the signal that lets a base-model eviction avoid discarding live + // LoRA hints for the same token-derived hash. An adapter-aware producer always + // sets it. The server ALSO narrows on a non-empty adapter_id when this is false, + // so a pre-adapter_scoped producer's LoRA eviction keeps narrowing on a + // mixed-version fleet; only an empty adapter_id with adapter_scoped=false takes + // the conservative cross-partition sweep. Removal is soft state: an over-eager + // drop costs a cache miss, never a wrong answer. A new bool rather than making + // adapter_id `optional`, which the wire-compat gate rejects as a cardinality + // change. Additive; ignored outside PREFIX_EVICTED. bool adapter_scoped = 8; }