Hermes

Hermes is a lightweight pub-sub middleware in Go, inspired by ROS 2/DDS internals. It implements broker-managed topics, reliable and best-effort transports, Redis-backed durability, schema enforcement, node discovery, observability, and benchmark tooling in a compact distributed-systems project.

Overview

Hermes is built around a central broker that accepts publishers and subscribers over TCP/UDP, persists accepted messages to Redis Streams, routes messages to active subscribers, and exposes system state through a CLI, HTTP dashboard, and Prometheus metrics.

The project is intentionally scoped like a miniature robotics/distributed-systems middleware: it has explicit QoS behavior, durable replay, observable backpressure, realistic benchmark workloads, and a clean Go API for building nodes.

Architecture

Go nodes / CLI
    |
    | TCP reliable publish/subscribe
    | UDP best-effort publish/subscribe
    v
Hermes broker
    |
    |-- topic registry
    |-- reliable subscriber queues
    |-- UDP subscriber address registry
    |-- schema validation
    |-- node discovery
    |-- routing metrics
    |
    |--> Redis Streams durability
    |--> per-topic JSONL logs
    |--> DLQ streams
    |--> HTTP dashboard
    |--> Prometheus metrics

Features

• Broker-managed pub-sub

  • Central topic registry
  • Publisher/subscriber session management
  • Message routing by topic
  • Per-topic counters, rates, queue depth, and delivery metrics

• Go client API

  • hermes.NewNode(name, brokerAddr)
  • node.Publisher(topic, qos)
  • node.Subscribe(topic, qos, callback)
  • publisher.Publish(ctx, []byte)
  • node.Replay(topic, offset, callback)

• QoS modes

  • Reliable: TCP transport with subscriber acknowledgement
  • BestEffort: UDP transport for fire-and-forget sensor-style traffic
  • Reliable subscribers use bounded delivery queues so backpressure is visible

• Durability and replay

  • Every accepted message is appended to Redis Stream hermes:stream:<topic>
  • Late subscribers and CLI users can replay from Redis Stream offsets
  • Per-topic JSONL logs provide human-readable audit trails

• Reliability controls

  • ACK tracking for reliable subscribers
  • Bounded retry behavior
  • Dead-letter queue for failed reliable deliveries
  • CLI and dashboard inspection for DLQ entries

• Schema enforcement

  • Optional JSON Schema registration per topic
  • Broker-side validation before Redis persistence, logging, or routing
  • Reliable publishers receive validation errors
  • Best-effort invalid UDP payloads are discarded
  • Current schema subset supports type, required, properties, items, enum, bounds, and additionalProperties: false

• Node discovery

  • Nodes tracked through publish, subscribe, inspect, and heartbeat activity
  • Broker records publisher/subscriber topic relationships
  • CLI and dashboard expose live node/topic topology

• Observability

  • Built-in HTTP dashboard
  • JSON metrics API at /api/metrics
  • Prometheus-compatible metrics at /metrics
  • Grafana + Prometheus Docker Compose stack
  • Broker latency metrics for schema validation, Redis append, log write, and total accept path

Performance

• Reliable fan-out: 512 subscribers, 1 KiB payload

  • 256,000/256,000 deliveries
  • 0 drops
  • p99 latency: 48.248143ms

• Reliable fan-out: 1,024 subscribers, 2 KiB payload

  • 102,400/102,400 deliveries
  • 0 drops
  • p99 latency: 440.282617ms

• Warehouse simulation: 32 robots, mixed QoS

  • 98 topics
  • 225 subscriber sessions
  • 20,530/20,530 deliveries
  • 0 drops

• Warehouse simulation: 2,048 robot stress probe

  • 6,146 topics
  • 14,337 subscriber sessions
  • reliable traffic: 24,286/24,286, 0 drops
  • best-effort traffic: degraded under pressure, as expected for UDP QoS

Performance interpretation:

• Reliable mode favors delivery guarantees over dropping data; under extreme pressure, the practical failure mode is tail-latency growth.
• Best-effort mode behaves like sensor/map traffic in robotics systems: it drops stale data under load instead of preserving every sample.
• Reliable control/health streams remained lossless while best-effort sensor/map streams degraded first.

Full benchmark tables live in docs/testing-and-benchmarking.md.

Benchmarking

Hermes includes benchmark tooling for latency, throughput, fan-out, and realistic multi-node workloads.

• bench

  • One publisher, one subscriber
  • Reports sent, received, dropped, throughput, p50, p95, p99
  • Supports reliable TCP and best-effort UDP
  • Supports paced or burst publishing

• bench-suite

  • Repeatable benchmark matrix
  • Covers reliable burst, reliable paced, large payloads, schema-enabled reliable traffic, UDP burst, UDP paced, and UDP large payloads
  • Writes Markdown or JSON for before/after comparisons

• bench-fanout

  • Measures broker routing cost as one topic fans out to many subscribers
  • Useful for validating reliable ACK path, subscriber queues, and tail latency growth
  • Supports subscriber sweeps such as 1,2,4,8,16,32,64,128,256,512

• bench-sim

  • Models an autonomous warehouse fleet
  • Creates named robot nodes, shared planner/map topics, per-robot pose/lidar/battery topics, and mixed reliable/best-effort traffic
  • Uses realistic message rates and payload sizes:
    • reliable pose: 20 Hz, 256 B
    • reliable battery: 2 Hz, 160 B
    • reliable task assignment: 512 B, fan-out to all robots
    • best-effort lidar: 10 Hz, 4 KiB
    • best-effort occupancy grid: 2 KiB, fan-out to all robots

• Metrics during benchmarks

  • /api/metrics: JSON broker metrics
  • /metrics: Prometheus text exposition
  • Dashboard: live rate, average rate, subscribers, queue depth, drops, DLQ, schema status, replay, node graph
  • Grafana dashboard: Prometheus-backed broker/topic panels

Example commands:

go run ./cmd/hermes bench --topic bench --qos reliable --messages 1000 --size 256

go run ./cmd/hermes bench-suite \
  --broker 127.0.0.1:7447 \
  --http http://127.0.0.1:8080 \
  --format markdown \
  --out /tmp/hermes-bench-suite.md

go run ./cmd/hermes bench-fanout \
  --broker 127.0.0.1:7447 \
  --qos reliable \
  --subscribers 1,2,4,8,16,32,64 \
  --messages 1000 \
  --size 1024 \
  --format markdown \
  --out /tmp/hermes-fanout.md

go run ./cmd/hermes bench-sim \
  --broker 127.0.0.1:7447 \
  --http http://127.0.0.1:8080 \
  --robots 8 \
  --duration 10s \
  --format markdown \
  --out /tmp/hermes-warehouse-sim.md

Quickstart

Requirements:

• Go 1.22+
• Redis
• Make

Start Redis:

make redis-start

Start the broker:

make run-broker

Open the dashboard:

http://127.0.0.1:8080

Publish a reliable message:

go run ./cmd/hermes pub --topic sensor.temp --qos reliable --data '{"c":24}'

Subscribe:

go run ./cmd/hermes sub --topic sensor.temp --qos reliable

Replay from Redis Streams:

go run ./cmd/hermes replay --topic sensor.temp --from 0

Inspect topics and nodes:

go run ./cmd/hermes topics
go run ./cmd/hermes nodes

Register a JSON schema:

go run ./cmd/hermes schema set --topic sensor.temp --file sensor-temp.schema.json

Inspect dead letters:

go run ./cmd/hermes dlq --topic sensor.temp

Inspect broker latency metrics:

go run ./cmd/hermes metrics

Run tests:

go test ./...

Prometheus And Grafana

Start Hermes so Docker containers can reach the HTTP metrics endpoint:

go run ./cmd/hermes-broker \
  --tcp 127.0.0.1:17447 \
  --udp 127.0.0.1:17447 \
  --http 0.0.0.0:18080 \
  --redis 127.0.0.1:16379 \
  --log-dir /tmp/hermes-demo-logs

Start Prometheus and Grafana:

docker compose up -d prometheus grafana

Open:

Hermes dashboard: http://127.0.0.1:18080
Prometheus:       http://127.0.0.1:9090
Grafana:          http://127.0.0.1:3000

Grafana login:

admin / admin

The provisioned dashboard is Hermes / Hermes Overview.

Public API

node := hermes.NewNode("example", "127.0.0.1:7447")
pub, _ := node.Publisher("sensor.temp", hermes.Reliable)
_ = pub.Publish(context.Background(), []byte(`{"c":24}`))

_ = node.Subscribe(context.Background(), "sensor.temp", hermes.Reliable, func(msg hermes.Message) {
    fmt.Println(string(msg.Payload))
})