Karma

Karma is a small TCP service for fast, day-bucketed limit usage accounting.

Use it when an application needs fresh usage totals by limit, subject, and UTC day on the hot path. Karma keeps counters in memory, persists accepted writes with snapshots and a write-ahead log (WAL), and speaks newline-delimited JSON over TCP.

Russian documentation: README.ru.md.

When to use it

Karma is built for this shape of problem:

application receives a request
  -> records usage for a subject
  -> reads the current usage total for the same limit
  -> decides whether the subject is still within the limit

Good fits:

• API request, email, export, job, or storage usage limits;
• many subjects with small day-bucketed counters;
• fresh totals that are cheaper to read from a focused read model than from an analytical database;
• at-least-once producers that need idempotent writes.

Not a fit:

• arbitrary time-series tags or analytical queries;
• multi-master replication;
• automatic leader election or quorum writes;
• strong cross-node read-after-write guarantees from replicas.

Quick start

Requirements:

• Crystal 1.17.1
• Shards

Build and run a local node:

shards build --release
bin/karma \
  --bind=127.0.0.1 \
  --port=8080 \
  --directory=.karma-data \
  --restore=true \
  --wal=true

Write one usage event:

printf '{"v":2,"op":"counter.increment","series":"api_requests","key":42,"value":1}\n' \
  | nc 127.0.0.1 8080

Read the current total:

printf '{"v":2,"op":"counter.sum","series":"api_requests","key":42}\n' \
  | nc 127.0.0.1 8080

Successful responses use the v2 envelope:

{"protocol_version":2,"success":true,"response":1,"error_code":null}

Run with Docker:

docker build -t karma:local .
docker run --rm \
  -p 8080:8080 \
  -v karma-data:/data \
  karma:local \
  --bind=0.0.0.0 \
  --port=8080 \
  --directory=/data \
  --restore=true \
  --wal=true \
  --wal-fsync=true

For production, use a persistent volume, keep WAL enabled, keep --wal-fsync=true unless you have a measured reason not to, scrape metrics, and create regular snapshots with snapshot.create_all or SIGUSR1.

Core concepts

Term	Meaning
series	Limit name, for example api_requests or emails_sent.
key	Unsigned 64-bit subject id inside a limit, for example account, user, workspace, or project id.
bucket	UTC day in YYYYMMDD format. If omitted on writes, Karma uses the current UTC day.
value	Unsigned 64-bit usage amount. Counters never go below zero.

The public v2 protocol still contains some historical tree.* operation names. For new examples and clients, use limit-usage language: series, key, bucket, and value.

Read commands do not create missing series. A missing series returns not_found; a missing key inside an existing series returns zero or an empty result.

Protocol

Karma 1.0 accepts protocol v2 only.

• One request is one JSON object followed by \n.
• One response is one JSON object followed by \r\n.
• Every request must include "v": 2 and op.
• If --auth-token is set, requests must include token.
• If --read-auth-token is set, that token can run only read-only commands.

Example request:

{"v":2,"op":"counter.increment","series":"api_requests","key":42,"bucket":20260620,"value":1}

Example error:

{
  "protocol_version": 2,
  "success": false,
  "response": "Field tree or series is required",
  "error_code": "validation_error"
}

Stable error codes include invalid_json, unsupported_protocol, unknown_command, validation_error, not_found, unauthorized, forbidden, request_too_large, response_too_large, query_timeout, idempotency_conflict, replication_gap, replication_error, and internal_error.

Common operations

Create a limit explicitly when you want setup to be visible:

{"v":2,"op":"tree.create","series":"api_requests"}

Record and read usage:

{"v":2,"op":"counter.increment","series":"api_requests","key":42,"value":1}
{"v":2,"op":"counter.increment","series":"api_requests","key":42,"bucket":20260620,"value":10}
{"v":2,"op":"counter.decrement","series":"api_requests","key":42,"bucket":20260620,"value":1}
{"v":2,"op":"counter.sum","series":"api_requests","key":42}
{"v":2,"op":"counter.sum","series":"api_requests","key":42,"range":{"from":20260601,"to":20260620}}
{"v":2,"op":"counter.series","series":"api_requests","key":42,"range":{"from":20260601,"to":20260620}}

Use batches when the application can collapse events before sending them:

{"v":2,"op":"series.batch_add","series":"api_requests","items":[[42,20260620,37],[43,20260620,12]]}
{"v":2,"op":"series.batch_set","series":"api_requests","items":[[42,20260620,100],[43,20260620,0]]}
{"v":2,"op":"counter.batch_sum","series":"api_requests","keys":[41,42,43]}
{"v":2,"op":"counter.multi_sum","items":[{"series":"api_requests","key":101},{"series":"emails_sent","key":101}]}

series.batch_set writes exact bucket values. A zero value deletes that bucket. Large requests must fit --max-request-bytes.

Inspect and maintain a limit:

{"v":2,"op":"tree.list"}
{"v":2,"op":"tree.info","series":"api_requests"}
{"v":2,"op":"tree.keys","series":"api_requests","limit":1000,"cursor":0}
{"v":2,"op":"tree.summary","series":"api_requests","range":{"from":20260601,"to":20260620}}
{"v":2,"op":"tree.top","series":"api_requests","limit":100}
{"v":2,"op":"series.delete_before","series":"api_requests","before":20260601}
{"v":2,"op":"series.compact","series":"api_requests"}

Reset or delete usage when an application needs to remove stored buckets:

{"v":2,"op":"counter.reset","series":"api_requests","key":42}
{"v":2,"op":"counter.delete_range","series":"api_requests","key":42,"range":{"from":20260601,"to":20260620}}
{"v":2,"op":"tree.reset","series":"api_requests"}

Idempotent writes

Mutating commands can include idempotency_key. Karma stores the first successful response for that key. Repeating the same payload returns the saved response with "idempotent": true; reusing the key with a different payload returns idempotency_conflict.

{"v":2,"op":"series.batch_add","series":"api_requests","items":[[42,20260620,37]],"idempotency_key":"usage-offsets-1000-1499"}

Use idempotency for writes that may be retried by a queue, job runner, import, or at-least-once event source. Do not retry mutating writes blindly.

Idempotency retention is controlled by --idempotency-max-records, --idempotency-max-age-seconds, and manual pruning:

{"v":2,"op":"idempotency.prune","before":"2026-06-20T00:00:00Z","limit":10000}

Streaming ingest

Streaming ingest is for rebuilds, backfills, and large imports. It supports:

Mode	Behavior
add	Add item values to the live series.
set	Set exact item bucket values in the live series.
replace_series	Build a staged series and atomically replace the live series on commit.

{"v":2,"op":"ingest.begin","stream_id":"import-20260620","mode":"add","granularity":"day"}
{"v":2,"op":"ingest.chunk","stream_id":"import-20260620","series":"api_requests","chunk_seq":1,"items":[[42,20260620,10]]}
{"v":2,"op":"ingest.commit","stream_id":"import-20260620"}

Duplicate chunks are skipped. Out-of-order chunks are rejected before they are applied. A committed stream is remembered durably, including after restart, snapshot restore, or replication bootstrap.

Persistence and recovery

Karma stores live counters in memory and persists data through:

• snapshots: MessagePack .tree files, one per series;
• WAL: newline-delimited JSON entries in karma.wal.

With --restore=true, startup loads the latest snapshot per series and replays WAL entries after the snapshot LSN.

Snapshot commands:

{"v":2,"op":"snapshot.create","series":"api_requests"}
{"v":2,"op":"snapshot.create_all"}
{"v":2,"op":"snapshot.list"}
{"v":2,"op":"snapshot.info"}
{"v":2,"op":"snapshot.verify"}

snapshot.create_all writes atomic snapshots, fsyncs them, truncates WAL after successful snapshotting, and prunes old snapshots according to --dump-retention-per-tree.

Recovery markers for external pipelines:

{"v":2,"op":"recovery.checkpoint","source":"usage-export","offset":"export-2026-06-20","event_id":"batch-42"}
{"v":2,"op":"recovery.status"}
{"v":2,"op":"reconciliation.report","checked_points":1000,"mismatch_count":2,"absolute_drift":15,"max_abs_delta":10}

Replication

Karma supports asynchronous master-to-slave replication. A slave bootstraps from master snapshots and then polls WAL entries.

bin/karma \
  --role=slave \
  --port=8081 \
  --directory=/var/lib/karma-slave \
  --restore=true \
  --replication-source-host=127.0.0.1 \
  --replication-source-port=8080 \
  --replication-token=read-secret

Useful checks:

{"v":2,"op":"replication.status"}
{"v":2,"op":"replication.entries","after_lsn":120,"limit":1000}

Replication boundaries:

• replication is asynchronous;
• slave nodes reject direct mutating client commands;
• failover is manual;
• the old master must be stopped before promoting a slave.

Runbook: docs/replication-operations-runbook.md.

Operations

Health and metrics:

{"v":2,"op":"system.ping"}
{"v":2,"op":"system.health"}
{"v":2,"op":"system.stats"}
{"v":2,"op":"system.metrics"}

Production runbook: docs/runbook.md.

Signals:

• SIGINT: stop accepting clients, snapshot all series, truncate WAL after successful snapshots, and exit with status 0.
• SIGUSR1: snapshot all series, truncate WAL after successful snapshots, and keep running.

Common startup options:

Option	Environment	Default	Meaning
--bind=host	KARMA_HOST	0.0.0.0	Address to listen on.
--port=port	KARMA_PORT	8080	TCP port.
--directory=path	KARMA_DUMP_DIR	.	Directory for snapshots, WAL, and metadata.
--role=master|slave	KARMA_ROLE	master	Node role.
--restore=true|false	KARMA_RESTORE	true	Restore snapshots and replay WAL on startup.
--wal=true|false	KARMA_WAL	true	Persist mutating commands to WAL.
--wal-fsync=true|false	KARMA_WAL_FSYNC	true	Fsync WAL writes and truncation.
--auth-token=token	KARMA_AUTH_TOKEN	unset	Token required for all commands.
--read-auth-token=token	KARMA_READ_AUTH_TOKEN	unset	Token allowed only for read-only commands.
--max-request-bytes=bytes	KARMA_MAX_REQUEST_BYTES	4096	Maximum JSON request line size.
--max-response-bytes=bytes	KARMA_MAX_RESPONSE_BYTES	1048576	Maximum JSON response size; 0 disables the limit.
--query-timeout-ms=ms	KARMA_QUERY_TIMEOUT_MS	1000	Timeout for large reads; 0 disables it.

Run bin/karma --help for the full flag list.

Clients

Crystal:

dependencies:
  karma_client:
    path: clients/crystal

KarmaClient.with_client do |karma|
  karma.record_usage("api_requests", subject_id: 42, amount: 1, day: Time.utc)
  karma.usage("api_requests", subject_id: 42)
end

See clients/crystal.

Ruby/Rails:

gem "karma_client", path: "clients/ruby"

KarmaClient.with_client do |karma|
  karma.increment(series: "api_requests", key: 42, bucket: Date.current, value: 1)
  karma.sum(series: "api_requests", key: 42, from: 7.days.ago.to_date, to: Date.current)
end

See clients/ruby.

Performance

Local performance depends on CPU, disk, filesystem, runtime, network, and workload mix. Treat the included results as regression checks, not universal promises.

Recent local checks showed:

• in-process single reads and writes in the hundreds of thousands of ops/sec with WAL off;
• batched add and batched sum paths above one million items or keys per second in local profiles;
• persisted single increments becoming WAL-bound when WAL is enabled;
• 10M-key scalability checks using about 5 GiB of reserved heap for 70M daily data points in one local run.

Charts:

Development

Developer guide: docs/development.ru.md.

Baseline checks:

crystal spec
shards build --release

Focused suites:

crystal spec spec/command_spec.cr
crystal spec spec/wal_spec.cr
crystal spec spec/replication_spec.cr
crystal spec spec/idempotency_spec.cr
crystal spec spec/bucketed_counter
crystal spec clients/crystal/spec
ruby clients/ruby/test/karma_client_test.rb

Do not commit generated runtime data such as .crystal-cache-*, .karma-data, .spec_*, bin/, snapshots, WAL files, or temporary files.

Support and status

Current shard version: 1.0.1.

The supported public API is protocol v2. Legacy/v1 requests are intentionally not supported in Karma 1.0.

For a bug report, include the Karma version, startup flags, request JSON, response envelope, and relevant logs. For production triage, start with docs/runbook.md and, for replicas, docs/replication-operations-runbook.md.

License

MIT