Skip to content

libdb 5.3.34: incomplete replication commit lock lists can violate client transaction isolation #140

Description

@xint-io

Summary

libdb 5.3.34 can violate transaction isolation during replication apply on a client. The
master-side trigger is a logged top-level DB_TXN_SNAPSHOT update transaction on a database opened
with DB_MULTIVERSION. The transaction retains both write locks and DB_LOCK_SIREAD locks at
commit.

__lock_vec allocates the temporary descriptor array used to build the replication commit lock
list from nwrites. This count excludes SIREAD locks, but DB_LOCK_PUT_READ populates a descriptor
for each retained lock of either type. The extra descriptor is written past the allocation. If
execution continues after this write, __lock_fix_list serializes only the first nwrites
descriptors. When a later-acquired SIREAD lock for another object is visited first, its object can
displace the write-lock object for a modified page.

The replication apply path on the client uses this list to acquire write locks. The built-in redo
path for the modified page does not reacquire the omitted page lock. Apply can therefore change the
page despite a read lock retained by a separate client transaction at the default isolation level.
This can make a repeated lookup observe a record that was absent earlier. The same count mismatch
also causes an out-of-bounds write on the master. The client isolation consequence follows from
source analysis and has not been directly observed.

Environment

Item Value
libdb Berkeley DB 5.3.34: (August 3, 2026), commit c4811dc871e313033993e95baa5b6525057c5911
Release build dist/configure 'CFLAGS=-O2 -g'
Diagnostic build dist/configure --enable-debug --enable-diagnostic 'CFLAGS=-g -O0 -fno-omit-frame-pointer'
AddressSanitizer build dist/configure --enable-debug 'CFLAGS=-fsanitize=address -fno-omit-frame-pointer -g -O1' LDFLAGS=-fsanitize=address
Host Ubuntu 26.04, Linux 7.0.0-30-generic, x86_64, gcc 15.2.0
Database B-tree with DB_MULTIVERSION
Environment flags DB_CREATE|DB_INIT_LOCK|DB_INIT_LOG|DB_INIT_MPOOL|DB_INIT_TXN|DB_INIT_REP
Reproducer transaction DB_TXN_SNAPSHOT, one write followed by one read
Reproducer topology single-site master using the Base Replication API; no client

Steps to reproduce

Start in an empty directory. The reproducer uses public db.h calls to start a single-site master,
write one record, read another in the same transaction, and commit. The --control option removes
only the read. In the tested tree, the fill phase and small page size place the two keys on
different B-tree pages. This separation is required to create a distinct SIREAD lock and makes the
displaced write-lock object visible in the printed page numbers. The reproducer measures the
incomplete commit lock list and the heap write on the master; it does not run a replication client.

  1. Check out the tested source, then configure and build release, diagnostic, and AddressSanitizer
    trees.

    git clone https://github.com/berkeleydb/libdb.git
    cd libdb
    git checkout c4811dc871e313033993e95baa5b6525057c5911
    mkdir build-release build-diagnostic build-asan
    (cd build-release && env CFLAGS='-O2 -g' ../dist/configure)
    (cd build-diagnostic && env CFLAGS='-g -O0 -fno-omit-frame-pointer' \
      ../dist/configure --enable-debug --enable-diagnostic)
    (cd build-asan && env \
      CFLAGS='-fsanitize=address -fno-omit-frame-pointer -g -O1' \
      LDFLAGS='-fsanitize=address' ../dist/configure --enable-debug)
    make -C build-release -j12
    make -C build-diagnostic -j12
    make -C build-asan -j8
  2. Save the reproducer below as repro.c in the source directory.

Complete repro.c
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include <db.h>

#define DATABASE "ledger.db"
#define NRECORDS 2000
#define PAGE_SIZE 512
#define MAX_RETRIES 20

static int broadcast_calls;
static int targeted_calls;

static const char *
code_name(int ret)
{
	if (ret == 0)
		return ("DB_SUCCESS");
	if (ret == DB_LOCK_DEADLOCK)
		return ("DB_LOCK_DEADLOCK");
	if (ret == DB_LOCK_NOTGRANTED)
		return ("DB_LOCK_NOTGRANTED");
	if (ret == DB_SNAPSHOT_CONFLICT)
		return ("DB_SNAPSHOT_CONFLICT");
	if (ret == DB_SNAPSHOT_UNSAFE)
		return ("DB_SNAPSHOT_UNSAFE");
	if (ret == DB_NOTFOUND)
		return ("DB_NOTFOUND");
	return (db_strerror(ret));
}

static int
fail(const char *call, int ret)
{
	fprintf(stderr, "%s: %s (%d)\n", call, code_name(ret), ret);
	return (ret == 0 ? EINVAL : ret);
}

static int
retryable(int ret)
{
	return (ret == DB_LOCK_DEADLOCK || ret == DB_LOCK_NOTGRANTED ||
	    ret == DB_SNAPSHOT_CONFLICT || ret == DB_SNAPSHOT_UNSAFE);
}

static int
send_message(DB_ENV *env, const DBT *control, const DBT *record,
    const DB_LSN *lsn, int eid, u_int32_t flags)
{
	(void)env;
	(void)control;
	(void)record;
	(void)lsn;
	(void)eid;
	(void)flags;
	if (eid == DB_EID_BROADCAST) {
		broadcast_calls++;
		return (0);
	}
	targeted_calls++;
	return (EIO);
}

static int
open_environment(DB_ENV **envp)
{
	int ret;

	*envp = NULL;
	if ((ret = db_env_create(envp, 0)) != 0)
		return (fail("db_env_create", ret));
	if ((ret = (*envp)->rep_set_transport(
	    *envp, 1, send_message)) != 0)
		return (fail("DB_ENV->rep_set_transport", ret));
	if ((ret = (*envp)->open(*envp, ".",
	    DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL |
	    DB_INIT_TXN | DB_INIT_REP, 0600)) != 0)
		return (fail("DB_ENV->open", ret));
	if ((ret = (*envp)->set_lk_detect(
	    *envp, DB_LOCK_DEFAULT)) != 0)
		return (fail("DB_ENV->set_lk_detect", ret));
	if ((ret = (*envp)->rep_start(
	    *envp, NULL, DB_REP_MASTER)) != 0)
		return (fail("DB_ENV->rep_start", ret));
	return (0);
}

static int
open_database(DB_ENV *env, DB **dbp, int create)
{
	u_int32_t flags;
	int ret;

	*dbp = NULL;
	if ((ret = db_create(dbp, env, 0)) != 0)
		return (fail("db_create", ret));
	if ((ret = (*dbp)->set_pagesize(*dbp, PAGE_SIZE)) != 0)
		return (fail("DB->set_pagesize", ret));
	flags = DB_AUTO_COMMIT | DB_MULTIVERSION;
	if (create)
		flags |= DB_CREATE;
	if ((ret = (*dbp)->open(*dbp, NULL, DATABASE, NULL,
	    DB_BTREE, flags, 0600)) != 0)
		return (fail("DB->open", ret));
	return (0);
}

static void
make_dbt(DBT *dbt, void *data, size_t size)
{
	memset(dbt, 0, sizeof(*dbt));
	dbt->data = data;
	dbt->size = (u_int32_t)size;
}

static int
put_value(DB *db, DB_TXN *txn, const char *key_text,
    const char *value_text)
{
	DBT key, value;

	make_dbt(&key, (void *)key_text, strlen(key_text));
	make_dbt(&value, (void *)value_text, strlen(value_text));
	return (db->put(db, txn, &key, &value, 0));
}

static int
get_value(DB *db, DB_TXN *txn, const char *key_text, DBT *value)
{
	DBT key;

	make_dbt(&key, (void *)key_text, strlen(key_text));
	memset(value, 0, sizeof(*value));
	return (db->get(db, txn, &key, value, 0));
}

static int
fill_database(DB *db)
{
	char key[32], value[32];
	int attempt, i, ret;

	for (i = 0; i < NRECORDS; i++) {
		(void)snprintf(key, sizeof(key), "account%06d", i);
		(void)snprintf(value, sizeof(value), "balance%06d", i);
		for (attempt = 0; attempt < MAX_RETRIES; attempt++) {
			ret = put_value(db, NULL, key, value);
			if (ret == 0)
				break;
			if (!retryable(ret))
				return (fail("DB->put (fill)", ret));
		}
		if (attempt == MAX_RETRIES)
			return (fail("DB->put (fill retries)", EBUSY));
	}
	return (0);
}

static int
run_transaction(DB_ENV *env, DB *db, int with_read)
{
	DB_TXN *txn;
	DBT value;
	int attempt, ret, t_ret;

	for (attempt = 0; attempt < MAX_RETRIES; attempt++) {
		txn = NULL;
		if ((ret = env->txn_begin(
		    env, NULL, &txn, DB_TXN_SNAPSHOT)) != 0)
			return (fail("DB_ENV->txn_begin", ret));

		ret = put_value(db, txn, "journal000001", "transfer000001");
		if (ret == 0 && with_read)
			ret = get_value(db, txn, "account001554", &value);
		if (ret != 0) {
			t_ret = txn->abort(txn);
			if (t_ret != 0)
				return (fail("DB_TXN->abort", t_ret));
			if (retryable(ret))
				continue;
			return (fail("transaction operation", ret));
		}

		ret = txn->commit(txn, 0);
		if (ret == 0) {
			printf("DB_TXN->commit: %s (%d)\n",
			    code_name(ret), ret);
			return (0);
		}
		if (!retryable(ret))
			return (fail("DB_TXN->commit", ret));
	}
	return (fail("transaction retries", EBUSY));
}

int
main(int argc, char **argv)
{
	DB_ENV *env;
	DB *db;
	DBT value;
	int create, ret, t_ret, with_read;

	if (argc != 2 ||
	    (strcmp(argv[1], "--trigger") != 0 &&
	    strcmp(argv[1], "--control") != 0)) {
		fprintf(stderr, "usage: %s --trigger|--control\n", argv[0]);
		return (EXIT_FAILURE);
	}
	with_read = strcmp(argv[1], "--trigger") == 0;
	printf("libdb version: %s\n", db_version(NULL, NULL, NULL));
	printf("environment: DB_CREATE|DB_INIT_LOCK|DB_INIT_LOG|DB_INIT_MPOOL|"
	    "DB_INIT_TXN|DB_INIT_REP\n");
	printf("database: DB_BTREE|DB_MULTIVERSION; transaction: "
	    "DB_TXN_SNAPSHOT; role: DB_REP_MASTER\n");
	printf("workload: one write followed by %d read%s\n",
	    with_read, with_read == 1 ? "" : "s");

	env = NULL;
	db = NULL;
	create = 1;
	if ((ret = open_environment(&env)) != 0)
		goto out;
	if ((ret = open_database(env, &db, create)) != 0)
		goto out;
	if ((ret = fill_database(db)) != 0)
		goto out;
	if ((ret = run_transaction(env, db, with_read)) != 0)
		goto out;

	if ((ret = db->close(db, 0)) != 0) {
		db = NULL;
		(void)fail("DB->close", ret);
		goto out;
	}
	db = NULL;
	printf("DB->close: DB_SUCCESS (0)\n");
	if ((ret = env->close(env, 0)) != 0) {
		env = NULL;
		(void)fail("DB_ENV->close", ret);
		goto out;
	}
	env = NULL;
	printf("DB_ENV->close: DB_SUCCESS (0)\n");

	if ((ret = open_environment(&env)) != 0)
		goto out;
	if ((ret = open_database(env, &db, 0)) != 0)
		goto out;
	if ((ret = get_value(db, NULL, "journal000001", &value)) != 0) {
		(void)fail("DB->get after clean close", ret);
		goto out;
	}
	printf("post-close DB->get: %s (%d), key=journal000001 value=%.*s\n",
	    code_name(ret), ret, (int)value.size, (char *)value.data);
	if (value.size != strlen("transfer000001") ||
	    memcmp(value.data, "transfer000001", value.size) != 0) {
		ret = fail("post-close value mismatch", EINVAL);
		goto out;
	}
	if (targeted_calls != 0) {
		ret = fail("unexpected transport callback", EIO);
		goto out;
	}
	printf("transport callbacks: %d broadcast, %d targeted\n",
	    broadcast_calls, targeted_calls);

out:
	if (db != NULL) {
		t_ret = db->close(db, 0);
		if (ret == 0 && t_ret != 0)
			ret = t_ret;
	}
	if (env != NULL) {
		t_ret = env->close(env, 0);
		if (ret == 0 && t_ret != 0)
			ret = t_ret;
	}
	return (ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE);
}
  1. Compile against the release tree, then run the trigger and control in separate directories.

    RELEASE=$PWD/build-release
    cc -O2 -g -I"$RELEASE" repro.c "$RELEASE/libdb.a" \
      $(sed -n 's/^LDFLAGS=[[:space:]]*//p' "$RELEASE/Makefile" | head -1) \
      $(sed -n 's/^LIBS=[[:space:]]*//p' "$RELEASE/Makefile" | head -1) \
      -ldl -pthread -o repro-release
    mkdir release-trigger release-control
    (cd release-trigger && ../repro-release --trigger)
    (cd release-control && ../repro-release --control)
  2. Extract the write record and the matching commit record by transaction ID. This avoids selecting
    one of the unrelated commit records produced while the database is filled.

Log extraction command
(
  set -eu
  RELEASE=$PWD/build-release
  show_txn_pages()
  {
    tree=$1
    directory=$2
    output="$directory.printlog"
    LD_LIBRARY_PATH="$tree/.libs" \
      "$tree/.libs/db_printlog" -h "$directory" >"$output"

    key_line=$(grep -n 'data: journal000001' "$output" | tail -1 | cut -d: -f1)
    test -n "$key_line"
    start=$((key_line - 6))
    header=$(sed -n "${start},${key_line}p" "$output" |
      grep '__db_addrem:' | tail -1)
    txn=$(printf '%s\n' "$header" |
      sed -n 's/.* txnp \([^ ]*\) .*/\1/p')
    test -n "$txn"
    commit_line=$(grep -n "__txn_regop: rec: 10 txnp $txn " "$output" |
      tail -1 | cut -d: -f1)
    test -n "$commit_line"

    write_page=$(sed -n "${start},${key_line}p" "$output" |
      awk '/^[[:space:]]*pgno:/ { page=$2 } END { print page }')
    commit_page_fields=$(awk -v start="$commit_line" '
      NR < start { next }
      NR > start && NF == 0 { exit }
      /^[[:space:]]*\(/ {
        line = $0
        sub(/^[^)]*\)[[:space:]]*/, "", line)
        print line
      }
    ' "$output")
    commit_pages=$(printf '%s\n' "$commit_page_fields" |
      awk '{ for (i = 1; i <= NF; i++) print $i }')
    printf '%s\n' "$commit_pages" |
      awk 'NF && $0 !~ /^[0-9]+$/ { bad = 1 } END { exit bad }'
    commit_count=$(printf '%s\n' "$commit_pages" |
      awk 'NF { count++ } END { print count + 0 }')
    test -n "$write_page"
    test "$commit_count" -eq 1
    printf 'write record page: %s\n' "$write_page"
    printf 'commit lock-list page count: %s\n' "$commit_count"
    printf 'commit lock-list page: %s\n' "$commit_pages"
  }

  show_txn_pages "$RELEASE" release-trigger
  test "$write_page" != "$commit_pages" || {
    echo 'trigger did not reach the distinct-page condition' >&2
    exit 1
  }
  show_txn_pages "$RELEASE" release-control
  test "$write_page" = "$commit_pages" || {
    echo 'control commit lock list does not contain the written page' >&2
    exit 1
  }
)
  1. Compile and run the reproducer against the diagnostic and AddressSanitizer trees. The
    AddressSanitizer tree omits --enable-diagnostic, so the DB_ASSERT bounds check does not stop
    execution before the out-of-bounds write.
Instrumented build and run commands
(
  set -eu
  ulimit -c 0

  DIAGNOSTIC=$PWD/build-diagnostic
  cc -g -O0 -fno-omit-frame-pointer -I"$DIAGNOSTIC" \
    repro.c "$DIAGNOSTIC/libdb.a" \
    $(sed -n 's/^LDFLAGS=[[:space:]]*//p' "$DIAGNOSTIC/Makefile" | head -1) \
    $(sed -n 's/^LIBS=[[:space:]]*//p' "$DIAGNOSTIC/Makefile" | head -1) \
    -ldl -pthread -o repro-diagnostic
  mkdir diagnostic-trigger diagnostic-control
  diagnostic_status=0
  (cd diagnostic-trigger && ../repro-diagnostic --trigger) \
    >diagnostic-trigger.log 2>&1 || diagnostic_status=$?
  test "$diagnostic_status" -eq 134
  grep -F 'BDB0059 assert failure:' diagnostic-trigger.log
  (cd diagnostic-control && ../repro-diagnostic --control) \
    >diagnostic-control.log 2>&1
  ! grep -Fq 'BDB0059 assert failure:' diagnostic-control.log

  ASAN=$PWD/build-asan
  cc -g -O1 -fno-omit-frame-pointer -fsanitize=address -I"$ASAN" \
    repro.c "$ASAN/libdb.a" \
    $(sed -n 's/^LDFLAGS=[[:space:]]*//p' "$ASAN/Makefile" | head -1) \
    $(sed -n 's/^LIBS=[[:space:]]*//p' "$ASAN/Makefile" | head -1) \
    -ldl -pthread -o repro-asan
  mkdir asan-trigger asan-control
  asan_status=0
  (cd asan-trigger && env \
    ASAN_OPTIONS='detect_leaks=0:halt_on_error=1:abort_on_error=1' \
    ../repro-asan --trigger) >asan-trigger.log 2>&1 || asan_status=$?
  test "$asan_status" -eq 134
  grep -F 'AddressSanitizer: heap-buffer-overflow' asan-trigger.log
  grep -F 'WRITE of size 8' asan-trigger.log
  grep -F 'src/lock/lock.c:457' asan-trigger.log
  grep -F '0 bytes after 40-byte region' asan-trigger.log
  (cd asan-control && env ASAN_OPTIONS='detect_leaks=0' \
    ../repro-asan --control) >asan-control.log 2>&1
  ! grep -Fq 'AddressSanitizer:' asan-control.log
)

Expected result

Replication apply must not modify a page while a client transaction retains a conflicting read
lock. Replication clients permit concurrent reads, and transactional page locks remain held until
the transaction commits
(docs_src/guides/programmer_reference/rep.md#L116-L120,
docs_src/guides/programmer_reference/lock_am_conv.md#L16-L20).
__rep_process_txn therefore acquires the objects in the commit lock list as write locks before it
applies any log record
(src/rep/rep_record.c#L1572-L1658).
The list must contain every write-lock object retained by the master transaction. An object from a
SIREAD lock must not displace a write-lock object. Building the list must remain within its
allocation.

Actual result

The release trigger exits 0 after commit:

DB_TXN->commit: DB_SUCCESS (0)

The extraction command matches the write and commit records by transaction ID:

write record page: 179
commit lock-list page count: 1
commit lock-list page: 140

The commit lock list therefore omits the page modified by the transaction. Removing only the read
restores that page as the list's only page:

write record page: 179
commit lock-list page count: 1
commit lock-list page: 179

AddressSanitizer trigger (addresses and source-path prefix normalized; unrelated frames omitted):

WRITE of size 8 at <address> thread T0
<address> is located 0 bytes after 40-byte region [...]
SUMMARY: AddressSanitizer: heap-buffer-overflow <source>/src/lock/lock.c:457 in __lock_vec

Diagnostic trigger (source-path prefix normalized):

BDB0059 assert failure: <source>/src/lock/lock.c/454: "(u_int8_t *)np < (u_int8_t *)objlist->data + objlist->size"

Both instrumented triggers exit with status 134. Their no-read controls exit 0 without an
assertion or an AddressSanitizer report.

The reproducer starts no replication client. These results directly prove the incomplete commit
lock list and the out-of-bounds heap write. They do not directly observe a client isolation
violation.

Analysis

Root cause

nwrites counts write locks, while DB_LOCK_PUT_READ populates descriptors for retained write and
SIREAD locks. This mismatch both under-sizes the temporary descriptor array and truncates the
serialized commit lock list.

  1. DB_TXN_SNAPSHOT sets the internal TXN_SNAPSHOT and TXN_SNAPSHOT_SAFE flags. On a
    DB_MULTIVERSION handle, __db_lget converts the page-read request to DB_LOCK_SIREAD. Newly
    granted locks enter the head of the locker's heldby list. A SIREAD lock acquired after a write
    lock is therefore visited first
    (src/txn/txn.c#L233-L252,
    src/db/db_meta.c#L1178-L1194,
    src/lock/lock.c#L1166-L1184).
  2. During a logged top-level commit on a replication master, __txn_commit asks __lock_vec to
    build the commit lock list while both lock types remain on heldby. nwrites counts only
    IS_WRITELOCK modes, so the array has one descriptor slot per write lock. It has no additional
    slots for retained SIREAD locks.
    DB_LOCK_PUT_READ releases ordinary read locks but retains both SIREAD and write locks. The
    traversal can therefore populate more descriptors than the array can hold. __lock_sicommit
    does not detach the SIREAD locks until the later __txn_end path, immediately before
    DB_LOCK_PUT_ALL
    (src/txn/txn.c#L847-L882,
    src/dbinc/lock.h#L52-L58,
    src/lock/lock.c#L1448-L1455,
    src/lock/lock.c#L386-L460,
    src/txn/txn.c#L1769-L1787).
  3. Each descriptor populated after the first nwrites descriptors starts outside the allocation.
    Each retained SIREAD lock increases the number of populated descriptors without increasing the
    nwrites-sized allocation. The number of out-of-bounds descriptors therefore grows with the
    retained SIREAD set.
    DB_ASSERT is the only bounds check, and it is inactive without DIAGNOSTIC. AddressSanitizer
    directly observes the 8-byte np->data write. The source then assigns np->size through the
    same out-of-range descriptor
    (src/lock/lock.c#L453-L460,
    src/dbinc/debug.h#L35-L40).
  4. In the tested release build, execution continues to __lock_fix_list. It receives nwrites
    instead of the number of descriptors actually populated. It therefore serializes only that
    prefix. Because a later-acquired SIREAD lock is visited first, its object can occupy the prefix
    and displace a write-lock object. In the reproduced transaction, __txn_regop_log therefore
    records the read page and omits the written page
    (src/lock/lock.c#L465-L468,
    src/lock/lock_list.c#L119-L140,
    src/txn/txn.c#L875-L882).
  5. On a client, __rep_process_txn takes write locks only for the serialized objects. It then
    collects the transaction's log records independently and applies them with DB_TXN_APPLY. The
    affected __db_addrem_recover path uses a recovery cursor and does not acquire the omitted
    logical page lock. Redo can therefore modify that page without encountering a read lock held by
    a separate client transaction
    (src/rep/rep_record.c#L1572-L1658,
    src/dbinc/db_am.h#L87-L107,
    src/db/db_meta.c#L1166-L1176,
    src/db/db_rec.c#L54-L124).

Impact and scope

The primary impact is that replication apply can violate transaction isolation on a client if the
master sends a commit log record containing the incomplete commit lock list. A separate read-only
client transaction at the default isolation level can search the omitted page and retain its read
lock. Because apply does not request the conflicting page lock, it can change the page without
resolving that conflict. A later lookup for the same key in the same client transaction can find a
record that the earlier lookup did not find. This violates B-tree's documented degree 3 isolation
guarantee
(docs_src/guides/programmer_reference/am_misc_stability.md#L8-L14,
docs_src/guides/programmer_reference/lock_am_conv.md#L16-L20).

The secondary impact is memory corruption on the master. AddressSanitizer directly observes an
out-of-bounds heap write during list construction. The tested release run continues far enough to
record the incomplete commit lock list, but the unchecked write can instead crash or corrupt
process memory.

The commit lock list does not select the log records or pages used for redo. The client still
follows the transaction's log chain, so the current evidence does not establish a dropped write,
persistent database corruption, data loss, or replica divergence. __txn_commit builds the commit
lock list only for a logged top-level transaction on a replication master. Non-replication
environments do not enter this branch
(src/txn/txn.c#L847-L872).
Observing the isolation violation also requires a concurrent client transaction to retain a read
lock on an omitted object.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions