release: 10.11.1.0

CHANGES.md

@@ -1,5 +1,17 @@
 # Changes
 
+## 10.11.1.0 08/13/2026
+
+* **Fix infinite recursion in `LONFTokenConverter.Write` — the metadata of an NFT transaction could not be serialized at all** — regression introduced in 10.3.0.0 with the `Newtonsoft.Json` → `System.Text.Json` migration; affects every release from 10.3.0.0 on. `JsonSerializer.Serialize(tx.Meta)` threw `JsonException: A possible object cycle was detected` for any transaction whose `AffectedNodes` contain an `NFTokenPage`, which is every `NFTokenMint`, `NFTokenBurn`, `NFTokenAcceptOffer` and `NFTokenModify` that touched a page. Verified against mainnet on all six NFT transaction types — the four above failed, `NFTokenCreateOffer` and `NFTokenCancelOffer` (no page in their metadata) went through:
+  * The converter broke its own recursion the way the other polymorphic converters do — strip itself from `options.Converters` via `JsonSerializerOptionsCache.WithoutConverter<T>` and re-enter the serializer. That works only for a converter that is *registered in the list*. `LONFTokenConverter` is declared as a `[JsonConverter]` **attribute on the `NFToken` type itself** (`LONFTokenPage.cs`), and a converter attached to a type outranks the options list, so System.Text.Json handed the value straight back to `Write` no matter what the list looked like. The frame repeated until the writer hit `MaxDepth`. Raising `MaxDepth` is not a workaround: at 64 and 128 it is a catchable `JsonException`, at 256 the stack overflows and the process dies
+  * `NFToken` has two fields, so `Write` now emits them directly instead of delegating. The wire shape is unchanged — `{"NFToken":{"NFTokenID":"…","URI":"…"}}`, the envelope `Read` already looks for — and the documented null behaviour is preserved by honouring `options.DefaultIgnoreCondition` rather than hard-coding one: `XrplJsonOptions.Default` (`WhenWritingNull`) omits a null `URI`, plain options keep it as `null`
+  * The six other converter types that call `WithoutConverter` — `LOConverter`, `GenericStringConverter<T>`, `MetaBinaryConverter`, `LedgerBinaryConverter`, `TransactionRequestConverter` and `TransactionResponseConverter` — were audited against the same two conditions — declared as a type-level attribute **and** re-serializing that same declared type. None hit both. `LOConverter` is registered in the options list (its one attribute use is property-level) and writes the concrete runtime type; `GenericStringConverter<T>`, `MetaBinaryConverter`, `LedgerBinaryConverter` and `TransactionRequestConverter` are only ever attached to properties. The three node converters (`CreatedNodeConverter`, `ModifiedNodeConverter`, `DeletedNodeConverter`) do not call `WithoutConverter` at all and so are not among those six, but they are type-level and were checked for the same trap anyway: they serialize a *different* class (`value.NewFields.GetType()`). `TransactionResponseConverter` is the one other type-level case, and the same trap was already defused there by the `TransactionResponseUnknown` sentinel, so that no value ever carries the annotated type at runtime. Nothing else was changed
+  * `TestULONFTokenConverter` had `Read` coverage only, which is how the bug survived. It now pins the written shape, both round trips (`URI` set and null), null handling under `XrplJsonOptions.Default` and under plain options, a multi-token `NFTokenPage`, and — the regression test proper — serializing a `Meta` carrying an `NFTokenPage` in `CreatedNode.NewFields`, `ModifiedNode.FinalFields`, `ModifiedNode.PreviousFields` and `DeletedNode.FinalFields`, since the page can arrive in any of them. All offline, on prepared JSON
+* **WebSocket message assembly was quadratic in the number of receive chunks** — `ReceiveLoopAsync` grew a multi-chunk message with `byteResult = byteResult.Concat(buffer.Take(result.Count)).ToArray()`. Every chunk allocated a fresh array the size of everything received so far and refilled it one byte at a time through a LINQ enumerator, so a message split into *k* chunks copied roughly `k/2` times its own length; every intermediate array was well past the 85 KB threshold and therefore landed on the uncompacted large object heap. `ledger_data` at `limit=2048` is a few megabytes and arrives in dozens of chunks over a real link, which is exactly where the cost concentrates. Chunks are now `Buffer.BlockCopy`-ed into a scratch buffer that grows to the largest message on the connection and is reused from then on; a message that arrives whole in one chunk skips the scratch entirely, and both the receive buffer and that scratch buffer are rented from `ArrayPool` rather than allocated per connection (measured on .NET 10: the shared pool does hand back the same multi-megabyte array after a return, so this is a real saving and not just indirection). Measured on a local fragmenting WebSocket server, 300 messages of 2 MiB, allocation per message: 3.50x payload at one chunk, 6.50x at eight, 18.52x at thirty-two — now a flat 3.01x, which is the floor (the exact-sized `byte[]` plus the UTF-16 string handed to the callback). Through the full client stack, a 3000-page `ledger_data` crawl with each page arriving in 32 chunks and a consumer retaining every object: 100.7 s → 55.8 s, 158.4 GiB → 67.3 GiB allocated, 891 → 398 gen2 collections, and the last-decile-to-first-decile page time drops from 1.39x to 1.13x. `ReceiveChunkSize` was measured at 1 MiB and 64 KiB and left at 1 MiB — now that the buffer is pooled, shrinking it changed nothing outside run-to-run noise. `TestUWebSocketMessageAssembly` pins the byte-exactness of a 96-chunk message, that a short message after a long one picks up no stale bytes from the reused buffer, and that per-message allocation stays under 12x payload at 64 chunks (34.5x before the fix). A dead `timedOut` local, declared and tested but never assigned since it appeared, is gone
+* **Request timeout timers outlived their requests** — `RequestManager.Resolve`/`Reject` called `timer.Stop()`. `System.Timers.Timer` derives from `Component` and carries a finalizer, so every completed request left a finalizable object behind, each of them holding its request's serialized text alive through the `Elapsed` closure; over a long paged crawl that is thousands of them. `Dispose()` stops the timer and takes it off the finalization queue. A second, worse case sat next to it: a token that is **already cancelled** runs its `Register` callback inline, so `Reject` completed the request in the middle of the factory method — before the timeout timer existed and therefore with nothing to remove. The factory then registered the timer for a promise that was already gone, and when it fired, `Reject` took its missing-promise early return without removing it, so the entry stayed in `timeoutsAwaitingResponse` for the life of the process. The `CancellationTokenRegistration` leaked on the same path, its assignment to `TaskInfo` happening after `DeletePromise` had already run. Both factories now check whether the promise survived and clean up after themselves; timer removal moved into `DisposeTimeout`, which is also called on the early returns of `Resolve` and `Reject` and so closes the narrow race with a concurrent cancellation as well. `TestURequestManagerCancellation` pins that an already-cancelled token leaves neither timer nor promise behind in either factory, and that a live request still arms its timeout and releases it on completion
+* **Reflection on the per-response path is gone** — `Resolve`, `Reject` and `ObserveTaskException` reached for `TrySetResult`, `TrySetException` and `Task` through `GetType().GetMethod(...)` + `Invoke` on every single response. `TaskInfo` now carries typed `SetResult`/`SetException` delegates and the `CompletionTask` itself, wired when the request is created. The properties were added rather than substituted: `TaskInfo` is public, so instances built outside `RequestManager` keep the old reflective path
+* **Dead `tasks` field removed from `XrplClient`** — `private readonly ConcurrentDictionary<int, TaskInfo> tasks` was never assigned and never read, so it was permanently null; a leftover from when the client tracked pending requests itself, which `RequestManager` has done for a long time
+
 ## 10.11.0.0 08/04/2026
 
 * **MPT path steps (`0x40`)** — `PathSet` only knew the three classic hop-type bits (`0x01` account, `0x10` currency, `0x20` issuer). rippled added `STPathElement::TypeMpt = 0x40` in **3.2.0**, so a hop can now carry a 24-byte `MPTokenIssuanceID` instead of a currency. The gap was silent in both directions: `FromParser` matched none of its masks on a `0x40` byte, produced an empty hop and left the 24 MPTID bytes unread — every following byte was then parsed at the wrong offset — while `SynthesizeType` had no way to emit the bit at all. Now handled end to end:

Tests/Xrpl.Tests/Client/BenchmarkLedgerDataCrawl.cs

@@ -0,0 +1,177 @@
+using Microsoft.VisualStudio.TestTools.UnitTesting;
+
+using System;
+using System.Collections.Generic;
+using System.Diagnostics;
+using System.Threading.Tasks;
+
+using Xrpl.Client;
+
+namespace Xrpl.Tests.ClientLib
+{
+    /// <summary>
+    /// Manual benchmark of a long paged crawl through the full client stack (socket receive loop,
+    /// message routing, RequestManager, JSON round-trip). Deliberately named outside the
+    /// TestU/TestI filters so it never runs in CI; invoke it explicitly:
+    /// <c>dotnet test --filter "FullyQualifiedName~BenchmarkLedgerDataCrawl"</c>.
+    /// Knobs: CRAWL_PAGES, CRAWL_PAYLOAD_BYTES, CRAWL_FRAGMENTS, CRAWL_RETAIN_PER_PAGE.
+    /// CRAWL_RETAIN_PER_PAGE models a consumer that keeps every crawled object alive (a full
+    /// ledger-state snapshot), which is what makes each forced gen2 collection progressively
+    /// more expensive as the crawl advances.
+    /// </summary>
+    [TestClass]
+    public class BenchmarkLedgerDataCrawl
+    {
+        private static int EnvInt(string name, int fallback)
+        {
+            string? raw = Environment.GetEnvironmentVariable(name);
+            return int.TryParse(raw, out int value) && value >= 0 ? value : fallback;
+        }
+
+        /// <summary>Stand-in for one crawled ledger object the consumer keeps in its snapshot.</summary>
+        private sealed class RetainedEntry
+        {
+            public RetainedEntry(string index)
+            {
+                Index = index;
+            }
+
+            public string Index { get; }
+        }
+
+        [TestMethod]
+        public async Task BenchmarkSequentialPaging()
+        {
+            int pages = EnvInt("CRAWL_PAGES", 2000);
+            int payloadBytes = EnvInt("CRAWL_PAYLOAD_BYTES", 2 * 1024 * 1024);
+            int fragments = EnvInt("CRAWL_FRAGMENTS", 32);
+            int retainPerPage = EnvInt("CRAWL_RETAIN_PER_PAGE", 0);
+
+            List<RetainedEntry> snapshot = new List<RetainedEntry>(pages * retainPerPage);
+
+            using PagedResponseServer server = new PagedResponseServer(payloadBytes, fragments);
+            using XrplClient client = new XrplClient(server.Url);
+
+            await client.Connect().ConfigureAwait(false);
+
+            // One warm-up page so JIT and pooled buffers are not charged to the measured window.
+            await RequestPageAsync(client).ConfigureAwait(false);
+
+            GC.Collect(2, GCCollectionMode.Forced, blocking: true);
+            GC.WaitForPendingFinalizers();
+            GC.Collect(2, GCCollectionMode.Forced, blocking: true);
+
+            long allocatedBefore = GC.GetTotalAllocatedBytes(precise: true);
+            int gen0Before = GC.CollectionCount(0);
+            int gen1Before = GC.CollectionCount(1);
+            int gen2Before = GC.CollectionCount(2);
+            long heapBefore = GC.GetTotalMemory(false);
+            long lohBefore = LohBytes();
+
+            double[] pageMs = new double[pages];
+            long startTicks = Stopwatch.GetTimestamp();
+
+            for (int i = 0; i < pages; i++)
+            {
+                long before = Stopwatch.GetTimestamp();
+                await RequestPageAsync(client).ConfigureAwait(false);
+
+                for (int entry = 0; entry < retainPerPage; entry++)
+                {
+                    snapshot.Add(new RetainedEntry(((long)i * retainPerPage + entry).ToString("X16")));
+                }
+
+                pageMs[i] = (Stopwatch.GetTimestamp() - before) * 1000.0 / Stopwatch.Frequency;
+            }
+
+            double totalSeconds = (Stopwatch.GetTimestamp() - startTicks) / (double)Stopwatch.Frequency;
+            long allocated = GC.GetTotalAllocatedBytes(precise: true) - allocatedBefore;
+            long heapAfter = GC.GetTotalMemory(false);
+            long lohAfter = LohBytes();
+
+            await client.Disconnect().ConfigureAwait(false);
+
+            Console.WriteLine("=== ledger_data crawl benchmark (full client stack) ===");
+            Console.WriteLine($"pages             : {pages}");
+            Console.WriteLine($"payload           : {payloadBytes / 1024.0 / 1024.0:F2} MiB");
+            Console.WriteLine($"fragments per page: {fragments}");
+            Console.WriteLine($"retained objects  : {snapshot.Count:N0} ({retainPerPage}/page)");
+            Console.WriteLine($"total time        : {totalSeconds:F2} s ({pages / totalSeconds:F2} pages/s)");
+            Console.WriteLine($"first decile avg  : {DecileAverage(pageMs, 0):F1} ms/page");
+            Console.WriteLine($"last decile avg   : {DecileAverage(pageMs, 9):F1} ms/page");
+            Console.WriteLine($"trend (last/first): {DecileAverage(pageMs, 9) / DecileAverage(pageMs, 0):F2}x");
+            Console.WriteLine($"p50 / p99 / max   : {Percentile(pageMs, 50):F1} / {Percentile(pageMs, 99):F1} / " +
+                              $"{Percentile(pageMs, 100):F1} ms");
+            Console.WriteLine($"allocated total   : {allocated / 1024.0 / 1024.0 / 1024.0:F2} GiB");
+            Console.WriteLine($"allocated per page: {allocated / (double)pages / 1024.0 / 1024.0:F1} MiB " +
+                              $"({allocated / (double)pages / payloadBytes:F1}x payload)");
+            Console.WriteLine($"gen0/gen1/gen2    : {GC.CollectionCount(0) - gen0Before} / " +
+                              $"{GC.CollectionCount(1) - gen1Before} / {GC.CollectionCount(2) - gen2Before}");
+            Console.WriteLine($"managed heap      : {heapBefore / 1024.0 / 1024.0:F1} -> {heapAfter / 1024.0 / 1024.0:F1} MiB");
+            Console.WriteLine($"LOH size          : {lohBefore / 1024.0 / 1024.0:F1} -> {lohAfter / 1024.0 / 1024.0:F1} MiB");
+            Console.WriteLine("decile profile (ms/page): " + string.Join(" | ", DecileProfile(pageMs)));
+        }
+
+        private static async Task RequestPageAsync(XrplClient client)
+        {
+            Dictionary<string, object> request = new Dictionary<string, object>
+            {
+                ["command"] = "ledger_data",
+                ["ledger_index"] = 96000000,
+                ["binary"] = true,
+                ["limit"] = 2048
+            };
+
+            Dictionary<string, object> response = await client.Request(request).ConfigureAwait(false);
+            if (response == null)
+            {
+                throw new InvalidOperationException("empty ledger_data response");
+            }
+        }
+
+        private static long LohBytes()
+        {
+            GCMemoryInfo info = GC.GetGCMemoryInfo();
+            ReadOnlySpan<GCGenerationInfo> generations = info.GenerationInfo;
+            return generations.Length > 3 ? generations[3].SizeAfterBytes : 0;
+        }
+
+        private static double DecileAverage(double[] values, int decile)
+        {
+            int size = Math.Max(1, values.Length / 10);
+            int from = decile * size;
+            int to = Math.Min(values.Length, from + size);
+            if (from >= to)
+            {
+                return 0;
+            }
+
+            double sum = 0;
+            for (int i = from; i < to; i++)
+            {
+                sum += values[i];
+            }
+
+            return sum / (to - from);
+        }
+
+        private static string[] DecileProfile(double[] values)
+        {
+            string[] profile = new string[10];
+            for (int i = 0; i < 10; i++)
+            {
+                profile[i] = DecileAverage(values, i).ToString("F1");
+            }
+
+            return profile;
+        }
+
+        private static double Percentile(double[] values, int percentile)
+        {
+            double[] sorted = (double[])values.Clone();
+            Array.Sort(sorted);
+            int index = (int)Math.Round((percentile / 100.0) * (sorted.Length - 1));
+            return sorted[Math.Clamp(index, 0, sorted.Length - 1)];
+        }
+    }
+}