Skip to content

fix(jsonrpc): enforce log filter cap and improve match efficiency#6732

Open
317787106 wants to merge 23 commits intotronprotocol:developfrom
317787106:hotfix/fix_newFilter
Open

fix(jsonrpc): enforce log filter cap and improve match efficiency#6732
317787106 wants to merge 23 commits intotronprotocol:developfrom
317787106:hotfix/fix_newFilter

Conversation

@317787106
Copy link
Copy Markdown
Collaborator

@317787106 317787106 commented Apr 29, 2026

Problem

Three issues exist in the JSON-RPC log filter subsystem (eth_newFilter / eth_getLogs), reported in #6510:

  1. Unbounded memory growtheth_newFilter imposes no limit on the number of active filters held in memory. A client can register filters indefinitely and eventually exhaust heap on the node.

  2. Correctness bugLogMatch.matchBlockOneByOne evaluated the MAX_RESULT guard after addAll, so the result list could transiently contain more than MAX_RESULT entries before the exception was thrown.

  3. Performance bottleneck under high filter counthandleLogsFilter iterated the filter map with an Iterator and called result.add() once per element, which is slow and contention-prone when thousands of filters are active.


Changes

1. Enforce a configurable cap on active log filters

Adds node.jsonrpc.maxLogFilterNum (default: 20000). When the cap is reached, eth_newFilter immediately returns JsonRpcExceedLimitException (JSON-RPC code -32005) instead of growing without bound.

node {
  jsonrpc {
    # Maximum number of concurrent eth_newFilter registrations (0 = unlimited)
    maxLogFilterNum = 20000
  }
}

2. Fix the MAX_RESULT boundary check in LogMatch.matchBlockOneByOne (correctness)

Moves the size + matchedLog.size() > MAX_RESULT guard before addAll. The result list now never exceeds MAX_RESULT entries regardless of how many logs a single block contributes.

3. Optimize handleLogsFilter for large filter maps

Condition Before After
filter count ≤ 10,000 (common case) Iterator loop, result.add() per element ConcurrentHashMap.forEach with local list + single addAll
filter count > 10,000 same slow path bounded ForkJoinPool(3) + parallelStream

Key improvements:

  • Reduced lock contention: elements matched per filter are collected into a local list first, then inserted with a single addAll on the shared LinkedBlockingQueue.
  • Bounded parallelism: logsFilterPool is a ForkJoinPool(3) created once per TronJsonRpcImpl instance and shut down with it, avoiding unbounded thread creation under spike load.
  • Observability: a logger.debug timing line records dispatch cost and filter-map size.

4. Decouple filter state from static class scope

The four filter maps (eventFilter2ResultFull, blockFilter2ResultFull, eventFilter2ResultSolidity, blockFilter2ResultSolidity), logsFilterPool, and filterParallelThreshold are now instance fields. handleBLockFilter, handleLogsFilter, and processLogFilterEntry are now instance methods.

Motivation: with static shared state, any test that constructs a TronJsonRpcImpl instance would silently read and write the same filter maps used by the running node, making test isolation impossible. Switching to instance state means each instance owns its own filter maps; tests create a lightweight instance with no Spring dependencies (new TronJsonRpcImpl(null, null, null)) and operate on an isolated map.

As a consequence, BlockFilterCapsule and LogsFilterCapsule now receive the TronJsonRpcImpl instance at construction time and call handleBLockFilter / handleLogsFilter on it directly, eliminating the static imports that were the only way to invoke those methods before.

5. node.jsonrpc.maxBlockFilterNum = 0 means block filter umber is not limited.

It has the same meaning as node.jsonrpc.maxLogFilterNum =0 for log filter number.


Testing

  • Unit tests (new / updated):
    • HandleLogsFilterTest — 10 cases covering event dispatch, block-range filtering, expired-filter removal, solidified/non-solidified routing, and 3 parallel-path cases (all-filters receive events, expired eviction, solidified routing — each with filterParallelThreshold lowered via reflection to keep the test fast)
    • LogMatchOverLimitTest — 4 cases covering under-limit, exact-limit, exceed-limit (verifies exception is thrown before addAll), and empty-block skip
    • WalletCursorTest.testNewFilter_exceedsCapThrowsException — verifies the cap throws the correct exception with the limit value in the message; fixed to populate the map on the same instance that calls newFilter (the original test was populating the static map then constructing a new instance, so it never actually tested the cap)
    • BlockFilterCapsuleTest, LogsFilterCapsuleTest, ConcurrentHashMapTest — updated to pass a TronJsonRpcImpl instance to constructors after the static-to-instance refactoring
  • Manual testing

Closes #6510

Comment thread common/src/main/resources/reference.conf
Comment thread framework/src/main/java/org/tron/core/services/jsonrpc/TronJsonRpcImpl.java Outdated
Comment thread framework/src/test/java/org/tron/core/jsonrpc/LogMatchOverLimitTest.java Outdated
Comment thread framework/src/main/java/org/tron/core/services/jsonrpc/TronJsonRpcImpl.java Outdated
Comment thread framework/src/test/java/org/tron/core/jsonrpc/LogMatchOverLimitTest.java Outdated
@Slf4j(topic = "API")
public class JsonRpcApiUtil {

static SecureRandom random = new SecureRandom();
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[NIT] Hoisting random to a static field is a nice perf win👍. One small follow-up: consider tightening it to private static final:
private static final SecureRandom random = new SecureRandom();
As written (package-private, non-final), anything in org.tron.core.services.jsonrpc can reassign it. A well-meaning test or perf experiment could quietly do: JsonRpcApiUtil.random = new Random(42);// compiles, no warning
…and filter IDs silently become predictable, which breaks the unguessability assumption clients rely on. private final closes that path at compile time, and also gives JMM safe-publication guarantees for free.

@317787106 317787106 force-pushed the hotfix/fix_newFilter branch from e6354d0 to ad0d688 Compare April 30, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parallelizing eth_newFilter event matching

4 participants