Memory leak: SearchIndex / AsyncSearchIndex instances are never garbage collected

[rbs333]

Summary

SearchIndex and AsyncSearchIndex instances are never garbage collected. Every index
ever constructed is retained for the lifetime of the process, so applications that build
an index per request grow unboundedly and are eventually OOM-killed.

The root cause is a weakref.finalize misuse in redisvl/index/index.py:

# SearchIndex.__init__
if self._owns_redis_client:
    weakref.finalize(self, self.disconnect)          # bound method -> strong ref to self

# AsyncSearchIndex.__init__
if self._owns_redis_client:
    weakref.finalize(self, sync_wrapper(self.disconnect))

self.disconnect is a bound method, so the finalizer callback holds a strong
reference to self. Per the weakref.finalize docs,
"the callback must not hold a reference to obj, otherwise obj will never be garbage
collected." weakref.finalize stores its callback in a module-level registry that is a
genuine GC root, so this creates a self-sustaining reference the cyclic GC cannot break.
The finalizer never fires, disconnect() is never called, and the index — along with its
schema, all field objects, its lock, and its Redis client — is retained forever.

A finalizer intended to release connections is the very thing preventing their release.

Reproduction (no Redis server required)

import gc
from redisvl.index import SearchIndex, AsyncSearchIndex

D = {
    "index": {"name": "probe", "prefix": "p", "storage_type": "hash"},
    "fields": [
        {"name": "a", "type": "tag"},
        {"name": "b", "type": "numeric"},
        {"name": "v", "type": "vector",
         "attrs": {"dims": 8, "algorithm": "hnsw",
                   "distance_metric": "cosine", "datatype": "float32"}},
    ],
}

for cls in (SearchIndex, AsyncSearchIndex):
    gc.collect()
    before = sum(1 for o in gc.get_objects() if type(o) is cls)
    for _ in range(300):
        i = cls.from_dict(D)
        del i
    gc.collect()
    after = sum(1 for o in gc.get_objects() if type(o) is cls)
    print(f"{cls.__name__:17s}: {before} -> {after} live after 300 create/delete")

Actual:

SearchIndex      : 0 -> 300 live after 300 create/delete
AsyncSearchIndex : 0 -> 300 live after 300 create/delete

Expected: both -> 0. Every instance survives an explicit del plus a full gc.collect().

Confirmed on main (0.24.0) and reproduced back to 0.18.2 — the offending line is
unchanged across that range.

Real-world impact

Reported by a user running a document-ingestion service that constructs a SearchIndex
once per upload request: ~33.5 KB leaked per request, perfectly linear over 5,000
requests with no plateau; container RSS climbed 3.82 GB → 5.72 GB over 42 hours, strictly
monotonic, and pods reached their 7 GiB limit and were OOM-killed roughly every 4 days.
The leak is spread thinly across many small retained objects, so it is largely invisible
to a heap profiler and shows up only in RSS — making it hard to attribute.

Suggested fix

Register a finalizer callback that does not reference self. Note the client is
created lazily (it is None at __init__ when the index owns it), so the callback must
be able to reach the current client at finalization time — capturing the constructor
argument by value is not sufficient. A small mutable holder shared between the instance
and a module-level close helper satisfies both constraints.

Workaround for users

Reuse a single SearchIndex per logical index rather than constructing one per operation.
Note: calling disconnect() explicitly does not help — the finalizer registration
itself is what retains the object, regardless of whether the callback ever runs.

Environment

redisvl	0.18.2 through 0.24.0 (all affected)
redis-py	7.4.1
Python	3.12.11
OS	Linux (container)

Reported by Ivan Devitsyn. Also reproduced with the async client.