feat(store): use RocksDB-backed persistent account state forest#2020
feat(store): use RocksDB-backed persistent account state forest#2020
Conversation
kkovaacs
force-pushed
the
krisztian/persistent-account-state-forest
branch
from
f32e0ca to
d1cd96e
Compare
kkovaacs
force-pushed
the
krisztian/persistent-account-state-forest
branch
from
d1cd96e to
a2b020f
Compare
|
Note: this PR is incorrect in the sense that account state forest get/set operations are now potentially blocking, so those should be performed on a blocking task. We already have this issue with the account and nullifier SMTs, and I'd prefer to solve that in a follow-up PR. |
Mirko-von-Leipzig
left a comment
Mirko-von-Leipzig
left a comment
There was a problem hiding this comment.
I think this looks good. Will maybe do another pass later.
| #[cfg(feature = "rocksdb")] | ||
| pub(crate) type AccountStateForestBackend = ForestPersistentBackend; | ||
| #[cfg(not(feature = "rocksdb"))] | ||
| pub(crate) type AccountStateForestBackend = ForestInMemoryBackend; |
There was a problem hiding this comment.
FYI I believe the latest rust version now supports the equivalent of cfg_if - cfg_select!.
I would also love to drop this feature flag now and exclusively use the rocksdb backend. iirc this was added while the rocksdb performance and stability werre questionable.
Co-authored-by: Mirko <48352201+Mirko-von-Leipzig@users.noreply.github.com>
| /// | ||
| /// This check ensures persisted account state forest storage has not diverged from the latest | ||
| /// account states in SQLite. When the forest is rebuilt from the database, it will naturally | ||
| /// match; when loaded from persistent storage, this catches corruption or incomplete shutdown. |
There was a problem hiding this comment.
nit: The terminology of persistent vs DB I find counterintuitive. My understanding is that persistent is referring to RocksDB while DB is referring to SQLite? Is there some more literal terminology we could use (given both are persistent, and both are databases). cc @Mirko-von-Leipzig
There was a problem hiding this comment.
Its a fair point, we should come up with something more concrete somehow.
sqlite and kv/key-value? Or alternatively we just use the raw names ala sqlite and rocksdb more consistently, avoiding database?
3 participants
As described in #1978 we're currently not using a persistent backend for the the AccountStateForest and rebuilding can take a long time (up to 9 minutes on our last testnet).
This PR implements the RocksDB-based persistent storage backend for the account state forest while keeping the existing logic for reconstructing its state from SQLite.
Upon startup the account state forest is checked. If it's empty we start reconstructing the state from SQLite. For a non-empty account state forest we check if vault and storage map commitments in the forest match data in SQLite for all public accounts.
For reference: I've used the stress test tool to generate a database with 50k public accounts, each with 2 vault assets and 64 storage map entries (in a single slot). Initializing the in-memory account state forest takes about 5 minutes, while initializing the RocksDB backed forest about 10 minutes. Loading and verifying persistent state (once persistent storage has been initialized) takes about 1.3s.