FinOrg

This is a demonstration repository, showcasing a potential monorepo setup including one microservice (ChainMonitor). It was created in a relatively short period, so caveat emptor.

Note:

• It is not production ready in any way.
• It is missing unit tests and it has not been e2e tested.
• It has a ton of TODO comments awaiting a batter day.

ChainMonitor

ChainMonitor is a microservice which monitors specified blockchain addresses in real time across multiple networks (Bitcoin, Ethereum, Solana), and produces a Kafka message whenever a transaction occurs on those addresses.

Specifically:

• gRPC API
  • basic lightweight API layer for managing user addresses to be monitored (add/remove), passing requests to the Address Repository
• Address Repository
  • PostreSQL-backed DAO layer for storing user addresses which should be monitored
  • Features an in-memory cache so that we don't hit the database on every blockchain block/transaction. The cache is seeded at startup and periodically refreshed. It's assumed that the delay in addresses being added/removed is acceptable.
• Blockchain Monitor
  • Blockchain Monitor consists of a series of blockchain readers (one per supported blockchain network), each launched as a separate goroutine, which:
    • connects to the blockchain network using a Blockdaemon client
    • at appropriate interval (defined on per-network basis) fetches new blocks (with transactions included)
    • attempts at parsing the block of transactions into a unified format (and does a terrible job at it - TODO)
    • passes the block to the Transaction Processor (TxProcessor)
• Transaction Processor
  • For every new transactions block:
    • Consults Redis (used as a block processing log) to verify whether the block has already been processed
    • If not, processes all transactions in the block:
      • Verifies whether the transaction concerns the user addresses being monitored (checking the Address Repository, and more specifically - its in-memory cache)
      • If it does - passes the transaction event to the Publisher
      • Marks the block as processed in Redis (if all transactions in the block have been successfully processed)
    • If it has been processed - skips it
• Publisher
  • Publishes the transaction event to a Kafka topic (or to STDOUT, depending on the implementation used)
  • Leverages Protobuf definitions for the serialization of the Kafka messages (including protovalidate for validation)

Structure

go/microservices/chainmonitor/internal/
├── api/v1/          # gRPC service handlers
├── monitor/         # Blockchain monitoring logic
└── repo/            # Database & caching layer

go/pkg/
├── clients/blockdaemon/  # BlockDaemon RPC client
├── messaging/            # Kafka producer/consume
├── pb/                   # Generated Protocol Buffers
└── util/                # Utility packages (grpc testing, logging, etc.)

proto/
├── finorg/api         # API gRPC Protobufs
└── finorg/messages    # Kafka message Protobufs

Setup

This code actually compiles. Who would have thought?

1. Make sure that the config in go/microservices/chainmonitor/main.go includes:

• Blockdaemon API key (BlockdaemonAPIKey)
• MonitorAll set to true (the DB will be initially empty, so let's just dump all transactions)

2. Start the DB and throw schema at it (it's not really necessary, with MonitorAll set to true, but the wire-up is too stupid to ignore it)

make start-db

3. Start the service and look at the lovely logs

go run ./go/microservices/chainmonitor/cmd/main.go

You should be able to use grpcurl to speak to the server. Hopefully.