Python IPLD

This project aims to speed up The AT Protocol SDK by using Rust for the heavy lifting. Only atproto related parts are implemented first.

Code snippet:

import libipld

# CID
print(libipld.decode_cid('bafyreig7jbijxpn4lfhvnvyuwf5u5jyhd7begxwyiqe7ingwxycjdqjjoa'))
# Output: {'hash': {'size': 32, 'code': 18, 'digest': b'\xdfHP\x9b\xbd\xbcYOV\xd7\x14\xb1{N\xa7\x07\x1f\xc2C^\xd8D\t\xf44\xd6\xbe\x04\x91\xc1)p'}, 'version': 1, 'codec': 113}
print(libipld.encode_cid(b'\x01q\x12 \xb6\x81\x1a\x1d\x7f\x8c\x17\x91\xdam\x1bO\x13m\xc0\xe2&y\xea\xfe\xaaX\xd6M~/\xaa\xd5\x89\x0e\x9d\x9c'))
# Output: bafyreifwqenb274mc6i5u3i3j4jw3qhcez46v7vkldle27rpvlkysdu5tq

# DAG-CBOR
print(libipld.decode_dag_cbor(b'\xa2aa\x0cabfhello!'))
# Output: {'a': 12, 'b': 'hello!'}
print(libipld.encode_dag_cbor({'a': 12, 'b': 'hello!'}))
# Output: b'\xa2aa\x0cabfhello!'

# multibase
print(libipld.decode_multibase('ueWVzIG1hbmkgIQ'))
# Output: ('u', b'yes mani !')
print(libipld.encode_multibase('u', b'yes mani !'))
# Output: ueWVzIG1hbmkgIQ

Features

🔗 CID (Content Identifier) Operations

• decode_cid(data: str | bytes) -> dict - Decode CIDs from string representation (e.g., 'bafy...') or raw bytes into structured data containing version, codec, and hash information
• encode_cid(data: str | bytes) -> str - Encode CID raw bytes to string representation, or return string CIDs as-is

📦 DAG-CBOR (Directed Acyclic Graph CBOR) Operations

• decode_dag_cbor(data: bytes) -> Any - Decode DAG-CBOR binary data into Python objects (dicts, lists, primitives)
• decode_dag_cbor_multi(data: bytes) -> list[Any] - Decode multiple concatenated DAG-CBOR objects from a single byte stream
• encode_dag_cbor(data: Any) -> bytes - Encode Python objects into DAG-CBOR binary format

🌐 Multibase Operations

• decode_multibase(data: str) -> tuple[str, bytes] - Decode multibase-encoded strings, returning the base identifier and decoded data
• encode_multibase(code: str, data: str | bytes) -> str - Encode data using specified multibase encoding (e.g., base58btc with code 'u')

🚗 CAR (Content Addressable Archives) Operations

• decode_car(data: bytes) -> tuple[dict, dict[bytes, dict]] - Decode CAR files into header metadata and a mapping of CID bytes to block data

Requirements

• Python 3.8 or higher.

Installing

You can install or upgrade libipld via

pip install -U libipld

Performance

Benchmarks against cbrrr (C), py-ipld-dag (Python wrapper over Rust-backed cbor2) and dag_cbor (pure Python), measured on the four classic nativejson-benchmark fixtures (round-tripped through DAG-CBOR). Bars are operations/second relative to pure-Python dag_cbor; higher is better.

Measured on Apple M1, macOS 15 (Darwin 24.6.0), CPython 3.14.0, libipld installed from PyPI (PGO + LTO wheel).

Deserialization

Serialization

Reproduce locally:

cd benchmark && ./run.sh

See benchmark/README.md for details.

Contributing

Contributions of all sizes are welcome. See CONTRIBUTING.md for setup, build, test, and benchmarking workflow.

License

MIT – see LICENSE.