Skip to content

Releases: MarshalX/python-libipld

v3.4.1

27 May 17:31
7361420

Choose a tag to compare

What's Changed

This is a focused decode-side optimization. The numbers below come from pytest-benchmark on Apple M1 / CPython 3.14 (PGO-built wheels), comparing v3.4.0 → v3.4.1 head-to-head.

⚡ Decode (DAG-CBOR)

Fixture v3.4.0 v3.4.1 Speedup
canada 4.75 ms 4.43 ms 1.07x (+7%)
github 69 µs 66 µs 1.05x (+5%)
twitter 790 µs 766 µs 1.03x (+3%)
citm_catalog 1.79 ms 1.77 ms 1.01x (+1%)

The decoder used to fetch the Python recursion limit on every recursive step. We now read it once per top-level decode call and pass it through, closing the last decode gap with cbrrr (we're within 0.7% of it on canada and 30–40% ahead on the others).

🔧 Optimizations

  • Optimize DAG-CBOR decode by reading Py_GetRecursionLimit once per call by @MarshalX in #NNN

Deserialization v3.4.0 → v3.4.1

image

Serialization v3.4.0 → v3.4.1

image

Full Changelog: v3.4.0...v3.4.1

v3.4.0

26 May 19:51
d8bee00

Choose a tag to compare

What's Changed

❗ Up to ~2x faster DAG-CBOR encode and decode!

This release is all about throughput. The numbers below come from pytest-benchmark on an Apple M1 / CPython 3.14, comparing v3.3.2 → v3.4.0 head-to-head.

⚡ Decode (DAG-CBOR)

Fixture v3.3.2 v3.4.0 Speedup
github 138 µs 70 µs 1.96x (+96%)
twitter 1.44 ms 0.80 ms 1.81x (+81%)
citm_catalog 2.87 ms 1.81 ms 1.59x (+59%)
canada 5.49 ms 4.77 ms 1.15x (+15%)

⚡ Encode (DAG-CBOR)

Fixture v3.3.2 v3.4.0 Speedup
canada 2.31 ms 1.17 ms 1.98x (+98%)
citm_catalog 2.25 ms 1.26 ms 1.79x (+79%)
twitter 902 µs 635 µs 1.42x (+42%)
github 75.4 µs 69.7 µs 1.08x (+8%)

🔧 Optimizations

  • Optimize DAG-CBOR encode/decode and CAR block parsing by @MarshalX in #104
  • Optimize DAG-CBOR string decode with an ASCII fast path by @MarshalX in #105
  • Optimize DAG-CBOR map decode with a key-intern cache and KnownHash insert by @MarshalX in #106
  • Optimize int encode with a direct PyLong digit read by @MarshalX in #107

🧰 Toolchain & Build

  • Bump to Rust 1.95 and PyO3 0.28 by @MarshalX in #102
  • Drop the macos-13 PGO runner; cross-compile x86_64 macOS wheels from arm64 by @MarshalX in #108
  • Bump CI actions (Node 24) and pytest-codspeed v5 by @MarshalX in #103

📊 Benchmarks & Docs

  • Add a head-to-head benchmark against other Python DAG-CBOR libs by @MarshalX in #109
  • Update README and copyright years by @MarshalX in #110

Deserialization v3.3.2 → v3.4.0

image

Serialization v3.3.2 → v3.4.0

image
🏎️ Old-style performance comparison. Before (v3.3.2) VS After (v3.4.0): image

Full Changelog: v3.3.2...v3.4.0

v3.3.2

05 Dec 12:36
e88185d

Choose a tag to compare

What's Changed

Full Changelog: v3.3.1...v3.3.2

v3.3.1

03 Dec 15:44
7ab7633

Choose a tag to compare

What's Changed

  • Add DASL testing harnesses to CI workflow by @MarshalX in #97
  • Fix error handling in array and map length checks by @MarshalX in #95
  • Fix error handling in CID and ULEB128 processing functions by @MarshalX in #96

Full Changelog: v3.3.0...v3.3.1

v3.3.0

01 Dec 14:28
ff16379

Choose a tag to compare

What's Changed

❗ Almost x2 speed for CAR decoding (Bluesky firehose), ~25-35% faster DAG-CBOR decoding, and ~17% faster CID decoding!

I do have more ideas on what to try to speed it up even further! For example, applying BOLT on top of the existing PGO.

⚡ Optimizations

  • Huge performance gain for DAG-CBOR decoding and CID by replacing deprecated Rust crate libipld with cbor4ii by @vmx in #80 and even contributed upstream!
  • Optimize DAG-CBOR encoding for arrays and maps by @MarshalX in #88
  • Optimize CAR decoding by reducing allocations by @MarshalX in #90
  • Optimize executable size (panic = "abort") by @MarshalX in #86

Other

🏎️ Performance comparison. Before (v3.2.0) VS After (v3.3.0):

image

You can find more benchmark results here: #92

New Contributors

  • @vmx made their first contribution in #80

Full Changelog: v3.2.0...v3.3.0

v3.2.0

22 Oct 14:55
53b68cd

Choose a tag to compare

What's Changed

❗ Possible breaking change for you: dropped support for PyPy 3.9 and PyPy 3.10 (both no longer supported upstream) in #77. CPython is still supported from 3.8 to 3.14.

New Contributors

Full Changelog: v3.1.1...v3.2.0

v3.1.1

24 Jun 22:37
3a0647c

Choose a tag to compare

What's Changed

Full Changelog: v3.1.0...v3.1.1

v3.1.0

20 Jun 23:15
d23c69a

Choose a tag to compare

What's Changed

New Contributors

Full Changelog: v3.0.1...v3.1.0

v3.0.1

18 Feb 10:48
a3b2a64

Choose a tag to compare

What's Changed

  • Update PyO3 to 0.23.4 by @MarshalX in #54
  • Specify the dynamic version in pyproject.toml by @fxcoudert in #53
  • Fix stack overflow when decoding deeply nested structures like arrays and maps by @MarshalX in #51

Possible breaking change for you: dropped support for PyPy 3.8

New Contributors

Full Changelog: v3.0.0...v3.0.1

v3.0.0

24 Oct 11:33
7bf9fb9

Choose a tag to compare

What's Changed

❗ Breaking changes

🆕 Features

Other

  • Update PyO3 to 0.22.5 by @MarshalX in #50
  • Update decode_car signature, bytes CIDs instead of str in README by @snarfed in #49

New Contributors

Full Changelog: v2.0.0...v3.0.0