Switch "crypto" implementation to Python

[jlaine]

This simplifies the build process and avoids needing to ship a vendor'd OpenSSL.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (483a66f) to head (aca1ddd).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #457   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           25        26    +1     
  Lines         5121      5186   +65     
=========================================
+ Hits          5121      5186   +65     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jlaine]

Initial testing suggest a 10x slowdown of packet protection compared to the current C code.

[rthalley]

I haven't tried measuring it yet. My prior experience is that cryptography is pretty fast, and I'm wondering if the header protection part might be the slow thing. If it were, it might be possible to do the crypto with cryptography and then do the masking with a tiny bit of C code.

[jlaine]

I haven't tried measuring it yet. My prior experience is that cryptography is pretty fast, and I'm wondering if the header protection part might be the slow thing. If it were, it might be possible to do the crypto with cryptography and then do the masking with a tiny bit of C code.

AFAICT it's the actual crypto code unfortunately that is slower.

[rthalley]

Can you say how you are benchmarking? I'm guessing I'm doing something wrong, but I'm testing by using the example client to pull 20MiB from nginx, and while it is slower it's not too bad, only around 1.074 times the non-cryptography version.

Regular aioquic (main from your branch with python-crypto):

2024-02-27 08:04:13,412 INFO client Response received for GET /foo.bin : 20971520 bytes in 0.4 s (428.916 Mbps)
2024-02-27 08:04:13,414 INFO quic [1da750bf1e625c3a] Connection close sent (code 0x100, reason )
python examples/http3_client.py https://localhost:8080/foo.bin  0.46s user 0.11s system 84% cpu 0.674 total

Your python-crypto branch:

024-02-27 08:12:30,026 INFO client Response received for GET /foo.bin : 20971520 bytes in 0.4 s (389.185 Mbps)
2024-02-27 08:12:30,028 INFO quic [254cea9a35deb3ff] Connection close sent (code 0x100, reason )
python examples/http3_client.py https://localhost:8080/foo.bin  0.41s user 0.07s system 66% cpu 0.724 total

[jlaine]

Here is what I use:

import binascii
import time

from aioquic.quic.crypto import CryptoPair

from tests.test_crypto import (
    LONG_CLIENT_PACKET_NUMBER,
    LONG_CLIENT_PLAIN_HEADER,
    LONG_CLIENT_PLAIN_PAYLOAD,
    PROTOCOL_VERSION,
)

pair = CryptoPair()
pair.setup_initial(
    cid=binascii.unhexlify("8394c8f03e515708"),
    is_client=True,
    version=PROTOCOL_VERSION,
)
start = time.time()
for i in range(100000):
    pair.encrypt_packet(
        LONG_CLIENT_PLAIN_HEADER,
        LONG_CLIENT_PLAIN_PAYLOAD,
        LONG_CLIENT_PACKET_NUMBER,
    )
elapsed = time.time() - start
print(elapsed)

A typical run for me puts the C code at about 70ms and the Python code at around 800ms.

Notes:

• Make sure you purge your build directory when switching to the python-crypto branch, otherwise you will still have the C version in the form of a .so :)
• The main branch of cryptography may perform differently as more code has been moved to rust.
• ChaCha20 is going to regress even further, but there is a WIP patch which helps: ChaCha20 API does not allow updating the nonce pyca/cryptography#10193

[rthalley]

Yeah, I discovered the need to purge while I was benchmarking. The timings I reported are after making sure I was testing in clean setups.

[rthalley]

And of course both of our testing could be accurate, as you're testing just the crypto whereas I'm testing what the overall effect in typical use is.

[jlaine]

I wish cryptography 43 were already out, preferably featuring:

pyca/cryptography#10437

As things currently stand, the build of the C code has broken again through no change of our own:

https://github.com/jlaine/aioquic/actions/runs/9150448224

[mvollrath]

Ran the benchmark today to see how well it performs on the optimized upstream cryptography. I modified the above posted bench to refer to test_crypto_v2 instead of test_crypto and increased the number of iterations to 1,000,000.

This branch, with the _crypto.abi3.so removed: 3.882s
main branch: 1.642s

Tested with Python 3.14 via uv, which resolved this tree:

$ uv tree
Resolved 14 packages in 1ms
aioquic
├── certifi v2025.11.12
├── cryptography v46.0.3
│   └── cffi v2.0.0
│       └── pycparser v2.23
├── pylsqpack v0.3.23
├── pyopenssl v25.3.0
│   └── cryptography v46.0.3 (*)
├── service-identity v24.2.0
│   ├── attrs v25.4.0
│   ├── cryptography v46.0.3 (*)
│   ├── pyasn1 v0.6.1
│   └── pyasn1-modules v0.4.2
│       └── pyasn1 v0.6.1
└── coverage[toml] v7.12.0 (extra: dev)
(*) Package tree already displayed

I think 250k encrypts/s is well within what Python users expect from Python.