`from_reader()` assumes single-byte varints — breaks for multi-byte codec codes

[sumanjeet0012]

from_reader() reads the multihash code and length as single bytes, assuming they are always 1-byte varints. For codec codes > 0x7F (e.g., dag-cbor at 0x71, blake2b-256 at 0xB220), the varint encoding uses multiple bytes, causing incorrect parsing.

Problem

In cid/cid.py, the CIDv1 path of from_reader():

elif version == 1:
    # Read codec (varint)
    codec_bytes = bytearray()
    codec_bytes.append(first_byte[0])
    bytes_read = 1

    while True:
        byte = reader.read(1)
        codec_bytes.append(byte[0])
        bytes_read += 1
        if (byte[0] & 0x80) == 0:
            break

    # Now read multihash
    peek = reader.read(2)           # ← Assumes 2 bytes for mh code + length
    mh_length = int(peek[1])        # ← Assumes length is second byte
    remaining = mh_length
    multihash_bytes = reader.read(remaining)

The peek = reader.read(2) assumes the multihash code is 1 byte and the length is 1 byte. But:

• Multihash codes > 0x7F use multi-byte varints (e.g., blake2b-256 = 0xB220 = 3 bytes)
• Digest lengths > 127 use multi-byte varints

Similarly, the CIDv0 path has the same problem:

if version == 0:
    peek = reader.read(2)
    mh_length = int(peek[1])        # ← Assumes single-byte code and length

Proposed Solution

Use proper varint reading for all fields:

def from_reader(reader):
    import varint

    first_byte = reader.read(1)
    if not first_byte:
        raise ValueError("Not enough data to read CID")

    version = int(first_byte[0])

    if version == 0:
        # CIDv0: first byte is actually the multihash code
        # Read the multihash properly
        mh_code = version  # The first byte IS the mh code for CIDv0
        # ... use varint.decode_stream() for length ...
    elif version == 1:
        # Read codec varint
        codec_code = varint.decode_stream(reader)
        # Read multihash code varint
        mh_code = varint.decode_stream(reader)
        # Read multihash length varint
        mh_length = varint.decode_stream(reader)
        # Read digest
        digest = reader.read(mh_length)
        # ...

Related

• File: cid/cid.py, from_reader() function
• Dependency: varint library (already imported)