[Bug][QDP] Fix List<T> readers failing on nullable outer rows

[0lai0]

Related Issues

Closes #1401
Part of #1338

Changes

• Bug fix
• New feature
• Refactoring
• Documentation
• Test
• CI/CD pipeline
• Other

Why

Reading a nullable List<T> column with a null outer row (e.g. [[1, 2], null, [3, 4]]) fails with:
InvalidInput("Inconsistent sample sizes: expected 2, got 0")
Root cause: sample-size validation calls ListArray::value_length(i) without checking is_null(i). For a null outer row, Arrow returns 0, which the reader treats as a length-0 list.

If a non-null row was seen first → validation fails with expected N, got 0
If the null row is row 0 → sample_size is seeded to 0 and corrupts all subsequent rows
This is pre-existing (not introduced by #1393) and affects three readers:

• ParquetReader
• ParquetStreamingReader
• ArrowIPCReader

NullHandling::FillZero does not help because the failure happens at list-length validation, before value filling runs.

How

Null outer row semantics

Policy	Behavior
Reject	Return InvalidInput with a clear message
FillZero	Fill sample_size zeros once sample_size is known
sample_size unknown	Skip the null row (no zeros, not counted in num_samples) — e.g. leading all-null batch before any non-null row establishes size

Code changes

1. ParquetReader (parquet.rs)

   • Add is_null(i) guard in sample-size validation loop
   • Split fast path (no null outer rows) vs slow path (per-row null handling per NullHandling)

2. ParquetStreamingReader (parquet.rs)

   • Seed sample_size from first non-null row instead of row 0
   • Skip all-null batches when size is unknown
   • Same null-handling policy as batch reader
   • Defensive .max(1) on sample-size divisor in read_batch

3. ArrowIPCReader (arrow_ipc.rs)

   • Two-phase List handling
   • Phase 1: validate size from non-null rows only
   • Phase 2: collect data with the same null policy

Checklist

• Added or updated unit tests for all changes
• Added or updated documentation for all changes

[rich7420]

nit: In Reject mode, when the first batch is entirely null and sample_size is still unknown, None => continue returns before null_handling is consulted — so this batch is silently skipped instead of rejected. The batch ParquetReader errors on the same input, so the two readers diverge here. It's also the one Reject path without test coverage. Suggest either erroring on Reject here too, or documenting the difference explicitly.

[rich7420]

@0lai0 thanks for the patch!

[ryankert01]

lgtm!