fix(delta): null partition value for composite generated-column partitions

[alealandreev]

What is the purpose of the pull request

When a Delta table is partitioned by a composite generated-column partition — e.g. year/month/day columns all generated from a single source column — the partition extractor reconstructs the partition value by joining the component values with Collectors.joining("-").

If the source column value is null, every generated component is null, and the join produced the literal string "null-null-null" (StringJoiner renders null elements as "null"). That corrupted value was then passed to the date parser, throwing ParseException: Unable to parse partition value and failing the snapshot.

This affected both the Spark-based DeltaPartitionExtractor and the DeltaKernelPartitionExtractor, which carried identical logic. A null partition value is the correct result here: Delta stores null partition values as null in the AddFile partitionValues map, and the __HIVE_DEFAULT_PARTITION__ sentinel only ever appears in the physical directory path.

Brief change log

• DeltaPartitionExtractor#getSerializedPartitionValue: return null when any component of a composite partition is null, consistent with the single-field branch (getOrDefault(name, null)).
• DeltaKernelPartitionExtractor#getSerializedPartitionValue: same fix.
• Added regression tests (unit + integration).

Verify this pull request

• Unit: testGeneratedPartitionValueExtractionWithNullSource in both TestDeltaPartitionExtractor and TestDeltaKernelPartitionExtractor — asserts the composite partition value resolves to null instead of "null-null-null".
• Integration: ITDeltaConversionSource#getCurrentSnapshotGenColPartitionedWithNullSourceTest — creates a Delta table partitioned by year/month/day generated from a nullable event_time, inserts a row with a null timestamp, and verifies the snapshot reads back one file with a null partition value (previously this threw while parsing the partition value).
• Verified locally: ITDeltaConversionSource — Tests run: 15, Failures: 0, Errors: 0.

[the-other-tim-brown]

@alealandreev are you running into these cases in datasets you are working with? I am curious how the dataset can get into a state where one of the fields is missing.

[vinishjail97]

@the-other-tim-brown The three partition columns are all generated from one source (event_time), so a row with a null event_time makes year/month/day all null at once — not one-of-three. It's just a nullable-timestamp scenario (optional / late-arriving / backfilled rows; generated columns are nullable unless constrained).

On write, Delta puts that file under …=__HIVE_DEFAULT_PARTITION__/… and parses it back into the AddFile partitionValues as genuine nulls — DelayedCommitProtocol.parsePartitions (v2.4.0 L121-146) casts the NULL literal produced by PartitionUtils (v2.4.0 L586-589) to a null string. On read both Delta (kernel literalForPartitionValue, v4.0.0 L479-482) and xtable take partition values from that map, where they are nulls — the __HIVE_DEFAULT_PARTITION__ sentinel only ever lives in the physical path.

So with all three components null, the old composite branch did Collectors.joining("-") over [null, null, null], which yields the literal string "null-null-null" (StringJoiner appends "null" for null elements — no NPE). convertFromDeltaPartitionValue then skips its null guard and, since this is a DAY transform, runs formatter.parse("null-null-null"), which throws org.apache.xtable.model.exception.ParseException: Unable to parse partition value. The fix returns null for the composite when any component is null, matching Delta's own read semantics.

This was never caught before because no existing partitioned test exercised a null partition value: testVariousOperations uses a single-field level:VALUE partition (never the multi-field join, never null), and the only generated partition (YEAR(birthDate)) is a single column. The new test is the first to hit the composite branch with a null source.