~1.5x perf drop in Parquet query performance at commit 5bf0db0

[philrz]

When querying a Parquet file, the following query shows a ~1.5x performance drop at commit 5bf0db0 compared to the commit just prior. On my Intel-based laptop, the drop is from ~6.7 seconds to ~10.5 seconds.

$ aws s3 cp --quiet s3://brim-sampledata/mgbench/bench3.parquet . &&
  super -version &&  
  hyperfine "super -c \"
from 'bench3.parquet' | head 3000000
| join (from 'bench3.parquet' | head 3000000) on left.log_time=right.log_time
| count()\""

Version: v0.3.0-13-g5bf0db0d1
Benchmark 1: super -c "
from 'bench3.parquet' | head 3000000
| join (from 'bench3.parquet' | head 3000000) on left.log_time=right.log_time
| count()"
  Time (mean ± σ):     10.456 s ±  0.571 s    [User: 19.904 s, System: 0.624 s]
  Range (min … max):    9.558 s … 10.919 s    10 runs

$ super -version &&
  hyperfine "super -c \"
from 'bench3.parquet' | head 3000000
| join (from 'bench3.parquet' | head 3000000) on left.log_time=right.log_time
| count()\""

Version: v0.3.0-12-g2afddff09
Benchmark 1: super -c "
from 'bench3.parquet' | head 3000000
| join (from 'bench3.parquet' | head 3000000) on left.log_time=right.log_time
| count()"
  Time (mean ± σ):      6.726 s ±  0.537 s    [User: 11.685 s, System: 0.517 s]
  Range (min … max):    6.143 s …  7.273 s    10 runs

Details

Repro is with super commit 5bf0db0, which is associated with the merge of #6776. The query above is a simplification of the mgbench bench3/q2 query where the symptom was first noticed.

After performing the binary search to spot where the perf changed, I used Claude to help create the simplified repro query above and offer its take on the root cause and ideas for addressing it, which is captured in a Gist titled "Performance regression: TypeDefs.LookupTypeRecord re-resolves the full type tree on every call, doubling join-heavy queries".

[philrz]

Verified in super commit 9ec409f.

There's been many other changes in the code since this perf drop was first logged, so here's a re-baseline at commit 04bf8c3 right before the merge of the change from linked PR #7252. At this commit the time of the repro query on my Intel Macbook comes in at ~8.3 seconds, which happens to be better than the ~10.5 seconds it was at when this issue was first logged.

$ super -version &&  
  hyperfine "super -c \"
from 'bench3.parquet' | head 3000000
| join (from 'bench3.parquet' | head 3000000) on left.log_time=right.log_time
| count()\""

Version: v0.3.0-350-g04bf8c3bd
Benchmark 1: super -c "
from 'bench3.parquet' | head 3000000
| join (from 'bench3.parquet' | head 3000000) on left.log_time=right.log_time
| count()"
  Time (mean ± σ):      8.300 s ±  0.491 s    [User: 14.774 s, System: 0.532 s]
  Range (min … max):    7.947 s …  9.249 s    10 runs

And now at commit 9ec409f that has the benefit of the changes from #7252, it's improved to ~7.0 seconds, which is much improved and just shy of the ~6.7 seconds we were at before the issues was originally filed.

$ super -version &&     hyperfine "super -c \"
from 'bench3.parquet' | head 3000000
| join (from 'bench3.parquet' | head 3000000) on left.log_time=right.log_time
| count()\""

Version: v0.3.0-351-g9ec409fe3
Benchmark 1: super -c "
from 'bench3.parquet' | head 3000000
| join (from 'bench3.parquet' | head 3000000) on left.log_time=right.log_time
| count()"
  Time (mean ± σ):      6.968 s ±  0.486 s    [User: 11.962 s, System: 0.513 s]
  Range (min … max):    6.163 s …  7.405 s    10 runs

As a separate exercise, I'll soon get results from a full re-run of the mgbench benchmarks to log the "official" state of the recovery on the reference AWS EC2 instance type that's used for formally tracking perf regressions.

Thanks @nwt!