Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
3763370
enable CometLocalTableScanExec by default
mbutrovich May 21, 2026
6830b09
Merge branch 'main' into enable_localtablescan
mbutrovich May 22, 2026
810e5d5
add NullType to toArrowType
mbutrovich May 22, 2026
174c939
add NullType to shuffles
mbutrovich May 22, 2026
3790c10
fix windowexec test and nulltype. fix timetype issues
mbutrovich May 22, 2026
18cd14b
Fix TimeType test.
mbutrovich May 22, 2026
fc40d59
fix null value type in map in native shuffle
mbutrovich May 22, 2026
92cc260
Merge branch 'main' into enable_localtablescan
mbutrovich May 22, 2026
cf0c1df
Merge branch 'main' into enable_localtablescan
mbutrovich May 26, 2026
8c088a7
avoid reuse in LocalTableScanExec
mbutrovich May 26, 2026
0cdfafe
Merge branch 'main' into enable_localtablescan
mbutrovich May 26, 2026
bd04fb4
Replace Comet's bespoke CometBatchIterator JNI input path with the ca…
mbutrovich May 27, 2026
2742b49
Merge branch 'main' into enable_localtablescan
mbutrovich May 27, 2026
04597c0
Merge branch 'main' into enable_localtablescan
mbutrovich May 27, 2026
b6db996
Unpack dictionaries.
mbutrovich May 27, 2026
5ca923f
Merge branch 'main' into enable_localtablescan
mbutrovich May 27, 2026
cf7bb6e
Fix shading issue.
mbutrovich May 27, 2026
2560e6f
Merge remote-tracking branch 'origin/enable_localtablescan' into enab…
mbutrovich May 27, 2026
82c9a1b
Try again to fix shading issue.
mbutrovich May 27, 2026
6adf124
Fix alignment issue for FFI Decimal128 with ArrowArrayStreamReader
mbutrovich May 27, 2026
3da08dc
Merge branch 'refs/heads/main' into enable_localtablescan
mbutrovich May 28, 2026
0e08018
Fix schema mismatch in CometArrowStream.
mbutrovich May 28, 2026
a5046e3
Fix nullability mismatch in CometArrowStreamSuite.
mbutrovich May 28, 2026
8f1c35a
Fix format.
mbutrovich May 28, 2026
07e7944
remove non-ascii
mbutrovich May 29, 2026
14633f7
Merge branch 'main' into enable_localtablescan
mbutrovich Jun 2, 2026
6afbdce
add CometArrowStreamSuite to CI workflows
mbutrovich Jun 2, 2026
deb697a
fix withInfo use
mbutrovich Jun 2, 2026
51a3a0d
Merge branch 'main' into enable_localtablescan
mbutrovich Jun 2, 2026
f7fd3bd
Merge branch 'main' into enable_localtablescan
mbutrovich Jun 2, 2026
3a847fc
Merge branch 'main' into enable_localtablescan
mbutrovich Jun 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion spark/src/main/scala/org/apache/comet/CometConf.scala
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,7 @@ object CometConf extends ShimCometConf {
val COMET_EXEC_TAKE_ORDERED_AND_PROJECT_ENABLED: ConfigEntry[Boolean] =
createExecEnabledConfig("takeOrderedAndProject", defaultValue = true)
val COMET_EXEC_LOCAL_TABLE_SCAN_ENABLED: ConfigEntry[Boolean] =
createExecEnabledConfig("localTableScan", defaultValue = false)
createExecEnabledConfig("localTableScan", defaultValue = true)

val COMET_NATIVE_COLUMNAR_TO_ROW_ENABLED: ConfigEntry[Boolean] =
conf(s"$COMET_EXEC_CONFIG_PREFIX.columnarToRow.native.enabled")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,15 +108,7 @@ class CometWindowExecSuite extends CometTestBase {
val cometShuffles = collect(df2.queryExecution.executedPlan) {
case _: CometShuffleExchangeExec => true
}
if (shuffleMode == "jvm" || shuffleMode == "auto") {
assert(cometShuffles.length == 1)
} else {
// we fall back to Spark for shuffle because we do not support
// native shuffle with a LocalTableScan input, and we do not fall
// back to Comet columnar shuffle due to
// https://github.com/apache/datafusion-comet/issues/1248
assert(cometShuffles.isEmpty)
}
assert(cometShuffles.length == 1)
}
}
}
Expand Down
Loading