Issues

• External sort OOM error tells Comet users to change datafusion.runtime.memory_limit and datafusion.execution.sort_spill_reservation_bytes

  requires-triage
  apache/datafusion-comet#7102 · Filter by author viirya opened on Oct 18, 2026

• docs: update Scala comments that reference the removed iceberg-writes.md page

  requires-triage
  apache/datafusion-comet#7101 · Filter by author andygrove opened on Oct 18, 2026

• perf: native Parquet scans fetch each file and row group only after the previous one is decoded

  area:scanenhancementperformance
  apache/datafusion-comet#7093 · Filter by author andygrove opened on Oct 17, 2026 · 1.2.0
  1

  

• BoundedWindowAggExec reserves no memory for the rows its window frames keep

  area:memorybugrequires-triage
  apache/datafusion-comet#7092 · Filter by author andygrove opened on Oct 17, 2026

• Spill limit error tells Comet users to raise datafusion.runtime.max_temp_directory_size

  enhancementrequires-triage
  apache/datafusion-comet#7091 · Filter by author andygrove opened on Oct 17, 2026
  1

• fair_unified's per-consumer limit does not bind when tasks run concurrently, so one consumer can starve another

  bugrequires-triage
  apache/datafusion-comet#7088 · Filter by author andygrove opened on Oct 17, 2026 · 1.2.0
  

• Replace remaining unwrap calls on protobuf input in the native planner with errors

  requires-triage
  apache/datafusion-comet#7086 · Filter by author 0lai0 opened on Oct 17, 2026
  1

  

• map_from_entries dispatches a calendar-interval result into the microsecond overflow

  requires-triage
  apache/datafusion-comet#7083 · Filter by author dwsmith1983 opened on Oct 17, 2026
  1

  

• bench: add the shuffle benchmark cases that the #5905 review found missing

  area:shuffleenhancementgood first issueperformancerequires-triage
  apache/datafusion-comet#7080 · Filter by author andygrove opened on Oct 16, 2026
  1

  

• perf: Celeborn push completion is polled every 10 ms and the transport hook proxies every channel of the shared factory

  area:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7079 · Filter by author andygrove opened on Oct 16, 2026

• perf: the native Celeborn writer pays a large fixed cost per frame (admission, schema, estimates, re-encodes)

  area:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7078 · Filter by author andygrove opened on Oct 16, 2026

• perf: the native Celeborn writer sends one frame per partition chunk with no coalescing

  area:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7077 · Filter by author andygrove opened on Oct 16, 2026

• perf: the native Celeborn writer reserves about 7x a batch and holds it through the network round trip

  area:memoryarea:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7076 · Filter by author andygrove opened on Oct 16, 2026

• perf: the JVM shuffle's native radix sort always runs six passes, and the JVM reserves scratch it never uses

  area:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7075 · Filter by author andygrove opened on Oct 16, 2026

• perf: the JVM columnar shuffle insert loop boxes, locks and copies every row more than once

  area:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7074 · Filter by author andygrove opened on Oct 16, 2026

• perf: a shuffle over a native plan takes the JVM row shuffle for round-robin and nested hash partitioning

  area:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7073 · Filter by author andygrove opened on Oct 16, 2026

• perf: the native multi-partition shuffle writer copies each partition's tail batch and flushes the output file once per partition

  area:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7071 · Filter by author andygrove opened on Oct 16, 2026
  

• perf: NativeBatchDecoderIterator discards its direct buffer after every block over 128 KiB

  area:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7070 · Filter by author andygrove opened on Oct 16, 2026

• perf: investigate decoding shuffle blocks ahead of the consumer in native shuffle read

  area:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7069 · Filter by author andygrove opened on Oct 16, 2026

• perf: a stage that reads shuffle and ends in columnar-to-row crosses the FFI boundary twice per batch

  area:ffiarea:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7068 · Filter by author andygrove opened on Oct 16, 2026

• perf: the shuffle reader builds a new decompressor and zeroes a fresh body buffer for every block

  area:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7067 · Filter by author andygrove opened on Oct 16, 2026

• perf: Arrow IPC write paths still copy through Channels.newChannel's 8 KiB buffer

  enhancementperformancerequires-triage
  apache/datafusion-comet#7064 · Filter by author andygrove opened on Oct 16, 2026

• Native hashing of NullType and ANSI interval values fails with "Unsupported data type in hasher"

  area:expressionsarea:shufflebugpriority:highrequires-triage
  apache/datafusion-comet#7063 · Filter by author andygrove opened on Oct 16, 2026

• perf: the sort-based JVM columnar shuffle is 2x slower than Spark at 2,000 partitions because every partition write pays builder, file and codec setup

  area:shuffleenhancementperformancerequires-triage
  apache/datafusion-comet#7061 · Filter by author andygrove opened on Oct 16, 2026

• perf: every native plan creates and deletes a temp directory on each Spark local disk, even when it never spills

  enhancementperformancerequires-triage
  apache/datafusion-comet#7060 · Filter by author andygrove opened on Oct 16, 2026