perf(sql): rewrite trivial expressions over same column in GROUP BY queries

[nwoolmer]

Closes #4141

This relates to performance around Clickbench Q35.

For Clickbench Q35 on M2 Mac Mini, this speeds up the query from 1.7s to 0.9s.

The rewritten query runs using a Rosti implementation and an early limit, instead of async group by and a late limit.

With changes but async group by instead of rosti, it runs in 1.18s.

Query:

SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, COUNT(*) AS c 
FROM hits 
GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 
ORDER BY c DESC LIMIT 10;

Before change:

Sort light lo: 10
  keys: [c desc]
    VirtualRecord
      functions: [ClientIP,column,column1,column2,c]
        Async Group By workers: 8
          keys: [ClientIP,column,column1,column2]
          values: [count(*)]
          filter: null
            DataFrame
                Row forward scan
                Frame forward scan on: hits

Execute: 1.78s

After change:

VirtualRecord
  functions: [ClientIP,ClientIP-1,ClientIP-2,ClientIP-3,c]
    Sort light lo: 10
      keys: [c desc]
        GroupBy vectorized: true workers: 8
          keys: [ClientIP]          values: [count(*)]
            DataFrame
                Row forward scan
                Frame forward scan on: hits

Execute: 915.35ms

[nwoolmer]

nit: maybe this could be safely relaxed

[nwoolmer]

nit: is there a lighter option?

[puzpuzpuz]

You could move this map into a field and reuse it between the invocations.

[nwoolmer]

more tests would be good

[ideoma]

[PR Coverage check]

😍 pass : 49 / 50 (98.00%)

file detail

	path	covered line	new line	coverage
🔵	io/questdb/griffin/SqlOptimiser.java	48	49	97.96%
🔵	io/questdb/griffin/model/QueryModel.java	1	1	100.00%

[puzpuzpuz]

We need to mitigate perf degradation on the c6a.metal box (192 cores, 384GB RAM): 0.6s for Java vs 2.2s for Rosti. The reason is that Java code implements hash table sharding while C++ code doesn't. Maybe we could mitigate this by limiting the max number of Rosti tables in use here:

questdb/core/src/main/java/io/questdb/griffin/engine/groupby/vect/GroupByRecordCursorFactory.java

Line 82 in fe2aeca

this.workerCount = workerCount;

[puzpuzpuz]

We need to mitigate perf degradation on the c6a.metal box (192 cores, 384GB RAM): 0.6s for Java vs 2.2s for Rosti. The reason is that Java code implements hash table sharding while C++ code doesn't. Maybe we could mitigate this by limiting the max number of Rosti tables in use here

I have a better idea: we should disable keyed Rosti for all types but SYMBOL. That's because SYMBOL type has low cardinality, while it's not always the case for INT or IPv4. So, for high cardinality columns Java-based GROUP BY will be faster than the Rosti one.

@bluestreak01 WDYT?