Thank you for your question. You have hit on an important design choice.
SynSQL relies on a strict fan out across all partitions.
- Wide tables: As you can see on the multi_omic live demo, queries include the hash key. And the hash key is always used in an implicit "order by" clause. Each chunk involved executes a query focused on the columns it contains.
- Deep tables: All data brokers execute the incoming query on their data set (partition).
Why fan out even for 'where pk = 1' ? The engine broadcasts to all partitions. It delegates expression evaluation downstream. It is designed for parallel scans. The overhead is perfectly acceptable for its target: analytical workload.
Thank you for your question. You have hit on an important design choice. SynSQL relies on a strict fan out across all partitions.
- Wide tables: As you can see on the multi_omic live demo, queries include the hash key. And the hash key is always used in an implicit "order by" clause. Each chunk involved executes a query focused on the columns it contains.
- Deep tables: All data brokers execute the incoming query on their data set (partition).
Why fan out even for 'where pk = 1' ? The engine broadcasts to all partitions. It delegates expression evaluation downstream. It is designed for parallel scans. The overhead is perfectly acceptable for its target: analytical workload.
Best, Stéphane.