ttmahdy
Joined 10 years ago
Karma
8
Submissions
4
Member since
Oct 27
Recent submissions
Comment · 5 years ago
With Snowflake data is locked away in a proprietary format not accessible by other compute platforms. You need to export/copy your data to a different system to train an ML model in python or R.
With the Databricks, you can use python, R and Scala, (not just SQL) to interface with your data. You can use multiple compute engines (Spark, presto and other engines that support Delta) so you are not locked into one compute engine.
Comment · 5 years ago
Exactly, companies learnt from Exasol
Out of the box performance is the name of the game
Executing a benchmark as complex as TPC-DS without tuning by Databricks or Snowflake is a big accomplishment
Comment · 5 years ago
Usually making a 1 ms query execute in 0.5 ms is a lot harder than making a 10 second query execute in 5 second.
One of the benefits of geometric mean is that all queries have "equal" weight in the metric, this keeps vendors from focusing on the long running queries and ignoring the short running ones. It is one way to balance between long and short query performance.
A similar concept is applied to TPC-DS for data load, single user run (Power), multi user run (Throughput) and data maintenance (Concurrent Delete and Inserts).
Check clause 7.6.3.1 in the TPC-Ds spec in http://tpc.org/tpc_documents_current_versions/pdf/tpc-ds_v3....
Comment · 5 years ago
The TPC audit process tends to be thorough and strict.
Possibly you missed a configuration that was included in the Full Disclosure Report or Supporting Files?
The Databricks official, audited benchmark was executed against Databricks SQL which is a PaaS service that doesn't allow special tuning btw.