I still didn’t get why they built this, there’s a better explanation of the feature set in the FAQ comparison with parquet: https://docs.arcticdb.io/latest/faq/
> How does ArcticDB differ from Apache Parquet?¶
> Both ArcticDB and Parquet enable the storage of columnar data without requiring additional infrastructure.
> ArcticDB however uses a custom storage format that means it offers the following functionality over Parquet:
> Versioned modifications ("time travel") - ArcticDB is bitemporal.
> Timeseries indexes. ArcticDB is a timeseries database and as such is optimised for slicing and dicing timeseries data containing billions of rows.
> Data discovery - ArcticDB is built for teams. Data is structured into libraries and symbols rather than raw filepaths.
> Support for streaming data. ArcticDB is a fully functional streaming/tick database, enabling the storage of both batch and streaming data.
> Support for "dynamic schemas" - ArcticDB supports datasets with changing schemas (column sets) over time.
> Support for automatic data deduplication.
The other answer I was looking for was why not kdb since this is a hedge fund.
> How does ArcticDB differ from Apache Parquet?¶
> Both ArcticDB and Parquet enable the storage of columnar data without requiring additional infrastructure.
> ArcticDB however uses a custom storage format that means it offers the following functionality over Parquet:
> Versioned modifications ("time travel") - ArcticDB is bitemporal. > Timeseries indexes. ArcticDB is a timeseries database and as such is optimised for slicing and dicing timeseries data containing billions of rows. > Data discovery - ArcticDB is built for teams. Data is structured into libraries and symbols rather than raw filepaths. > Support for streaming data. ArcticDB is a fully functional streaming/tick database, enabling the storage of both batch and streaming data. > Support for "dynamic schemas" - ArcticDB supports datasets with changing schemas (column sets) over time. > Support for automatic data deduplication.
The other answer I was looking for was why not kdb since this is a hedge fund.