jbleonesio
Joined 3 years ago
Karma
30
Submissions
7
Member since
Sep 13
Recent submissions
Comment · 2 years ago
While Squey does not claim to be as versatile as Paraview (it is not designed to visualize 3D mesh data for example) it is on the other hand focused on conducting iterative analyses over massive columnar datasets to improve its understanding and find weak signals and anomalies through the use of parallel coordinates, data series and scatter plots.
Comment · 2 years ago
Data needs to fit in RAM and graphics in VRAM. Let's say 100GB or more if you filter some rows during import. Data is ingested in a in-house database designed to refresh the ever changing selected rows as quickly as possible to conduct a true investigation. You can load as many parquet files as you want in one go provided they have the same structure. Any outlier in any visual representation will be drawn as this is a requirement to detect weak signals and anomalies
Comparisons with the tools you mentioned would indeed be interesting, writing a blog post would be a good idea I guess! I wrote a comparison with ELK here : https://squey.org/domains/cybersecurity/pentesteracademy-mac...
Comment · 2 years ago
Thanks!
It depends on the number of rows/columns and the types of the values, but the application displays a dialog asking you if you want to stop the import before completion when it feels like resources are being exhausted.
The software was specifically developed to be able to handle as much data as possible while remaining responsive so the workstation resources will likely be the bottleneck here.
On my 32GB development machine, I can easily load tens of millions rows with tens of columns.
Comment · 2 years ago
We are using BuildStream to export a flatpak application but the build system is indeed CMake with both Clang and GCC compiling the project without warnings.
Feel free to open an issue[1] on the project repository to further discuss about a MacOS port :)
Comment · 2 years ago
Version five indeed because it already has quite a bit of an history as an ex-proprietary product.
We pronounce it "Skwey" (like in "query") but you can really pronounce it as you wish since its not even an existing word x)
Comment · 2 years ago
Thanks for your feedback. Unfortunately there is currently only a Linux build (which happens to also be running under Windows thanks to WSL2) because there is a lot of dependencies[1] to build. Any help to implement a MacOS build would of course be warmly welcomed :)
In the meantime, you can deploy the software from AWS Marketplace[2] and use it through your web browser but note that this is an on-demand paying product.
[1]: https://gitlab.com/squey/squey/-/tree/main/buildstream/eleme...
[2]: https://aws.amazon.com/marketplace/pp/prodview-l363lrih42bhm