sagia

I had similiar issue, but for aggreagations. Use case was to "compress" large datasets into smaller aggregations for insertion into a costly db. At first we used duckdb but memory became an issue there and we also bumped into a couple of issues with how duckdb handles arrays. We then moved this workload to clickhouse local, which was faster and had more fine tuning options to our liking. in this case was limiting ram usage with i.e. max_bytes_before_external_group_by

    Location: Israel
    Remote: Yes
    Willing to relocate: Yes
    Technologies: go-lang, java, kotlin, javascript (ts if i must), linux, bash, k8s, helm, clickhouse, druid
    Other-notable-skills: People management, tech project management, AWS->GCP
    Résumé/CV: Available on request
    Email: asagia [at] icloud [dot] com

Hi, My name is sagi. In my current role I am a software engineering manager, in which I (hands-on) lead a very small team (5-6) of full stack engineers. Coming from ad-tech, I have managed to tackle multiple challenges in a variety of technologies and teams - be it Android SDKs to mid-scale (~2-10k qps) backends. Also, worked on multiple other products spanning from clickhouse analytical systems to full stack applications in whatever-gets-the-job-done fashion. I have 12 years (7 years in the current company, after 2 exits) of software engineering experience, 4 of them as a lead. Looking forward hearing about opportunities that might require someone who doesn't see any barriers between data-eng/backend/frontend/mobile/ops.

I wonder if a similar (but maybe more bloated) implementation using interfaces (and probably generics too?) will work in this case.

Decided to give a shot and do a short write up regarding an odd moment of writing unit tests for some functions that required randomness. hope this might help someone in the future stumbling on similar use cases

Quite impressive, it's amazing how embedded that "stop" animation the character does when he runs in my memory, used to play this quite a bit when i was small