dserban

  Location: Europe, Role Type: Fractional, Remote / Worldwide
  Remote: Yes
  Willing to relocate: Yes
  Technologies: Kafka, Flink, Spark, Cassandra, Scala, Rust / PyO3 / Tokio, Python, Zookeeper, Databricks, Delta Lake, BigQuery, Redshift, Hive, Kinesis, Airbyte, Airflow, Temporal, DBT, Aerospike, Snowflake, PrestoSQL, Trino, Clickhouse, SQL, Vector DBs, Golang, gRPC/protobuf, Terraform, CUDA
  Resume: https://drive.google.com/file/d/1lZuSpneLDVzzYoeaA8or-m5m2dfYAMB4/view
  Email: in the profile (please do mention "Via HN" in the subject for your email to land in the right place in my inbox)

[Remote Worldwide, Fractional] I'm pursuing a role in the high-scalability distributed systems space. I'm a well-rounded Scala/Rust/Python dev, well-versed in data engineering, with deep knowledge of the internals of distributed datastores. I have experience with data modeling for high throughput database activity and a strong understanding of which workloads and data access patterns are scalable and what datastore the data should reside in. I have a highly confident ability to lead a data / MLops project from start to finish. Hire me as fractional data engineer.

Core Focus Areas:

● Cassandra (Data Modeling, Troubleshooting Performance And Operational Issues)

● Apache Iceberg (Scaling, Tuning, Self-Hosted Setup)

● Stream Processing At Scale: Kafka, Flink, Spark Streaming, Storm

● Custom-Crafted Contextualized Embeddings, Vector-Based Semantic Search, Deep Intent Recognition In Search Engine Queries

● Languages: Scala, Rust, Python, SQL (proficient), Golang (ramping up)

● Expert-level kdb+/q, engine internals, query planning/optimization, C API extensions, performance analysis, hardware acceleration

Educational Background: Computer Science.

Solid experience working remotely and working with teams that are distributed geographically. I typically work Pacific Time hours.

Ask: $385K base, pro-rated.

Adding to your 6 and 7, Ed Zitron's Better Offline podcast has a good series on how the path was paved to the cost reckoning of the present day.

At some point, I needed to write a function which, given a collection of product titles, picks one that is neither the longest nor the shortest, it should pick the one which best captures the essence of the product while not being excessively verbose. For example, given the product titles below, it should pick "Portable Two-Way Translator, Handheld".

Portable Two-Way Translator, Handheld

Portable Handheld Translator

Handheld Two-Way Translator

Electronic Two-Way Portable Instant Voice Translator, 40 Languages, Handheld

Based on previous experience with centroid-based algos, the function I wrote does a first pass throwing all words from all product titles into one big bag, then computing a centroid (frequency histogram with low-frequency words removed). The second pass is to compute a cosine similarity score for each product title (its own frequency histogram against the centroid). Whichever product title is the most similar to the centroid wins.

That algo may have existed already in some academic paper somewhere, but I came up with it independently.

These roles are North America only, based on your careers page. Please consider adding a caveat to the "Remote" designation.