blr246
Joined 10 years ago
Joined 10 years ago
I like your breakdown, and I've observed similar things in my experience as an engineering focused data person! I've had many discussions with my colleagues about how to manage effectively these different blends of roles and skills.
I'm looking for someone for an engineering type of data role right now. Is there a way to get in touch with you about it?
Our product helps companies listen to their customers by unifying natural language feedback across various channels, applying signals using various natural language modeling techniques, then aggregating them to help teams deliver better outcomes using more relevant information.
Hope to hear from you (brandon at frame.ai) :)
edit: forgot to share agreement for your breakdown
The 1960s and 1970s crisis had a lot to do with the end of NYC's industrial epoch. Suburban development and globalization eliminated manufacturing and pulled workers and residents out of the city. The recovery of NYC was bringing high-value services, retail, and tourism back along with arts and culture.
In the time since, NYC has become increasingly a luxury experience, which is indeed part of its strength but also its weakness, since it accelerates decline when people can up and leave without having roots.
The feels correct to me.
I lived in New York City for 15 years. Until last year. I've thought about this theme all year. Decades of policy supporting foreign investment and developer speculation gutted the chance for even affluent upper middle class New Yorkers to afford housing and setup a home base, and so many left. The situation has been incomparably more challenging for low income residents.
I agree the urban collapse meme is much easier to spread than a thoughtful discussion about policy and priorities and how to balance the economic strength of a city's major players with the daily priorities of everyday citizens. I hope the New York remainders shift priorities and initiate a different kind of prosperous era than the one I got to enjoy.
The article discusses a low-dimensional KNN problem. The curse of dimensionality guides intuition that the methods here likely will not apply to extremely high-dimensional problems.
faiss actually comes with a lot of excellent documentation that describes the problems unique to KNN on embedding vectors. In particular, for extremely large datasets, most of the tractable methods are approximations that make use of clustering, quantization, and centriod-difference tricks to make computation efficient.
See https://github.com/facebookresearch/faiss/wiki/Faiss-indexes and related links for more information.
We've frequently had the same dream of adding more native support for nearest-neighbor type queries, since that is the workhorse of so many useful techniques in the modern NLP stack.
Right now, we have lots of dense vectors stored in massive toast tables in PG. It's faster to fetch them rather than recompute them, especially since there are a number of preprocessing steps that limit what we pay attention to.
The discussion here about full text search versus semantic search is interesting. In our experience, both are highly relevant. Sometimes it's most useful for our customers to segment their conversation data by exact text matches, and other times semantic clustering is most effective. I think there's plenty of reason to offer both kinds of capabilities.
I can't find a way to disable this feature. Does anybody know how?
The reason this bad practice is common is that it is allowed by the spec in https://tools.ietf.org/html/rfc6749#section-6 as an optional action to take on refresh grants. Please, do not do this.
They want to have a rapid response path (little to no delay using staging envs) to respond to emergencies. The old SOP allowed all releases to use the emergency path. By not using it in the SOP anymore, I'd be concerned that it would break silently from some other refactor or change.
Your notion is to maintain the emergency rollout as a relaxation of the new SOP such that the time in staging is reduced to almost nothing. That sounds like a good idea since it avoids maintaining two processes and having greater risk of breakage. So, same logic but using different thresholds versus two independent processes.
5. Changing the SOP to do staged rollouts of rules in
the same manner used for other software at Cloudflare
while retaining the ability to do emergency global
deployment for active attacks.
One concern I'd have is whether or not I'm exercising the global rollout procedure often enough to be confident it works when it's needed. Of the hundreds of WAF rule changes rolled out every month, how many are global emergencies?It's a fact of managing process that branches are liability and the hot path is the thing that will have the highest level of reliability. I wonder if anyone there has concerns about diluting the rapid response path (the one having the highest associated risk) by making this process change.
edit: fix verbatim formatting
Good riddance to password expiration.
Each shard allows at most 5 GetRecords operations per second. If you want to fan out to many consumers, you will reach those limits quickly and have to implement a significant latency/throughput tradeoff to make it work.
For API limits, see: https://docs.aws.amazon.com/kinesis/latest/APIReference/API_...