squarecog
Joined 17 years ago
Karma
350
Submissions
72
Member since
Aug 13
Recent submissions
Comment · 2 years ago
Comment · 3 years ago
LLMs are Foundational Models, but not all Foundational Models are LLMs.
If you are interested in this space, it does probably make sense to look up what Stable Diffusion (StabilityAI) and Claude (Anthropic) are. Not recognizing them is fine, the pace is pretty incredible in this space and there's lots to learn, but it doesn't make the point you think you are making.
Comment · 4 years ago
LinkedDataHub, a "RDF-native notebook", is not to be confused with LinkedIn DataHub, which is a metadata store/crawler/ui for your data systems: https://datahubproject.io/.
Comment · 4 years ago
I should've added, there's an obvious example for the "SaaS control plane" separation, which is equivalent: "stop processing job X that is destabilizing the cluster" should be processed without needing to fight for resources with job X. Same for ACL changes, user deactivations, etc etc. It's generally a good idea to have your control stuff not be subject to whatever instabilities you might be controlling against.
Comment · 4 years ago
These terms usually show up in the context of networking protocols. Cloudflare has a very quick explainer: https://www.cloudflare.com/learning/network-layer/what-is-th.... To make it even shorter: a control plane is where all the coordination that controls activity (data) happens. The data plane is where the data actually moves around.
Explainers seem to not cover _why_ you would want to separate these "planes". There are several reasons, and I'm no authority, but for starters: * control messages will have different expectations around them: their amount and frequency, delivery guarantees, urgency with which they are processed. Treating this traffic separately means you can engineer appropriately for data and control traffic. * last thing you want is the control message "stop processing traffic from IP x.x.x.x port y" to be stuck behind traffic from said IP/port...
In this context, the meaning is somewhat different. They are referring to administrative traffic vs "actual work" traffic. Auth, billing/accounting, configuration updates, that sort of thing. If you are running a SaaS, and your customer is very security conscious and wants none of their precious data to ever leave their VPC, you have 2 options: deploy your software into their VPC completely, making it hard to do a variety of things like upgrades, and increasing complexity; or you can separate control actions from your "worker nodes" and storage, and only deploy the latter into the VPC. You can then work on your control panels, monitor usage, continuously evolve various admin panels and config options, etc, using normal SaaS approaches while the security conscious customer knows that their core data is not leaving their virtual walls and only "bob ran a thing and stored results" goes to the vendor.
This post is about abstracting out common bits of how one implements that, and allowing SaaS offerings to provide that sort of separation easier.
Comment · 5 years ago
Now? No idea. At the time I left in 2016, all Storm usage was replaced by Heron, that was the point of making it API-compatible.
Comment · 5 years ago
I was in charge of the Twitter data platform team at the time we developed Heron and deprecated Storm. The Mesos component of your retelling is not quite right. Take a look at this comment I wrote around the time we started talking about Heron, addressing the same misconception: https://news.ycombinator.com/item?id=10056479
Comment · 5 years ago
Donated to Apache in 2018. https://blog.twitter.com/engineering/en_us/topics/open-sourc... (and open-sourced in 2016, having started development in 2014).
Comment · 5 years ago
Why is this odd?
If he started two years later and there was not a trace of the Prize work at the company, that would be an indicator that the competition was not important. If he started and could still see knock-on effects from the competition, that's an indicator that it was important.
Plus, he didn't just start at Netflix. He "took over the small team that was working and maintaining the rating prediction algorithm that included the first year Progress Prize solution."
Yeah, that sounds like he has some authority on the matter.
Comment · 6 years ago
> So basically all research projects have to redirect funds to cover part of the "open access fees"
Yes, the cost of publishing results, as well as review, editing, etc, should be incorporated into the research budget (researchers can of course choose other means of sharing their findings than through publication). In an open access model, the publishing costs are offset by money saved in not paying for access to other articles the research project needs. I would hope that opting out means that projects have to pay fees on articles they'd otherwise get for free through this agreement, rather than riding on everyone else doing their bit.
One would imagine the fee schedule is set in the agreement UCB reached with Springer (presumably the agreement will be published at some point...)
Both very technical, both with very little "sell" despite being given by a Camunda co-founder and a Temporal principal eng.
https://www.youtube.com/watch?v=zt9DFMkjkEA "Balancing Choreography and Orchestration" by Bernd Rücker
https://www.youtube.com/watch?v=EaBVzjtSK6A "Building event-driven, reactive applications with Temporal: Workflows vs Sagas" by Dominik Turnow