jwcrux
Joined 14 years ago
Joined 14 years ago
I’m less familiar with LangSmith, but browsing their site suggests they happen to offer observability into LLM interactions in addition to other parts of the workflow lifecycle. This just seems to handle logging and you have to pass all the data yourself- it’s not instrumenting an LLM client, for example.
Isn’t this blog post effectively “we patched our firewall, things broke, we made a support case, and the vendor investigated and filed a CVE”?
I noticed that you have a page on records that have been set. It looks like Ercot released this data today that might be of interest: https://www.ercot.com/news/release/2023-09-14-ercot-provides...
I’m curious why there appears to be a pretty significant delta between their data and yours.
Also, if you’re open to suggestions, I had trouble finding the pricing for gridstatus. Entirely possible I was missing something obvious, but I wanted to see how much it might cost to get the long term Ercot generation-by-source dataset and couldn’t seem to get a clear answer.
Regardless, building things like this takes a ton of effort, and I appreciate all you’re doing. Keep up the great work!
Edit - I see you mention that Tailscale uses userland WireGuard. Is that the biggest difference between the two? Do you foresee yourselves running into issues by not using the userland implementation?
In addition to this, I’ve also heard anecdotally that many of the unvaccinated (by choice) patients have animosity towards the healthcare workers themselves because the patients see this virus as politicized.
Overall I can imagine that it’s resulted in a more-hostile-than-average working environment which is bound to be stressful.
That's the thing. If we're watching syscalls, we see these checks. These would be things like attempted file-reads. Would they be enough to set off alarms? Maybe, maybe not.
This is generally the cat/mouse game of malware detonation in general. There are attempts to make sandboxes appear realistic, but I'd argue that our use case is even simpler since running commands or making network connections during installation is not a normal thing. It might be benign, but it's abnormal enough to warrant investigation.
There will always be ways to try and get around the system, but I'm pretty firm that this will significantly raise the bar which is a Good Thing.
1) The “observable window” is the entire installation time. If they make installs take forever, that’ll affect everyone which should raise alarms pretty quick.
2) The conditional execution is possible but the installation is done using a vanilla alpine container which will match many legitimate hosts too. And any fingerprinting activities that involve syscalls would be detected in the process.
All this to say, there’s always room to continue raising the bar!
I'm a big believer that functions like this should be centralized under a foundation like that, and have really close connections to package manager maintainers so that we can work together towards solving the problem.
If you're looking for a tl;dr you can find one on Twitter (with pictures!) [0]
This research was a blast to do, and I learned a ton. Happy to answer questions!
Instead, if you build on the standard library, you can compose your application from there- a good muxer, some standard middleware that are generic http.Handler's, a session library, etc.