roshanj
Joined 12 years ago
Karma
204
Submissions
34
Member since
Sep 5
Recent submissions
Comment · 2 years ago
Comment · 2 years ago
You already mention Materialize -- have you looked at the SUBSCRIBE feature? https://materialize.com/docs/sql/subscribe/
It's fairly straightforward to write a websocket endpoint in whatever backend you have that issues a SUBSCRIBE query on connect and then continuously forwards the updates to the client. The complicated parts you'd need to build are on the frontend -- handling updates and merging them into whatever state you render from.
Disclaimer -- I work at Materialize -- but it does seem like the perfect fit for what you need (efficiently updated real-time materialized views).
Comment · 2 years ago
This sounds very similar to Ovio, which matches open-source projects with contributors that have relevant skills. There isn't a rating system but there are tools to browse and find good projects & issues to contribute to for aspiring OSS devs
Comment · 2 years ago
This is great! I've had this exact idea for a specific robotics use-case but never got around to implementing it: a fleet of robots that each expose an HTTP service for debugging purposes. These robots connect to the internet through cellular or hop around among a set of wifi access points, such that long-lived connections are often interrupted and each robot IP address intermittently changes.
Many other reverse proxy / tunneling solutions use TCP-based protocols or require the target hosts to be accessible by the proxy server, but in this case QUIC connection migration avoids the reconnection handshakes needed for dropped TCP connections, and your client->server model allows the robots to register themselves from restrictive networks.
The only missing feature would be to allow some sort of auth plugin - perhaps as a sub-request made to an external auth service that contains the identifier of the client the request will be routed to, similar to nginx's auth_request (https://nginx.org/en/docs/http/ngx_http_auth_request_module....)
Comment · 2 years ago
> Of course it's possible to solve. But I think the solutions (and the cons of those solutions) are often pretty arduous or unreasonable.
> worst of all, you don't have any transactional consistency across those joins, it's possible to join when you've only consumed one half of the streaming data (e.g. one of two topics).
This is exactly right! Most streaming solutions out there overly simplify real use-cases where you have multiple upstream tables and need strongly consistent joins in the streamed data, such that transactional guarantees are propagated downstream. It's very hard to achieve this with classic CDC + Kafka style systems.
We provide these guarantees in the product I work on and one of our co-founders talks a bit about the benefits here: https://materialize.com/blog/operational-consistency/ .
It's often something that folks overlook when choosing a streaming system and then get bitten when they realize they can't easily join across the tables ingested from their upstream db and get correct results.
Comment · 2 years ago
Another open-source SFU I've had great experience with is Livekit[0]. Great docs, modern, easy to deploy (it's a golang binary), and supports a number of egress options too if you want to record the media during a stream to an external system. With their cloud product they've also built a really cool 'mesh-based' SFU-CDN that allows peers to connect to an SFU closest to them and have their media broadcast to other SFUs internal to their network, which allows for easy scaling during broadcast-style use-cases.
Comment · 4 years ago
Thank you for writing sish! We’ve been using sish as a self-hosted alternative to ngrok because of how easy and flexible it is to configure and because it allows us to tunnel to devices without installing any extra device-side software (since it uses ssh).
I highly recommend it among all the other great projects on the awesome-tunneling list
Comment · 4 years ago
We've been building a live-streaming & remote operation system for drones using LiveKit. I've been extremely impressed at the speed they've delivered improvements and grateful of how supportive DZ & the LiveKit team have been of us. It's definitely the right choice if you're looking for a modern SFU system to run. Congrats on reaching 1.0!
Comment · 5 years ago
Here's a non-paywalled article: https://techcrunch.com/2021/02/28/autonomous-drone-maker-sky...
Comment · 6 years ago
Check out https://ovio.org/ -- they target matching developers with Open Source projects, and many of them are for non-profit causes (several COVID-19 related projects right now: https://explore.ovio.org/covid-19)
I'm a former Skydio[1] employee and our motion-planning & state-estimation teams implemented this on all our drones years ago. It wasn't completely fool-proof, but in many non-high-speed scenarios a loss of a motor was not catastrophic and the drone could 'emergency land' within a few meters of where it was flying. I wish I had proof for you but unfortunately no longer have access to any internal videos :)
[1] https://skydio.com