lithdew
Joined 6 years ago
Karma
65
Submissions
8
Member since
Apr 3
Recent submissions
Comment · 6 years ago
Comment · 6 years ago
Honestly, it was during my deep dive into p2p networking a year back that I started scratching my head and thought "Huh, you could pretty much replace most of the reasons you'd need Kubernetes and gRPC and Consul with this and make coding 10x simpler..."
I only got to finally try it out just two months ago when Mozilla opened up a program for building MVPs that aim to fix problems with the Internet. Big thanks to them for that!
Comment · 6 years ago
All communication is quiescent unless some given node A sends data to node B (apart from TCP keepalive packets of course). For routing messages, it is expected O(log n) much like ordinary Kademlia.
There isn't much resilience carefully integrated into the system for untrusted peers (complete S/Kademlia is in progress though! Static/dynamic crypto puzzles are fully implemented to prevent Sybil attacks on IDs, and the FIND_NODE RPC call is implemented to meet he guarantees documented in S/Kademlia's paper), since this framework is intended to be used in a closed environment for a set of microservices.
The underlying library for networking, Monte, fully supports providing your own net.Conn's and io.Reader/Writers - all you need to do is return your modified net.Conn via implementing the monte.Handshaker interface here: https://github.com/lithdew/monte/blob/master/net.go#L36
Monte will everything else related to bidirectional RPC management. Details of the bare minimum Monte prefixes to your packets is documented on the README for it.
Hope you enjoy using either Monte or Flatend!
Comment · 6 years ago
Contributions would be great! Open up a Github issue or come join us on Discord (link in the readme) and let's talk about it there :).
Comment · 6 years ago
Hello! OP here.
One of the biggest frustrations I've had prying through Darklang [1], Firebase [2], Supabase [3], and a slew of other low-code projects is that although they do help me build backends faster, they come with a big tradeoff of having to deal with either (a) vendor lock-in, (b) tie-in to specific tools/databases/coding patterns, or (c) a large monthly bill.
At the same time, coding without these tools, I would have to deal with either configuration hell [4] with Kubernetes/Istio/Consul or take the time to stitch a bunch of libraries together and spend a lot of time having to deal with devops/networking/backend architecture myself.
So, I built this framework called Flatend over the last two months for Mozilla Spring MVP Labs, which:
1) lets you write functions using any tools/databases/programming languages/frameworks/hosting environments, and
2) turn them into microservices stitched together into a mesh network with service discovery/load balancing/routing using the Kademlia p2p overlay networking protocol popularized in BitTorrent/Ethereum [5].
Taking advantage of p2p networking, Flatend has taken off a lot of the pains and time spent dealing with bidirectional RPC, service discovery, load balancing, scaling, deployment patterns, and end-to-end-encryption, and has led me to spending more of my time writing productive business logic code in NodeJS/Go.
Using Flatend I've now deployed four different sites/apps racking roughly 200k+ visitors on two 5$ DigitalOcean instances, doing red/blue deployments and upgrades to my microservices really easily.
But that being said, Flatend still is very much experimental and is a heavy work in progress.
Nonetheless, I still do hope for Flatend to be immeasurably useful for the people here at HN looking to speed up the backend development of their next product/service.
[1] https://news.ycombinator.com/item?id=20985429
[2] https://news.ycombinator.com/item?id=17661391
[3] https://news.ycombinator.com/item?id=23319901
To clarify, the API gateway I provided is just* another node/peer in the network which advertises a few services that deal with exposing other nodes' services via HTTP/WebSocket/gRPC (rather than an ordinary API gateway server).
This means that you can run several API gateway nodes on different machines, and kill several of them, and they would still discover and process one another's request/data without any extra work, as they are independent nodes that are not directly tied by their addresses to other nodes/microservices.
So, I guess the easiest way to point out the p2p aspect here is on how Flatend deals with the separation of concerns across "services"/nodes.