urbit

Unfortunately I wouldn't expect a lot of substantive technical discussion for a post like this on HN -- this post will expire before anyone has time to read it properly and produce thoughtful responses.

(Which is broadly the problem with posting enormous applied CS-research braindumps to HN. Don't know a better place to do it, though. And I see people are upvoting you. Also, the absence of thoughtless responses is itself a great response.)

The key point here, as far as I can tell, is really building a CRDT system as a true platform layer, rather than as a one-off solution for a special-purpose app. I think it's fairly clear that generic sync is a pretty essential part of a modern decentralized computing environment, and I don't think it's clear at all what the best way to solve it is.

But... it seems to me that you've solved a large piece of the problem but not the whole thing. Because the most important document sync and collaboration platform is, of course, source code control. If you have a document sync and collaboration model that doesn't at least generalize to classic revision control, why not?

Now, there's a sensible reason to separate these problems -- CRDT and OT solutions tend to specialize in the zero-maintenance case where a user-resolved merge is just impossible. Whereas if it was possible to build a zero-maintenance revision-control system, which automatically resolved all merges and conflicts, someone would have done so already.

This certainly suggests that the two are different problems. But generalizing across slight differences is what system software does. Maybe one is a special case of the other?

A generalized layer for lightweight collaboration is pretty powerful. But it certainly seems like the case that if you could generalize across lightweight collaboration and heavyweight revision control, you would have something that would be incredibly powerful.

Or is this too ambitious? I don't know so I'm asking you.

Definitely not dead -- it got stalled for a while because Urbit tends to use deep stacks and both Graal and the JVM seem to favor shallow ones. The author (not me) came up with a "meta-stack" design that may mitigate this issue and will go back and try it soon.

But the performance seen on a trivial Ackerman function was very good -- Nock via Graal got close to C performance. That's impressive for what's basically a Lisp.

Here's a more optimistic take which is unavoidably a plug.

Urbit is maybe a sort of "extreme langsec." "Immune" is a dangerous word. Urbit is "premitigated" against Spectre for the following reasons, some cool and others stupid:

(1) The Nock VM has no O(n) data structures, ie, arrays. This was not anything to brag about till now.

(2) Urbit events are interpreted in their own process, which is one trust domain (one process, one "personal server").

(3) A personal server isn't a browser and doesn't habitually run code its owner doesn't trust.

(4) Urbit is purely functional, so you only know the time if someone passes you the time.

(5) Top-level event handlers are passed the time of the event. This is easy to fuzz.

(6) If you are running untrusted foreign code, it is probably just a function. Giving this function a measured time interval from top-level event times would be strange. And it would need another channel to exfiltrate the stolen data.

Never say never, but this combination of premitigations makes me worry about Spectre very little. Although, when we do have a less-trusted tier of applications (walk before run), ain't nobody is going to be telling them the time.

Besides the plug, the lesson there is that one of the reasons Spectre is a problem is that langsec is an imperfect bandaid.

Urbit is premitigated because it was designed as a functional system from the ground up. The JS environment is functional lite -- it allows typeless functional programming, but it also retains many mutable/imperative systems design tropes.

It's these tropes that have become clever attack vectors. If you don't want to get Spectred, build systems that are formal and functional all the way down.

Um, I think you just did. :-)

Of course, like every abstraction, this abstraction breaks down as you get into the details...

If I can distill my objection to the pure capability model: there are two different statements of the capability manifesto, one strong and one weak.

The weak capability position is that it should be possible to construct an opaque combination of designator and authority, ie, a capability. The strong capability position is that designator and authority must always follow the same path.

So in the "pure capability model," you are actually subtracting a feature from the system. Now, of course, it's often the case that subtracting a feature makes a system better. But one needs to be really clear about confusing this with the positive virtues of just having capabilities.

Also, while I agree that capabilities are really great (Arvo has a somewhat capability-like internal communication model), it's important to point out that (a) capability vs. identity is a duality; and (b) it's cheating to compare a good capability system to a bad identity system.

For (a), note that identity systems can implement capability features like delegation and revocation; they just have to do it actively rather than passively. If I am willing to act as your deputy, I can delegate my power to act to you. Or some restricted subset thereof. Tell me what to do and I'll do it.

For (b), if you put a decent capability system next to the Internet, a network whose identity model is a burning-tire dumpster fire. If your identity system actually works and makes sense, competing is harder for the capability design.

I do think Urbit should have some kind of capability model. However, this has the feel of a 2.0 or even 3.0 feature. I hate to solve a problem until it is actually in my way. I feel we have come nowhere near the limits of the common case. A simple map from identity to privilege is certainly that.

And while capability UIs may be doable, identity UIs are trivial. This would not be the first case of humans being funky, hence screwing up our elegant architectural logic.

If you have a system software project and it can advance without being lashed to a company, definitely do it that way! It's always a poor fit -- they have to succeed on different timelines.

One thing I know: once we all have personal servers, today's cloud will look pretty lame in retrospect...

Protip: if no one's ahead of you, you're leading :-)

This is a great answer. I think the impedance-mismatch question is a big part of the problem. Personally I prefer to boil the ocean rather than drowning in the impedance-mismatch tarpit. Both of these strategies are, of course, profoundly insane.

It's true that a file-open dialog box easily becomes a powerbox. But of course, you are using the ambient authority of the user's login session to ascribe authority to the path that gets typed. So in a way it's the exception that proves the rule. And does a URL bar easily become a powerbox? What about a command-line shell?

Perhaps we're both just agreeing that there is always trivial ambient authority, and the right way to scale up to nontrivial cases is a capability model.

I just worry that when too many people discover the beautiful screwdriver that is capabilities, they decide that everything should be a screw. But the world has nails. It may even be true that the world should have nails. And then you find yourself pounding in nails with the butt of your screwdriver.

Have you considered an ICO? Asking for a friend :-)

This is a great question.

One possible answer: maybe the right system would use "ambient authority" (ACLs, etc) for the common case, but scale up to true capabilities for interesting corner cases.

Creating capability UIs that compete, in the common case, with the normal ambient-authority design, is ferociously hard. This seems to be where most capability systems falter. The hardest UI problem in the leading modern capability OS, Sandstorm, is certainly the capability "powerbox."

Think about all the places you can type a filename or a URL. If the designator and the authorization have to travel together, and the pair is an unforgeable capability, then user-entered designators (like paths or URLs) are not practical. Designators can't travel through unsecured paths, through the human mind, etc. This is a painful limitation.

That said, the OP is the best explanation of capabilities I've ever seen. Everyone really should read it.

Wadler's criticisms of jets are pretty common and not super hard to respond to.

Nothing at all stops any jet-based interpreter from using normal standards practices to specify a standard jet profile. We can even expect aggressive vendors to "embrace and extend" that profile.

But, in practice as well as theory, this isn't going to cause incompatibilities, monopolies, etc. Your semantic compatibility is perfect.

In fact, jet competition should have excellent effects on performance across a healthy market. If Google hosts a jet-propelled interpreter, it's natural for it to both offer a tensor processing library, and hook that library up to a jet that talks to a hardware TPU. Your code will work the same if you move off Google -- you're not physically locked in.

What we just got rid of: the bureaucratic overhead of standardizing a TPU opcode. Look at all the crypto algorithms you have to run in JS on the browser, when half the time there's a hardware instruction on the chip. It's technically trivial to connect these two things -- but not politically. Politically, you need a committee to standardize the builtin.

It's true that performance disparities are absurd if you don't jet-propel basic arithmetic. But no one would do that. If a system works both in theory and in practice, it works. You can't really say that in theory, it doesn't work in practice.

(Disclaimer: jet inventor here. But Simplicity's jets are much more badass because of the Coq equivalency proofs.)

On first glance, Cell feels like a very serious effort and I wouldn’t be surprised if it has an impact.

The embedding model is a nice and thoughtful touch. I think a lot of imperative systems could benefit by converting their semantic cores into code in an engine that’s reactive, structurally typed, transactional, and functional-lite (Cell appears to not even have generalized higher order programming, a bold but defensible choice). The relational automata (actors?) are also quite interesting.

Urbit | San Francisco, CA | FULL TIME | ONSITE

Urbit is hiring:

Urbit is a clean-slate system software stack defined as a single frozen function. We're looking for:

- Systems generalists who are comfortable at every layer of the stack and want to build beautiful code. Experience in functional programming, compilers, kernels, etc, is nice.

- Designers who can code, or frontend devs with an attention to detail.

Email: [email protected]

The first question (economics) is one we've thought a bit about. Not affiliated with Filecoin/IPFS/Protocol Labs, so these answers may be wrong:

Filecoin creates a market for storage. The price of B2 is an important variable in this market, because it creates an arbitrage opportunity where you can use B2 to mine Filecoin.

But why not just use B2 directly? Three reasons.

One, B2's replication isn't perfect (at least, not as perfect as S3), and both B2 and S3 have nonzero political risk (a sovereign could order them to destroy your data). B2 plus a layer of blockchainy replication could be quite attractive to some low-price-sensitivity customers.

Two, to work with B2, you need a business relationship with B2. This is obviously no big deal in the normal world of multiuser, single-server apps that we live in today.

However, if we can somehow migrate to a world in which users have their own servers, it's much easier for those personal servers to have simple, maintenance-free automated relationships with decentralized token-based services. Think about how many centralized service providers a typical startup has relationships with (and how much they cost). This design can't really be scaled down from industrial computing to personal computing.

Three, Filecoin has an interesting consensus design in which the word "miner" is used in two senses: "miners" participate in storage transactions, and also receive block rewards based on proof of stake, where stake is defined as the storage of the whole system. The effect of this on the economics of Filecoin is extremely hard to predict, but it can only be positive.

The one point that seems unresolved about the economics of Filecoin is this interesting interaction between FC as a cryptocurrency and FC as a payment mechanism in a storage market. It does seem that you could take FC's storage market and port it to an existing blockchain. But there may be more subtle technical requirements from a blockchain -- a storage market is certainly a demanding application.

I agree - I see this Benjamin Franklin ("if we don't all stand together, we shall all hang separately") attitude everywhere.

But I don't think this new attitude is an accident. It's not just because we're good people, or even good engineers. It's an inevitable consequence of what we're trying to do.

Fundamentally, all decentralized systems are one big federation. If two decentralized systems can't talk to each other, that's because no one needs to connect them yet. There is never a question of whether the connection can be made, as with centralized systems.

In short: for decentralized systems, the n in Metcalfe's law ("the value of a network is proportional to n^2, where n is its user count" - I think it's more like n log n, but whatever) is the number of users not in any decentralized network, but in all decentralized networks. (It also should add any centralized networks that you can gateway to.)

What this means in practice for Urbit and Matrix: if both Urbit and Matrix succeed, a gateway is inevitable. It's inevitable because (a) it will be demanded and (b) no one can prevent it. Then, Urbit will benefit from Matrix's network effect and vice versa.

So economically, we're not competitors. We're colleagues. So building a positive relationship is a professional no-brainer. In the new decentralized world, the only reason to be a jerk to someone else is that you're actually a jerk. You'll see this, but it isn't common -- at all.

(However, our domain is googleable and yours isn't. Owned! We'll just have to link to you :-)

As a competitor (or "competitor," I don't think decentralized services actually compete with each other in the traditional sense), I just want to say:

Matrix is an amazing project with an incredibly hard-working team. If you can afford to, please stop what you're doing right now and give them money. Or code. Or both. We can win this thing!

-- Urbit