lbradstreet

It's not that I wouldn't recommend it, I would just think twice recommending it for very small ops teams for certain use cases. It's not so hard to manage, but having it be managed by Confluent is a great option.

I love Kafka and the log orientated streaming model, but I often have to think twice before recommending it to clients who would have to manage the ops themselves. Having a managed service on GCP, and Confluent's existing cloud offering on AWS really brings down the barrier to entry. There aren't really any AWS/GCP serverless equivalents (Kinesis has 7 day retention maximum, no key compaction, and less surrounding tooling such as KStreams/KSQL).

PubSub is fundamentally a different abstraction than Kafka which is log orientated. With Kafka you can maintain the ability to replay your log/stream, in order, at any time.

This enables different kinds of use cases, and can be easier to reason about.

Our approach archives topics to cheap, highly durable and available object stores, while keeping the data available for blending between warehoused and live data sets.

This reduces operational complexity significantly vs scaling nodes up, dealing with rebalancing, under replicated partitions, etc.

We will be launching support for native excision and data anonymization soon, as these are extremely important to storing streaming data for the long term.

Workarounds for excision in Kafka, such as key compaction, are often not possible to use as they depend on the key scheme used.

As a side note, we provide tooling to maintain Amazon Athena/Hive,Presto support over your Pyrostore archived data in S3, while maintaining its ability to be streamed in-order.

Pyrostore co-founder here. When practitioners archive their data from Kafka to other storage products (S3, SQL database, etc) today, they are giving up on the log ordered structure of the data their ability to consume their data in its original ordering, with its original offsets and timestamps. Pyrostore structures and indexes your data in S3 in order to provide a consumer that implements the Kafka consumer interfaces, ensuring you are always able to stream from hot and cold storage alike.

On the face of it, the lender has no more incentive, but having an unforgeable trail of ownership and payment may improve the value of the loan as it passes from hand to hand.

That all said, a blockchain is not really required. Better standards and a trusted third party would do fine.

You still need to be able to transfer the loan from party to party, and allow parties to prove ownership of the loan. That's not a complex contract obviously, but it's important.