bladecatcher
Joined 17 years ago
Karma
816
Submissions
128
Member since
Dec 3
Recent submissions
Comment · 6 years ago
Comment · 7 years ago
> Now if it were possible for it to accept an event from Kafka but not guarantee that the event will eventually make it into the materialized view but may be lost, that'd be a problem.
This is usually not a problem these days as it’s possible to guarantee exactly once ingestion using Kafka offsets
Comment · 7 years ago
You’re right. You don’t get read-your-own-writes guarantees in a typical Kdb tickerplant architecture, as the event streams are propagated to the real-time nodes asynchronously.
Comment · 7 years ago
I think the usecase for this is streaming queries and aggregations where the query output is materialised asynchronously
Comment · 7 years ago
In Indian cities like Mumbai, Bangalore, one can easily spend 3 to 4 hours a day commuting
Comment · 7 years ago
I suppose one can measure the risk of losing messages - I’m not suggesting that it’s always possible, but calling it an unmeasurable risk is probably not true
I’ve worked on fairly complex systems using kdb/q that are a joy to debug because the codebase is a handful of files. q is admittedly a more simplified and “mainstream” cousin of J, but allows a fair bit of expressiveness. The powerful data store (kdb) and q-sql that comes along with it makes it one of the most concise data platforms ever. Too bad it’s not free for commercial use!
[0] https://www.jsoftware.com/help/jforc/tacit_programs.htm
[1] https://www.jsoftware.com/help/learning/09.htm