I've been really interested in this architecture since Jay Kreps' blog post on it. One part that I'm less clear on is how this fits in with request-response style communication between, say, a Web browser and a Web server.
In a simple Web-app-writes-to-DB scenario, it's easy to read my writes, but with a async log processing system, how am I supposed to organize my code so I can read my writes and respond with useful information?
Maybe the solution is to eschew request-response entirely and have all requests return 200, then poll or use two-way communication?
Alternatively, I could have my log-appending operation return a value indicating the position in the totally-ordered log, which I could pass to the query interfaces as a way of indicating "don't return until you've processed at least to here." Does anyone do that?
Am I totally off base here? I'd love to hear from anyone who is using these kinds of systems today.
In a simple Web-app-writes-to-DB scenario, it's easy to read my writes, but with a async log processing system, how am I supposed to organize my code so I can read my writes and respond with useful information?
Maybe the solution is to eschew request-response entirely and have all requests return 200, then poll or use two-way communication?
Alternatively, I could have my log-appending operation return a value indicating the position in the totally-ordered log, which I could pass to the query interfaces as a way of indicating "don't return until you've processed at least to here." Does anyone do that?
Am I totally off base here? I'd love to hear from anyone who is using these kinds of systems today.