As a contributor, I think it's interesting when comments focus on the language (Python vs Rust) vs the architecture (local vs remote). Inference is embarrassingly parallelizable, with Python Flask or Postgres replicas. I think the interesting thing is that data retrieval costs tend to dominate other costs, and yet are often ignored.
ML algorithms get a lot focus and hype. Data retrieval, not as much.
ML algorithms get a lot focus and hype. Data retrieval, not as much.