raufakdemir
Joined 2 years ago
Joined 2 years ago
As for intelligently - but probabilistically - determining confidentiality (if I read that correctly), that does sound pretty interesting in scenarios where metadata is just simply insufficient. Also tricky. Sounds like you thought about these problems pretty deeply.
We usually sync per user. That way we make sure that no information leaks to another interface.
We plan to implement unified ACL syncs to dedupe the data or even have 1 sync per org, but that’s mostly a cost optimization; Airweave will just scale horizontally until then.
1) the integrations are done traditionally so with REST/SQL. The MCP/REST search layer rests on the data that gets synced.
2) most providers are painless. Slack doesn’t want major exports in one go but most developers point at a single channel anyway so the rate limit errors don’t bite too much.
3) this is all orchestrated by the platform itself. Incremental syncs will receive the latest “watermark state” and sync from there. Hashes are used to compare data for persist actions (update/insert/keep)