syncliteio

Few code samples are shown here: https://github.com/syncliteio/SyncLite and https://github.com/syncliteio/SyncLite/tree/main/synclite-co...

A jsp-servlet sample web app is here: https://github.com/syncliteio/synclite-sample-web-app

SyncLite logger transactionally captures the exact sql statements (DDLs and DMls) as executed on the underlying embedded databases, into sqllog files.

The log files for each device (i.e. database) are shipped to a configured staging storage(Local FS/SFTP/S3/MinIO/Kafka etc).

SyncLite consolidator continuously looks for new devices and new logs for each devices.

The received raw sql logs are not directly applicable on final destination as they are logical statements, may involve complex dependencies/joins e.g. insert into select <cols> from t1 JOIN t2 ...

Consolidator applies the incoming sql logs transactionally to a native embedded database (SQLite) and in the process generates record level CDC logs by identifying exact changed records and tables

The CDC logs are record-level INSERT, UPDATE, DELETE statements, DDL statements : CREATE TABLE, ALTER TABLE, DROP TABLE, RENAME TABLE.

The CDC log files as generated are applied to final destination databases.

A summary of this is here : https://www.synclite.io/synclite/sync-ready-apps

Thank you for the feedback and we are improving the documentation.

- SyncLite Logger, a wrapper JDBC driver, supporting popular embedded databases (SQLite, DuckDB, Apache Derby, H2, HyperSQL), creates transactional sql logs for embedded dbs, ships them to configured staging storage ( S3/MinIO/SFTP/Kafka/OneDrive/GoogleDriver etc)

- Several application instances each creating multiple embedded databases/devices are synchronized in real-time on the staging storage.

- SyncLite consolidator, a centralized tool, continuously processes device logs, generates CDC logs, replicates and consolidates staged devices into a wide range of industry leading database, data warehouses and data lakes.

Look forward to feedback and suggestions.