knur
Joined 12 years ago
Joined 12 years ago
The conflict of interest should be disclaimed in the very first sentence of the post.
- if only one process can write to the db, how do you architect your system so that writes are performed by just one of the instances (the writer) - how do you gracefully upgrade your app without any downtime?
Sometimes an unexpected snack makes its way into my belly, leaving behind an annoying dirty plate that needs some cleaning.
Thanks for trying though
Lately I have been migrating all my self-hosted stuff into a raspberry pi (instead of running a public instance in the cloud). It gives me a bit of piece of mind knowing that it adds an extra layer of security (to hit any of my endpoints/apps you would need to infiltrate my VPN). And it will save me a lot of money on hosting.
I don't need to expose my computers publicly or enable upnp or anything. It just works.
The bot names and places the file in the right folder for paperless to ingest.
If you want to build anything mildly interesting, you need to have a solid background on software engineering (building data pipelines in Spark, Flink, etc. goes way beyond knowing SQL), you need to really understand your runtime (e.g. the JVM, and how to tune it when working with massive amounts of data), you need a bit of knowledge about infrastructure, because some of the most specialized and powerful tools do not have yet an established "way of doing things", and the statefulness nature of them make them different from your typical web app deployment.
Maybe if you want to become a data analyst you only need SQL, and I would still doubt it. But data engineering is a bit different.