> The fundamental problem with general log monitoring is that logs are functionally unstructured.
JSON format logs?
One pattern I’ve used before - in a Java app with an RDBMS, create an error table, and every unexpected Java exception gets logged to that table. Then have a system to match problems in that table to known errors, or flag new ones for analysis. (I just clone the table from the production DB into a non-production DB, then have a bunch of views I wrote to analyse them - the fact it is cloned to the non-production DB means I don’t need release management approval to update the definitions of my views.)
At a certain scale this may lead to problems with performance or data volumes, but generally not if the app is small to medium scale (as in practice many apps are), and especially if exceptions are sufficiently infrequent (one hopes so!)
JSON format logs?
One pattern I’ve used before - in a Java app with an RDBMS, create an error table, and every unexpected Java exception gets logged to that table. Then have a system to match problems in that table to known errors, or flag new ones for analysis. (I just clone the table from the production DB into a non-production DB, then have a bunch of views I wrote to analyse them - the fact it is cloned to the non-production DB means I don’t need release management approval to update the definitions of my views.)
At a certain scale this may lead to problems with performance or data volumes, but generally not if the app is small to medium scale (as in practice many apps are), and especially if exceptions are sufficiently infrequent (one hopes so!)