sea6ear

I guess I am reading Elric of Melnibone - although I can't say I'm really enjoying it, and looking forward to getting back to something like Earthsea that is more hopeful.

For programming I am reading HTML5 Games: Novice to Ninja, which I quite like so far. And JavaScript for Kids - both with the intent to entice my kids into game programming.

For work I am reading Learn dbatools in a Month of Lunches and outside of lingering questions about whether anything command line oriented matters in the age of AI, it has been really helpful in encapsulating solutions to things that are distinct problems in administering SQL Server.

I loved your 'expand' utility - and wrote a truly heinous Tcl to PHP framework/translation layer using it, because I preferred Tcl to PHP at the time - still do, but the early hatred for PHP has muted - that powered an application at the college I attended for an embarrassingly long time.

I think this was created by kazinator who has posed a fair amount about lisp here over the years.

I've been playing around with the Godot game engine as a sort of modern successor to Delphi / Lazarus. I'm currently messing around with trying to create some database server management software using it.

GDScript is pretty similar in feel to Python, and you can also use C# if you want to. It has some level of GUI controls in the framework (not sure how many yet, but all of the GUI controls used to build the editor are available for use).

I want to believe the 3d capabilities might be useful for some kind of UI stuff, but I don't really have a real idea how to make that work - just a "wouldn't it be neat if..." question about it right now.

I love the framing in this video in terms of the decisions I need to make when I'm doing personal projects.

With the caveat that you probably shouldn't listen to me (or anyone else on here) since you are the only one who knows how much pain each choice will be ...

I think that given that you are not really dealing with structured data - you've said that different sites have different structures, and I assume even with processing, you may not be able to generate identical metadata structures from each entry.

I think I would go for one column of XML, plus maybe another column that holds a parsed data structure that represents the result of your processing (basically a cache holding the post-processed version of each site). Hopefully that could be re-evaluated by whatever language (Python?) you are using for your application. That way you don't have to do the full parsing each time you want to examine the entry, but you have access to something that can quickly give you whatever metadata is associated with it, but which doesn't tie you to the rigid structure of a table based database.

Once you know what you are really doing with the data, then you could add additional metadata columns that are more rigid, and which can be queried directly in SQL as you identify patterns that are useful for performance.