I have a similar desire, I'm considering trying out AWS' RDS Aurora Serverless v2. (ugh, that's a mouthful) Full Postgres/MySQL but extremely quick scale up and down so as long as you have small use, it shouldn't cost a ton.
I'm also super interested in Cloudflare D1, looking to get my hands on it and try it out.
I'm also super interested in Cloudflare D1, looking to get my hands on it and try it out.