Comment by Maultasche

We discovered that many towns in Iceland have municipal pool/sport complexes, and those were great to visit. They were quite affordable for single visits, the pools were great (with various pool temperatures to choose from), and many of them had extras like pool toys and waterslides, which were a hit with the kids.