This is correct. There's been a lot of interest in e-values and non-parametric confidence sequences in recent literature. It's usually refered to as anytime-valid inference [1]. Evan Miller explored a similar idea in [2]. For some practical examples, see my Python library [3] implementing multinomial and time inhomogeneous Bernoulli / Poisson process tests based in [4]. See [5] for linear models / t-tests.
[1] https://arxiv.org/abs/2210.0194
[2] https://www.evanmiller.org/sequential-ab-testing.html
[3] https://github.com/assuncaolfi/savvi/
[4] https://openreview.net/forum?id=a4zg0jiuVi
[5] https://arxiv.org/abs/2210.08589