You need to know how much LLM output you need to get your product working, before you even know what you're hoping for regarding a target cost per million tokens. When you do get PMF, can some of the work be offloaded to a smaller and cheaper model? Can you determine this division of labour yet?
Consider also that "computer" used to be a job title, that since then the cost of doing computations has reduced by a factor of at least 1e14, and yet that you're only asking this question at all because you're still compute limited.
You need to know how much LLM output you need to get your product working, before you even know what you're hoping for regarding a target cost per million tokens. When you do get PMF, can some of the work be offloaded to a smaller and cheaper model? Can you determine this division of labour yet?
Consider also that "computer" used to be a job title, that since then the cost of doing computations has reduced by a factor of at least 1e14, and yet that you're only asking this question at all because you're still compute limited.