There are cities that are trying actively to become startup hubs to compete with SV, which are currently cheap but probably won't stay so. Kansas City will stay cheap long enough for a startup to take off.
Also, if we're talking about being frugal rather than relying on VC money, why not pick an area with a good university but not so much of a "city scene"/VC-land-ness, and hire college students? For example, Urbana-Champaign, IL. That way you're drawing from a pool where not all the talent has been snapped up yet, and the rent is usually reasonable because students have to live there.
Also, if we're talking about being frugal rather than relying on VC money, why not pick an area with a good university but not so much of a "city scene"/VC-land-ness, and hire college students? For example, Urbana-Champaign, IL. That way you're drawing from a pool where not all the talent has been snapped up yet, and the rent is usually reasonable because students have to live there.