Not just startups (though the focus might have shifted toward them in recent years, not sure). The Wikipedia article says that Fellows are funded to "drop out of school and pursue other work, which could involve scientific research, creating a startup, or working on a social movement".
https://en.wikipedia.org/wiki/Thiel_Fellowship