Ask HN: Criteria for making a good take-home dev quiz?

I have never been a fan of the take home development test when your interviewing for a job. After hiring over the last few years I see the importance now. What are some good problems to include when creating a take home project? I assume something technically related to our product or space, but do you make them start from scratch or give them a broken app and have them fix it?