zazerbayev
Joined 6 years ago
Karma
14
Submissions
3
Member since
Jul 20
Joined 6 years ago
You must ask yourself why the government is able to retain word-class managers and jurists despite stiff competition from the private sector. James Comey was an executive at the world's largest hedge fund, why did he quit for a government job where he was making 100x less? Any federal judge could've made a fortune in corporate law, why didn't they choose to? Because there is a sense of prestige and national duty about their posts. Creating a similar status for technologists in government would go a long way towards recruiting and retaining top talent.
If we want to convince people that being a government engineer is worthwhile and respected, starting with students seems reasonable.
You can check out our evaluation library (https://github.com/wellecks/lm-evaluation-harness) for the exact benchmark implementations we used, including prompting.
In particular, the prompt that starts at line 27 in this file (https://github.com/wellecks/lm-evaluation-harness/blob/maste...) is quite good for high school/olympiad problems. We took this prompt from Google's Minerva paper.