GPT-4 Turbo is dramatically worse at one task I often try:
Read the following passage from [new ML article]. Identify their assumptions, and tell me which mathematical operations or procedures they use depend upon these assumptions.
GPT-4: Usually correctly identifies the assumptions, and often quotes the correct mathematics in its reply.
GPT-4 Turbo: Sometimes identifies the assumptions, and is guaranteed to stop trying at that point and then give me a Wikipedia-like summary about the assumptions rather than finish the task. Further prompting will not improve its result.
Read the following passage from [new ML article]. Identify their assumptions, and tell me which mathematical operations or procedures they use depend upon these assumptions.
GPT-4: Usually correctly identifies the assumptions, and often quotes the correct mathematics in its reply.
GPT-4 Turbo: Sometimes identifies the assumptions, and is guaranteed to stop trying at that point and then give me a Wikipedia-like summary about the assumptions rather than finish the task. Further prompting will not improve its result.