Comment

Comment by antirez

Because of masked attention in LLMs, if you put the options before the body (the email to analyze), the transformer already knows what it needs to look for, and can use more tokens to create state to address that specific task (BERT has no mask in the attention, so tokens attend also to next tokens). You could also do a few examples in the system prompt to improve calibration.

Another trick that works is to repeat the question two times: "I'm repeating the task and labels for clarity: ..."

parent antirez 1 month ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton