Comment Comment by ted537 I don't think it would be too hard to scrape useful data out of my LLM convos.If human response is "That's BS", "fuck off", or something similar, mark as bad assistant message.If human response is "huh" or "cool", mark as good assistant message.If on ChatGPT, watch how much scrolling user does. If there's a lot, its somewhat likely that the LLM outputted something useful.That strategy would have holes of course but as long as its better than guessing something like that would be a useful heuristic. parent ted537 1 year ago
If human response is "That's BS", "fuck off", or something similar, mark as bad assistant message.
If human response is "huh" or "cool", mark as good assistant message.
If on ChatGPT, watch how much scrolling user does. If there's a lot, its somewhat likely that the LLM outputted something useful.
That strategy would have holes of course but as long as its better than guessing something like that would be a useful heuristic.