kawin

This is great advice!

I'd like to add that if you don't have pairwise preference data (A > B) but do have binary data (A is good for x_1, B is good for x_2, etc.), then Kahneman-Tversky Optimization (KTO) might be a better fit. Despite learning with a weaker signal, it works as well or better than dpo in practice.

That's a fair point. I was initially thinking of targeting businesses rather than individual consumers, so I thought a flexible pricing plan would be helpful (given variations in headcount, sector, etc.). It looks like individual consumers are interested in it as well, so I'll add a more transparent pricing tier for them. Thanks for the feedback!

This is a fascinating idea - it's definitely possible, though easier for some groups of people than others (I don't think I'll be able to hire doctors for $25/article). I think the human and AI analyses would contrast each other nicely -- the model would estimate how well people will react on average and the human analysis, although idiosyncratic, will pick up on more abstract qualities of writing, like the flow. I'll put this down as a feature to implement in the long-term -- thanks for the idea!

Thanks for the feedback! User privacy is definitely important to me, especially since users might be entering recruiting / product-related data into the app. The text that you analyze is not stored on any Toasted database (the most recent text you analyze is stored as a cookie so that you don't lose progress if you accidentally close the tab). Moving forward, I'll work on options to allow for anonymous payment.

I definitely wasn't planning to base pricing on what users were submitting or how frequently they were using the app. I was initially thinking of selling to businesses rather than individual consumers, and enterprise pricing is pretty variable -- based on factors like head count, which categories (e.g., 'accountants', 'doctors') would be useful, etc. Hence the flexible pricing model. If there's enough interest among individual users though -- which there seems to be -- I'd be happy to offer a basic tier that's transparent.

Thanks for taking the time to write this out! I agree on the UX suggestions. The suggestions for each word are a list of terms that could work, with the score being conditional on the cohort selected. The darker the alternative, the worse it is -- the pinker, the better.

The reason why technical articles -- or in your case, very domain-specific ones -- get average scores is that the model was intended to be used for recruiting and ad-copy. So at the moment, it works on a common vocabulary for all groups, whereas one reason your article might be intriguing to programmers is because it discusses a lot of domain-specific terms (e.g., discussing 'exploits' and 'network security'). If I modified the model to consider more domain-specific terms, I think the article you provided would rank very highly. There are other reasons as well, like the pithiness and clarity of it, but I do think the vocabulary is an important part of it.

From the little beta-testing I've done, the people who find it most useful are -- for example -- people without a CS/medical/accounting background who have to interact with, recruit, and sell to CS/medical/accounting people. For those folks, I would say there's some value, even at the vocabulary-level, though I agree that there's much more to writing style than just vocabulary.

Got it. The system does use a state-of-the-art POS tagger, but I think 'key' was mistakenly tagged as an adjective instead of as a noun for your sentence. I'll try to fix that -- thank for pointing it out!

Textio looks like a a great product, but IIUC its use case is very specific -- how can you make your job description more enticing and gender-neutral?

Toasted is meant to be more of a big tent product: you're not just working with men/women, but very specific groups of people (e.g., retirees and accountants) and you want to use language in the way that they're using it, which is helpful for things like writing ad copy as well.

Glad you like the idea! At the moment, the model requires a fairly large amount of data, which is why the app only offers large well-defined cohorts to select from (e.g., accountants, retirees). I like your idea though, and it may be possible to use some of transfer learning to work on smaller groups of people or even individual customers. I'll try to implement that at some point in the future.

Thanks for trying it out! The possible changes you can make stays the same across different cohorts of people, but the score assigned to the word changes (e.g., 'security' has a higher score for retirees than college students). The pinker the word, the higher the score.

Would it be better if only the top 3-4 options were given for each word?