skorniienko

Hi, one fellow HN commenter suggested to use a credible list of sources in my AI fact-checker skills. I promised to test and publish the results.

Test failed, and what I've found is interesting - as of 47 tested news outlet sources - 16 blocked AI requests with 400 error. (Reuters, WSJ, The New York Times, The Guardian, etc)

Blocked content still arriving from news aggregators, but model always prefer the source over any citation. Sometimes, data from aggregators were even different to original post.

If you have experienced this and found a solution - I'd be thrilled to hear about it. As for now - it's a gap I acknowledge and cannot overcome.

exactly, that is one of the use cases I see for these skills

What if you use OpenAI as a judge and check both responses? Not defending Gemini, just curious if it was that wrong, or Fable - that aggressive.

Not sure that I get your idea about short-form content. It already works with tiktok or yt shorts, especially if it has generated transcription/captions. Otherwise, skill will download video locally and ask your permission to run local models like whisper to transcribe video.

Checking before you publish - that's interesting usecase I haven't thought about. Nothing stops you to fact-check your own content and decide what to do with it. Point is a report itself not more video content from it

can't, at least with this design. Report itself takes minutes to proof-check. Realistic version is some kind of extension/plugin idea from earlier in a thread, but I'm not there yet

True, and it doesn't do logic. It decomposes content into claims and search for every claim citations. It catches a false premise, not a bad inference. Someone with valid reasoning from true facts to a wrong conclusion goes straight through.

Empirical sources - would be a first candidate to be implemented in a skill. And you are right about 10 credible sources are actually 2. It will look like 10 independent sources to it, real gap. Skill does prefer primary source over secondary. For example it'll prefer NASA article over blog post who cited NASA article.

I'll work on empirical sources improvement in a next releases.