My website already includes AI-related files such as llms.txt.
I'm considering creating a separate JSON file for every page and article so AI systems can understand the content more easily and accurately. I would reference this JSON file from the page's <head> using a <link> tag.
The JSON file could contain information such as:
Page URL
Canonical URL
Title
Summary / Description
Main Content (clean article content)
Author
Published Date
Last Updated
Entities (people, companies, places, products, etc.)
Keywords / Topics
FAQ
...
My idea is that AI crawlers could read this structured JSON instead of having to extract the main content from noisy HTML that contains navigation menus, sidebars, ads, comments, JavaScript, tables, and other non-essential elements.
I have two questions:
Could this approach reduce the chances of AI crawlers misunderstanding a page or extracting incorrect information from HTML, advertisements, tables, comments, or other noisy content?
Do you think a page-level JSON file like this could help AI systems better understand a page and potentially improve AI recommendations, citations, or other AI-generated responses in the future? Why or why not?
I'm considering creating a separate JSON file for every page and article so AI systems can understand the content more easily and accurately. I would reference this JSON file from the page's <head> using a <link> tag.
The JSON file could contain information such as:
Page URL
Canonical URL
Title
Summary / Description
Main Content (clean article content)
Author
Published Date
Last Updated
Entities (people, companies, places, products, etc.)
Keywords / Topics
FAQ
...
My idea is that AI crawlers could read this structured JSON instead of having to extract the main content from noisy HTML that contains navigation menus, sidebars, ads, comments, JavaScript, tables, and other non-essential elements.
I have two questions:
Could this approach reduce the chances of AI crawlers misunderstanding a page or extracting incorrect information from HTML, advertisements, tables, comments, or other noisy content?
Do you think a page-level JSON file like this could help AI systems better understand a page and potentially improve AI recommendations, citations, or other AI-generated responses in the future? Why or why not?