> can they really look at a DOM tree and tell what it is/does
Yes, if you encode the DOM as a list of options for ChatGPT to choose from. In fact I developed a proof of concept of this for a client. https://jarvys.ai/ although they seem to have pivoted from automating just the browser to automating all software.
Yes, if you encode the DOM as a list of options for ChatGPT to choose from. In fact I developed a proof of concept of this for a client. https://jarvys.ai/ although they seem to have pivoted from automating just the browser to automating all software.