jurl is a CLI. Tell it what you want from a page and it picks exactly that out: the paragraphs that answer a question, the code, the links worth following, or a photo you describe. On Wikipedia's Cologne page it finds the cathedral among 73 images in 2.3 s; on the Jevons page, the two paragraphs out of ~240 that explain his paradox.
The link above lets you try it in the browser, no key needed. Or in your terminal, with your own TypeSafe key (it asks for it on first run):
brew install rayoplateado/tap/jurl
jurl -q "What is the Jevons paradox?" en.wikipedia.org/wiki/William_Stanley_Jevons
jurl --code github.com/BurntSushi/ripgrep
jurl -f "a cathedral" en.wikipedia.org/wiki/Cologne
jurl -l -n 3 news.ycombinator.com | xargs -n1 jurl -n 1
How it works: jurl splits the page into blocks, links and images, and turns each one into a yes/no question ("does this block answer the question?"). A decision model, Jev from TypeSafe, scores them all in a single request. For images, Clef on Cloudflare Workers AI looks at the pixels. The models return probabilities, never text: jurl picks, it doesn't write. Everything it prints is on the page, so it's safe to pipe into xargs curl.
Most pages take under a second and cost a fraction of a cent, about a thousand pages per dollar, paid straight to the APIs: jurl calls them directly with your own keys (TypeSafe for text, plus Cloudflare only if you want it to look at images).
Written in Rust, MIT or Apache-2.0. Code: https://github.com/rayoplateado/jurl
I'd especially like to hear about pages where it picks the wrong thing.
P.S. It's pronounced /xaɾl/, "harl": the name is a nod to the Spanish comedian Chiquito de la Calzada and his "jarl".
The link above lets you try it in the browser, no key needed. Or in your terminal, with your own TypeSafe key (it asks for it on first run):
How it works: jurl splits the page into blocks, links and images, and turns each one into a yes/no question ("does this block answer the question?"). A decision model, Jev from TypeSafe, scores them all in a single request. For images, Clef on Cloudflare Workers AI looks at the pixels. The models return probabilities, never text: jurl picks, it doesn't write. Everything it prints is on the page, so it's safe to pipe into xargs curl.Most pages take under a second and cost a fraction of a cent, about a thousand pages per dollar, paid straight to the APIs: jurl calls them directly with your own keys (TypeSafe for text, plus Cloudflare only if you want it to look at images).
Written in Rust, MIT or Apache-2.0. Code: https://github.com/rayoplateado/jurl
I'd especially like to hear about pages where it picks the wrong thing.
P.S. It's pronounced /xaɾl/, "harl": the name is a nod to the Spanish comedian Chiquito de la Calzada and his "jarl".