alexblackwell_

Feel free to try it out and let us know if you have any issues. I was personally skeptical, but it now does about 70% of my "reverse engineering" work. Sometimes needs a bit of guidance on really complex fields, but I think the tool calls we have given it allow it to be pretty smart. Just my biased $0.02 though

Web apps like this are few and far between, and piggy-backing on the session/anti-bot tokens from your browser usually works. In really complex cases we do help companies on a white glove case-by-case basis.

Interesting. We essentially do the same thing, but with MITM. We have a chrome extension internally, but have found it's a bit of a clunky interface. Might be releasing one soon. The approach with executing script in webpage is interesting. Best of luck!

You can prompt the mcp to do this. Honestly considering adding this as a skill in the agent chat. Internally we do this all of the time for our white glove integrations.

The requests still route through your servers/the data still lives with you. Kampala is a powerful tool but I don't see people replacing the actual apps with it. Most of our customers use it for automating repetitive actions in legacy dashboards.

I wouldn't consider what we do evasion really. We are using real tokens that you have received from your browser as a result of browsing the web. Any good anti-bot will have enforcement for abuses of that token.

My broader point is that these ToS clauses are often so broad and vague that they're essentially unenforceable and not meaningful in practice. For example, "Do not use bots" covers a pretty substantial amount of ground, and intention isn't exactly something you can screen for. Is an autofill chrome extension a bot? If so what separates that autofill from accessibility extensions? Is someone using Whispr flow to fill forms considered a bot? AirBNB doesn't block Google's crawler. Why? A company can enforce its TOS as it wishes. My general point is that the waters are murky, and that automation is a sort of sliding scale.

sorry a bit confused on your question here. If you're asking about JSON RPC we handle this via parsing. The AI can then handle deducing structure most of the time given enough context

gRPC obscures the keys not the values. Enums and signed ints are sort of tricky, but the latter is just a mapping problem and the former can be figured out through some logical deduction. gRPC isn't designed to obscure request content, but for over the wire efficiency.

The goal is not to scrape sites en-masse, but to allow people to automate their existing workflows and actions that they perform already via a browser. I understand the concerns around this being unethical, and it's something I spent a lot of time thinking about when I worked on automations previously. I've written a decent amount about how I don't think that sneaker bots or ticket bots are ethical. I don't support mass scraping websites/making the web more inaccessible for others.

I do have to push back on the ToS comments though. Automation is used daily by nearly all companies. RPA is a billion dollar industry. Browserbase raised at 300M valuation. Is using puppeteer to automate a form submission a violation of ToS? If so then why is using a screen reader not? Is it the intention? Why is hitting network requests directly different? I personally don't think that automation is unethical (as long as it is not affecting server capacity). I don't think the answer to the ethical problems in scraping is just not to automate at all. Open to disagreement here though.

Yep essentially. I would argue that we're probably closer to a MITM proxy like Proxyman than Wireshark. We don't do general packet sniffing (yet), although internally we use our own packet sniffing tools for reverse engineering on-prem installations.

Definitely get that. Being hammered by scrapers is a massive PITA (especially with latest aggressive AI crawlers). We focus primarily on allowing people to automate their existing workflows. For all hosted workflows we have rate limits to prevent mass scraping/affecting server workload in any real capacity. In fact, because we don't load js/html and hit endpoints directly I would guess that we consume less server resources in the end.

Yep we handle gRPC and websocket. gRPC is a bit sketch/hard to do because of the way the protocol is designed. FWIW not many sites implement gRPC (some google sites and spotify being the only two I can think of), and if they do they usually have decent APIs. Feel free to try and lmk if you have any issues!

Yeah agreed this messaging is a bit confusing. Our focus is on helping people build automations, not do any mass-scale scraping.