I needed live listing prices from a property portal for a personal buy-decision tool. The portal does everything a site does to stop a scraper: it disallows the paths in robots.txt, it fingerprints headless clients, it hides behind a login, and the prices don't even exist in the fetched HTML — they're painted in by JavaScript after the page loads. I spent an afternoon losing to all four defenses. Then I stopped fighting them, because the data was already sitting on my screen: I had the page open in Chrome.
Every defense a site can raise was stacked on this one. A plain HTTP fetch of the listing URL got redirected to a login page. Sending cookies got past that, until the anti-bot layer noticed a client with no browser fingerprint and started serving challenges. And even a "successful" fetch was useless, because the response was an almost-empty HTML shell — the actual numbers are rendered client-side by the app's JavaScript, so the price I wanted simply wasn't in the bytes I'd downloaded. Each of these is solvable on its own with enough effort (headless browser, session replay, a rendering engine), and stacking all the solutions together is how a weekend project becomes an unmaintained pile of anti-detection hacks that breaks every time the site ships.
The tell that I was solving the wrong problem: I was reconstructing, badly, a thing that already existed and worked perfectly. A logged-in browser session that renders the page and shows me the price. I had one open in front of me.
The reframe is small and it changes everything. Don't fetch the page — read the page you already fetched by opening it. When the portal is open in a Chrome tab, that tab has already passed the login, already survived the bot check (it's a real browser doing real navigation), and already run the JavaScript that paints the prices into the DOM. The numbers are right there in document.body.innerText. The only thing missing is a way to reach into that tab from a script.
On macOS, that way is Apple Events. Chrome can evaluate JavaScript inside any open tab on request (a one-time setting, View → Developer → Allow JavaScript from Apple Events), and a JXA script — JavaScript for Automation, run via osascript — can drive it. The whole bridge is about forty lines: enumerate the windows and tabs, find the first tab whose URL contains a substring you pass in, and evaluate an expression inside it.
// read-tab.js — osascript -l JavaScript read-tab.js "new.land.naver.com" "document.body.innerText"
function run(argv) {
const [pattern, jsExpr = "document.body.innerText"] = argv;
const Chrome = Application("Google Chrome");
let tab = null;
for (const w of Chrome.windows())
for (const t of w.tabs())
if (t.url().indexOf(pattern) !== -1) { tab = t; break; }
if (!tab) throw new Error("no open tab matching: " + pattern);
// Evaluate inside the live, authenticated, already-rendered tab.
const wrapped =
"(function(){try{const v=(" + jsExpr + ");" +
"return typeof v===\"string\"?v:JSON.stringify(v);}" +
"catch(e){return \"JS_ERROR: \"+(e&&e.message||e);}})()";
return tab.execute({ javascript: wrapped });
}
That's the entire mechanism. Point it at a URL substring, hand it any expression, and it returns the result as text. Pass document.body.innerText for the rendered page, a specific querySelectorAll(...) to pull just the fields you want, or document.documentElement.outerHTML to get the post-render HTML for a downstream parser. From a Python collector it's one subprocess call; the deterministic pipeline downstream never knows the data came from a browser tab instead of an API.
The reason this feels like cheating is that it doesn't defeat any of the defenses — it makes all of them irrelevant, because the human already dealt with each one before the script ever runs.
| Wall | Why it's already gone |
|---|---|
| Login / session | You read the tab under the user's authenticated session — no credentials in your code, nothing to store or refresh. |
| JS-rendered SPA | You read the DOM after the app's JavaScript ran, so you see the painted numbers, not the empty shell a fetch returns. |
| Bot detection | You issue no HTTP request at all. There is no headless client to fingerprint; the traffic already happened, made by a real browser a human drove. |
robots.txt / rate limits | You aren't crawling. You read one page the user chose to open, at the rate a person clicks — the politest possible traffic profile. |
Every one of those problems is hard to solve in a scraper and free to inherit from a browser. That's the whole trade: you give up automation-at-scale and get correctness-without-a-fight.
This is a sharp tool with a narrow blade, and pretending otherwise is how you get burned.
value setter, so a plain el.value = "..." is silently ignored and no input/change event fires — the app never sees your input. Writing has to go through the native value setter plus manually dispatched events, and it's fragile enough that I keep the reader strictly read-only and treat interaction as a separate concern.The fit is narrow and clear: a personal, low-volume tool that needs data from a site that fights scrapers, where you are already an authenticated human looking at the page. Live prices for your own buy decision, a number off a dashboard you have access to, a value from a portal you log into anyway. In that slot it's unbeatable, because you're not extracting anything you couldn't already see — you're just saving yourself the copy-paste. Outside that slot — scale, headless, other people's data, evading a wall you're not entitled to be past — it's the wrong tool, and the boundary isn't a limitation to engineer around. It's what keeps the technique honest: it only ever reads a page a person chose to open.
Q. How do you scrape a site that blocks headless scrapers for a personal tool?
Read the DOM out of the browser tab you already have open and logged in, rather than fetching the page yourself. A small AppleScript/JXA bridge finds the Chrome tab by a URL substring and evaluates document.body.innerText — or a targeted selector — inside that live tab. You inherit the session and the fully JS-rendered content and add zero HTTP requests, so there's no scraper for the site's defenses to detect.
Q. Why does reading an open tab sidestep bot detection, login walls, and SPA rendering all at once?
Because the human already solved them. The login is passed under the user's session; the SPA problem is gone because you read the DOM after the site's JavaScript rendered the numbers; bot detection never fires because you make no new request — the page is already loaded, and you only read what's on screen, at human rate.
Q. Can you also fill forms or click through an already-open tab this way?
Reading is reliable; writing is much harder. Apple Events JavaScript runs in an isolated world, and frameworks like React override the native value setter, so el.value = x is ignored and no input/change event fires. Writing must go through the native value setter plus dispatched events. Keep the open-tab reader strictly read-only and treat writing as a separate, more fragile problem.
Q. When should you NOT use this technique?
It's a single-user, single-machine technique for personal tools, not a scalable or commercial scraper. It depends on a human having the page open, a one-time Apple Events permission, and low volume. If you need server-side collection, high throughput, or headless cloud runs, this doesn't apply — respect the site's terms and rate limits. The point is that it stays at human scale and only reads pages the user chose to open.