gaunt ridge
I'm trying to access https://www.thechronicle.com.au/ (News Corp Australia) but getting blocked.
Error: "You might have been detected and blocked as a crawler bot!"
Tried:
• web_fetch - Cookie restrictions
• agent-browser - Bot detection triggered
Setup: OpenClaw 2026.2.17, Agent Browser skill, headless mode
Questions:
Goal: News article monitoring
glad schoonerBOT
barren rootBOT
That block message is the site’s anti-bot / anti-automation doing its job. OpenClaw (and Playwright-style “agent browser” automation in general) will often get flagged there, especially in headless mode. I can’t help with “Cloudflare bypass”, stealth tricks, CAPTCHA-solving services, or residential proxy tactics meant to evade those controls.
What is reasonable (and usually compliant) for “news monitoring”:
If you tell me what “monitoring” means concretely (e.g., “track homepage headlines”, “track a topic search”, “watch a specific author page”, “fetch full article text”), I can suggest a workflow that’s more robust and stays on the right side of access/ToS.
gaunt ridge
@barren root Setting up Browser Relay for news sites - need detailed guidance
Goal: Access full news articles (The Chronicle, etc.)
Questions:
Browser Choice
Extensions - Allowed vs Problematic
âś“ Allowed: Ad blockers (uBlock)? Password managers?
âś— Problematic: Privacy Badger, VPN extensions, fingerprint randomizers?
Profile Setup
Workflow
Does this work for other sites?
Security
Troubleshooting
Current: OpenClaw 2026.2.17, can install any browser
What's your recommended browser+extension stack?
barren rootBOT
For “news sites with bot detection”, Browser Relay (Chrome extension takeover) is usually the most reliable legit approach because it uses a real, interactive browser session (your cookies, your login, normal rendering) instead of a fresh headless automation context.
Docs worth bookmarking:
Use Chrome stable, headed (normal window).
Why:
Notes:
For bot-sensitive publishers, aim for a minimal extension stack in the profile you use for relay.
Usually OK
Often causes weird breakage / detection / missing content
Recommendation: create a dedicated “OpenClaw Relay” Chrome profile with as few extensions as possible.
Best practice: Dedicated Chrome profile → log into the news site(s) there → use relay on tabs in that profile.
On the machine that runs Chrome:
openclaw browser extension install
openclaw browser extension path
Then in Chrome:
chrome://extensionsopenclaw browser extension pathON when attachedOpenClaw only controls tabs you explicitly attach (it does not auto-attach random tabs).
From CLI (optional sanity check):
openclaw browser --browser-profile chrome tabs
In agent/tool calls: use the browser tool with profile="chrome".
There’s no universal safe number. Practical advice:
From the docs’ security notes: when attached, the model can read and interact with the page content of that attached tab (click/type/navigate, etc.). It is not isolated like the dedicated managed browser.
Practical safety checklist:
!Most common causes:
If your Gateway is remote: run a node host on the machine that runs Chrome (so the gateway can proxy browser actions to that node). (This is in the Chrome extension doc.)
If your sessions are sandboxed, you may need:
{
agents: {
defaults: {
sandbox: {
browser: { allowHostControl: true }
}
}
}
}
…and then ensure you’re targeting the host browser control (not the sandbox browser).
Non-evasion steps that actually help:
If you tell me where your Gateway is running (same machine as Chrome vs remote VPS) and whether you’re using sandboxing/Docker, I can give the exact “you should run node host here / set this config knob” path for your setup.
gaunt ridge
@barren root
Gateway: Same machine as Chrome
Should be straightforward - no "node host" config needed?
barren rootBOT
Yep — if Gateway + Chrome are on the same machine and you’re not sandboxed, you normally don’t need any node host or extra routing. The two things people miss are (a) the extension must be loaded/unpacked, and (b) you must explicitly attach a tab (badge ON).
openclaw browser extension install
openclaw browser extension path
Then in Chrome: chrome://extensions → Developer mode → Load unpacked → select that folder → pin the extension.
https://www.thechronicle.com.au/ in ChromeONopenclaw browser status --browser-profile chrome
openclaw browser tabs --browser-profile chrome
If you see “relay running but no tab connected”, it simply means the tab isn’t attached (or you attached in a different Chrome profile/window than you think).
! (relay not reachable)On the same machine, that usually means the local relay server isn’t up or ports are blocked. Quick check:
openclaw gateway status
openclaw browser status --browser-profile chrome
Also open the extension Options page — it shows whether the relay is reachable.
Make sure the agent is using the relay profile:
browser tool calls should use profile="chrome" (that’s the extension relay profile).Docs (exact flow + security notes): https://docs.openclaw.ai/tools/chrome-extension
If you paste the output of openclaw browser status --browser-profile chrome (and whether the badge shows ON, …, or !), I can tell you exactly which part is failing.
gaunt ridge
@barren root Two quick questions:
Gateway token mismatch after update - what's the correct rotate command? Tried:
Can I use Bitwarden/1Password in the dedicated Chrome profile for auto-login to subscription sites? Safe?
Setup: Local gateway, no sandbox, Chrome 145.
barren rootBOT
openclaw devices rotate … is for paired device tokens (nodes/clients), not the Gateway’s shared auth token. It also requires --device + --role. Docs: https://docs.openclaw.ai/cli/devicesFor the Gateway auth token, rotate by setting a new value for gateway.auth.token, then restart:
# show current
openclaw config get gateway.auth.token
# set a new one (pick one)
openclaw config set gateway.auth.token "$(openssl rand -hex 32)"
# or if openssl isn’t available:
openclaw config set gateway.auth.token "$(node -e "console.log(require('crypto').randomBytes(32).toString('hex'))")"
# apply
openclaw gateway restart
If what you’re seeing is “token mismatch” in the Dashboard, it’s often just the UI caching the old token in localStorage. Fix: paste the current token into Control → Overview → Gateway Access (token field), then reconnect. (There are two auth inputs side-by-side: token + password.)
If it still doesn’t match, run openclaw gateway status and check whether Config (cli) and Config (service) point to different configs (service using an older file is a common gotcha).
Mostly “yes, but be intentional about it”.
What’s safe-ish:
Main risks (real ones):
Mitigations I recommend:
If you want, tell me whether you’re using Bitwarden or 1Password and I can point to the specific settings toggles to reduce autofill/phishing risk.