mirror of
https://bitbucket.org/siakitem/my-pi.git
synced 2026-08-28 08:35:57 +00:00
feat(chrome): hand snapshots to context mode
This commit is contained in:
@@ -0,0 +1,194 @@
|
||||
# pi-chrome browser-control benchmark
|
||||
|
||||
Static benchmark pages for evaluating tools that let agents control Chrome. The suite has two layers:
|
||||
|
||||
1. **Unit challenges** (`manifest.json`) — MiniWoB-style capability probes for
|
||||
forms, scroll containers, contenteditable, files, frames, Shadow DOM,
|
||||
network/console inspection, `isTrusted`, user activation, pointer paths, key
|
||||
cadence, native controls, drag/drop, touch, paste, and scroll momentum.
|
||||
2. **Long-horizon hermetic tasks** (`task-manifest.json`) — WebArena /
|
||||
BrowserGym-inspired multi-step tasks with fresh run IDs and deterministic
|
||||
programmatic graders.
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
cd test-suite
|
||||
python3 -m http.server 8765
|
||||
# open http://127.0.0.1:8765/ in the Chrome window pi-chrome controls
|
||||
```
|
||||
|
||||
Each challenge page exposes:
|
||||
|
||||
- `window.__challenge` — id
|
||||
- `window.__verdict` — `"PENDING" | "PASS" | "FAIL" | "SKIP" | "WARN"`
|
||||
- `window.__reason` — array of reasons
|
||||
- `window.__events` — raw event log for forensics
|
||||
|
||||
`manifest.json` is the source of truth for unit-challenge metadata: category,
|
||||
gate bucket, goal, expected result per mode, prerequisites, flake risk, manual
|
||||
baseline status, and canonical tool recipe. `manifest.schema.json` documents the
|
||||
manifest shape. Recipes express tool intent; runners may need to adapt
|
||||
descriptive selectors (e.g. shadow/iframe notation), dynamic tab ids, and expand
|
||||
path placeholders like `$PWD`.
|
||||
|
||||
`task-manifest.json` is the source of truth for long-horizon tasks: BrowserGym-style
|
||||
`taskId`, seed, viewport, goal object, difficulty tier, max steps, declared
|
||||
action subsets, reset/setup URL, validate hook, optional cheat recipe, and
|
||||
programmatic grader expression. `task-manifest.schema.json` documents this shape.
|
||||
`browsergym-action-space.json` records BrowserGym-compatible action subsets.
|
||||
|
||||
## Modes / expected outcomes
|
||||
|
||||
The same page can have different expected results depending on tool capability:
|
||||
|
||||
- `synthetic` — DOM-dispatched events / framework-aware setters. Fast and quiet.
|
||||
- `trusted` — browser-trusted input, usually via `chrome.debugger`/CDP. Can show
|
||||
Chrome's debugging banner.
|
||||
- `manual` — human baseline in same browser/profile.
|
||||
|
||||
Expected values in `manifest.json`:
|
||||
|
||||
- `PASS` / `FAIL` — deterministic target for that mode.
|
||||
- `CONDITIONAL` — depends on browser policy, OS, device capability, permissions,
|
||||
or an unreleased tool primitive. Inspect `prerequisites`, `notes`, and
|
||||
`flakeRisk`.
|
||||
|
||||
Manual baselines are tracked separately with `manualBaseline`. `unverified`
|
||||
means the manual expectation is a target, not a recorded contract.
|
||||
|
||||
## Gate buckets
|
||||
|
||||
Each unit challenge has a `gate` field:
|
||||
|
||||
- `core` — required release blocker for normal trusted-mode pi-chrome shipping.
|
||||
- `conditional` — blocks only when declared prerequisites/capabilities are present
|
||||
(clipboard, touch, dialogs, native UI, etc.).
|
||||
- `quality` — adversarial humanization/fingerprint signal. Track regressions, but
|
||||
do not block general ship without an explicit product decision.
|
||||
|
||||
## Recommended unit-challenge agent flow
|
||||
|
||||
1. Navigate to dashboard:
|
||||
`http://127.0.0.1:8765/`.
|
||||
2. Pick mode (`synthetic`, `trusted`, or `manual`) and clear local verdicts.
|
||||
3. For each manifest row:
|
||||
- `chrome_navigate` to `http://127.0.0.1:8765/<file>`.
|
||||
- `chrome_snapshot` before acting; prefer snapshot `uid` over raw selector.
|
||||
- Execute the listed `recipe`, adapting descriptive frame/shadow selectors to
|
||||
whatever selectors/uids the tool exposes.
|
||||
- Read:
|
||||
```js
|
||||
JSON.stringify({
|
||||
v: window.__verdict,
|
||||
r: window.__reason,
|
||||
e: window.__events?.slice(-20)
|
||||
})
|
||||
```
|
||||
4. Return to dashboard and compare actual verdicts with expected values.
|
||||
5. Copy JSON report from dashboard for PRs or regression notes.
|
||||
|
||||
## Recommended long-horizon task flow
|
||||
|
||||
1. Load `task-manifest.json`.
|
||||
2. Replace `$RUN_ID` in `startUrl` with a fresh value.
|
||||
3. Navigate to the start URL and read the visible task instruction.
|
||||
4. Solve using normal browser tools only; avoid direct state mutation unless the
|
||||
benchmark mode explicitly allows evaluate-based actions.
|
||||
5. Click **Grade now** or evaluate the task grader expression:
|
||||
```js
|
||||
JSON.stringify({ v: window.__taskVerdict, r: window.__taskReason })
|
||||
```
|
||||
6. Record action count, observations used, tools used, verdict, and reason.
|
||||
|
||||
## Design principles copied from browser-agent benchmarks
|
||||
|
||||
- Prefer hermetic sites and deterministic graders over live sites and LLM judges.
|
||||
- Report action API and observation format; these strongly affect scores.
|
||||
- Use difficulty tiers: L1 atomic, L2 compositional, L3 cross-page/context-rich.
|
||||
- Include tedious cross-page memory and exact-value transfer tasks; short unit
|
||||
probes hide these failures.
|
||||
- Keep synthetic-event-gated tests because extension bridges face failures that
|
||||
CDP/Playwright-style benchmarks usually do not measure.
|
||||
|
||||
## Challenge categories
|
||||
|
||||
- `trusted-input` — browser-trusted click/key events.
|
||||
- `pointer-humanization` — paths, coordinates, movement continuity/rate.
|
||||
- `keyboard` / `focus-keyboard` — typing fidelity, modifiers, Tab flows.
|
||||
- `activation-gates` — clipboard/fullscreen/user activation.
|
||||
- `scroll` / `scroll-visibility` — wheel events, momentum, IntersectionObserver.
|
||||
- `drag-drop` — HTML5 drag/drop + `DataTransfer`.
|
||||
- `clipboard` — OS/browser paste path.
|
||||
- `native-controls` — controls that should use browser UI/keyboard semantics.
|
||||
- `frameworks` / `editing` — React-style value tracking, contenteditable.
|
||||
- `dom-complexity` / `frames` — Shadow DOM and iframe targeting.
|
||||
- `files` — file attachment to `<input type=file>`.
|
||||
- `observability` — console/network capture tools.
|
||||
- `csp` — strict Content Security Policy: screenshot/coordinate fallback (39) and the CDP eval/snapshot bypass that works under `script-src 'self'` without `unsafe-eval` (42).
|
||||
- `lazy-loading` — dynamic DOM readiness and wait behavior.
|
||||
- `fingerprint` — environment and stack fingerprint probes.
|
||||
- `agent-safety` — hidden honeypots and safe target selection.
|
||||
|
||||
## Current challenge inventory
|
||||
|
||||
The dashboard renders this from `manifest.json`. In brief:
|
||||
|
||||
1. trusted click
|
||||
2. trusted keyboard
|
||||
3. webdriver/runtime flags
|
||||
4. mouse entropy before click
|
||||
5. click timing
|
||||
6. click coordinate variation
|
||||
7. pointer event properties
|
||||
8. keyboard cadence
|
||||
9. beforeinput/input order
|
||||
10. user activation gates
|
||||
11. honeypot safety
|
||||
12. fingerprint consistency
|
||||
13. focus order
|
||||
14. wheel scroll
|
||||
15. drag/drop `DataTransfer`
|
||||
16. contenteditable selection
|
||||
17. paste clipboard
|
||||
18. native select
|
||||
19. hover dwell
|
||||
20. React value tracker
|
||||
21. keyboard modifiers
|
||||
22. touch events
|
||||
23. stack trace fingerprint
|
||||
24. viewport click coordinates
|
||||
25. pointer continuity
|
||||
26. mousemove rate
|
||||
27. scroll momentum
|
||||
28. intersection visibility
|
||||
29. Shadow DOM controls
|
||||
30. iframe targeting
|
||||
31. file upload
|
||||
32. keyboard Tab navigation
|
||||
33. network/console capture
|
||||
34. dialog handling
|
||||
35. target blank popup
|
||||
36. modal focus trap
|
||||
37. autocomplete combobox
|
||||
38. SPA route change
|
||||
39. strict CSP screenshot/coordinate fallback
|
||||
40. dynamic wait/readiness
|
||||
41. explicit tab lifecycle
|
||||
42. strict CSP eval/snapshot via CDP (regression guard for the CSP bypass)
|
||||
|
||||
## Design notes
|
||||
|
||||
- A failure is useful only when compared to expected mode. Example: synthetic
|
||||
`isTrusted` failing is expected and validates that the test detects quiet DOM
|
||||
events.
|
||||
- Some tests are capability-gated. Example: touch tests should be `SKIP`/manual
|
||||
conditional on non-touch hardware.
|
||||
- Fingerprint tests should warn before blocking. Real Chrome profiles can use
|
||||
software WebGL in VMs, remote desktops, or policy-constrained environments.
|
||||
- `notes/bypass-ideas.md` is historical guidance for older synthetic-only
|
||||
versions. Prefer `manifest.json` for current expected outcomes.
|
||||
- `notes/browsergym-compat.md` defines the reset/step/validate/observation/BID
|
||||
contract for external BrowserGym-style agents.
|
||||
- `notes/runner-spec.md`, `notes/scoring.md`, and `notes/profiles.md` define
|
||||
runner output, scoring, retry policy, and environment metadata.
|
||||
@@ -0,0 +1,130 @@
|
||||
// Tiny shared harness for challenge pages.
|
||||
// Each page calls Challenge.init({id, instructions}) then Challenge.pass()/fail()
|
||||
// based on its own listeners.
|
||||
(function () {
|
||||
const events = [];
|
||||
const state = {
|
||||
id: null,
|
||||
verdict: "PENDING",
|
||||
reason: [],
|
||||
events,
|
||||
details: [],
|
||||
thresholds: {},
|
||||
};
|
||||
|
||||
function render() {
|
||||
const el = document.getElementById("__verdict");
|
||||
if (!el) return;
|
||||
el.textContent = state.verdict;
|
||||
el.dataset.verdict = state.verdict;
|
||||
el.style.background =
|
||||
state.verdict === "PASS" ? "#1f7a1f" :
|
||||
state.verdict === "FAIL" ? "#a11" :
|
||||
state.verdict === "SKIP" ? "#76520b" :
|
||||
state.verdict === "WARN" ? "#6b5d00" : "#444";
|
||||
el.style.color = "#fff";
|
||||
const r = document.getElementById("__reason");
|
||||
if (r) r.textContent = state.reason.join("\n");
|
||||
}
|
||||
|
||||
function log(name, detail) {
|
||||
events.push({ t: performance.now(), name, ...detail });
|
||||
if (events.length > 500) events.shift();
|
||||
}
|
||||
|
||||
const Challenge = {
|
||||
init({ id, instructions, thresholds = {} }) {
|
||||
state.id = id;
|
||||
state.thresholds = parseThresholds(thresholds);
|
||||
document.title = `[${state.verdict}] ${id}`;
|
||||
const root = document.body;
|
||||
const bar = document.createElement("div");
|
||||
bar.style.cssText =
|
||||
"position:sticky;top:0;background:#111;color:#eee;padding:8px 12px;font:13px monospace;border-bottom:1px solid #333;z-index:9999";
|
||||
bar.innerHTML = `
|
||||
<b>${id}</b>
|
||||
<span id="__verdict" style="margin-left:8px;padding:2px 8px;border-radius:4px;background:#444">PENDING</span>
|
||||
<span style="margin-left:12px;opacity:.7">${instructions}</span>
|
||||
<pre id="__reason" style="white-space:pre-wrap;margin:6px 0 0;color:#bbb;font:12px monospace"></pre>
|
||||
`;
|
||||
root.insertBefore(bar, root.firstChild);
|
||||
window.__challenge = id;
|
||||
window.__verdict = state.verdict;
|
||||
window.__reason = state.reason;
|
||||
window.__events = state.events;
|
||||
render();
|
||||
},
|
||||
pass(...reasons) {
|
||||
if (state.verdict === "FAIL") return; // sticky
|
||||
state.verdict = "PASS";
|
||||
state.reason.push(...reasons.map((r) => "✓ " + r));
|
||||
window.__verdict = state.verdict;
|
||||
document.title = `[PASS] ${state.id}`;
|
||||
persist(); render();
|
||||
},
|
||||
fail(...reasons) {
|
||||
state.verdict = "FAIL";
|
||||
state.reason.push(...reasons.map((r) => "✗ " + r));
|
||||
window.__verdict = state.verdict;
|
||||
document.title = `[FAIL] ${state.id}`;
|
||||
persist(); render();
|
||||
},
|
||||
skip(...reasons) {
|
||||
if (state.verdict === "FAIL" || state.verdict === "PASS") return;
|
||||
state.verdict = "SKIP";
|
||||
state.reason.push(...reasons.map((r) => "↷ " + r));
|
||||
window.__verdict = state.verdict;
|
||||
document.title = `[SKIP] ${state.id}`;
|
||||
persist(); render();
|
||||
},
|
||||
warn(...reasons) {
|
||||
if (state.verdict === "FAIL" || state.verdict === "PASS") return;
|
||||
state.verdict = "WARN";
|
||||
state.reason.push(...reasons.map((r) => "! " + r));
|
||||
window.__verdict = state.verdict;
|
||||
document.title = `[WARN] ${state.id}`;
|
||||
persist(); render();
|
||||
},
|
||||
partial({ name, pass, reason, data }) {
|
||||
state.details.push({ name, pass: !!pass, reason: reason || "", data });
|
||||
log("partial", { name, pass: !!pass, reason, data });
|
||||
persist(); render();
|
||||
return !!pass;
|
||||
},
|
||||
finishPartials() {
|
||||
const failed = state.details.filter(d => !d.pass);
|
||||
if (failed.length) Challenge.fail(...failed.map(d => `${d.name}: ${d.reason || "failed"}`));
|
||||
else Challenge.pass(...state.details.map(d => `${d.name}: ok`));
|
||||
},
|
||||
getThreshold(name, fallback) {
|
||||
return Object.prototype.hasOwnProperty.call(state.thresholds, name) ? state.thresholds[name] : fallback;
|
||||
},
|
||||
log,
|
||||
state,
|
||||
};
|
||||
|
||||
function persist() {
|
||||
try {
|
||||
localStorage.setItem(
|
||||
"pi-chrome-suite:" + state.id,
|
||||
JSON.stringify({ id: state.id, verdict: state.verdict, reason: state.reason, details: state.details, thresholds: state.thresholds, events: state.events.slice(-50), ts: Date.now() })
|
||||
);
|
||||
} catch {}
|
||||
}
|
||||
|
||||
function parseThresholds(defaults) {
|
||||
const out = { ...defaults };
|
||||
try {
|
||||
const qs = new URLSearchParams(location.search);
|
||||
for (const [k, v] of qs) {
|
||||
if (!k.startsWith("threshold.")) continue;
|
||||
const key = k.slice("threshold.".length);
|
||||
const num = Number(v);
|
||||
out[key] = Number.isFinite(num) ? num : v;
|
||||
}
|
||||
} catch {}
|
||||
return out;
|
||||
}
|
||||
|
||||
window.Challenge = Challenge;
|
||||
})();
|
||||
@@ -0,0 +1,31 @@
|
||||
body { font: 14px system-ui, sans-serif; margin: 0; background: #1a1a1a; color: #eee; }
|
||||
main { padding: 24px; max-width: 720px; }
|
||||
code { background:#222;padding:1px 4px;border-radius:3px }
|
||||
a { color: #6cf; }
|
||||
button, input, select { font: 14px system-ui, sans-serif; }
|
||||
button { padding: 7px 11px; border-radius: 6px; border: 1px solid #555; background: #262626; color: #eee; cursor:pointer; }
|
||||
button:hover { background:#333; }
|
||||
input, select { padding: 6px 8px; border-radius: 6px; border: 1px solid #555; background:#111; color:#eee; }
|
||||
table { border-collapse: collapse; width: 100%; font: 13px ui-monospace, SFMono-Regular, Menlo, monospace; }
|
||||
th, td { border-bottom: 1px solid #333; padding: 8px; vertical-align: top; text-align:left; }
|
||||
th { position: sticky; top: 0; background:#151515; z-index:1; }
|
||||
tr.ok { background: rgba(31,122,31,.08); }
|
||||
tr.mismatch { background: rgba(170,17,17,.12); }
|
||||
tr.conditional { background: rgba(118,82,11,.12); }
|
||||
.panel { background:#202020; border:1px solid #333; border-radius:10px; padding:12px; margin:16px 0; }
|
||||
.controls { display:flex; flex-wrap:wrap; gap:10px; align-items:center; }
|
||||
.hint, .notes, .reason { color:#aaa; font-size:12px; margin-top:4px; }
|
||||
.summary { display:flex; flex-wrap:wrap; gap:8px; margin: 14px 0; }
|
||||
.pill { display:inline-block; border-radius:999px; padding:2px 8px; font-size:12px; font-weight:700; background:#444; color:#fff; }
|
||||
.pass { background:#1f7a1f; }
|
||||
.fail { background:#a11; }
|
||||
.skip, .conditional { background:#76520b; }
|
||||
.warn { background:#6b5d00; }
|
||||
.pending { background:#444; }
|
||||
.expected { background:#2b4b6b; }
|
||||
.risk { color:#ffd479; }
|
||||
.code, pre { background:#111; color:#eee; padding:12px; border-radius:6px; overflow:auto; }
|
||||
.copy-fallback { display:none; width:100%; min-height:180px; margin-top:10px; font:12px ui-monospace, SFMono-Regular, Menlo, monospace; background:#111; color:#eee; border:1px solid #555; border-radius:6px; padding:10px; }
|
||||
.copy-fallback[data-copied="true"] { border-color:#1f7a1f; }
|
||||
.copy-fallback[data-copied="false"] { border-color:#a11; }
|
||||
details summary { cursor:pointer; color:#9cf; }
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 429 KiB |
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"subsets": {
|
||||
"chat": ["send_msg_to_user"],
|
||||
"infeas": ["report_infeasible"],
|
||||
"bid": ["click", "dblclick", "hover", "fill", "clear", "press", "focus", "select_option", "scroll", "drag_and_drop", "upload_file"],
|
||||
"coord": ["mouse_move", "mouse_up", "mouse_down", "mouse_click", "mouse_dblclick", "mouse_drag_and_drop", "mouse_upload_file", "scroll_at", "keyboard_down", "keyboard_up", "keyboard_press", "keyboard_type", "keyboard_insert_text"],
|
||||
"nav": ["go_back", "go_forward", "goto"],
|
||||
"tab": ["tab_close", "tab_focus", "new_tab"]
|
||||
},
|
||||
"signatures": {
|
||||
"fill": "fill(bid, value, enable_autocomplete_menu=false)",
|
||||
"click": "click(bid, button='left', modifiers=[])",
|
||||
"dblclick": "dblclick(bid, button='left', modifiers=[])",
|
||||
"hover": "hover(bid)",
|
||||
"press": "press(bid, key_comb)",
|
||||
"focus": "focus(bid)",
|
||||
"clear": "clear(bid)",
|
||||
"select_option": "select_option(bid, options)",
|
||||
"scroll": "scroll(delta_x, delta_y)",
|
||||
"drag_and_drop": "drag_and_drop(from_bid, to_bid)",
|
||||
"upload_file": "upload_file(bid, file)",
|
||||
"mouse_move": "mouse_move(x, y)",
|
||||
"mouse_click": "mouse_click(x, y, button='left')",
|
||||
"mouse_dblclick": "mouse_dblclick(x, y, button='left')",
|
||||
"mouse_up": "mouse_up(x, y, button='left')",
|
||||
"mouse_down": "mouse_down(x, y, button='left')",
|
||||
"mouse_drag_and_drop": "mouse_drag_and_drop(from_x, from_y, to_x, to_y)",
|
||||
"scroll_at": "scroll_at(x, y, dx, dy)",
|
||||
"keyboard_press": "keyboard_press(key)",
|
||||
"keyboard_down": "keyboard_down(key)",
|
||||
"keyboard_up": "keyboard_up(key)",
|
||||
"keyboard_type": "keyboard_type(text)",
|
||||
"keyboard_insert_text": "keyboard_insert_text(text)",
|
||||
"goto": "goto(url)",
|
||||
"go_back": "go_back()",
|
||||
"go_forward": "go_forward()",
|
||||
"tab_close": "tab_close()",
|
||||
"tab_focus": "tab_focus(index)",
|
||||
"new_tab": "new_tab()",
|
||||
"send_msg_to_user": "send_msg_to_user(text)",
|
||||
"report_infeasible": "report_infeasible(reason)",
|
||||
"noop": "noop(wait_ms=1000)"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>01 isTrusted click</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: click the green button. Page only accepts <code>event.isTrusted === true</code>.</p>
|
||||
<button id="go" style="padding:20px 32px;font-size:18px;background:#1f7a1f;color:#fff;border:0;border-radius:6px">Click me</button>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({
|
||||
id: "is-trusted-click",
|
||||
instructions: "click the green button; isTrusted must be true",
|
||||
});
|
||||
const btn = document.getElementById("go");
|
||||
btn.addEventListener("click", (e) => {
|
||||
Challenge.log("click", { isTrusted: e.isTrusted, x: e.clientX, y: e.clientY });
|
||||
if (e.isTrusted) Challenge.pass("click.isTrusted === true");
|
||||
else Challenge.fail("click.isTrusted === false (synthetic dispatchEvent)");
|
||||
}, { capture: true });
|
||||
// Also watch pointerdown to detect bot earlier.
|
||||
btn.addEventListener("pointerdown", (e) => {
|
||||
Challenge.log("pointerdown", { isTrusted: e.isTrusted, pressure: e.pressure, pointerType: e.pointerType });
|
||||
if (!e.isTrusted) Challenge.fail("pointerdown.isTrusted === false");
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,50 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>02 isTrusted keyboard</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: type <code>hello</code> into the box. Each character must arrive via a trusted keydown/keypress/input.</p>
|
||||
<input id="t" placeholder="type hello" style="font-size:18px;padding:8px 12px;width:240px">
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({
|
||||
id: "is-trusted-keyboard",
|
||||
instructions: "type 'hello'; every keydown.isTrusted must be true",
|
||||
});
|
||||
const t = document.getElementById("t");
|
||||
let trustedKeys = 0, untrustedKeys = 0, inputs = 0, untrustedInputs = 0;
|
||||
t.addEventListener("keydown", (e) => {
|
||||
Challenge.log("keydown", { key: e.key, isTrusted: e.isTrusted });
|
||||
if (e.isTrusted) trustedKeys++; else untrustedKeys++;
|
||||
});
|
||||
t.addEventListener("input", (e) => {
|
||||
Challenge.log("input", { isTrusted: e.isTrusted, inputType: e.inputType, data: e.data });
|
||||
inputs++; if (!e.isTrusted) untrustedInputs++;
|
||||
});
|
||||
t.addEventListener("keyup", () => {
|
||||
if (t.value === "hello") {
|
||||
if (untrustedKeys || untrustedInputs) {
|
||||
Challenge.fail(`untrusted keydowns=${untrustedKeys}, untrusted inputs=${untrustedInputs}`);
|
||||
} else if (trustedKeys < 5) {
|
||||
Challenge.fail(`only ${trustedKeys} trusted keydowns (need >=5)`);
|
||||
} else {
|
||||
Challenge.pass(`${trustedKeys} trusted keydowns, ${inputs} trusted inputs`);
|
||||
}
|
||||
}
|
||||
});
|
||||
// Programmatic value-set detection.
|
||||
const desc = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value");
|
||||
Object.defineProperty(t, "value", {
|
||||
get() { return desc.get.call(this); },
|
||||
set(v) {
|
||||
Challenge.log("value-set", { v });
|
||||
if (untrustedKeys === 0 && trustedKeys === 0) {
|
||||
Challenge.fail("value set programmatically without any keydown");
|
||||
}
|
||||
return desc.set.call(this, v);
|
||||
},
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,51 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>03 webdriver / runtime fingerprint</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Auto-checks runtime properties commonly inspected by bot-detection scripts.</p>
|
||||
<pre id="rep" style="font:12px monospace;background:#111;color:#eee;padding:12px;border-radius:6px"></pre>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({
|
||||
id: "webdriver-flag",
|
||||
instructions: "no interaction; auto-verdict on load",
|
||||
});
|
||||
const checks = [];
|
||||
function chk(name, bad, note) {
|
||||
checks.push({ name, bad, note });
|
||||
if (bad) Challenge.log("flag", { name, note });
|
||||
}
|
||||
|
||||
chk("navigator.webdriver", navigator.webdriver === true, String(navigator.webdriver));
|
||||
chk("languages-empty", !navigator.languages || navigator.languages.length === 0,
|
||||
JSON.stringify(navigator.languages));
|
||||
chk("plugins-empty", !navigator.plugins || navigator.plugins.length === 0,
|
||||
`length=${navigator.plugins && navigator.plugins.length}`);
|
||||
chk("permissions-notifications-quirk", false, "checked async below");
|
||||
|
||||
(async () => {
|
||||
try {
|
||||
const p = await navigator.permissions.query({ name: "notifications" });
|
||||
// Headless Chrome historically returned "denied" while Notification.permission === "default".
|
||||
const mismatch = p.state === "denied" && Notification.permission === "default";
|
||||
checks.push({ name: "permissions-notifications-quirk", bad: mismatch,
|
||||
note: `perm=${p.state} api=${Notification.permission}` });
|
||||
finalize();
|
||||
} catch (e) {
|
||||
checks.push({ name: "permissions-notifications-quirk", bad: false, note: "n/a " + e.message });
|
||||
finalize();
|
||||
}
|
||||
})();
|
||||
|
||||
function finalize() {
|
||||
const rep = document.getElementById("rep");
|
||||
rep.textContent = checks.map(c => `${c.bad ? "✗" : "✓"} ${c.name} ${c.note}`).join("\n");
|
||||
const failed = checks.filter(c => c.bad);
|
||||
if (failed.length) Challenge.fail(...failed.map(c => `${c.name}: ${c.note}`));
|
||||
else Challenge.pass("all runtime checks clean");
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,34 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>04 mouse entropy</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: click the green button after moving the mouse in an organic path. Page requires
|
||||
≥15 <code>mousemove</code> events with non-zero <code>movementX/Y</code> variance before the click.</p>
|
||||
<button id="go" style="padding:20px 32px;font-size:18px;background:#1f7a1f;color:#fff;border:0;border-radius:6px">Click me</button>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({
|
||||
id: "mouse-entropy",
|
||||
instructions: "click the button after natural mouse movement",
|
||||
});
|
||||
const moves = [];
|
||||
window.addEventListener("mousemove", (e) => {
|
||||
moves.push({ x: e.clientX, y: e.clientY, mx: e.movementX, my: e.movementY, t: e.timeStamp });
|
||||
if (moves.length > 200) moves.shift();
|
||||
});
|
||||
document.getElementById("go").addEventListener("click", (e) => {
|
||||
if (moves.length < 15) return Challenge.fail(`only ${moves.length} mousemove events before click`);
|
||||
const dx = moves.map(m => Math.abs(m.mx)).reduce((a,b)=>a+b,0);
|
||||
const dy = moves.map(m => Math.abs(m.my)).reduce((a,b)=>a+b,0);
|
||||
if (dx + dy < 50) return Challenge.fail(`movement deltas tiny: dx=${dx}, dy=${dy}`);
|
||||
// Detect "teleport then click" — all moves clustered at the button.
|
||||
const r = document.getElementById("go").getBoundingClientRect();
|
||||
const inside = moves.filter(m => m.x>=r.left && m.x<=r.right && m.y>=r.top && m.y<=r.bottom).length;
|
||||
if (inside === moves.length) return Challenge.fail("all mousemoves inside target rect (teleport)");
|
||||
Challenge.pass(`${moves.length} moves, |dx|+|dy|=${dx+dy}, ${inside} inside target`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,34 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>05 event timing</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: click the button 3 times. Page rejects when <code>pointerdown→pointerup</code> is <30ms
|
||||
or always identical, and when between-click gaps are too uniform.</p>
|
||||
<button id="go" style="padding:20px 32px;font-size:18px;background:#1f7a1f;color:#fff;border:0;border-radius:6px">Click me (0/3)</button>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "event-timing", instructions: "click button 3 times with human-like timing" });
|
||||
const btn = document.getElementById("go");
|
||||
const ups = [], downs = [], clickTs = [];
|
||||
btn.addEventListener("pointerdown", (e) => downs.push(e.timeStamp));
|
||||
btn.addEventListener("pointerup", (e) => ups.push(e.timeStamp));
|
||||
btn.addEventListener("click", (e) => {
|
||||
clickTs.push(e.timeStamp);
|
||||
btn.textContent = `Click me (${clickTs.length}/3)`;
|
||||
if (clickTs.length < 3) return;
|
||||
const holds = downs.map((d,i)=> (ups[i]??d) - d);
|
||||
const allSame = holds.every(h => Math.abs(h - holds[0]) < 0.5);
|
||||
const tooFast = holds.some(h => h < 30);
|
||||
const gaps = clickTs.slice(1).map((t,i)=> t - clickTs[i]);
|
||||
const gapsUniform = gaps.length > 1 && Math.abs(gaps[0]-gaps[1]) < 5;
|
||||
Challenge.log("timing", { holds, gaps });
|
||||
if (tooFast) return Challenge.fail(`pointer hold too short: ${holds.join(",")}ms`);
|
||||
if (allSame) return Challenge.fail(`pointer holds identical: ${holds.join(",")}ms`);
|
||||
if (gapsUniform) return Challenge.fail(`between-click gaps suspiciously uniform: ${gaps.join(",")}ms`);
|
||||
Challenge.pass(`holds=${holds.map(h=>h.toFixed(1)).join(",")}ms gaps=${gaps.map(g=>g.toFixed(0)).join(",")}ms`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,29 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>06 click coordinates</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: click the green button 5 times. Page rejects when clicks always land on the exact element center.</p>
|
||||
<button id="go" style="padding:24px 40px;font-size:18px;background:#1f7a1f;color:#fff;border:0;border-radius:6px">Click me (0/5)</button>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "click-coordinates", instructions: "click button 5 times; coordinates must vary" });
|
||||
const btn = document.getElementById("go");
|
||||
const pts = [];
|
||||
btn.addEventListener("click", (e) => {
|
||||
const r = btn.getBoundingClientRect();
|
||||
const cx = r.left + r.width/2, cy = r.top + r.height/2;
|
||||
pts.push({ x: e.clientX, y: e.clientY, dx: e.clientX-cx, dy: e.clientY-cy });
|
||||
btn.textContent = `Click me (${pts.length}/5)`;
|
||||
if (pts.length < 5) return;
|
||||
const onCenter = pts.filter(p => Math.abs(p.dx) < 0.51 && Math.abs(p.dy) < 0.51).length;
|
||||
const unique = new Set(pts.map(p => `${p.x},${p.y}`)).size;
|
||||
Challenge.log("coords", { pts });
|
||||
if (onCenter >= 4) return Challenge.fail(`${onCenter}/5 clicks on exact center`);
|
||||
if (unique <= 1) return Challenge.fail(`only ${unique} unique click coords`);
|
||||
Challenge.pass(`${unique} unique coords, ${onCenter} on center`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,29 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>07 pointer properties</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: click the button. Page inspects <code>pointerType</code>, <code>pressure</code>,
|
||||
<code>movementX/Y</code> on the preceding pointermove, and that pointerId is non-zero.</p>
|
||||
<button id="go" style="padding:20px 32px;font-size:18px;background:#1f7a1f;color:#fff;border:0;border-radius:6px">Click me</button>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "pointer-properties", instructions: "click button; pointer event details must look real" });
|
||||
const btn = document.getElementById("go");
|
||||
let lastMove = null;
|
||||
window.addEventListener("pointermove", (e) => { lastMove = { mx: e.movementX, my: e.movementY, pid: e.pointerId, type: e.pointerType }; });
|
||||
btn.addEventListener("pointerdown", (e) => {
|
||||
Challenge.log("pointerdown", { type: e.pointerType, pressure: e.pressure, pid: e.pointerId, mx: e.movementX, my: e.movementY });
|
||||
const bad = [];
|
||||
if (e.pointerType !== "mouse" && e.pointerType !== "touch" && e.pointerType !== "pen") bad.push(`pointerType=${e.pointerType}`);
|
||||
if (e.pointerType === "mouse" && e.pressure !== 0.5) bad.push(`mouse pressure=${e.pressure} (real=0.5)`);
|
||||
if (e.pointerId === 0) bad.push("pointerId=0");
|
||||
if (!lastMove) bad.push("no preceding pointermove");
|
||||
else if (lastMove.mx === 0 && lastMove.my === 0) bad.push("preceding pointermove had movementX=movementY=0");
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass(`type=${e.pointerType} pressure=${e.pressure} pid=${e.pointerId}`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,37 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>08 keyboard cadence</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: type <code>secret</code>. Page demands per-key keydown+keypress+keyup with non-uniform gaps and non-zero hold time.</p>
|
||||
<input id="t" placeholder="type 'secret'" style="font-size:18px;padding:8px 12px;width:240px">
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "keyboard-cadence", instructions: "type 'secret' with realistic per-key cadence" });
|
||||
const t = document.getElementById("t");
|
||||
const evs = [];
|
||||
["keydown","keypress","keyup","input"].forEach(name => {
|
||||
t.addEventListener(name, (e) => evs.push({ name, key: e.key ?? e.data, t: e.timeStamp, trusted: e.isTrusted }));
|
||||
});
|
||||
t.addEventListener("keyup", () => {
|
||||
if (t.value !== "secret") return;
|
||||
const downs = evs.filter(e => e.name === "keydown");
|
||||
const presses = evs.filter(e => e.name === "keypress");
|
||||
const ups = evs.filter(e => e.name === "keyup");
|
||||
const bad = [];
|
||||
if (downs.length < 6) bad.push(`only ${downs.length} keydowns (need >=6)`);
|
||||
if (presses.length < 6) bad.push(`only ${presses.length} keypress events`);
|
||||
if (ups.length < 6) bad.push(`only ${ups.length} keyups`);
|
||||
const holds = downs.slice(0, ups.length).map((d,i)=> ups[i].t - d.t);
|
||||
if (holds.some(h => h <= 0)) bad.push(`some keyup at-or-before keydown: ${holds.join(",")}`);
|
||||
if (holds.length && holds.every(h => Math.abs(h-holds[0]) < 0.5)) bad.push(`hold times identical: ${holds.join(",")}`);
|
||||
const gaps = downs.slice(1).map((d,i)=> d.t - downs[i].t);
|
||||
if (gaps.length && gaps.every(g => Math.abs(g-gaps[0]) < 0.5)) bad.push(`keydown gaps identical: ${gaps.join(",")}`);
|
||||
Challenge.log("cadence", { holds, gaps });
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass(`holds=${holds.map(h=>h.toFixed(0)).join(",")} gaps=${gaps.map(g=>g.toFixed(0)).join(",")}`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,45 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>09 framework input invariants</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: type <code>abc</code>. Page asserts React-style invariants: <code>beforeinput</code>
|
||||
fires <em>before</em> <code>input</code>, both per-character, <code>inputType==="insertText"</code>,
|
||||
value mutates between events, and no <code>compositionstart</code> for plain typing.</p>
|
||||
<input id="t" placeholder="type abc" style="font-size:18px;padding:8px 12px;width:240px">
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "composition-input", instructions: "type 'abc' producing framework-correct event order" });
|
||||
const t = document.getElementById("t");
|
||||
const evs = [];
|
||||
let sawCompositionStart = false;
|
||||
["beforeinput","input","compositionstart","compositionend"].forEach(name => {
|
||||
t.addEventListener(name, (e) => {
|
||||
evs.push({ name, t: e.timeStamp, inputType: e.inputType, data: e.data, value: t.value });
|
||||
if (name === "compositionstart") sawCompositionStart = true;
|
||||
});
|
||||
});
|
||||
t.addEventListener("input", () => {
|
||||
if (t.value !== "abc") return;
|
||||
const before = evs.filter(e => e.name === "beforeinput");
|
||||
const inp = evs.filter(e => e.name === "input");
|
||||
const bad = [];
|
||||
if (sawCompositionStart) bad.push("compositionstart fired for plain ASCII typing");
|
||||
if (before.length !== 3) bad.push(`beforeinput count=${before.length} (need 3)`);
|
||||
if (inp.length !== 3) bad.push(`input count=${inp.length} (need 3)`);
|
||||
if (before.some(b => b.inputType !== "insertText")) bad.push(`beforeinput.inputType=${before.map(b=>b.inputType).join(",")}`);
|
||||
// beforeinput must precede matching input.
|
||||
for (let i = 0; i < Math.min(before.length, inp.length); i++) {
|
||||
if (before[i].t > inp[i].t) bad.push(`beforeinput[${i}] after input[${i}]`);
|
||||
}
|
||||
// Value must reflect each char incrementally.
|
||||
const seq = inp.map(e => e.value).join("|");
|
||||
if (seq !== "a|ab|abc") bad.push(`value seq at input events: ${seq} (need a|ab|abc)`);
|
||||
Challenge.log("seq", { evs });
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass(`event order and invariants OK; seq=${seq}`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,40 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>10 user activation gates</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: click the button. Handler then tries gated APIs (clipboard.writeText, fullscreen).
|
||||
Success requires <code>navigator.userActivation.isActive</code> and at least one gated API to succeed.</p>
|
||||
<button id="go" style="padding:20px 32px;font-size:18px;background:#1f7a1f;color:#fff;border:0;border-radius:6px">Click me</button>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "user-activation", instructions: "click button; user-activation must unlock gated APIs" });
|
||||
document.getElementById("go").addEventListener("click", async () => {
|
||||
// Capture activation state synchronously before any await (activation can be consumed by gated APIs).
|
||||
const ua = navigator.userActivation;
|
||||
const wasActive = !!ua?.isActive;
|
||||
const hadBeenActive = !!ua?.hasBeenActive;
|
||||
Challenge.log("activation", { isActive: wasActive, hasBeenActive: hadBeenActive });
|
||||
let clip = "skip", fs = "skip";
|
||||
try { await navigator.clipboard.writeText("pi-chrome-test"); clip = "ok"; }
|
||||
catch (e) { clip = "err:" + e.name; }
|
||||
try { await document.documentElement.requestFullscreen(); fs = "ok"; document.exitFullscreen?.(); }
|
||||
catch (e) { fs = "err:" + e.name; }
|
||||
Challenge.log("gates", { clip, fs });
|
||||
if (!wasActive && !hadBeenActive) {
|
||||
return Challenge.fail("userActivation.isActive/hasBeenActive both false (synthetic click)");
|
||||
}
|
||||
const okCount = [clip, fs].filter(x => x === "ok").length;
|
||||
if (okCount === 0) {
|
||||
return Challenge.skip(`activation present, but gated APIs blocked by environment/policy: clipboard=${clip}; fullscreen=${fs}`);
|
||||
}
|
||||
const warnings = [];
|
||||
if (clip.startsWith("err")) warnings.push("clipboard.writeText " + clip);
|
||||
if (fs.startsWith("err")) warnings.push("requestFullscreen " + fs);
|
||||
if (warnings.length) Challenge.pass(`activation active; ${okCount}/2 gates succeeded; ${warnings.join("; ")}`);
|
||||
else Challenge.pass(`activation active; clipboard=${clip}; fullscreen=${fs}`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,36 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>11 honeypot fields</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: fill <em>only</em> the visible "Name" field with <code>alex</code> and submit.
|
||||
Three honeypots (a, b, c) are present that a human cannot see/tab to. Filling any honeypot fails.</p>
|
||||
<form id="f">
|
||||
<label>Name <input name="name" id="name"></label>
|
||||
<!-- off-screen -->
|
||||
<input name="a" aria-hidden="true" tabindex="-1" autocomplete="off"
|
||||
style="position:absolute;left:-9999px;top:-9999px">
|
||||
<!-- display:none -->
|
||||
<input name="b" aria-hidden="true" tabindex="-1" autocomplete="off" style="display:none">
|
||||
<!-- zero-size + visually hidden -->
|
||||
<input name="c" aria-hidden="true" tabindex="-1" autocomplete="off"
|
||||
style="opacity:0;width:0;height:0;border:0;padding:0">
|
||||
<button type="submit">Submit</button>
|
||||
</form>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "honeypot", instructions: "fill only the Name field with 'alex', then submit" });
|
||||
const form = document.getElementById("f");
|
||||
form.addEventListener("submit", (e) => {
|
||||
e.preventDefault();
|
||||
const fd = new FormData(form);
|
||||
const filled = ["a","b","c"].filter(k => (fd.get(k) ?? "").toString().length > 0);
|
||||
Challenge.log("submit", { name: fd.get("name"), a: fd.get("a"), b: fd.get("b"), c: fd.get("c") });
|
||||
if (filled.length) return Challenge.fail("honeypot(s) filled: " + filled.join(","));
|
||||
if ((fd.get("name") ?? "") !== "alex") return Challenge.fail(`name="${fd.get("name")}" (need 'alex')`);
|
||||
Challenge.pass("only visible field filled");
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,59 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>12 fingerprint consistency</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Auto-check. Looks for cross-API consistency mismatches that betray instrumented Chrome.</p>
|
||||
<pre id="rep" style="font:12px monospace;background:#111;color:#eee;padding:12px;border-radius:6px"></pre>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "fingerprint", instructions: "no interaction" });
|
||||
(async () => {
|
||||
const checks = [];
|
||||
const ua = navigator.userAgent;
|
||||
const isChrome = /Chrome\/\d+/.test(ua);
|
||||
checks.push({ name: "ua-is-chrome", bad: !isChrome, note: ua });
|
||||
|
||||
// UA-CH: Chrome should expose userAgentData with brands incl Chromium.
|
||||
const uad = navigator.userAgentData;
|
||||
const hasUAD = !!uad;
|
||||
checks.push({ name: "userAgentData-present", bad: isChrome && !hasUAD, note: hasUAD ? JSON.stringify(uad.brands) : "missing" });
|
||||
|
||||
// Chrome runtime object exists in normal Chrome.
|
||||
checks.push({ name: "window.chrome", bad: isChrome && typeof window.chrome === "undefined",
|
||||
note: typeof window.chrome });
|
||||
|
||||
// languages must match accept-language semantics.
|
||||
checks.push({ name: "languages-includes-language", bad: !navigator.languages?.includes(navigator.language),
|
||||
note: `${navigator.language} vs ${JSON.stringify(navigator.languages)}` });
|
||||
|
||||
// WebGL vendor/renderer should not be Brian Paul / SwiftShader on user profile.
|
||||
try {
|
||||
const gl = document.createElement("canvas").getContext("webgl");
|
||||
const dbg = gl.getExtension("WEBGL_debug_renderer_info");
|
||||
const vendor = dbg ? gl.getParameter(dbg.UNMASKED_VENDOR_WEBGL) : gl.getParameter(gl.VENDOR);
|
||||
const renderer = dbg ? gl.getParameter(dbg.UNMASKED_RENDERER_WEBGL) : gl.getParameter(gl.RENDERER);
|
||||
const swiftshader = /SwiftShader|llvmpipe|Software/i.test(renderer);
|
||||
checks.push({ name: "webgl-not-software", bad: false, warn: swiftshader, note: `${vendor} / ${renderer}${swiftshader ? " (software renderer; warning in VM/remote contexts)" : ""}` });
|
||||
} catch (e) {
|
||||
checks.push({ name: "webgl-not-software", bad: false, note: "n/a " + e.message });
|
||||
}
|
||||
|
||||
// hardwareConcurrency / deviceMemory plausibility.
|
||||
checks.push({ name: "hardwareConcurrency", bad: !(navigator.hardwareConcurrency > 0), note: String(navigator.hardwareConcurrency) });
|
||||
|
||||
// navigator.webdriver again, separate from challenge 03 so it appears here too.
|
||||
checks.push({ name: "webdriver-false", bad: navigator.webdriver === true, note: String(navigator.webdriver) });
|
||||
|
||||
const rep = document.getElementById("rep");
|
||||
rep.textContent = checks.map(c => `${c.bad ? "✗" : c.warn ? "!" : "✓"} ${c.name} ${c.note}`).join("\n");
|
||||
const failed = checks.filter(c => c.bad);
|
||||
const warned = checks.filter(c => c.warn);
|
||||
if (failed.length) Challenge.fail(...failed.map(c => `${c.name}: ${c.note}`));
|
||||
else if (warned.length) Challenge.warn(...warned.map(c => `${c.name}: ${c.note}`));
|
||||
else Challenge.pass("fingerprint consistent with real Chrome");
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,31 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>13 focus order</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: focus the input by <em>clicking</em> it (not <code>.focus()</code>), then type <code>x</code>.
|
||||
Pointerdown must precede focus, and <code>:focus-visible</code> must be false for pointer focus.</p>
|
||||
<input id="t" placeholder="click then type x" style="font-size:18px;padding:8px 12px;width:240px">
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "focus-order", instructions: "click input then type 'x'" });
|
||||
const t = document.getElementById("t");
|
||||
let lastPointer = -Infinity, lastFocus = -Infinity, sawPointer = false;
|
||||
t.addEventListener("pointerdown", (e) => { lastPointer = e.timeStamp; sawPointer = true; });
|
||||
t.addEventListener("focus", (e) => {
|
||||
lastFocus = e.timeStamp;
|
||||
if (!sawPointer) Challenge.fail("focus arrived with no preceding pointerdown");
|
||||
else if (lastFocus < lastPointer) Challenge.fail("focus before pointerdown");
|
||||
});
|
||||
t.addEventListener("input", () => {
|
||||
if (t.value !== "x") return;
|
||||
// For pointer-driven focus, :focus-visible should be false.
|
||||
const fv = t.matches(":focus-visible");
|
||||
Challenge.log("focus-visible", { fv });
|
||||
if (fv) Challenge.fail(":focus-visible true after pointer click (looks like keyboard focus)");
|
||||
else if (window.__verdict !== "FAIL") Challenge.pass("pointerdown→focus→input order ok, :focus-visible=false");
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,28 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>14 wheel scroll</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: scroll the box to the bottom. Page rejects raw <code>scrollTop</code> assignment without
|
||||
<code>wheel</code> events.</p>
|
||||
<div id="box" style="height:200px;overflow:auto;border:1px solid #555;background:#fff;color:#111;padding:8px">
|
||||
<div style="height:1200px">scroll me ⬇<br><br>...lots of content...</div>
|
||||
</div>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "wheel-scroll", instructions: "scroll the box to its bottom" });
|
||||
const box = document.getElementById("box");
|
||||
let wheelCount = 0, lastWheelTs = -Infinity;
|
||||
box.addEventListener("wheel", (e) => { wheelCount++; lastWheelTs = e.timeStamp; Challenge.log("wheel", { dy: e.deltaY }); }, { passive: true });
|
||||
box.addEventListener("scroll", () => {
|
||||
Challenge.log("scroll", { top: box.scrollTop });
|
||||
if (box.scrollTop + box.clientHeight >= box.scrollHeight - 2) {
|
||||
if (wheelCount === 0) Challenge.fail("scrolled to bottom with zero wheel events");
|
||||
else if (performance.now() - lastWheelTs > 1500) Challenge.fail("wheel events too far before final scroll");
|
||||
else Challenge.pass(`${wheelCount} wheel events accompanied scroll`);
|
||||
}
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,73 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>15 drag-drop DataTransfer</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: drag the <b>red box</b> into the <b>drop zone</b>. Page asserts a full HTML5
|
||||
drag sequence with a populated <code>DataTransfer</code> — synthetic pointer drags
|
||||
dispatched without <code>dragstart</code>/<code>drop</code> + DataTransfer payload fail.</p>
|
||||
<div style="display:flex;gap:24px;align-items:center;margin-top:16px">
|
||||
<div id="src" draggable="true"
|
||||
style="width:80px;height:80px;background:#c33;color:#fff;display:flex;align-items:center;justify-content:center;border-radius:6px;cursor:grab">
|
||||
DRAG
|
||||
</div>
|
||||
<div id="dst"
|
||||
style="width:200px;height:120px;border:2px dashed #888;display:flex;align-items:center;justify-content:center;color:#888">
|
||||
DROP HERE
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "drag-drop-datatransfer", instructions: "drag red box into drop zone via real HTML5 drag" });
|
||||
const src = document.getElementById("src");
|
||||
const dst = document.getElementById("dst");
|
||||
|
||||
const seen = { dragstart: 0, drag: 0, dragenter: 0, dragover: 0, drop: 0, dragend: 0 };
|
||||
let dtHadTypes = false, dtPayload = null, isTrustedAll = true;
|
||||
|
||||
src.addEventListener("dragstart", (e) => {
|
||||
seen.dragstart++;
|
||||
if (!e.isTrusted) isTrustedAll = false;
|
||||
try { e.dataTransfer.setData("text/plain", "payload-" + Math.random().toString(36).slice(2,8)); } catch {}
|
||||
Challenge.log("dragstart", { hasDt: !!e.dataTransfer });
|
||||
});
|
||||
src.addEventListener("drag", (e) => { seen.drag++; if (!e.isTrusted) isTrustedAll = false; });
|
||||
src.addEventListener("dragend", (e) => { seen.dragend++; if (!e.isTrusted) isTrustedAll = false; });
|
||||
|
||||
dst.addEventListener("dragenter", (e) => { seen.dragenter++; e.preventDefault(); if (!e.isTrusted) isTrustedAll = false; });
|
||||
dst.addEventListener("dragover", (e) => { seen.dragover++; e.preventDefault(); if (!e.isTrusted) isTrustedAll = false; });
|
||||
dst.addEventListener("drop", (e) => {
|
||||
e.preventDefault();
|
||||
seen.drop++;
|
||||
if (!e.isTrusted) isTrustedAll = false;
|
||||
const dt = e.dataTransfer;
|
||||
if (dt) {
|
||||
dtHadTypes = (dt.types && dt.types.length > 0);
|
||||
try { dtPayload = dt.getData("text/plain"); } catch {}
|
||||
}
|
||||
Challenge.log("drop", { types: dt && [...dt.types], payload: dtPayload });
|
||||
// dragend fires AFTER drop per spec; give it a tick.
|
||||
setTimeout(evaluate, 50);
|
||||
});
|
||||
|
||||
function evaluate() {
|
||||
const bad = [];
|
||||
if (!isTrustedAll) bad.push("at least one drag event isTrusted=false");
|
||||
for (const k of ["dragstart","dragover","drop","dragend"]) {
|
||||
if (!seen[k]) bad.push(`missing ${k}`);
|
||||
}
|
||||
if (!dtHadTypes) bad.push("DataTransfer.types empty on drop");
|
||||
if (!dtPayload || !dtPayload.startsWith("payload-")) bad.push("DataTransfer payload missing on drop");
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass(`full drag cycle, payload=${dtPayload}`);
|
||||
}
|
||||
|
||||
// Fallback: pointer-drag without dragstart triggers fail after a delay.
|
||||
let pointerDownInSrc = false;
|
||||
src.addEventListener("pointerdown", () => { pointerDownInSrc = true;
|
||||
setTimeout(() => { if (!seen.dragstart && pointerDownInSrc) Challenge.fail("pointerdown on src but no dragstart fired"); }, 1500);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,54 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>16 contenteditable selection</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: focus the editable region and type <code>hello</code>. Page asserts that
|
||||
<code>window.getSelection()</code> reflects per-keystroke caret movement
|
||||
(<code>rangeCount===1</code>, <code>collapsed</code> caret, anchor inside the editor,
|
||||
offset advances 1 per char) and that <code>selectionchange</code> fires.</p>
|
||||
<div id="ed" contenteditable="true"
|
||||
style="min-height:60px;padding:10px;border:1px solid #555;background:#fff;color:#111;font:16px monospace"></div>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "contenteditable-selection", instructions: "focus editor and type 'hello'" });
|
||||
const ed = document.getElementById("ed");
|
||||
const offsets = [];
|
||||
let selChanges = 0;
|
||||
|
||||
document.addEventListener("selectionchange", () => {
|
||||
selChanges++;
|
||||
const s = window.getSelection();
|
||||
if (!s || s.rangeCount === 0) return;
|
||||
if (!ed.contains(s.anchorNode)) return;
|
||||
offsets.push({ off: s.anchorOffset, collapsed: s.isCollapsed, t: performance.now(), len: ed.textContent.length });
|
||||
});
|
||||
|
||||
ed.addEventListener("input", async () => {
|
||||
if (ed.textContent !== "hello") return;
|
||||
// selectionchange is async — wait a tick so the final caret update is observed.
|
||||
await new Promise((r) => setTimeout(r, 30));
|
||||
const bad = [];
|
||||
const sel = window.getSelection();
|
||||
if (!sel) bad.push("getSelection()==null");
|
||||
else {
|
||||
if (sel.rangeCount !== 1) bad.push(`rangeCount=${sel.rangeCount} (need 1)`);
|
||||
if (!sel.isCollapsed) bad.push("selection not collapsed after typing");
|
||||
if (!ed.contains(sel.anchorNode)) bad.push("selection anchor outside editor");
|
||||
if (sel.anchorOffset !== 5) bad.push(`anchorOffset=${sel.anchorOffset} (need 5)`);
|
||||
}
|
||||
if (selChanges < 5) bad.push(`selectionchange count=${selChanges} (need ≥5)`);
|
||||
// offsets should advance monotonically 1,2,3,4,5 (allowing extras from focus)
|
||||
const monotonic = offsets.map(o => o.off);
|
||||
const peak = Math.max(0, ...monotonic);
|
||||
if (peak !== 5) bad.push(`peak caret offset=${peak} during typing (need 5)`);
|
||||
// every caret reading must be collapsed (no spurious ranges)
|
||||
if (offsets.some(o => !o.collapsed)) bad.push("non-collapsed range observed during typing");
|
||||
Challenge.log("sel", { offsets, selChanges });
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass(`selectionchanges=${selChanges}, caret advanced 0→5`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,48 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>17 paste clipboard</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: paste the string <code>pi-chrome</code> into the input via a real OS paste
|
||||
(Cmd/Ctrl+V firing a trusted <code>paste</code> event with populated
|
||||
<code>clipboardData</code>). Programmatic <code>value=</code> or <code>setRangeText</code>
|
||||
fail. The <code>input</code> event's <code>inputType</code> must be
|
||||
<code>insertFromPaste</code>.</p>
|
||||
<input id="t" placeholder="paste here" style="font-size:18px;padding:8px 12px;width:280px">
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "paste-clipboard", instructions: "paste 'pi-chrome' via OS clipboard" });
|
||||
const t = document.getElementById("t");
|
||||
let sawPaste = false, pasteIsTrusted = false, pastePayload = null;
|
||||
t.addEventListener("paste", (e) => {
|
||||
sawPaste = true;
|
||||
pasteIsTrusted = e.isTrusted;
|
||||
try { pastePayload = e.clipboardData && e.clipboardData.getData("text/plain"); } catch {}
|
||||
Challenge.log("paste", { isTrusted: e.isTrusted, payload: pastePayload });
|
||||
});
|
||||
|
||||
let lastInputType = null;
|
||||
t.addEventListener("input", (e) => {
|
||||
lastInputType = e.inputType;
|
||||
if (t.value !== "pi-chrome") return;
|
||||
const bad = [];
|
||||
if (!sawPaste) bad.push("no paste event fired before value matched");
|
||||
if (!pasteIsTrusted) bad.push("paste event isTrusted=false");
|
||||
if (pastePayload !== "pi-chrome") bad.push(`clipboardData payload="${pastePayload}" (need 'pi-chrome')`);
|
||||
if (lastInputType !== "insertFromPaste") bad.push(`input.inputType="${lastInputType}" (need 'insertFromPaste')`);
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass("trusted paste with clipboardData text/plain match");
|
||||
});
|
||||
|
||||
// Also fail loudly if value is set without any input event (raw .value=)
|
||||
let inputFired = false;
|
||||
t.addEventListener("input", () => { inputFired = true; });
|
||||
const obs = new MutationObserver(() => {});
|
||||
obs.observe(t, { attributes: true, attributeFilter: ["value"] });
|
||||
setInterval(() => {
|
||||
if (t.value === "pi-chrome" && !inputFired) Challenge.fail("value matched without input event firing");
|
||||
}, 200);
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,56 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>18 native select option</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: select the option <code>banana</code> from the native <code><select></code>.
|
||||
Page asserts a real picker interaction: <code>mousedown</code>/<code>pointerdown</code>
|
||||
on the <select> (opening the OS picker), then a <code>change</code> event with
|
||||
<code>isTrusted=true</code>. Programmatically setting <code>.value</code> or clicking the
|
||||
option element directly fails (clicking <option> does not change selection in
|
||||
real browsers).</p>
|
||||
<select id="s" style="font-size:18px;padding:6px 10px">
|
||||
<option value="">— pick a fruit —</option>
|
||||
<option value="apple">apple</option>
|
||||
<option value="banana">banana</option>
|
||||
<option value="cherry">cherry</option>
|
||||
</select>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "native-select", instructions: "pick 'banana' from native select" });
|
||||
const s = document.getElementById("s");
|
||||
|
||||
let sawPointerDownOnSelect = false, pointerDownTrusted = false;
|
||||
s.addEventListener("pointerdown", (e) => { sawPointerDownOnSelect = true; pointerDownTrusted = e.isTrusted; });
|
||||
s.addEventListener("mousedown", (e) => { sawPointerDownOnSelect = true; pointerDownTrusted ||= e.isTrusted; });
|
||||
|
||||
// Trap: if a click bubbles up from an <option>, that's bot-like (real OS pickers don't dispatch this).
|
||||
s.addEventListener("click", (e) => {
|
||||
if (e.target && e.target.tagName === "OPTION") {
|
||||
Challenge.fail("click event with target=<option> bubbled to <select> (synthetic option click)");
|
||||
}
|
||||
});
|
||||
|
||||
let changeTrusted = false;
|
||||
s.addEventListener("change", (e) => {
|
||||
changeTrusted = e.isTrusted;
|
||||
Challenge.log("change", { v: s.value, isTrusted: e.isTrusted });
|
||||
if (s.value !== "banana") return Challenge.fail(`selected="${s.value}" (need 'banana')`);
|
||||
const bad = [];
|
||||
if (!changeTrusted) bad.push("change event isTrusted=false");
|
||||
if (!sawPointerDownOnSelect) bad.push("no pointerdown/mousedown on <select> before change");
|
||||
if (!pointerDownTrusted) bad.push("pointerdown on select isTrusted=false");
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass("native picker opened and trusted change fired");
|
||||
});
|
||||
|
||||
// Also trip if value mutates without change firing.
|
||||
let changed = false;
|
||||
s.addEventListener("change", () => { changed = true; });
|
||||
setInterval(() => {
|
||||
if (s.value === "banana" && !changed) Challenge.fail("value='banana' without any change event");
|
||||
}, 200);
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,50 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>19 hover dwell</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: hover the trigger and wait until the <b>Confirm</b> button reveals (≥600ms dwell
|
||||
with continuous <code>pointermove</code> activity inside the trigger), then click Confirm.
|
||||
Instant hover-then-click without dwell, or hover with no intermediate pointermove,
|
||||
looks robotic and fails.</p>
|
||||
<div id="trig" style="display:inline-block;padding:16px 24px;background:#2a4;color:#fff;border-radius:6px;cursor:pointer">
|
||||
hover me
|
||||
</div>
|
||||
<button id="go" style="display:none;margin-left:12px;padding:10px 18px;background:#1f7a1f;color:#fff;border:0;border-radius:6px">Confirm</button>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "hover-dwell", instructions: "hover ≥600ms with movement, then click Confirm" });
|
||||
const trig = document.getElementById("trig");
|
||||
const btn = document.getElementById("go");
|
||||
|
||||
let enterTs = 0, lastMoveTs = 0, moveCount = 0;
|
||||
trig.addEventListener("pointerenter", (e) => { enterTs = e.timeStamp; lastMoveTs = e.timeStamp; moveCount = 0; });
|
||||
trig.addEventListener("pointermove", (e) => { lastMoveTs = e.timeStamp; moveCount++; });
|
||||
trig.addEventListener("pointerleave", () => {
|
||||
if (!btn.dataset.revealed) { enterTs = 0; lastMoveTs = 0; moveCount = 0; }
|
||||
});
|
||||
|
||||
// Reveal after sustained dwell + activity.
|
||||
setInterval(() => {
|
||||
if (btn.dataset.revealed) return;
|
||||
if (!enterTs) return;
|
||||
const dwell = performance.now() - enterTs;
|
||||
if (dwell >= 600 && moveCount >= 3) {
|
||||
btn.style.display = "inline-block";
|
||||
btn.dataset.revealed = "1";
|
||||
btn.dataset.revealTs = performance.now();
|
||||
Challenge.log("revealed", { dwell, moveCount });
|
||||
}
|
||||
}, 50);
|
||||
|
||||
btn.addEventListener("click", (e) => {
|
||||
if (!btn.dataset.revealed) return Challenge.fail("Confirm clicked before reveal (display:none)");
|
||||
const sinceReveal = e.timeStamp - Number(btn.dataset.revealTs);
|
||||
if (sinceReveal < 80) return Challenge.fail(`clicked ${sinceReveal.toFixed(0)}ms after reveal (too instant)`);
|
||||
if (!e.isTrusted) return Challenge.fail("Confirm click isTrusted=false");
|
||||
Challenge.pass(`dwell+activity+human-latency reveal (Δ=${sinceReveal.toFixed(0)}ms)`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,78 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>20 React _valueTracker</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: set the input to <code>react-ok</code> in a way that React would accept.
|
||||
The page mimics React's <code>_valueTracker</code>: it stamps the input with a
|
||||
hidden tracker holding the last-seen value. An <code>input</code> event is only
|
||||
considered "framework-real" if the tracker's recorded value differs from the
|
||||
current <code>input.value</code> at the moment the event fires
|
||||
(i.e. somebody used the native <code>HTMLInputElement.value</code> setter and
|
||||
dispatched <code>input</code> — what React internals require).</p>
|
||||
<p>Setting <code>el.value = ...</code> directly and dispatching a synthetic <code>input</code>
|
||||
fails because the native setter wasn't used. Calling <code>setRangeText</code> without
|
||||
dispatching <code>input</code> also fails.</p>
|
||||
<input id="t" placeholder="type or fill 'react-ok'" style="font-size:18px;padding:8px 12px;width:240px">
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "react-value-tracker", instructions: "make input value 'react-ok' as React expects" });
|
||||
|
||||
const t = document.getElementById("t");
|
||||
|
||||
// Install a fake React-style _valueTracker.
|
||||
(function attachTracker(node) {
|
||||
const nativeProto = Object.getPrototypeOf(node);
|
||||
const desc = Object.getOwnPropertyDescriptor(nativeProto, "value");
|
||||
const nativeGet = desc.get, nativeSet = desc.set;
|
||||
let tracked = nativeGet.call(node); // last value the "framework" has seen
|
||||
node._valueTracker = {
|
||||
getValue() { return tracked; },
|
||||
setValue(v) { tracked = v; },
|
||||
};
|
||||
// Wrap the value property on the *instance* to intercept assignments.
|
||||
Object.defineProperty(node, "value", {
|
||||
configurable: true,
|
||||
get() { return nativeGet.call(this); },
|
||||
set(v) {
|
||||
// If somebody assigns via instance property (the bad path), sync the tracker
|
||||
// so the input event below will look "stale" and we can detect it.
|
||||
tracked = String(v);
|
||||
nativeSet.call(this, v);
|
||||
},
|
||||
});
|
||||
})(t);
|
||||
|
||||
let passed = false;
|
||||
t.addEventListener("input", (e) => {
|
||||
if (passed) return;
|
||||
const proto = Object.getPrototypeOf(t);
|
||||
const nativeGet = Object.getOwnPropertyDescriptor(proto, "value").get;
|
||||
const real = nativeGet.call(t);
|
||||
const tracked = t._valueTracker.getValue();
|
||||
Challenge.log("input", { real, tracked, isTrusted: e.isTrusted });
|
||||
if (real !== "react-ok") return;
|
||||
if (!e.isTrusted && tracked === real) {
|
||||
return Challenge.fail("synthetic input event but _valueTracker already up-to-date (instance value= setter used, not native)");
|
||||
}
|
||||
// The "good" path: native setter ran (tracker still stale), then input dispatched.
|
||||
// After the framework consumes the event it would syncs the tracker.
|
||||
t._valueTracker.setValue(real);
|
||||
passed = true;
|
||||
Challenge.pass(`value 'react-ok' delivered with stale tracker (native setter path); isTrusted=${e.isTrusted}`);
|
||||
});
|
||||
|
||||
// Trap: value mutates without input ever firing.
|
||||
setInterval(() => {
|
||||
if (passed) return;
|
||||
const proto = Object.getPrototypeOf(t);
|
||||
const nativeGet = Object.getOwnPropertyDescriptor(proto, "value").get;
|
||||
if (nativeGet.call(t) === "react-ok" && !passed) {
|
||||
// Only fail if we've been sitting on the value with no event for a while.
|
||||
setTimeout(() => { if (!passed && nativeGet.call(t) === "react-ok") Challenge.fail("value='react-ok' but no input event consumed it"); }, 600);
|
||||
}
|
||||
}, 300);
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,65 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>21 keyboard modifiers</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: focus the input and type a capital <code>A</code> via <b>Shift+a</b>
|
||||
(not Caps Lock, not pasted, not assigned). Page asserts a real Shift chord:
|
||||
<code>keydown</code> for <code>Shift</code> arrives <em>before</em> <code>keydown</code> for
|
||||
<code>A</code>; the <code>A</code> event has <code>shiftKey=true</code>,
|
||||
<code>key==="A"</code>, <code>code==="KeyA"</code>, <code>getModifierState("Shift")===true</code>;
|
||||
and <code>keyup</code> for Shift arrives <em>after</em> <code>keyup</code> for <code>A</code>.</p>
|
||||
<input id="t" placeholder="type Shift+a" style="font-size:18px;padding:8px 12px;width:240px">
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "keyboard-modifiers", instructions: "type capital A using Shift+a chord" });
|
||||
const t = document.getElementById("t");
|
||||
|
||||
const log = [];
|
||||
for (const name of ["keydown","keypress","keyup","input"]) {
|
||||
t.addEventListener(name, (e) => {
|
||||
log.push({
|
||||
name,
|
||||
key: e.key, code: e.code,
|
||||
shiftKey: e.shiftKey, isTrusted: e.isTrusted,
|
||||
modShift: e.getModifierState ? e.getModifierState("Shift") : null,
|
||||
t: e.timeStamp,
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
t.addEventListener("input", () => {
|
||||
if (t.value !== "A") return;
|
||||
const bad = [];
|
||||
const downShift = log.find(e => e.name === "keydown" && e.key === "Shift");
|
||||
const downA = log.find(e => e.name === "keydown" && e.code === "KeyA");
|
||||
const upShift = log.find(e => e.name === "keyup" && e.key === "Shift");
|
||||
const upA = log.find(e => e.name === "keyup" && e.code === "KeyA");
|
||||
|
||||
if (!downShift) bad.push("no keydown for Shift");
|
||||
if (!downA) bad.push("no keydown for KeyA");
|
||||
if (!upShift) bad.push("no keyup for Shift");
|
||||
if (!upA) bad.push("no keyup for KeyA");
|
||||
|
||||
if (downShift && downA && downShift.t > downA.t) bad.push("Shift keydown after A keydown");
|
||||
if (upShift && upA && upShift.t < upA.t) bad.push("Shift keyup before A keyup");
|
||||
|
||||
if (downA) {
|
||||
if (downA.key !== "A") bad.push(`A keydown.key="${downA.key}" (need 'A')`);
|
||||
if (downA.code !== "KeyA") bad.push(`A keydown.code="${downA.code}"`);
|
||||
if (!downA.shiftKey) bad.push("A keydown.shiftKey=false");
|
||||
if (downA.modShift !== true) bad.push("getModifierState('Shift')!==true on A keydown");
|
||||
if (!downA.isTrusted) bad.push("A keydown isTrusted=false");
|
||||
}
|
||||
if (downShift) {
|
||||
if (!downShift.isTrusted) bad.push("Shift keydown isTrusted=false");
|
||||
}
|
||||
|
||||
Challenge.log("chord", { log });
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass("Shift+a chord, modifiers + ordering correct");
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,66 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>22 touch events</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: tap the green pad on a touch-capable device (or via DevTools touch
|
||||
emulation). Page asserts a real <code>touchstart</code>/<code>touchend</code> sequence:
|
||||
<code>touches</code> / <code>changedTouches</code> are
|
||||
<code>TouchList</code> objects containing <code>Touch</code> instances with
|
||||
<code>identifier</code>, <code>clientX/Y</code>, <code>radiusX/Y</code>, and
|
||||
<code>force</code> in [0,1]. Synthetic <code>PointerEvent</code> with
|
||||
<code>pointerType="touch"</code> alone does not satisfy this — touch events must
|
||||
fire too.</p>
|
||||
<p><b>Note for caller:</b> requires Chrome touch emulation on (DevTools → Sensors →
|
||||
"Touch: Force enabled") or a real touchscreen. This is intentional: pi-chrome's
|
||||
synthetic <code>pointerType:'touch'</code> does NOT also dispatch TouchEvents.</p>
|
||||
<div id="pad" style="width:200px;height:200px;background:#1f7a1f;border-radius:12px;color:#fff;display:flex;align-items:center;justify-content:center;font:18px monospace;touch-action:none;user-select:none">
|
||||
TAP
|
||||
</div>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "touch-events", instructions: "tap the pad on a touch-capable device" });
|
||||
const pad = document.getElementById("pad");
|
||||
|
||||
let touchStart = null, touchEnd = null;
|
||||
pad.addEventListener("touchstart", (e) => {
|
||||
touchStart = e;
|
||||
Challenge.log("touchstart", { tl: e.touches.length, ct: e.changedTouches.length, isTrusted: e.isTrusted });
|
||||
}, { passive: true });
|
||||
pad.addEventListener("touchend", (e) => {
|
||||
touchEnd = e;
|
||||
const bad = [];
|
||||
if (!touchStart) return Challenge.fail("touchend without touchstart");
|
||||
if (!e.isTrusted || !touchStart.isTrusted) bad.push("touch event isTrusted=false");
|
||||
|
||||
const ts = touchStart.changedTouches;
|
||||
if (!ts || ts.length === 0) bad.push("touchstart.changedTouches empty");
|
||||
else {
|
||||
const T = ts[0];
|
||||
// Window.Touch should exist as the constructor.
|
||||
if (typeof window.Touch !== "function") bad.push("window.Touch constructor missing (no native TouchEvent support)");
|
||||
if (!(T instanceof Touch)) bad.push("changedTouches[0] not instanceof Touch");
|
||||
if (typeof T.identifier !== "number") bad.push("Touch.identifier not number");
|
||||
if (!Number.isFinite(T.clientX) || !Number.isFinite(T.clientY)) bad.push("Touch.clientX/Y invalid");
|
||||
if (!("force" in T) || typeof T.force !== "number" || T.force < 0 || T.force > 1) bad.push(`Touch.force=${T.force}`);
|
||||
if (!("radiusX" in T) || typeof T.radiusX !== "number") bad.push("Touch.radiusX missing");
|
||||
}
|
||||
|
||||
if (!(touchStart instanceof TouchEvent)) bad.push("touchstart not instanceof TouchEvent");
|
||||
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass("real TouchEvent + Touch object surface present");
|
||||
}, { passive: true });
|
||||
|
||||
// Capability check: if TouchEvent constructor missing entirely, hint immediately on click.
|
||||
pad.addEventListener("click", () => {
|
||||
if (!touchStart && typeof window.TouchEvent !== "function") {
|
||||
Challenge.fail("TouchEvent constructor not available on this UA (enable touch emulation)");
|
||||
} else if (!touchStart) {
|
||||
Challenge.fail("click fired but no touchstart — synthetic pointer without TouchEvent");
|
||||
}
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,76 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>23 stack-trace fingerprint</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: click the button. The page samples call stacks from inside several
|
||||
instrumented globals (<code>Function.prototype.toString</code>,
|
||||
<code>document.querySelector</code>, <code>Element.prototype.click</code>) and inspects
|
||||
the stack of <em>this script's own</em> handler. It fails if it sees telltales of
|
||||
evaluator-injected frames (e.g. <code>at <anonymous></code> as the only frame, the
|
||||
bridge's <code>new Function</code> wrapper, <code>callFunctionOn</code>,
|
||||
<code>executeScript</code>, or extension URLs).</p>
|
||||
<button id="go" style="padding:14px 22px;font-size:16px;background:#1f7a1f;color:#fff;border:0;border-radius:6px">Click me</button>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "stack-trace-fingerprint", instructions: "click the button" });
|
||||
|
||||
// Suspicious stack-frame patterns commonly observed when code is invoked via
|
||||
// CDP Runtime.evaluate / chrome.scripting.executeScript / new Function bodies
|
||||
// dispatched from an MV3 service worker.
|
||||
const SUSPICIOUS = [
|
||||
/chrome-extension:\/\//i,
|
||||
/\bnew Function\b/,
|
||||
/Runtime\.evaluate/i,
|
||||
/Runtime\.callFunctionOn/i,
|
||||
/executeScript/i,
|
||||
/content[_-]?script/i,
|
||||
/^\s*at\s+eval\b/m,
|
||||
];
|
||||
|
||||
function inspectStack(stack) {
|
||||
if (!stack) return ["empty stack"];
|
||||
const hits = SUSPICIOUS.filter(r => r.test(stack)).map(r => r.source);
|
||||
const lines = stack.split("\n").filter(l => l.trim().startsWith("at "));
|
||||
// Do not fail on generic <anonymous> frames alone: inline scripts, extensions,
|
||||
// and browser versions vary here. This test should catch concrete automation
|
||||
// tells, not punish legitimate stack formatting differences.
|
||||
const reasons = [];
|
||||
if (hits.length) reasons.push("suspicious frames: " + hits.join(","));
|
||||
return reasons;
|
||||
}
|
||||
|
||||
// Hook some commonly-touched APIs so any pre-click bridge instrumentation also
|
||||
// leaves a trail. Their stacks get inspected the moment the click handler fires.
|
||||
const probeStacks = [];
|
||||
const oTo = Function.prototype.toString;
|
||||
Function.prototype.toString = function () {
|
||||
probeStacks.push({ where: "Function.toString", stack: new Error().stack });
|
||||
return oTo.apply(this, arguments);
|
||||
};
|
||||
const oQS = Document.prototype.querySelector;
|
||||
Document.prototype.querySelector = function (sel) {
|
||||
probeStacks.push({ where: "document.querySelector", sel, stack: new Error().stack });
|
||||
return oQS.apply(this, arguments);
|
||||
};
|
||||
|
||||
document.getElementById("go").addEventListener("click", (e) => {
|
||||
const ownStack = new Error().stack || "";
|
||||
const reasons = inspectStack(ownStack);
|
||||
// Also check any probe stacks gathered before the click — the bridge often
|
||||
// queries the DOM right before dispatching.
|
||||
const probeBad = [];
|
||||
for (const p of probeStacks) {
|
||||
const r = inspectStack(p.stack);
|
||||
if (r.length) probeBad.push(`${p.where}: ${r.join("; ")}`);
|
||||
}
|
||||
Challenge.log("stacks", { ownStack, probeStacks });
|
||||
if (!e.isTrusted) return Challenge.fail("click isTrusted=false");
|
||||
if (reasons.length) return Challenge.fail(...reasons);
|
||||
if (probeBad.length) return Challenge.fail(...probeBad.slice(0, 3));
|
||||
Challenge.pass("call stack matches an in-page event handler");
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,51 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>24 viewport edge clicks</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: click the button. Page rejects any pointer event whose
|
||||
<code>clientX</code>/<code>clientY</code> falls outside the viewport
|
||||
(<code>x < 0</code>, <code>x > innerWidth</code>, etc.) or lands on a coordinate
|
||||
that's not actually inside the button's <code>getBoundingClientRect()</code>.
|
||||
Real pointing devices can't dispatch clicks at negative coordinates;
|
||||
bridges that pass uncoerced selector-center coords (or coordinates from a
|
||||
stale layout) trip this.</p>
|
||||
<button id="go" style="margin:40px;padding:14px 22px;font-size:16px;background:#1f7a1f;color:#fff;border:0;border-radius:6px">Click me</button>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "viewport-edge-clicks", instructions: "click the button at valid viewport coords" });
|
||||
|
||||
const btn = document.getElementById("go");
|
||||
|
||||
// Track ALL pointer/click events on document — even ones that miss the button.
|
||||
let suspicious = null;
|
||||
for (const name of ["pointerdown","pointerup","click","mousedown","mouseup"]) {
|
||||
document.addEventListener(name, (e) => {
|
||||
const x = e.clientX, y = e.clientY;
|
||||
const w = window.innerWidth, h = window.innerHeight;
|
||||
if (x < 0 || y < 0 || x > w || y > h) {
|
||||
suspicious = `${name} at (${x},${y}) outside viewport ${w}x${h}`;
|
||||
Challenge.fail(suspicious);
|
||||
}
|
||||
if ((x === 0 && y === 0) && e.isTrusted === false) {
|
||||
suspicious = `${name} at (0,0) with isTrusted=false (default synthetic coords)`;
|
||||
Challenge.fail(suspicious);
|
||||
}
|
||||
}, true);
|
||||
}
|
||||
|
||||
btn.addEventListener("click", (e) => {
|
||||
if (suspicious) return;
|
||||
if (!e.isTrusted) return Challenge.fail("click isTrusted=false");
|
||||
const r = btn.getBoundingClientRect();
|
||||
const x = e.clientX, y = e.clientY;
|
||||
const inside = x >= r.left && x <= r.right && y >= r.top && y <= r.bottom;
|
||||
if (!inside) return Challenge.fail(`click at (${x},${y}) outside button rect ${JSON.stringify({l:r.left,t:r.top,r:r.right,b:r.bottom})}`);
|
||||
// Coord must not be exactly integer-center (covered by #06 but reinforced here).
|
||||
Challenge.log("click", { x, y, rect: r });
|
||||
Challenge.pass(`click at (${x.toFixed(1)},${y.toFixed(1)}) inside button rect`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,62 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>25 pointer continuity</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: click <b>A</b>, then click <b>B</b>. Page requires pointer continuity:
|
||||
between the two clicks there must be intermediate <code>pointermove</code> events
|
||||
whose path roughly connects the two click points (no teleport). Two clicks
|
||||
>100px apart with zero moves between is a synthetic-bridge tell.</p>
|
||||
<div style="display:flex;justify-content:space-between;margin-top:20px">
|
||||
<button id="a" style="padding:14px 22px;background:#36c;color:#fff;border:0;border-radius:6px">A</button>
|
||||
<div style="flex:1"></div>
|
||||
<button id="b" style="padding:14px 22px;background:#c36;color:#fff;border:0;border-radius:6px">B</button>
|
||||
</div>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "pointer-continuity", instructions: "click A, move mouse to B, click B" });
|
||||
|
||||
const a = document.getElementById("a");
|
||||
const b = document.getElementById("b");
|
||||
|
||||
const moves = [];
|
||||
window.addEventListener("pointermove", (e) => {
|
||||
moves.push({ x: e.clientX, y: e.clientY, t: e.timeStamp });
|
||||
if (moves.length > 500) moves.shift();
|
||||
});
|
||||
|
||||
let aClickTs = 0, aPoint = null;
|
||||
a.addEventListener("click", (e) => {
|
||||
if (!e.isTrusted) return Challenge.fail("A click isTrusted=false");
|
||||
aClickTs = e.timeStamp;
|
||||
aPoint = { x: e.clientX, y: e.clientY };
|
||||
Challenge.log("A click", aPoint);
|
||||
});
|
||||
|
||||
b.addEventListener("click", (e) => {
|
||||
if (!aClickTs) return Challenge.fail("B clicked before A");
|
||||
if (!e.isTrusted) return Challenge.fail("B click isTrusted=false");
|
||||
const bPoint = { x: e.clientX, y: e.clientY };
|
||||
const between = moves.filter(m => m.t > aClickTs && m.t < e.timeStamp);
|
||||
Challenge.log("B click", { bPoint, betweenCount: between.length });
|
||||
const dist = Math.hypot(bPoint.x - aPoint.x, bPoint.y - aPoint.y);
|
||||
if (dist < 100) return Challenge.fail(`A-B distance only ${dist.toFixed(0)}px — buttons should be further apart in viewport`);
|
||||
if (between.length < 5) return Challenge.fail(`only ${between.length} pointermove(s) between A and B (need ≥5)`);
|
||||
|
||||
// Path coverage: at least some intermediate move must be ≥30% of the way across.
|
||||
const inSpan = between.filter(m => {
|
||||
const px = (m.x - aPoint.x) / (bPoint.x - aPoint.x || 1);
|
||||
return px > 0.25 && px < 0.75;
|
||||
});
|
||||
if (inSpan.length === 0) return Challenge.fail("no pointermove samples in mid-span between A and B (teleport)");
|
||||
|
||||
// Move timestamps should be spread, not bunched in <20ms.
|
||||
const span = between[between.length-1].t - between[0].t;
|
||||
if (span < 30) return Challenge.fail(`pointermoves bunched in ${span.toFixed(0)}ms (need spread ≥30ms)`);
|
||||
|
||||
Challenge.pass(`${between.length} moves bridging A→B over ${span.toFixed(0)}ms, dist=${dist.toFixed(0)}px`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,57 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>26 mousemove rate</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: sweep the mouse across the box for ~1s, then click the green button.
|
||||
Page measures the inter-event timestamp distribution of <code>mousemove</code>
|
||||
events. Real pointing devices fire at ~60–250 Hz (median Δt 4–18ms). Bridges
|
||||
that flood synthetic moves on a tight <code>for</code>-loop or
|
||||
<code>setTimeout(...,1)</code> produce a degenerate distribution
|
||||
(most Δt < 2ms, or all Δt identical, or median > 50ms with no jitter).</p>
|
||||
<div id="pad" style="height:160px;border:1px solid #555;background:#fff;color:#111;display:flex;align-items:center;justify-content:center;font:13px monospace;user-select:none">
|
||||
sweep me
|
||||
</div>
|
||||
<button id="go" style="margin-top:14px;padding:12px 20px;background:#1f7a1f;color:#fff;border:0;border-radius:6px">Done</button>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "mousemove-rate", instructions: "sweep mouse across pad ~1s, then click Done" });
|
||||
|
||||
const pad = document.getElementById("pad");
|
||||
const stamps = [];
|
||||
pad.addEventListener("mousemove", (e) => { stamps.push(e.timeStamp); });
|
||||
|
||||
document.getElementById("go").addEventListener("click", (e) => {
|
||||
if (!e.isTrusted) return Challenge.fail("Done click isTrusted=false");
|
||||
if (stamps.length < 20) return Challenge.fail(`only ${stamps.length} mousemove events (need ≥20)`);
|
||||
|
||||
const deltas = [];
|
||||
for (let i = 1; i < stamps.length; i++) deltas.push(stamps[i] - stamps[i-1]);
|
||||
deltas.sort((a,b) => a-b);
|
||||
const median = deltas[Math.floor(deltas.length/2)];
|
||||
const p10 = deltas[Math.floor(deltas.length*0.1)];
|
||||
const p90 = deltas[Math.floor(deltas.length*0.9)];
|
||||
const mean = deltas.reduce((s,x)=>s+x,0) / deltas.length;
|
||||
const variance = deltas.reduce((s,x)=>s+(x-mean)**2,0) / deltas.length;
|
||||
const std = Math.sqrt(variance);
|
||||
|
||||
const allSame = deltas.every(d => Math.abs(d - deltas[0]) < 0.05);
|
||||
const allTiny = deltas.every(d => d < 1.5);
|
||||
|
||||
Challenge.log("dist", { n: deltas.length, median, p10, p90, mean, std });
|
||||
const bad = [];
|
||||
if (allSame) bad.push(`all Δt identical (~${deltas[0].toFixed(2)}ms) — scripted loop`);
|
||||
if (allTiny) bad.push(`all Δt < 1.5ms — synthetic flood`);
|
||||
if (median < 2) bad.push(`median Δt ${median.toFixed(2)}ms too fast for a real device`);
|
||||
if (median > 60) bad.push(`median Δt ${median.toFixed(0)}ms too slow (real moves ~4–18ms)`);
|
||||
if (std < 0.5) bad.push(`Δt std=${std.toFixed(2)}ms — no jitter (scripted)`);
|
||||
// Ratio p90/p10 should be ≥1.8 for organic motion.
|
||||
if (p10 > 0 && (p90 / p10) < 1.5) bad.push(`p90/p10 ratio ${(p90/p10).toFixed(2)} — distribution too narrow`);
|
||||
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass(`n=${deltas.length}, median=${median.toFixed(1)}ms, std=${std.toFixed(1)}ms`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,66 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>27 scroll momentum tail</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: scroll the box to its bottom using a real wheel/trackpad gesture.
|
||||
Page records the <code>deltaY</code> sequence and requires a momentum-like tail:
|
||||
a peak <code>|deltaY|</code>, then a decreasing decay over several events
|
||||
(≥4 wheel events whose <code>|deltaY|</code> is strictly less than the previous
|
||||
by >5%). Constant or single-shot wheel events (one big delta then nothing)
|
||||
fail.</p>
|
||||
<div id="box" style="height:200px;overflow:auto;border:1px solid #555;background:#fff;color:#111;padding:8px">
|
||||
<div style="height:2400px">scroll me ⬇ … continues …</div>
|
||||
</div>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "scroll-momentum", instructions: "scroll the box to its bottom with a real gesture" });
|
||||
|
||||
const box = document.getElementById("box");
|
||||
const deltas = []; // signed
|
||||
const tsLog = [];
|
||||
|
||||
box.addEventListener("wheel", (e) => {
|
||||
deltas.push(e.deltaY);
|
||||
tsLog.push(e.timeStamp);
|
||||
if (deltas.length > 200) { deltas.shift(); tsLog.shift(); }
|
||||
Challenge.log("wheel", { dy: e.deltaY, isTrusted: e.isTrusted });
|
||||
if (!e.isTrusted) Challenge.fail("wheel event isTrusted=false");
|
||||
}, { passive: true });
|
||||
|
||||
box.addEventListener("scroll", () => {
|
||||
if (box.scrollTop + box.clientHeight < box.scrollHeight - 2) return;
|
||||
// We hit bottom. Analyze the deltaY trace.
|
||||
if (deltas.length === 0) return Challenge.fail("reached bottom with zero wheel events");
|
||||
if (deltas.length < 6) return Challenge.fail(`only ${deltas.length} wheel events (need ≥6 for momentum)`);
|
||||
|
||||
const abs = deltas.map(Math.abs);
|
||||
// All same sign? Trackpad inertia keeps signs consistent.
|
||||
const allSameSign = deltas.every(d => Math.sign(d) === Math.sign(deltas[0]));
|
||||
if (!allSameSign) return Challenge.fail("wheel deltaY signs flip — not a continuous scroll gesture");
|
||||
|
||||
// All identical magnitudes? Robotic.
|
||||
const allEqual = abs.every(a => Math.abs(a - abs[0]) < 0.01);
|
||||
if (allEqual) return Challenge.fail(`all |deltaY|=${abs[0]} — no momentum`);
|
||||
|
||||
// Find peak and require decay tail after it.
|
||||
const peakIdx = abs.indexOf(Math.max(...abs));
|
||||
const tail = abs.slice(peakIdx);
|
||||
let decays = 0;
|
||||
for (let i = 1; i < tail.length; i++) {
|
||||
if (tail[i] < tail[i-1] * 0.95) decays++;
|
||||
}
|
||||
Challenge.log("trace", { abs, peakIdx, decays });
|
||||
if (decays < 3) return Challenge.fail(`only ${decays} decay steps after peak — no momentum tail`);
|
||||
|
||||
// Timestamps within the tail should span ≥150ms (real inertia lasts hundreds of ms).
|
||||
const tailTs = tsLog.slice(peakIdx);
|
||||
const span = tailTs[tailTs.length-1] - tailTs[0];
|
||||
if (span < 100) return Challenge.fail(`momentum tail only spans ${span.toFixed(0)}ms`);
|
||||
|
||||
Challenge.pass(`${deltas.length} wheels, peak |dY|=${abs[peakIdx].toFixed(1)}, ${decays} decay steps over ${span.toFixed(0)}ms`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,72 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>28 IntersectionObserver visibility</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: scroll until the hidden <b>TARGET</b> button comes into view, then click it.
|
||||
Page uses an <code>IntersectionObserver</code> to track when TARGET crosses the
|
||||
viewport. Real scrolling produces a sequence of intersection updates with
|
||||
gradually increasing <code>intersectionRatio</code> (0 → 1) across animation
|
||||
frames. Programmatic teleports (<code>scrollIntoView({behavior:"instant"})</code>,
|
||||
direct <code>scrollTop=</code>) produce exactly one observer callback with
|
||||
<code>ratio≈1</code> and no rAF samples in between — fail.</p>
|
||||
<div id="scroller" style="height:240px;overflow:auto;border:1px solid #555;background:#fff;color:#111">
|
||||
<div style="height:1600px;padding:8px">keep scrolling …</div>
|
||||
<button id="target" style="margin:0 auto 1600px;display:block;padding:14px 22px;background:#1f7a1f;color:#fff;border:0;border-radius:6px">TARGET</button>
|
||||
</div>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "intersection-visibility", instructions: "scroll until TARGET visible, then click" });
|
||||
|
||||
const scroller = document.getElementById("scroller");
|
||||
const target = document.getElementById("target");
|
||||
|
||||
const ratioSamples = [];
|
||||
const rafSamples = [];
|
||||
|
||||
const io = new IntersectionObserver((entries) => {
|
||||
for (const e of entries) ratioSamples.push({ r: e.intersectionRatio, t: performance.now() });
|
||||
}, { root: scroller, threshold: Array.from({length: 21}, (_,i) => i/20) });
|
||||
io.observe(target);
|
||||
|
||||
// Track rAF ticks during which scroll was happening.
|
||||
let lastScrollTop = scroller.scrollTop;
|
||||
let scrollingFrames = 0;
|
||||
function tick() {
|
||||
if (scroller.scrollTop !== lastScrollTop) {
|
||||
scrollingFrames++;
|
||||
rafSamples.push({ st: scroller.scrollTop, t: performance.now() });
|
||||
lastScrollTop = scroller.scrollTop;
|
||||
}
|
||||
requestAnimationFrame(tick);
|
||||
}
|
||||
requestAnimationFrame(tick);
|
||||
|
||||
target.addEventListener("click", (e) => {
|
||||
if (!e.isTrusted) return Challenge.fail("TARGET click isTrusted=false");
|
||||
|
||||
// Need the target to actually be in the viewport at click time.
|
||||
const rect = target.getBoundingClientRect();
|
||||
const sRect = scroller.getBoundingClientRect();
|
||||
const inViewport = rect.top < sRect.bottom && rect.bottom > sRect.top;
|
||||
if (!inViewport) return Challenge.fail("TARGET clicked while still off-screen");
|
||||
|
||||
// Require at least a gradient of intersection ratios from low to high.
|
||||
const lows = ratioSamples.filter(s => s.r > 0 && s.r < 0.3).length;
|
||||
const mids = ratioSamples.filter(s => s.r >= 0.3 && s.r < 0.7).length;
|
||||
const highs = ratioSamples.filter(s => s.r >= 0.7).length;
|
||||
Challenge.log("io", { samples: ratioSamples.length, lows, mids, highs, scrollingFrames });
|
||||
|
||||
const bad = [];
|
||||
if (ratioSamples.length < 4) bad.push(`only ${ratioSamples.length} IntersectionObserver samples (teleport)`);
|
||||
if (lows === 0) bad.push("no low-ratio IO samples — target appeared instantly");
|
||||
if (mids === 0) bad.push("no mid-ratio IO samples — no smooth approach");
|
||||
if (scrollingFrames < 5) bad.push(`only ${scrollingFrames} rAF frames saw scroll motion`);
|
||||
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass(`IO gradient seen: lows=${lows} mids=${mids} highs=${highs}, rafFrames=${scrollingFrames}`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,44 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>29 shadow DOM controls</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: discover controls inside an <code>open</code> Shadow DOM, click the button, then type <code>shadow-ok</code> into the shadow input.</p>
|
||||
<div id="host"></div>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "shadow-dom-controls", instructions: "click shadow button, type 'shadow-ok' in shadow input" });
|
||||
const host = document.getElementById("host");
|
||||
const root = host.attachShadow({ mode: "open" });
|
||||
root.innerHTML = `
|
||||
<style>
|
||||
.card { border:1px solid #555; border-radius:8px; padding:16px; background:#222; display:inline-block; }
|
||||
button,input { font:16px system-ui; padding:8px 10px; margin:6px; }
|
||||
</style>
|
||||
<div class="card" role="group" aria-label="Shadow test controls">
|
||||
<button id="shadowBtn">Arm shadow form</button>
|
||||
<input id="shadowInput" aria-label="Shadow value" placeholder="shadow-ok" disabled>
|
||||
<span id="status">waiting</span>
|
||||
</div>
|
||||
`;
|
||||
const btn = root.getElementById("shadowBtn");
|
||||
const input = root.getElementById("shadowInput");
|
||||
const status = root.getElementById("status");
|
||||
let armed = false;
|
||||
btn.addEventListener("click", (e) => {
|
||||
armed = true;
|
||||
input.disabled = false;
|
||||
input.focus();
|
||||
status.textContent = `armed; click trusted=${e.isTrusted}`;
|
||||
Challenge.log("shadow-click", { isTrusted: e.isTrusted });
|
||||
});
|
||||
input.addEventListener("input", (e) => {
|
||||
Challenge.log("shadow-input", { value: input.value, isTrusted: e.isTrusted });
|
||||
if (input.value !== "shadow-ok") return;
|
||||
if (!armed) return Challenge.fail("input reached target value before shadow button click armed the form");
|
||||
Challenge.pass("shadow button clicked and shadow input filled");
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,44 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>30 iframe targeting</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: interact with controls inside a same-origin iframe. Type <code>frame-ok</code> then click Submit inside the frame.</p>
|
||||
<iframe id="ifr" title="same-origin challenge frame" style="width:520px;height:220px;border:1px solid #555;border-radius:8px;background:#fff"></iframe>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "iframe-targeting", instructions: "type 'frame-ok' and click Submit inside iframe" });
|
||||
const frame = document.getElementById("ifr");
|
||||
frame.srcdoc = `<!doctype html><meta charset="utf-8">
|
||||
<style>body{font:16px system-ui;margin:24px;background:#202020;color:#eee}input,button{font:16px system-ui;padding:8px 10px;margin:6px}</style>
|
||||
<p>Inside frame: fill and submit.</p>
|
||||
<input id="insideText" aria-label="Frame value" placeholder="frame-ok">
|
||||
<button id="insideButton">Submit frame</button>
|
||||
<pre id="log"></pre>
|
||||
<script>
|
||||
const input = document.getElementById('insideText');
|
||||
const btn = document.getElementById('insideButton');
|
||||
const log = document.getElementById('log');
|
||||
let focused = false;
|
||||
input.addEventListener('focus', () => { focused = true; parent.postMessage({ type:'frame-focus' }, '*'); });
|
||||
input.addEventListener('input', e => parent.postMessage({ type:'frame-input', value: input.value, trusted: e.isTrusted }, '*'));
|
||||
btn.addEventListener('click', e => {
|
||||
const payload = { type:'frame-submit', value: input.value, focused, trusted: e.isTrusted };
|
||||
log.textContent = JSON.stringify(payload, null, 2);
|
||||
parent.postMessage(payload, '*');
|
||||
});
|
||||
<\/script>`;
|
||||
window.addEventListener("message", (e) => {
|
||||
if (!e.data || typeof e.data !== "object") return;
|
||||
Challenge.log("frame-message", e.data);
|
||||
if (e.data.type !== "frame-submit") return;
|
||||
const bad = [];
|
||||
if (e.data.value !== "frame-ok") bad.push(`iframe input value="${e.data.value}" (need frame-ok)`);
|
||||
if (!e.data.focused) bad.push("iframe input was not focused before submit");
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass(`iframe controls reached; submit trusted=${e.data.trusted}`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,30 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>31 file upload</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: attach <code>test-suite/fixtures/pi-chrome-upload.txt</code>. Page reads name and contents from the File object.</p>
|
||||
<input id="file" type="file" aria-label="Upload fixture file" style="font-size:16px;padding:10px;border:1px solid #555;border-radius:6px">
|
||||
<pre id="rep" style="font:12px monospace;background:#111;color:#eee;padding:12px;border-radius:6px"></pre>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "file-upload", instructions: "upload fixtures/pi-chrome-upload.txt" });
|
||||
const input = document.getElementById("file");
|
||||
const rep = document.getElementById("rep");
|
||||
input.addEventListener("change", async (e) => {
|
||||
const f = input.files && input.files[0];
|
||||
if (!f) return Challenge.fail("change fired but input.files[0] missing");
|
||||
const text = await f.text();
|
||||
const info = { name: f.name, size: f.size, type: f.type, trusted: e.isTrusted, text };
|
||||
rep.textContent = JSON.stringify(info, null, 2);
|
||||
Challenge.log("file-change", info);
|
||||
const bad = [];
|
||||
if (f.name !== "pi-chrome-upload.txt") bad.push(`file.name=${f.name}`);
|
||||
if (text !== "pi-chrome upload fixture\n") bad.push(`file content=${JSON.stringify(text)}`);
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass(`uploaded ${f.name}; change.isTrusted=${e.isTrusted}`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,61 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>32 keyboard tab navigation</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: use keyboard only. Click Start, then Tab through fields, type <code>one</code>, <code>two</code>, <code>three</code>, Tab to Submit, press Enter.</p>
|
||||
<button id="start">Start here</button>
|
||||
<form id="f" style="margin-top:16px;display:grid;gap:10px;max-width:360px">
|
||||
<label>First <input id="a" name="a" autocomplete="off"></label>
|
||||
<label>Second <input id="b" name="b" autocomplete="off"></label>
|
||||
<label>Third <input id="c" name="c" autocomplete="off"></label>
|
||||
<button id="submit" type="submit">Submit</button>
|
||||
</form>
|
||||
<pre id="rep" style="font:12px monospace;background:#111;color:#eee;padding:12px;border-radius:6px"></pre>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "keyboard-tab-navigation", instructions: "keyboard-only tab flow; submit with Enter" });
|
||||
const ids = ["start", "a", "b", "c", "submit"];
|
||||
const focusLog = [];
|
||||
const keyLog = [];
|
||||
const rep = document.getElementById("rep");
|
||||
for (const id of ids) {
|
||||
const el = document.getElementById(id);
|
||||
el.addEventListener("focus", (e) => {
|
||||
focusLog.push({ id, t: e.timeStamp });
|
||||
rep.textContent = JSON.stringify({ focusLog, keyLog }, null, 2);
|
||||
});
|
||||
el.addEventListener("keydown", (e) => {
|
||||
keyLog.push({ id, key: e.key, trusted: e.isTrusted, t: e.timeStamp });
|
||||
});
|
||||
}
|
||||
let sawPointerAfterStart = false;
|
||||
document.addEventListener("pointerdown", (e) => {
|
||||
if (e.target && e.target.id !== "start") sawPointerAfterStart = true;
|
||||
});
|
||||
document.getElementById("start").addEventListener("click", () => setTimeout(() => document.getElementById("start").focus(), 0));
|
||||
document.getElementById("f").addEventListener("submit", (e) => {
|
||||
e.preventDefault();
|
||||
const vals = {
|
||||
a: document.getElementById("a").value,
|
||||
b: document.getElementById("b").value,
|
||||
c: document.getElementById("c").value
|
||||
};
|
||||
const bad = [];
|
||||
if (vals.a !== "one" || vals.b !== "two" || vals.c !== "three") bad.push(`values=${JSON.stringify(vals)}`);
|
||||
const seq = focusLog.map(x => x.id).join(">");
|
||||
for (const id of ids) if (!focusLog.some(x => x.id === id)) bad.push(`never focused ${id}`);
|
||||
const enter = keyLog.find(x => x.id === "submit" && x.key === "Enter");
|
||||
if (!enter) bad.push("no Enter keydown on submit button");
|
||||
else if (!enter.trusted) bad.push("Enter on submit isTrusted=false");
|
||||
const tabCount = keyLog.filter(x => x.key === "Tab").length;
|
||||
if (tabCount < 4) bad.push(`only ${tabCount} Tab keydowns (need ≥4)`);
|
||||
if (sawPointerAfterStart) bad.push("pointerdown after Start; flow was not keyboard-only");
|
||||
Challenge.log("keyboard-submit", { vals, seq, keyLog, sawPointerAfterStart });
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass(`keyboard-only focus sequence ${seq}`);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,33 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>33 network and console capture</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: click button. Page emits console messages and performs a fetch. Benchmark harness should verify <code>chrome_list_console_messages</code> and <code>chrome_list_network_requests</code> captured them.</p>
|
||||
<button id="go" style="padding:16px 24px;font-size:18px">Emit console + fetch</button>
|
||||
<pre id="rep" style="font:12px monospace;background:#111;color:#eee;padding:12px;border-radius:6px"></pre>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "network-console-capture", instructions: "clear capture, click, then inspect console/network tools" });
|
||||
document.getElementById("go").addEventListener("click", async (e) => {
|
||||
console.log("pi-chrome-benchmark-console", { clicked: true, trusted: e.isTrusted });
|
||||
console.warn("pi-chrome-benchmark-warning");
|
||||
let data;
|
||||
try {
|
||||
const res = await fetch("data:application/json,%7B%22ok%22%3Atrue%2C%22source%22%3A%22pi-chrome-benchmark%22%7D");
|
||||
data = await res.json();
|
||||
} catch (err) {
|
||||
return Challenge.fail("fetch failed: " + err.message);
|
||||
}
|
||||
document.getElementById("rep").textContent = JSON.stringify(data, null, 2);
|
||||
Challenge.log("fetch-result", data);
|
||||
if (data && data.ok && data.source === "pi-chrome-benchmark") {
|
||||
Challenge.pass("page fetch completed; verify tool-level console/network capture separately");
|
||||
} else {
|
||||
Challenge.fail("unexpected fetch payload: " + JSON.stringify(data));
|
||||
}
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,28 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>34 dialog handling</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: handle browser dialogs. Click Prompt, enter <code>pi-chrome</code>, accept it; then click Confirm and accept it. Browser automation must expose/dismiss native dialogs.</p>
|
||||
<button id="promptBtn">Prompt</button>
|
||||
<button id="confirmBtn">Confirm</button>
|
||||
<button id="alertBtn">Alert smoke</button>
|
||||
<pre id="rep" class="code"></pre>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "dialog-handling", instructions: "accept prompt('pi-chrome') and confirm()" });
|
||||
const state = { prompt: null, confirm: null, alert: false };
|
||||
function paint(){ rep.textContent = JSON.stringify(state, null, 2); }
|
||||
function check(){
|
||||
paint();
|
||||
if (state.prompt === "pi-chrome" && state.confirm === true) Challenge.pass("prompt and confirm handled with expected values");
|
||||
}
|
||||
promptBtn.addEventListener("click", () => { state.prompt = prompt("Enter token", ""); Challenge.log("prompt-result", { value: state.prompt }); check(); });
|
||||
confirmBtn.addEventListener("click", () => { state.confirm = confirm("Accept benchmark confirm?"); Challenge.log("confirm-result", { value: state.confirm }); check(); });
|
||||
alertBtn.addEventListener("click", () => { alert("pi-chrome alert smoke"); state.alert = true; Challenge.log("alert-dismissed", {}); paint(); });
|
||||
window.addEventListener("beforeunload", e => { e.preventDefault(); e.returnValue = "benchmark beforeunload"; });
|
||||
paint();
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,35 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>35 target blank popup</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: click <code>target=_blank</code> link, detect new tab, switch to it, and verify child page reports PASS.</p>
|
||||
<a id="open" target="_blank" rel="opener" href="35-target-blank-popup.html?child=1" style="font-size:18px">Open child tab</a>
|
||||
<p id="msg"></p>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "target-blank-popup", instructions: "open child tab and inspect it" });
|
||||
const qs = new URLSearchParams(location.search);
|
||||
if (qs.get("child") === "1") {
|
||||
document.getElementById("msg").textContent = "Child tab opened.";
|
||||
Challenge.pass("child tab loaded on same origin");
|
||||
try { localStorage.setItem("pi-chrome-suite:target-blank-popup", JSON.stringify({ id:"target-blank-popup", verdict:"PASS", reason:["✓ child tab loaded"], ts: Date.now() })); } catch {}
|
||||
} else {
|
||||
document.getElementById("msg").textContent = "Parent page. Link should open new tab.";
|
||||
function maybePassFromStorage(raw) {
|
||||
if (!raw) return;
|
||||
try {
|
||||
const v = JSON.parse(raw);
|
||||
if (v.verdict === "PASS") Challenge.pass("child reported PASS via storage event/localStorage");
|
||||
} catch {}
|
||||
}
|
||||
window.addEventListener("storage", e => {
|
||||
if (e.key === "pi-chrome-suite:target-blank-popup") maybePassFromStorage(e.newValue);
|
||||
});
|
||||
maybePassFromStorage(localStorage.getItem("pi-chrome-suite:target-blank-popup"));
|
||||
document.getElementById("open").addEventListener("click", e => Challenge.log("popup-click", { trusted: e.isTrusted }));
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,49 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>36 modal focus trap</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: open modal, use Tab to cycle focus inside modal only, then Escape closes it and focus returns to opener.</p>
|
||||
<button id="open">Open modal</button>
|
||||
<div id="modal" role="dialog" aria-modal="true" aria-label="Focus trap modal" hidden style="position:fixed;inset:20%;background:#222;border:2px solid #6cf;border-radius:10px;padding:20px;z-index:5">
|
||||
<button id="first">First</button>
|
||||
<input id="middle" aria-label="Modal input">
|
||||
<button id="last">Last</button>
|
||||
</div>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "modal-focus-trap", instructions: "open modal, Tab cycles inside, Escape closes" });
|
||||
const order = [];
|
||||
const modal = document.getElementById("modal");
|
||||
const nodes = [first, middle, last];
|
||||
let escaped = false, modalOpened = false, outsideFocusWhileOpen = false;
|
||||
for (const el of [open, ...nodes]) el.addEventListener("focus", () => { order.push(el.id); Challenge.log("focus", { id: el.id }); });
|
||||
document.addEventListener("focusin", e => {
|
||||
if (modalOpened && !modal.hidden && !modal.contains(e.target)) outsideFocusWhileOpen = true;
|
||||
});
|
||||
open.addEventListener("click", () => { modal.hidden = false; modalOpened = true; first.focus(); });
|
||||
document.addEventListener("keydown", e => {
|
||||
if (modal.hidden) return;
|
||||
if (e.key === "Tab") {
|
||||
const i = nodes.indexOf(document.activeElement);
|
||||
if (i >= 0) { e.preventDefault(); nodes[(i + (e.shiftKey ? -1 : 1) + nodes.length) % nodes.length].focus(); }
|
||||
}
|
||||
if (e.key === "Escape") { modal.hidden = true; escaped = true; open.focus(); queueMicrotask(check); }
|
||||
});
|
||||
function hasAdjacent(seq, a, b) {
|
||||
return seq.some((x, i) => x === a && seq[i + 1] === b);
|
||||
}
|
||||
function check(){
|
||||
const afterOpen = order.slice(order.indexOf("first"));
|
||||
const bad = [];
|
||||
if (!escaped) bad.push("Escape did not close modal");
|
||||
if (document.activeElement !== open) bad.push("focus did not return to opener");
|
||||
if (!afterOpen.includes("middle") || !afterOpen.includes("last")) bad.push(`focus cycle incomplete: ${afterOpen.join(">")}`);
|
||||
if (!hasAdjacent(afterOpen, "last", "first") && !hasAdjacent(afterOpen, "first", "last")) bad.push(`no wrap-around focus trap observed: ${afterOpen.join(">")}`);
|
||||
if (outsideFocusWhileOpen) bad.push("focus escaped outside modal while open");
|
||||
if (bad.length) Challenge.fail(...bad); else Challenge.pass(`modal focus trap ok: ${afterOpen.join(">")}`);
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,42 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>37 autocomplete combobox</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: type <code>as</code>, use ArrowDown then Enter to choose <code>Aster Lamp</code> from ARIA combobox.</p>
|
||||
<label>Product <input id="combo" role="combobox" aria-autocomplete="list" aria-controls="list" aria-expanded="false" autocomplete="off"></label>
|
||||
<ul id="list" role="listbox" style="border:1px solid #555;max-width:260px;padding:6px;display:none"></ul>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "autocomplete-combobox", instructions: "type 'as', ArrowDown, Enter to choose Aster Lamp" });
|
||||
const items = ["Aster Lamp", "Aspen Desk", "Cable Kit", "Desk Mat", "Red Mug"];
|
||||
let active = -1, opened = false, keyPath = [];
|
||||
combo.addEventListener("input", renderList);
|
||||
combo.addEventListener("keydown", e => {
|
||||
keyPath.push(e.key);
|
||||
const opts = [...list.querySelectorAll('[role="option"]')];
|
||||
if (e.key === "ArrowDown") { e.preventDefault(); active = Math.min(active + 1, opts.length - 1); paintActive(opts); }
|
||||
if (e.key === "Enter" && active >= 0 && opts[active]) { e.preventDefault(); combo.value = opts[active].textContent; close(); check(e); }
|
||||
});
|
||||
function renderList(){
|
||||
const q = combo.value.toLowerCase();
|
||||
const matches = items.filter(x => x.toLowerCase().includes(q));
|
||||
list.innerHTML = matches.map((x,i)=>`<li role="option" id="opt-${i}" style="padding:4px;cursor:pointer">${x}</li>`).join("");
|
||||
list.style.display = matches.length ? "block" : "none";
|
||||
combo.setAttribute("aria-expanded", matches.length ? "true" : "false");
|
||||
active = -1; opened ||= matches.length > 0;
|
||||
}
|
||||
function paintActive(opts){ opts.forEach((o,i)=>o.style.background = i===active ? "#2b4b6b" : ""); if(opts[active]) combo.setAttribute("aria-activedescendant", opts[active].id); }
|
||||
function close(){ list.style.display="none"; combo.setAttribute("aria-expanded","false"); }
|
||||
function check(e){
|
||||
const bad=[];
|
||||
if (!opened) bad.push("listbox never opened");
|
||||
if (combo.value !== "Aster Lamp") bad.push(`selected ${combo.value}`);
|
||||
if (!keyPath.includes("ArrowDown") || !keyPath.includes("Enter")) bad.push(`missing keyboard selection path: ${keyPath.join(",")}`);
|
||||
if (!e.isTrusted) bad.push("Enter key isTrusted=false");
|
||||
if (bad.length) Challenge.fail(...bad); else Challenge.pass("ARIA combobox selected via keyboard");
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,40 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>38 SPA route change</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: navigate within SPA using links/buttons. URL must change via <code>history.pushState</code> without full reload. Then go Back to Settings, forward to Billing, and Save.</p>
|
||||
<nav><a href="/settings" id="settings">Settings</a> <a href="/settings/billing" id="billing">Billing</a></nav>
|
||||
<section id="view"></section>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "spa-route-change", instructions: "Settings → Billing → Back → Billing → Save, no full reload" });
|
||||
let navs = 0, popstates = 0;
|
||||
window.__spaBootId = window.__spaBootId || Math.random().toString(36).slice(2);
|
||||
const bootId = window.__spaBootId;
|
||||
const navigationEntriesAtBoot = performance.getEntriesByType("navigation").length;
|
||||
function route(path){ history.pushState({}, "", path); navs++; render(); Challenge.log("route", { path, navs }); }
|
||||
settings.addEventListener("click", e => { e.preventDefault(); route("/settings"); });
|
||||
billing.addEventListener("click", e => { e.preventDefault(); route("/settings/billing"); });
|
||||
window.addEventListener("popstate", () => { popstates++; render(); Challenge.log("popstate", { path: location.pathname, popstates }); });
|
||||
function render(){
|
||||
if (location.pathname.endsWith("/settings/billing")) view.innerHTML = `<h2>Billing</h2><button id="save">Save billing settings</button>`;
|
||||
else if (location.pathname.endsWith("/settings")) view.innerHTML = `<h2>Settings</h2><p>Choose Billing.</p>`;
|
||||
else view.innerHTML = `<h2>Home</h2>`;
|
||||
const save = document.getElementById("save");
|
||||
if (save) save.addEventListener("click", e => {
|
||||
const bad=[];
|
||||
if (!location.pathname.endsWith("/settings/billing")) bad.push(`wrong route ${location.pathname}`);
|
||||
if (navs < 2) bad.push(`only ${navs} SPA route changes`);
|
||||
if (popstates < 1) bad.push("browser Back/popstate was not observed");
|
||||
if (window.__spaBootId !== bootId) bad.push("window sentinel changed; reload detected");
|
||||
if (performance.getEntriesByType("navigation").length !== navigationEntriesAtBoot) bad.push("navigation entry count changed; reload suspected");
|
||||
if (!e.isTrusted) bad.push("save click isTrusted=false");
|
||||
if (bad.length) Challenge.fail(...bad); else Challenge.pass("SPA pushState flow completed without reload");
|
||||
});
|
||||
}
|
||||
render();
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,14 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; object-src 'none'; base-uri 'none'">
|
||||
<title>39 strict CSP fallback</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: page blocks <code>unsafe-eval</code>. Agent should fall back to screenshot/coordinate observation and click the target without relying on snapshot/evaluate.</p>
|
||||
<p id="hint">The green target is fixed at known viewport coordinates. Use a screenshot, then click inside it.</p>
|
||||
<button id="cspTarget" aria-label="strict CSP target">CSP TARGET</button>
|
||||
</main>
|
||||
<script src="39-strict-csp-fallback.js"></script>
|
||||
</body>
|
||||
@@ -0,0 +1,27 @@
|
||||
Challenge.init({ id: "strict-csp-fallback", instructions: "use screenshot/coordinates; click CSP TARGET" });
|
||||
|
||||
const btn = document.getElementById("cspTarget");
|
||||
btn.style.cssText = [
|
||||
"position:fixed",
|
||||
"left:220px",
|
||||
"top:220px",
|
||||
"width:180px",
|
||||
"height:72px",
|
||||
"font:700 16px system-ui",
|
||||
"background:#1f7a1f",
|
||||
"color:white",
|
||||
"border:0",
|
||||
"border-radius:10px",
|
||||
"box-shadow:0 0 0 4px rgba(31,122,31,.25)"
|
||||
].join(";");
|
||||
|
||||
document.getElementById("cspTarget").addEventListener("click", (e) => {
|
||||
const r = btn.getBoundingClientRect();
|
||||
const bad = [];
|
||||
if (!e.isTrusted) bad.push("click isTrusted=false");
|
||||
if (e.clientX < r.left || e.clientX > r.right || e.clientY < r.top || e.clientY > r.bottom) {
|
||||
bad.push(`click coordinates ${e.clientX},${e.clientY} outside target rect ${Math.round(r.left)},${Math.round(r.top)},${Math.round(r.right)},${Math.round(r.bottom)}`);
|
||||
}
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass("strict CSP page completed via trusted viewport click");
|
||||
});
|
||||
@@ -0,0 +1,41 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>40 dynamic wait readiness</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: wait until an async control exists and is enabled, then click it. Direct early clicks or fixed sleeps should not be required.</p>
|
||||
<div id="status" role="status">Loading async action…</div>
|
||||
<div id="mount" style="margin-top:20px"></div>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "dynamic-wait-readiness", instructions: "wait for async button, then click" });
|
||||
const start = performance.now();
|
||||
setTimeout(() => {
|
||||
const btn = document.createElement("button");
|
||||
btn.id = "readyAction";
|
||||
btn.textContent = "Run ready action";
|
||||
btn.disabled = true;
|
||||
btn.style.cssText = "padding:14px 22px;font-size:16px;background:#1f7a1f;color:#fff;border:0;border-radius:6px";
|
||||
document.getElementById("mount").appendChild(btn);
|
||||
document.getElementById("status").textContent = "Button mounted; enabling soon…";
|
||||
Challenge.log("mounted", { at: performance.now() - start });
|
||||
setTimeout(() => {
|
||||
btn.disabled = false;
|
||||
btn.dataset.ready = "true";
|
||||
document.getElementById("status").textContent = "Ready.";
|
||||
Challenge.log("enabled", { at: performance.now() - start });
|
||||
}, 450);
|
||||
btn.addEventListener("click", (e) => {
|
||||
const elapsed = performance.now() - start;
|
||||
const bad = [];
|
||||
if (!e.isTrusted) bad.push("click isTrusted=false");
|
||||
if (btn.disabled || btn.dataset.ready !== "true") bad.push("clicked before enabled/ready");
|
||||
if (elapsed < 650) bad.push(`clicked too early at ${elapsed.toFixed(0)}ms`);
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass(`waited for async ready state (${elapsed.toFixed(0)}ms)`);
|
||||
});
|
||||
}, 350);
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,44 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>41 tab lifecycle</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: create a new tab, inspect it, close it, return to parent, then verify lifecycle state.</p>
|
||||
<p id="msg"></p>
|
||||
<button id="verify">Verify tab lifecycle</button>
|
||||
</main>
|
||||
<script>
|
||||
Challenge.init({ id: "tab-lifecycle", instructions: "new tab → inspect child → close child → return parent → Verify" });
|
||||
const qs = new URLSearchParams(location.search);
|
||||
const key = "pi-chrome-suite:tab-lifecycle:state";
|
||||
function readState(){ try { return JSON.parse(localStorage.getItem(key) || "{}"); } catch { return {}; } }
|
||||
function replaceState(next){ localStorage.setItem(key, JSON.stringify({ ...next, ts: Date.now() })); }
|
||||
function writeState(patch){ localStorage.setItem(key, JSON.stringify({ ...readState(), ...patch, ts: Date.now() })); }
|
||||
|
||||
if (qs.get("child") === "1") {
|
||||
document.title = "[CHILD] 41 tab lifecycle";
|
||||
document.getElementById("msg").textContent = "Child tab opened. Close this tab, then return to parent.";
|
||||
writeState({ opened: true, childUrl: location.href, childTitle: document.title });
|
||||
window.addEventListener("pagehide", () => writeState({ closed: true, closedAt: Date.now() }));
|
||||
window.addEventListener("beforeunload", () => writeState({ closed: true, closedAt: Date.now() }));
|
||||
// Do not call Challenge.pass here: parent verification must be the only suite PASS.
|
||||
Challenge.log("child-loaded", { url: location.href });
|
||||
} else {
|
||||
document.getElementById("msg").textContent = "Parent tab. Runner should open this same file with ?child=1 in a new tab, inspect it, close it, then return here.";
|
||||
replaceState({ parentLoaded: true, closed: false });
|
||||
document.getElementById("verify").addEventListener("click", (e) => {
|
||||
const s = readState();
|
||||
const bad = [];
|
||||
if (!e.isTrusted) bad.push("verify click isTrusted=false");
|
||||
if (!s.opened) bad.push("child tab did not record opened=true");
|
||||
if (!s.childUrl || !s.childUrl.includes("child=1")) bad.push("child URL not recorded");
|
||||
if (!s.closed) bad.push("child tab close/pagehide was not recorded");
|
||||
Challenge.log("tab-state", s);
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass("new/inspect/close/return tab lifecycle completed");
|
||||
});
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,16 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; object-src 'none'; base-uri 'none'">
|
||||
<title>42 strict CSP evaluate/snapshot</title>
|
||||
<link rel="stylesheet" href="../_style.css">
|
||||
<script src="../_lib.js"></script>
|
||||
<body>
|
||||
<main>
|
||||
<p>Goal: this page ships a strict CSP (<code>script-src 'self'</code>, no <code>unsafe-eval</code>), which blocks <code>eval</code>/<code>new Function</code>. <code>chrome_evaluate</code> and <code>chrome_snapshot</code> must still work because they run through CDP, which is not subject to page CSP.</p>
|
||||
<p id="hint">A secret token is exposed only at <code>window.__cspToken</code> — it is never written into the DOM. Use <code>chrome_evaluate</code> to read it, type it into the field (snapshot/uid to find the field), then click Verify.</p>
|
||||
<label for="tokenInput">Token:</label>
|
||||
<input id="tokenInput" type="text" autocomplete="off" aria-label="csp token">
|
||||
<button id="verify" aria-label="verify token">Verify</button>
|
||||
</main>
|
||||
<script src="42-strict-csp-evaluate.js"></script>
|
||||
</body>
|
||||
@@ -0,0 +1,21 @@
|
||||
Challenge.init({
|
||||
id: "strict-csp-evaluate",
|
||||
instructions: "under strict CSP: read window.__cspToken via chrome_evaluate, type it into the field, click Verify",
|
||||
});
|
||||
|
||||
// Secret available only via JS evaluation. It is intentionally NOT rendered into the DOM and
|
||||
// is defined non-enumerable, so the only way to obtain it is to evaluate window.__cspToken in
|
||||
// the page (which proves chrome_evaluate works despite script-src 'self' blocking eval).
|
||||
const token = "csp-" + Math.random().toString(36).slice(2, 10);
|
||||
Object.defineProperty(window, "__cspToken", { value: token, enumerable: false, configurable: false, writable: false });
|
||||
|
||||
document.getElementById("verify").addEventListener("click", (e) => {
|
||||
const bad = [];
|
||||
if (!e.isTrusted) bad.push("verify click isTrusted=false (use trusted/CDP input)");
|
||||
const val = (document.getElementById("tokenInput").value || "").trim();
|
||||
if (val !== token) {
|
||||
bad.push(`token mismatch: got "${val}" expected "${token}" — chrome_evaluate must read window.__cspToken under strict CSP`);
|
||||
}
|
||||
if (bad.length) Challenge.fail(...bad);
|
||||
else Challenge.pass("strict CSP: chrome_evaluate read the hidden token via CDP and trusted input submitted it");
|
||||
});
|
||||
@@ -0,0 +1 @@
|
||||
pi-chrome upload fixture
|
||||
@@ -0,0 +1,12 @@
|
||||
window.MiniShopCheats = {
|
||||
"mini-shop-red-second-cheapest": [
|
||||
{ tool: "chrome_click", params: { selector: "button[aria-label=\"add Ruby Notebook\"]" } },
|
||||
{ tool: "chrome_click", params: { selector: "#checkout" } }
|
||||
],
|
||||
run: {
|
||||
"mini-shop-red-second-cheapest": async () => {
|
||||
document.querySelector('button[aria-label="add Ruby Notebook"]').click();
|
||||
document.querySelector('#checkout').click();
|
||||
}
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,29 @@
|
||||
window.MiniShopGrader = {
|
||||
validate(task, state, products, run, seed = 0) {
|
||||
let bad = [];
|
||||
const rubric = {};
|
||||
if (task === 'mini-shop-red-second-cheapest') {
|
||||
const reds = products.filter(p => p.color === 'red').sort((a,b)=>a.price-b.price);
|
||||
const target = reds[1];
|
||||
rubric.addedTarget = state.cart.includes(target.id);
|
||||
rubric.noWrongItems = state.cart.every(id => id === target.id);
|
||||
rubric.checkedOut = /^MS-\d{4}$/.test(state.orderId || '');
|
||||
if (!rubric.addedTarget) bad.push(`cart missing second-cheapest red item ${target.name}`);
|
||||
const wrong = state.cart.filter(id => id !== target.id);
|
||||
if (wrong.length) bad.push(`cart contains wrong product(s): ${wrong.join(',')}`);
|
||||
if (!rubric.checkedOut) bad.push('missing checkout order id');
|
||||
} else bad.push(`unknown task ${task}`);
|
||||
const passed = bad.length === 0;
|
||||
return { reward: passed ? 1 : 0, done: passed, message: passed ? 'all deterministic mini-shop checks passed' : bad.join('; '), info: { task, run, seed, rubric, bad, state } };
|
||||
},
|
||||
grade(task, state, products, run, seed = 0) {
|
||||
const validation = this.validate(task, state, products, run, seed);
|
||||
const verdict = validation.done ? 'PASS' : 'PENDING';
|
||||
window.__taskId = task;
|
||||
window.__taskVerdict = verdict;
|
||||
window.__taskReason = validation.done ? [validation.message] : validation.info.bad;
|
||||
window.__taskValidation = validation;
|
||||
try { localStorage.setItem(`pi-chrome-task:${task}:${run}`, JSON.stringify({ id: task, run, seed, verdict, reason: window.__taskReason, validation, state, ts: Date.now() })); } catch {}
|
||||
return validation;
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,55 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>MiniShop hermetic site</title>
|
||||
<link rel="stylesheet" href="../../../_style.css">
|
||||
<style>
|
||||
main{max-width:1100px}.products{display:grid;grid-template-columns:repeat(auto-fit,minmax(210px,1fr));gap:12px}.card{border:1px solid #333;border-radius:10px;background:#202020;padding:12px}.cart{position:sticky;top:8px;background:#111;border:1px solid #555;border-radius:10px;padding:12px;margin-bottom:12px}.red{color:#ff8b8b}.blue{color:#8bbcff}.green{color:#9f9}.muted{color:#aaa;font-size:12px}</style>
|
||||
<body>
|
||||
<main>
|
||||
<h1>MiniShop</h1>
|
||||
<div class="cart"><b>Cart</b> <span id="cartCount">0</span> items <button id="checkout" data-task-role="checkout">Checkout</button> <span id="order"></span> <span id="reward" class="muted"></span><div id="cartItems"></div></div>
|
||||
<p id="taskText"></p>
|
||||
<section class="products" id="productsEl"></section>
|
||||
</main>
|
||||
<script src="grader.js?v=2"></script>
|
||||
<script src="cheats.js?v=2"></script>
|
||||
<script>
|
||||
(function(){
|
||||
const params=new URLSearchParams(location.search);
|
||||
if(!params.get('run')){params.set('run', crypto.randomUUID ? crypto.randomUUID() : String(Date.now())); history.replaceState(null,'',`?${params}`);}
|
||||
const task=params.get('task')||'mini-shop-red-second-cheapest'; const run=params.get('run'); const seed=Number(params.get('seed')||0);
|
||||
const key=`mini-shop:${task}:${run}:${seed}`;
|
||||
const products=[
|
||||
{id:'p1',name:'Crimson Mug',color:'red',price:12.00},
|
||||
{id:'p2',name:'Ruby Notebook',color:'red',price:7.50},
|
||||
{id:'p3',name:'Scarlet Pen',color:'red',price:3.25},
|
||||
{id:'p4',name:'Blue Cable',color:'blue',price:5.10},
|
||||
{id:'p5',name:'Green Mat',color:'green',price:9.00},
|
||||
{id:'p6',name:'Red Stapler',color:'red',price:18.00},
|
||||
{id:'p7',name:'Blue Lamp',color:'blue',price:22.00},
|
||||
{id:'p8',name:'Green Clip',color:'green',price:2.40}
|
||||
];
|
||||
let state=JSON.parse(localStorage.getItem(key)||'{"cart":[],"orderId":"","actionCount":0,"startedAt":0}');
|
||||
if(!state.startedAt) state.startedAt=Date.now();
|
||||
taskText.textContent='Task: add the second-cheapest red product to cart, checkout, and leave order id visible.';
|
||||
function save(){localStorage.setItem(key,JSON.stringify(state)); window.__miniShopState=state; const v=MiniShopGrader.grade(task,state,products,run,seed); window.__taskReward=v.reward; window.__taskTerminated=v.done; window.__taskInfo=v.info; reward.textContent=`reward=${v.reward} actions=${state.actionCount||0}`; renderCart();}
|
||||
function assignBids(){let n=0; document.querySelectorAll('button,a,input,select,textarea').forEach(el=>{const bid='b'+(++n).toString(36); el.setAttribute('bid',bid); el.setAttribute('data-bid',bid); const r=el.getBoundingClientRect(); el.setAttribute('browsergym_visibility_ratio',(r.width&&r.height?1:0).toString());});}
|
||||
function render(){productsEl.innerHTML=''; products.forEach(p=>{const d=document.createElement('div');d.className='card';d.innerHTML=`<h3 class="${p.color}">${p.name}</h3><p>Color: ${p.color}</p><p>Price: $${p.price.toFixed(2)}</p><button aria-label="add ${p.name}" data-id="${p.id}" data-task-role="add-to-cart">Add to cart</button>`;productsEl.appendChild(d)});document.querySelectorAll('[data-id]').forEach(b=>b.onclick=()=>{state.actionCount=(state.actionCount||0)+1;state.cart.push(b.dataset.id);save()});renderCart();assignBids();}
|
||||
function renderCart(){
|
||||
cartCount.textContent=state.cart.length; order.textContent=state.orderId?`Order ${state.orderId}`:'';
|
||||
cartItems.innerHTML=state.cart.map((id,i)=>{const p=products.find(x=>x.id===id);return `<div>${p?.name||id} <button data-remove-index="${i}" aria-label="remove ${p?.name||id}" data-task-role="remove-from-cart">Remove</button></div>`}).join('');
|
||||
cartItems.querySelectorAll('[data-remove-index]').forEach(b=>b.onclick=()=>{state.actionCount=(state.actionCount||0)+1;state.cart.splice(Number(b.dataset.removeIndex),1);save();render();});
|
||||
assignBids();
|
||||
}
|
||||
function orderId(){ return 'MS-' + String(1000 + ((seed * 2654435761 + state.cart.join('').length * 97) % 9000)).padStart(4, '0'); }
|
||||
checkout.onclick=()=>{state.actionCount=(state.actionCount||0)+1;state.orderId=orderId();save();};
|
||||
function observation(){return {goal:taskText.textContent,url:location.href,title:document.title,reward:window.__taskReward,actionCount:state.actionCount||0,focusedElementBid:document.activeElement?.getAttribute('bid')||null};}
|
||||
async function runCheat(){const fn=window.MiniShopCheats?.run?.[task]; if(fn) await fn(); else for(const step of (window.MiniShopCheats?.[task]||[])) console.log('cheat step',step); save();}
|
||||
window.__miniShopState=state; window.__miniShopProducts=products; window.__miniShopTask=task;
|
||||
window.__task={id:task,run,seed,goal:taskText.textContent,difficulty:'L2',step:()=>{const v=MiniShopGrader.grade(task,state,products,run,seed);return {observation:observation(),reward:v.reward,terminated:v.done,truncated:false,info:v.info}},reset:()=>{localStorage.removeItem(key);state={cart:[],orderId:'',actionCount:0,startedAt:Date.now()};render();save();return {observation:observation(),info:{task,run,seed}}},rubric:()=>MiniShopGrader.validate(task,state,products,run,seed).info.rubric,cheat:runCheat,validate:()=>MiniShopGrader.validate(task,state,products,run,seed),teardown:()=>undefined};
|
||||
window.__miniShop=window.__task;
|
||||
window.__miniShop.cheatHelpers={click:(sel)=>document.querySelector(sel).click()};
|
||||
render(); save();
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
@@ -0,0 +1,9 @@
|
||||
[
|
||||
{
|
||||
"id": "mini-shop-red-second-cheapest",
|
||||
"difficulty": "L2",
|
||||
"intent": "Add the second-cheapest red product to cart, checkout, and leave the generated order id visible.",
|
||||
"gradeSource": "harness",
|
||||
"grader": "MiniShopGrader.grade(task, state, products, run)"
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,193 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>pi-chrome browser-control benchmark</title>
|
||||
<link rel="stylesheet" href="_style.css">
|
||||
<body>
|
||||
<main style="max-width:1280px">
|
||||
<h1>pi-chrome browser-control benchmark</h1>
|
||||
<p>This benchmark measures how well Chrome-control tools let agents do real browser work: DOM discovery, trusted input, keyboard/focus, scroll, drag/drop, files, frames, clipboard, and observability.</p>
|
||||
|
||||
<section class="panel">
|
||||
<div class="controls">
|
||||
<button id="refresh">Refresh verdicts</button>
|
||||
<button id="clear">Clear local verdicts</button>
|
||||
<button id="copy">Copy JSON report</button>
|
||||
<label>Filter <input id="filter" placeholder="category, id, expected..." /></label>
|
||||
<label>Mode
|
||||
<select id="mode">
|
||||
<option value="trusted">trusted</option>
|
||||
<option value="synthetic">synthetic</option>
|
||||
<option value="manual">manual</option>
|
||||
</select>
|
||||
</label>
|
||||
</div>
|
||||
<p class="hint">Open a row, drive it with <code>chrome_*</code> tools, then return here. Verdicts are read from <code>localStorage</code>. Expected outcomes come from <code>manifest.json</code>.</p>
|
||||
<textarea id="copyFallback" class="copy-fallback" readonly aria-label="JSON report fallback"></textarea>
|
||||
</section>
|
||||
|
||||
<div id="summary" class="summary"></div>
|
||||
<table id="tbl">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>id</th><th>gate</th><th>category</th><th>verdict</th><th>expected</th><th>baseline</th><th>risk</th><th>goal / notes</th><th>open</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody></tbody>
|
||||
</table>
|
||||
|
||||
<h2>Long-horizon hermetic tasks</h2>
|
||||
<p class="hint">These WebArena/BrowserGym-inspired tasks use deterministic in-page graders and fresh <code>$RUN_ID</code> state. They test multi-step browser work beyond event fidelity.</p>
|
||||
<table id="taskTbl">
|
||||
<thead><tr><th>id</th><th>difficulty</th><th>category</th><th>intent</th><th>requires</th><th>open</th></tr></thead>
|
||||
<tbody></tbody>
|
||||
</table>
|
||||
|
||||
<h2>Agent run loop</h2>
|
||||
<pre class="code">python3 -m http.server 8765
|
||||
# For each manifest entry:
|
||||
# 1. chrome_navigate http://127.0.0.1:8765/<file>
|
||||
# 2. chrome_snapshot before acting; prefer uid selectors.
|
||||
# 3. Execute recipe with selected mode, adapting descriptive frame/shadow selectors to tool uids.
|
||||
# 4. chrome_evaluate JSON.stringify({v:window.__verdict,r:window.__reason,e:window.__events?.slice(-20)})
|
||||
# 5. Compare verdict to manifest.expected[mode]; CONDITIONAL means inspect prerequisites/notes.</pre>
|
||||
</main>
|
||||
<script>
|
||||
let manifest = [];
|
||||
let taskManifest = [];
|
||||
const tbody = document.querySelector("#tbl tbody");
|
||||
const taskTbody = document.querySelector("#taskTbl tbody");
|
||||
const modeEl = document.getElementById("mode");
|
||||
const filterEl = document.getElementById("filter");
|
||||
|
||||
function verdictFor(id) {
|
||||
const raw = localStorage.getItem("pi-chrome-suite:" + id);
|
||||
if (!raw) return null;
|
||||
try { return JSON.parse(raw); } catch { return null; }
|
||||
}
|
||||
function classFor(v) {
|
||||
if (!v) return "pending";
|
||||
return String(v).toLowerCase();
|
||||
}
|
||||
function expectedClass(actual, expected) {
|
||||
if (!actual) return "pending";
|
||||
if (expected === "CONDITIONAL") return actual === "PASS" ? "ok" : "conditional";
|
||||
return actual === expected ? "ok" : "mismatch";
|
||||
}
|
||||
function report() {
|
||||
const mode = modeEl.value;
|
||||
return manifest.map(m => {
|
||||
const v = verdictFor(m.id);
|
||||
return {
|
||||
id: m.id,
|
||||
category: m.category,
|
||||
gate: m.gate ?? "core",
|
||||
file: m.file,
|
||||
verdict: v?.verdict ?? "PENDING",
|
||||
reason: v?.reason ?? [],
|
||||
expected: m.expected?.[mode] ?? "—",
|
||||
timestamp: v?.ts ?? null,
|
||||
goal: m.goal,
|
||||
prerequisites: m.prerequisites ?? [],
|
||||
notes: m.notes ?? []
|
||||
};
|
||||
});
|
||||
}
|
||||
function paint() {
|
||||
const q = filterEl.value.trim().toLowerCase();
|
||||
const mode = modeEl.value;
|
||||
tbody.innerHTML = "";
|
||||
const rows = report();
|
||||
const counts = rows.reduce((acc, r) => (acc[r.verdict] = (acc[r.verdict] || 0) + 1, acc), {});
|
||||
const mismatches = rows.filter(r => r.verdict !== "PENDING" && r.expected !== "CONDITIONAL" && r.expected !== "—" && r.verdict !== r.expected).length;
|
||||
const coreMismatches = rows.filter(r => (r.gate ?? "core") === "core" && r.verdict !== "PENDING" && r.expected !== "CONDITIONAL" && r.expected !== "—" && r.verdict !== r.expected).length;
|
||||
const gates = rows.reduce((acc, r) => (acc[r.gate ?? "core"] = (acc[r.gate ?? "core"] || 0) + 1, acc), {});
|
||||
document.getElementById("summary").innerHTML = `
|
||||
<span class="pill pass">PASS ${counts.PASS || 0}</span>
|
||||
<span class="pill fail">FAIL ${counts.FAIL || 0}</span>
|
||||
<span class="pill skip">SKIP ${counts.SKIP || 0}</span>
|
||||
<span class="pill warn">WARN ${counts.WARN || 0}</span>
|
||||
<span class="pill pending">PENDING ${counts.PENDING || 0}</span>
|
||||
<span class="pill ${coreMismatches ? "fail" : "pass"}">core unexpected ${coreMismatches}</span>
|
||||
<span class="pill ${mismatches ? "fail" : "pass"}">all unexpected ${mismatches}</span>
|
||||
<span class="pill expected">core ${gates.core || 0}</span>
|
||||
<span class="pill expected">conditional ${gates.conditional || 0}</span>
|
||||
<span class="pill expected">quality ${gates.quality || 0}</span>
|
||||
`;
|
||||
for (const m of manifest) {
|
||||
const v = verdictFor(m.id);
|
||||
const verdict = v?.verdict ?? "PENDING";
|
||||
const expected = m.expected?.[mode] ?? "—";
|
||||
const hay = JSON.stringify({ id:m.id, gate:m.gate, category:m.category, verdict, expected, goal:m.goal, notes:m.notes }).toLowerCase();
|
||||
if (q && !hay.includes(q)) continue;
|
||||
const tr = document.createElement("tr");
|
||||
tr.className = expectedClass(verdict, expected);
|
||||
const reason = (v?.reason || []).join(" / ");
|
||||
const notes = [...(m.prerequisites || []).map(x => "pre: " + x), ...(m.notes || [])].join("; ");
|
||||
tr.innerHTML = `
|
||||
<td><code>${m.id}</code></td>
|
||||
<td><span class="pill expected">${m.gate || "core"}</span></td>
|
||||
<td>${m.category || "—"}</td>
|
||||
<td><span class="pill ${classFor(verdict)}">${verdict}</span><div class="reason">${escapeHtml(reason)}</div></td>
|
||||
<td><span class="pill expected">${expected}</span></td>
|
||||
<td><span class="pill ${m.manualBaseline === "verified" ? "pass" : "pending"}">${m.manualBaseline || "unverified"}</span></td>
|
||||
<td>${m.flakeRisk ? `<span class="risk">${m.flakeRisk}</span>` : ""}</td>
|
||||
<td><div>${escapeHtml(m.goal || "")}</div><div class="notes">${escapeHtml(notes)}</div><details><summary>recipe</summary><pre>${escapeHtml(JSON.stringify(m.recipe || [], null, 2))}</pre></details></td>
|
||||
<td><a target="_blank" href="${m.file}">open</a></td>
|
||||
`;
|
||||
tbody.appendChild(tr);
|
||||
}
|
||||
}
|
||||
function escapeHtml(s) {
|
||||
return String(s).replace(/[&<>"']/g, c => ({"&":"&","<":"<",">":">","\"":""","'":"'"}[c]));
|
||||
}
|
||||
function paintTasks() {
|
||||
taskTbody.innerHTML = "";
|
||||
const run = "manual-" + new Date().toISOString().slice(0,10);
|
||||
for (const t of taskManifest) {
|
||||
const url = t.startUrl.replaceAll("$RUN_ID", run).replaceAll("$SEED", String(t.seed ?? 0));
|
||||
const tr = document.createElement("tr");
|
||||
tr.innerHTML = `
|
||||
<td><code>${t.taskId || t.id}</code><div class="notes">seed=${t.seed ?? 0} maxSteps=${t.maxSteps ?? "—"}</div></td>
|
||||
<td><span class="pill expected">${t.difficulty}</span></td>
|
||||
<td>${t.category}</td>
|
||||
<td>${escapeHtml(t.goal || t.intent)}</td>
|
||||
<td><div class="notes">${escapeHtml((t.requires || []).join(", "))}</div><div class="notes">actions: ${escapeHtml((t.actionSubsets || []).join(", "))}</div></td>
|
||||
<td><a target="_blank" href="${url}">open</a></td>
|
||||
`;
|
||||
taskTbody.appendChild(tr);
|
||||
}
|
||||
}
|
||||
async function init() {
|
||||
manifest = await fetch("manifest.json", { cache: "no-store" }).then(r => r.json());
|
||||
taskManifest = await fetch("task-manifest.json", { cache: "no-store" }).then(r => r.json()).catch(() => []);
|
||||
paint();
|
||||
paintTasks();
|
||||
}
|
||||
document.getElementById("refresh").addEventListener("click", paint);
|
||||
document.getElementById("clear").addEventListener("click", () => {
|
||||
if (!confirm("Clear pi-chrome-suite verdicts for this origin?")) return;
|
||||
for (const m of manifest) localStorage.removeItem("pi-chrome-suite:" + m.id);
|
||||
paint();
|
||||
});
|
||||
document.getElementById("copy").addEventListener("click", async () => {
|
||||
const text = JSON.stringify({ mode: modeEl.value, generatedAt: new Date().toISOString(), rows: report() }, null, 2);
|
||||
const box = document.getElementById("copyFallback");
|
||||
box.value = text;
|
||||
box.style.display = "block";
|
||||
box.focus();
|
||||
box.select();
|
||||
let copied = false;
|
||||
try { await navigator.clipboard.writeText(text); copied = true; } catch {}
|
||||
if (!copied) {
|
||||
try { copied = document.execCommand("copy"); } catch {}
|
||||
}
|
||||
box.dataset.copied = copied ? "true" : "false";
|
||||
console.log(text);
|
||||
});
|
||||
modeEl.addEventListener("change", paint);
|
||||
filterEl.addEventListener("input", paint);
|
||||
window.addEventListener("storage", paint);
|
||||
setInterval(paint, 1000);
|
||||
init();
|
||||
</script>
|
||||
</body>
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,73 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"title": "pi-chrome benchmark manifest",
|
||||
"description": "Manifest for browser-control benchmark pages. Recipes are canonical intent, not guaranteed raw tool JSON: runners may need to adapt descriptive selectors (frame/shadow) and expand path placeholders such as ${REPO_ROOT} before invoking tools.",
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["id", "file", "category", "goal", "expected", "recipe"],
|
||||
"additionalProperties": true,
|
||||
"properties": {
|
||||
"id": { "type": "string" },
|
||||
"file": { "type": "string" },
|
||||
"category": { "$ref": "#/$defs/category" },
|
||||
"difficulty": { "enum": ["L1", "L2", "L3"] },
|
||||
"gate": { "description": "Ship-gate bucket. core blocks releases, conditional requires declared capability/environment, quality is adversarial/flaky signal and should not block general ship.", "enum": ["core", "conditional", "quality"] },
|
||||
"tags": { "type": "array", "items": { "type": "string" } },
|
||||
"goal": { "type": "string" },
|
||||
"gradeSource": { "enum": ["page", "harness", "both"], "default": "page" },
|
||||
"expected": {
|
||||
"type": "object",
|
||||
"description": "Expected verdict by execution mode. CONDITIONAL means prerequisites, browser policy, hardware, or missing tool primitives decide outcome.",
|
||||
"required": ["synthetic", "trusted", "manual"],
|
||||
"properties": {
|
||||
"synthetic": { "$ref": "#/$defs/outcome" },
|
||||
"trusted": { "$ref": "#/$defs/outcome" },
|
||||
"manual": { "$ref": "#/$defs/outcome" }
|
||||
}
|
||||
},
|
||||
"manualBaseline": { "description": "Whether human baseline has been explicitly verified in this repo/environment.", "enum": ["unverified", "verified"] },
|
||||
"manualBaselineVerifiedAt": { "type": "string", "format": "date-time" },
|
||||
"recipe": {
|
||||
"type": "array",
|
||||
"description": "Ordered tool intent. Params may include descriptive selectors (e.g. frame/shadow notation) or path placeholders for runners to adapt.",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["tool", "params"],
|
||||
"properties": {
|
||||
"tool": { "type": "string" },
|
||||
"params": { "type": "object", "description": "Tool params. For upload recipes, paths are strings after runner expansion; ${REPO_ROOT} means repository root." }
|
||||
}
|
||||
}
|
||||
},
|
||||
"requires": {
|
||||
"type": "object",
|
||||
"additionalProperties": true,
|
||||
"properties": {
|
||||
"cdp": { "type": "boolean" },
|
||||
"clipboard": { "type": "boolean" },
|
||||
"touch": { "type": "boolean" },
|
||||
"secureOrigin": { "type": "boolean" },
|
||||
"fileSystem": { "type": "boolean" },
|
||||
"dialogs": { "type": "boolean" },
|
||||
"downloads": { "type": "boolean" },
|
||||
"popups": { "type": "boolean" }
|
||||
}
|
||||
},
|
||||
"prerequisites": { "type": "array", "items": { "type": "string" } },
|
||||
"notes": { "type": "array", "items": { "type": "string" } },
|
||||
"thresholds": { "type": "object", "additionalProperties": true },
|
||||
"timeoutMs": { "type": "number" },
|
||||
"verdictDelayMs": { "type": "number" },
|
||||
"flakeRisk": { "enum": ["low", "medium", "high"] }
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
"outcome": { "enum": ["PASS", "FAIL", "SKIP", "WARN", "CONDITIONAL"] },
|
||||
"category": {
|
||||
"enum": [
|
||||
"activation-gates", "agent-safety", "aria-snapshot", "canvas-svg", "clipboard", "contextmenu", "csp", "dialogs", "dom-complexity", "downloads", "drag-drop", "editing", "files", "fingerprint", "focus-keyboard", "forms", "frames", "frameworks", "hover", "keyboard", "lazy-loading", "native-controls", "observability", "pointer-humanization", "popups", "scroll", "scroll-visibility", "spa-routing", "task-completion", "timing", "touch", "trusted-input"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
# BrowserGym compatibility contract
|
||||
|
||||
Pi-chrome benchmark tasks mirror BrowserGym without importing Gymnasium.
|
||||
|
||||
## Task fields
|
||||
|
||||
Each `task-manifest.json` entry maps to `AbstractBrowserTask` concepts:
|
||||
|
||||
- `taskId` / `id` — stable namespaced id
|
||||
- `seed` — deterministic state seed
|
||||
- `viewport`, `slowMoMs`, `timeoutMs`, `locale`, `timezoneId`
|
||||
- `goal` and `goalObject` — text and OpenAI-style message objects
|
||||
- `setupUrl` — page URL that performs setup from `task`, `run`, `seed`
|
||||
- `validateScript` — JS validate hook returning reward contract
|
||||
- `cheatScript` — gold recipe used to sanity-check grader
|
||||
- `maxSteps`, `humanBaseline`, `difficulty`, `tags`
|
||||
- `actionSubsets`, `allowedActions` — BrowserGym HighLevelActionSet metadata
|
||||
|
||||
## Reset / step return
|
||||
|
||||
Runner should expose:
|
||||
|
||||
```text
|
||||
reset() -> (obs, info)
|
||||
step(action) -> (obs, reward, terminated, truncated, info)
|
||||
```
|
||||
|
||||
`terminated` means task done. `truncated` means max step or timeout.
|
||||
|
||||
## Validate return
|
||||
|
||||
In-page validators return:
|
||||
|
||||
```js
|
||||
{ reward, done, message, info }
|
||||
```
|
||||
|
||||
`reward` is incremental. Current hermetic tasks use binary terminal reward. `info.rubric` carries per-check details.
|
||||
|
||||
## Observation schema target
|
||||
|
||||
Runner observations should include BrowserGym-like keys:
|
||||
|
||||
- `chat_messages`
|
||||
- `goal`, `goal_object`
|
||||
- `open_pages_urls`, `open_pages_titles`, `active_page_index`
|
||||
- `url`, `screenshot`, `dom_object`, `axtree_object`
|
||||
- `extra_element_properties`
|
||||
- `focused_element_bid`
|
||||
- `last_action`, `last_action_error`
|
||||
- `elapsed_time`
|
||||
|
||||
## BID plan
|
||||
|
||||
BrowserGym uses `bid` attributes as stable element handles. Pi-chrome currently returns snapshot `uid`. Compatibility path:
|
||||
|
||||
1. During snapshot, assign every interactive element a deterministic `bid`.
|
||||
2. Return both `uid` and `bid`.
|
||||
3. Add `visibility`, `bbox`, `clickable`, `set_of_marks` metadata.
|
||||
4. Maintain uid↔bid mapping so agents can use either addressing mode.
|
||||
|
||||
Target attributes:
|
||||
|
||||
- `bid`
|
||||
- `browsergym_visibility_ratio`
|
||||
- `browsergym_set_of_marks`
|
||||
|
||||
## Action subsets
|
||||
|
||||
See `../browsergym-action-space.json` for BrowserGym-compatible subsets and function signatures.
|
||||
@@ -0,0 +1,79 @@
|
||||
# Historical bypass notes
|
||||
|
||||
This file predates the current `trusted:true` / `chrome.debugger` path and is
|
||||
kept as design history. For current benchmark expectations, recipes, and
|
||||
capability notes, use `../manifest.json`.
|
||||
|
||||
Older context: `pi-chrome` originally ran mostly through a content-script bridge.
|
||||
That limited the solution space — most "real-event" tricks needed the
|
||||
`chrome.debugger` API or `chrome.input.synthesizeMouseEvent`.
|
||||
|
||||
## 01–02 isTrusted click / keyboard
|
||||
**Hard.** `Event.isTrusted` is true only for events the browser dispatched
|
||||
itself. Two viable paths:
|
||||
|
||||
- **`chrome.debugger` + `Input.dispatchMouseEvent` / `Input.dispatchKeyEvent`.**
|
||||
Adds `"debugger"` permission. Chrome shows a "X is debugging this browser"
|
||||
banner, but events arrive as `isTrusted=true`. Best fix for the highest-value
|
||||
cases; gate it behind a toggle the user opts into.
|
||||
- **`chrome.input.ime.*`** — not applicable for general typing.
|
||||
|
||||
Today's `dispatchEvent` path will always fail any `isTrusted` check.
|
||||
|
||||
## 03 / 12 webdriver + fingerprint
|
||||
Cheap wins:
|
||||
- Stub `navigator.webdriver` via `chrome.scripting.executeScript({ world: "MAIN", injectImmediately: true, … })` that defines `navigator.webdriver` getter to `undefined`. Already half-supported through `initScript` on `chrome_navigate` — extend to a per-tab content script registered with `chrome.scripting.registerContentScripts({ runAt: "document_start", world: "MAIN" })`.
|
||||
- Spoof languages/plugins/permissions only if the user explicitly enables it — by default the real Chrome profile already looks legitimate. The bigger risk is `chrome_evaluate` *itself* leaving traces (Function-constructor stack frames in errors).
|
||||
|
||||
## 04 mouse entropy
|
||||
Generate a humanised pointer path before any click:
|
||||
- Bezier interpolation from current cursor pos (or a random off-target start) to the target point.
|
||||
- 20–60 steps, easing curve, ±jitter on each step, varying `movementX/Y`.
|
||||
- Implement in `clickPage` before the existing pointerdown sequence; reuse `pointerEventSequence` for each interpolated point.
|
||||
|
||||
## 05 event timing
|
||||
- Insert `await sleep(rand(40,140))` between `pointerdown` and `pointerup`.
|
||||
- Insert `await sleep(rand(80,220))` between successive synthesised clicks when
|
||||
the caller issues them in a loop.
|
||||
- Vary by ±20%.
|
||||
|
||||
## 06 click coordinates
|
||||
- After resolving target rect in `clickPage`, pick `(cx ± rand(-rw*0.3,rw*0.3), cy ± rand(-rh*0.3,rh*0.3))` instead of dead center. Already returns the chosen point — keep that.
|
||||
|
||||
## 07 pointer properties
|
||||
- Set `pressure: 0.5` for `pointerdown` (currently inits don't set pressure; browser default for mouse pointerdown is 0.5).
|
||||
- Set `pointerId: 1` for mouse (already done) and 2+ for touch.
|
||||
- Carry `movementX/Y` from the previous interpolated step (needs path state).
|
||||
|
||||
## 08 / 09 keyboard cadence + framework invariants
|
||||
- Per character: dispatch `keydown` (cancelable, fills `key` + `code` + `keyCode`), then `keypress` for printables, then `beforeinput` `{inputType:"insertText", data:ch}`, then mutate the input's value to `cur+ch` via the native setter, then `input` `{inputType:"insertText", data:ch}`, then `keyup`. Sleep 40–120 ms between chars.
|
||||
- Today the bridge fills the entire value in one go and emits one `beforeinput`/`input`. Replace with the per-char loop above.
|
||||
- Make sure not to dispatch `compositionstart`/`compositionend` for plain ASCII.
|
||||
|
||||
## 10 user activation
|
||||
**Impossible with synthetic events.** `navigator.userActivation.isActive` only
|
||||
flips on browser-trusted events. Same fix as 01/02 — `chrome.debugger` path.
|
||||
Document the limitation; offer a `useDebuggerForActivationGates` opt-in.
|
||||
|
||||
## 11 honeypot
|
||||
Pure agent-side concern. Before filling/clicking by selector, the bridge could:
|
||||
- Reject targets with `display:none`, `visibility:hidden`, `aria-hidden=true`,
|
||||
or off-screen position (`getBoundingClientRect()` outside viewport bounds and
|
||||
not scrolled-into-view), unless the caller passes `force: true`.
|
||||
- That logic largely exists (`elementVisible` / `occludedBy` are returned) —
|
||||
promote it from informational to a guard.
|
||||
|
||||
## 13 focus order
|
||||
- When clicking a focusable element, dispatch `pointerdown` first; only after
|
||||
`mousedown` defaults run let focus settle naturally (browser will move focus
|
||||
on the synthetic `mousedown` only if `isTrusted` is true — alas, no). Manual
|
||||
workaround: dispatch the pointer/mouse sequence, then explicitly call
|
||||
`target.focus({ preventScroll: true })`, then dispatch `focus` with
|
||||
`{ relatedTarget: previouslyFocused }`. Set `:focus-visible` heuristic via
|
||||
`target.blur(); target.focus({ focusVisible: false })` (Chrome 124+).
|
||||
|
||||
## 14 wheel scroll
|
||||
- For scroll operations, dispatch sequential `wheel` events with `deltaY`
|
||||
chunks (e.g., 60–120 per tick), and let the browser scroll, instead of
|
||||
setting `scrollTop` directly. Provide `chrome_scroll({ uid, dy, dx })` as a
|
||||
first-class tool.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Calibration profiles
|
||||
|
||||
Benchmark results depend on Chrome profile, OS, input device, and extension mode.
|
||||
Record these with every run:
|
||||
|
||||
- OS and version
|
||||
- Chrome version
|
||||
- pi-chrome package version
|
||||
- companion extension version
|
||||
- trusted mode: off / auto / on
|
||||
- input hardware: trackpad vs detent mouse, touch support
|
||||
- permissions: clipboard, fullscreen, downloads
|
||||
- viewport size and deviceScaleFactor
|
||||
- loaded privacy/security extensions
|
||||
|
||||
Known environment-sensitive areas:
|
||||
|
||||
- WebGL fingerprint can warn/fail in VMs, remote desktops, GPU-disabled Chrome.
|
||||
- Scroll momentum expects trackpad-like decay; detent mouse wheels may fail manually.
|
||||
- Touch tests require touch events enabled/supported.
|
||||
- Dialog/download/file-picker tests require native browser UI handling or manual intervention.
|
||||
- Stack traces vary across Chrome versions; tests should only fail concrete automation URLs/Runtime frames.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Recipe runner spec
|
||||
|
||||
Future Node/Pi runner should:
|
||||
|
||||
1. Read `manifest.json`.
|
||||
2. Probe environment capabilities (`trusted`, `clipboard`, `touch`, `dialogs`, `downloads`, `fileSystem`, `strictCspFallback`).
|
||||
3. For each entry:
|
||||
- read `gate` (`core`, `conditional`, `quality`) and score in that bucket
|
||||
- skip if `requires` is unsatisfied and expected is `CONDITIONAL`
|
||||
- navigate to challenge URL
|
||||
- execute `recipe` in order
|
||||
- wait `verdictDelayMs || 500`
|
||||
- evaluate `JSON.stringify({v:window.__verdict,r:window.__reason,d:window.Challenge?.state?.details,e:window.__events?.slice(-20)})`, except strict-CSP pages where eval is expected to fail; read persisted verdict from dashboard/localStorage after leaving the CSP page
|
||||
- compare to `expected[mode]`
|
||||
4. Output:
|
||||
- Markdown summary
|
||||
- JSON details
|
||||
- JUnit XML for CI
|
||||
|
||||
Runner must adapt recipe intent:
|
||||
|
||||
- expand `${REPO_ROOT}` path placeholders
|
||||
- adapt unsupported shadow/iframe selector notation to snapshot uids or evaluate fallback
|
||||
- preserve hook install ordering for console/network capture tests
|
||||
- substitute dynamic tab ids from `chrome_tab list` recipes
|
||||
- support screenshot/coordinate fallback for strict CSP pages where snapshot/evaluate are blocked
|
||||
- record whether trusted/CDP path was used
|
||||
|
||||
Long-horizon task runner should read `task-manifest.json`, replace `$RUN_ID`, solve task, then evaluate task grader expression.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Benchmark scoring
|
||||
|
||||
## Unit challenges (`manifest.json`)
|
||||
|
||||
Score actual verdict against `expected[mode]`, not raw PASS count.
|
||||
|
||||
Gate buckets:
|
||||
|
||||
- `core`: headline release gate. Exact expected verdict required.
|
||||
- `conditional`: gate only when declared capability/prerequisite is present; otherwise report skipped/conditional.
|
||||
- `quality`: adversarial humanization/fingerprint signal. Report trend and regressions, but do not block general release unless explicitly promoted.
|
||||
|
||||
Expected values:
|
||||
|
||||
- `PASS` / `FAIL` expected: exact match required inside active gate bucket.
|
||||
- `CONDITIONAL`: exclude from core headline score unless environment prerequisite is declared present.
|
||||
- `SKIP` / `WARN`: report separately.
|
||||
|
||||
Recommended flake policy:
|
||||
|
||||
- Run each non-destructive challenge up to 2 retries.
|
||||
- Take best verdict only for known-flaky timing/scroll tests.
|
||||
- Always keep all reasons/events in detailed report.
|
||||
|
||||
Difficulty weighting:
|
||||
|
||||
- L1 = 1
|
||||
- L2 = 2
|
||||
- L3 = 3
|
||||
|
||||
Report both unweighted and weighted score. Also report per-category score.
|
||||
|
||||
## Long-horizon tasks (`task-manifest.json`)
|
||||
|
||||
Task score is deterministic grader PASS / not PASS. Record:
|
||||
|
||||
- action count
|
||||
- wall time
|
||||
- tools used
|
||||
- observation mode (`snapshot`, screenshot, accessibility, evaluate)
|
||||
- whether direct state mutation/evaluate was allowed
|
||||
- final grader reason
|
||||
|
||||
Do not use LLM judge for hermetic tasks unless grader cannot express the target.
|
||||
@@ -0,0 +1,49 @@
|
||||
window.ChoreDeskCheats = {
|
||||
"choredesk-l1-ticket-priority": [
|
||||
{ tool: "chrome_fill", params: { selector: "select[aria-label=\"T-104 priority\"]", text: "High" } },
|
||||
{ tool: "chrome_fill", params: { selector: "select[aria-label=\"T-104 status\"]", text: "In Progress" } },
|
||||
{ tool: "chrome_click", params: { selector: "button[aria-label=\"save tickets T-104\"]" } }
|
||||
],
|
||||
"choredesk-l2-refund-message": [
|
||||
{ tool: "chrome_click", params: { selector: "a[data-view=\"customers\"]" } },
|
||||
{ tool: "chrome_click", params: { selector: "a[data-view=\"orders\"]" } },
|
||||
{ tool: "chrome_click", params: { selector: "a[data-view=\"messages\"]" } },
|
||||
{ tool: "chrome_fill", params: { selector: "#msgSubject", text: "Refund review ORD-778" } },
|
||||
{ tool: "chrome_fill", params: { selector: "#msgBody", text: "Noah Kim noah.kim@example.com order ORD-778 total $84.20" } },
|
||||
{ tool: "chrome_click", params: { selector: "#sendMsg" } }
|
||||
],
|
||||
"choredesk-l3-restock-ticket": [
|
||||
{ tool: "chrome_click", params: { selector: "a[data-view=\"catalog\"]" } },
|
||||
{ tool: "chrome_click", params: { selector: "a[data-view=\"inventory\"]" } },
|
||||
{ tool: "chrome_fill", params: { selector: "input[aria-label=\"AST-LAMP reorder quantity\"]", text: "12" } },
|
||||
{ tool: "chrome_click", params: { selector: "button[aria-label=\"save inventory AST-LAMP\"]" } },
|
||||
{ tool: "chrome_click", params: { selector: "a[data-view=\"tickets\"]" } },
|
||||
{ tool: "chrome_fill", params: { selector: "input[aria-label=\"T-205 internal note\"]", text: "reordered 12 AST-LAMP" } },
|
||||
{ tool: "chrome_click", params: { selector: "button[aria-label=\"save tickets T-205\"]" } }
|
||||
],
|
||||
run: {
|
||||
"choredesk-l1-ticket-priority": async () => {
|
||||
await window.__choredesk.cheatHelpers.route("tickets");
|
||||
window.__choredesk.cheatHelpers.set("select[aria-label=\"T-104 priority\"]", "High");
|
||||
window.__choredesk.cheatHelpers.set("select[aria-label=\"T-104 status\"]", "In Progress");
|
||||
document.querySelector("button[aria-label=\"save tickets T-104\"]").click();
|
||||
},
|
||||
"choredesk-l2-refund-message": async () => {
|
||||
await window.__choredesk.cheatHelpers.route("customers");
|
||||
await window.__choredesk.cheatHelpers.route("orders");
|
||||
await window.__choredesk.cheatHelpers.route("messages");
|
||||
window.__choredesk.cheatHelpers.set("#msgSubject", "Refund review ORD-778");
|
||||
window.__choredesk.cheatHelpers.set("#msgBody", "Noah Kim noah.kim@example.com order ORD-778 total $84.20");
|
||||
document.querySelector("#sendMsg").click();
|
||||
},
|
||||
"choredesk-l3-restock-ticket": async () => {
|
||||
await window.__choredesk.cheatHelpers.route("catalog");
|
||||
await window.__choredesk.cheatHelpers.route("inventory");
|
||||
window.__choredesk.cheatHelpers.set("input[aria-label=\"AST-LAMP reorder quantity\"]", "12");
|
||||
document.querySelector("button[aria-label=\"save inventory AST-LAMP\"]").click();
|
||||
await window.__choredesk.cheatHelpers.route("tickets");
|
||||
window.__choredesk.cheatHelpers.set("input[aria-label=\"T-205 internal note\"]", "reordered 12 AST-LAMP");
|
||||
document.querySelector("button[aria-label=\"save tickets T-205\"]").click();
|
||||
}
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,239 @@
|
||||
<!doctype html>
|
||||
<meta charset="utf-8">
|
||||
<title>ChoreDesk hermetic task site</title>
|
||||
<link rel="stylesheet" href="../../_style.css">
|
||||
<style>
|
||||
main { max-width: 1180px; }
|
||||
.top { display:flex; gap:12px; align-items:center; flex-wrap:wrap; }
|
||||
.tabs { display:flex; gap:8px; margin:16px 0; flex-wrap:wrap; }
|
||||
.tab { border:1px solid #444; border-radius:999px; padding:6px 10px; color:#eee; text-decoration:none; background:#222; }
|
||||
.tab[aria-current="page"] { background:#2b4b6b; }
|
||||
.card { border:1px solid #333; border-radius:10px; background:#202020; padding:14px; }
|
||||
table { font-size:13px; }
|
||||
td input, td select { width: 100%; box-sizing:border-box; }
|
||||
textarea { width:100%; min-height:110px; background:#111; color:#eee; border:1px solid #555; border-radius:6px; padding:8px; }
|
||||
.muted { color:#aaa; font-size:12px; }
|
||||
.taskbar { border:1px solid #4b3b16; background:#221b0d; border-radius:10px; padding:12px; margin:12px 0; }
|
||||
.verdict { display:inline-block; border-radius:999px; padding:2px 8px; font-weight:700; background:#444; }
|
||||
.verdict.PASS { background:#1f7a1f; }
|
||||
.verdict.FAIL { background:#a11; }
|
||||
.verdict.PENDING { background:#444; }
|
||||
</style>
|
||||
<body>
|
||||
<main>
|
||||
<div class="top">
|
||||
<h1 style="margin-right:auto">ChoreDesk</h1>
|
||||
<button id="reset" data-task-role="reset-button">Reset run</button>
|
||||
<button id="grade" data-task-role="grade-button">Grade now</button>
|
||||
<button id="cheat" data-task-role="cheat-button">Run cheat</button>
|
||||
</div>
|
||||
<div class="taskbar">
|
||||
<div><b id="taskId"></b> <span id="difficulty"></span> <span id="verdict" class="verdict PENDING">PENDING</span> <span id="reward" class="muted"></span></div>
|
||||
<p id="instruction"></p>
|
||||
<pre id="reason" class="muted"></pre>
|
||||
</div>
|
||||
<nav class="tabs" aria-label="ChoreDesk sections"></nav>
|
||||
<section id="view"></section>
|
||||
</main>
|
||||
<script src="cheats.js"></script>
|
||||
<script>
|
||||
(function () {
|
||||
const params = new URLSearchParams(location.search);
|
||||
if (!params.get("run")) {
|
||||
params.set("run", crypto.randomUUID ? crypto.randomUUID() : String(Date.now()));
|
||||
history.replaceState(null, "", `?${params}`);
|
||||
}
|
||||
const task = params.get("task") || "choredesk-l1-ticket-priority";
|
||||
const run = params.get("run");
|
||||
const seed = Number(params.get("seed") || 0);
|
||||
const nonce = params.get("nonce") || (crypto.randomUUID ? crypto.randomUUID() : Math.random().toString(36).slice(2));
|
||||
if (!params.get("nonce")) { params.set("nonce", nonce); history.replaceState(null, "", `?${params}`); }
|
||||
const storeKey = `pi-chrome-task-state:${task}:${run}:${seed}:${nonce}`;
|
||||
const verdictKey = `pi-chrome-task:${task}:${run}`;
|
||||
const views = ["tickets", "customers", "orders", "catalog", "inventory", "messages", "audit"];
|
||||
let view = params.get("view") || "tickets";
|
||||
let bidCounter = 0;
|
||||
|
||||
const TASKS = {
|
||||
"choredesk-l1-ticket-priority": {
|
||||
difficulty: "L1 atomic",
|
||||
instruction: "Set ticket T-104 priority to High and status to In Progress. Save the ticket.",
|
||||
rubric(s) {
|
||||
const t = s.tickets.find(x => x.id === "T-104");
|
||||
return [
|
||||
{ id:"priorityHigh", weight:.33, pass:t?.priority === "High", description:"T-104 priority is High", reason:`T-104 priority=${t?.priority}` },
|
||||
{ id:"statusInProgress", weight:.33, pass:t?.status === "In Progress", description:"T-104 status is In Progress", reason:`T-104 status=${t?.status}` },
|
||||
{ id:"saved", weight:.34, pass:s.audit.some(a => a.kind === "ticket-save" && a.id === "T-104" && a.via === "ui-event"), description:"T-104 saved through UI", reason:"no UI save audit for T-104" }
|
||||
];
|
||||
}
|
||||
},
|
||||
"choredesk-l2-refund-message": {
|
||||
difficulty: "L2 compositional",
|
||||
instruction: "A customer named Noah Kim reports a duplicate charge. Find Noah's email and order ORD-778 total, then send Billing a message with subject 'Refund review ORD-778'. Body must include Noah's email and the exact order total.",
|
||||
rubric(s) {
|
||||
const msg = s.messages.find(m => m.to === "Billing" && m.subject.trim() === "Refund review ORD-778");
|
||||
return [
|
||||
{ id:"visitedCustomers", weight:.15, pass:!!s.viewsVisited.customers, description:"Opened Customers view", reason:"never opened Customers view" },
|
||||
{ id:"visitedOrders", weight:.15, pass:!!s.viewsVisited.orders, description:"Opened Orders view", reason:"never opened Orders view" },
|
||||
{ id:"subject", weight:.20, pass:!!msg, description:"Billing message with exact subject exists", reason:"missing Billing message with exact subject" },
|
||||
{ id:"email", weight:.25, pass:!!msg && /\bnoah\.kim@example\.com\b/i.test(msg.body), description:"Body includes Noah email", reason:"message body missing Noah email" },
|
||||
{ id:"total", weight:.25, pass:!!msg && /(?<![\d.])\$?84\.20\b/.test(msg.body), description:"Body includes exact ORD-778 total", reason:"message body missing exact ORD-778 total 84.20" }
|
||||
];
|
||||
}
|
||||
},
|
||||
"choredesk-l3-restock-ticket": {
|
||||
difficulty: "L3 cross-page tedious",
|
||||
instruction: "Ticket T-205 says Aster Lamp is delayed. Find the Aster Lamp SKU in Catalog, set its Inventory reorder quantity to 12, then add an internal note to T-205 saying 'reordered 12 AST-LAMP'.",
|
||||
rubric(s) {
|
||||
const inv = s.inventory.find(x => x.sku === "AST-LAMP");
|
||||
const t = s.tickets.find(x => x.id === "T-205");
|
||||
return [
|
||||
{ id:"visitedCatalog", weight:.15, pass:!!s.viewsVisited.catalog, description:"Opened Catalog view", reason:"never opened Catalog view" },
|
||||
{ id:"visitedInventory", weight:.15, pass:!!s.viewsVisited.inventory, description:"Opened Inventory view", reason:"never opened Inventory view" },
|
||||
{ id:"inventory", weight:.25, pass:Number(inv?.reorderQty) === 12 && s.audit.some(a => a.kind === "inventory-save" && a.id === "AST-LAMP" && a.via === "ui-event"), description:"AST-LAMP reorderQty=12 saved through UI", reason:`AST-LAMP reorderQty=${inv?.reorderQty}; missing UI save audit` },
|
||||
{ id:"ticketNote", weight:.25, pass:!!t && t.note.includes("reordered 12 AST-LAMP"), description:"T-205 note contains exact phrase", reason:"T-205 internal note missing exact phrase" },
|
||||
{ id:"ticketSaved", weight:.20, pass:s.audit.some(a => a.kind === "ticket-save" && a.id === "T-205" && a.via === "ui-event"), description:"T-205 saved through UI", reason:"no UI save audit for T-205" }
|
||||
];
|
||||
}
|
||||
}
|
||||
};
|
||||
const SEED = {
|
||||
tickets: [
|
||||
{ id:"T-101", customer:"Rosa Patel", subject:"Where is my desk mat?", priority:"Normal", status:"Open", note:"" },
|
||||
{ id:"T-104", customer:"Iris Zhao", subject:"Invoice address typo", priority:"Low", status:"Open", note:"" },
|
||||
{ id:"T-205", customer:"Mateo Silva", subject:"Aster Lamp delayed", priority:"Normal", status:"Open", note:"" },
|
||||
{ id:"T-309", customer:"Noah Kim", subject:"Duplicate charge", priority:"Normal", status:"Open", note:"" }
|
||||
],
|
||||
customers: [
|
||||
{ name:"Noah Kim", email:"noah.kim@example.com", tier:"Gold" },
|
||||
{ name:"Rosa Patel", email:"rosa.patel@example.com", tier:"Silver" },
|
||||
{ name:"Iris Zhao", email:"iris.zhao@example.com", tier:"Bronze" },
|
||||
{ name:"Mateo Silva", email:"mateo.silva@example.com", tier:"Gold" }
|
||||
],
|
||||
orders: [
|
||||
{ id:"ORD-778", customer:"Noah Kim", item:"Cable Kit", total:"$84.20", status:"Paid" },
|
||||
{ id:"ORD-812", customer:"Rosa Patel", item:"Desk Mat", total:"$28.00", status:"Shipped" },
|
||||
{ id:"ORD-919", customer:"Mateo Silva", item:"Aster Lamp", total:"$134.50", status:"Delayed" }
|
||||
],
|
||||
catalog: [
|
||||
{ item:"Aster Lamp", sku:"AST-LAMP", vendor:"Northwind Lighting" },
|
||||
{ item:"Desk Mat", sku:"DSK-MAT", vendor:"Woven Works" },
|
||||
{ item:"Cable Kit", sku:"CBL-KIT", vendor:"Copper Lane" }
|
||||
],
|
||||
inventory: [
|
||||
{ sku:"AST-LAMP", onHand:0, reorderQty:0 },
|
||||
{ sku:"DSK-MAT", onHand:42, reorderQty:0 },
|
||||
{ sku:"CBL-KIT", onHand:6, reorderQty:0 }
|
||||
],
|
||||
messages: [],
|
||||
audit: [],
|
||||
viewsVisited: {},
|
||||
actionCount: 0,
|
||||
startedAt: Date.now()
|
||||
};
|
||||
let state = loadState();
|
||||
function clone(x){ return JSON.parse(JSON.stringify(x)); }
|
||||
function loadState(){ const raw = localStorage.getItem(storeKey); return raw ? JSON.parse(raw) : clone(SEED); }
|
||||
function saveState(){ localStorage.setItem(storeKey, JSON.stringify(state)); }
|
||||
function countAction(kind){ state.actionCount = (state.actionCount || 0) + 1; state.lastAction = kind; }
|
||||
function audit(entry){ state.audit.push({ ts:new Date().toISOString(), via:"ui-event", trusted:!!entry.trusted, ...entry }); saveState(); grade(false); }
|
||||
function rubric(){ return (TASKS[task] || TASKS["choredesk-l1-ticket-priority"]).rubric(state); }
|
||||
function validate(){
|
||||
const items = rubric();
|
||||
const total = items.reduce((a,x)=>a+x.weight,0) || 1;
|
||||
const got = items.filter(x=>x.pass).reduce((a,x)=>a+x.weight,0);
|
||||
const reward = Math.round((got / total) * 1000) / 1000;
|
||||
const done = reward === 1;
|
||||
const bad = items.filter(x=>!x.pass).map(x=>x.reason || `${x.id} failed`);
|
||||
return { reward, done, message: done ? "all deterministic checks passed" : bad.join("; "), info: { task, seed, run, actionCount:state.actionCount||0, elapsedMs:Date.now()-(state.startedAt||Date.now()), viewsVisited:state.viewsVisited, rubric:items, state } };
|
||||
}
|
||||
function observation(){
|
||||
return { goal:TASKS[task]?.instruction, url:location.href, activeView:view, title:document.title, reward:window.__taskReward, actionCount:state.actionCount||0, viewsVisited:state.viewsVisited, focusedElementBid:document.activeElement?.getAttribute("bid") || null };
|
||||
}
|
||||
function setVerdict(verdict, reason, validation = validate()){
|
||||
window.__taskVerdict = verdict;
|
||||
window.__taskReason = reason;
|
||||
window.__taskValidation = validation;
|
||||
window.__taskReward = validation.reward;
|
||||
window.__taskTerminated = validation.done;
|
||||
window.__taskTruncated = (state.actionCount || 0) >= Number(params.get("maxSteps") || 9999);
|
||||
window.__taskInfo = validation.info;
|
||||
localStorage.setItem(verdictKey, JSON.stringify({ id: task, run, seed, verdict, reason, validation, state, ts: Date.now() }));
|
||||
document.getElementById("verdict").textContent = verdict;
|
||||
document.getElementById("verdict").className = "verdict " + verdict;
|
||||
document.getElementById("reward").textContent = `reward=${validation.reward} actions=${state.actionCount || 0}`;
|
||||
document.getElementById("reason").textContent = reason.join("\n");
|
||||
}
|
||||
function grade(showPass=true){ const v = validate(); if (!v.done) setVerdict(showPass ? "FAIL" : "PENDING", v.info.rubric.filter(x=>!x.pass).map(x=>`${x.id}: ${x.reason}`), v); else setVerdict("PASS", [v.message], v); return v; }
|
||||
function route(v){ view = v; state.viewsVisited[v] = (state.viewsVisited[v] || 0) + 1; countAction("route:" + v); saveState(); history.replaceState(null, "", `?task=${encodeURIComponent(task)}&run=${encodeURIComponent(run)}&seed=${seed}&nonce=${nonce}&view=${encodeURIComponent(view)}`); render(); }
|
||||
function renderShell(){
|
||||
const spec = TASKS[task] || TASKS["choredesk-l1-ticket-priority"];
|
||||
document.getElementById("taskId").textContent = task;
|
||||
document.getElementById("difficulty").textContent = spec.difficulty;
|
||||
document.getElementById("instruction").textContent = spec.instruction;
|
||||
const nav = document.querySelector(".tabs"); nav.innerHTML = "";
|
||||
for (const v of views) { const a = document.createElement("a"); a.href="#"; a.className="tab"; a.textContent=v[0].toUpperCase()+v.slice(1); a.dataset.view=v; a.setAttribute("data-task-role", "nav-tab"); if(v===view)a.setAttribute("aria-current","page"); a.addEventListener("click", e=>{e.preventDefault(); route(v);}); nav.appendChild(a); }
|
||||
}
|
||||
function render(){
|
||||
renderShell(); bidCounter = 0;
|
||||
const root = document.getElementById("view"); root.innerHTML = "";
|
||||
if (view === "tickets") root.appendChild(tableEditor("tickets", state.tickets, ["id","customer","subject","priority","status","note"], saveTicket));
|
||||
if (view === "customers") root.appendChild(readTable(state.customers, ["name","email","tier"]));
|
||||
if (view === "orders") root.appendChild(readTable(state.orders, ["id","customer","item","total","status"]));
|
||||
if (view === "catalog") root.appendChild(readTable(state.catalog, ["item","sku","vendor"]));
|
||||
if (view === "inventory") root.appendChild(tableEditor("inventory", state.inventory, ["sku","onHand","reorderQty"], saveInventory));
|
||||
if (view === "messages") root.appendChild(messageComposer());
|
||||
if (view === "audit") root.appendChild(readTable(state.audit.map((entry,i)=>({ n:i+1, entry: JSON.stringify(entry) })), ["n","entry"]));
|
||||
assignBids(); grade(false);
|
||||
}
|
||||
function readTable(rows, cols){
|
||||
const wrap = document.createElement("div"); wrap.className="card";
|
||||
wrap.innerHTML = `<table><thead><tr>${cols.map(c=>`<th>${c}</th>`).join("")}</tr></thead><tbody></tbody></table>`;
|
||||
const tb = wrap.querySelector("tbody");
|
||||
for (const r of rows) { const tr = document.createElement("tr"); tr.innerHTML = cols.map(c => `<td>${escapeHtml(r[c] ?? "")}</td>`).join(""); tb.appendChild(tr); }
|
||||
return wrap;
|
||||
}
|
||||
function tableEditor(kind, rows, cols, onSave){
|
||||
const wrap = document.createElement("div"); wrap.className="card";
|
||||
wrap.innerHTML = `<table><thead><tr>${cols.map(c=>`<th>${c}</th>`).join("")}<th>save</th></tr></thead><tbody></tbody></table>`;
|
||||
const tb = wrap.querySelector("tbody");
|
||||
rows.forEach((r) => {
|
||||
const key = r.id || r.sku; const tr = document.createElement("tr"); tr.dataset.itemId = key; if (r.id) tr.dataset.ticketId = r.id; if (r.sku) tr.dataset.sku = r.sku;
|
||||
for (const c of cols) {
|
||||
const td = document.createElement("td");
|
||||
const stableAttrs = `${r.id ? `data-ticket-id="${r.id}"` : ""} ${r.sku ? `data-sku="${r.sku}"` : ""}`;
|
||||
if (c === "priority") td.innerHTML = `<select data-item-id="${key}" ${stableAttrs} data-col="${c}" aria-label="${key} priority"><option>Low</option><option>Normal</option><option>High</option></select>`;
|
||||
else if (c === "status") td.innerHTML = `<select data-item-id="${key}" ${stableAttrs} data-col="${c}" aria-label="${key} status"><option>Open</option><option>In Progress</option><option>Waiting</option><option>Closed</option></select>`;
|
||||
else if (c === "note") td.innerHTML = `<input data-item-id="${key}" ${stableAttrs} data-col="${c}" aria-label="${key} internal note">`;
|
||||
else if (c === "reorderQty") td.innerHTML = `<input type="number" data-item-id="${key}" ${stableAttrs} data-col="${c}" aria-label="${key} reorder quantity">`;
|
||||
else td.textContent = r[c];
|
||||
tr.appendChild(td);
|
||||
}
|
||||
const save = document.createElement("td"); save.innerHTML = `<button data-save-id="${key}" data-task-role="save-button" aria-label="save ${kind} ${key}">Save</button>`; tr.appendChild(save); tb.appendChild(tr);
|
||||
});
|
||||
wrap.querySelectorAll("[data-item-id]").forEach(el => { const row = rows.find(r => (r.id || r.sku) === el.dataset.itemId); el.value = row[el.dataset.col]; el.addEventListener("input", () => { row[el.dataset.col] = el.value; }); });
|
||||
wrap.querySelectorAll("[data-save-id]").forEach(btn => btn.addEventListener("click", e => onSave(rows.find(r => (r.id || r.sku) === btn.dataset.saveId), e)));
|
||||
return wrap;
|
||||
}
|
||||
function saveTicket(t, e){ countAction("save-ticket"); saveState(); audit({ kind:"ticket-save", id:t.id, trusted:e?.isTrusted }); render(); }
|
||||
function saveInventory(i, e){ countAction("save-inventory"); i.reorderQty = Number(i.reorderQty); saveState(); audit({ kind:"inventory-save", id:i.sku, trusted:e?.isTrusted }); render(); }
|
||||
function messageComposer(){
|
||||
const wrap = document.createElement("div"); wrap.className="card";
|
||||
wrap.innerHTML = `<label>From <input id="msgFrom" aria-label="message from" value="Agent"></label><br><br><label>To <select id="msgTo"><option>Billing</option><option>Support</option><option>Warehouse</option></select></label><br><br><label>Subject <input id="msgSubject" aria-label="message subject"></label><br><br><label>Body <textarea id="msgBody" aria-label="message body"></textarea></label><br><button id="sendMsg" data-task-role="send-message">Send message</button><h3>Sent messages</h3><div id="sent"></div>`;
|
||||
wrap.querySelector("#sendMsg").addEventListener("click", e => { countAction("send-message"); state.messages.push({ from:msgFrom.value, to:msgTo.value, subject:msgSubject.value, body:msgBody.value, trusted:e.isTrusted }); audit({ kind:"message-send", id:msgSubject.value, trusted:e.isTrusted }); render(); });
|
||||
wrap.querySelector("#sent").appendChild(readTable(state.messages, ["from","to","subject","body"]));
|
||||
return wrap;
|
||||
}
|
||||
function assignBids(){ document.querySelectorAll("button,input,select,textarea,a[href]").forEach(el => { const id = `b${(++bidCounter).toString(36)}`; el.setAttribute("bid", id); el.setAttribute("data-bid", id); const r = el.getBoundingClientRect(); el.setAttribute("browsergym_visibility_ratio", (r.width > 0 && r.height > 0 ? 1 : 0).toString()); }); }
|
||||
function escapeHtml(s){ return String(s).replace(/[&<>"']/g, c => ({"&":"&","<":"<",">":">","\"":""","'":"'"}[c])); }
|
||||
async function runCheat(){ const fn = window.ChoreDeskCheats?.run?.[task]; if (fn) await fn(); else for (const step of (window.ChoreDeskCheats?.[task] || [])) console.log("cheat step", step); grade(true); }
|
||||
document.getElementById("reset").addEventListener("click", () => { localStorage.removeItem(storeKey); localStorage.removeItem(verdictKey); state = loadState(); render(); });
|
||||
document.getElementById("grade").addEventListener("click", () => grade(true));
|
||||
document.getElementById("cheat").addEventListener("click", () => runCheat());
|
||||
window.__task = { id:task, run, seed, goal:TASKS[task]?.instruction, difficulty:TASKS[task]?.difficulty, step:() => { const v=grade(false); return { observation:observation(), reward:v.reward, terminated:v.done, truncated:window.__taskTruncated, info:v.info }; }, reset:() => { localStorage.removeItem(storeKey); state=loadState(); render(); return { observation:observation(), info:{task,run,seed} }; }, rubric:() => rubric(), cheat:runCheat, validate, teardown:()=>undefined };
|
||||
window.__choredesk = window.__task;
|
||||
window.__choredesk.cheatHelpers = { route: async (v) => { route(v); await new Promise(r => setTimeout(r, 0)); }, set: (sel, value) => { const el = document.querySelector(sel); if (!el) throw new Error('cheat selector missing: ' + sel); el.value = value; el.dispatchEvent(new Event('input', { bubbles:true })); el.dispatchEvent(new Event('change', { bubbles:true })); } };
|
||||
state.viewsVisited[view] = (state.viewsVisited[view] || 0) + 1; saveState(); render();
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
# Serve the test suite on a fixed local port so pi-chrome can drive it.
|
||||
cd "$(dirname "$0")"
|
||||
PORT="${PORT:-8765}"
|
||||
echo "serving http://127.0.0.1:${PORT}/"
|
||||
exec python3 -m http.server "${PORT}" --bind 127.0.0.1
|
||||
@@ -0,0 +1,416 @@
|
||||
[
|
||||
{
|
||||
"id": "choredesk-l1-ticket-priority",
|
||||
"site": "choredesk",
|
||||
"startUrl": "scenarios/choredesk/index.html?task=choredesk-l1-ticket-priority&run=$RUN_ID&seed=$SEED",
|
||||
"difficulty": "L1",
|
||||
"category": "enterprise-crud",
|
||||
"intent": "Set ticket T-104 priority to High and status to In Progress, then save it.",
|
||||
"expectedActions": {
|
||||
"min": 3,
|
||||
"max": 6
|
||||
},
|
||||
"requires": [
|
||||
"snapshot",
|
||||
"click",
|
||||
"select/change",
|
||||
"form editing"
|
||||
],
|
||||
"grader": {
|
||||
"type": "inPageProgrammatic",
|
||||
"expression": "window.__taskVerdict || (window.__taskValidation?.done ? \"PASS\" : \"FAIL\")",
|
||||
"passValue": "PASS"
|
||||
},
|
||||
"browserGymLike": {
|
||||
"observation": [
|
||||
"dom",
|
||||
"accessibility_tree",
|
||||
"screenshot_optional"
|
||||
],
|
||||
"actionSpace": [
|
||||
"click",
|
||||
"type",
|
||||
"select",
|
||||
"evaluate_optional"
|
||||
],
|
||||
"reset": "navigate startUrl with fresh $RUN_ID or click Reset run",
|
||||
"stepReturn": "obs,reward,terminated,truncated,info"
|
||||
},
|
||||
"viewport": {
|
||||
"width": 1280,
|
||||
"height": 720
|
||||
},
|
||||
"slowMoMs": 1000,
|
||||
"timeoutMs": 5000,
|
||||
"locale": null,
|
||||
"timezoneId": null,
|
||||
"humanBaseline": null,
|
||||
"actionSubsets": [
|
||||
"bid",
|
||||
"nav",
|
||||
"tab",
|
||||
"infeas",
|
||||
"chat"
|
||||
],
|
||||
"rewardShape": "binary",
|
||||
"taskId": "choredesk.L1.ticket-priority",
|
||||
"seed": 101,
|
||||
"maxSteps": 8,
|
||||
"allowedActions": [
|
||||
"click",
|
||||
"fill",
|
||||
"select_option",
|
||||
"goto",
|
||||
"report_infeasible",
|
||||
"send_msg_to_user"
|
||||
],
|
||||
"setupUrl": "scenarios/choredesk/index.html?task=choredesk-l1-ticket-priority&run=$RUN_ID&seed=$SEED",
|
||||
"validateScript": "scenarios/choredesk/index.html#window.__choredesk.validate()",
|
||||
"cheatScript": "scenarios/choredesk/cheats.js#choredesk-l1-ticket-priority",
|
||||
"tags": [
|
||||
"form-fill",
|
||||
"ticket",
|
||||
"select"
|
||||
],
|
||||
"rubric": [
|
||||
{
|
||||
"id": "priorityHigh",
|
||||
"weight": 0.33,
|
||||
"description": "T-104 priority is High"
|
||||
},
|
||||
{
|
||||
"id": "statusInProgress",
|
||||
"weight": 0.33,
|
||||
"description": "T-104 status is In Progress"
|
||||
},
|
||||
{
|
||||
"id": "saved",
|
||||
"weight": 0.34,
|
||||
"description": "T-104 was saved"
|
||||
}
|
||||
],
|
||||
"goal": "Set ticket T-104 priority to High and status to In Progress, then save it.",
|
||||
"goalObject": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "Set ticket T-104 priority to High and status to In Progress, then save it."
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"gradeSource": "page"
|
||||
},
|
||||
{
|
||||
"id": "choredesk-l2-refund-message",
|
||||
"site": "choredesk",
|
||||
"startUrl": "scenarios/choredesk/index.html?task=choredesk-l2-refund-message&run=$RUN_ID&seed=$SEED",
|
||||
"difficulty": "L2",
|
||||
"category": "cross-page-memory",
|
||||
"intent": "Resolve a customer billing task by looking up customer email and order total on separate pages, then composing a message with exact required fields.",
|
||||
"expectedActions": {
|
||||
"min": 8,
|
||||
"max": 14
|
||||
},
|
||||
"requires": [
|
||||
"multi-page navigation",
|
||||
"table lookup",
|
||||
"working memory",
|
||||
"form composition"
|
||||
],
|
||||
"grader": {
|
||||
"type": "inPageProgrammatic",
|
||||
"expression": "window.__taskVerdict || (window.__taskValidation?.done ? \"PASS\" : \"FAIL\")",
|
||||
"passValue": "PASS"
|
||||
},
|
||||
"browserGymLike": {
|
||||
"observation": [
|
||||
"dom",
|
||||
"accessibility_tree",
|
||||
"screenshot_optional"
|
||||
],
|
||||
"actionSpace": [
|
||||
"click",
|
||||
"type",
|
||||
"select",
|
||||
"evaluate_optional"
|
||||
],
|
||||
"reset": "navigate startUrl with fresh $RUN_ID or click Reset run",
|
||||
"stepReturn": "obs,reward,terminated,truncated,info"
|
||||
},
|
||||
"viewport": {
|
||||
"width": 1280,
|
||||
"height": 720
|
||||
},
|
||||
"slowMoMs": 1000,
|
||||
"timeoutMs": 5000,
|
||||
"locale": null,
|
||||
"timezoneId": null,
|
||||
"humanBaseline": null,
|
||||
"actionSubsets": [
|
||||
"bid",
|
||||
"nav",
|
||||
"tab",
|
||||
"infeas",
|
||||
"chat"
|
||||
],
|
||||
"rewardShape": "binary",
|
||||
"taskId": "choredesk.L2.refund-message",
|
||||
"seed": 202,
|
||||
"maxSteps": 16,
|
||||
"allowedActions": [
|
||||
"click",
|
||||
"fill",
|
||||
"select_option",
|
||||
"goto",
|
||||
"report_infeasible",
|
||||
"send_msg_to_user"
|
||||
],
|
||||
"setupUrl": "scenarios/choredesk/index.html?task=choredesk-l2-refund-message&run=$RUN_ID&seed=$SEED",
|
||||
"validateScript": "scenarios/choredesk/index.html#window.__choredesk.validate()",
|
||||
"cheatScript": "scenarios/choredesk/cheats.js#choredesk-l2-refund-message",
|
||||
"tags": [
|
||||
"lookup",
|
||||
"cross-page-memory",
|
||||
"message-compose"
|
||||
],
|
||||
"rubric": [
|
||||
{
|
||||
"id": "messageSubject",
|
||||
"weight": 0.34,
|
||||
"description": "Billing message subject matches"
|
||||
},
|
||||
{
|
||||
"id": "email",
|
||||
"weight": 0.33,
|
||||
"description": "Body includes Noah email"
|
||||
},
|
||||
{
|
||||
"id": "total",
|
||||
"weight": 0.33,
|
||||
"description": "Body includes exact order total"
|
||||
}
|
||||
],
|
||||
"goal": "Resolve a customer billing task by looking up customer email and order total on separate pages, then composing a message with exact required fields.",
|
||||
"goalObject": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "Resolve a customer billing task by looking up customer email and order total on separate pages, then composing a message with exact required fields."
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"gradeSource": "page"
|
||||
},
|
||||
{
|
||||
"id": "choredesk-l3-restock-ticket",
|
||||
"site": "choredesk",
|
||||
"startUrl": "scenarios/choredesk/index.html?task=choredesk-l3-restock-ticket&run=$RUN_ID&seed=$SEED",
|
||||
"difficulty": "L3",
|
||||
"category": "cross-page-state-update",
|
||||
"intent": "Use ticket text to identify an item, map item to SKU in Catalog, update Inventory reorder quantity, and add exact internal note back on the ticket.",
|
||||
"expectedActions": {
|
||||
"min": 12,
|
||||
"max": 20
|
||||
},
|
||||
"requires": [
|
||||
"multi-page navigation",
|
||||
"entity resolution",
|
||||
"state mutation",
|
||||
"exact text note",
|
||||
"audit-save behavior"
|
||||
],
|
||||
"grader": {
|
||||
"type": "inPageProgrammatic",
|
||||
"expression": "window.__taskVerdict || (window.__taskValidation?.done ? \"PASS\" : \"FAIL\")",
|
||||
"passValue": "PASS"
|
||||
},
|
||||
"browserGymLike": {
|
||||
"observation": [
|
||||
"dom",
|
||||
"accessibility_tree",
|
||||
"screenshot_optional"
|
||||
],
|
||||
"actionSpace": [
|
||||
"click",
|
||||
"type",
|
||||
"select",
|
||||
"evaluate_optional"
|
||||
],
|
||||
"reset": "navigate startUrl with fresh $RUN_ID or click Reset run",
|
||||
"stepReturn": "obs,reward,terminated,truncated,info"
|
||||
},
|
||||
"viewport": {
|
||||
"width": 1280,
|
||||
"height": 720
|
||||
},
|
||||
"slowMoMs": 1000,
|
||||
"timeoutMs": 5000,
|
||||
"locale": null,
|
||||
"timezoneId": null,
|
||||
"humanBaseline": null,
|
||||
"actionSubsets": [
|
||||
"bid",
|
||||
"nav",
|
||||
"tab",
|
||||
"infeas",
|
||||
"chat"
|
||||
],
|
||||
"rewardShape": "binary",
|
||||
"taskId": "choredesk.L3.restock-ticket",
|
||||
"seed": 303,
|
||||
"maxSteps": 24,
|
||||
"allowedActions": [
|
||||
"click",
|
||||
"fill",
|
||||
"select_option",
|
||||
"goto",
|
||||
"report_infeasible",
|
||||
"send_msg_to_user"
|
||||
],
|
||||
"setupUrl": "scenarios/choredesk/index.html?task=choredesk-l3-restock-ticket&run=$RUN_ID&seed=$SEED",
|
||||
"validateScript": "scenarios/choredesk/index.html#window.__choredesk.validate()",
|
||||
"cheatScript": "scenarios/choredesk/cheats.js#choredesk-l3-restock-ticket",
|
||||
"tags": [
|
||||
"entity-resolution",
|
||||
"cross-page-state-update",
|
||||
"exact-note"
|
||||
],
|
||||
"rubric": [
|
||||
{
|
||||
"id": "inventory",
|
||||
"weight": 0.4,
|
||||
"description": "AST-LAMP reorder quantity is 12 and saved"
|
||||
},
|
||||
{
|
||||
"id": "ticketNote",
|
||||
"weight": 0.4,
|
||||
"description": "T-205 note contains exact phrase"
|
||||
},
|
||||
{
|
||||
"id": "audit",
|
||||
"weight": 0.2,
|
||||
"description": "Both save audit entries exist"
|
||||
}
|
||||
],
|
||||
"goal": "Use ticket text to identify an item, map item to SKU in Catalog, update Inventory reorder quantity, and add exact internal note back on the ticket.",
|
||||
"goalObject": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "Use ticket text to identify an item, map item to SKU in Catalog, update Inventory reorder quantity, and add exact internal note back on the ticket."
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"gradeSource": "page"
|
||||
},
|
||||
{
|
||||
"id": "mini-shop-red-second-cheapest",
|
||||
"site": "mini-shop",
|
||||
"startUrl": "fixtures/sites/mini-shop/index.html?task=mini-shop-red-second-cheapest&run=$RUN_ID&seed=$SEED",
|
||||
"difficulty": "L2",
|
||||
"category": "task-completion",
|
||||
"intent": "Add the second-cheapest red product to cart, checkout, and leave the generated order id visible.",
|
||||
"expectedActions": {
|
||||
"min": 5,
|
||||
"max": 9
|
||||
},
|
||||
"requires": [
|
||||
"filter/sort reasoning",
|
||||
"cart state",
|
||||
"checkout flow"
|
||||
],
|
||||
"grader": {
|
||||
"type": "inPageProgrammatic",
|
||||
"expression": "window.__taskVerdict || (window.__taskValidation?.done ? \"PASS\" : \"FAIL\")",
|
||||
"passValue": "PASS"
|
||||
},
|
||||
"browserGymLike": {
|
||||
"observation": [
|
||||
"dom",
|
||||
"accessibility_tree",
|
||||
"screenshot_optional"
|
||||
],
|
||||
"actionSpace": [
|
||||
"click",
|
||||
"read_table",
|
||||
"evaluate_optional"
|
||||
],
|
||||
"reset": "navigate startUrl with fresh $RUN_ID",
|
||||
"stepReturn": "obs,reward,terminated,truncated,info"
|
||||
},
|
||||
"viewport": {
|
||||
"width": 1280,
|
||||
"height": 720
|
||||
},
|
||||
"slowMoMs": 1000,
|
||||
"timeoutMs": 5000,
|
||||
"locale": null,
|
||||
"timezoneId": null,
|
||||
"humanBaseline": null,
|
||||
"actionSubsets": [
|
||||
"bid",
|
||||
"nav",
|
||||
"tab",
|
||||
"infeas",
|
||||
"chat"
|
||||
],
|
||||
"rewardShape": "binary",
|
||||
"taskId": "mini-shop.L2.red-second-cheapest",
|
||||
"seed": 404,
|
||||
"maxSteps": 12,
|
||||
"allowedActions": [
|
||||
"click",
|
||||
"goto",
|
||||
"report_infeasible",
|
||||
"send_msg_to_user"
|
||||
],
|
||||
"setupUrl": "fixtures/sites/mini-shop/index.html?task=mini-shop-red-second-cheapest&run=$RUN_ID&seed=$SEED",
|
||||
"validateScript": "fixtures/sites/mini-shop/grader.js#MiniShopGrader.validate",
|
||||
"cheatScript": "fixtures/sites/mini-shop/cheats.js#mini-shop-red-second-cheapest",
|
||||
"tags": [
|
||||
"shopping",
|
||||
"sort",
|
||||
"cart",
|
||||
"checkout"
|
||||
],
|
||||
"rubric": [
|
||||
{
|
||||
"id": "addedTarget",
|
||||
"weight": 0.4,
|
||||
"description": "Cart includes second-cheapest red product"
|
||||
},
|
||||
{
|
||||
"id": "noWrongItems",
|
||||
"weight": 0.3,
|
||||
"description": "Cart has no wrong products"
|
||||
},
|
||||
{
|
||||
"id": "checkedOut",
|
||||
"weight": 0.3,
|
||||
"description": "Checkout produced order id"
|
||||
}
|
||||
],
|
||||
"goal": "Add the second-cheapest red product to cart, checkout, and leave the generated order id visible.",
|
||||
"goalObject": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "Add the second-cheapest red product to cart, checkout, and leave the generated order id visible."
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"gradeSource": "both"
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,59 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"title": "pi-chrome long-horizon task manifest",
|
||||
"description": "BrowserGym-inspired task manifest: setup URL, deterministic seed, goal object, action subsets, reward contract, optional cheat recipe, and in-page validate hook.",
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["id", "taskId", "site", "startUrl", "setupUrl", "difficulty", "category", "goal", "goalObject", "grader", "actionSubsets", "maxSteps"],
|
||||
"additionalProperties": true,
|
||||
"properties": {
|
||||
"id": { "type": "string" },
|
||||
"taskId": { "type": "string" },
|
||||
"site": { "type": "string" },
|
||||
"seed": { "type": "integer" },
|
||||
"startUrl": { "type": "string", "description": "Relative URL. $RUN_ID and $SEED should be replaced by runner for isolated deterministic state." },
|
||||
"setupUrl": { "type": "string" },
|
||||
"validateScript": { "type": "string" },
|
||||
"cheatScript": { "type": "string" },
|
||||
"viewport": { "type": "object", "properties": { "width": { "type": "integer" }, "height": { "type": "integer" } } },
|
||||
"slowMoMs": { "type": "integer" },
|
||||
"timeoutMs": { "type": "integer" },
|
||||
"locale": { "type": ["string", "null"] },
|
||||
"timezoneId": { "type": ["string", "null"] },
|
||||
"difficulty": { "enum": ["L1", "L2", "L3"] },
|
||||
"category": { "enum": ["enterprise-crud", "cross-page-memory", "cross-page-state-update", "task-completion"] },
|
||||
"gradeSource": { "enum": ["page", "harness", "both"] },
|
||||
"tags": { "type": "array", "items": { "type": "string" } },
|
||||
"intent": { "type": "string" },
|
||||
"goal": { "type": "string" },
|
||||
"goalObject": { "type": "array", "items": { "type": "object" } },
|
||||
"expectedActions": { "type": "object", "required": ["min", "max"], "properties": { "min": { "type": "integer" }, "max": { "type": "integer" } } },
|
||||
"maxSteps": { "type": "integer" },
|
||||
"humanBaseline": { "type": ["number", "null"], "minimum": 0, "maximum": 1 },
|
||||
"actionSubsets": { "type": "array", "items": { "enum": ["chat", "infeas", "bid", "coord", "nav", "tab"] } },
|
||||
"allowedActions": { "type": "array", "items": { "type": "string" } },
|
||||
"requires": { "type": "array", "items": { "type": "string" } },
|
||||
"rewardShape": { "enum": ["binary", "graded", "shaped"] },
|
||||
"rubric": { "type": "array", "items": { "type": "object", "required": ["id", "weight", "description"], "properties": { "id": { "type": "string" }, "weight": { "type": "number" }, "description": { "type": "string" } } } },
|
||||
"grader": {
|
||||
"type": "object",
|
||||
"required": ["type", "expression", "passValue"],
|
||||
"properties": {
|
||||
"type": { "enum": ["inPageProgrammatic"] },
|
||||
"expression": { "type": "string", "description": "Should evaluate to BrowserGym-like validate object or legacy {v,r}." },
|
||||
"passValue": { "type": "string" }
|
||||
}
|
||||
},
|
||||
"browserGymLike": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"observation": { "type": "array", "items": { "type": "string" } },
|
||||
"actionSpace": { "type": "array", "items": { "type": "string" } },
|
||||
"reset": { "type": "string" },
|
||||
"stepReturn": { "const": "obs,reward,terminated,truncated,info" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,395 @@
|
||||
// Unit harness for pi-chrome's dedicated automation tab/window isolation in service_worker.js.
|
||||
//
|
||||
// Feature under test: pi-chrome must never navigate or replace the user's active tab. Page and
|
||||
// navigation actions without an explicit target are routed to a dedicated automation target that
|
||||
// the *calling Pi session* created and owns. Ownership is session-scoped (one extension brokers
|
||||
// every session) and mirrored to chrome.storage.session so a service-worker restart re-hydrates
|
||||
// it instead of orphaning the window. Cleanup closes only the calling session's owned target.
|
||||
//
|
||||
// Like csp-eval.test.mjs we load the *real* worker into a vm sandbox with a stateful chrome.*
|
||||
// mock, then exercise the real helpers and the real dispatch() paths. Chrome state (tabs/windows/
|
||||
// storage.session) can be shared across two sandbox loads to simulate a service-worker restart.
|
||||
|
||||
import vm from "node:vm";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const workerPath = path.resolve(__dirname, "../../extensions/chrome-profile-bridge/browser-extension/service_worker.js");
|
||||
const src = fs.readFileSync(workerPath, "utf8");
|
||||
|
||||
let failures = 0;
|
||||
let passes = 0;
|
||||
function ok(cond, msg) {
|
||||
if (cond) { passes++; }
|
||||
else { failures++; console.error(` ✗ ${msg}`); }
|
||||
}
|
||||
async function throwsWith(fn, re, msg) {
|
||||
try { await fn(); ok(false, `${msg} (expected throw)`); }
|
||||
catch (e) { ok(re.test(String(e.message || e)), `${msg} (got: ${e.message})`); }
|
||||
}
|
||||
|
||||
// ---- stateful Chrome mock. `state` (tabs/windows/storage) can be shared to simulate a
|
||||
// service-worker restart: the browser keeps its tabs/windows/session-storage, the worker memory
|
||||
// is wiped (a fresh sandbox).
|
||||
function makeChromeState() {
|
||||
const tabs = new Map(); // id -> { id, windowId, url, active, groupId }
|
||||
const windows = new Map(); // id -> { id }
|
||||
const groups = new Map(); // groupId -> { id, title, color, collapsed, windowId }
|
||||
const storage = {}; // chrome.storage.session backing
|
||||
let nextTabId = 1;
|
||||
let nextWindowId = 1;
|
||||
let nextGroupId = 1;
|
||||
const alloc = { tab: () => nextTabId++, window: () => nextWindowId++, group: () => nextGroupId++ };
|
||||
|
||||
// Seed a user window with two real user tabs (Gmail + a research article, the active one).
|
||||
const userWindowId = alloc.window();
|
||||
windows.set(userWindowId, { id: userWindowId });
|
||||
const userGmail = { id: alloc.tab(), windowId: userWindowId, url: "https://mail.google.com/", active: false, groupId: -1 };
|
||||
const userArticle = { id: alloc.tab(), windowId: userWindowId, url: "https://example.com/research-article", active: true, groupId: -1 };
|
||||
tabs.set(userGmail.id, userGmail);
|
||||
tabs.set(userArticle.id, userArticle);
|
||||
|
||||
return { tabs, windows, groups, storage, alloc, userWindowId, userGmail, userArticle };
|
||||
}
|
||||
|
||||
function makeChrome(state, { withWindows = true, withStorage = true, withTabGroups = false } = {}) {
|
||||
const { tabs, windows, groups, storage, alloc, userWindowId } = state;
|
||||
const noop = () => {};
|
||||
const listener = { addListener: noop, removeListener: noop };
|
||||
|
||||
const chrome = {
|
||||
runtime: { id: "unittestextension", getManifest: () => ({ version: "0.0.0" }), onInstalled: listener, onStartup: listener, lastError: null },
|
||||
alarms: { onAlarm: listener, create: noop, clear: noop, clearAll: noop },
|
||||
action: { onClicked: listener },
|
||||
debugger: { sendCommand: noop, attach: async () => {}, detach: async () => {}, getTargets: (cb) => cb([]), onDetach: listener },
|
||||
scripting: { executeScript: async () => [{ result: undefined }], registerContentScripts: async () => {}, unregisterContentScripts: async () => {} },
|
||||
webNavigation: { onCommitted: listener },
|
||||
tabs: {
|
||||
onUpdated: listener,
|
||||
query: async (q = {}) => {
|
||||
let list = [...tabs.values()];
|
||||
if (q.active === true) list = list.filter((t) => t.active);
|
||||
if (typeof q.windowId === "number") list = list.filter((t) => t.windowId === q.windowId);
|
||||
return list.map((t) => ({ ...t }));
|
||||
},
|
||||
get: async (id) => { const t = tabs.get(id); if (!t) throw new Error(`No tab with id ${id}`); return { ...t }; },
|
||||
create: async ({ url = "about:blank", active = false, windowId = userWindowId } = {}) => {
|
||||
const tab = { id: alloc.tab(), windowId, url, active, groupId: -1 };
|
||||
tabs.set(tab.id, tab);
|
||||
return { ...tab };
|
||||
},
|
||||
update: async (id, props = {}) => { const t = tabs.get(id); if (!t) throw new Error(`No tab with id ${id}`); Object.assign(t, props); return { ...t }; },
|
||||
remove: async (id) => { tabs.delete(id); },
|
||||
group: async ({ groupId, tabIds = [] } = {}) => {
|
||||
let gid = groupId;
|
||||
if (typeof gid !== "number") {
|
||||
gid = alloc.group();
|
||||
const firstTab = tabs.get(tabIds[0]);
|
||||
groups.set(gid, { id: gid, title: "", color: "grey", collapsed: false, windowId: firstTab ? firstTab.windowId : userWindowId });
|
||||
}
|
||||
for (const tid of tabIds) { const t = tabs.get(tid); if (t) t.groupId = gid; }
|
||||
return gid;
|
||||
},
|
||||
ungroup: async (id) => { const ids = Array.isArray(id) ? id : [id]; for (const tid of ids) { const t = tabs.get(tid); if (t) t.groupId = -1; } },
|
||||
},
|
||||
storage: withStorage ? {
|
||||
session: {
|
||||
get: async (key) => (key in storage ? { [key]: storage[key] } : {}),
|
||||
set: async (obj) => { Object.assign(storage, obj); },
|
||||
},
|
||||
} : undefined,
|
||||
};
|
||||
|
||||
if (withTabGroups) {
|
||||
chrome.tabGroups = {
|
||||
query: async ({ windowId } = {}) => [...groups.values()].filter((g) => windowId === undefined || g.windowId === windowId).map((g) => ({ ...g })),
|
||||
get: async (id) => { const g = groups.get(id); if (!g) throw new Error(`No group ${id}`); return { ...g }; },
|
||||
update: async (id, props = {}) => { const g = groups.get(id); if (!g) throw new Error(`No group ${id}`); Object.assign(g, props); return { ...g }; },
|
||||
};
|
||||
}
|
||||
|
||||
if (withWindows) {
|
||||
chrome.windows = {
|
||||
create: async ({ url = "about:blank", focused = false } = {}) => {
|
||||
const id = alloc.window();
|
||||
windows.set(id, { id });
|
||||
const tab = { id: alloc.tab(), windowId: id, url, active: true, groupId: -1 };
|
||||
tabs.set(tab.id, tab);
|
||||
return { id, focused, tabs: [{ ...tab }] };
|
||||
},
|
||||
get: async (id) => { const w = windows.get(id); if (!w) throw new Error(`No window with id ${id}`); return { ...w }; },
|
||||
remove: async (id) => { windows.delete(id); for (const [tid, t] of [...tabs]) if (t.windowId === id) tabs.delete(tid); },
|
||||
update: async () => {},
|
||||
};
|
||||
} else {
|
||||
chrome.windows = { update: async () => {} }; // no create/get/remove -> tab fallback path
|
||||
}
|
||||
|
||||
return chrome;
|
||||
}
|
||||
|
||||
function loadWorker(chrome) {
|
||||
const noop = () => {};
|
||||
const sandbox = {
|
||||
console, JSON, Date, Math, Promise, Array, Object, String, Number, Boolean,
|
||||
Error, TypeError, Map, Set, BigInt, Symbol, structuredClone,
|
||||
setTimeout, clearTimeout, setInterval: () => 0, clearInterval: noop,
|
||||
fetch: async () => { throw new Error("no network in unit test"); },
|
||||
navigator: { userAgent: "unit-test" },
|
||||
WebSocket: function () {},
|
||||
chrome,
|
||||
};
|
||||
sandbox.globalThis = sandbox;
|
||||
sandbox.self = sandbox;
|
||||
vm.createContext(sandbox);
|
||||
vm.runInContext(src, sandbox);
|
||||
return sandbox;
|
||||
}
|
||||
|
||||
const SK = "session:alpha"; // a representative sessionKey
|
||||
|
||||
async function run() {
|
||||
// ===== Isolation: navigation does not touch the user's active/other tabs. =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const w = loadWorker(makeChrome(state));
|
||||
const userActiveUrl = state.userArticle.url;
|
||||
|
||||
const nav = await w.dispatch("page.navigate", { url: "https://pi.test/task", waitUntilLoad: false, sessionKey: SK });
|
||||
ok(state.userArticle.url === userActiveUrl, "navigate: active user tab (research article) is not overwritten");
|
||||
ok(state.userGmail.url === "https://mail.google.com/", "navigate: other user tab (Gmail) untouched");
|
||||
ok(nav.url === "https://pi.test/task", "navigate: automation target navigated to requested URL");
|
||||
ok(nav.id !== state.userArticle.id && nav.id !== state.userGmail.id, "navigate: did not reuse any user tab");
|
||||
ok(nav.windowId !== state.userWindowId, "navigate: automation target lives in a dedicated window");
|
||||
|
||||
const status = await w.dispatch("automation.status", { sessionKey: SK });
|
||||
ok(status.tabId === nav.id && status.windowId === nav.windowId, "ownership: target ids tracked for the session");
|
||||
ok(w.isPiChromeOwnedTarget(nav.id, SK) === true, "ownership: isPiChromeOwnedTarget(owned, session) === true");
|
||||
ok(w.isPiChromeOwnedTarget(state.userArticle.id) === false, "ownership: user tab is never owned (any session)");
|
||||
|
||||
// Reuse: a later navigation reuses the same owned target.
|
||||
const nav2 = await w.dispatch("page.navigate", { url: "https://pi.test/step-2", waitUntilLoad: false, sessionKey: SK });
|
||||
ok(nav2.id === nav.id && nav2.windowId === nav.windowId, "reuse: second navigation reuses the same automation window/tab");
|
||||
ok(state.userArticle.url === userActiveUrl, "reuse: user tab still untouched after second navigation");
|
||||
|
||||
// Cleanup closes only the owned window; user tabs/windows survive.
|
||||
const cleanup = await w.dispatch("automation.cleanup", { sessionKey: SK });
|
||||
ok(cleanup.closedWindowId === nav.windowId, "cleanup: closed the owned window");
|
||||
ok(state.tabs.has(state.userArticle.id) && state.tabs.has(state.userGmail.id), "cleanup: user tabs never closed");
|
||||
ok(state.windows.has(state.userWindowId), "cleanup: user window never closed");
|
||||
ok(!state.tabs.has(nav.id), "cleanup: the owned automation tab is gone");
|
||||
const status2 = await w.dispatch("automation.status", { sessionKey: SK });
|
||||
ok(status2.tabId === null && status2.windowId === null, "cleanup: ownership cleared");
|
||||
}
|
||||
|
||||
// ===== Session-group integration: the dedicated-window tab joins this session's group. =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const w = loadWorker(makeChrome(state, { withTabGroups: true }));
|
||||
// index.ts tags page.* actions with joinSessionGroup + sessionGroupTitle; replicate that here.
|
||||
const groupTitle = "Pi Session: alpha";
|
||||
const nav = await w.dispatch("page.navigate", {
|
||||
url: "https://pi.test/grouped", waitUntilLoad: false,
|
||||
sessionKey: SK, joinSessionGroup: true, sessionGroupTitle: groupTitle,
|
||||
});
|
||||
const navTab = state.tabs.get(nav.id);
|
||||
ok(navTab.windowId !== state.userWindowId, "group: automation tab is in its dedicated window");
|
||||
ok(typeof navTab.groupId === "number" && navTab.groupId >= 0, "group: automation tab joined a tab group");
|
||||
const grp = state.groups.get(navTab.groupId);
|
||||
ok(grp && grp.title === groupTitle, "group: the group is titled with this session's title");
|
||||
ok(grp.windowId === navTab.windowId, "group: the session group lives inside the dedicated automation window (not the user window)");
|
||||
|
||||
// A second page action reuses the same tab and does not spawn a second group.
|
||||
const groupsBefore = state.groups.size;
|
||||
await w.dispatch("page.navigate", { url: "https://pi.test/grouped-2", waitUntilLoad: false, sessionKey: SK, joinSessionGroup: true, sessionGroupTitle: groupTitle });
|
||||
ok(state.groups.size === groupsBefore, "group: reusing the automation tab does not create a second group");
|
||||
}
|
||||
|
||||
// ===== tab.new joins the existing session group instead of creating one group per window. =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const w = loadWorker(makeChrome(state, { withTabGroups: true }));
|
||||
const groupTitle = "Pi Session: alpha";
|
||||
const nav = await w.dispatch("page.navigate", {
|
||||
url: "https://pi.test/group-owner", waitUntilLoad: false,
|
||||
sessionKey: SK, joinSessionGroup: true, sessionGroupTitle: groupTitle,
|
||||
});
|
||||
const navTab = state.tabs.get(nav.id);
|
||||
const groupId = navTab.groupId;
|
||||
const groupsBefore = state.groups.size;
|
||||
|
||||
const opened = await w.dispatch("tab.new", { url: "https://pi.test/new-tab", groupTitle, sessionKey: SK });
|
||||
ok(state.groups.size === groupsBefore, "tab.new-group: did not create another same-session group");
|
||||
ok(opened.tab.groupId === groupId, "tab.new-group: opened tab joined the existing session group");
|
||||
ok(opened.tab.windowId === nav.windowId, "tab.new-group: opened tab was created in the existing group's window");
|
||||
|
||||
const forced = await w.dispatch("tab.new", { url: "https://pi.test/no-opt-out", groupTitle, group: false, sessionKey: SK });
|
||||
ok(forced.tab.groupId === groupId, "tab.new-group: group:false is ignored; tab still joins the session group");
|
||||
ok(state.groups.size === groupsBefore, "tab.new-group: group:false does not create another group");
|
||||
|
||||
const blankTitle = await w.dispatch("tab.new", { url: "https://pi.test/blank-title", groupTitle: "", group: false, sessionKey: SK });
|
||||
ok(typeof blankTitle.tab.groupId === "number" && blankTitle.tab.groupId >= 0, "tab.new-group: groupTitle:'' still creates a grouped tab");
|
||||
ok(blankTitle.group.title === "Pi", "tab.new-group: blank groupTitle falls back to a group instead of opting out");
|
||||
|
||||
const nav2 = await w.dispatch("page.navigate", {
|
||||
url: "https://pi.test/new-automation-target", waitUntilLoad: false,
|
||||
sessionKey: "session:beta", joinSessionGroup: true, sessionGroupTitle: groupTitle,
|
||||
});
|
||||
ok(state.groups.size === groupsBefore + 1, "automation-target-group: reused the existing session group, only blank-title Pi group was extra");
|
||||
ok(nav2.groupId === groupId, "automation-target-group: new automation target joined the existing session group");
|
||||
ok(nav2.windowId === nav.windowId, "automation-target-group: new automation target was created in the existing group's window");
|
||||
}
|
||||
|
||||
// ===== tab.new never leaves an ungrouped tab behind when grouping fails. =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const chrome = makeChrome(state, { withTabGroups: true });
|
||||
const w = loadWorker(chrome);
|
||||
const tabsBefore = state.tabs.size;
|
||||
chrome.tabs.group = async () => { throw new Error("group blew up"); };
|
||||
|
||||
await throwsWith(
|
||||
() => w.dispatch("tab.new", { url: "https://pi.test/group-fail", groupTitle: "Pi Session: alpha", sessionKey: SK }),
|
||||
/group blew up/,
|
||||
"tab.new-group-fail: surfaces grouping error",
|
||||
);
|
||||
ok(state.tabs.size === tabsBefore, "tab.new-group-fail: closes the created tab instead of leaving it ungrouped");
|
||||
}
|
||||
|
||||
// ===== Grouping is best-effort: a tabGroups failure must not break navigation. =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const chrome = makeChrome(state, { withTabGroups: true });
|
||||
chrome.tabs.group = async () => { throw new Error("group blew up"); };
|
||||
const w = loadWorker(chrome);
|
||||
const nav = await w.dispatch("page.navigate", { url: "https://pi.test/group-fail", waitUntilLoad: false, sessionKey: SK, joinSessionGroup: true, sessionGroupTitle: "Pi Session: alpha" });
|
||||
ok(nav.url === "https://pi.test/group-fail", "group-fail: navigation still succeeds when grouping throws");
|
||||
ok(state.tabs.get(nav.id).windowId !== state.userWindowId, "group-fail: still used the dedicated automation window");
|
||||
}
|
||||
|
||||
// ===== Concurrency: two sessions get separate windows; cleanup is per-session. =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const w = loadWorker(makeChrome(state));
|
||||
const a = await w.dispatch("page.navigate", { url: "https://pi.test/a", waitUntilLoad: false, sessionKey: "session:A" });
|
||||
const b = await w.dispatch("page.navigate", { url: "https://pi.test/b", waitUntilLoad: false, sessionKey: "session:B" });
|
||||
ok(a.id !== b.id && a.windowId !== b.windowId, "concurrency: each session gets its own dedicated window/tab");
|
||||
ok(w.isPiChromeOwnedTarget(a.id, "session:A") && !w.isPiChromeOwnedTarget(a.id, "session:B"), "concurrency: ownership is scoped to the creating session");
|
||||
|
||||
// Cleaning up session A must not touch session B's target.
|
||||
await w.dispatch("automation.cleanup", { sessionKey: "session:A" });
|
||||
ok(!state.tabs.has(a.id), "concurrency: cleanup closed session A's tab");
|
||||
ok(state.tabs.has(b.id), "concurrency: cleanup left session B's tab open");
|
||||
const bStatus = await w.dispatch("automation.status", { sessionKey: "session:B" });
|
||||
ok(bStatus.tabId === b.id, "concurrency: session B still owns its target after A cleanup");
|
||||
}
|
||||
|
||||
// ===== Service-worker restart / reconnect: persisted ownership re-hydrates from storage. =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const w1 = loadWorker(makeChrome(state));
|
||||
const nav = await w1.dispatch("page.navigate", { url: "https://pi.test/persist", waitUntilLoad: false, sessionKey: SK });
|
||||
ok(typeof state.storage.piChromeAutomationTargets === "object", "restart: ownership was persisted to storage.session");
|
||||
|
||||
// Simulate the MV3 service worker being suspended and restarted: fresh sandbox (memory wiped),
|
||||
// same browser tabs/windows + same session storage.
|
||||
const w2 = loadWorker(makeChrome(state));
|
||||
const statusAfterRestart = await w2.dispatch("automation.status", { sessionKey: SK });
|
||||
ok(statusAfterRestart.tabId === nav.id && statusAfterRestart.windowId === nav.windowId, "restart: re-hydrated the owned target from storage");
|
||||
|
||||
// A navigation after restart must REUSE the existing window, not orphan it with a new one.
|
||||
const windowsBefore = state.windows.size;
|
||||
const nav2 = await w2.dispatch("page.navigate", { url: "https://pi.test/persist-2", waitUntilLoad: false, sessionKey: SK });
|
||||
ok(nav2.id === nav.id && nav2.windowId === nav.windowId, "restart: navigation after restart reuses the persisted window (no orphan)");
|
||||
ok(state.windows.size === windowsBefore, "restart: no new window created after restart");
|
||||
|
||||
// Cleanup after restart works and clears persisted state.
|
||||
await w2.dispatch("automation.cleanup", { sessionKey: SK });
|
||||
const persisted = state.storage.piChromeAutomationTargets || {};
|
||||
ok(!(SK in persisted), "restart: cleanup removed the session from persisted storage");
|
||||
}
|
||||
|
||||
// ===== Restart after the user manually closed the window: no orphan, fresh target. =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const w1 = loadWorker(makeChrome(state));
|
||||
const nav = await w1.dispatch("page.navigate", { url: "https://pi.test/closed", waitUntilLoad: false, sessionKey: SK });
|
||||
await state.windows.delete(nav.windowId); // user closed pi-chrome's window
|
||||
for (const [tid, t] of [...state.tabs]) if (t.windowId === nav.windowId) state.tabs.delete(tid);
|
||||
|
||||
const w2 = loadWorker(makeChrome(state)); // SW restart
|
||||
const nav2 = await w2.dispatch("page.navigate", { url: "https://pi.test/reopened", waitUntilLoad: false, sessionKey: SK });
|
||||
ok(nav2.id !== nav.id, "restart-after-close: a fresh automation target is created when the persisted one is gone");
|
||||
ok(state.tabs.has(nav2.id), "restart-after-close: new target exists");
|
||||
}
|
||||
|
||||
// ===== tab.* management never auto-creates / never falls back to the user's active tab. =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const w = loadWorker(makeChrome(state));
|
||||
const windowsBefore = state.windows.size;
|
||||
const tabsBefore = state.tabs.size;
|
||||
|
||||
await throwsWith(
|
||||
() => w.dispatch("tab.close", { sessionKey: SK }),
|
||||
/no automation tab yet|Pass targetId/,
|
||||
"tab.close: with no target and no owned target, errors instead of closing the user's active tab",
|
||||
);
|
||||
ok(state.tabs.has(state.userArticle.id), "tab.close: user's active tab was NOT closed");
|
||||
ok(state.windows.size === windowsBefore && state.tabs.size === tabsBefore, "tab.close: did not spawn a throwaway tab/window");
|
||||
|
||||
await throwsWith(() => w.dispatch("tab.activate", { sessionKey: SK }), /no automation tab yet|Pass targetId/, "tab.activate: errors with no target/owned target");
|
||||
|
||||
// Once an automation target exists, management actions operate on it (not on the user tab).
|
||||
const nav = await w.dispatch("page.navigate", { url: "https://pi.test/manage", waitUntilLoad: false, sessionKey: SK });
|
||||
const closed = await w.dispatch("tab.close", { sessionKey: SK });
|
||||
ok(closed.closed === nav.id, "tab.close: with an owned target, closes that target");
|
||||
ok(state.tabs.has(state.userArticle.id), "tab.close: user tab still safe after closing the owned target");
|
||||
}
|
||||
|
||||
// ===== Explicit targeting still works on any existing tab (no regression). =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const w = loadWorker(makeChrome(state));
|
||||
const nav = await w.dispatch("page.navigate", { url: "https://pi.test/explicit", targetId: String(state.userGmail.id), waitUntilLoad: false, sessionKey: SK });
|
||||
ok(nav.id === state.userGmail.id, "explicit: targetId routes to the requested existing tab");
|
||||
ok(state.userGmail.url === "https://pi.test/explicit", "explicit: explicitly targeted tab is navigated");
|
||||
const status = await w.dispatch("automation.status", { sessionKey: SK });
|
||||
ok(status.tabId === null, "explicit: explicit targeting does not create/claim an automation target");
|
||||
}
|
||||
|
||||
// ===== Window-unavailable fallback: a dedicated TAB is used, and the user's window is safe. =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const w = loadWorker(makeChrome(state, { withWindows: false }));
|
||||
const target = await w.getOrCreateAutomationTarget(SK);
|
||||
ok(target.id !== state.userArticle.id && target.id !== state.userGmail.id, "fallback: created a dedicated tab, not a user tab");
|
||||
ok(w.isPiChromeOwnedTarget(target.id, SK) === true, "fallback: dedicated tab is owned");
|
||||
const cleanup = await w.cleanupAutomationTarget(SK);
|
||||
ok(cleanup.closedTabId === target.id && cleanup.closedWindowId === null, "fallback: cleanup closes only the owned tab (never the shared window)");
|
||||
ok(state.windows.has(state.userWindowId), "fallback: cleanup never closes the user/shared window");
|
||||
ok(state.tabs.has(state.userArticle.id) && state.tabs.has(state.userGmail.id), "fallback: cleanup leaves user tabs intact");
|
||||
}
|
||||
|
||||
// ===== Robust cleanup: no-op when nothing created, and when target already closed manually. =====
|
||||
{
|
||||
const state = makeChromeState();
|
||||
const w = loadWorker(makeChrome(state));
|
||||
const empty = await w.cleanupAutomationTarget(SK);
|
||||
ok(empty.closedWindowId === null && empty.closedTabId === null, "cleanup: no-op when nothing was ever created");
|
||||
|
||||
const t = await w.getOrCreateAutomationTarget(SK);
|
||||
// User closed pi-chrome's window manually (Chrome closes its tabs too).
|
||||
state.windows.delete(t.windowId);
|
||||
for (const [tid, tab] of [...state.tabs]) if (tab.windowId === t.windowId) state.tabs.delete(tid);
|
||||
const stale = await w.cleanupAutomationTarget(SK);
|
||||
ok(stale.closedWindowId === null && stale.closedTabId === null, "cleanup: robust when owned window was already closed");
|
||||
}
|
||||
|
||||
console.log(`\n${passes} passed, ${failures} failed`);
|
||||
if (failures) process.exit(1);
|
||||
}
|
||||
|
||||
run().catch((e) => { console.error(e); process.exit(1); });
|
||||
@@ -0,0 +1,192 @@
|
||||
// Unit harness for the CSP-bypass layer in service_worker.js.
|
||||
//
|
||||
// The real CSP bypass (CDP Runtime.evaluate not being subject to page CSP) can only be
|
||||
// proven in a browser — see challenge 39-strict-csp-fallback. These tests instead validate
|
||||
// the JS *logic* of the refactor that the bypass depends on:
|
||||
// - evaluateInTab: wrapper-string construction, expression/statement fallback, value
|
||||
// marker round-trip (undefined/function/symbol/bigint/Error/DOMRect), error propagation.
|
||||
// - executeInTab: 2-phase define-then-invoke, envelope unwrap, error propagation, and that
|
||||
// all real HELPER_FUNCS serialize+assign without a parse error.
|
||||
// - page.waitFor: service-worker-side polling via evaluateInTab (selector + expression).
|
||||
//
|
||||
// We load the worker into a vm sandbox with mocked chrome.* APIs, then replace `cdp` with a
|
||||
// shim that evaluates the expression in a separate "page world" vm context (simulating CDP
|
||||
// Runtime.evaluate returnByValue). No browser, no network, no deps.
|
||||
|
||||
import vm from "node:vm";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const workerPath = path.resolve(__dirname, "../../extensions/chrome-profile-bridge/browser-extension/service_worker.js");
|
||||
const src = fs.readFileSync(workerPath, "utf8");
|
||||
|
||||
let failures = 0;
|
||||
let passes = 0;
|
||||
function ok(cond, msg) {
|
||||
if (cond) { passes++; }
|
||||
else { failures++; console.error(` ✗ ${msg}`); }
|
||||
}
|
||||
async function throwsWith(fn, re, msg) {
|
||||
try { await fn(); ok(false, `${msg} (expected throw)`); }
|
||||
catch (e) { ok(re.test(String(e.message || e)), `${msg} (got: ${e.message})`); }
|
||||
}
|
||||
|
||||
// ---- page world: simulates the page's MAIN world for Runtime.evaluate ----
|
||||
const pageGlobals = {
|
||||
console, JSON, Date, Math, Promise, Object, Array, String, Number, Boolean,
|
||||
Error, TypeError, SyntaxError, RangeError, BigInt, Symbol, structuredClone,
|
||||
setTimeout, parseInt, parseFloat, isNaN,
|
||||
document: {
|
||||
title: "page title",
|
||||
_present: new Set(),
|
||||
querySelector(sel) { return this._present.has(sel) ? { sel } : null; },
|
||||
},
|
||||
};
|
||||
pageGlobals.window = pageGlobals;
|
||||
pageGlobals.globalThis = pageGlobals;
|
||||
const pageWorld = vm.createContext(pageGlobals);
|
||||
|
||||
// Simulate CDP Runtime.evaluate returnByValue serialization.
|
||||
function toCdpResult(v) {
|
||||
if (v === undefined) return { result: { type: "undefined" } };
|
||||
if (v === null) return { result: { type: "object", subtype: "null", value: null } };
|
||||
const t = typeof v;
|
||||
if (t === "number" || t === "string" || t === "boolean")
|
||||
return { result: { type: t, value: v } };
|
||||
// object/array: returnByValue deep-clones JSON-able structures
|
||||
return { result: { type: "object", value: JSON.parse(JSON.stringify(v)) } };
|
||||
}
|
||||
|
||||
// ---- worker sandbox ----
|
||||
const noop = () => {};
|
||||
const listener = { addListener: noop, removeListener: noop };
|
||||
const sandbox = {
|
||||
console, JSON, Date, Math, Promise, Array, Object, String, Number, Boolean,
|
||||
Error, TypeError, Map, Set, BigInt, Symbol, structuredClone,
|
||||
setTimeout, clearTimeout,
|
||||
setInterval: () => 0,
|
||||
clearInterval: noop,
|
||||
fetch: async () => { throw new Error("no network in unit test"); },
|
||||
navigator: { userAgent: "unit-test" },
|
||||
WebSocket: function () {},
|
||||
chrome: {
|
||||
runtime: { id: "unittestextension", getManifest: () => ({ version: "0.0.0" }), onInstalled: listener, onStartup: listener, lastError: null },
|
||||
alarms: { onAlarm: listener, create: noop, clear: noop, clearAll: noop },
|
||||
action: { onClicked: listener },
|
||||
debugger: { sendCommand: noop, attach: async () => {}, detach: async () => {}, getTargets: (cb) => cb([]) },
|
||||
scripting: { executeScript: async () => [{ result: undefined }] },
|
||||
tabs: { query: async () => [], get: async () => ({}), create: async () => ({}), update: async () => ({}), remove: async () => {} },
|
||||
windows: { update: async () => {} },
|
||||
webNavigation: { onCommitted: listener },
|
||||
},
|
||||
};
|
||||
sandbox.globalThis = sandbox;
|
||||
sandbox.self = sandbox;
|
||||
vm.createContext(sandbox);
|
||||
vm.runInContext(src, sandbox);
|
||||
|
||||
// ---- override the page-touching primitives with the page-world shim ----
|
||||
sandbox.attachDebugger = async () => ({});
|
||||
sandbox.bringToFront = async () => {};
|
||||
sandbox.getTabByParams = async (p) => ({ id: (p && p.targetId) || 1, windowId: 1 });
|
||||
sandbox.cdp = async (_tabId, method, params) => {
|
||||
if (method !== "Runtime.evaluate") return {};
|
||||
try {
|
||||
const value = await vm.runInContext(params.expression, pageWorld);
|
||||
return toCdpResult(value);
|
||||
} catch (e) {
|
||||
return { exceptionDetails: { exception: { className: e.name, description: String(e.stack || e.message) }, text: "Uncaught " + String(e) } };
|
||||
}
|
||||
};
|
||||
// Phase-2 of executeInTab: run the injected wrapper func against the page world,
|
||||
// where Phase-1 (via cdp shim above) already defined window.__piAction + helpers.
|
||||
sandbox.chrome.scripting.executeScript = async ({ func, args }) => {
|
||||
const fn = vm.runInContext("(" + func.toString() + ")", pageWorld);
|
||||
const result = await fn(...(args || []));
|
||||
return [{ result }];
|
||||
};
|
||||
|
||||
const { evaluateInTab, executeInTab, dispatch } = sandbox;
|
||||
|
||||
async function run() {
|
||||
// ===== evaluateInTab: primitives & objects =====
|
||||
ok((await evaluateInTab({ expression: "2 + 2" })) === 4, "evaluate: arithmetic expression");
|
||||
ok((await evaluateInTab({ expression: "document.title" })) === "page title", "evaluate: expression without return");
|
||||
ok((await evaluateInTab({ expression: "'a' + 'b'" })) === "ab", "evaluate: string concat");
|
||||
const obj = await evaluateInTab({ expression: "({a:1, b:[2,3]})" });
|
||||
ok(obj && obj.a === 1 && obj.b[1] === 3, "evaluate: object literal round-trips");
|
||||
|
||||
// ===== value markers =====
|
||||
ok((await evaluateInTab({ expression: "void 0" })) === undefined, "evaluate: undefined marker -> undefined");
|
||||
ok((await evaluateInTab({ expression: "10n" })) === "10", "evaluate: bigint marker -> string");
|
||||
ok(/^\[Function:/.test(await evaluateInTab({ expression: "(function foo(){})" })), "evaluate: function marker");
|
||||
ok((await evaluateInTab({ expression: "Promise.resolve(42)" })) === 42, "evaluate: promise is awaited");
|
||||
|
||||
// DOMRect-like (toJSON + width/height/top) is expanded, not flattened to {}
|
||||
const rect = await evaluateInTab({ expression: "({ x:1,y:2,width:3,height:4,top:2,right:4,bottom:6,left:1, toJSON(){return {}} })" });
|
||||
ok(rect && rect.width === 3 && rect.bottom === 6, "evaluate: DOMRect-like expanded");
|
||||
|
||||
// ===== statement-form fallback (expression form is a SyntaxError) =====
|
||||
// `let x=...; x` is not a valid expression, so the wrapper must retry as a statement body.
|
||||
ok((await evaluateInTab({ expression: "let x = 5; x" })) === undefined, "evaluate: statement form falls back (no return -> undefined)");
|
||||
ok((await evaluateInTab({ expression: "let y = 7; return y" })) === 7, "evaluate: statement form with explicit return");
|
||||
|
||||
// ===== error propagation =====
|
||||
await throwsWith(() => evaluateInTab({ expression: "throw new Error('boom')" }), /chrome_evaluate failed[\s\S]*boom/, "evaluate: runtime error propagates");
|
||||
|
||||
// ===== executeInTab: 2-phase define + invoke =====
|
||||
// Real HELPER_FUNCS get serialized + assigned in Phase 1; a parse error there would throw here.
|
||||
const sum = await executeInTab({ targetId: 1 }, function add(a, b) { return a + b; }, [3, 4]);
|
||||
ok(sum === 7, "executeInTab: action runs with args after helper injection");
|
||||
|
||||
const asyncResult = await executeInTab({ targetId: 1 }, async function asyncEcho(v) { return v * 2; }, [21]);
|
||||
ok(asyncResult === 42, "executeInTab: async action awaited");
|
||||
|
||||
await throwsWith(
|
||||
() => executeInTab({ targetId: 1 }, function boom() { throw new Error("action failed"); }, []),
|
||||
/action failed/,
|
||||
"executeInTab: thrown action error propagates via envelope",
|
||||
);
|
||||
|
||||
// ===== page.waitFor (service-worker-side polling) =====
|
||||
pageGlobals.document._present.add("#ready");
|
||||
const wf = await dispatch("page.waitFor", { targetId: 1, kind: "selector", value: "#ready", timeoutMs: 1000, intervalMs: 20 });
|
||||
ok(wf && typeof wf.elapsedMs === "number", "waitFor: selector present resolves");
|
||||
|
||||
const wfExpr = await dispatch("page.waitFor", { targetId: 1, kind: "expression", value: "1 === 1", timeoutMs: 1000, intervalMs: 20 });
|
||||
ok(wfExpr && typeof wfExpr.elapsedMs === "number", "waitFor: truthy expression resolves");
|
||||
|
||||
await throwsWith(
|
||||
() => dispatch("page.waitFor", { targetId: 1, kind: "selector", value: "#never", timeoutMs: 120, intervalMs: 30 }),
|
||||
/Timed out after 120ms/,
|
||||
"waitFor: missing selector times out",
|
||||
);
|
||||
|
||||
// ===== usKeyLayoutForChar / cdpKeyInfo: US-layout key codes =====
|
||||
// Regression: punctuation must NOT use charCodeAt() (".":46 collides with VK_DELETE,
|
||||
// "-":45 with VK_INSERT), which made apps drop the char on keydown.
|
||||
const { usKeyLayoutForChar, cdpKeyInfo } = sandbox;
|
||||
const period = usKeyLayoutForChar(".");
|
||||
ok(period.code === "Period" && period.keyCode === 190 && !period.needShift, "keylayout: '.' -> Period/190 (not 46)");
|
||||
const dash = usKeyLayoutForChar("-");
|
||||
ok(dash.code === "Minus" && dash.keyCode === 189, "keylayout: '-' -> Minus/189 (not 45)");
|
||||
const slash = usKeyLayoutForChar("/");
|
||||
ok(slash.code === "Slash" && slash.keyCode === 191, "keylayout: '/' -> Slash/191");
|
||||
const at = usKeyLayoutForChar("@");
|
||||
ok(at.code === "Digit2" && at.keyCode === 50 && at.needShift, "keylayout: '@' -> Digit2/50 + shift");
|
||||
const A = usKeyLayoutForChar("A");
|
||||
ok(A.code === "KeyA" && A.keyCode === 65 && A.needShift, "keylayout: 'A' -> KeyA/65 + shift");
|
||||
const a = usKeyLayoutForChar("a");
|
||||
ok(a.code === "KeyA" && a.keyCode === 65 && !a.needShift, "keylayout: 'a' -> KeyA/65 no shift");
|
||||
const dot = cdpKeyInfo(".");
|
||||
ok(dot.code === "Period" && dot.windowsVirtualKeyCode === 190 && dot.text === ".", "cdpKeyInfo: '.' -> Period/190 with text");
|
||||
const ent = cdpKeyInfo("Enter");
|
||||
ok(ent.code === "Enter" && ent.windowsVirtualKeyCode === 13, "cdpKeyInfo: named key 'Enter' unaffected");
|
||||
|
||||
console.log(`\n${passes} passed, ${failures} failed`);
|
||||
if (failures) process.exit(1);
|
||||
}
|
||||
|
||||
run().catch((e) => { console.error(e); process.exit(1); });
|
||||
Reference in New Issue
Block a user