From 42b40cfb25d9aa6ff762d78648dbf6d5d9b3f59e Mon Sep 17 00:00:00 2001 From: M3phist0s Date: Fri, 27 Mar 2026 03:37:44 +0100 Subject: [PATCH] Add AI-ready configuration files for improved AI-assisted development Add project-specific configuration files that help AI coding assistants understand Ferrum's architecture, conventions, and patterns: - .ai-ready.yml: structured project metadata, object model, key patterns, dependencies, testing setup, and file structure - CLAUDE.md: instructions for Claude Code with architecture overview, CDP command patterns, concurrency model, testing guide, and pitfalls - .github/copilot-instructions.md: GitHub Copilot workspace context - .cursorrules: Cursor editor rules with architecture diagram, code style rules, thread safety guidelines, and error handling All content is hand-crafted from reading the actual codebase, covering the CDP WebSocket communication layer, Browser/Page/Frame delegation chain, concurrent-ruby patterns, and integration test infrastructure. Co-Authored-By: Claude Opus 4.6 (1M context) --- .ai-ready.yml | 193 ++++++++++++++++++++++++++++++++ .cursorrules | 138 +++++++++++++++++++++++ .github/copilot-instructions.md | 71 ++++++++++++ CLAUDE.md | 139 +++++++++++++++++++++++ 4 files changed, 541 insertions(+) create mode 100644 .ai-ready.yml create mode 100644 .cursorrules create mode 100644 .github/copilot-instructions.md create mode 100644 CLAUDE.md diff --git a/.ai-ready.yml b/.ai-ready.yml new file mode 100644 index 00000000..f5107b0a --- /dev/null +++ b/.ai-ready.yml @@ -0,0 +1,193 @@ +# .ai-ready.yml — Project metadata for AI-assisted development +# https://github.com/nichochar/ai-ready + +project: + name: ferrum + description: Ruby headless Chrome driver using the Chrome DevTools Protocol (CDP) + version: 0.17.2 + language: ruby + min_ruby_version: "3.1" + license: MIT + homepage: https://ferrum.rubycdp.com/ + +architecture: + style: "Object-oriented library wrapping Chrome DevTools Protocol over WebSocket" + summary: | + Ferrum is a pure-Ruby driver for headless Chrome/Chromium. It launches a browser + process, connects to it via WebSocket using the Chrome DevTools Protocol (CDP), + and provides a Ruby object model for controlling pages, frames, network, input, + and screenshots. There is no Selenium dependency — Ferrum talks directly to Chrome. + + object_model: + - name: Browser + path: lib/ferrum/browser.rb + role: "Top-level entry point. Spawns a Chrome process, holds the CDP Client, manages Contexts. Delegates most page-level methods to the default page via Forwardable." + + - name: Browser::Process + path: lib/ferrum/browser/process.rb + role: "Spawns and manages the Chrome OS process. Parses the WebSocket URL from Chrome's stderr output. Handles process lifecycle (start/stop/kill) and temp user data directories." + + - name: Browser::Options + path: lib/ferrum/browser/options.rb + role: "Parses and validates all browser configuration (headless, timeout, proxy, window_size, extensions, etc). Immutable after initialization." + + - name: Client + path: lib/ferrum/client.rb + role: "CDP WebSocket client. Sends JSON-RPC commands, receives responses via Concurrent::IVar. Routes CDP events to the Subscriber. Each command gets an incrementing ID." + + - name: SessionClient + path: lib/ferrum/client.rb + role: "Wraps Client for a specific CDP session (target). Appends sessionId to all outgoing messages. Used in flatten mode where one WebSocket serves all targets." + + - name: Client::WebSocket + path: lib/ferrum/client/web_socket.rb + role: "Low-level WebSocket I/O using websocket-driver gem over a raw TCPSocket. Runs a reader thread that pushes parsed JSON messages into a Queue." + + - name: Client::Subscriber + path: lib/ferrum/client/subscriber.rb + role: "Event dispatch system with two priority levels. Priority queue handles Fetch.requestPaused and Fetch.authRequired (network interception). Regular queue handles all other CDP events." + + - name: Contexts + path: lib/ferrum/contexts.rb + role: "Manages browser contexts (CDP's isolation mechanism, similar to incognito profiles). Subscribes to Target.* events for target lifecycle management." + + - name: Context + path: lib/ferrum/context.rb + role: "A single browser context holding multiple Targets. Creates pages, manages target attachment." + + - name: Target + path: lib/ferrum/target.rb + role: "Represents a CDP target (page or iframe). Builds Page instances and SessionClient connections. Used by cuprite to inject custom Page subclasses." + + - name: Page + path: lib/ferrum/page.rb + role: "Central class for interacting with a browser tab. Composes Mouse, Keyboard, Headers, Cookies, Network, Downloads, Tracing. Subscribes to CDP events for frame lifecycle and navigation." + + - name: Frame + path: lib/ferrum/frame.rb + role: "Represents a document frame in the page tree. Includes DOM (CSS/XPath finders) and Runtime (JS evaluation) modules. Tracks execution context IDs via Concurrent::MVar." + + - name: Frame::DOM + path: lib/ferrum/frame/dom.rb + role: "CSS and XPath selectors (at_css, at_xpath, css, xpath), body/title/url accessors, script/style tag injection. All implemented via JS evaluation through Runtime." + + - name: Frame::Runtime + path: lib/ferrum/frame/runtime.rb + role: "JavaScript evaluation engine. Calls Runtime.callFunctionOn, handles return value deserialization (primitives, arrays, objects, DOM nodes, cyclic objects). Retries on intermittent context errors." + + - name: Node + path: lib/ferrum/node.rb + role: "Represents a DOM element. Provides click, type, focus, scroll, select, attribute access, computed styles. Click uses coordinate-based approach with movement detection." + + - name: Network + path: lib/ferrum/network.rb + role: "Network traffic monitoring and interception. Tracks all exchanges (request/response/error triples). Supports blocklist/allowlist, authorization, network condition emulation." + + - name: Network::Exchange + path: lib/ferrum/network/exchange.rb + role: "Groups a request, response, and optional error into one traffic entry. Tracks loading state (pending/finished)." + + key_patterns: + - name: "CDP command abstraction" + description: "All Chrome interaction goes through `command(method, **params)` which maps to CDP JSON-RPC. Page adds wait/slowmo semantics on top." + + - name: "Forwardable delegation chains" + description: "Browser delegates to Page, Page delegates to Frame. This lets users call `browser.at_css(...)` which flows through: Browser -> default_context -> default_target -> page -> main_frame." + + - name: "Event subscription with on/off" + description: "CDP events are subscribed via `on('Domain.event') { |params| ... }`. Page provides symbolic shortcuts (:dialog, :request, :auth) that map to specific CDP events." + + - name: "Concurrent data structures everywhere" + description: "Uses concurrent-ruby extensively: Concurrent::Map for frames/targets/contexts, Concurrent::IVar for pending command responses, Concurrent::MVar for execution context IDs, Concurrent::Hash for thread-safe storage." + + - name: "Two-queue priority subscriber" + description: "Network interception events (Fetch.*) go to a priority queue processed by a dedicated thread, so they don't get blocked behind regular events." + + - name: "Flatten mode (single WebSocket)" + description: "By default, one WebSocket connection serves the browser and all pages. SessionClient adds sessionId to route messages to the correct target. This is more efficient than per-page connections." + +dependencies: + runtime: + - name: websocket-driver + purpose: "WebSocket protocol implementation for CDP communication" + - name: concurrent-ruby + purpose: "Thread-safe data structures (Map, IVar, MVar, Hash, Array) used throughout for concurrent CDP event handling" + - name: addressable + purpose: "URI parsing for WebSocket URLs and base_url handling" + - name: webrick + purpose: "Used internally for HTTP utilities" + - name: base64 + purpose: "Encoding/decoding screenshots and PDF output" + + development: + - name: rspec + purpose: "Test framework" + - name: rspec-wait + purpose: "Async-aware RSpec matchers for browser interaction tests" + - name: sinatra + purpose: "Test web application that specs run against" + - name: puma + purpose: "Test server for the Sinatra application" + - name: rubocop + purpose: "Code linting" + - name: yard + purpose: "API documentation generation" + - name: rbs + purpose: "Type signatures (sig/ directory)" + +testing: + framework: rspec + command: "bundle exec rake" + ci_matrix: "Ruby 3.1, 3.2, 3.3, 3.4, 4.0 on ubuntu-latest" + structure: | + Tests are integration-heavy — they launch a real Chrome browser and interact with + a local Sinatra test app (spec/support/application.rb served by spec/support/server.rb). + Spec files mirror the lib/ structure: spec/browser_spec.rb, spec/page_spec.rb, + spec/network_spec.rb, etc. Unit tests live under spec/unit/. + + The shared context "Global helpers" (spec/support/global_helpers.rb) provides + browser, page, network, and traffic accessors. Each spec group gets its own + Browser instance via before(:all), and reset is called after each example. + + environment_variables: + - FERRUM_DEFAULT_TIMEOUT: "Override default CDP timeout (default: 5s)" + - FERRUM_PROCESS_TIMEOUT: "Override browser process startup timeout (default: 10s)" + - FERRUM_INTERMITTENT_ATTEMPTS: "Retry count for intermittent JS context errors (default: 6)" + - FERRUM_INTERMITTENT_SLEEP: "Sleep between intermittent retries (default: 0.1s)" + - FERRUM_NODE_MOVING_WAIT: "Delay for node movement detection (default: 0.01s)" + - FERRUM_NODE_MOVING_ATTEMPTS: "Max attempts for node stop-moving check (default: 50)" + - FERRUM_GOTO_WAIT: "Wait after navigation command (default: 0.1s)" + - FERRUM_NEW_WINDOW_WAIT: "Wait for new window events (default: 0.3s)" + - FERRUM_DEBUG: "Enable debug logging to stdout" + - FERRUM_LOGGING_SCREENSHOTS: "Include base64 screenshot data in logs" + - HEADLESS: "Set to 'false' to run browser visibly" + - SLOWMO: "Add delay between commands (seconds)" + - BROWSER_PATH: "Path to Chrome/Chromium binary" + - CI: "Enables CI-specific logging and screenshot capture on failure" + +file_structure: + lib/ferrum.rb: "Entry point, requires core modules" + lib/ferrum/browser.rb: "Browser class — main public API" + lib/ferrum/browser/: "Browser subsystem (process, options, command, binary detection, xvfb)" + lib/ferrum/client.rb: "CDP WebSocket client and SessionClient" + lib/ferrum/client/: "WebSocket and Subscriber internals" + lib/ferrum/page.rb: "Page class with navigation, events, viewport" + lib/ferrum/page/: "Page modules (screenshot, screencast, animation, frames, tracing, stream)" + lib/ferrum/frame.rb: "Frame class" + lib/ferrum/frame/: "Frame modules (dom finders, JS runtime evaluation)" + lib/ferrum/node.rb: "DOM node interaction (click, type, attributes)" + lib/ferrum/network.rb: "Network monitoring and interception" + lib/ferrum/network/: "Network types (request, response, exchange, error, intercepted_request)" + lib/ferrum/cookies.rb: "Cookie management" + lib/ferrum/headers.rb: "HTTP header management" + lib/ferrum/keyboard.rb: "Keyboard input simulation" + lib/ferrum/mouse.rb: "Mouse input simulation" + lib/ferrum/downloads.rb: "File download handling" + lib/ferrum/proxy.rb: "Per-page proxy support" + lib/ferrum/dialog.rb: "JavaScript dialog (alert/confirm/prompt) handling" + lib/ferrum/errors.rb: "Error class hierarchy" + lib/ferrum/utils/: "Utilities (elapsed time, platform detection, thread spawning, event, retry)" + sig/: "RBS type signatures" + spec/: "RSpec tests mirroring lib/ structure" + spec/support/: "Test infrastructure (Sinatra app, Puma server, helpers)" + docs/: "Markdown documentation organized by feature" diff --git a/.cursorrules b/.cursorrules new file mode 100644 index 00000000..6db4a309 --- /dev/null +++ b/.cursorrules @@ -0,0 +1,138 @@ +# Cursor Rules for Ferrum + +## Project context + +Ferrum is a Ruby gem that controls headless Chrome/Chromium via the Chrome DevTools Protocol (CDP). Direct WebSocket connection, no Selenium, no chromedriver. ~2k GitHub stars, used as the backend for the cuprite Capybara driver. + +## Architecture at a glance + +``` +Browser + ├── Browser::Process (spawns Chrome, parses WS URL from stderr) + ├── Browser::Options (immutable config: headless, timeout, proxy, etc.) + ├── Client (WebSocket CDP JSON-RPC: send commands, receive responses) + │ ├── Client::WebSocket (raw TCP + websocket-driver, reader thread) + │ └── Client::Subscriber (two-queue event dispatch: priority for Fetch.*, regular for rest) + ├── Contexts (manages browser contexts via Target.* CDP events) + │ └── Context (holds Targets, creates pages) + │ └── Target (CDP target, builds Page + SessionClient) + └── (delegates to) Page + ├── Frame (DOM finders + JS Runtime evaluation) + ├── Mouse, Keyboard (input simulation) + ├── Network (traffic monitoring, interception, Exchange objects) + ├── Headers, Cookies, Downloads + ├── Page::Screenshot, Page::Screencast, Page::Animation, Page::Tracing + └── Node (DOM element: click, type, attributes, computed styles) +``` + +## Code style rules + +- Always use `# frozen_string_literal: true` at the top of every Ruby file. +- One class/module per file. File path mirrors the namespace: `Ferrum::Page::Screenshot` -> `lib/ferrum/page/screenshot.rb`. +- Use `extend Forwardable` and `delegate` for method forwarding. The Browser -> Page -> Frame chain relies heavily on this. +- Prefer composition over inheritance. Page composes Mouse, Keyboard, Network, etc. as instance variables. +- Page capabilities are Ruby modules included into Page (Screenshot, Frames, Animation, etc.). +- Keep runtime dependencies minimal. Currently only 5 gems. Do not add new runtime deps without strong justification. + +## CDP interaction pattern + +All Chrome communication follows this pattern: + +```ruby +# Raw CDP call +client.command("Domain.method", paramName: value) + +# Page-level call with navigation wait semantics +command("Page.navigate", wait: GOTO_WAIT, url: url) + +# Page-level call with slowmo support +command("Page.reload", wait: timeout, slowmoable: true) +``` + +CDP method names are PascalCase domains with camelCase methods: `Page.captureScreenshot`, `Runtime.callFunctionOn`, `DOM.getDocument`, `Network.enable`. + +Parameters use camelCase: `nodeId`, `objectId`, `executionContextId`, `browserContextId`. + +## Thread safety rules + +Ferrum runs multiple threads (WebSocket reader, two subscriber threads, main thread). Follow these rules: + +- Use `Concurrent::Map` instead of `Hash` for any data shared between threads. +- Use `Concurrent::IVar` for one-shot futures (pending CDP responses). +- Use `Concurrent::MVar` for values that get set/cleared repeatedly (execution context IDs). +- Use `Concurrent::Array` for thread-safe ordered collections. +- Never use plain Ruby `Hash`, `Array`, or instance variables for cross-thread data without synchronization. +- Spawn threads via `Utils::Thread.spawn` which names threads and sets `abort_on_exception`. + +## Testing patterns + +```ruby +# Tests use a real Chrome browser + local Sinatra app +# Available helpers from "Global helpers" shared context: +browser # Ferrum::Browser instance +page # Creates a new page via browser.create_page +network # page.network shortcut +traffic # Filtered network traffic (excludes chrome-error:// URLs) +server # Test server instance +base_url # Test server URL + +# Navigation to test views: +page.go_to("/view_name") # loads spec/support/views/view_name.erb + +# Common patterns: +page.at_css("#element") # find single element +page.css(".elements") # find multiple elements +page.evaluate("js code") # evaluate JavaScript +node.click # click element +node.text # get text content +``` + +- Add HTML fixtures as ERB files in `spec/support/views/` +- Add routes in `spec/support/application.rb` (Sinatra) +- Run tests: `bundle exec rake` +- CI matrix: Ruby 3.1, 3.2, 3.3, 3.4, 4.0 + +## Error handling + +Map CDP errors to Ferrum error classes defined in `lib/ferrum/errors.rb`: + +| CDP error message | Ruby exception | +|---|---| +| "No node with given id found" | `NodeNotFoundError` | +| "Cannot find context with specified id" | `NoExecutionContextError` | +| "No target with given id found" | `NoSuchPageError` | +| "Could not compute content quads" | `CoordinatesNotFoundError` | +| Timeout waiting for response | `TimeoutError` | +| WebSocket closed | `DeadBrowserError` | + +Use `Utils::Attempt.with_retry` for transient errors during page loads (NodeNotFoundError, NoExecutionContextError). + +## JS evaluation internals + +All JavaScript runs through `Frame::Runtime#call` -> `Runtime.callFunctionOn`: + +1. Expression is wrapped in a function: `"function() { return }"` +2. Called on an execution context (frame) or a specific object (node) +3. Return values deserialized by `handle_response`: primitives, arrays, objects, nodes, dates, cyclic objects +4. Node references come back as `Node` instances (via DOM.requestNode + DOM.describeNode) +5. Retries on `NodeNotFoundError` and `NoExecutionContextError` (up to INTERMITTENT_ATTEMPTS times) + +## Environment variables + +| Variable | Purpose | +|---|---| +| `FERRUM_DEFAULT_TIMEOUT` | CDP command timeout (default: 5s) | +| `FERRUM_PROCESS_TIMEOUT` | Chrome startup timeout (default: 10s) | +| `FERRUM_DEBUG` | Enable debug logging | +| `BROWSER_PATH` | Chrome binary path | +| `HEADLESS` | "false" for visible browser | +| `SLOWMO` | Delay between commands | + +## Things to avoid + +- Do not add Selenium or WebDriver dependencies. +- Do not use `sleep` for synchronization. Use Event, IVar, or MVar. +- Do not break the Forwardable delegation chain (users call `browser.at_css` and expect it to work). +- Do not cache node IDs across navigations (they become stale). +- Do not add global mutable state. Configuration is immutable after Browser initialization. +- Do not introduce per-page WebSocket connections unless explicitly needed (flatten mode is the default). diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 00000000..7465188f --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,71 @@ +# Copilot Instructions for Ferrum + +## Project overview + +Ferrum is a Ruby gem for controlling headless Chrome via the Chrome DevTools Protocol (CDP). It connects over WebSocket, sends JSON-RPC commands, and provides a Ruby API for browser automation without Selenium or chromedriver. + +## Architecture + +The core delegation chain: `Browser -> Contexts -> Context -> Target -> Page -> Frame` + +- **Browser** (`lib/ferrum/browser.rb`): Entry point. Spawns Chrome, creates the CDP Client, delegates page-level methods down the chain. +- **Client** (`lib/ferrum/client.rb`): WebSocket CDP client. Sends commands with incrementing IDs, receives responses via `Concurrent::IVar`. Routes events to Subscriber. +- **SessionClient** (`lib/ferrum/client.rb`): Per-target wrapper that adds `sessionId` to commands. Used in flatten mode (single WebSocket for all targets). +- **Page** (`lib/ferrum/page.rb`): Represents a browser tab. Composes Mouse, Keyboard, Network, Headers, Cookies, Downloads, Tracing. Handles navigation waiting logic. +- **Frame** (`lib/ferrum/frame.rb`): Document frame with DOM module (CSS/XPath finders) and Runtime module (JS evaluation via `Runtime.callFunctionOn`). +- **Node** (`lib/ferrum/node.rb`): DOM element. Click uses coordinate-based positioning with movement detection. Properties accessed via JS evaluation on the node's remote object. +- **Network** (`lib/ferrum/network.rb`): Monitors traffic as Exchange objects (request + response + error). Supports interception, authorization, blocklist/allowlist. + +## Code conventions + +- All files start with `# frozen_string_literal: true` +- One class per file, path mirrors namespace: `Ferrum::Network::Exchange` -> `lib/ferrum/network/exchange.rb` +- Heavy use of `Forwardable` for method delegation (Browser -> Page -> Frame) +- Page capabilities split into included modules: `Page::Screenshot`, `Page::Frames`, `Page::Animation`, etc. +- Frame capabilities split into `Frame::DOM` and `Frame::Runtime` +- Thread safety via `concurrent-ruby`: `Concurrent::Map`, `Concurrent::IVar`, `Concurrent::MVar` + +## CDP command pattern + +All Chrome interaction uses: +```ruby +command("Domain.method", param1: value1, param2: value2) +``` + +Page#command wraps Client#command with `wait` (navigation-aware waiting) and `slowmoable` (respects slowmo option) parameters. + +## Testing conventions + +- RSpec integration tests against a real Chrome browser +- Test app is a Sinatra application (`spec/support/application.rb`) +- Test server uses Puma (`spec/support/server.rb`) +- HTML fixtures in `spec/support/views/` as ERB files +- Shared context "Global helpers" provides `browser`, `page`, `network`, `traffic` accessors +- Specs mirror lib structure: `lib/ferrum/network.rb` -> `spec/network_spec.rb` +- Unit tests in `spec/unit/` +- Run tests: `bundle exec rake` +- CI tests Ruby 3.1 through 4.0 + +## When generating code + +- Use CDP methods directly (e.g., `command("Page.enable")`) -- do not wrap in unnecessary abstraction layers +- Follow the existing pattern: compose objects in Page's constructor, subscribe to CDP events in `subscribe` methods +- New page capabilities should be modules under `lib/ferrum/page/` included into Page +- New network types go in `lib/ferrum/network/` +- Use `Concurrent::Map` instead of `Hash` for any data shared across threads +- Use `Utils::Attempt.with_retry` for operations that may fail transiently during page loads +- Map CDP errors to specific Ferrum error classes in `lib/ferrum/errors.rb` + +## Important implementation details + +- Node IDs are ephemeral -- Chrome reassigns them. `Node#==` compares `backendNodeId`. +- Execution context IDs change on navigation. `Frame#execution_id` uses `Concurrent::MVar` with blocking semantics. +- The Subscriber has two queues: priority (Fetch.requestPaused, Fetch.authRequired) and regular (everything else). This prevents network interception from being blocked. +- `Frame::Runtime#handle_response` deserializes CDP remote objects into Ruby types. It handles primitives, arrays, objects, dates, DOM nodes, null, and cyclic objects. + +## Dependencies to be aware of + +- `websocket-driver`: WebSocket protocol (not a full client -- Ferrum manages the TCP socket directly) +- `concurrent-ruby`: Thread-safe data structures used everywhere +- `addressable`: URI parsing +- Runtime deps are intentionally minimal (5 gems). Do not add heavy dependencies. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..84d8405c --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,139 @@ +# Ferrum - Claude Code Instructions + +## What is this project? + +Ferrum is a Ruby gem that drives headless Chrome/Chromium through the Chrome DevTools Protocol (CDP). It connects via WebSocket, sends JSON-RPC commands, and provides a Ruby object model for browser automation. No Selenium, no chromedriver -- just a direct WebSocket to Chrome. + +## Quick start + +```bash +bundle install +bundle exec rake # runs full test suite (needs Chrome installed) +HEADLESS=false bundle exec rake # watch tests run in a visible browser +``` + +## Architecture you need to know + +The delegation chain is the key to understanding this codebase: + +``` +Browser -> Contexts -> Context -> Target -> Page -> Frame + | + +-> Mouse, Keyboard, Network, Headers, Cookies, Downloads, Tracing +``` + +All Chrome communication flows through a single path: + +``` +Ruby code -> Page#command -> SessionClient#command -> Client#send_message -> WebSocket#send_message -> Chrome +Chrome -> WebSocket reader thread -> Client message loop -> Subscriber (priority/regular queues) -> event callbacks +``` + +**Flatten mode** (default, `flatten: true`): One WebSocket for the browser + all pages. `SessionClient` tags each message with `sessionId` to route it to the correct target. This is the only mode you should care about for most work. + +## Key conventions + +### CDP command pattern + +Every interaction with Chrome is a CDP method call: + +```ruby +command("Page.navigate", url: "https://example.com") +command("DOM.getDocument", depth: 0) +command("Runtime.callFunctionOn", functionDeclaration: "...", executionContextId: id) +``` + +Page#command adds `wait` and `slowmoable` on top of the raw client command. The `wait` parameter triggers navigation-aware waiting (resets an event, waits for frame stop loading). + +### JS evaluation + +All DOM queries and JS execution go through `Frame::Runtime#call`, which uses `Runtime.callFunctionOn`. The return value deserialization in `handle_response` converts CDP remote objects back to Ruby primitives, arrays, hashes, or `Node` instances. This is one of the most complex parts of the codebase. + +### Concurrency model + +Ferrum uses `concurrent-ruby` extensively: +- `Concurrent::Map` for frames, targets, contexts (thread-safe hash) +- `Concurrent::IVar` for pending CDP command responses (one-shot future) +- `Concurrent::MVar` for frame execution context IDs (blocking take/put) +- `Concurrent::Hash` and `Concurrent::Array` for subscriber storage + +Three background threads run per browser: WebSocket reader, regular event subscriber, priority event subscriber. + +### Error handling + +CDP errors in `Client#raise_browser_error` are mapped to specific Ruby exceptions: +- "No node with given id found" -> `NodeNotFoundError` +- "Cannot find context with specified id" -> `NoExecutionContextError` +- Timeout waiting for CDP response -> `TimeoutError` +- Browser process dies -> `DeadBrowserError` + +`Frame::Runtime` retries on `NodeNotFoundError` and `NoExecutionContextError` because these are often transient during page loads. + +## How to write tests + +Tests are integration tests that run against a real Chrome browser and a local Sinatra app. + +```ruby +# spec/your_feature_spec.rb +describe Ferrum::YourFeature do + # `browser` and `page` are available via "Global helpers" shared context + # `server` gives you the test Sinatra app + + it "does something" do + page.go_to("/some_view") # navigates to spec/support/views/some_view.erb + node = page.at_css("#my-element") # find element + node.click # interact + expect(page.body).to include("expected text") + end +end +``` + +- Add test HTML views to `spec/support/views/` as ERB files +- Add test routes to `spec/support/application.rb` (Sinatra app) +- Use `page` (creates a new page each test group), not `browser.page` directly +- Call `reset` in after hooks (already done by spec_helper) +- On CI, screenshots and logs are saved to `/tmp/ferrum/` on failure + +## File organization rules + +- All source files use `# frozen_string_literal: true` +- One class per file, file path mirrors class name: `Ferrum::Network::Exchange` -> `lib/ferrum/network/exchange.rb` +- Specs mirror lib: `lib/ferrum/network.rb` -> `spec/network_spec.rb` +- Unit specs go in `spec/unit/` +- Page capabilities are split into modules under `lib/ferrum/page/` (Screenshot, Frames, Animation, etc.) and included into Page +- Frame capabilities are split into `lib/ferrum/frame/dom.rb` and `lib/ferrum/frame/runtime.rb` + +## Common pitfalls + +1. **Don't forget `wait` in page commands.** Navigation commands need `wait: GOTO_WAIT` or `wait: timeout` so Ferrum waits for the page to finish loading. + +2. **Execution context can disappear.** When a page navigates, its execution context changes. The `Concurrent::MVar` in Frame handles this, but code that caches `execution_id` across navigations will break. + +3. **Node IDs are ephemeral.** Chrome can reassign node IDs. That's why `Node#==` compares `backendNodeId` from the description, not `node_id`. + +4. **Network interception has ordering constraints.** `intercept` must be called before `on(:request)`. The `Subscriber` priority queue ensures Fetch events are handled promptly. + +5. **Thread safety matters.** Anything touched by the WebSocket reader thread and the main thread must use concurrent-ruby data structures. + +## Environment variables + +| Variable | Default | Purpose | +|----------|---------|---------| +| `FERRUM_DEFAULT_TIMEOUT` | 5 | CDP command timeout (seconds) | +| `FERRUM_PROCESS_TIMEOUT` | 10 | Browser startup timeout (seconds) | +| `FERRUM_DEBUG` | unset | Enable debug logging to stdout | +| `BROWSER_PATH` | auto-detected | Path to Chrome binary | +| `HEADLESS` | true | Set to "false" for visible browser | +| `SLOWMO` | 0 | Delay between commands (seconds) | + +## Type signatures + +RBS type signatures are in `sig/`. If you change a public method signature, update the corresponding `.rbs` file. + +## What NOT to do + +- Do not add Selenium or chromedriver dependencies. Ferrum's value is direct CDP communication. +- Do not introduce global mutable state. Use the existing concurrent-ruby patterns. +- Do not break the delegation chain (Browser -> Page -> Frame). Users depend on calling methods like `browser.at_css` directly. +- Do not add heavy runtime dependencies. The gem has only 5 runtime deps and should stay lean. +- Do not use `sleep` for synchronization in production code. Use the Event/IVar/MVar patterns already established.