diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9aae1162..2e675d9d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -68,6 +68,7 @@
- `Browser::Process#default_user_agent`, `#browser_version`, `#protocol_version`, `#v8_version` and `#webkit_version`,
use `Browser#default_user_agent` and `Browser#version` instead
- Ruby 3.1 support, Ferrum requires Ruby 3.2 or newer
+- `#evaluate_async`, `#evaluate_func` and `#evaluate_on`, use `#evaluate` and `Node#evaluate`
## [0.19.1](https://github.com/rubycdp/ferrum/compare/v0.19.0...v0.19.1) (October 16, 2026) ##
@@ -104,6 +105,9 @@
loaded (on CDP's `Network.loadingFinished`, not `Network.responseReceived`, so `exchange.response.body` is always
available), yielding the request's `Network::Exchange`. Doesn't require `network.intercept` to be set up, and is
never fired for requests that fail to load [#294]
+- `#evaluate`/`#execute` take named arguments that become the script's parameters, `page.evaluate("a + b", a: 1,
+ b: 2)`, a function declaration as the script, `await`, and `timeout:`
+- `#evaluate_handle` returning a `Ferrum::RemoteObject`, and `Node#execute`/`Node#evaluate_handle`
- `Ferrum::Browser#quit`/`Ferrum::Browser::Process#stop` accept `wait: false` to return immediately and run process
killing and user-data-directory cleanup on a background thread instead of blocking; the call returns the `Thread`
so callers can `#join` it if they need cleanup to have finished, e.g. before process exit or before reusing a
@@ -119,6 +123,10 @@
- `Ferrum::Page::Stream#stream` now closes the CDP stream handle (`IO.close`) once it's been fully read, so streams
opened for `Ferrum::Browser#pdf` and `Ferrum::Page::Tracing#record` no longer keep their backing storage alive in
the browser; a failed `IO.close` is raised to the caller.
+- `#evaluate`/`#execute` await a returned promise or thenable, keywords are always named parameters and can't be
+ mixed with positional arguments, a thrown non-`Error` raises, and `Node#evaluate` resolves nodes and dates, see
+ [upgrading](docs/13-javascript.md#upgrading-to-021)
+- `Ferrum::ScriptTimeoutError` is a `Ferrum::TimeoutError`
### Fixed
- An out-of-process iframe, e.g. Chrome's PDF viewer for an embedded PDF, appearing while a web worker was connected
diff --git a/README.md b/README.md
index 12358157..524dd303 100644
--- a/README.md
+++ b/README.md
@@ -85,6 +85,57 @@ JS
browser.quit
```
+Pass arguments by name, so the script can use parameters instead of digging
+through `arguments[0]`:
+
+```ruby
+page.evaluate("a + b", a: 1, b: 2) # => 3
+page.evaluate("el.getAttribute(name)", el: page.at_css("a"), name: "href")
+```
+
+A script that starts with a function or arrow declaration is used as-is, which
+is how you run more than one statement:
+
+```ruby
+page.evaluate(<<~JS, a: 1, b: 2)
+ function(a, b) {
+ const sum = a + b;
+ return sum * 3;
+ }
+JS
+```
+
+`await` works directly, and a returned promise is awaited:
+
+```ruby
+page.evaluate("await fetch(url).then(r => r.text())", url: "/api")
+```
+
+Pass `timeout:` (seconds, defaulting to the page timeout) to bound how long the
+script may take before `Ferrum::ScriptTimeoutError` is raised, or `0` to let it
+run for as long as it takes. `timeout:` and `args:` are the only reserved
+keywords; use `args:` when a JavaScript parameter needs one of those names:
+
+```ruby
+page.evaluate("timeout * 2", args: { timeout: 21 }) # => 42
+```
+
+Use `#execute` when you only want the side effects, `#evaluate_handle` when you
+want to keep a value in the browser and pass it back in later, and
+`Ferrum::Node#evaluate` to run a script with `this` bound to an element:
+
+```ruby
+page.execute("window.scrollBy(0, 100)") # => true
+
+list = page.evaluate_handle("document.querySelectorAll('li')")
+page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list)
+
+page.at_css("input").evaluate("this.value")
+```
+
+Coming from 0.20 or earlier? See
+[upgrading to 0.21](docs/13-javascript.md#upgrading-to-021).
+
Do any mouse movements you like:
```ruby
diff --git a/docs/13-javascript.md b/docs/13-javascript.md
index 362d095b..d53d5ad9 100644
--- a/docs/13-javascript.md
+++ b/docs/13-javascript.md
@@ -4,43 +4,76 @@ sidebar_position: 13
# JavaScript
-#### evaluate(expression, \*args)
+#### evaluate(expression, \*args, timeout: nil, \*\*named)
-Evaluate and return result for given JS expression
+Evaluate JavaScript and return the result, converted to Ruby.
-* expression `String` should be valid JavaScript
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
- simple value.
+* expression `String` a JavaScript expression, or a function/arrow declaration, which is used as-is and is how you
+ run more than one statement
+* args positional arguments, reachable as `arguments[0]` and up. Can't be mixed with named ones
+* timeout `Numeric` seconds to wait for the script and any promise it returns, defaults to the page timeout, `0`
+ waits for as long as it takes. Raises `Ferrum::ScriptTimeoutError`, a `Ferrum::TimeoutError`, when exceeded
+* named named arguments, which become the script's parameters. A `Ferrum::Node` arrives as the live element. Use
+ `args: { timeout: 1 }` for a parameter named `timeout` or `args`
```ruby
-page.evaluate("[window.scrollX, window.scrollY]")
+page.evaluate("[window.scrollX, window.scrollY]") # => [0, 0]
+page.evaluate("a + b", a: 1, b: 2) # => 3
+page.evaluate("el.getAttribute(name)", el: page.at_css("a"), name: "href")
+page.evaluate("await fetch(url).then(r => r.status)", url: "/") # => 200
+page.evaluate(<<~JS, c: 3) # => 9
+ function(c) {
+ const sum = 1 + 2;
+ return sum * c;
+ }
+JS
```
-#### evaluate_async(expression, wait_time, \*args)
-
-Evaluate asynchronous expression and return result
+#### evaluate_handle(expression, \*args, timeout: nil, \*\*named) : `Ferrum::RemoteObject`
-* expression `String` should be valid JavaScript
-* wait_time How long we should wait for Promise to resolve or reject
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
- simple value.
+Same as `evaluate`, but keeps the value in the browser and returns a reference you can pass back in as an argument.
+Call `#value` to convert it to Ruby, and `#release` to free it.
```ruby
-page.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
+list = page.evaluate_handle("document.querySelectorAll('li')")
+page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list)
```
-#### execute(expression, \*args)
+#### execute(expression, \*args, timeout: nil, \*\*named) : `Boolean`
+
+Run JavaScript for its side effects. The script is used as a function body, so statements need no wrapping.
-Execute expression. Doesn't return the result
+```ruby
+page.execute("window.scrollBy(0, 100)") # => true
+```
-* expression `String` should be valid JavaScript
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
- simple value.
+`Ferrum::Node` has the same three methods, with `this` bound to the element. Arrow functions don't get `this`, so use a
+`function` declaration when you need it.
```ruby
-page.execute(%(1 + 1)) # => true
+page.at_css("input").evaluate("this.value")
+page.at_css("input").execute("this.value = text", text: "hello")
```
+#### Upgrading to 0.21
+
+0.21 finishes the move to the API above. Ferrum 0.20 runs old-style scripts as before and warns about each change below,
+pointing at your code, so upgrade to 0.20 first and fix its warnings.
+
+| Script | 0.20 | 0.21 | Rewrite |
+|---|---|---|---|
+| Keyword arguments read through `arguments` | one object in `arguments[0]` | named parameters | `evaluate("options.foo", options: { foo: 1 })` |
+| Mixing positional and keyword arguments | keywords appended as an object | `ArgumentError` | pass the object in braces, or name every argument |
+| More than one expression, `"a(); b"` | returns `a()`, skips the rest | `SyntaxError` | `evaluate("function() { a(); return b }")` |
+| A line break before the script | `nil`, nothing runs | runs and returns its value | check it's safe to run, or remove it |
+| Returns a promise | `{}`, not awaited | awaited, raises if rejected | prefix with `void` to keep it from waiting |
+| Returns an object with a `then` method | the object | awaited like a promise | wrap it, `({ value: object })` |
+| Throws something that isn't an `Error` | the thrown value | `Ferrum::JavaScriptError` | return the value instead |
+| `Node#evaluate` returns a node, nodes or a `Date` | `{}` | `Ferrum::Node`, or the date's string | check the code using the result |
+| `evaluate_async(expression, wait, *args)` | deprecated | removed | `evaluate("await …", timeout: wait)` |
+| `evaluate_func(function, *args, on:)` | deprecated | removed | `evaluate(function, a: 1)`, or `node.evaluate(function)` |
+| `evaluate_on(node:, expression:)` | deprecated | removed | `node.evaluate(expression)` |
+
#### evaluate_on_new_document(source, \*\*options) : `String`
Runs JavaScript in every new document of the page and its in-process frames, before the document's own scripts.
@@ -62,7 +95,7 @@ JS
#### isolated_world(name) : `IsolatedWorld`
Returns the frame's isolated world with this name, its own JavaScript globals over the frame's DOM, so the page's
-scripts can't see or override what runs there. It has `evaluate`, `evaluate_async`, `execute` and `evaluate_func`,
+scripts can't see or override what runs there. It has `evaluate`, `evaluate_handle` and `execute`,
takes and returns nodes, and comes back empty in every new document. On a page it's the main frame's world, use
`frame.isolated_world(name)` for an iframe. It's the same world `evaluate_on_new_document` runs in with
`world_name:`.
diff --git a/docs/21-workers.md b/docs/21-workers.md
index 7b8d3d10..32fc6e58 100644
--- a/docs/21-workers.md
+++ b/docs/21-workers.md
@@ -36,7 +36,7 @@ Returns service worker targets registered in this context. Unlike `workers`, the
are not connected to -- call `attach_target` first if you want to interact with one.
```ruby
-page.evaluate_async(%(navigator.serviceWorker.register("/sw.js").then(arguments[0])), 5)
+page.evaluate(%(await navigator.serviceWorker.register("/sw.js")))
sleep 0.1
@@ -91,7 +91,7 @@ Same API as `page.console` -- see [JavaScript](/docs/ferrum/javascript#console).
worker.console.map(&:text) # => ["from worker"]
```
-#### evaluate / execute / evaluate_async
+#### evaluate / execute / evaluate_handle
Same API as on `Page`/`Frame` -- see [JavaScript](/docs/ferrum/javascript). A worker has a single execution context,
so there's no frame to target.
diff --git a/lib/ferrum.rb b/lib/ferrum.rb
index bf1567af..676de110 100644
--- a/lib/ferrum.rb
+++ b/lib/ferrum.rb
@@ -9,6 +9,7 @@
require "ferrum/utils/attempt"
require "ferrum/errors"
require "ferrum/browser"
+require "ferrum/remote_object"
require "ferrum/node"
#
diff --git a/lib/ferrum/browser.rb b/lib/ferrum/browser.rb
index 6d698242..bfa04216 100644
--- a/lib/ferrum/browser.rb
+++ b/lib/ferrum/browser.rb
@@ -36,7 +36,7 @@ class Browser
screenshot pdf mhtml viewport_size device_pixel_ratio
start_screencast stop_screencast
frames frame_by main_frame
- evaluate evaluate_on evaluate_async execute evaluate_func isolated_world
+ evaluate evaluate_handle execute isolated_world
add_script_tag add_style_tag bypass_csp
evaluate_on_new_document remove_script_on_new_document
on position position=
diff --git a/lib/ferrum/client.rb b/lib/ferrum/client.rb
index 48af6f63..613f9a4a 100644
--- a/lib/ferrum/client.rb
+++ b/lib/ferrum/client.rb
@@ -209,7 +209,9 @@ def command(method, async: false, timeout: nil, **params)
#
# @param [Numeric, nil] timeout
# How long to wait for the browser to take the message, and then for its
- # response. Defaults to `protocol_timeout`.
+ # response. Defaults to `protocol_timeout`. `Float::INFINITY` waits for
+ # the response for as long as it takes, still giving the browser
+ # `protocol_timeout` to take the message.
#
# @return [Boolean, Hash]
# `true` when sent asynchronously, otherwise the parsed `"result"`
@@ -217,9 +219,10 @@ def command(method, async: false, timeout: nil, **params)
#
def send_message(message, async:, timeout: nil)
timeout ||= protocol_timeout
+ write_timeout = timeout.infinite? ? protocol_timeout : timeout
if async
- @ws.send_message(message, timeout)
+ @ws.send_message(message, write_timeout)
true
else
raise DeadBrowserError if @ws.messages.closed?
@@ -227,8 +230,8 @@ def send_message(message, async:, timeout: nil)
pending = Concurrent::IVar.new
@pendings[message[:id]] = pending
begin
- @ws.send_message(message, timeout)
- data = pending.value!(timeout)
+ @ws.send_message(message, write_timeout)
+ data = pending.value!(timeout.infinite? ? nil : timeout)
ensure
@pendings.delete(message[:id])
end
diff --git a/lib/ferrum/errors.rb b/lib/ferrum/errors.rb
index 82e35ba9..22f89aea 100644
--- a/lib/ferrum/errors.rb
+++ b/lib/ferrum/errors.rb
@@ -87,7 +87,7 @@ def awaited
end
# Raised when an evaluated script takes too long to return a value.
- class ScriptTimeoutError < Error
+ class ScriptTimeoutError < TimeoutError
#
# Explains that the evaluated script timed out.
#
@@ -216,7 +216,7 @@ class JavaScriptError < BrowserError
def initialize(response)
@class_name, @message = response["exception"]&.values_at("className", "description")
- @message ||= response["text"]
+ @message ||= [response["text"], response.dig("exception", "value")&.inspect].compact.join(" ")
@stack_trace = response["stackTrace"]
super(response.merge("message" => @message))
end
diff --git a/lib/ferrum/frame.rb b/lib/ferrum/frame.rb
index c53c344d..affa06a1 100644
--- a/lib/ferrum/frame.rb
+++ b/lib/ferrum/frame.rb
@@ -167,12 +167,11 @@ def parent
# frame.body # =>
lol
#
def content=(html)
- evaluate_async(%(
+ execute(<<~JS, html: html)
document.open();
- document.write(arguments[0]);
+ document.write(html);
document.close();
- arguments[1](true);
- ), @page.timeout, html)
+ JS
@page.document_node_id
end
alias set_content content=
diff --git a/lib/ferrum/frame/dom.rb b/lib/ferrum/frame/dom.rb
index 3cf1492b..44d34c05 100644
--- a/lib/ferrum/frame/dom.rb
+++ b/lib/ferrum/frame/dom.rb
@@ -27,32 +27,38 @@ class Frame
#
module DOM
SCRIPT_SRC_TAG = <<~JS
- const script = document.createElement("script");
- script.src = arguments[0];
- script.type = arguments[1];
- script.onload = arguments[2];
- document.head.appendChild(script);
+ function(url, type) {
+ const script = document.createElement("script");
+ script.src = url;
+ script.type = type;
+ document.head.appendChild(script);
+ return new Promise(resolve => script.onload = resolve);
+ }
JS
SCRIPT_TEXT_TAG = <<~JS
- const script = document.createElement("script");
- script.text = arguments[0];
- script.type = arguments[1];
- document.head.appendChild(script);
- arguments[2]();
+ function(content, type) {
+ const script = document.createElement("script");
+ script.text = content;
+ script.type = type;
+ document.head.appendChild(script);
+ }
JS
STYLE_TAG = <<~JS
- const style = document.createElement("style");
- style.type = "text/css";
- style.appendChild(document.createTextNode(arguments[0]));
- document.head.appendChild(style);
- arguments[1]();
+ function(content) {
+ const style = document.createElement("style");
+ style.type = "text/css";
+ style.appendChild(document.createTextNode(content));
+ document.head.appendChild(style);
+ }
JS
LINK_TAG = <<~JS
- const link = document.createElement("link");
- link.rel = "stylesheet";
- link.href = arguments[0];
- link.onload = arguments[1];
- document.head.appendChild(link);
+ function(url) {
+ const link = document.createElement("link");
+ link.rel = "stylesheet";
+ link.href = url;
+ document.head.appendChild(link);
+ return new Promise(resolve => link.onload = resolve);
+ }
JS
#
@@ -160,7 +166,7 @@ def xpath(selector, within: nil)
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector: selector, within: within)
end
#
@@ -187,7 +193,7 @@ def at_xpath(selector, within: nil)
return xpath.snapshotItem(0);
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector: selector, within: within)
end
#
@@ -214,7 +220,7 @@ def css(selector, within: nil)
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector: selector, within: within)
end
#
@@ -241,7 +247,7 @@ def at_css(selector, within: nil)
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector: selector, within: within)
end
#
@@ -307,17 +313,17 @@ def wait_for_selector(css: nil, xpath: nil, within: nil, timeout: @page.timeout,
# browser.add_script_tag(url: "http://example.com/stylesheet.css") # => true
#
def add_script_tag(url: nil, path: nil, content: nil, type: "text/javascript")
- expr, *args = if url
- [SCRIPT_SRC_TAG, url, type]
- elsif path || content
- if path
- content = File.read(path)
- content += "\n//# sourceURL=#{path}"
- end
- [SCRIPT_TEXT_TAG, content, type]
- end
+ if url
+ evaluate(SCRIPT_SRC_TAG, url: url, type: type)
+ elsif path || content
+ if path
+ content = File.read(path)
+ content += "\n//# sourceURL=#{path}"
+ end
+ evaluate(SCRIPT_TEXT_TAG, content: content, type: type)
+ end
- evaluate_async(expr, @page.timeout, *args)
+ true
end
#
@@ -333,17 +339,17 @@ def add_script_tag(url: nil, path: nil, content: nil, type: "text/javascript")
# browser.add_style_tag(content: "h1 { font-size: 40px; }") # => true
#
def add_style_tag(url: nil, path: nil, content: nil)
- expr, *args = if url
- [LINK_TAG, url]
- elsif path || content
- if path
- content = File.read(path)
- content += "\n//# sourceURL=#{path}"
- end
- [STYLE_TAG, content]
- end
+ if url
+ evaluate(LINK_TAG, url: url)
+ elsif path || content
+ if path
+ content = File.read(path)
+ content += "\n//# sourceURL=#{path}"
+ end
+ evaluate(STYLE_TAG, content: content)
+ end
- evaluate_async(expr, @page.timeout, *args)
+ true
end
end
end
diff --git a/lib/ferrum/frame/runtime.rb b/lib/ferrum/frame/runtime.rb
index 535e9123..aa968e74 100644
--- a/lib/ferrum/frame/runtime.rb
+++ b/lib/ferrum/frame/runtime.rb
@@ -1,6 +1,7 @@
# frozen_string_literal: true
require "singleton"
+require "ferrum/frame/script"
module Ferrum
#
@@ -22,164 +23,320 @@ def inspect
class Frame
#
- # Evaluates and executes JavaScript in a frame's execution context via
+ # Runs JavaScript in a frame's execution context via
# `Runtime.callFunctionOn`, converting arguments and return values
# between Ruby and JS, and resolving object/array/node results
# (including cyclic ones, via {CyclicObject}) into Ruby equivalents.
#
+ # There are three entry points, all of which accept the script in the
+ # same two shapes and pass arguments the same way:
+ #
+ # * {#evaluate} returns the value, serialized into Ruby.
+ # * {#evaluate_handle} returns a {RemoteObject} that stays in the browser.
+ # * {#execute} discards the value and returns `true`.
+ #
+ # ## Script shapes
+ #
+ # A script is either a bare expression, which gets wrapped in a function
+ # for you, or a function declaration, which is used as-is. Use the latter
+ # whenever you need more than one statement.
+ #
+ # page.evaluate("window.scrollY")
+ # page.evaluate("function() { const a = 1; return a + 1 }")
+ #
+ # Arrow functions keep the `this` of where they're defined, so on a
+ # {Node} use a `function` declaration when the script needs `this`.
+ #
+ # ## Arguments
+ #
+ # Keyword arguments become the function's parameters, so the script can
+ # name what it receives instead of digging through `arguments[0]`:
+ #
+ # page.evaluate("a + b", a: 1, b: 2)
+ #
+ # Each keyword is sent as its own protocol argument, so a {Node} arrives
+ # in JavaScript as the live element rather than as serialized JSON. When
+ # the script is a function declaration, arguments bind to its parameters
+ # by name, and one it doesn't declare raises an `ArgumentError`.
+ #
+ # Positional arguments still work and land in `arguments[0]` and up, but
+ # can't be mixed with keyword ones.
+ #
+ # `timeout:` and `args:` are reserved. Pass `args:` explicitly when you
+ # need a JavaScript parameter that happens to be named after one of them:
+ #
+ # page.evaluate("timeout * 2", args: { timeout: 21 })
+ #
+ # ## Promises
+ #
+ # A returned promise, or any object with a `then` method, is awaited, so
+ # `async`/`await` works directly and there is no separate asynchronous
+ # method:
+ #
+ # page.evaluate("await fetch(url).then(r => r.text())", url: "/api")
+ #
+ # If the script hasn't settled within `timeout:` seconds (defaulting to
+ # the page's timeout) a {ScriptTimeoutError} is raised. `timeout: 0` lets
+ # it run for as long as it takes.
+ #
module Runtime
INTERMITTENT_ATTEMPTS = ENV.fetch("FERRUM_INTERMITTENT_ATTEMPTS", 6).to_i
INTERMITTENT_SLEEP = ENV.fetch("FERRUM_INTERMITTENT_SLEEP", 0.1).to_f
+ # Stands in for a declared parameter the caller didn't pass, which CDP
+ # sends as `undefined`.
+ UNDEFINED = Object.new.freeze
+ private_constant :UNDEFINED
+
#
- # Evaluate and return result for given JS expression.
+ # Evaluates JavaScript and returns the result, serialized into Ruby.
#
# @param [String] expression
- # The JavaScript to evaluate.
+ # A JavaScript expression, or a function declaration.
#
- # @param [Array] args
- # Additional arguments to pass to the JavaScript code.
+ # @param [Array] positional
+ # Arguments reachable as `arguments[0]` and up. Can't be combined
+ # with named ones.
+ #
+ # @param [Hash] keywords
+ # Named arguments, which become the function's parameters, plus:
+ #
+ # @option keywords [Numeric, nil] :timeout
+ # How long to wait for the script, and any promise it returns, to
+ # settle, in seconds. Defaults to the page timeout. `0` disables it.
+ #
+ # @option keywords [Hash] :args
+ # Named arguments whose names would collide with `timeout` or `args`.
+ #
+ # @return [Object]
+ # The result. DOM nodes come back as {Node}, arrays and plain objects
+ # are converted recursively, and cyclic values become {CyclicObject}.
+ #
+ # @raise [ScriptTimeoutError]
+ # The script didn't settle within `timeout`.
+ #
+ # @raise [JavaScriptError]
+ # The script threw.
#
# @example
- # browser.evaluate("[window.scrollX, window.scrollY]")
+ # browser.evaluate("[window.scrollX, window.scrollY]") # => [0, 0]
+ # browser.evaluate("a + b", a: 1, b: 2) # => 3
+ # browser.evaluate("await fetch(url).then(r => r.status)", url: "/") # => 200
#
- def evaluate(expression, *args)
- expression = format("function() { return %s }", expression)
- call(expression: expression, arguments: args)
+ def evaluate(expression, *positional, **keywords)
+ run(expression, positional, keywords, mode: :value)
end
#
- # Evaluate asynchronous expression and return result.
- #
- # @param [String] expression
- # The JavaScript to evaluate.
+ # Same as {#evaluate}, but returns a {RemoteObject} that keeps the value
+ # in the browser instead of serializing it. Handles can be passed back
+ # in as arguments. Primitives are returned as-is, and DOM nodes as
+ # {Node}, since both are already usable from Ruby.
#
- # @param [Integer] wait
- # How long we should wait for Promise to resolve or reject.
+ # @param (see #evaluate)
+ # @option (see #evaluate)
#
- # @param [Array] args
- # Additional arguments to pass to the JavaScript code.
+ # @return [RemoteObject, Node, Object]
#
# @example
- # browser.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
- #
- def evaluate_async(expression, wait, *args)
- template = <<~JS
- function() {
- return new Promise((__f, __r) => {
- try {
- arguments[arguments.length] = r => __f(r);
- arguments.length = arguments.length + 1;
- setTimeout(() => __r(new Error("timed out promise")), %s);
- %s
- } catch(error) {
- __r(error);
- }
- });
- }
- JS
-
- expression = format(template, wait * 1000, expression)
- call(expression: expression, arguments: args, awaitPromise: true)
+ # list = page.evaluate_handle("document.querySelectorAll('li')")
+ # page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list)
+ #
+ def evaluate_handle(expression, *positional, **keywords)
+ run(expression, positional, keywords, mode: :handle)
end
#
- # Execute expression. Doesn't return the result.
+ # Runs JavaScript for its side effects and discards the result. Unlike
+ # {#evaluate}, the script is used as a function body rather than an
+ # expression, so multiple statements need no wrapping.
#
- # @param [String] expression
- # The JavaScript to evaluate.
+ # @param (see #evaluate)
+ # @option (see #evaluate)
#
- # @param [Array] args
- # Additional arguments to pass to the JavaScript code.
+ # @return [Boolean]
+ # Always `true`.
#
# @example
- # browser.execute(%(1 + 1)) # => true
+ # browser.execute("window.scrollBy(0, 100)") # => true
+ # browser.execute(<<~JS, url: "/next")
+ # history.pushState({}, "", url);
+ # window.dispatchEvent(new Event("popstate"));
+ # JS
#
- def execute(expression, *args)
- expression = format("function() { %s }", expression)
- call(expression: expression, arguments: args, handle: false, returnByValue: true)
+ def execute(expression, *positional, **keywords)
+ run(expression, positional, keywords, mode: :none)
true
end
#
- # Evaluates a raw JS function declaration (unlike {#evaluate}, which
- # wraps the given expression in one), optionally on a specific remote
- # object instead of the frame's global execution context.
+ # @api private
#
- # @param [String] expression
- # A JS function declaration, e.g. `"function(a, b) { return a + b }"`.
+ # Backs {Node#evaluate}, {Node#evaluate_handle}, {Node#execute} and
+ # {RemoteObject#value}. Runs the script with `this` bound to `object`,
+ # using the same argument and script conventions as {#evaluate}.
+ #
+ # @param [Node, RemoteObject] object
+ # The node or object to bind `this` to.
#
- # @param [Array] args
- # Arguments to pass to the function.
+ # @param [Symbol] mode
+ # `:value`, `:handle` or `:none`.
+ #
+ # @return [Object]
+ #
+ def evaluate_in(object, expression, positional, keywords, mode: :value)
+ run(expression, positional, keywords, mode: mode, on: object)
+ end
+
+ private
+
#
- # @param [Node, nil] on
- # Remote object to invoke the function on.
+ # Wraps the caller's script and hands it to {#call}. `mode` decides both
+ # how the result is converted and whether the script's value is kept.
#
- def evaluate_func(expression, *args, on: nil)
- call(expression: expression, arguments: args, on: on)
+ def run(expression, positional, keywords, mode:, on: nil)
+ script = Script.new(expression)
+ named = keywords.except(:timeout, :args)
+ params, values = arguments_for(script, positional, named_arguments(keywords[:args], named), mode)
+ call(wrap(script, params, mode), values, on: on, mode: mode, timeout: script_timeout(keywords[:timeout]))
end
#
- # Evaluates an expression against a given node's remote object (+this+
- # refers to the node), returning the raw JS value rather than
- # resolving it to a {Node}/Hash/Array.
+ # Normalizes the two argument styles into a parameter list and a list of
+ # values.
#
- # @param [Node] node
- # The node to evaluate the expression on.
+ def arguments_for(script, positional, named, mode)
+ if positional.any?
+ raise ArgumentError, "Pass arguments either positionally or by name, not both" if named.any?
+
+ return [[], positional]
+ end
+
+ return [[], []] if named.empty?
+ return [[], bind_to_parameters(script, named)] if function?(script, mode)
+
+ [named.keys.map { |key| validate_parameter!(key) }, named.values]
+ end
+
+ def named_arguments(args, named)
+ return named if args.nil?
+ raise ArgumentError, "args: takes a Hash of named arguments, got #{args.class}" unless args.is_a?(Hash)
+
+ args.merge(named)
+ end
+
#
- # @param [String] expression
- # The JavaScript to evaluate.
+ # Orders named arguments to match a declaration's own parameter list, so
+ # that `evaluate("function(a, b) { … }", b: 2, a: 1)` binds by name.
+ # Parameters left out are passed as `undefined`. Falls back to insertion
+ # order when the parameters can't be read, e.g. when destructured.
#
- # @param [Boolean] by_value
- # Whether to return the plain JS value instead of a handle.
+ def bind_to_parameters(script, named)
+ names = script.parameters
+ return named.values unless names
+
+ values = named.transform_keys(&:to_s)
+ unknown = values.keys - names
+ if unknown.any?
+ raise ArgumentError, "#{unknown.map(&:inspect).join(', ')} not declared by the function, " \
+ "which takes (#{names.join(', ')})"
+ end
+
+ last = names.rindex { |name| values.key?(name) }
+ names.take(last.to_i + 1).map { |name| values.fetch(name, UNDEFINED) }
+ end
+
+ def validate_parameter!(key)
+ name = key.to_s
+ return name if name.match?(Script::JS_IDENTIFIER) && !Script::RESERVED_WORDS.include?(name)
+
+ raise ArgumentError, "#{name.inspect} is not a valid JavaScript parameter name"
+ end
+
+ # A named function in a function body is a statement defining it, so
+ # {#execute} runs it as a body rather than calling it.
+ def function?(script, mode)
+ script.declaration? && !(mode == :none && script.named_function?)
+ end
+
#
- # @param [Integer] wait
- # Passed through to the underlying `Runtime.callFunctionOn` command.
+ # Builds the function declaration that is actually sent to the browser:
+ # the caller's declaration as-is, or an expression wrapped in one.
#
- def evaluate_on(node:, expression:, by_value: true, wait: 0)
- options = { handle: true }
- expression = format("function() { return %s }", expression)
- options = { handle: false, returnByValue: true } if by_value
- call(expression: expression, on: node, wait: wait, **options)
+ def wrap(script, params, mode)
+ return script.code if function?(script, mode)
+
+ # Parenthesized on its own lines so that neither automatic semicolon
+ # insertion nor a trailing `//` comment in the caller's expression can
+ # swallow it.
+ body = mode == :none || script.code.empty? ? script.code : "return (\n#{script.code}\n);"
+ "#{'async ' if script.awaits?}function(#{params.join(', ')}) {\n#{body}\n}"
end
- private
+ # Waiting for the response is what bounds the script, so a page that
+ # replaces `Promise` or `setTimeout`, or throttles timers, can't defeat
+ # it.
+ def script_timeout(timeout)
+ seconds = (timeout || @page.timeout).to_f
+ seconds.positive? ? seconds : Float::INFINITY
+ end
- def call(expression:, arguments: [], on: nil, wait: 0, handle: true, **options)
+ #
+ # @param [Symbol] mode
+ # `:value` resolves the result into Ruby, `:handle` returns a
+ # {RemoteObject}, and `:none` discards it.
+ #
+ def call(declaration, values, on:, mode:, timeout:)
# do not rescue -> retry if we operate on an existing node
errors = on ? [] : [NodeNotFoundError, NoExecutionContextError]
Utils::Attempt.with_retry(errors: errors, max: INTERMITTENT_ATTEMPTS, wait: INTERMITTENT_SLEEP) do
- params = options.dup
+ target = call_target(on)
+ target[:returnByValue] = true if mode == :none
- params = params.merge(objectId: resolve_node(on)) if on
+ response = call_function(declaration, values, target, timeout)
+ handle_error(response)
- if params[:executionContextId].nil? && params[:objectId].nil?
- params = params.merge(executionContextId: execution_id!)
+ result = response["result"]
+ case mode
+ when :none then nil
+ when :handle then handle_remote_object(result)
+ else handle_response(result)
end
+ end
+ end
- response = @page.command("Runtime.callFunctionOn",
- wait: wait, slowmoable: true,
- **params.merge(functionDeclaration: expression,
- arguments: prepare_args(arguments)))
- handle_error(response)
- response = response["result"]
+ def call_function(declaration, values, target, timeout)
+ @page.command("Runtime.callFunctionOn",
+ timeout: timeout, slowmoable: true, awaitPromise: true,
+ functionDeclaration: declaration, arguments: prepare_args(values), **target)
+ rescue TimeoutError
+ raise ScriptTimeoutError
+ end
- handle ? handle_response(response) : response["value"]
+ def call_target(on)
+ if on.is_a?(RemoteObject)
+ { objectId: on.remote_id }
+ elsif on
+ { objectId: resolve_node(on) }
+ else
+ { executionContextId: execution_id! }
end
end
- # FIXME: We should have a central place to handle all type of errors
+ # A returned `Error` raises too, as it does when thrown.
def handle_error(response)
result = response["result"]
- return if result["subtype"] != "error"
+ details = response["exceptionDetails"]
+ return if details.nil? && result["subtype"] != "error"
- case result["description"]
- when /\AError: timed out promise/
- raise ScriptTimeoutError
- else
- raise JavaScriptError, response["exceptionDetails"]
- end
+ raise JavaScriptError, details || { "exception" => result }
+ end
+
+ def handle_remote_object(result)
+ return handle_response(result) if result["objectId"].nil? || result["subtype"] == "node"
+
+ RemoteObject.new(self, @page, result)
end
def handle_response(response, check_cyclic: true)
@@ -227,10 +384,15 @@ def resolve_node_options
def prepare_args(args)
args.map do |arg|
- if arg.is_a?(Node)
+ case arg
+ when Node
{ objectId: resolve_node(arg) }
- elsif arg.is_a?(Hash) && arg["objectId"]
- { objectId: arg["objectId"] }
+ when RemoteObject
+ { objectId: arg.remote_id }
+ when UNDEFINED
+ {}
+ when Hash
+ arg["objectId"] ? { objectId: arg["objectId"] } : { value: arg }
else
{ value: arg }
end
diff --git a/lib/ferrum/frame/script.rb b/lib/ferrum/frame/script.rb
new file mode 100644
index 00000000..07f425d9
--- /dev/null
+++ b/lib/ferrum/frame/script.rb
@@ -0,0 +1,193 @@
+# frozen_string_literal: true
+
+require "strscan"
+
+module Ferrum
+ class Frame
+ #
+ # @api private
+ #
+ # A script passed to {Runtime#evaluate} and friends. Reads just enough
+ # JavaScript to tell a function declaration from an expression, to find
+ # its parameters and where the code ends, skipping strings, template
+ # literals, regular expressions and comments so that a `//` inside
+ # `"http://…"` isn't mistaken for one.
+ #
+ class Script
+ FUNCTION_DECLARATION = /\A(?:async\s+)?function[\s*(]/
+ NAMED_FUNCTION = /\A(?:async\s+)?function\s*\*?\s*[A-Za-z_$]/
+ SINGLE_PARAMETER_ARROW = /\A(?:async\s+)?([A-Za-z_$][\w$]*)\s*=>/
+ ASYNC_PREFIX = /\A(?:async\s*)?\z/
+ ARROW = /\A\s*=>/
+
+ # A parameter's name, followed by its default value if any.
+ PARAMETER = /\A([A-Za-z_$][\w$]*)\s*(?:=(?!>)|\z)/
+
+ JS_IDENTIFIER = /\A[A-Za-z_$][A-Za-z0-9_$]*\z/
+
+ # Words that can't name a parameter.
+ RESERVED_WORDS = %w[
+ arguments await break case catch class const continue debugger default delete do else enum eval export extends
+ false finally for function if implements import in instanceof interface let new null package private protected
+ public return static super switch this throw true try typeof var void while with yield
+ ].freeze
+
+ NOISE = %r{\s+|//[^\n]*|/\*.*?(?:\*/|\z)}m
+ STRING = /"(?:\\.|[^"\\\n])*"?|'(?:\\.|[^'\\\n])*'?/m
+ TEMPLATE_CHUNK = /(?:\\.|[^`\\$]|\$(?!\{))*/m
+ REGEXP = %r{/(?:\\.|\[(?:\\.|[^\]\\\n])*\]|[^/\\\n\[])+/[A-Za-z]*}
+ WORD = /[\p{Alnum}_$]+/
+
+ # Characters ending a value, after which a `/` is a division.
+ VALUE_END = %r{[\p{Alnum}_$)\]\}"'`/]}
+
+ # Words after which a `/` starts a regular expression, not a division.
+ REGEXP_KEYWORDS = %w[return typeof instanceof in of new delete void throw case do else yield await].freeze
+
+ # @return [String] the script without surrounding whitespace, comments
+ # and trailing semicolons.
+ attr_reader :code
+
+ #
+ # @param [String] source
+ #
+ def initialize(source)
+ @source = source.to_s
+ @code_start = @code_end = 0
+ @parameter_commas = []
+ @words = Set.new
+ scan
+ @code = @source[@code_start...@code_end]
+ end
+
+ #
+ # Whether the script is a complete function or arrow declaration,
+ # rather than an expression that happens to begin with one, such as the
+ # immediately invoked `function() { … }()`.
+ #
+ # @return [Boolean]
+ #
+ def declaration?
+ arrow? || (@code.match?(FUNCTION_DECLARATION) && @body_end == @code_end)
+ end
+
+ #
+ # Whether the script is a declaration of a named function, which as a
+ # function body is a statement defining it rather than a value.
+ #
+ # @return [Boolean]
+ #
+ def named_function?
+ declaration? && @code.match?(NAMED_FUNCTION)
+ end
+
+ #
+ # Whether the script uses `await`, and so has to run in an `async`
+ # function.
+ #
+ # @return [Boolean]
+ #
+ def awaits?
+ @words.include?("await")
+ end
+
+ #
+ # The parameter names of a declaration, or `nil` when they can't be
+ # read plainly, e.g. because of destructuring or rest parameters.
+ #
+ # @return [Array, nil]
+ #
+ def parameters
+ if (single = @code[SINGLE_PARAMETER_ARROW, 1])
+ return [single]
+ end
+ return unless @paren_close
+
+ bounds = [@paren_open, *@parameter_commas, @paren_close]
+ pieces = bounds.each_cons(2).map { |from, to| @source[(from + 1)...to].strip }
+ pieces.pop if pieces.last.empty?
+ names = pieces.map { |piece| piece[PARAMETER, 1] }
+ names if names.all?
+ end
+
+ private
+
+ def arrow?
+ return true if @code.match?(SINGLE_PARAMETER_ARROW)
+ return false unless @paren_close
+
+ @source[@code_start...@paren_open].match?(ASYNC_PREFIX) && @source[(@paren_close + 1)..].match?(ARROW)
+ end
+
+ def scan
+ scanner = StringScanner.new(@source)
+ stack = []
+ previous = nil
+
+ until scanner.eos?
+ next if scanner.skip(NOISE)
+
+ start = scanner.pos
+ previous = scan_token(scanner, stack, previous)
+ @code_start = start if @code_end.zero?
+ @code_end = scanner.pos unless previous == ";"
+ end
+ end
+
+ def scan_token(scanner, stack, previous)
+ if scanner.scan(WORD)
+ @words << scanner.matched unless previous == "."
+ scanner.matched
+ elsif scanner.scan(STRING) || (regexp_allowed?(previous) && scanner.scan(REGEXP))
+ scanner.matched
+ elsif scanner.skip(/`/)
+ scan_template(scanner, stack)
+ else
+ scan_punctuator(scanner, stack)
+ end
+ end
+
+ def scan_punctuator(scanner, stack)
+ char = scanner.getch
+ track_parameters(char, scanner.pos - 1, stack)
+ case char
+ when "{", "(", "[" then stack.push(char)
+ when ")", "]" then stack.pop
+ when "}"
+ return scan_template(scanner, stack) if stack.pop == :template
+
+ @body_end ||= scanner.pos if stack.empty?
+ end
+ char
+ end
+
+ # Remembers where the first top-level parenthesized group, the
+ # parameter list of a declaration, opens, closes and is split by commas.
+ def track_parameters(char, position, stack)
+ return if @paren_close
+
+ case char
+ when "(" then @paren_open ||= position if stack.empty?
+ when ")" then @paren_close = position if @paren_open && stack == ["("]
+ when "," then @parameter_commas << position if @paren_open && stack == ["("]
+ end
+ end
+
+ def scan_template(scanner, stack)
+ scanner.skip(TEMPLATE_CHUNK)
+ if scanner.skip(/\$\{/)
+ stack.push(:template)
+ else
+ scanner.skip(/`/)
+ end
+ "`"
+ end
+
+ def regexp_allowed?(previous)
+ return true if previous.nil? || REGEXP_KEYWORDS.include?(previous)
+
+ previous.length == 1 && !previous.match?(VALUE_END)
+ end
+ end
+ end
+end
diff --git a/lib/ferrum/node.rb b/lib/ferrum/node.rb
index f8ca9b45..d9cab2e7 100644
--- a/lib/ferrum/node.rb
+++ b/lib/ferrum/node.rb
@@ -17,6 +17,7 @@ class Node
attr_reader :page, :target_id, :description, :tag_name
def initialize(frame, target_id, description, object_id: nil, node_id: nil)
+ @runtime = frame
@page = frame.page
@target_id = target_id
@description = description
@@ -210,9 +211,9 @@ def scroll_into_view
#
# @return [Boolean]
def in_viewport?(of: nil)
- function = <<~JS
- function(element, scope) {
- const rect = element.getBoundingClientRect();
+ evaluate(<<~JS, scope: of)
+ function(scope) {
+ const rect = this.getBoundingClientRect();
const [height, width] = scope
? [scope.offsetHeight, scope.offsetWidth]
: [window.innerHeight, window.innerWidth];
@@ -222,7 +223,6 @@ def in_viewport?(of: nil)
rect.right <= width;
}
JS
- page.evaluate_func(function, self, of)
end
#
@@ -369,15 +369,14 @@ def attribute(name)
# @return [Array]
#
def selected
- function = <<~JS
- function(element) {
- if (element.nodeName.toLowerCase() !== 'select') {
+ evaluate(<<~JS)
+ function() {
+ if (this.nodeName.toLowerCase() !== 'select') {
throw new Error('Element is not a