Early errors may result from parsing the source text of a test file, but
they may also result from parsing some other source text as referenced
through the ES2015 module syntax. The latter form of early error is not
necessarily detectable by ECMAScript parsers, however. Because of this,
the label "early" is not sufficiently precise for all Test262 consumers
to correctly interpret all tests.
Update the "phase" name of "early" to "parse" for all those negative
tests that describe errors resulting from parsing of the file's source
text directly. A forthcoming commit will update the remaining tests to
use a "phase" name that is more specific to module resolution.
Modify $262.uncallableAndIsHTMLDDA() to an optional $262.IsHTMLDDA (whose use must be guarded by a feature of the same name), and narrowly/correctly prescribe its requirements consistent with `document.all`'s behavior in HTML.
Changes to the instructions for interpreting tests will likely produce
new failures for consumers who are updating between revisions of
Test262. Introduce a machine-readable convention for signaling
substantive changes.
Previously, test consumers were encouraged to insert a `throw` statement
as the first statement of tests for early errors. This recommendation
made tests harder to consume, and as an optional transformation,
consumers may have ignored it or simply been unaware it was made. By
explicitly including such a `throw` statement, the tests become more
literal, making them easier to consume and more transparent in their
expectations.
Document expectation for all tests for early errors to include an
explicit `throw` statement. Extend linting script to verify that
contributors are automatically notified of violations and to ensure that
future contributions satisfy this expectation.
Define the expected behavior of new host-defined utilities. These will
facilitate forthcoming tests that concern cross-realm and cross-script
semantics (which are currently untestable using standard ECMAScript
alone).
The project's CONTRIBUTING.md was written with test authors in mind. It
contains details on non-technical metadata (e.g. "author" and "es6id"),
helper function usage, and preferred code structure. In addition, it
elides certain low-level technical details on the requirements for the
runtime environment.
Introduce a new document targeted towards those executing the tests.
Formalize all expectations regarding how the runtime environment should
be defined, how metadata should be interpreted, and how results should
be understood. This information has overlap with the CONTRIBUTING.md
file, but it also contains details that are irrelevant to test authors.
This document can serve as a more formal contract between Test262 and
the implementors who consume it. This allows Test262 to unambiguously
document future modifications to the formal requirements which in turn
supports consumers who maintain their own test harnesses.
Rick Waldron
Mike Pennisi
André Bargull
Jeff Walden
Leo Balter
Thomas Wood
Shu-yu Guo