8.9 KiB

Raw Blame History

Interpreting Test262 Tests

All tests are declared as text files located within this project's test directory. In order to execute Test262 tests, runtimes must observe the following semantics.

Test Execution

Test262 tests are only valid under the runtime environment conditions described here. Test environments may be further modified according to the metadata contained with each test--refer to the "Metadata" section for more details.

Realm Isolation

Each test must be executed in a new ECMAScript realm dedicated to that test. Unless configured otherwise (via the module flag), source text must be interpreted as global code.

Test262-Defined Bindings

The contents of the following files must be evaluated in the test realm's global scope prior to test execution:

harness/assert.js
harness/sta.js

Host-Defined Functions

The following values must be defined as writable, configurable, non-enumerable properties of the global scope prior to test execution.

print A function that exposes the string value of its first argument to the test runner. This is used as a communication mechanism for asynchronous tests (via the async flag, described below).
$ An ordinary object with the following properties:
- createRealm - a function which creates a new ECMAScript Realm, defines this API on the new realm's global object, and returns the $ property of the new realm's global object
- detachArrayBuffer - a function which implements the DetachArrayBuffer abstract operation
- evalScript - a function which accepts a string value as its first argument and executes is as an ECMAScript script according to the following algorithm:
```
1. Let hostDefined be any host-defined values for the provided
   sourceText (obtained in an implementation dependent manner)
2. Let realm be the current Realm Record.
3. Let s be ParseScript(sourceText, realm, hostDefined).
4. If s is a List of errors, then
   a. Let error be the first element of s.
   b. Return
      Completion{[[Type]]: throw, [[Value]]: error, [[Target]]: empty}.
5. Let status be ScriptEvaluation(s).
6. Return Completion(status).
```
- global - a reference to the global object on which $ was initially defined

Strict Mode

Unless configured otherwise (via the noStrict, onlyStrict, module, or raw flags), each test must be executed twice: once in ECMAScript's non-strict mode, and again in ECMAScript's strict mode. To run in strict mode, the test contents must be modified prior to execution--a "use strict" directive must be inserted as the initial character sequence of the file, followed by a semicolon (;) and newline character (\n):

"use strict";

This must precede any additional text modifications described by test metadata.

Modules

Test262 includes tests for ECMAScript 2015 module code, denoted by the "module" metadata flag. Files bearing a name ending in _FIXTURE.js should not be interpreted as standalone tests; they are intended to be referenced by test files.

All module specifiers used by Test262 begin with the character sequence ./. The remaining characters should be interpreted as the name of a file within the same directory as the file under test. The contents of this file must be interpreted as UTF-8-encoded text and supplied to the Source Text Module Record's ParseModule abstract operation. The result of that operation must be returned by the implementation-defined HostResolveImportedModule directly.

For example, consider a test file located at test/language/import/nested/index.js with the following contents:

import * as ns from './dep.js';

Implementers should attempt to resolve this module specifier by loading a file located at test/language/import/nested/dep.js.

Test Results

By default, tests signal failure by generating an uncaught exception. If execution completes without generating an exception, the test must be interpreted as "passing." Any uncaught exception must be interpreted as test failure. These semantics may be modified by any test according to the metadata declared within the test itself (via the negative attribute and the async flag, described below).

Metadata

Each test file may define metadata that describe additional requirements. This information is delimited by the token sequence /*--- and ---*/ and is structured as YAML.

`negative`

These tests are expected to generate an uncaught exception. The value of this attribute is a YAML dictonary with two keys:

phase - the stage of the test interpretation process that the error is expected to be produced; either "early" (meaning, "prior to evaluation") or "runtime" (meaning, "during evaluation"); in the case of "early", additional test transformation may be required--see below
type - the name of the constructor of the expected error

If a test configured with the negative attribute completes without throwing an exception, or if the name of the thrown exception's constructor does not match the specified constructor name, or if the error occurs at a phase that differs from the indicated phase, the test must be interpreted as "failing."

Example:

/*---
negative:
  phase: runtime
  type: ReferenceError
---*/
unresolvable;

Consumers are free to assert the "early" phase as they see fit.

For example, it is possible to insert a throw statement with a unique error type at the beginning of the test file. In this case, the statement should be inserted after the directive desribed in the section titled "Strict Mode" (where appropriate), though it must not be inserted for tests containing the "raw" flag.

`includes`

One or more files whose content must be evaluated in the test realm's global scope prior to test execution. These files are located within the harness/ directory of the Test262 project.

Example

/*---
includes: [testBuildInObject.js]
---*/
testBuiltInObject(Number.prototype.toLocaleString, true, false, [], 0);

`flags`

The flags attribute is an optional value that specifies one or more of the following strings:

onlyStrict The test must be executed just once--in strict mode, only. This must be accomplished using the transformation described in the section titled "Strict Mode".

Example
```
/*---
flags: [onlyStrict]
---*/
var thisVal = null;
[null].forEach(function() {
  thisVal = this;
});
assert.sameValue(thisVal, undefined);
```

noStrict The test must be executed just once--in non-strict mode, only. In other words, the transformation described by the section titled "Strict Mode" must not be applied to these tests.