These tests have been re-factored to expand coverage of the "default
parameter" language feature and to more closely adhere to this project's
preferred file organization.
Ensure that when HasBinding of an Object environment record returns an
abrupt completion, that same completion is returned to the runtime.
Update the meta-data of related tests for consistency with this new
test.
The latest revision of ECMA262 makes special provisions for classes
which extend the `null` value [1]. Update the relevant tests
accordingly.
[1] https://github.com/tc39/ecma262/issues/543
Utilize the test generation tool to increase coverage of destructuring
assignment semantics. Previously, only destructuring assignment in the
AssignmentExpression position was tested. With this change applied, the
same tests will assert expected behavior for destructuring assignment in
`for..of` statements, as well.
A limited number of tests are applied to the `for..in` statement as
well, but due to the iteration protocol observed by that statement, many
destructuring tests are not relevant, and others cannot be automatically
generated from this format.
This change set does not include a test for restrictions relating to
template literals because such a test already exists in the project.
While a form of this test for string literals in strict mode code
existed previously, it is less precise and relies on unrelated
semantics. Remove the previous form and replace with a more direct
version.
Add tests that assert the management of the running execution context's
LexicalEnvironment and VariableEnvironment components, as created by the
following abstract operations:
- NewDeclarativeEnvironment
- NewObjectEnvironment
- NewFunctionEnvironment
Many tests require the use of non-strict direct eval, meaning they may
not be run in strict mode. This does not effect coverage because the
semantics in these cases are not observable from strict mode code.
Some situations require the creation of a binding, but this binding has
no relevance to the test itself. In these cases, use names consisting
solely of the underscore character (`_`).
Avoid the use of Block statements wherever possible, as these trigger
the creation of additional environments which may interfere with the
behavior under test.
Closes#572
Introduce tests for new semantics for ES2015 features such as
lexically-scoped bindings. Also add tests for semantics defined in prior
editions of the specification but not yet covered in this test suite.
Limit tests in `language/expressions/call/` and `built-ins/eval/` to
only assert details that are directly related to the invocation pattern.
Re-organize all other tests within the `language/eval-code/` directory,
further categorizing each as `direct` or `indirect` as appropriate.
Remove the extraneous "executable" flag from those files which carried
it.
This harness function is not necessary in the majority of cases in which
it is used. Remove its usage to simplify tests and decrease the amount
of domain-specific knowledge necessary to contribute to the test suite.
Persist the harness function itself for use by future tests for ES2015
modules (such a helper is necessary for tests that are interpreted as
module code).
Except for a small set of expected input/output pairs, both ES5 and
ES2015 define the expected return value of these methods in terms of an
"implementation-dependent approximation." This makes it inappropriate to
enforce expectations for specific values, even if expressed imprecisely.
The files in this patch are organized according to the following naming
scheme:
Prefix | Grammar production
-------------------|-------------------
`array-empty-` | ArrayAssignmentPattern : [ ]
`array-elision-` | ArrayAssignmentPattern : [ Elision ]
`array-rest-` | ArrayAssignmentPattern : [ Elisionopt AssignmentRestElement ]
`array-elem-` | ArrayAssignmentPattern : [ AssignmentElementList ]
`array-elem-trlg-` | ArrayAssignmentPattern : [ AssignmentElementList , Elisionopt AssignmentRestElementopt ]
Suffix | Intent
-------------------|-------
`-abpt-close-err` | The assignment evaluation returns an abrupt completion, and the iterator's `return` method throws an error
`-abpt-close-skip` | The assignment evaluation returns an abrupt completion, but IteratorClose is not invoked
`-abpt-close` | The assignment evaluation returns an abrupt completion, and IteratorClose is invoked as specified
`-get-err` | Abrupt completion returned from GetIterator
`-nrml-close-err` | The assignment evaluation completes, and the iterator's `return` method throws an error
`-nrml-close-null` | The assignment evaluation completes, and the iterator's `return` method returns a non-Object value (there is no corresponding `-abpt-` suffix because the algorithm does not reference the return value in those cases)
`-nrml-close-skip` | The assignment evaluation completes, but IteratorClose is not invoked
`-nrml-close` | The assignment evaluation completes, and IteratorClose is invoked as specified
Not all suffixes are appropriate for all productions. Suffixes have been
simplified in cases where less specificity is necessary to disambiguate
test cases.
Some tests for `eval` assert the equality of key enumeration on the
global object, comparing the ordering when the object is referenced
through eval.
Based on the test bodies and the "info" field in their metadata, these
tests appear to have been written under the mistaken impression that
erroneous creation of an environment record would be observable through
the `this` value.
In reality, the value in such cases resolves to the global object. That
renders these tests redundant and overly complex--none of the
distinctions between each test actually demonstrates a different
behavior.
Remove the redundant tests and introduce three new tests asserting the
correct resolution of the `this` keyword for direct eval code from
within the global scope and from within function scope.
Files whose name ends in `_.js` are not themselves valid Test262 tests
and should not be interpreted as such by test runners.
---
The tests for the GetBindingValue method of the module Environment
Record are very minimal. This is because GetBindingValue is necessary to
assert any aspect of binding creation/initialization/mutation. In this
way, GetBindingValue is being implicitly tested by every test that
references a binding value.
Files whose name ends in `_.js` are not themselves valid Test262 tests
and should not be interpreted as such by test runners.
---
Because the tests in this patch concern declaration *instantiation*,
care has been taken to avoid asserting binding values following
evaluation. Because a given module's dependencies are evaluated prior to
the module itself, this is only observable in modules which import their
own bindings.
A separate patch dedicated to the evaluation of module code asserts the
behavior of bindings following evaluation.
---
For tests that concern the creation of a module namespace object, this
patch relies on the semantics of the `in` operator. The `in` operator
uses the [[HasProperty]] internal method and avoids testing unrelated
semantics concerning binding resolution. Those semantics should be
explicitly asserted with a separate set of tests dedicated to that
purpose.
---
One test case which is notably missing is error resulting from a cycle
due to an `import` declaration (under the current file naming scheme,
such a test might be named `instn-named-err-circular.js`). Due to the
recursive nature of ModuleDeclarationInstantiation, it is not
technically possible for a circular request to be found in step 12.c.i.
Cycles rely on at least 2 `export` declarations, and because these are
resolved *before* imports, any cycle would trigger failure prior to step
12.c.
---
One aspect of *module* resolution that makes ordering observable is the
fact that resolution can fail in two distinct ways (i.e. with a
SyntaxError or with a ReferenceError). This patch includes tests that
leverage this detail in order to assert that modules are resolved in the
correct sequence.
However, from the perspective of the ECMA-262 specification, the
ordering of *export* (e.g. binding) resolution is not observable. This
is due to restrictions on the grammar, where each export is independent.
When *export* resolution fails, it does so with instances of SyntaxError
alone, precluding the above strategy.
So while ModuleDeclarationInstantiation resolves the exports of the
module's dependencies in the following order:
1. "Indirect" exports, e.g.
- `export { x } from './y.js';`
- `export { x as z } from './y.js';`
- `import { x } from './y.js'; export { x };`
2. "Star" imports
- `import * as ns from './y.js';`
3. "Named" (my word) imports
- `import x from './y.js';`
- `import { x } from './y.js';`
- `import { x as z } from './y.js';`
Intentional failures cannot be used to discern resolution ordering.
The runtime semantics of this statement are host-defined and therefore
untestable, but the statement's affect on the formal grammar should be
consistent across all implementations.
V8 ran into an issue where the YAML parser our test setup is using
didn't understand the newline, and failed to parser the negative
test expectation below, causing the test to fail. This patch fixes
the issue.
Assert that ImportDeclaration and ExportDeclaration match only the
ModuleItem symbol.
According to the definition of HostResolveImportedModule, it is
acceptable for an implementation to throw a SyntaxError in the event
that a requested module can neither be found nor created:
> If a Module Record corresponding to the pair referencingModule,
> specifier does not exist or cannot be created, an exception must be
> thrown.
In order to reliably detect a SyntaxError in response to the correct
interpretation of the grammar (and not a SyntaxError from an *incorrect*
interpretation of the grammar followed by a failure to resolve the
requested module), the ModuleSpecifier of ExportDeclarations should
describe a valid resource.
- Prefix file names to explicitly describe the "head" position
- Remove statement name suffix as this information is reflected by each
file's location within the file hierarchy
The harness file `Test262Error.js` has not contained executable code since it
was introduced in this project [1]. The definition of the `Test262Error`
function has consistently been located in the `sta.js` harness file which test
runners are expected to inject into the test environment.
Remove the file and all references to it.
[1] See commit c33bf0e043
Test262 defines tests for expression-producing syntactic forms within
the `language/expressions/` directory. Most tests for object literals
conform to this structure, but 12 such tests were added to the
`language/object-literal/` directory. Move these tests to the canonical
location for object literals.
ECMAScript 2015 introduced tail call optimization for function calls
occuring in a number of positions in the grammar. Assert expected
behavior by triggering a large (but configurable) number of recursive
function calls in these positions. Compliant runtimes will execute such
programs without error; non-compliant runtimes are expected to fail
these tests by throwing an error or crashing when system resources are
exhausted.
The ES2016 draft further refines the completion values for `if` and
`with` statements. Two tests must be removed outright because the
completion value in those cases is no longer accessible from the
runtime.
In order to facilitate proper tail calls, ES2015 modified the completion
value of a number of statements.
These tests use `eval` to verify the new values.
Although the `for..in` statement allows Expressions to define the
iterator, only an AssignmentExpression may occupy this position in the
`for..of` statement.
This change set includes tests for most invocations of the
SetFunctionName abstract operation in the ES2015 specification.
Practical testing considerations preclude the introduction of tests for
certain invocations:
- The project is still vetting methods to sustainably test the semantics
of the Destructuring Binding pattern across all valid productions.
- 13.3.3.6 Runtime Semantics: IteratorBindingInitialization
- 13.3.3.7 Runtime Semantics: KeyedBindingInitialization
- Without a loader, there is no way to access a function object declared
in an ExportDeclaration, so `name` assignment cannot be tested in
these cases
- 14.1.19 Runtime Semantics: InstantiateFunctionObject
- 14.4.12 Runtime Semantics: InstantiateFunctionObject
- 14.5.15 Runtime Semantics: BindingClassDeclarationEvaluation
- 15.2.3.11 Runtime Semantics: Evaluation
- Add missing 'negative: ReferenceError' in S7.9_A5.7_T1
- Remove stale 'negative: ReferenceError' in block-local-closure-get-before-initialization
- Remove all tail-call expressions in test/built-ins/Function
- And update code in test/built-ins/Function to simply use 'f()' instead of 'var r = f(); return r'
Fixes#383
The `negative` frontmatter tag expresses an expectation for the behavior
of the test file as a whole. The `assert.throws` helper function offers
more fine-grained control over expectations because it may be applied to
specific statements and expressions. This makes it preferable in cases
where it may be used (i.e. when the test body does not describe a syntax
error or early error).
Re-implement assertions for errors to use the `assert.throws` helper
function wherever possible.
The `test/language/class/` directory contains a small subset of
Test262's tests for ES2015 classes. The majority of tests for classes
are organized within `test/language/statements/class/`.
- Move the tests that rely on the ClassDeclaration syntactic form from
`test/language/class/` to `test/language/statements/class/`.
- Move the test that relies on the ClassExpression syntactic form from
`test/language/class/` to `test/language/expressions/class/`.
- Re-organize existing tests for identifiers
Name files according to their content (not their ES5 ID). Move tests
for IdentifierName and LabelIdentifier to more appropriate directories.
- Simplify and correct tests
Instead of asserting successful runtime evaluation using `eval`, rely
on the test runner's ability to detect syntax errors. Update the test
bodies to test the grammar referenced by their ES5 ID and
description--the IdentifierStart pattern.
- Use `negative` frontmatter to assert SyntaxErrors
- Remove redundant tests
- Use `assert.equal` helper function
- Add equivalent tests for literal unicode chars
- Add tests for variable-length unicode escape
The `test/language/generators/` directory contained a single file that
concerned generators derived both from syntactic form and from a
built-in function.
Refactor this test into two files and place each in the appropriate
directory.
Some tests involving the directive prologue are invalidated by source
text transformations that insert executable code in the beginning of the
script. Implement a `raw` flag that allows these tests to opt-out of
this transformation. Update the relevant tests to use this flag (and
remove references to globals only available when code is injected).
Update the Python runner accordingly:
- Do not run tests marked as "raw" in strict mode
- Reject invalid test configurations
Update the browser runner accordingly:
- Do not modify the script body of tests marked as "raw"
In ECMAScript 5, assignment to a non-reference value throws a runtime
ReferenceError. ECMAscript 6 specifies an early ReferenceError in these
cases. Tests for this behavior have been authored to pass in both cases.
Simplify these tests to describe and assert the early error.
The `fnExists` function defines a generic way to determine if any number
of values are function instances. Because it is only used by a single
test, the additional complexity required by the generalized code (and
the organizational drawbacks to maintaining another "include" file) are
not justified. Remove the file and update the test to assert the
function's existence directly.
Some tests specifically concern the application of the `use strict`
directive as it appears in JavaScript source code. These tests should
*not* be run with the `onlyStrict` flag because relying on the test
runner to enable strict mode makes the semantics of the source code
irrelevant. Update these tests to use the `noStrict` flag.
Other tests concern language semantics that are only valid in strict
mode, but the mechanism for enabling strictness is inconseqential.
Update these tests to use the `onlyStrict` flag and remove any redundant
`use strict` directive prologues contained within.
Still other tests are valid both within and outside of strict mode.
In keeping with the majority of other tests, do not specify any
restrictions on the environments in which these tests may be run.
The `$PRINT` helper function has no effect on test behavior. The
following tests use it to report assertion outcomes (and as a result
will fail silently):
- test/language/function-code/S10.2.1_A4_T2.js
- test/language/function-code/S10.2.1_A5.1_T1.js
- test/language/function-code/S10.2.1_A5.1_T2.js
- test/language/function-code/S10.2.1_A5.2_T1.js
Remove the function definition and all references within tests. Update
tests that use it as an error reporting mechanism to instead use an
appropriate `assert` helper function.
In neglecting to assert the type of error thrown (or that any error was
thrown at all), these tests cannot fail. Refactor the tests to use the
`assert.throws` helper method, which takes these details into
consideration.
This function is equivalent to `$ERROR` (which is automatically included
in test environments). Remove the harness file that defines the
function, remove references to the file from test `includes` lists, and
update scripts to instead invoke the `$ERROR` function.
The previous description of the 11.4.1-4.a-5 test case implied that an environment object could not be deleted while inside a with. However, the actual test was to test whether the variable declaration could be deleted. The description has been updated to better reflect this.
Test 11.4.1-4.a-5 states that it verifies that an environment object cannot be deleted. However, this was giving a false positive. It was actually testing where a "var" declaration on an environment object cannot be deleted (there are other tests for this). This test case fails on Chrome 43, Firefox 38 and Internet Explorer 11.
Test case 10.4.2-3-c-2-s verifies that eval can't instantiate variables in the calling context. However, it does not verify that the variable isn't matched to the global context either. Added test case 10.4.2-3-c-3-s to check for this.
- Add missing ".prototype" lookup in Array.prototype tests (Follow-up of #280)
- Add noStrict flag in global/S10.2.3_A2.2_*.js (implicit this-binding is part of the test)
- Provide correct this-binding in function/13.2-*.js
- Add missing assert.throws in "set-iterator-close-after-add-failure.js"
From the ES6 specification:
> ## PropertyDefinition : IdentifierReference
>
> 1. Let propName be StringValue of IdentifierReference.
> 2. Let exprValue be the result of evaluating IdentifierReference.
> 3. ReturnIfAbrupt(exprValue).
> 4. Let propValue be GetValue(exprValue).
> 5. ReturnIfAbrupt(propValue).
> 6. Assert: enumerable is true.
> 7. Return CreateDataPropertyOrThrow(object, propName, propValue).
The increment/decrement operator evaluates its operand expression once. When
the operand expression is a property accessor, RequireObjectCoercible
and ToPropertyKey are called on the property accessor in the correct order.
The assignment operator evaluates its operands from left to right. When
the left-hand side expression is a property accessor, RequireObjectCoercible
and ToPropertyKey are called on the property accessor before the right-hand
side expression is evaluated.
- Add missing "var" declarations and noStrict flags
- Add strict mode tests for:
- arguments-object/10.6-6-3 -> arguments-object/10.6-6-3-s
- arguments-object/10.6-6-4 -> arguments-object/10.6-6-4-s
- Remove try-finally clean-up code
- Add strict mode compatible tests for eval-code/S10.4.2_A1.1_T*
- No strict mode compatible files added for eval-code/S10.4.2_A1.2_T*, because it doesn't really make sense in this context.
- (S10.4.2_A1.1 and S10.4.2_A1.2 should probably be removed, because the tested behaviour is not required by the spec.)
- Split S8.5_A10, S8.5_A4 and S8.1_A3 into declaration (both modes) and assignment tests (non-strict only)
Part of issue #35.
These tests are derived from the following files within the Google V8
project:
test/mjsunit/es6/generators-iteration.js
test/mjsunit/es6/generators-objects.js
test/mjsunit/es6/generators-runtime.js
test/mjsunit/es6/generators-states.js
- Duplicate properties are allowed in object literals: test/language/expressions/object/*.js
- RegExpBuiltinExec was changed to use ToLength(lastIndex): test/built-ins/RegExp/prototype/exec/S15.10.6.2_A5_T3.js
- Non-undefined flags arguments allowed in RegExp constructor call: test/built-ins/RegExp/*.js
- Array.prototype.push throws TypeError if new length exceeds Number.MAX_SAFE_INTEGER: test/built-ins/Array/prototype/push/S15.4.4.7_A2_T2.js
- .length property of bound functions is configurable: test/built-ins/Function/prototype/bind/15.3.4.5-15-2.js
- Array.prototype changed back to exotic Array object: test/built-ins/Array/prototype/*.js, test/built-ins/Array/isArray/15.4.3.2-0-5.js
- in-statement-position-label-statement.js is an Annex-B test (B.3.2 Labelled Function Declarations), moved to annexB directory
- identifier-let-allowed-as-lefthandside-expression-strict.js needs to check for a SyntaxError (ES6, 12.1.1)
- Remove sort() calls in test/built-ins/Object/getOwnPropertyNames/*.js
- Remove sort() calls in test/language/block-scope/syntax/for-in/acquire-properties-from-*.js
- verifyConfigurable() needs to called last in test/built-ins/Object/is/length.js
- All productions within ClassBody are implicitly strict, update test/language/class/method-definition/yield-as-*.js accordingly
- Remove unnecessary noStrict flag in test/language/class/method-definition/yield-as-generator-method-binding-identifier.js
- Check own symbols are returned in property creation order from Object.getOwnPropertySymbols():
- test/language/computed-property-names/basics/symbol.js
- test/language/computed-property-names/class/method/symbol.js
- test/language/computed-property-names/class/static/method-symbol.js
- test/language/computed-property-names/object/method/symbol.js
- Fix copy-paste error in test/language/expressions/object/method-definition/yield-as-function-expression-binding-identifier.js
- Expand ambiguous assertion messages and assert execution paths more
finely.
- Improve variable names in `for..of` tests
While the object created by a GeneratorFunction may be considered an
"iterable", it is being used as an iterator in these tests. Naming the
variable according to the way it is used improves the readability of
the test body.
- Add 'features' attribute to test frontmatter
- Move tests
- Introduce additional `for..of` control flow tests
These tests are derived from the following files within the Google V8
project:
test/mjsunit/es6/typed-array-iterator.js
test/mjsunit/es6/arguments-iterator.js
test/mjsunit/es6/string-iterator.js
test/mjsunit/es6/collection-iterator.js
These tests are derived from the following files within the Google V8 project:
test/mjsunit/es6/regress/regress-2506.js
test/mjsunit/es6/regress/regress-3426.js
test/mjsunit/es6/regress/regress-3683.js
Add missing variable declarations (issue #35)
Split S12.6.3_A10 and S12.6.3_A10.1 because both files seem to test implicit global variables (issue #35)
Changes (issue #35)
- Add missing noStrict flags.
- Change 13.2-15-1 and 13.2-18-1 to use assert.js and propertyHelper.js (simplifies writable and configurable checks while in strict mode).
- Add variable declarations for globals.
- Create copies of S13.2.1_A6_T1 and S13.2.1_A6_T2 instead of adding variable declarations, because both files seem to test implicit global variables.
- Split S13_A14 and S13_A16 to work in strict mode.
- Remove assignment to .name property.
Add missing noStrict flags (issue #35)
Add missing noStrict flags and variable declarations (issue #35)
Add missing noStrict flags (issue #35)
According to the new test organization scheme, specification
identification references should be encoded within the test file itself,
and the file should be named according to the semantics of the test.
These tests are derived from the following files within the Google V8
project:
test/mjsunit/es6/templates.js
Some of these tests include non-printable characters, causing git to
infer that they are binary files. Introduce a `.gitattributes` file to
configure git to consistently display the source text of all JavaScript
files.
These tests are derived from the following files within the Google V8 project:
test/mjsunit/harmony/class-computed-property-names-super.js
test/mjsunit/harmony/class-property-name-eval-arguments.js
test/mjsunit/harmony/classes-experimental.js
test/mjsunit/harmony/classes-lazy-parsing.js
test/mjsunit/harmony/classes.js
These tests are derived from the following files within the Google V8 project:
test/mjsunit/harmony/method-name-eval-arguments.js
test/mjsunit/harmony/object-literals-method.js
test/mjsunit/harmony/object-literals-property-shorthand.js
test/mjsunit/harmony/object-literals-super.js
These tests are derived from the following files within the Google V8
project:
test/mjsunit/harmony/computed-property-names-classes.js
test/mjsunit/harmony/computed-property-names-object-literals-methods.js
test/mjsunit/harmony/computed-property-names-super.js
test/mjsunit/harmony/computed-property-names.js
These tests are derived from the following files within the Google V8 project:
test/mjsunit/harmony/block-conflicts.js
test/mjsunit/harmony/block-for.js
test/mjsunit/harmony/block-leave.js
test/mjsunit/harmony/block-let-declaration.js
test/mjsunit/harmony/block-let-semantics.js
test/mjsunit/harmony/block-scoping.js
- Add missing noStrict flags when tests explicitly require non-strict semantics (e.g. unqualified delete, with statement)
- Change `throw new Exception(...)` to `$ERROR(...)` in call/*
- Replace dynamic strict mode check with noStrict flag in call/11.2.3-3_8
- Align assignment/11.13.1_A3.1 with 11.13.1_A3.2 to avoid creating implicit global variable
- Split test into multiple files when undeclared variable are under test
- addition/S11.6.1_A2.4_T3
- division/11.5.2_A2.4_T3
- does-not-equals/11.9.2_A2.4_T3
- equals/S11.9.1_A2.4_T3
- greater-than/S11.8.2_A2.4_T3
- greater-than-or-equal/S11.8.4_A2.4_T3
- in/S11.8.7_A2.4_T3
- instanceof/11.8.6_A2.4_T3
- left-shift/S11.7.1_A2.4_T3
- less-than/S11.8.1_A2.4_T3
- less-than-or-equal/S11.8.3_A2.4_T3
- modulus/S11.5.3_A2.4_T3
- multiplication/11.5.1_A2.4_T3
- right-shift/11.7.2_A2.4_T3
- strict-does-not-equals/11.9.5_A2.4_T3
- strict-equals/11.9.4_A2.4_T3
- subtraction/11.6.2_A2.4_T3
- unsigned-right-shift/11.7.3_A2.4_T3
- Add declaration when implicit global variable creation not part of the test
- assignment/11.13.1_A4_T1
- compound-assignment/S11.13.2_A3.1_T*
- compound-assignment/S11.13.2_A3.2_T*
- grouping/11.1.6_A3_T5
- instanceof/S15.3.5.3_*
- Split test into multiple files when unqualified delete is used
- delete/S11.4.1_A2.2_T1.js
- delete/11.4.1_A3.2
- grouping/S11.1.6_A2
- grouping/S11.1.6_A3_T6
This change adds 'var' declarations for global variables to allow the tests to run in strict mode (see issue #35).
Extra care was taken to ensure the changes do not alter the test behavior, for example when implicit creation of global variables are part of the test.
Note: The change does not fix all strict mode errors due to missing 'var' declarations.
Mike Pennisi
Leo Balter
Ian Halliday
Gorkem Yakin
Dan Ehrenberg
André Bargull
Rick Waldron
Jeff Walden
Brian Terlson
Catherine Farman
unknown
Kevin Gibbons
Thomas Wood
Pieter van Ginkel
Caitlin Potter
smikes
Brian Terlson