The templates are being used for many tests reusing the same available function forms.
The format they are provided allow us to extend tests with cases for other tests relying
in the same formats.
After @rwaldron's feedback:
The purpose of the `!` operator is to evaluate an UnaryExpression,
coerce the result to a boolean value and then return the negated
value of that operation. But that's not what you're trying to do at
all—you just want to evaluate the expression to the right of the
operator, nothing more, nothing less. In this specific case, you
don't even really care about the evaluation, the goal is write
valid (or invalid, as the case may be) syntax that is will be
parsed according to a specific grammar rule that requires some
operator to signal that the thing is an expression and not a Block
Statement.
Tests doesn't use async functionality and don't call $DONE, so remove
"async" flag:
- src/params/error/async-gen-named-func-expr.template
- test/language/expressions/async-generator/params-named-dflt-abrupt.js
- test/language/expressions/async-generator/params-named-dflt-ref-later.js
- test/language/expressions/async-generator/params-named-dflt-ref-self.js
Intl.PluralRules.prototype is no longer a Intl.Prototype instance:
- test/intl402/PluralRules/prototype/prototype.js
Intl.PluralRules throws an error when called as a function:
- test/intl402/PluralRules/undefined-newtarget-throws.js
Module namespace objects call OrdinaryDelete for symbol properties:
- test/language/module-code/namespace/internals/delete-non-exported.js
Async generators no longer retrieves "done" property twice:
- src/async-generators/yield-star-async-next.case
- src/async-generators/yield-star-async-return.case
- src/async-generators/yield-star-async-throw.case
Minor units of CLF is 4, so we need to test with maximumFractionDigits=3
to get an error:
- test/intl402/NumberFormat/dft-currency-mnfd-range-check-mxfd.js
DateTimeFormat.prototype.formatToParts length property was changed from
0 to 1:
- test/intl402/DateTimeFormat/prototype/formatToParts/length.js
minimumSignificantDigits and maximumSignificantDigits properties are
only retrieved once:
- test/intl402/NumberFormat/11.1.1_32.js
test/annexB/built-ins/Date/prototype/setYear/time-clip.js
test/built-ins/Date/prototype/setFullYear/new-value-time-clip.js
test/built-ins/Date/prototype/setMonth/new-value-time-clip.js
- Don't try to test time-clip at the end points, because this is near
impossible to get right (needs to consider time zone offset, dst, local
mean time because of Africa/Monrovia, etc.).
test/built-ins/DataView/prototype/setFloat64/detached-buffer-after-toindex-byteoffset.js
test/built-ins/DataView/prototype/setInt16/detached-buffer-after-toindex-byteoffset.js
- Wasn't update to expect RangeError
test/built-ins/Function/internals/Construct/derived-this-uninitialized-realm.js
- Change ClassDeclaration -> ClassExpression to get completion value
test/built-ins/Function/prototype/toString/AsyncFunction.js
- Add missing \n in expected string
- Also fixed in gh-847
test/built-ins/global/global-object.js
- Add 'var' to make test pass in strict-mode
test/language/block-scope/syntax/redeclaration-in-block/attempt-to-redeclare-function-declaration-with-function-declaration.js
- This is allowed in sloppy mode when Annex B is implemented
test/language/expressions/async-generators/expression-yield-as-statement.js
- Fix calls to then()
test/language/module-code/namespace/internals/own-property-keys-binding-types.js
test/language/module-code/namespace/internals/own-property-keys-sort.js
- Tests weren't updated after removal of @@iterator from module
namespace objects
test/language/module-code/namespace/internals/set-prototype-of-null.js
- Fix syntax error
test/language/statements/async-function/early-errors-no-async-generator.js
- No longer valid now that async iteration proposal is at stage 3
* Re-organize coverage for YieldExpression
* Extend test coverge for YieldExpression
* fixup! Extend test coverge for YieldExpression
Remove unused variables
A recent change to the specification [1] introduces parse-time errors
for certain usages of `super` within eval code. Modify all tests that
are affected by this change:
- Update the test bodies to accurately enforce the new semantics
- Rename files to better reflect the section of the specification that
they enforce
- Update test meta-data
- Change the `esid` meta-data to reflect the location of the relevant
specification text
- Remove the `es6id` meta-data as the behavior is no longer relatable
to that specification
- Introduce the `features` meta-data in cases where the test file's
new location no longer reflects all required language features
[1] "Normative: Clarify rules around super inside eval"
https://github.com/tc39/ecma262/pull/685
* Add tests for prototype realm inference
* Add tests for miscellaneous realm concerns
* Add tests for realm of spec-created Errors
In some cases, Error objects produced by the specification are
observable from ECMAScript code. Among these cases, some are further
differentiated in that they occur outside of any built-in function and
may be triggered through syntactic production directly. The current
realm record is commonly interpreted incorrectly under these
circumstances.
Add tests asserting that the expected realm record is used when
constructing such Error objects.
* Add tests for realm use in ArraySpeciesCreate
* Add tests for function realm retrieval
* Add tests for cross-realm behaviors of Symbols
* Add tests for GetValue and PutValue
* Add tests for realm of spec-created Arrays
In some cases, Arrays produced by CreateArrayFromList are observable
from ECMAScript code. Among these cases, two occur outside of any
built-in function and may be triggered through syntactic production
directly. The current realm record is commonly interpreted incorrectly
under these circumstances.
Add tests asserting that the expected realm record is used when
constructing arrays.
* Add test for spec-created object
* fixup! Add tests for realm of spec-created Errors
* fixup! Add tests for realm of spec-created Errors
* fixup! Add tests for prototype realm inference
* fixup! Add tests for miscellaneous realm concerns
Because expectations regarding error "phase" are now expressed via test
meta-data, the test runner may now enforce this requirement on negative
tests.
Remove the "NotEarlyError" from the project source. This reduces the
amount of domain knowledge required to author tests and lessens the
potential for inconsistencies between tests.
Authored via the following command:
$ find test -type f -print0 | \
xargs -0 sed \
-i 's/^\(\s*\)negative:\s*SyntaxError\s*$/\1negative:\n\1 phase: early\n\1 type: SyntaxError/g'
This test is being added because the committee is considering changing
this evaluation order (as discussed at the May 2016 Munich meeting). The
consequences of this spec change will be more clear as a test change
than a simple test addition.
* Assert creation of 'arguments' object
Ensure that the 'arguments' object is created in cases where it is not
required by the body but is required by the parameters.
* Add tests for cases that disable "arguments" map
* Add tests for NewTarget override of bound function
* Add test for properties of exotic String objects
* Add tests for early errors in functions
* Improve tests for class accessors
Use the `propertyHelper.js` utility in order to functionally test the
property descriptors of class methods.
* Remove redundant tests
The semantics of an IdentifierReference as a PropertyDefinition within
an object initializer are exhaustively tested by the files in this
directory whose name match the pattern `prop-def-id-*.js`.
Delete the redundant tests in favor of the more descriptively-named and
more exhaustive alternatives.
* Rename tests
* Update test names to be more descriptive
* Add tests for property descriptors of accessors
* Add tests for runtime error during method dfn
* Add test for observable iteration
* Extend tests for class "name" inference
Ensure that when a class defines a static "name" method, that method
definition prevents the "name" inference behavior.
* Re-generate tests
This is a pure miscellaneous change to use global values of NaN and Infinity instead of their namespaced equivalents.
Ref 9ae0567a50 (r69049894)
cc @suwc
This re-factors some existing Sputnik tests to be more targeted and to use a
pattern that can be generalized to other forms. We could test these all day,
but I've limited myself to forms introduced in ES2015, specifically
YieldExpression and new.target. Note that SpiderMonkey incorrectly throws a
SyntaxError for these.
I thoughtlessly wrote ReferenceError tests for yield = 1 until I realized
that such productions are not actually recognized by the grammar, so the early
errors do not apply. Instead, I've added a negative syntax test for that case.
* Refactor test for valid cover
* Add tests for ValidSimpleAssignmentTarget
Ensure that constructs introduced in ES2015 are disallowed as assignment
targets, with or without a "cover" grammar.
* Add test for grammar precedence of YieldExpression
* Add missing test for early error
* Add missing test for WithBaseObject
* Improve coverage for `new.target`
* Add test for deletion of SuperReference
* Add tests for `in` keyword restrictions
* fixup! Improve coverage for `new.target`
The global isNaN is not precise at all, and Number.isNaN is an ES6 feature that makes it preferrable to use assert's sameValue for NaN values, as it handles it internally using the comparison.
* Improve test coverage for `super` keyword
Add tests for SuperCall and SuperProperty, organized together in the
`test/language/expressions/super/` directory. For SuperProperty, include
tests for usage from within Object initializers and class bodies because
a different set of semantics are observable from each context.
Utilize the test generation tool to increase coverage of destructuring
assignment semantics. Previously, only destructuring assignment in the
AssignmentExpression position was tested. With this change applied, the
same tests will assert expected behavior for destructuring assignment in
`for..of` statements, as well.
A limited number of tests are applied to the `for..in` statement as
well, but due to the iteration protocol observed by that statement, many
destructuring tests are not relevant, and others cannot be automatically
generated from this format.
Add tests that assert the management of the running execution context's
LexicalEnvironment and VariableEnvironment components, as created by the
following abstract operations:
- NewDeclarativeEnvironment
- NewObjectEnvironment
- NewFunctionEnvironment
Many tests require the use of non-strict direct eval, meaning they may
not be run in strict mode. This does not effect coverage because the
semantics in these cases are not observable from strict mode code.
Some situations require the creation of a binding, but this binding has
no relevance to the test itself. In these cases, use names consisting
solely of the underscore character (`_`).
Avoid the use of Block statements wherever possible, as these trigger
the creation of additional environments which may interfere with the
behavior under test.
Closes#572
Introduce tests for new semantics for ES2015 features such as
lexically-scoped bindings. Also add tests for semantics defined in prior
editions of the specification but not yet covered in this test suite.
Limit tests in `language/expressions/call/` and `built-ins/eval/` to
only assert details that are directly related to the invocation pattern.
Re-organize all other tests within the `language/eval-code/` directory,
further categorizing each as `direct` or `indirect` as appropriate.
Remove the extraneous "executable" flag from those files which carried
it.
This harness function is not necessary in the majority of cases in which
it is used. Remove its usage to simplify tests and decrease the amount
of domain-specific knowledge necessary to contribute to the test suite.
Persist the harness function itself for use by future tests for ES2015
modules (such a helper is necessary for tests that are interpreted as
module code).
Except for a small set of expected input/output pairs, both ES5 and
ES2015 define the expected return value of these methods in terms of an
"implementation-dependent approximation." This makes it inappropriate to
enforce expectations for specific values, even if expressed imprecisely.
The files in this patch are organized according to the following naming
scheme:
Prefix | Grammar production
-------------------|-------------------
`array-empty-` | ArrayAssignmentPattern : [ ]
`array-elision-` | ArrayAssignmentPattern : [ Elision ]
`array-rest-` | ArrayAssignmentPattern : [ Elisionopt AssignmentRestElement ]
`array-elem-` | ArrayAssignmentPattern : [ AssignmentElementList ]
`array-elem-trlg-` | ArrayAssignmentPattern : [ AssignmentElementList , Elisionopt AssignmentRestElementopt ]
Suffix | Intent
-------------------|-------
`-abpt-close-err` | The assignment evaluation returns an abrupt completion, and the iterator's `return` method throws an error
`-abpt-close-skip` | The assignment evaluation returns an abrupt completion, but IteratorClose is not invoked
`-abpt-close` | The assignment evaluation returns an abrupt completion, and IteratorClose is invoked as specified
`-get-err` | Abrupt completion returned from GetIterator
`-nrml-close-err` | The assignment evaluation completes, and the iterator's `return` method throws an error
`-nrml-close-null` | The assignment evaluation completes, and the iterator's `return` method returns a non-Object value (there is no corresponding `-abpt-` suffix because the algorithm does not reference the return value in those cases)
`-nrml-close-skip` | The assignment evaluation completes, but IteratorClose is not invoked
`-nrml-close` | The assignment evaluation completes, and IteratorClose is invoked as specified
Not all suffixes are appropriate for all productions. Suffixes have been
simplified in cases where less specificity is necessary to disambiguate
test cases.
Test262 defines tests for expression-producing syntactic forms within
the `language/expressions/` directory. Most tests for object literals
conform to this structure, but 12 such tests were added to the
`language/object-literal/` directory. Move these tests to the canonical
location for object literals.
ECMAScript 2015 introduced tail call optimization for function calls
occuring in a number of positions in the grammar. Assert expected
behavior by triggering a large (but configurable) number of recursive
function calls in these positions. Compliant runtimes will execute such
programs without error; non-compliant runtimes are expected to fail
these tests by throwing an error or crashing when system resources are
exhausted.
This change set includes tests for most invocations of the
SetFunctionName abstract operation in the ES2015 specification.
Practical testing considerations preclude the introduction of tests for
certain invocations:
- The project is still vetting methods to sustainably test the semantics
of the Destructuring Binding pattern across all valid productions.
- 13.3.3.6 Runtime Semantics: IteratorBindingInitialization
- 13.3.3.7 Runtime Semantics: KeyedBindingInitialization
- Without a loader, there is no way to access a function object declared
in an ExportDeclaration, so `name` assignment cannot be tested in
these cases
- 14.1.19 Runtime Semantics: InstantiateFunctionObject
- 14.4.12 Runtime Semantics: InstantiateFunctionObject
- 14.5.15 Runtime Semantics: BindingClassDeclarationEvaluation
- 15.2.3.11 Runtime Semantics: Evaluation
The `negative` frontmatter tag expresses an expectation for the behavior
of the test file as a whole. The `assert.throws` helper function offers
more fine-grained control over expectations because it may be applied to
specific statements and expressions. This makes it preferable in cases
where it may be used (i.e. when the test body does not describe a syntax
error or early error).
Re-implement assertions for errors to use the `assert.throws` helper
function wherever possible.
The `test/language/class/` directory contains a small subset of
Test262's tests for ES2015 classes. The majority of tests for classes
are organized within `test/language/statements/class/`.
- Move the tests that rely on the ClassDeclaration syntactic form from
`test/language/class/` to `test/language/statements/class/`.
- Move the test that relies on the ClassExpression syntactic form from
`test/language/class/` to `test/language/expressions/class/`.
- Re-organize existing tests for identifiers
Name files according to their content (not their ES5 ID). Move tests
for IdentifierName and LabelIdentifier to more appropriate directories.
- Simplify and correct tests
Instead of asserting successful runtime evaluation using `eval`, rely
on the test runner's ability to detect syntax errors. Update the test
bodies to test the grammar referenced by their ES5 ID and
description--the IdentifierStart pattern.
- Use `negative` frontmatter to assert SyntaxErrors
- Remove redundant tests
- Use `assert.equal` helper function
- Add equivalent tests for literal unicode chars
- Add tests for variable-length unicode escape
In ECMAScript 5, assignment to a non-reference value throws a runtime
ReferenceError. ECMAscript 6 specifies an early ReferenceError in these
cases. Tests for this behavior have been authored to pass in both cases.
Simplify these tests to describe and assert the early error.
The `fnExists` function defines a generic way to determine if any number
of values are function instances. Because it is only used by a single
test, the additional complexity required by the generalized code (and
the organizational drawbacks to maintaining another "include" file) are
not justified. Remove the file and update the test to assert the
function's existence directly.
Some tests specifically concern the application of the `use strict`
directive as it appears in JavaScript source code. These tests should
*not* be run with the `onlyStrict` flag because relying on the test
runner to enable strict mode makes the semantics of the source code
irrelevant. Update these tests to use the `noStrict` flag.
Other tests concern language semantics that are only valid in strict
mode, but the mechanism for enabling strictness is inconseqential.
Update these tests to use the `onlyStrict` flag and remove any redundant
`use strict` directive prologues contained within.
Still other tests are valid both within and outside of strict mode.
In keeping with the majority of other tests, do not specify any
restrictions on the environments in which these tests may be run.
This function is equivalent to `$ERROR` (which is automatically included
in test environments). Remove the harness file that defines the
function, remove references to the file from test `includes` lists, and
update scripts to instead invoke the `$ERROR` function.
The previous description of the 11.4.1-4.a-5 test case implied that an environment object could not be deleted while inside a with. However, the actual test was to test whether the variable declaration could be deleted. The description has been updated to better reflect this.
Test 11.4.1-4.a-5 states that it verifies that an environment object cannot be deleted. However, this was giving a false positive. It was actually testing where a "var" declaration on an environment object cannot be deleted (there are other tests for this). This test case fails on Chrome 43, Firefox 38 and Internet Explorer 11.
Josh Wolfe
Valerie R Young
Leo Balter
Caitlin Potter
Rick Waldron
Daniel Ehrenberg
Josh Wolfe
Robin Templeton
Caio Lima
André Bargull
Marshall Roch
Mike Pennisi
Shu-yu Guo
⭐caitp⭐
Michael Ficarra
dop251
Tim Disney
Brian Terlson
Jeff Morrison
Mathias Bynens
Ian Halliday
Gorkem Yakin
Brian Terlson
Catherine Farman
unknown
Thomas Wood
Pieter van Ginkel