The templates are being used for many tests reusing the same available function forms.
The format they are provided allow us to extend tests with cases for other tests relying
in the same formats.
After @rwaldron's feedback:
The purpose of the `!` operator is to evaluate an UnaryExpression,
coerce the result to a boolean value and then return the negated
value of that operation. But that's not what you're trying to do at
all—you just want to evaluate the expression to the right of the
operator, nothing more, nothing less. In this specific case, you
don't even really care about the evaluation, the goal is write
valid (or invalid, as the case may be) syntax that is will be
parsed according to a specific grammar rule that requires some
operator to signal that the thing is an expression and not a Block
Statement.
Tests doesn't use async functionality and don't call $DONE, so remove
"async" flag:
- src/params/error/async-gen-named-func-expr.template
- test/language/expressions/async-generator/params-named-dflt-abrupt.js
- test/language/expressions/async-generator/params-named-dflt-ref-later.js
- test/language/expressions/async-generator/params-named-dflt-ref-self.js
Intl.PluralRules.prototype is no longer a Intl.Prototype instance:
- test/intl402/PluralRules/prototype/prototype.js
Intl.PluralRules throws an error when called as a function:
- test/intl402/PluralRules/undefined-newtarget-throws.js
Module namespace objects call OrdinaryDelete for symbol properties:
- test/language/module-code/namespace/internals/delete-non-exported.js
Async generators no longer retrieves "done" property twice:
- src/async-generators/yield-star-async-next.case
- src/async-generators/yield-star-async-return.case
- src/async-generators/yield-star-async-throw.case
Minor units of CLF is 4, so we need to test with maximumFractionDigits=3
to get an error:
- test/intl402/NumberFormat/dft-currency-mnfd-range-check-mxfd.js
DateTimeFormat.prototype.formatToParts length property was changed from
0 to 1:
- test/intl402/DateTimeFormat/prototype/formatToParts/length.js
minimumSignificantDigits and maximumSignificantDigits properties are
only retrieved once:
- test/intl402/NumberFormat/11.1.1_32.js
test/annexB/built-ins/Date/prototype/setYear/time-clip.js
test/built-ins/Date/prototype/setFullYear/new-value-time-clip.js
test/built-ins/Date/prototype/setMonth/new-value-time-clip.js
- Don't try to test time-clip at the end points, because this is near
impossible to get right (needs to consider time zone offset, dst, local
mean time because of Africa/Monrovia, etc.).
test/built-ins/DataView/prototype/setFloat64/detached-buffer-after-toindex-byteoffset.js
test/built-ins/DataView/prototype/setInt16/detached-buffer-after-toindex-byteoffset.js
- Wasn't update to expect RangeError
test/built-ins/Function/internals/Construct/derived-this-uninitialized-realm.js
- Change ClassDeclaration -> ClassExpression to get completion value
test/built-ins/Function/prototype/toString/AsyncFunction.js
- Add missing \n in expected string
- Also fixed in gh-847
test/built-ins/global/global-object.js
- Add 'var' to make test pass in strict-mode
test/language/block-scope/syntax/redeclaration-in-block/attempt-to-redeclare-function-declaration-with-function-declaration.js
- This is allowed in sloppy mode when Annex B is implemented
test/language/expressions/async-generators/expression-yield-as-statement.js
- Fix calls to then()
test/language/module-code/namespace/internals/own-property-keys-binding-types.js
test/language/module-code/namespace/internals/own-property-keys-sort.js
- Tests weren't updated after removal of @@iterator from module
namespace objects
test/language/module-code/namespace/internals/set-prototype-of-null.js
- Fix syntax error
test/language/statements/async-function/early-errors-no-async-generator.js
- No longer valid now that async iteration proposal is at stage 3
* Re-organize coverage for YieldExpression
* Extend test coverge for YieldExpression
* fixup! Extend test coverge for YieldExpression
Remove unused variables
A recent change to the specification [1] introduces parse-time errors
for certain usages of `super` within eval code. Modify all tests that
are affected by this change:
- Update the test bodies to accurately enforce the new semantics
- Rename files to better reflect the section of the specification that
they enforce
- Update test meta-data
- Change the `esid` meta-data to reflect the location of the relevant
specification text
- Remove the `es6id` meta-data as the behavior is no longer relatable
to that specification
- Introduce the `features` meta-data in cases where the test file's
new location no longer reflects all required language features
[1] "Normative: Clarify rules around super inside eval"
https://github.com/tc39/ecma262/pull/685
* Add tests for prototype realm inference
* Add tests for miscellaneous realm concerns
* Add tests for realm of spec-created Errors
In some cases, Error objects produced by the specification are
observable from ECMAScript code. Among these cases, some are further
differentiated in that they occur outside of any built-in function and
may be triggered through syntactic production directly. The current
realm record is commonly interpreted incorrectly under these
circumstances.
Add tests asserting that the expected realm record is used when
constructing such Error objects.
* Add tests for realm use in ArraySpeciesCreate
* Add tests for function realm retrieval
* Add tests for cross-realm behaviors of Symbols
* Add tests for GetValue and PutValue
* Add tests for realm of spec-created Arrays
In some cases, Arrays produced by CreateArrayFromList are observable
from ECMAScript code. Among these cases, two occur outside of any
built-in function and may be triggered through syntactic production
directly. The current realm record is commonly interpreted incorrectly
under these circumstances.
Add tests asserting that the expected realm record is used when
constructing arrays.
* Add test for spec-created object
* fixup! Add tests for realm of spec-created Errors
* fixup! Add tests for realm of spec-created Errors
* fixup! Add tests for prototype realm inference
* fixup! Add tests for miscellaneous realm concerns
Because expectations regarding error "phase" are now expressed via test
meta-data, the test runner may now enforce this requirement on negative
tests.
Remove the "NotEarlyError" from the project source. This reduces the
amount of domain knowledge required to author tests and lessens the
potential for inconsistencies between tests.
Authored via the following command:
$ find test -type f -print0 | \
xargs -0 sed \
-i 's/^\(\s*\)negative:\s*SyntaxError\s*$/\1negative:\n\1 phase: early\n\1 type: SyntaxError/g'
This test is being added because the committee is considering changing
this evaluation order (as discussed at the May 2016 Munich meeting). The
consequences of this spec change will be more clear as a test change
than a simple test addition.
* Assert creation of 'arguments' object
Ensure that the 'arguments' object is created in cases where it is not
required by the body but is required by the parameters.
* Add tests for cases that disable "arguments" map
* Add tests for NewTarget override of bound function
* Add test for properties of exotic String objects
* Add tests for early errors in functions
* Improve tests for class accessors
Use the `propertyHelper.js` utility in order to functionally test the
property descriptors of class methods.
* Remove redundant tests
The semantics of an IdentifierReference as a PropertyDefinition within
an object initializer are exhaustively tested by the files in this
directory whose name match the pattern `prop-def-id-*.js`.
Delete the redundant tests in favor of the more descriptively-named and
more exhaustive alternatives.
* Rename tests
* Update test names to be more descriptive
* Add tests for property descriptors of accessors
* Add tests for runtime error during method dfn
* Add test for observable iteration
* Extend tests for class "name" inference
Ensure that when a class defines a static "name" method, that method
definition prevents the "name" inference behavior.
* Re-generate tests
This is a pure miscellaneous change to use global values of NaN and Infinity instead of their namespaced equivalents.
Ref 9ae0567a50 (r69049894)
cc @suwc
This re-factors some existing Sputnik tests to be more targeted and to use a
pattern that can be generalized to other forms. We could test these all day,
but I've limited myself to forms introduced in ES2015, specifically
YieldExpression and new.target. Note that SpiderMonkey incorrectly throws a
SyntaxError for these.
I thoughtlessly wrote ReferenceError tests for yield = 1 until I realized
that such productions are not actually recognized by the grammar, so the early
errors do not apply. Instead, I've added a negative syntax test for that case.
* Refactor test for valid cover
* Add tests for ValidSimpleAssignmentTarget
Ensure that constructs introduced in ES2015 are disallowed as assignment
targets, with or without a "cover" grammar.
* Add test for grammar precedence of YieldExpression
* Add missing test for early error
* Add missing test for WithBaseObject
* Improve coverage for `new.target`
* Add test for deletion of SuperReference
* Add tests for `in` keyword restrictions
* fixup! Improve coverage for `new.target`
The global isNaN is not precise at all, and Number.isNaN is an ES6 feature that makes it preferrable to use assert's sameValue for NaN values, as it handles it internally using the comparison.
* Improve test coverage for `super` keyword
Add tests for SuperCall and SuperProperty, organized together in the
`test/language/expressions/super/` directory. For SuperProperty, include
tests for usage from within Object initializers and class bodies because
a different set of semantics are observable from each context.
Utilize the test generation tool to increase coverage of destructuring
assignment semantics. Previously, only destructuring assignment in the
AssignmentExpression position was tested. With this change applied, the
same tests will assert expected behavior for destructuring assignment in
`for..of` statements, as well.
A limited number of tests are applied to the `for..in` statement as
well, but due to the iteration protocol observed by that statement, many
destructuring tests are not relevant, and others cannot be automatically
generated from this format.
Add tests that assert the management of the running execution context's
LexicalEnvironment and VariableEnvironment components, as created by the
following abstract operations:
- NewDeclarativeEnvironment
- NewObjectEnvironment
- NewFunctionEnvironment
Many tests require the use of non-strict direct eval, meaning they may
not be run in strict mode. This does not effect coverage because the
semantics in these cases are not observable from strict mode code.
Some situations require the creation of a binding, but this binding has
no relevance to the test itself. In these cases, use names consisting
solely of the underscore character (`_`).
Avoid the use of Block statements wherever possible, as these trigger
the creation of additional environments which may interfere with the
behavior under test.
Closes#572
Introduce tests for new semantics for ES2015 features such as
lexically-scoped bindings. Also add tests for semantics defined in prior
editions of the specification but not yet covered in this test suite.
Limit tests in `language/expressions/call/` and `built-ins/eval/` to
only assert details that are directly related to the invocation pattern.
Re-organize all other tests within the `language/eval-code/` directory,
further categorizing each as `direct` or `indirect` as appropriate.
Remove the extraneous "executable" flag from those files which carried
it.
This harness function is not necessary in the majority of cases in which
it is used. Remove its usage to simplify tests and decrease the amount
of domain-specific knowledge necessary to contribute to the test suite.
Persist the harness function itself for use by future tests for ES2015
modules (such a helper is necessary for tests that are interpreted as
module code).
Except for a small set of expected input/output pairs, both ES5 and
ES2015 define the expected return value of these methods in terms of an
"implementation-dependent approximation." This makes it inappropriate to
enforce expectations for specific values, even if expressed imprecisely.
The files in this patch are organized according to the following naming
scheme:
Prefix | Grammar production
-------------------|-------------------
`array-empty-` | ArrayAssignmentPattern : [ ]
`array-elision-` | ArrayAssignmentPattern : [ Elision ]
`array-rest-` | ArrayAssignmentPattern : [ Elisionopt AssignmentRestElement ]
`array-elem-` | ArrayAssignmentPattern : [ AssignmentElementList ]
`array-elem-trlg-` | ArrayAssignmentPattern : [ AssignmentElementList , Elisionopt AssignmentRestElementopt ]
Suffix | Intent
-------------------|-------
`-abpt-close-err` | The assignment evaluation returns an abrupt completion, and the iterator's `return` method throws an error
`-abpt-close-skip` | The assignment evaluation returns an abrupt completion, but IteratorClose is not invoked
`-abpt-close` | The assignment evaluation returns an abrupt completion, and IteratorClose is invoked as specified
`-get-err` | Abrupt completion returned from GetIterator
`-nrml-close-err` | The assignment evaluation completes, and the iterator's `return` method throws an error
`-nrml-close-null` | The assignment evaluation completes, and the iterator's `return` method returns a non-Object value (there is no corresponding `-abpt-` suffix because the algorithm does not reference the return value in those cases)
`-nrml-close-skip` | The assignment evaluation completes, but IteratorClose is not invoked
`-nrml-close` | The assignment evaluation completes, and IteratorClose is invoked as specified
Not all suffixes are appropriate for all productions. Suffixes have been
simplified in cases where less specificity is necessary to disambiguate
test cases.
Test262 defines tests for expression-producing syntactic forms within
the `language/expressions/` directory. Most tests for object literals
conform to this structure, but 12 such tests were added to the
`language/object-literal/` directory. Move these tests to the canonical
location for object literals.
ECMAScript 2015 introduced tail call optimization for function calls
occuring in a number of positions in the grammar. Assert expected
behavior by triggering a large (but configurable) number of recursive
function calls in these positions. Compliant runtimes will execute such
programs without error; non-compliant runtimes are expected to fail
these tests by throwing an error or crashing when system resources are
exhausted.
This change set includes tests for most invocations of the
SetFunctionName abstract operation in the ES2015 specification.
Practical testing considerations preclude the introduction of tests for
certain invocations:
- The project is still vetting methods to sustainably test the semantics
of the Destructuring Binding pattern across all valid productions.
- 13.3.3.6 Runtime Semantics: IteratorBindingInitialization
- 13.3.3.7 Runtime Semantics: KeyedBindingInitialization
- Without a loader, there is no way to access a function object declared
in an ExportDeclaration, so `name` assignment cannot be tested in
these cases
- 14.1.19 Runtime Semantics: InstantiateFunctionObject
- 14.4.12 Runtime Semantics: InstantiateFunctionObject
- 14.5.15 Runtime Semantics: BindingClassDeclarationEvaluation
- 15.2.3.11 Runtime Semantics: Evaluation
The `negative` frontmatter tag expresses an expectation for the behavior
of the test file as a whole. The `assert.throws` helper function offers
more fine-grained control over expectations because it may be applied to
specific statements and expressions. This makes it preferable in cases
where it may be used (i.e. when the test body does not describe a syntax
error or early error).
Re-implement assertions for errors to use the `assert.throws` helper
function wherever possible.
The `test/language/class/` directory contains a small subset of
Test262's tests for ES2015 classes. The majority of tests for classes
are organized within `test/language/statements/class/`.
- Move the tests that rely on the ClassDeclaration syntactic form from
`test/language/class/` to `test/language/statements/class/`.
- Move the test that relies on the ClassExpression syntactic form from
`test/language/class/` to `test/language/expressions/class/`.
- Re-organize existing tests for identifiers
Name files according to their content (not their ES5 ID). Move tests
for IdentifierName and LabelIdentifier to more appropriate directories.
- Simplify and correct tests
Instead of asserting successful runtime evaluation using `eval`, rely
on the test runner's ability to detect syntax errors. Update the test
bodies to test the grammar referenced by their ES5 ID and
description--the IdentifierStart pattern.
- Use `negative` frontmatter to assert SyntaxErrors
- Remove redundant tests
- Use `assert.equal` helper function
- Add equivalent tests for literal unicode chars
- Add tests for variable-length unicode escape
In ECMAScript 5, assignment to a non-reference value throws a runtime
ReferenceError. ECMAscript 6 specifies an early ReferenceError in these
cases. Tests for this behavior have been authored to pass in both cases.
Simplify these tests to describe and assert the early error.
The `fnExists` function defines a generic way to determine if any number
of values are function instances. Because it is only used by a single
test, the additional complexity required by the generalized code (and
the organizational drawbacks to maintaining another "include" file) are
not justified. Remove the file and update the test to assert the
function's existence directly.
Some tests specifically concern the application of the `use strict`
directive as it appears in JavaScript source code. These tests should
*not* be run with the `onlyStrict` flag because relying on the test
runner to enable strict mode makes the semantics of the source code
irrelevant. Update these tests to use the `noStrict` flag.
Other tests concern language semantics that are only valid in strict
mode, but the mechanism for enabling strictness is inconseqential.
Update these tests to use the `onlyStrict` flag and remove any redundant
`use strict` directive prologues contained within.
Still other tests are valid both within and outside of strict mode.
In keeping with the majority of other tests, do not specify any
restrictions on the environments in which these tests may be run.
This function is equivalent to `$ERROR` (which is automatically included
in test environments). Remove the harness file that defines the
function, remove references to the file from test `includes` lists, and
update scripts to instead invoke the `$ERROR` function.
The previous description of the 11.4.1-4.a-5 test case implied that an environment object could not be deleted while inside a with. However, the actual test was to test whether the variable declaration could be deleted. The description has been updated to better reflect this.
Test 11.4.1-4.a-5 states that it verifies that an environment object cannot be deleted. However, this was giving a false positive. It was actually testing where a "var" declaration on an environment object cannot be deleted (there are other tests for this). This test case fails on Chrome 43, Firefox 38 and Internet Explorer 11.
From the ES6 specification:
> ## PropertyDefinition : IdentifierReference
>
> 1. Let propName be StringValue of IdentifierReference.
> 2. Let exprValue be the result of evaluating IdentifierReference.
> 3. ReturnIfAbrupt(exprValue).
> 4. Let propValue be GetValue(exprValue).
> 5. ReturnIfAbrupt(propValue).
> 6. Assert: enumerable is true.
> 7. Return CreateDataPropertyOrThrow(object, propName, propValue).
The increment/decrement operator evaluates its operand expression once. When
the operand expression is a property accessor, RequireObjectCoercible
and ToPropertyKey are called on the property accessor in the correct order.
The assignment operator evaluates its operands from left to right. When
the left-hand side expression is a property accessor, RequireObjectCoercible
and ToPropertyKey are called on the property accessor before the right-hand
side expression is evaluated.
- Add missing "var" declarations and noStrict flags
- Add strict mode tests for:
- arguments-object/10.6-6-3 -> arguments-object/10.6-6-3-s
- arguments-object/10.6-6-4 -> arguments-object/10.6-6-4-s
- Remove try-finally clean-up code
- Add strict mode compatible tests for eval-code/S10.4.2_A1.1_T*
- No strict mode compatible files added for eval-code/S10.4.2_A1.2_T*, because it doesn't really make sense in this context.
- (S10.4.2_A1.1 and S10.4.2_A1.2 should probably be removed, because the tested behaviour is not required by the spec.)
- Split S8.5_A10, S8.5_A4 and S8.1_A3 into declaration (both modes) and assignment tests (non-strict only)
Part of issue #35.
These tests are derived from the following files within the Google V8
project:
test/mjsunit/es6/generators-iteration.js
test/mjsunit/es6/generators-objects.js
test/mjsunit/es6/generators-runtime.js
test/mjsunit/es6/generators-states.js
- Duplicate properties are allowed in object literals: test/language/expressions/object/*.js
- RegExpBuiltinExec was changed to use ToLength(lastIndex): test/built-ins/RegExp/prototype/exec/S15.10.6.2_A5_T3.js
- Non-undefined flags arguments allowed in RegExp constructor call: test/built-ins/RegExp/*.js
- Array.prototype.push throws TypeError if new length exceeds Number.MAX_SAFE_INTEGER: test/built-ins/Array/prototype/push/S15.4.4.7_A2_T2.js
- .length property of bound functions is configurable: test/built-ins/Function/prototype/bind/15.3.4.5-15-2.js
- Array.prototype changed back to exotic Array object: test/built-ins/Array/prototype/*.js, test/built-ins/Array/isArray/15.4.3.2-0-5.js
- in-statement-position-label-statement.js is an Annex-B test (B.3.2 Labelled Function Declarations), moved to annexB directory
- identifier-let-allowed-as-lefthandside-expression-strict.js needs to check for a SyntaxError (ES6, 12.1.1)
- Remove sort() calls in test/built-ins/Object/getOwnPropertyNames/*.js
- Remove sort() calls in test/language/block-scope/syntax/for-in/acquire-properties-from-*.js
- verifyConfigurable() needs to called last in test/built-ins/Object/is/length.js
- All productions within ClassBody are implicitly strict, update test/language/class/method-definition/yield-as-*.js accordingly
- Remove unnecessary noStrict flag in test/language/class/method-definition/yield-as-generator-method-binding-identifier.js
- Check own symbols are returned in property creation order from Object.getOwnPropertySymbols():
- test/language/computed-property-names/basics/symbol.js
- test/language/computed-property-names/class/method/symbol.js
- test/language/computed-property-names/class/static/method-symbol.js
- test/language/computed-property-names/object/method/symbol.js
- Fix copy-paste error in test/language/expressions/object/method-definition/yield-as-function-expression-binding-identifier.js
According to the new test organization scheme, specification
identification references should be encoded within the test file itself,
and the file should be named according to the semantics of the test.
- Add missing noStrict flags when tests explicitly require non-strict semantics (e.g. unqualified delete, with statement)
- Change `throw new Exception(...)` to `$ERROR(...)` in call/*
- Replace dynamic strict mode check with noStrict flag in call/11.2.3-3_8
- Align assignment/11.13.1_A3.1 with 11.13.1_A3.2 to avoid creating implicit global variable
- Split test into multiple files when undeclared variable are under test
- addition/S11.6.1_A2.4_T3
- division/11.5.2_A2.4_T3
- does-not-equals/11.9.2_A2.4_T3
- equals/S11.9.1_A2.4_T3
- greater-than/S11.8.2_A2.4_T3
- greater-than-or-equal/S11.8.4_A2.4_T3
- in/S11.8.7_A2.4_T3
- instanceof/11.8.6_A2.4_T3
- left-shift/S11.7.1_A2.4_T3
- less-than/S11.8.1_A2.4_T3
- less-than-or-equal/S11.8.3_A2.4_T3
- modulus/S11.5.3_A2.4_T3
- multiplication/11.5.1_A2.4_T3
- right-shift/11.7.2_A2.4_T3
- strict-does-not-equals/11.9.5_A2.4_T3
- strict-equals/11.9.4_A2.4_T3
- subtraction/11.6.2_A2.4_T3
- unsigned-right-shift/11.7.3_A2.4_T3
- Add declaration when implicit global variable creation not part of the test
- assignment/11.13.1_A4_T1
- compound-assignment/S11.13.2_A3.1_T*
- compound-assignment/S11.13.2_A3.2_T*
- grouping/11.1.6_A3_T5
- instanceof/S15.3.5.3_*
- Split test into multiple files when unqualified delete is used
- delete/S11.4.1_A2.2_T1.js
- delete/11.4.1_A3.2
- grouping/S11.1.6_A2
- grouping/S11.1.6_A3_T6
This change adds 'var' declarations for global variables to allow the tests to run in strict mode (see issue #35).
Extra care was taken to ensure the changes do not alter the test behavior, for example when implicit creation of global variables are part of the test.
Note: The change does not fix all strict mode errors due to missing 'var' declarations.
Rick Waldron
Valerie R Young
Josh Wolfe
Leo Balter
Caitlin Potter
Daniel Ehrenberg
Josh Wolfe
Robin Templeton
Caio Lima
André Bargull
Marshall Roch
Mike Pennisi
Shu-yu Guo
⭐caitp⭐
Michael Ficarra
dop251
Tim Disney
Brian Terlson
Jeff Morrison
Mathias Bynens
Ian Halliday
Gorkem Yakin
Brian Terlson
Catherine Farman
unknown
Thomas Wood
Pieter van Ginkel