Migrating all tests to this API is necessary to prevent tests from hanging indefinitely when a SAB is sent to a worker but the code in the worker attempts to create a non-sharable TypedArray (something that is not Int32Array or BigInt64Array). When that scenario occurs, an exception is thrown and the agent worker can no longer communicate with any other threads that control the SAB. If the main thread happens to be spinning in the $262.agent.waitUntil() while loop, it will never meet its termination condition and the test will hang indefinitely.
Because we've defined $262.agent.broadcast(SAB) in https://github.com/tc39/test262/blob/master/INTERPRETING.md, there are host implementations that assume compatibility, which must be maintained.
$262.agent.MAX_TIME_EPSILON was intended to be used for callers like `Atomics.wait(typedArray, indexNumber, valueNumber, timeoutNumber)` where all parameters with the `Number` suffix denote values which are already Number values. It should not be used for `Atomics.wait(typedArray, indexObject, valueObject, timeoutObject)` where all parameters with the `Object` suffix denote values which are Object values, because in that case we'd require that `ToNumber(objectValue)` (potentially invoked multiple times) completes in less than 100 milliseconds (the default value for MAX_TIME_EPSILON).
Also removes $262.agent.MAX_TIME_EPSILON because it is now no longer used.
Also fixes multiple issues and bugs in various Atomics tests.
CONTRIBUTING.md and INTERPRETING.md:
- Add missing description for `CanBlockIsFalse` flag and also introduce `CanBlockIsTrue` flag which is needed for some tests (see below).
harness/atomicsHelper.js:
- Updated `$262.agent.waitUntil` to clarify it can also be called with BigInt64Array objects.
- Added `$262.agent.timeouts` constants to unify the various, inconsistently used timeouts in the Atomics tests. Each timeout constant has a brief description and a usage example to clarify when it should or should not be used. The default values were tested in SpiderMonkey (locally and CI) under various system load levels and should hopefully also be valid for other engines.
- Added `$262.agent.tryYield` to yield control from the main thread. The default implementation simply calls `$262.agent.sleep`, but test262 hosts could theoretically provide a different implementation.
- Added `$262.agent.trySleep` to replace direct calls to `$262.agent.sleep` from the main thread. Motivation for this function: Some test262 hosts (like browsers) may be able to pause and sleep on the main thread, so they could provide their implementation for `trySleep` which performs a busy-wait or something do nothing.
harness/testAtomics.js:
- Replace `let`, `arrow-functions`, and `for-of` loops to avoid creating extremely long `features` lists in tests using this helper. Removed `Map`, `WeakMap`, `Set`, and `WeakSet` (these weren't even listed in features.yaml) for the same reason and added the missing `Symbol` entry for this file to features.yaml.
- Updated all files including "harness/testAtomics.js" to match the new feature requirements.
test/built-ins/Atomics/store/good-views.js and test/built-ins/Atomics/store/bigint/good-views.js:
- Replace `arrow-functions` and `for-of` loops with ES5 alternatives.
test/built-ins/Atomics/wait/bigint/*.js and test/built-ins/Atomics/wake/bigint/*.js
- Some tests were using `BigInt64Array.BYTES_PER_ELEMENT * 8` for the SharedArrayBuffer length, but their non-BigInt counterparts are using `Int32Array.BYTES_PER_ELEMENT * 4`. For consistency and to make it easier to compare the BigInt against the non-BigInt versions, I've changed it to `BigInt64Array.BYTES_PER_ELEMENT * 4`.
- Also aligned formatting and statement placement when they differed between the non-BigInt and the BigInt version of a test file. (I've diffed some of the non-BigInt and BigInt files against each other and different formatting was a nuisance.)
Test files using `$262.agent.monotonicNow()`:
- Moved `$262.agent.report()` calls outside of the block of code measured `$262.agent.monotonicNow()` to avoid measuring how long it takes to execute `$262.agent.report()`.
- Without this change some tests failed intermittently in certain test configurations in SpiderMonkey. For example with the flags `--ion-eager -- ion-offthread-compile=off` which forces early Ion compilation on the main thread. The `$262.agent.report()` implementation in the SpiderMonkey test262 host embedding uses a for-loop which was forcefully Ion compiled under these settings. And because Ion compilation can take some time, the test case ran longer than `$262.agent.MAX_TIME_EPSILON` which lead to intermittent failures.
Test files using `CanBlockIsFalse` / `CanBlockIsTrue`:
- Some of these tests actually expected that the main thread can wait and [[CanBlock]] is `true` for the agent record executing the test. Therefore I've added a new `CanBlockIsTrue` flag and replaced the flags where needed.
test/built-ins/Atomics/wait/**/*.js and test/built-ins/Atomics/wake/**/*.js:
- Use an atomic counter `RUNNING` in more tests to have better control when a worker agent was actually started.
- Replace the various `$262.agent.sleep(/* Sleep X ms to ensure worker actually sleeps */)` calls with the new `$262.agent.tryYield()` function. This `X` was sometimes as low as 10 milliseconds, which is definitely too short for CI systems under heavy load (observed by intermittent CI failures for SpiderMonkey) and sometimes as high as 500 milliseconds, which is probably much longer than needed even when the system is under heavy load.
- Removed duplicate strings in assertion messages, presumably from copy-pasting the messages between different files.
- Removed extra empty lines at the end of multiple files.
test/built-ins/Atomics/wake/bad-range.js, test/built-ins/Atomics/wake/bigint/bad-range.js, and test/built-ins/Atomics/wait/bigint/non-bigint64-typedarray-throws.js:
- Removed unnecessary `features` and `includes` from this file.
test/built-ins/Atomics/wait/waiterlist-order-of-operations-is-fifo.js and test/built-ins/Atomics/wait/bigint/waiterlist-order-of-operations-is-fifo.js:
- The test was actually broken and didn't test what it said it does. This probably explains #1530.
- The test wants to ensure the waiterlist is implemented as a FIFO structure. This requires that the waiting agents all wait on the same index position, because the waiterlists are defined by each index. But if the agents wait on different indices, each agent is inserted into a different FIFO structure and therefore we can't observe any FIFO ordering between the agents when they're woken up.
- All this requires a bit of synchronization between the main agent and the waiting agent, I hope the added comments help to review these changes.
test/built-ins/Atomics/wait/good-views.js:
- The agent sends multiple reports to the main agent, but only the first one was read.
- This error was introduced during previous refactorings. I've changed it to back to use a while-loop as used in the first version of this file.
test/built-ins/Atomics/wait/no-spurious-wakeup-no-operation.js:
- Only the BigInt version of this test was present, copied it so we also get code coverage for the non-BigInt case.
test/built-ins/Atomics/wait/waiterlist-block-indexedposition-wake.js:
- Added extra while loops to avoid intermittent failures when the agent worker haven't started to wait.
- This should help to avoid some of the intermittent failures we saw for SpiderMonkey.
test/built-ins/Atomics/wake/wake-all-on-loc.js and test/built-ins/Atomics/wake/bigint/wake-all-on-loc.js:
- This test was also no longer after previous refactoring sessions.
- The "B" agent only waited for 10 milliseconds, which made it likely that it already timed out before the main agent was able to call `Atomics.wake`, which in turn rendered the test useless, because the test case wants to ensure that `Atomics.wake` cannot wake "B". But if "B" was already timed out, it can trivially not be woken by `Atomics.wake`.
- Added some safety measure to catch the case when "B" timed out before `Atomics.wake` was called and made it a test error if that happens.
test/built-ins/Atomics/wake/count-defaults-to-infinity-missing.js, test/built-ins/Atomics/wake/count-defaults-to-infinity-undefined.js, and test/built-ins/Atomics/wake/undefined-index-defaults-to-zero.js:
- Changed the `$262.agent.start()` calls to use a for-loop to avoid code duplication.
- (Forgot to undo the code formatting around `assert.sameValue`, still need to change it back.)
- Also more while-loops around `Atomics.wake`.
- These changes should fix#1529 and #1566.
test/built-ins/Atomics/wake/wake-all.js:
- Removed "B" worker agent.
- Without this removal the test case would be exactly equal to test/built-ins/Atomics/wake/wake-all-on-loc.js.
test/built-ins/Atomics/wake/wake-in-order-one-time.js:
- Add for-loops to avoid code duplication and make the test more readable.
- Make the `Atomics.wake` assertion messages unique by adding the current loop counter.
- Add `$262.agent.tryYield()` to give the worker agents enough time to actually start waiting.
test/built-ins/Atomics/wake/wake-in-order.js:
- Removed the outer loop `attempt < 10` because it uses `$262.agent` in a way currently not required to work. And which actually also doesn't work in SpiderMonkey's implementation of `$262.agent`.
- According to INTERPRETING.md `$262.agent.broadcast()` broadcasts its message to all agents ever started by `$262.agent.start()` and then blocks until all agents received the broadcast. It is not required that the agents started by the first `$262.agent.broadcast()` call will all be disabled/destroyed/whatever when the second `$262.agent.broadcast()` call occurs, which then means the second `$262.agent.broadcast()` call still tries to reach the agents started in the first loop iteration, but these may no longer accept broadcasts and therefore won't acknowledge they've received the broadcast. Which then means the second `$262.agent.broadcast()` call waits forever.
* Add Intl.RelativeTimeFormat feature.
* Add tests for the shape of the Intl.RelativeTimeFormat API.
* fixup! Add tests for the shape of the Intl.RelativeTimeFormat API.
In order to satisfy the project's formatting rules, license information
was inserted into a number of files where it had been mistakenly omitted
by the original contributors [1]. In many cases, the license information
did not accurately describe the contributor or the time of contribution.
Update the information according to the information provided by the
contributors at the time each file was authored:
- atomicsHelper.js - a72ee6d912
- detachArrayBuffer.js - 70c7375be8
- nans.js - b17ffc0298
- nativeFunctionMatcher.js - 6b7cbb5035
- proxyTrapsHelper.js d530c87b41
- regexpUtils.js - 44b40e083e
- tcoHelper.js - 4dc81d3788
[1] 4ea2931f16
Previously, the error message generated by failed asynchronous tests was
generic and underspecified. Improve the format and explicitly document
it in project's interpreting guidelines.
`function nan() { return /* expression to compute NaN */ }` may not return the same implementation-distinguishable NaN value before and after JIT compilation.
* Fix bad references on tests for BigInt TypedArrays
* Remove bad conversions for BigInt TypedArray
* Cleanup the BigInt TypedArray harness file
Remove non used code (testBigIntTypedArrayConversions)
Move the constructors list to inside the exposed function, this prevents early implementations to fail before the function is called.
* Fix bad references in TypedArrays.of (BigInt)
* Remove BigInt tests from typedarray harness test
* Use BigInt for BigInt typedArrays
* Apply last fixings on BigInt TypedArray tests
* Apply fixes to last revision from @anba
- optimize file names
- BigInt folder for TAs.from and of
- copy tests preparing for bigint
- copy ta prototype methods for bigint
- Use an exclusive harness for bigint typedarrays
- add features
- use proper harness
- use preper harness
- Restore original tests for TypedArrays
- final fixes
- fix includes
- add @jakobkummerow's changes
- remove EOL whitespace
- use 'Array.isArray' instead of 'instanceof Array'
- check for BigInt type in toLocaleString tests
- specify TypedArray constructor list for non-BigInt tests
- update TypedArray harness test for BigInt
- add a missing type coercion
- disable more tests for Big(U)Int64Array
- check for BigInt before using BigInt TypedArray constructors
The use of `let` is unnecessary here, particularly in a
file that is needed to run any of the tests, even the ES5 tests.
Removing it allows older engines to run older tests.
typeCoercion.js is included in several tests which are not marked as `features: [BigInt]`.
Since BigInt is new feature, we should not make the above unrelated tests failed due to
lack of BigInt syntax support.
Close#1252.
* Move decimalToHexString into harness instead of duplicating it in multiple files
* Optimize decimalToHexString and support numbers greater than 65535
* Replace alternative decimalToHexString function with include for decimalToHexString.js
* Add decimalToHex2String to return the string representation of a two-digit hex-number
* Replace decimalToHex2String with decimalToPercentHexString to return the percent hex-encoded string of a two-digit hex-number
* Replace two String.fromCharCode calls with a single call
* Further reduce string concatentations in decodeURI[Component] tests
* Remove unnecessary Test262Error error handling in catch-clauses
* Remove try/catch wrappings in decodeURI/encodeURI tests
Because expectations regarding error "phase" are now expressed via test
meta-data, the test runner may now enforce this requirement on negative
tests.
Remove the "NotEarlyError" from the project source. This reduces the
amount of domain knowledge required to author tests and lessens the
potential for inconsistencies between tests.
Ensure that NaN values are canonicalized consistently by all invocations
of SetValueInBuffer. Also ensure that `%TypedArray%.prototype.set` and
`%TypedArray%.prototype.slice` preserve the bit-level encoding of the
source data.
Use a set of experimentally-derived expressions known to produce NaN
values with distinct bit patterns in various platforms.
Modify the regular expression for native functions' toString value to
satisfy all test cases. Correct the test file's reference to the harness
file. Re-format the test file's assertions to aid debugging in the event
of failure.
The millisecond representation of a given Date instance is dependent on
the local system's time zone settings. In order to pass consistently
across contexts, tests for this value must take the system configuration
into account.
Introduce a test harness utility function to encapsulate these concerns.
Re-use this function across all test files that assert the exact
millisecond representation of Date instances.
The harness file `Test262Error.js` has not contained executable code since it
was introduced in this project [1]. The definition of the `Test262Error`
function has consistently been located in the `sta.js` harness file which test
runners are expected to inject into the test environment.
Remove the file and all references to it.
[1] See commit c33bf0e043
ECMAScript 2015 introduced tail call optimization for function calls
occuring in a number of positions in the grammar. Assert expected
behavior by triggering a large (but configurable) number of recursive
function calls in these positions. Compliant runtimes will execute such
programs without error; non-compliant runtimes are expected to fail
these tests by throwing an error or crashing when system resources are
exhausted.
Some JavaScript source files are only relevant in the context of the
Test262 website. They should not be explicitly included by individual
tests, so their presence in the `harness/` directory alongside "include"
files is misleading.
Move the scripts to a location within the `website/` directory to
better-reflect their intended use. Update the relevant HTML templates
with the new locations.
Although test files once expressed dependencies on external files using
a global `$INCLUDE` function, that pattern was removed in favor of
declarative meta-data [1].
Remove the associated logic from the Python runner and the browser.
[1] See commit d4354d14d5.
The `fnExists` function defines a generic way to determine if any number
of values are function instances. Because it is only used by a single
test, the additional complexity required by the generalized code (and
the organizational drawbacks to maintaining another "include" file) are
not justified. Remove the file and update the test to assert the
function's existence directly.
The `environment.js` file has been empty since its initial introduction
to this project [1]. It has no effect on any of the contexts in which it
is currently referenced, so it may be safely removed.
[1] be82787a00
Unlike the console runner, the browser runner does not modify the
strictness of tests prior to running them. Regardless of a given test's
metadata, it runs every test exactly once, and it never enables strict
mode. This means that tests intended to function in strict mode must
declare the "use strict"; directive prologue in addition to the
`onlyStrict` flag.
For any test that specifies the `onlyStrict` metadata flag, transform
the source code by injecting a "use strict" directive prologue prior to
running the test.
Since the Python runner was updated to include `assert.js` in all tests
unconditionally, a number of tests have been written that implicitly
rely on its presence. The browser runner does not currently provide this
file's contents to these tests, so they fail unconditionally.
Update the browser runner to inject that file's contents into every test
context.
Note: the existing approach to file retrieval (namely loading via
synchronous XHR requests) is inefficient and deprecated in some
browsers. It is honored here for the sake of consistency and to minimize
the changeset necessary to fix the browser runner.
The `$PRINT` helper function has no effect on test behavior. The
following tests use it to report assertion outcomes (and as a result
will fail silently):
- test/language/function-code/S10.2.1_A4_T2.js
- test/language/function-code/S10.2.1_A5.1_T1.js
- test/language/function-code/S10.2.1_A5.1_T2.js
- test/language/function-code/S10.2.1_A5.2_T1.js
Remove the function definition and all references within tests. Update
tests that use it as an error reporting mechanism to instead use an
appropriate `assert` helper function.
This function is equivalent to `$ERROR` (which is automatically included
in test environments). Remove the harness file that defines the
function, remove references to the file from test `includes` lists, and
update scripts to instead invoke the `$ERROR` function.
- Remove no longer needed noStrict flags.
- Change tests to use propertyHelper.js for strict mode compatibility.
- Add tests for return value of `delete` operator, e.g. `delete Array.prototype === false`.
- Add non-writable tests for global NaN property.
- Split some tests to run in strict mode.
- Change tests with global `var length` declaration to use a different variable name for browser environments.
- Merge duplicated tests symbol-data-property-configurable-non-strict and symbol-data-property-configurable-strict.
- Merge duplicated tests symbol-data-property-writable-non-strict and symbol-data-property-writable-strict.
- (And a small change in propertyHelper to reduce code duplication in function call.)
The `testBuiltInObject` function verifies the presence of the specified
properties for *all* object types (not just function objects). Update
the error messages to reflect this.
Only attempt to re-set the property value in cases where it was
successfully modified as part of the function's execution. This avoids
errors when the underlying value is not writable. Rename the internal
result-tracking variable to make this more clear.
When invoked without a custom assertion message, `assert.sameValue` and
`assert.notSameValue` automatically create a message that describes the
actual and expected values.
Extend both assertion methods to also include this information in cases
where a custom message has been specified.
These tests are derived from the following files within the Google V8 project:
test/mjsunit/harmony/array-fill.js
test/mjsunit/harmony/array-find.js
test/mjsunit/harmony/array-findindex.js
test/mjsunit/harmony/array-from.js
test/mjsunit/harmony/array-of.js
- this change hits all files that use
dataPropertyAttributesAreCorrect or
acccessorPropertyAttributesAreCorrect
Major changes:
- remove include of runTestCase.js,
dataPropertyAttributesAreCorrect.js and
acccessorPropertyAttributesAreCorrect.js
- add include of propertyHelper.js
- remove function testCase() and call to
function runTestCase()
- instead of collecting test state into booleans,
which are eventually returned from runTestCase,
test conditions and throw errors immediately
- use negative: to check error types instead of
using `instanceof` in a catch() block
Selected commit logs follow:
manual conversion of accessorPropertyAttributesAreCorrect() test
- remove runTestCase
- remove wrapping testCase function
- unindent
- remove includes of runTestCase & accessor...
- add include of propertyHelper.js
- unpack final two args 'true, false' to explicit inline
tests of isEnumerable and isConfigurable
- unpack setter test into inline logic
- unpack getter test into inline logic
used script to replace includes: frontmatter
with propertyHelper.js ; manually added back
references to fnGlobalObject() where needed
additional helper functions
add helper functions for get and writable
duplicate runTestCase so can remove runTestCase.js from
includes of all tests in batch
use keyboard macros to simplify repetitive conversions
remove auto-save file accidentaly committed
minor changes, manually remove runTestCase from remaining files
lint etc. fixes
remove now-obsolete harness files
make strict/nonstrict variants
tests where behavior is different in strict/nonstrict
need two variants
- strict mode throws on assign to read-only element (no setter fn)
- strict mode separates named arguments from arguments[]
fix indentation
cleanup minor issues
reviewing 15.2.3.7-9 tests, found some minor issues
- indentation/spacing
- duplicate test (eg, direct test of value and dataProp...)
- remove needless try/finally
- use assert.. helpers in place of if(is..
- rename some assertions to 'e'
rename assertX to verifyX
new helper fn sameValue
test "set" as well as get
reverse order of checks
restore test of desc2
restore test of length
restore test of ownProperty
incorporate suggestions re order of ops, Object.prototype
add test of sameValue helper
restore return value in getter
restore second defineProperty call
restore check of explicit "false"
restore explicit test of TypeError
set noStrict flag on arguments changes
rename catch var to "e" b4ad0e6
remove dataPropertyAttributesAreCorrect from tests 0d83ff2
remove accessorPropertyAttributesAreCorrect from tests bb926f3
remove {data,accessor}Property... fns from harness
fix writable check on array.length
call $ERROR if expected exception not thrown
fix Epected => Expected typo
use assert._isSamevalue
Rick Waldron
Leo Balter
Ms2ger
Mathias Bynens
André Bargull
Ms2ger
jugglinmike
peterwmwong
Michael Ficarra
Robin Templeton
Aakash Patel
Yusuke Suzuki
Josh Wolfe
Doug Ilijev
Daniel Ehrenberg
Lars T Hansen
Shu-yu Guo
Kevin Gibbons
Gorkem Yakin
dtig
Michael Ficarra
Jordan Harband
Brian Terlson
smikes
Erik Arvidsson
Brian Terlson