e4f4abdcb2
This PR proposes changes to existing test262 tests to allow them to pass under Hardened JavaScript (see Secure ECMAScript proposal and Hardened JavaScript). Moddable uses Hardened JavaScript for JavaScript runtimes on resource constrained embedded devices, including those targeted by ECMA-419.
The changes fall into four groups:
1. Replace use of new Date() with new Date(1970). Scripts running inside a Compartment cannot retrieve the current time, so new Date() throws but new Date(1970) succeeds. Very few tests need the current time, but instead simply need a Date instance.
2. Use Object.defineProperty instead of setting existing built-in properties directly, such as toString and toValue. In Hardened JavaScript, prototypes of built-in objects are frozen. Consequently, setting properties of an instance that exist on the prototype throw (Hardened JavaScript is always in strict mode).
3. Eliminate use of Math.random(). Scripts running inside a Compartment cannot generate random numbers. One test identified so far uses Math.random() in a way that can easily be replaced with a counter.
4. Narrow the scope of exception tests. Consider the following
assert.throws(TypeError, () => {
var s1 = new Date();
s1.toString = Boolean.prototype.toString;
s1.toString();
});
This test passes, but only because new Date() fails by throwing a TypeError. If the invocation of the Date constructor is resolved by (1) above, then the assignment to toString fails as per (2) above. The script should be modified as below to ensure that assert.throws only tests the intended statement, s1.toString(). The modified script tests the intended functionality and passes under Hardened JavaScript
var s1 = new Date(1970);
Object.defineProperty(s1, "toString", {
value: Boolean.prototype.toString
});
assert.throws(TypeError, () => {
s1.toString();
});
This is an initial PR to begin the process of adapting test262 for use with Hardened JavaScript. Further changes are expected, with the vast majority likely to fall into the four groups described above.
Thank you to gibson042, kriskowal, and erights for their advice on this work.
Test262: ECMAScript Test Suite (ECMA TR/104)
Test262 is the implementation conformance test suite for the latest drafts (or most recent published edition) of the following Ecma specifications:
- ECMA-262, ECMAScript Language Specification
- ECMA-402, ECMAScript Internationalization API Specification
- ECMA-404, The JSON Data Interchange Format (pdf)
Test262 itself is described in ECMA TR/104 and is included in ECMA-414 (pdf).
Goals & State of Test262
The goal of Test262 is to provide test material that covers every observable behavior specified in the ECMA-414 Standards Suite. Development of Test262 is an on-going process. As of October 2017, Test262 consisted of over 29272 individual test files covering the majority of the pseudo-code algorithms and grammar productions defined in the ECMA-414 Standards Suite. Each of these files contains one or more distinct test cases. This marks the most comprehensive ECMAScript test suite to date. While test coverage is broad, TC39 does not consider coverage to be complete and as with any software project there exists the possibility of omissions and errors. This project welcomes any contributions to Test262 that help make test coverage of existing features more comprehensive.
ECMAScript feature proposals
As defined in the TC39 Process, Stage 4 Entrance Criteria requires tests for new feature proposals to advance. Tests may be written by proposal champions, implementers, or any interested community member.
A proposal champion is someone that worked on the feature proposal and specification directly.
An implementer is someone that works on implementing the proposal into a JavaScript engine, parser, runtime or embedding.
A community member is you, and we welcome you to contribute! If you're having trouble getting started, or even just want to ask a question, feel free to open an issue.
Contributing to Test262
Guidance for contributing to Test262 can be found in CONTRIBUTING.md.
Authors of contributions from non-Ecma member organizations must sign the Test262 CLA
Running Test262
Guidance for running Test262 and explanations of how a test file must be interpreted by a test runner is in INTERPRETING
Rationale
This project offers an explanation for many of its design decisions and maintenance practices--see rationale.md.
Test262 Runners
Volunteer-maintained projects that may be used to execute Test262 in various ECMAScript hosts:
- https://github.com/bterlson/test262-harness (platform: Node.js)
- https://github.com/test262-utils/test262-harness-py (platform: Python)
- https://bakkot.github.io/test262-web-runner/ (platform: web)
- https://github.com/Izhido/test262_harness_cpp (platform: C++)
- https://github.com/lahma/test262-harness-dotnet (platform: .NET)
How To Read CI Results
Test262 runs CI tests against every PR and commit. The only tests that are required to pass are visibly flagged as REQUIRED. The CI test results that are attributed to specific runs against specific engines should not be perceived as meaningful to anyone but the person that is reviewing the test material contained within the contributed changeset. These tests are almost always expected to fail, especially in the case of tests for new features. They may be helpful in determining whether or not a regression occurred, but that can only be determined by an actual human reviewing the results and comparing those outcomes to the expected outcomes of the tests.
Where did website/ go?
It's been removed. If you need to access the code that contained in that directory, you can find it here.