From 29f321a4e9327c8bde84479a33ed4ea997d89dcb Mon Sep 17 00:00:00 2001 From: Philip Chimento Date: Fri, 28 Oct 2022 17:10:10 -0700 Subject: [PATCH] Document staging folder Most of the information taken from the discussion in the issue. Closes: #3649 --- CONTRIBUTING.md | 23 +++++++++++++++++++++++ INTERPRETING.md | 8 ++++++-- 2 files changed, 29 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0c46598c69..cd5e833cd8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -246,6 +246,29 @@ This key identifies the section number from the portion of the ES6 standard that Read the [Test262 Technical Rationale Report](https://github.com/tc39/test262/wiki/Test262-Technical-Rationale-Report,-October-2017#specification-reference-ids) for reasoning behind deprecation. +## Staging + +There is a `test/staging/` folder, containing tests that are subject to fewer requirements, in order to get tests running across more than one implementation as early as possible, and promote interoperability of in-progress features. + +Tests in staging are not required to be split up into one test per file, or to conform to any particular style as long as they are runnable. +In particular, mechanically-converted tests from implementations' private test suites are encouraged, as sharing them with other implementations promotes interoperability. + +These are the requirements for adding a test to staging: + +1. Just as with a test outside of staging, it must test an existing language feature, a Stage 3 TC39 proposal, or a normative pull request. +2. It is correct. +3. It is runnable using the usual test262 runner. +4. It is reviewed by someone with permission to land tests in the staging folder, not necessarily a test262 maintainer. +5. If it requires the implementation to support ECMA-402, it must be in `test/staging/intl402/` or a subfolder thereof. + +Tests are intended to live temporarily in staging. +Once the feature they test has stabilized, contributors are encouraged to refine them to meet the additional requirements of the main test262 suite, and move them out of the staging folder. + +Tests in staging do not count towards the test262 coverage requirement for a TC39 proposal to reach Stage 4. + +Implementations may designate a group of people who have permission to review and land tests in the staging folder. +To add or remove people from this group, please open an issue. + ## Test Environment Each test case is run in a fresh JavaScript environment; in a browser, this will be a new <iframe>; for a console runner, this will be a new process. diff --git a/INTERPRETING.md b/INTERPRETING.md index 9f4f1eedfd..8cb36452a4 100644 --- a/INTERPRETING.md +++ b/INTERPRETING.md @@ -160,10 +160,14 @@ located at `test/language/import/nested/dep.js`. Files bearing a name ending in `.json` are intended to be interpreted as JSON. +### Staging + +Tests in the `test/staging/` folder are expected to be executed just like all the other tests, in order to promote interoperability as soon as possible. + ### Implementations without ECMA-402 -Tests in the `test/intl402/` folder assume that the implementation supports [ECMA-402, the ECMAScript Internationalization API Specification](https://tc39.github.io/ecma402/). -When testing an implementation lacking the capabilities in ECMA-402, the tests in that folder should be skipped. +Tests in the `test/intl402/` and `test/staging/intl402/` folders assume that the implementation supports [ECMA-402, the ECMAScript Internationalization API Specification](https://tc39.github.io/ecma402/). +When testing an implementation lacking the capabilities in ECMA-402, the tests in those folders should be skipped. ## Test Results