Commit Graph

316 Commits

Author SHA1 Message Date
Sam Mikes 5fbb0a3902 remove spurious flags: [path] 2014-10-24 15:24:57 +01:00
Sam Mikes 6b7e5d7747 bestPractice: supply error names to negative
pilot directory for replacing flags: [negative] with
negative: errorname

ch07-7.2,7.3: add error names
ch07-rest: insert error names rest of ch07
ch08: add expected error name
ch11: add error name to negative
ch12: put error name in negative

ch13: add error name to negative
only one test, did it manually

ch15: add error name to negative:

ch07: add error name to negative

these tests used flags: \n - negative and so
were not caught by the earlier naive grep for flags: [negative]

--- changes suggested by @anba

bestPractice - remove added NotEarlyError
ch07,ch11 - expect ReferenceError (req'd by ES6)
correct test descriptions to expect ReferenceError only
enforce parsing as block statement by adding `;`
remove needless include of $FAIL.js

effectively, a rebase of all changes onto master

    Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice

    Merge branch 'saved-bestPractice-negative' into negative-addErrorName-work

    Conflicts:
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js
    	test/suite/ch12/12.1/S12.1_A4_T1.js
    	test/suite/ch12/12.5/S12.5_A2.js

    incorporate changes made on PR branch

    re-remove added NotEarlyError

    Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice

    Conflicts:
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js
    	test/suite/ch12/12.1/S12.1_A4_T1.js
    	test/suite/ch12/12.5/S12.5_A2.js

    ch15: correct negative regexes

    ch10: avoid use of not-NotEarlyError regex

    ch14 error regexps

    ch13 error regexps

    ch12: negative error matching

    Merge branch 'console-runner-checkError' into negative-addErrorName-work

    test262.py: check negative tests with regex

    implement checking of negative tests =>

    negative: SyntaxError

    means that /SyntaxError/ must match stderr
    or test is reported as failure

    Fixes #78

    expect real errors

    bestPractice: supply error names to negative

    pilot directory for replacing flags: [negative] with
    negative: errorname

    ch07-7.2,7.3: add error names
    ch07-rest: insert error names rest of ch07
    ch08: add expected error name
    ch11: add error name to negative
    ch12: put error name in negative

    ch13: add error name to negative
    only one test, did it manually

    ch15: add error name to negative:

    ch07: add error name to negative

    these tests used flags: \n - negative and so
    were not caught by the earlier naive grep for flags: [negative]

    --- changes suggested by @anba

    bestPractice - remove added NotEarlyError
    ch07,ch11 - expect ReferenceError (req'd by ES6)
    correct test descriptions to expect ReferenceError only
    enforce parsing as block statement by adding `;`
    remove needless include of $FAIL.js

    Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice

    resolved Conflicts:
    	test/suite/ch07/7.9/S7.9_A5.7_T1.js
    	test/suite/ch11/11.13/11.13.1/S11.13.1_A2.1_T3.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T1.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T10.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T11.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T3.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T4.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T5.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T6.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T7.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T8.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T9.js
    	test/suite/ch11/11.3/11.3.1/S11.3.1_A2.1_T3.js
    	test/suite/ch11/11.3/11.3.2/S11.3.2_A2.1_T3.js
    	test/suite/ch11/11.4/11.4.4/S11.4.4_A2.1_T3.js
    	test/suite/ch11/11.4/11.4.5/S11.4.5_A2.1_T3.js
    	test/suite/ch12/12.5/S12.5_A2.js
    	test/suite/ch12/12.6/12.6.3/S12.6.3_A4_T2.js

    negative: second half of @anba notes

    correct test descriptions to expect ReferenceError only
    enforce parse as block statement by adding `;`
    remove needless include of $FAIL.js

    ch12: specify type

    negative: @anba fixes

    bestPractice - remove added NotEarlyError
    ch07,ch11 - expect ReferenceError (req'd by ES6)

    bestPractice: supply error names to negative

    pilot directory for replacing flags: [negative] with
    negative: errorname

    ch07-7.2,7.3: add error names
    ch07-rest: insert error names rest of ch07
    ch08: add expected error name
    ch11: add error name to negative
    ch12: put error name in negative

    ch13: add error name to negative
    only one test, did it manually

    ch15: add error name to negative:

    ch07: add error name to negative

    these tests used flags: \n - negative and so
    were not caught by the earlier naive grep for flags: [negative]

    expect a SyntaxError

    ch07: add error name to negative

    these tests used flags: \n - negative and so
    were not caught by the earlier naive grep for flags: [negative]

    ch15: add error name to negative:

    ch13: add error name to negative

    only one test, did it manually

    ch12: put error name in negative

    ch11: add error name to negative

    ch08: add expected error name
    ch07-rest: insert error names rest of ch07

    ch07-7.2,7.3: add error names

    negative-errorname: pilot

    pilot directory for replacing flags: [negative] with
    negative: errorname

    bestPractice: supply error names to negative

bestPractice: supply error names to negative

pilot directory for replacing flags: [negative] with
negative: errorname

ch07-7.2,7.3: add error names
ch07-rest: insert error names rest of ch07
ch08: add expected error name
ch11: add error name to negative
ch12: put error name in negative

ch13: add error name to negative
only one test, did it manually

ch15: add error name to negative:

ch07: add error name to negative

these tests used flags: \n - negative and so
were not caught by the earlier naive grep for flags: [negative]

bestPractice: supply error names to negative

negative-errorname: pilot

pilot directory for replacing flags: [negative] with
negative: errorname

ch07-7.2,7.3: add error names

ch07-rest: insert error names rest of ch07

ch08: add expected error name

ch11: add error name to negative

ch12: put error name in negative

ch13: add error name to negative

only one test, did it manually

ch15: add error name to negative:

ch07: add error name to negative

these tests used flags: \n - negative and so
were not caught by the earlier naive grep for flags: [negative]

expect a SyntaxError

negative: @anba fixes

bestPractice - remove added NotEarlyError
ch07,ch11 - expect ReferenceError (req'd by ES6)

ch12: specify type

negative: second half of @anba notes

correct test descriptions to expect ReferenceError only
enforce parse as block statement by adding `;`
remove needless include of $FAIL.js

expect real errors

ch12: negative error matching

ch13 error regexps

ch14 error regexps

add "description" header back

remove now-unused includes

remove needless include
2014-08-13 00:19:11 +01:00
Brian Terlson a794e0eb54 Merge pull request #76 from anba/issue-61/wrong-test-assertions
Correct test assertion (Fixes issue#61)
2014-08-05 23:52:06 -07:00
Brian Terlson 7f5310b4d5 Merge pull request #77 from anba/issue-63/correct-array-fill-test
Correct Array.prototype.fill tests (Fixes #63)
2014-08-05 23:51:28 -07:00
André Bargull f0ac90a5fd Move tests to Math.clz32 (Fixes #41)
Number.prototype.clz() was renamed to Math.clz32(). Also removed superfluous 'spec' entry in test descriptor.
2014-08-04 20:57:02 +02:00
André Bargull e256a76e9d Correct Array.prototype.fill tests (Fixes #63)
The expected results weren't correct.
2014-08-04 19:44:22 +02:00
André Bargull 28649e8772 Correct test assertion (Fixes issue#61)
Changed Number.isNaN_Boolean.js, String.prototype.endsWith_Fail.js and String.prototype.endsWith_Fail_2.js to test for the correct result. Also removed implicit coercion in a couple of other test files.
2014-08-04 19:10:41 +02:00
Sam Mikes 4205a1da64 harness: remove unused code
sta.js:
slight change to Test262Error() semantics; message property now always set (default "")
make $ERROR a var
set $ERROR to function $ERROR so it can be overridden if needed
remove 2009 copyright in favor of 2012 copyright
REVERTED: remove never-used fn testFailed
testFailed actually used by $FAIL
remove obsolete fn $INCLUDE

ed.js:
remove commented-out obsolete code

test262.py:
remove always-included harness file that provides no functions used by any extant test
add comment line to nonstrict

cth.js: define `print` for node, cscript

use cth (console test harness) to define functions
for console runner.  V8, Spidermonkey (js) and JavaScriptCore (jsc)
provide a function `print`.  Provide a default `print` for node
and cscript

set print_handle to 'print' by default (can still override)
for cscript: wrap tests in try/catch so we get syntax errors
2014-08-03 00:33:08 +01:00
Sam Mikes bd80bf7dfc es6/Math: use Number.isNaN
use number.isNaN to test for NaN
pointed out by @anba in #60
2014-07-31 13:53:12 -06:00
Sam Mikes c33bf0e043 tools, harness: support new YAML frontmatter
parseTestRecord: add support for YAML frontmatter
parseTestRecord: initial unit test for test record parser
parseTestRecord: refactor for testing

factor old parsing; add YAML parsing

runner: support "includes" from YAML frontmatter

support frontmatter "includes" in python runner
use test.includes if present instead of scanning test code with regex

harness: factor individual functions out into files

tools: handle YAML errors

tolerate missing keys in dictionary (flags, includes)
report filename when empty frontmatter block
new option --list-includes to test262.py

harness: factor helper functions into separate files

sth: remove extra close-paren (syntax error)

test_common: TDD; failing parse of YAML

common: use parseTestRecord (YAML-aware)
2014-07-30 15:39:04 -07:00
Brian Terlson 0defa37385 Add path attribute. 2014-07-30 15:39:01 -07:00
Brian Terlson d4354d14d5 Normalize testcase format
This commit normalizes the test case format used across test262. It applies the following transformations:

* Convert to YAML for frontmatter
* Remove of trailing whitespace
* Replace /r/n with /n except in chapters 6 and 7.
* Copyright header always uses // comments
* new includes attribute replaces $INCLUDE
* No implicit assumptions about test environment other than $ERROR. Everything else appears in the include array. This includes "runTestCase" which is now included in a substantial number of tests.
2014-07-30 15:38:26 -07:00
Mathias Bynens a824f44727 Delete `Number.toInteger` test
`Number.toInteger` was removed from the ES6 draft. Fixes #62.
2014-07-30 23:17:00 +02:00
Brian Terlson 509472bb74 Fix suspect formatting in S15.2.3.1_A3 2014-07-24 15:29:21 -07:00
smikes a8c49d8b47 sth: correct syntax error in new fn isAsyncTest 2014-07-22 10:41:32 -06:00
Brian Terlson 66aab6025d Fix comment formatting for S22.1.2.3_T1.js 2014-07-21 15:47:50 -07:00
Brian Terlson 66366d6961 Merge pull request #49 from smikes/faster-python-runner
test262.py: only include helper scripts when needed
2014-07-21 14:32:03 -07:00
Sam Mikes d4a3479a1e test262.py: only include helper scripts when needed
test262.py: only supply async helper scripts when test is async
sth.js: factor out function isAsyncTest()
timer.js: improve workaround for async tests when Promise is defined but setTimeout is noot

timer.js emulates setTimeout using Promise by doing a busy loop that checks
if `timeout` milliseconds have elapsed.  Modified check to (timeLeft > 0) instead
of (!timeLeft) to prevent infinite loop when check does not happen to run
at precise millisecond timeout expires.

Because test262.py did not support the $INCLUDE directive, some helper
scripts were added to every test -- notably testIntl, timer, and donePrintHandle
Now that $INCLUDE is supported, these can be dropped, speeding overall test run time
2014-07-21 15:15:04 +01:00
Hank Yates cae283b563 Adding Array.prototype#fill tests 2014-07-18 15:17:33 -07:00
Brian Terlson 96c6efe0ef Merge pull request #43 from smikes/initial-Promise-tests
Initial promise tests
2014-07-18 11:22:48 -07:00
Brian Terlson 821e06e3f1 Merge pull request #44 from smikes/fix-packager
packager: make packager runnable
2014-07-18 11:21:52 -07:00
smikes 7032af4db8 packager: make packager runnable
packager.py cannot run due to syntax errors in a few script files

packagerConfig: use git instead of hg
test262: use pop instead of delete to avoid throw if property missing
S22.1.2.1_T*: fix docString header comment: s/b /**
S22.1.2.1_T3: fix docString header comment: s/b /**, fix end of docstring * / => */
2014-07-18 08:53:29 +01:00
Sam Mikes 7e07cc138d async,promises: initial tests of Promises
doneprintHandle.js: make $DONE accept any falsy argument as meaning 'pass'
PromiseHelper.js: checkSequence: new helper fn for async tests
.gitignore: port .hgignore to .gitignore syntax
test262.py: support $INCLUDE directive in python test runner

S25.4.4.1*: tests to cover Section 25.4.4.1, Promise.all( iterable )
A1.1: Promise.all is callable
A1.2: Promise.all expects 1 argument
A2.1: Promise.all([]) is a Promise
A2.2: Promise.all([]) is resolved immediately
A2.3: Promise.all([]) is resolved with a new empty array
A3.1: Promise.all expects an iterable argument
2014-07-18 05:37:08 +01:00
Matthew Meyers c451a33299 Adds tests for Array.prototype.find 2014-07-17 11:24:50 -07:00
ryanmurakami eb0ff48938 first batch of ES6 tests 2014-07-16 17:31:37 -07:00
Hank Yates 9244107ebf Adding Array#of tests 2014-07-16 17:03:52 -07:00
Hank Yates 7edb891fff Adding test for Array.from 2014-07-16 13:34:37 -07:00
Thomas Dahlstrom 2f5ec53aad Bug 596 - Coverage: 15.4.4.11 - identical elements and array holes 2014-07-16 12:56:46 -07:00
Brian Terlson 50696f69f6 Merge pull request #34 from niksurya/master
Adding Support for Async Tests
2014-07-16 12:10:56 -07:00
NikSurya 9aa6b917fa Adding Support for Async Tests 2014-07-15 13:47:59 -07:00
Brian Terlson b0c6fb0272 Merge pull request #2 from bterlson/localeCompare-fix
15.5.4.9_CE should check for locale-sensitive comparison
2014-07-10 14:59:57 -07:00
Brian Terlson 916a08bb9a Merge pull request #19 from JaimeLynSchatz/JaimeLynSchatz/fixBugz1533
change test 15.3.5.4_2-89 to call bound function before returning (Fixes bugzilla 1533)
2014-07-10 14:53:34 -07:00
Brian Terlson ee409f9d7d Merge pull request #14 from JaimeLynSchatz/JaimeLynSchatz/fixBugz1561
fix error msgs in test S12.6.1_A1 from Bugzilla issue 1561
2014-07-10 14:32:37 -07:00
Brian Terlson 5920e4e9c9 Merge pull request #6 from JaimeLynSchatz/JaimeLynSchatz/fixBugz1159
change assert to not assume a sorted list of arguments indices (Fixes bugzilla 1159)
2014-07-10 14:25:42 -07:00
Brian Terlson 019a62a863 Merge pull request #7 from JaimeLynSchatz/JaimeLynSchatz/fixBugz1552
fix point of view typos from Bugzilla 1552
2014-07-10 13:45:37 -07:00
Brian Terlson 4f15bf990b Merge pull request #1 from JaimeLynSchatz/JaimeLynSchatz/typofix
fix typo in 10.5 (fixes Bugzilla 1795)
2014-07-10 13:40:27 -07:00
Brian Terlson cb8b373444 Merge pull request #26 from muratsu/intl402-alert
Remove the alert call from test file
2014-07-10 13:39:14 -07:00
José Roberto Vidal 7b6ef7ba88 Fix Bugzilla 1450 2014-02-11 19:56:32 -05:00
Murat Sutunc 7697770a4c removed alert call from test file 2014-01-30 16:00:57 -08:00
JaimeLynSchatz 6b47f8c585 change test 15.3.5.4_2-89 to call bound function before returning for Bugzilla issue 1533 2014-01-26 17:08:20 -08:00
JaimeLynSchatz f9afc0c847 fix error msgs in test S12.6.1_A1 from Bugzilla issue 1561 2014-01-26 16:21:28 -08:00
JaimeLynSchatz a5fda120de fix point of view typos from Bugzilla 1552 2014-01-26 14:55:44 -08:00
JaimeLynSchatz d7446f811a change assert to not assume a sorted list of arguments indices for Bugzilla 1159 2014-01-26 14:25:38 -08:00
Brian Terlson 47a92dbb24 15.5.4.9_CE should check for locale-sensitive comparison 2014-01-26 13:24:35 -08:00
JaimeLynSchatz 239c4b721a fix typo in 10.5 from Bugzilla 2014-01-26 11:36:47 -08:00
Brent Baker ada9da5aa9 Bug 1128: use Date.prototype.getFullYear() in S15.2.2.1_A2_T5
Bug 1131: use String.prototype.charAt() in 15.12.3-11-26

Both of these changes were accidentaly dropped when I regenerated patches for these bugs. I have grepped the testsuite to ensure that the annexB functions only appear in the annexB section of the testsuite.

NOTE: Also regenerated the website based on this change. (Still waiting for permission to actually update the live website)
2013-06-13 10:07:45 -04:00
Prashanth Srinivasan 87605ba3b5 Bug 1472: Adding JUnit Compatible XML and Logfile support for Python test runner 2013-06-10 11:51:24 -04:00
Brent Baker 91bb82fb7e Bug 1548: Update results handling for Annex B section (r+bterlson) 2013-06-07 07:05:37 -04:00
Brent Baker 0d01ab1c7f Bug 1189: use "===" instead of "=" for S13.2_A4_T1 and T2 2013-06-07 06:57:45 -04:00
Brent Baker 592bf451b2 Bug 1131: Move String.prototype.substr into annexB 2013-06-05 08:00:44 -04:00