Commit Graph

333 Commits

Author SHA1 Message Date
Domenic Denicola 66a39c0423 Add an example of using assert.throws 2014-12-05 14:41:06 -05:00
Domenic Denicola fa4c337570 Add assert.throws
Closes #57.
2014-12-05 14:41:04 -05:00
Domenic Denicola 0d94ef1e98 Example of using assert.js 2014-12-01 16:04:33 -05:00
Domenic Denicola 4bb8c027e4 Add initial assert helpers
Closes #110
2014-12-01 15:44:33 -05:00
Brian Terlson bbeafbd3c6 Remove "best practice" tests
Sbp_A1_T1 to Sbp_A5_T2, Sbp_12, Sbp_7*: Functions are now valid in blocks, tests syntactically invalid anyway
Sbp_A10_T1 to Sbp_A10_T2: Tests are broken
2014-11-19 15:28:04 -08:00
Brian Terlson b752d2fdde Merge pull request #117 from smikes/issue-80/obsolete-frontmatter
remove obsolete attributes from frontmatter fixes #80
2014-11-12 12:26:05 -08:00
Brian Terlson 0149045593 Merge pull request #86 from smikes/pr/83
browser runner: check negative regex
2014-11-12 12:25:05 -08:00
smikes 2d62a6df1f remove email attribute
remove spec attribute
remove assertion, section, name
2014-11-12 13:16:42 -07:00
Brian Terlson 0caf4eec31 Merge pull request #100 from smikes/promise-tests
add initial set of tests for Promise
2014-11-12 11:29:11 -08:00
André Bargull 5c5dffce8b es6 compatability updates
See also: https://github.com/tc39/test262/pull/29
2014-11-11 17:34:23 -08:00
smikes f544773583 split "expectedThis" cases into strict/nonstrict branches 2014-11-10 19:21:54 -07:00
smikes 26787ab09b add tests for 24.4.4.1 - Promise.all
add more tests of Promise.all

additional Promise test

add tests of Promise.prototype

add some tests for Promise.race

additional Promise tests

add Promise.reject and Promise.resolve tests

correct test description

rename badly-named files

use current license and minor style cleanup

correct understanding of undefined as thisArgument

incorporate line notes

Is this enough to make a constructor in ES6?

more accurate use of resolved,fulfilled etc.
remove some redundant tests
add new tests

remove "constant array" unclear language

better description

Update S25.4.2.1_A3.2_T1.js

address dangling ()

changes per @anba line notes
 - if GetIterator is abrupt, the Promise.race / Promise.all should reject
 - if Promise.race is called with nonconforming constructor as 'this',
   TypeError should be thrown (cannot reject if exeption is thrown from
   NewPromiseCapability; no promise exists yet...)

correct description of "this" testing in callbacks

fix whitespace, formatting

remove tab
add "next-turn" checking to sequencers
2014-11-10 16:37:36 -07:00
Brian Terlson eb08d871c5 Merge pull request #69 from smikes/ES5-Array-concat-ownProperty
ES5.S15: new tests to clarify Array.prototype.concat
2014-10-24 10:48:00 -07:00
Brian Terlson 0c0d1d3d46 Merge pull request #101 from smikes/addl-console-negative-fixes
additional fixes suggested by @anba
2014-10-24 10:31:39 -07:00
Sam Mikes 5fbb0a3902 remove spurious flags: [path] 2014-10-24 15:24:57 +01:00
Sam Mikes 89d71f56b2 additional fixes suggested by @anba 2014-10-24 15:18:37 +01:00
Sam Mikes c650b1a9b0 ES5.S15: new tests to clarify Array.concat
Test of spec language from 15.4.4.4,
5.b.iii.3.b states 'Call the [[DefineOwnProeprty]] internal method of A'
which means that the result of Array.prototype.concat
should return `true` from `hasOwnProperty` for every property
created by `concat`, up to `length`

Set length to 3.
a[0] is set to 0 on a
a[1] is unset
a[2] is set to 2 via a prototype of a

add assertions for values of b[0], b[1], b[2]
add assertions for b.hasOwnProperty('2')

Check with Array.prototype and Object.prototype, separate tests

fix typo in assertion message

test was correct, but assertion message was incorrect
2014-08-19 22:01:46 +01:00
Sam Mikes 4debe08707 browser runner: check negative regex
1. use negative regex (instead of ".") to check iframeError
2. make Test262Error.prototype.toString() always include
   the string Test262Error (no spaces) so it matches /Test262Error/
3. modify check for supportsWindowOnerror - require that
   first argument (message) to onerror() contains the error name
   by checking for /Error:/.

Change (3) above forces IE11 onto the !supportsWindowOnerror path.
Test262 tests are run inside an eval() on IE11, and errors are
caught and explicitly have toString() called.  Without this,
IE11 passes only the `message` property to onerror(), and regexes
that expect to match error name fail.

sth: revert to simple `onerror` checking

gs: refactor function `$DONE`

refactor logic tree
add support for async test failures

ed.js: crude support for error typing

S8.7.1_A2: cache result of delete

`delete` has a side-effect and its
return value depends on this; cache result of
delete so when reporting result, we are always
reporting the result that made us fail, not the
result of a new delete
[pedantic]

ed: explicitly pass cooked error to $DONE

gs: only let $DONE be called once
2014-08-13 12:24:12 +01:00
Sam Mikes 6b7e5d7747 bestPractice: supply error names to negative
pilot directory for replacing flags: [negative] with
negative: errorname

ch07-7.2,7.3: add error names
ch07-rest: insert error names rest of ch07
ch08: add expected error name
ch11: add error name to negative
ch12: put error name in negative

ch13: add error name to negative
only one test, did it manually

ch15: add error name to negative:

ch07: add error name to negative

these tests used flags: \n - negative and so
were not caught by the earlier naive grep for flags: [negative]

--- changes suggested by @anba

bestPractice - remove added NotEarlyError
ch07,ch11 - expect ReferenceError (req'd by ES6)
correct test descriptions to expect ReferenceError only
enforce parsing as block statement by adding `;`
remove needless include of $FAIL.js

effectively, a rebase of all changes onto master

    Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice

    Merge branch 'saved-bestPractice-negative' into negative-addErrorName-work

    Conflicts:
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js
    	test/suite/ch12/12.1/S12.1_A4_T1.js
    	test/suite/ch12/12.5/S12.5_A2.js

    incorporate changes made on PR branch

    re-remove added NotEarlyError

    Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice

    Conflicts:
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js
    	test/suite/ch12/12.1/S12.1_A4_T1.js
    	test/suite/ch12/12.5/S12.5_A2.js

    ch15: correct negative regexes

    ch10: avoid use of not-NotEarlyError regex

    ch14 error regexps

    ch13 error regexps

    ch12: negative error matching

    Merge branch 'console-runner-checkError' into negative-addErrorName-work

    test262.py: check negative tests with regex

    implement checking of negative tests =>

    negative: SyntaxError

    means that /SyntaxError/ must match stderr
    or test is reported as failure

    Fixes #78

    expect real errors

    bestPractice: supply error names to negative

    pilot directory for replacing flags: [negative] with
    negative: errorname

    ch07-7.2,7.3: add error names
    ch07-rest: insert error names rest of ch07
    ch08: add expected error name
    ch11: add error name to negative
    ch12: put error name in negative

    ch13: add error name to negative
    only one test, did it manually

    ch15: add error name to negative:

    ch07: add error name to negative

    these tests used flags: \n - negative and so
    were not caught by the earlier naive grep for flags: [negative]

    --- changes suggested by @anba

    bestPractice - remove added NotEarlyError
    ch07,ch11 - expect ReferenceError (req'd by ES6)
    correct test descriptions to expect ReferenceError only
    enforce parsing as block statement by adding `;`
    remove needless include of $FAIL.js

    Merge branch 'negative-addErrorName-work' into negative-addErrorName-bestPractice

    resolved Conflicts:
    	test/suite/ch07/7.9/S7.9_A5.7_T1.js
    	test/suite/ch11/11.13/11.13.1/S11.13.1_A2.1_T3.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T1.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T10.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T11.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T2.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T3.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T4.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T5.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T6.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T7.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T8.js
    	test/suite/ch11/11.13/11.13.2/S11.13.2_A2.2_T9.js
    	test/suite/ch11/11.3/11.3.1/S11.3.1_A2.1_T3.js
    	test/suite/ch11/11.3/11.3.2/S11.3.2_A2.1_T3.js
    	test/suite/ch11/11.4/11.4.4/S11.4.4_A2.1_T3.js
    	test/suite/ch11/11.4/11.4.5/S11.4.5_A2.1_T3.js
    	test/suite/ch12/12.5/S12.5_A2.js
    	test/suite/ch12/12.6/12.6.3/S12.6.3_A4_T2.js

    negative: second half of @anba notes

    correct test descriptions to expect ReferenceError only
    enforce parse as block statement by adding `;`
    remove needless include of $FAIL.js

    ch12: specify type

    negative: @anba fixes

    bestPractice - remove added NotEarlyError
    ch07,ch11 - expect ReferenceError (req'd by ES6)

    bestPractice: supply error names to negative

    pilot directory for replacing flags: [negative] with
    negative: errorname

    ch07-7.2,7.3: add error names
    ch07-rest: insert error names rest of ch07
    ch08: add expected error name
    ch11: add error name to negative
    ch12: put error name in negative

    ch13: add error name to negative
    only one test, did it manually

    ch15: add error name to negative:

    ch07: add error name to negative

    these tests used flags: \n - negative and so
    were not caught by the earlier naive grep for flags: [negative]

    expect a SyntaxError

    ch07: add error name to negative

    these tests used flags: \n - negative and so
    were not caught by the earlier naive grep for flags: [negative]

    ch15: add error name to negative:

    ch13: add error name to negative

    only one test, did it manually

    ch12: put error name in negative

    ch11: add error name to negative

    ch08: add expected error name
    ch07-rest: insert error names rest of ch07

    ch07-7.2,7.3: add error names

    negative-errorname: pilot

    pilot directory for replacing flags: [negative] with
    negative: errorname

    bestPractice: supply error names to negative

bestPractice: supply error names to negative

pilot directory for replacing flags: [negative] with
negative: errorname

ch07-7.2,7.3: add error names
ch07-rest: insert error names rest of ch07
ch08: add expected error name
ch11: add error name to negative
ch12: put error name in negative

ch13: add error name to negative
only one test, did it manually

ch15: add error name to negative:

ch07: add error name to negative

these tests used flags: \n - negative and so
were not caught by the earlier naive grep for flags: [negative]

bestPractice: supply error names to negative

negative-errorname: pilot

pilot directory for replacing flags: [negative] with
negative: errorname

ch07-7.2,7.3: add error names

ch07-rest: insert error names rest of ch07

ch08: add expected error name

ch11: add error name to negative

ch12: put error name in negative

ch13: add error name to negative

only one test, did it manually

ch15: add error name to negative:

ch07: add error name to negative

these tests used flags: \n - negative and so
were not caught by the earlier naive grep for flags: [negative]

expect a SyntaxError

negative: @anba fixes

bestPractice - remove added NotEarlyError
ch07,ch11 - expect ReferenceError (req'd by ES6)

ch12: specify type

negative: second half of @anba notes

correct test descriptions to expect ReferenceError only
enforce parse as block statement by adding `;`
remove needless include of $FAIL.js

expect real errors

ch12: negative error matching

ch13 error regexps

ch14 error regexps

add "description" header back

remove now-unused includes

remove needless include
2014-08-13 00:19:11 +01:00
Brian Terlson a794e0eb54 Merge pull request #76 from anba/issue-61/wrong-test-assertions
Correct test assertion (Fixes issue#61)
2014-08-05 23:52:06 -07:00
Brian Terlson 7f5310b4d5 Merge pull request #77 from anba/issue-63/correct-array-fill-test
Correct Array.prototype.fill tests (Fixes #63)
2014-08-05 23:51:28 -07:00
André Bargull f0ac90a5fd Move tests to Math.clz32 (Fixes #41)
Number.prototype.clz() was renamed to Math.clz32(). Also removed superfluous 'spec' entry in test descriptor.
2014-08-04 20:57:02 +02:00
André Bargull e256a76e9d Correct Array.prototype.fill tests (Fixes #63)
The expected results weren't correct.
2014-08-04 19:44:22 +02:00
André Bargull 28649e8772 Correct test assertion (Fixes issue#61)
Changed Number.isNaN_Boolean.js, String.prototype.endsWith_Fail.js and String.prototype.endsWith_Fail_2.js to test for the correct result. Also removed implicit coercion in a couple of other test files.
2014-08-04 19:10:41 +02:00
Sam Mikes 4205a1da64 harness: remove unused code
sta.js:
slight change to Test262Error() semantics; message property now always set (default "")
make $ERROR a var
set $ERROR to function $ERROR so it can be overridden if needed
remove 2009 copyright in favor of 2012 copyright
REVERTED: remove never-used fn testFailed
testFailed actually used by $FAIL
remove obsolete fn $INCLUDE

ed.js:
remove commented-out obsolete code

test262.py:
remove always-included harness file that provides no functions used by any extant test
add comment line to nonstrict

cth.js: define `print` for node, cscript

use cth (console test harness) to define functions
for console runner.  V8, Spidermonkey (js) and JavaScriptCore (jsc)
provide a function `print`.  Provide a default `print` for node
and cscript

set print_handle to 'print' by default (can still override)
for cscript: wrap tests in try/catch so we get syntax errors
2014-08-03 00:33:08 +01:00
Sam Mikes bd80bf7dfc es6/Math: use Number.isNaN
use number.isNaN to test for NaN
pointed out by @anba in #60
2014-07-31 13:53:12 -06:00
Sam Mikes c33bf0e043 tools, harness: support new YAML frontmatter
parseTestRecord: add support for YAML frontmatter
parseTestRecord: initial unit test for test record parser
parseTestRecord: refactor for testing

factor old parsing; add YAML parsing

runner: support "includes" from YAML frontmatter

support frontmatter "includes" in python runner
use test.includes if present instead of scanning test code with regex

harness: factor individual functions out into files

tools: handle YAML errors

tolerate missing keys in dictionary (flags, includes)
report filename when empty frontmatter block
new option --list-includes to test262.py

harness: factor helper functions into separate files

sth: remove extra close-paren (syntax error)

test_common: TDD; failing parse of YAML

common: use parseTestRecord (YAML-aware)
2014-07-30 15:39:04 -07:00
Brian Terlson 0defa37385 Add path attribute. 2014-07-30 15:39:01 -07:00
Brian Terlson d4354d14d5 Normalize testcase format
This commit normalizes the test case format used across test262. It applies the following transformations:

* Convert to YAML for frontmatter
* Remove of trailing whitespace
* Replace /r/n with /n except in chapters 6 and 7.
* Copyright header always uses // comments
* new includes attribute replaces $INCLUDE
* No implicit assumptions about test environment other than $ERROR. Everything else appears in the include array. This includes "runTestCase" which is now included in a substantial number of tests.
2014-07-30 15:38:26 -07:00
Mathias Bynens a824f44727 Delete `Number.toInteger` test
`Number.toInteger` was removed from the ES6 draft. Fixes #62.
2014-07-30 23:17:00 +02:00
Brian Terlson 509472bb74 Fix suspect formatting in S15.2.3.1_A3 2014-07-24 15:29:21 -07:00
smikes a8c49d8b47 sth: correct syntax error in new fn isAsyncTest 2014-07-22 10:41:32 -06:00
Brian Terlson 66aab6025d Fix comment formatting for S22.1.2.3_T1.js 2014-07-21 15:47:50 -07:00
Brian Terlson 66366d6961 Merge pull request #49 from smikes/faster-python-runner
test262.py: only include helper scripts when needed
2014-07-21 14:32:03 -07:00
Sam Mikes d4a3479a1e test262.py: only include helper scripts when needed
test262.py: only supply async helper scripts when test is async
sth.js: factor out function isAsyncTest()
timer.js: improve workaround for async tests when Promise is defined but setTimeout is noot

timer.js emulates setTimeout using Promise by doing a busy loop that checks
if `timeout` milliseconds have elapsed.  Modified check to (timeLeft > 0) instead
of (!timeLeft) to prevent infinite loop when check does not happen to run
at precise millisecond timeout expires.

Because test262.py did not support the $INCLUDE directive, some helper
scripts were added to every test -- notably testIntl, timer, and donePrintHandle
Now that $INCLUDE is supported, these can be dropped, speeding overall test run time
2014-07-21 15:15:04 +01:00
Hank Yates cae283b563 Adding Array.prototype#fill tests 2014-07-18 15:17:33 -07:00
Brian Terlson 96c6efe0ef Merge pull request #43 from smikes/initial-Promise-tests
Initial promise tests
2014-07-18 11:22:48 -07:00
Brian Terlson 821e06e3f1 Merge pull request #44 from smikes/fix-packager
packager: make packager runnable
2014-07-18 11:21:52 -07:00
smikes 7032af4db8 packager: make packager runnable
packager.py cannot run due to syntax errors in a few script files

packagerConfig: use git instead of hg
test262: use pop instead of delete to avoid throw if property missing
S22.1.2.1_T*: fix docString header comment: s/b /**
S22.1.2.1_T3: fix docString header comment: s/b /**, fix end of docstring * / => */
2014-07-18 08:53:29 +01:00
Sam Mikes 7e07cc138d async,promises: initial tests of Promises
doneprintHandle.js: make $DONE accept any falsy argument as meaning 'pass'
PromiseHelper.js: checkSequence: new helper fn for async tests
.gitignore: port .hgignore to .gitignore syntax
test262.py: support $INCLUDE directive in python test runner

S25.4.4.1*: tests to cover Section 25.4.4.1, Promise.all( iterable )
A1.1: Promise.all is callable
A1.2: Promise.all expects 1 argument
A2.1: Promise.all([]) is a Promise
A2.2: Promise.all([]) is resolved immediately
A2.3: Promise.all([]) is resolved with a new empty array
A3.1: Promise.all expects an iterable argument
2014-07-18 05:37:08 +01:00
Matthew Meyers c451a33299 Adds tests for Array.prototype.find 2014-07-17 11:24:50 -07:00
ryanmurakami eb0ff48938 first batch of ES6 tests 2014-07-16 17:31:37 -07:00
Hank Yates 9244107ebf Adding Array#of tests 2014-07-16 17:03:52 -07:00
Hank Yates 7edb891fff Adding test for Array.from 2014-07-16 13:34:37 -07:00
Thomas Dahlstrom 2f5ec53aad Bug 596 - Coverage: 15.4.4.11 - identical elements and array holes 2014-07-16 12:56:46 -07:00
Brian Terlson 50696f69f6 Merge pull request #34 from niksurya/master
Adding Support for Async Tests
2014-07-16 12:10:56 -07:00
NikSurya 9aa6b917fa Adding Support for Async Tests 2014-07-15 13:47:59 -07:00
Brian Terlson b0c6fb0272 Merge pull request #2 from bterlson/localeCompare-fix
15.5.4.9_CE should check for locale-sensitive comparison
2014-07-10 14:59:57 -07:00
Brian Terlson 916a08bb9a Merge pull request #19 from JaimeLynSchatz/JaimeLynSchatz/fixBugz1533
change test 15.3.5.4_2-89 to call bound function before returning (Fixes bugzilla 1533)
2014-07-10 14:53:34 -07:00
Brian Terlson ee409f9d7d Merge pull request #14 from JaimeLynSchatz/JaimeLynSchatz/fixBugz1561
fix error msgs in test S12.6.1_A1 from Bugzilla issue 1561
2014-07-10 14:32:37 -07:00