tests covering a given chapter, N, without delving into subsections. This in turn
broken the Results page which was by default displaying individual results for chapters
13(.0) and 14(.0). Fixed.
directory next*
SputnikConverter:
- ES5TestCase.cs
* Case of JSON-based property names was wrong. Fixed
* Use the tests' 'id' instead of 'path' as the GlobalScopeTests keys
* Added 'id' and 'path' as Global Scope test metadata. The correct/clean solution here is to simply use 'path'
as the key to GlobalScopeTests, but this refactoring needs to be undertaken later when we can convert the
'normal' test cases over to use 'path' as the key as well
* Turns out we cannot depend on the message received by window.onerror to have some form of "syntax"
contained within it. Instead, we'll just rely on the regular expression ".", matches any one character,
for the short term
- SputnikTestCase.cs
* Same case issue as for ES5TestCases.cs
* Don't trust the Sputnik metadata for the ES5 section name or even test case id to be correct. Instead,
generate this information from the file path of the test case
TestCasePackager.py:
- added a new global, GLOBAL_SCOPE_FILES, which is a list of JS files found directly under test\suite\*.js
which include metadata for so-called globally scoped tests. These files are imported directly by the HTML
test harness
test\suite\*:
- regenerated Sputnik tests based on new converter
default.html:
- import SputnikGlobalScope.js. Really TestCasePackager.py should generate the global scope imports to
default.html automatically...
website\resources\scripts\testcases\*:
- test cases have shuffled from existing *.json files into globalscope.json
- restructured the generated GlobalScope.js file such that the HTML test harness
can import multiple such files without worrying about one blowing aways another's
metadata
- inline with the last change, GlobalScope.js has been renamed to SputnikGlobalScope.js
and generated directly to test\suite\*
- the path and ID properties for most entries in GlobalScope.js were incorrect. That is,
they were based on the metadata contained in the original Sputnik tests and not the
new test262 directories and filenames
- generate global scope tests to test\suite\GlobalScope\* instead of
test\suite\sputnik_converted\GlobalScope\*. Going forward we need to move towards
integrating all test sources into test262 directories (e.g., "GlobalScope"===good;
"Sputnik\GlobalScope"===not-so-good)
- a few hacks to the sources (e.g., private vars to public vars) I'll fix in a refactoring
later. Just trying to get everything working first
Still needs a bit of integration work into the test harness.
A number of Sputnik tests were written outside the context of a chapter's sections. E.g.,
"Chapter 13" versus "Chapter 13, Section 1". We now fake a section number ("13.0" for the
example above) for such cases when generating test case metadata.
their test case objects. Created an automated tool, FixPathsAndIds.py,
to fix these.
TestCasePackager.py no longer throws when it encounters a non-*.js
file where it expects test cases. Instead, it simply ignores it.
rationale behind the addition of this file is it's meant to replace
TestCasePackager.ps1 outright and will be runnable on all major
operating systems. There's still a bit of cleanup work needed as
the Python port was hardly optimized for the language.
A few side effects of this change:
- it now takes under ten seconds to generate the test262 website (PS
version was taking over a minute)
- generated *.json files are ~5% smaller in size (PS was forcing
'\r\n' and the Python version preserves the tests' newlines whether
they're '\r\n' or '\n')
ES5 chapter. That is, ecma-262-toc.xml now has '.0' sections for all chapters
and I've manually modified (Sputnik Ch. 13 & 14) test case id's and paths
to utilize the '.0' suffix. Long term, SputnikConverter needs to perform
this transformation itself though.
to the following criteria:
1. If a test case directory directly contains *.js files at the root level, package
the entire directory as an XML regardless of how many *.js files are in
subdirectories
2. If the directory and subdirectories contain less than 1,000 test cases, package
the entire directory as an XML. Otherwise...
3. Perform steps 1 and 2 on each subdirectory of the directory
The net effect of this change is that no test case XMLs are larger than 1.5 megs any longer,
although the total number of test XMLs has jumped from about a dozen to ~75. This should
make running the test harness seem a little smoother across all browsers since we won't be
blocked downloading 4 meg files for several seconds any longer.
- 26 test case modifications. Most of these involved replacing instances 'window' with the
global object
- removal of 47 test cases. The bulk of these were dependent upon 'window' methods or properties
such as 'document' and 'alert'
- addition of 1,234 new test cases. These cover the following ES5.1 sections:
chapter07\7.6\7.6.1
chapter07\7.8\7.8.5
chapter11\11.8\11.8.2
chapter11\11.8\11.8.3
chapter12\12.6\12.6.3
chapter12\12.6\12.6.4
chapter12\12.7
chapter12\12.8
chapter12\12.9
chapter13\13.2
chapter15\15.10\15.10.2
chapter15\15.10\15.10.4
chapter15\15.10\15.10.6\15.10.6.2
chapter15\15.11\15.11.4\15.11.4.4
chapter15\15.1\15.1.1\15.1.1.1
chapter15\15.1\15.1.1\15.1.1.2
chapter15\15.1\15.1.1\15.1.1.3
chapter15\15.1\15.1.2\15.1.2.2
chapter15\15.1\15.1.2\15.1.2.3
chapter15\15.2\15.2.4\15.2.4.2
chapter15\15.3\15.3.4\15.3.4.5.1
chapter15\15.3\15.3.4\15.3.4.5.2
chapter15\15.4\15.4.4\15.4.4.10
chapter15\15.4\15.4.4\15.4.4.12
chapter15\15.4\15.4.4\15.4.4.4
chapter15\15.9\15.9.1\15.9.1.15
were actually unique across the test cases. This *was* needed prior to November as we weren't
running each test case in it's own private global environment. The situation now is that
we're running each test within it's own iframe => the modifications are no longer needed.
Few small improvements to SputnikConverter:
- App.config file locations have been fixed
- template files get pushed alongside generated tool binaries
- the root path for Sputnik conformance files is "Conformance", not "tests"
- allow the main exe to throw exceptions so they can be properly debugged with VS
"?sitedebug" to the end of the Test262 URL.
There's a mismatch between the Test262 test harness and IE Test Center's harness
which invalidated 15.4.4.17-5-1.js, 15.4.4.18-5-1.js, and 15.4.4.20-5-1.js on Test262.
Essentially what's going on is Test262 uses the 'window' object as 'this' for the test
case functions whereas IE Test Center uses the test case object housing the test case
function as this. This causes problems for test cases pertaining to scoping. I've gone
ahead and adjusted the Test262 tests to expect 'window' to be 'this', and will be updating
IE Test Center's harness to use the 'window' object as well so we don't run into this
problem again.
of the tests. Emit this to the activity bar
test\harness\sta.js: too many years of Python had me thinking JavaScript arrays have an append method:) Fixed.
Also, added a pickled representation of all test helper functions found in this file
test\harness\sth.js: detached most test helper functions from the iframe's document object (as globals) and
inject these into the actual test cases. It's a bit slower and not as elegant, but it is
cleaner from an ES5 purist perspective. Still need to move Sputnik helper functions into
sta.js
Extended Controller such that it now measures overall test execution time. Such a change
is very useful for measuring performance-impact changes such as the aforementioned
improvement
David Fugate