Commit Graph

245 Commits

Author SHA1 Message Date
André Bargull b218cf6dec Replace "imp" module with "importlib" in parseTestRecord
The "imp" module is deprecated in Python 3.4 and will be removed in
Python 3.12. Use it only if the current Python version is too old to use
the "importlib" module.
2023-11-15 09:15:47 -08:00
Liviu Rau be929de3ec Allow linter to use alternate features.txt file 2023-10-16 12:37:31 -07:00
Ms2ger 715dd1073b Only lint js and json files
CheckFileName is the only one that could arguably be valid for other files, but it doesn't seem worth the complexity to run it.
2023-10-06 13:45:37 +02:00
André Bargull a8bd958193
Don't require Inflect for linting files (#3839) 2023-08-28 16:22:19 +02:00
Philip Chimento a58ae41ea7 linter: Fix parsing features.txt
Previously, split() would split on whitespace, then the if-condition would
remove `#` - interpreting every word in every comment in the file as a
potential valid feature flag. We want splitlines() here.

Partial-line comments were inadvertently "supported" before, because of
this bug. Instead, support them explicitly by chopping off a `#`
character, anything after it, and any whitespace immediately preceding it.
2023-03-06 17:31:20 +01:00
Philip Chimento 6e1b737357 linter: Use same Python interpreter when executing script in subprocess
If a specific Python interpreter is used (e.g. by invoking the script with
"/path/to/python run.py") then we should use the same Python interpreter
to execute scripts in subprocesses, not the one from the environment.

The path to the running Python interpreter is given by sys.executable.
2023-03-06 17:31:20 +01:00
legendecas 2e7cdfbe18 Fix generator clean fail on .DS_Store 2022-05-05 17:03:58 -04:00
Csaba Osztrogonác 00f682e746 tools: enforce restriction on YAML negative key
The phase field must precede the type field for negative tests
to have a consistent style and be able to parse easier.
Related to the goal of https://github.com/tc39/test262/issues/1997

Added this check to the linting script and updated tests accordingly.
2022-01-10 13:50:38 -05:00
Csaba Osztrogonác bd1acb51a4 tools: enforce restriction on YAML includes key
Includes key should use flow notation to be able parsed easier
as suggested in https://github.com/tc39/test262/issues/1997

Added this check to the linting script and updated tests accordingly.
2022-01-05 17:27:58 -05:00
Mike Pennisi c4e54648c0 tools: enforce restriction on YAML
Some consumers have reported difficulty parsing an uncommon YAML
construction [1] [2] [3]. Extend the linter to help ensure that
construction is not used in future contributions.

[1] https://github.com/tc39/test262/issues/1997
[2] https://github.com/tc39/test262/pull/2505
[3] https://github.com/tc39/test262/issues/3171
2021-09-07 17:41:36 -04:00
Rick Waldron 304edd2296
chore: migrate $ERROR -> throw new Test262Error in tools/lint/test/fixtures/harness/propertyHelper.js (#3124) 2021-07-29 12:34:53 -07:00
Richard Gibson 1483cdee1a Skip parsing test files that don't need regeneration 2021-07-28 13:23:51 -04:00
Richard Gibson e07f577a9c Default to not regenerate test files that are newer than their case and template 2021-07-28 13:23:51 -04:00
Richard Gibson 1925fa1a3b Update Test to lazily parse generated content 2021-07-28 13:23:51 -04:00
Richard Gibson 172d75bacd Rename a variable for clarity 2021-07-28 13:23:51 -04:00
Richard Gibson 52ff4171ab Improve code clarity 2021-07-28 12:43:25 -04:00
Richard Gibson 20e06a147f Improve "Procedurally-generated tests" documentation 2021-07-28 12:43:25 -04:00
Richard Gibson bf0297c0a4 Fix typo 2021-07-19 13:28:31 -04:00
Richard Gibson a7fed497c6 Make it possible to express test case values as a sequence of code points 2021-07-16 09:49:17 -04:00
Richard Gibson d15066ec39
Cleanup generation code (#3041)
* Simplify find_cases

* Improve help text

* Improve YAML-capturing regex

* Use built-in dedenting

* Fix use of built-in dedenting

* Fix use of built-in dedenting for Python 3
2021-07-16 09:39:57 -04:00
jugglinmike 6c9d2222fb
lint: disallow duplicate values in "features" tag (#3010) 2021-06-24 13:38:18 -04:00
jugglinmike 64a8968246
lint: disallow extra fields in "negative" metadata (#3009) 2021-06-24 13:37:55 -04:00
Mike Pennisi 56ca8add7d Update project structure to support non-JS files
This change is in service of forthcoming tests for the "JSON modules"
language proposal [1]. Verifying the semantics of that proposal requires
modules whose source text is not valid ECMAScript; this change updates
the guidelines for contributing and interpreting tests so that such test
material can be handled consistently.

Differentiating JSON files with a distinct file name suffice will assist
consumers which require special handling of such files (e.g. web
browsers).

Change the pattern used to designate "fixture" files so that it may be
applied to files used for JSON modules.

Increment the project version number to alert consumers of this change
in interpreting instructions.

[1] https://github.com/tc39/proposal-json-modules
2021-05-28 20:02:59 -04:00
Mike Pennisi 1ebd34b53b CI: Do not report test failures as errors
Prior to this patch, the CircleCI continuous integration environment was
configured to report test failures in a negative light, displaying red
cross-marks and reporting that "some checks were not successful" in
commits and GitHub Pull Requests which included them.

The passing/failing status of tests does not influence their
desirability for Test262. (In practice, engines very commonly fail
newly-contributed tests.)

Although these conflicting interpretations does not technically
interfere with the maintainers' ability to merge new contributions, it
does create confusion for many contributors who interpreted the UI as a
rejection of their work.

In addition, this behavior made it impossible to distinguish between the
benign test failures and disruptive infrastructural problems (e.g. the
crashing of engines).

Reconfigure the continuous integration environment to accept passing and
failing tests equally, and to only report a problem when the Test262
project's testing infrastructure behaves unexpectedly.
2021-05-28 18:06:58 -04:00
Rick Waldron 1fd417b146 CI: show result of "git status --porcelain" 2020-09-25 13:16:06 -04:00
Rick Waldron 5b54058ad7 Enable configuration of templates for generated tests. Fixes gh-1049 2020-09-25 13:16:06 -04:00
Rick Waldron 99ceda42a4 Generation: support more complex globbing. Ref gh-1049 2020-09-25 13:16:06 -04:00
Rick Waldron 95d51c4a15 Generation: don't make negative parse SyntaxError tests that have flags: [async]. Fixes gh-1229 2020-09-10 16:22:34 -04:00
Rick Waldron 8c3a208ef6 Add globbing support to test generator 2020-09-10 11:14:18 -04:00
Leo Balter 0277ed4260 fixup! Rename usage of master to main 2020-07-15 15:47:15 -04:00
Leo Balter 2e127ec40e Rename usage of master to main
Ref #2699
2020-07-15 15:47:15 -04:00
Adrian Heine 271dad7c0a Read existing generated files with correct encoding
Otherwise we cannot parse the comments
2020-05-13 10:58:07 -04:00
André Bargull 033f31a8ed Add the language tag mappings python script
The file is derived from the same-named file for SpiderMonkey, therefore I've
kept the MPL license info.

The next commits use this script to generate language tag mappings data.
2020-04-09 09:20:55 -04:00
Gus Caplan 833825849d
add lint check for newline at end of file 2019-11-10 20:18:53 -08:00
Mike Pennisi a9111d7144 lint: add rule to verify use of harness files
Verify that every test file which references a harness file using the
"includes" directive also contains at least one reference to a value
defined in the harness file.

To support this check, extend each harness file with a list of values
which it defines.
2019-09-25 14:00:50 -04:00
Mike Pennisi bb5a6622d4 lint: Compute ergonomic name for tests
The command-line interface to Python's "unittest" module allows users to
run tests by name, e.g.

    $ test.py TestLinter.test_foo

If the tests include a period, they cannot be referenced in this way.
Rename the tests to omit the filename extension so that they may be
referenced from the command-line in this way.
2019-09-25 14:00:50 -04:00
Mike Pennisi f38748efc1 lint: Increase isolation between tests and project
Prior to this commit, the tests for the linter partially depended on
project files. This coupling risks churn in the tests: if the needs of
the project change (e.g. a harness file is modified), then the linter
tests would need to be updated. It also made the tests more difficult to
understand because the input was both larger and more complicated than
necessary to exercise the relevant functionality.

Execute the tests in the context of the fixture directory and introduce
minimal support files that serve the same purpose as the corresponding
project files.
2019-09-25 14:00:50 -04:00
Ms2ger 3ff5c0a115 Generation: Support Python 3 (#2288)
* Generation: Use Python 3-compatible imports.

* Generation: Use range() instead of xrange().

* Generation: Use list comprehensions instead of map().

* Generation: Explicitly use bytes in the Test class.

* Generation: Run unit tests on Python 3 as well.
2019-08-14 12:46:23 -04:00
Ms2ger f7896ada9e Lint: Use the print function rather than the statement. 2019-08-12 17:14:07 +02:00
Ms2ger f899f9524c Lint: Use items() rather than iteritems(). 2019-08-12 17:14:07 +02:00
Ms2ger 0b5ff0e151 Lint: Expect bytes rather than unicode from subprocess. 2019-08-12 17:14:07 +02:00
Ms2ger b78cb852e6 Lint: Rename _THROW_STMT_LEGACY to _THROW_STMT_RAW.
'Legacy' implies (to me) that it should no longer be allowed, but the lint
explicitly checks for its presence (in the 'raw' case).
2019-08-12 17:14:07 +02:00
Ms2ger 8946956f19 Lint: Make CheckHarnessFeatures output stable.
The iteration order for a set is not guaranteed to be stable, but this test expected it to be.
2019-08-12 17:14:07 +02:00
Ms2ger c4e5d69044 Tools: Update PyYAML version. (#2279)
Fixes #2278.
2019-08-12 09:49:14 -04:00
Ms2ger 8f5f6a1aa1 tools: update lint.py to work with pip 10+ some more (#2273)
main() and InstallRequirement were also removed from the public API.
2019-08-08 14:11:34 -04:00
Leo Balter 002d3484d6 Add tests for linter 2019-07-15 14:03:25 -04:00
Leo Balter 8b28871eca Update header regexp to prevent year ranges 2019-07-15 14:03:25 -04:00
Aleksey Shvayka a8d4eefa85 Skip hidden files in generator.py (#2183) 2019-05-29 10:58:53 -04:00
Leo Balter 3d4909b0fd
Use .. for git diffs in the CI routines (#2140)
the double dot gets only commits not in the left side of the double dot.

The right side assumes HEAD.

This will allow us to fetch the correct list of new or modified files in the current branch.
2019-04-25 17:56:55 -04:00
Leo Balter ebbe13e2b5
Add CircleCI (#2106)
Transfer Travis jobs to CircleCI
2019-03-20 21:04:41 -04:00