Commit Graph

268 Commits

Author SHA1 Message Date
Darren Tucker fb2f3a61bf Move unset to before we set anything. 2022-07-12 12:54:24 +10:00
Darren Tucker c483a5c0fb Test against openssl-3.0.5. 2022-07-06 11:52:54 +10:00
Darren Tucker 669a56bcfe Update sanitizer test targets:
- remove clang-sanitize-memory for now.  It takes so long that the test
   times out.
 - add gcc sanitize-address and sanitize-undefined test targets.
2022-07-05 18:35:53 +10:00
Darren Tucker 48cc68b691 Add GCC address sanitizer build/test. 2022-07-05 18:30:10 +10:00
Darren Tucker 55c60bdd39 Move sanitizer logs into regress for collection. 2022-07-05 18:30:10 +10:00
Darren Tucker 7394ed80c4 Add clang sanitizer tests. 2022-07-03 22:53:44 +10:00
Darren Tucker c71a1442d0 Update OpenSSL tests to the most recent releases. 2022-06-29 18:28:47 +10:00
Darren Tucker 1781f507c1 Test against OpenSSL 1.1.1o and 3.0.3. 2022-05-27 18:19:48 +10:00
Darren Tucker c53906e0c5 Test against LibreSSL 3.5.3. 2022-05-27 18:18:31 +10:00
Darren Tucker f38cf74f20 Also retest OpenBSD upstream on .yml changes. 2022-05-06 14:50:18 +10:00
Darren Tucker 217b518e0f Add ubsan minimal testcase on OpenBSD.
As suggested by djm@.
2022-05-06 14:39:34 +10:00
Darren Tucker 4835544d2d Add Mac OS X 12 test target. 2022-04-30 11:00:02 +10:00
Darren Tucker 97a6a8b8c1 Only run tests when source files change.
Also run tests on changes to V_9_0 branch.
2022-04-29 18:34:38 +10:00
Darren Tucker 7bf2eb958f Add debian-riscv64 test target. 2022-04-26 23:30:59 +10:00
Darren Tucker 3913c93552 Update OpenSSL and LibreSSL versions in tests. 2022-04-25 17:21:24 +10:00
Darren Tucker d2b888762b Retire fbsd6 test VM.
It's long since out of support, relatively slow (it's i686) and the
compiler has trouble with PIE.
2022-04-16 14:31:13 +10:00
Darren Tucker 3741df98ff Disable security key on fbsd6 test host. 2022-04-04 23:52:11 +10:00
Darren Tucker 32c12236f2 Specify TEST_SHELL=bash on AIX.
The system shells cause the agent-restrict test to fail due to some
quoting so explicitly specify bash until we can get configure to
autmatically work around that.
2022-04-04 15:16:51 +10:00
Darren Tucker 6c49eb5fab Only run regression tests on slow VMs. 2022-04-01 23:21:40 +11:00
Darren Tucker f67e479039 Increase test timeout to allow slow VMs to finish 2022-04-01 23:21:06 +11:00
Darren Tucker 5b054d7640 Skip slow tests on (very) slow test targets. 2022-04-01 13:16:47 +11:00
Darren Tucker e937514920 Add AIX 5.1 test target. 2022-03-28 17:51:03 +11:00
Darren Tucker 4bbe815ba9 Drop leading "v" from release version identifier.
It's present in the git tags but not in the release tarball names.
Also drop extra "/" from URL path.
2022-03-26 22:01:31 +11:00
Darren Tucker f5cdd3b3c2 Use tarballs when testing LibreSSL releases.
This means they'll still work when the combination of -portable and
openbsd github repos no longer match.
2022-03-26 16:28:04 +11:00
Darren Tucker d23efe4b12 Add OpenWRT mips and mipsel test targets. 2022-03-26 08:13:46 +11:00
Darren Tucker f2ff669347 No unused param warnings for clang-12 and gcc-11.
These have too many false positives in -Werror tests on the github CI
since we often provide empty stub functions for functionality not needed
for particular configurations.
2022-02-28 22:21:36 +11:00
Darren Tucker 96558ecd87 Add debian-i386 test target. 2022-02-26 14:10:41 +11:00
Darren Tucker 3016ba4703 Add tests for latest releases of {Libre,Open}SSL. 2022-02-25 11:39:34 +11:00
Darren Tucker 6520c488de Add Alpine Linux test VM. 2022-02-22 13:08:59 +11:00
Damien Miller 7c0d4ce911 disable agent-restrict test on minix3
Minix seems to have a platform-wide limit on the number of
select(2) syscalls that can be concurrently issued. This test
seems to exceed this limit.

Refer to:

https://github.com/Stichting-MINIX-Research-Foundation/minix/blob/R3.3.0/minix/servers/vfs/select.c#L114
https://github.com/Stichting-MINIX-Research-Foundation/minix/blob/R3.3.0/minix/servers/vfs/select.c#L30-L31
2022-02-22 11:14:51 +11:00
Darren Tucker 63bf4f49ed Add test configs for MUSL C library. 2022-02-18 12:12:21 +11:00
Darren Tucker 41417dbda9 Comment hurd test, the VM is currently broken. 2022-02-17 22:05:29 +11:00
Darren Tucker 766176cfdb Add cygwin-release test config.
This tests the flags used to build the cygwin release binaries.
2022-02-12 10:24:56 +11:00
Darren Tucker ea13fc830f Stop testing OpenBSD HEAD on 6.9 and 7.0.
HEAD is not guaranteed to work on previous stable branches, and at the
moment is broken due to libfido API changes.
2022-02-11 13:42:36 +11:00
Darren Tucker 3fe6800b60 Move more tests out of valgrind-1 runner. 2022-01-20 00:49:57 +11:00
Darren Tucker 6c435bd499 Stop on first test failure to minimize logs. 2022-01-13 09:01:05 +11:00
Darren Tucker 613a6545fc libhardended_malloc.so moved into out dir. 2022-01-11 20:56:01 +11:00
Darren Tucker bbf285164d Always save config.h as build artifact.
Should allow better comparison between failing and succeeding test
platforms.
2022-01-05 11:48:58 +11:00
Darren Tucker 03bd4ed0db Add OpenBSD 7.0 target. Retire 6.8. 2022-01-05 11:48:58 +11:00
Darren Tucker 3dafd3fe22 Increase timeout for test step. 2021-12-11 08:43:41 +11:00
Darren Tucker 5aefb05cd5 Update the list of tests that don't work on Minix.
While there, remove CC (configure will now find clang) and make the test
list easier to update via cut and paste.
2021-12-10 10:27:27 +11:00
Darren Tucker e665ed2d0c Switch from LibreSSL 3.4.0 to 3.4.1.
The LibreSSL 3.4.0 release has an OPENBSD_BRANCH that points to
"master" and that branch no longer has the files LibreSSL expects
and thus it will no longer build, breaking the test.
2021-11-12 22:57:51 +11:00
Darren Tucker d575cf4489 kitchensink test target now needs krb5. 2021-10-22 23:27:41 +11:00
Darren Tucker 4ae39cada2 Test both MIT KRB5 and Heimdal. 2021-10-22 22:54:33 +11:00
Darren Tucker a07664646b Source configs script so setup_ci can use settings 2021-10-22 14:00:05 +11:00
Darren Tucker 34df52c201 Install libedit and pam based on config flags. 2021-10-22 09:42:14 +11:00
Darren Tucker 8c626cc563 Don't use 'here string", it's not POSIX. 2021-10-21 16:53:39 +11:00
Darren Tucker 086a4b5977 Remove -Werror from compiler package to install. 2021-10-21 15:33:27 +11:00
Darren Tucker 5a7a468750 Build with -Werror on most recent gcc and clang. 2021-10-21 15:18:34 +11:00
Darren Tucker d4b38144c0 Add tcmalloc test target. 2021-10-12 23:01:40 +11:00
Darren Tucker 482f73be10 Include relevant env vars on command line.
Makes it easier to reproduce a build by cut/pasting the configure line.
2021-10-07 15:55:04 +11:00
Darren Tucker 52d4232b49 Disable security key on minix3.
The test doesn't work so disable.
2021-10-06 18:14:37 +11:00
Darren Tucker 8f4be526a3 Disable security key on NetBSD4 test.
sk-dummy used for the security key test includes both sha2.h and OpenSSL
causing the definitions conflict so disable security key support on this
platform.
2021-10-06 15:40:58 +11:00
Darren Tucker a3c6375555 Use backticks instead of $(..) for portability.
Older shells (eg /bin/sh on Solaris 10) don't support $() syntax.
2021-09-29 19:30:59 +10:00
Darren Tucker 958aaa0387 Skip file-based tests by default on Mac OS.
The file-based tests need OpenSSL so skip them.
2021-09-29 18:53:32 +10:00
Darren Tucker 55c8bdf6e9 Build without OpenSSL on Mac OS.
Modern versions don't ship enough libcrypto to build against.
2021-09-29 18:42:47 +10:00
Darren Tucker 5e6d28b787 Split c89 test openssl setting out. 2021-09-29 17:48:09 +10:00
Darren Tucker c8d92d3d4f Add make clean step to tests. 2021-09-29 13:28:56 +10:00
Darren Tucker 360fb41ef8 Test all available clang and gcc versions. 2021-09-29 12:05:50 +10:00
David Manouchehri f6a660e5bf Don't prompt for yes/no questions. 2021-09-24 07:52:04 +10:00
Darren Tucker 8a7a06ee50 Test against LibreSSL 3.2.6, 3.3.4, 3.4.0. 2021-09-17 13:03:31 +10:00
Darren Tucker c4565e69ff Really fix test on OpenSSL 1.1.1 stable. 2021-09-08 21:09:49 +10:00
Darren Tucker 79f1bb5f56 Correct OpenSSL 1.1.1 stable identifier. 2021-09-08 18:51:39 +10:00
Darren Tucker a3e92a6794 Tests for OpenSSL 3.0.0 release & 1.1.1 branch. 2021-09-08 13:45:10 +10:00
Darren Tucker 65bb011113 Test against OpenSSL 3 branch as well as dev.
Now that OpenSSL development has moved to 3.1, test against the most
recent version of the openssl-3.0 branch too.
2021-09-05 19:43:13 +10:00
Darren Tucker 6a24567a29 Improve github test driver script.
- use a trap to always output any failed regress logs (since the script
   sets -e, the existing log output is never invoked).
 - pass LTESTS and SKIP_LTESTS when re-running with sshd options (eg.
   UsePAM).
2021-08-16 14:42:48 +10:00
Darren Tucker b467cf1370 Remove deprecated ubuntu-16.04 test targets.
Github has deprecated ubuntu-16.04 and it will be removed on 20
September.
2021-08-16 11:32:23 +10:00
Darren Tucker 20e6eefcdf Skip agent ptrace test on hurd. 2021-08-15 23:25:26 +10:00
Darren Tucker 7c9115bbbf Add hurd test target. 2021-08-15 19:37:22 +10:00
Darren Tucker 7909a566f6 Skip scp3 tests on all dfly58 and 60 configs. 2021-08-15 12:45:10 +10:00
Darren Tucker e50635640f Test OpenSSH from OpenBSD head on 6.8 and 6.9. 2021-08-13 13:21:00 +10:00
Darren Tucker e0ba38861c Skip scp3 test on dragonfly 58 and 60.
The tests hang, so skip until we figure them out.
2021-08-13 13:00:14 +10:00
Darren Tucker a1f78e08bd Move SUDO to "make test" command line.
Environment variables don't get passed by vmrun, so move to command
line.
2021-07-26 12:45:30 +10:00
Darren Tucker 02e624273b Set SUDO for tests and cleanup. 2021-07-25 23:26:36 +10:00
Darren Tucker 460ae5d930 Pass OPENSSL=no to make tests too. 2021-07-25 22:37:55 +10:00
Darren Tucker b6673b1d2e Test OpenBSD upstream with and without OpenSSL. 2021-07-24 14:20:12 +10:00
Darren Tucker 8aa2f9aeb5 Make whitespace consistent. 2021-07-14 11:27:24 +10:00
Darren Tucker 4f4297ee9b Add ARM64 Linux self-hosted runner. 2021-07-14 11:27:24 +10:00
Darren Tucker 2c9e4b319f Disable rocky84 to figure out why agent test fails 2021-07-03 19:17:31 +10:00
Darren Tucker f93fdc4de1 Add rocky84 test target. 2021-06-28 13:06:37 +10:00
Darren Tucker da9f9acaac Save logs on failure for upstream test 2021-06-07 19:19:23 +10:00
Darren Tucker 76883c6016 Add obsdsnap-i386 upstream test target. 2021-06-07 14:38:17 +10:00
Darren Tucker ecc186e46e Retire fbsd7 test target.
It's the slowest of the selfhosted targets (since it's 32bit but has
most of the crypto algos). We still have coverage for 32bit i386.
2021-06-02 13:54:11 +10:00
Darren Tucker 1db69d1b65 Add dfly60 target. 2021-06-02 11:22:12 +10:00
Darren Tucker 7a3a1dd2c7 Put minix3 config in the host-specific block. 2021-06-01 09:38:26 +10:00
Darren Tucker 2063af7142 Add minix3 test target. 2021-05-27 21:13:38 +10:00
Darren Tucker 078a0e60c9 Rename README.md to ci-status.md.
The original intent was to provide a status page for the CIs configured
in that directory, but it had the side effect of replacing the top-level
README.md.
2021-05-25 11:40:47 +10:00
Darren Tucker 8e32e97e78 Add obsd69 test target. 2021-05-01 20:07:47 +10:00
Darren Tucker 70a8dc138a Add status badges for Actions-based tests. 2021-04-29 10:59:54 +10:00
Darren Tucker 40b59024cc Add obsdsnap (OpenBSD snapshot) test target. 2021-04-28 12:22:11 +10:00
Darren Tucker e627067ec8 Add test building upstream OpenBSD source. 2021-04-28 11:43:54 +10:00
Darren Tucker 1b8108ebd1 Test against OpenSSL 1.1.0h instead of 1.1.0g.
1.1.0g requires a perl glob module that's not installed by default.
2021-04-27 14:22:20 +10:00
Darren Tucker 9bc20efd39 Use the default VM type for libcrypto ver tests. 2021-04-27 12:37:59 +10:00
Darren Tucker 9f79e80dc4 Always build OpenSSL shared.
This is the default for current versions but we need it to test against
earlier versions.
2021-04-27 12:32:46 +10:00
Darren Tucker b3cc9fbdff Fix custom OpenSSL tests.
Check out specified OpenSSL version.  Install custom libcrypto where
configure expects to find it.  Remove unneeded OpenSSL config time
options.  Older OpenSSL versions were not make -j safe so remove it.
2021-04-27 11:44:29 +10:00
Darren Tucker 7753260987 Export CC and CFLAGS for c89 test. 2021-04-26 17:20:23 +10:00
Darren Tucker 33f62dfbe8 Add c89 here too. 2021-04-26 17:13:44 +10:00
Darren Tucker da9d59f526 Add test against OpenSSL w/out ECC. 2021-04-26 15:34:23 +10:00
Darren Tucker 29e194a752 Ensure we can still build with C89. 2021-04-26 14:50:58 +10:00
Darren Tucker a38016d369 Interop test agains PuTTY. 2021-04-26 14:50:58 +10:00
Darren Tucker 095b0307a7 Support testing against arbitary libcrytpo vers.
Add tests against various LibreSSL and OpenSSL versions.
2021-04-26 14:50:44 +10:00
Darren Tucker b16082aa11 Add fbsd10 test target. 2021-04-26 13:35:44 +10:00
Darren Tucker 2c805f16b2 Disable compiler hardening on nbsd4.
The system compiler supports -fstack-protector-all, but using it will
result in an internal compiler error on some files.
2021-04-25 14:15:02 +10:00
Darren Tucker 6a5d393056 Add nbsd3, nbsd4 and nbsd9 test targets. 2021-04-25 13:01:34 +10:00
Darren Tucker d1aed05bd2 Comment out nbsd2 test target for now. 2021-04-24 22:03:46 +10:00
Darren Tucker 3737c9f66e Replace "==" (a bashism) with "=". 2021-04-23 19:49:46 +10:00
Darren Tucker a116b6f5be Add nbsd2 test target. 2021-04-23 16:34:48 +10:00
Darren Tucker 196bf2a9bb Add obsd68 test target. 2021-04-23 14:54:10 +10:00
Darren Tucker e3ba6574ed Remove dependency on bash. 2021-04-23 14:53:32 +10:00
Darren Tucker db1f9ab8fe Add obsd67 test target. 2021-04-23 14:41:13 +10:00
Darren Tucker c039a6bf79 Re-add macos-11.0 test target. 2021-04-23 11:08:23 +10:00
Darren Tucker a6db3a47b5 Add openindiana test target. 2021-04-23 10:28:28 +10:00
Darren Tucker 3fe7e73b02 Test krb5 on Solaris 11 too. 2021-04-23 10:26:35 +10:00
Darren Tucker f57fbfe5eb Don't always set SUDO.
Rely on sourcing configs to set as appropriate.
2021-04-22 22:29:33 +10:00
Darren Tucker e428f29402 Remove now-unused 2nd arg to configs. 2021-04-22 22:29:33 +10:00
Darren Tucker cb4ff640d7 Add win10 test target. 2021-04-22 22:29:33 +10:00
Darren Tucker 4457837238 Add nbsd8 test target. 2021-04-22 22:29:33 +10:00
Darren Tucker bd4fba22e1 Add obsd51 target. 2021-04-22 22:29:33 +10:00
Darren Tucker 9403d0e805 Add fbsd13 target. 2021-04-22 22:29:33 +10:00
Darren Tucker f42b550c28 Add pattern for valgrind-unit. 2021-04-08 14:20:12 +10:00
Darren Tucker 19e5344627 Run unit tests under valgrind.
Run a separate build for the unit tests under Valgrind.  They take long
enough that running in parallel with the other Valgrind tests helps.
2021-04-08 13:35:14 +10:00
Darren Tucker 3674e33fa7 Further split Valgrind tests.
Even split in two, the Valgrind tests take by far the longest to run,
so split them four ways to further increase parallelism.
2021-04-07 17:02:27 +10:00
Darren Tucker b3ca08cb17 Install libcbor with libfido2. 2021-04-05 23:48:31 +10:00
Darren Tucker 7cd262c1c5 Save config.h and config.log on failure too. 2021-04-03 16:59:10 +11:00
Darren Tucker 867a7dcf00 Don't install moduli during tests.
Now that we have TEST_SSH_MODULI_FILE pointing to the moduli in the
soure directory we don't need to install the file to prevent warnings
about it being missing.
2021-03-13 13:52:53 +11:00
Darren Tucker de3866383b Only upload config logs if configure fails. 2021-03-01 21:13:24 +11:00
Darren Tucker 24a3a67bd7 Remove macos-11.00 PAM test target too.
These are failing apparently due to some kind of infrastructure problem,
making it look like every commit is busted.
2021-02-26 11:50:37 +11:00
Darren Tucker fd9fa76a34 Remove macos-11.0 from the test target list.
It has been consistently failing for the past few days with a github
actions internal error.
2021-02-26 01:15:10 +11:00
Darren Tucker f186a020f2 Add a couple more test VMs. 2021-02-23 16:05:22 +11:00
Darren Tucker ffcdd3d90e Valgrind test: split and move up list.
Since the valgrind test takes so long it approaches the limit allowed by
github, move it to the head of the list so it's the first one started and
split the longest tests out into a second instance that runs concurrently
with the first.
2021-02-23 11:18:49 +11:00
Darren Tucker 5fcb051494 Disable rlimit sandbox, doesn't work with valgrind
Only run regress tests, runing unit tests as well makes it run longer
than allowed y github.
2021-02-21 15:39:01 +11:00
Darren Tucker bb0b9bf453 Upload valgrind logs on failure. 2021-02-20 13:29:41 +11:00
Darren Tucker ebb3b75e97 Rename "vm" to "os" in selfhosted to match c-cpp.
Should make it easier to share code or maybe merge at some point.
2021-02-19 22:20:00 +11:00
Darren Tucker 76c0be0fe0 Upload regress failure logs in c-cpp too. 2021-02-19 22:19:38 +11:00
Darren Tucker 8751b6c313 Comment out Solaris 64bit PAM build...
until I can figure out why it's failing.
2021-02-19 22:13:36 +11:00
Darren Tucker e9f6d563c0 Actually run Valgrind tests. 2021-02-19 10:20:17 +11:00
Darren Tucker 41d232e226 Add test against Valgrind. 2021-02-19 10:16:56 +11:00
Darren Tucker e6528d91f1 Add fbsd12 test target. 2021-02-18 16:30:01 +11:00
Darren Tucker 6506cb2798 Remove unused arg. 2021-02-18 15:21:13 +11:00
Darren Tucker 93c31a6239 Add DEBUG_SK to kitchensink builds. 2021-02-18 14:54:07 +11:00
Darren Tucker 65085740d3 Add bbone test target (arm32). 2021-02-18 14:53:14 +11:00
Darren Tucker d0763c8d56 Fixing quoting for installing moduli on target guest. 2021-02-18 11:05:48 +11:00
Darren Tucker b3afc243bc Install moduli on target not host. 2021-02-18 10:41:58 +11:00
Darren Tucker bc9b0c2570 Skip unit tests on sol11 to speed things up. 2021-02-18 10:10:00 +11:00
Darren Tucker 161873035c Remove SKIP_UNIT as it needs to be a make arg. 2021-02-18 10:09:27 +11:00
Darren Tucker 1c293868e4 Always intall moduli.
Allows us to run tests without falling back to a fixed modulus.  Ensure that
the directory exists.
2021-02-18 10:05:03 +11:00
Darren Tucker 5c8f41ad10 Quote SSHD_CONFOPTS in case it contains spaces. 2021-02-18 09:59:09 +11:00
Darren Tucker 4653116c1f Fix labels on targets (dots vs underscores). 2021-02-18 09:51:18 +11:00
Darren Tucker 4512047f57 More compact representation of config matrix. 2021-02-17 21:47:48 +11:00
Darren Tucker 0406cd09f0 Skip unit tests on hosted VMs to speed things up. 2021-02-17 21:19:18 +11:00