Darren Tucker
8386886fb1
Run upstream obsdsnap tests on ephemeral runners.
2022-12-13 20:55:37 +11:00
Darren Tucker
b6e01459b5
Move obsdsnap test VMs to ephemeral runners.
2022-12-13 20:48:56 +11:00
Darren Tucker
ce3c3e78ce
Add SANDBOX_DEBUG to the kitchensink test build.
2022-12-07 18:58:25 +11:00
Darren Tucker
f84b9cffd5
Skip unit tests on slow riscv64 hardware.
2022-11-28 22:39:25 +11:00
Darren Tucker
9f2747e0be
Rework how selfhosted tests interact with runners.
...
Previously there was one runner per test target (mostly VMs). This had
a few limitations:
- multiple tests that ran on the same target (eg multiple build
configs) were serialized on availability or that runner.
- it needed manual balancing of VMs over host machines.
To address this, make VMs that use ephemeral disks (ie most of them)
all use a pool of runners with the "libvirt" label. This requires that
we distinguish between "host" and "target" for those. Native runners
and VMs with persistent disks (eg the constantly-updated snapshot ones)
specify the same host and target.
This should improve test throughput.
2022-11-28 22:39:08 +11:00
Darren Tucker
d664ddaec8
Run vmstartup from temp dir.
...
This will allow us to create ephemeral disk images per-runner.
2022-11-27 12:19:37 +11:00
Darren Tucker
0fa16e952b
Make "config" in matrix singular and pass in env.
...
This will allow the startup scripts to adapt their behaviour based on
the type and config.
2022-11-27 12:14:00 +11:00
Darren Tucker
e8857043af
Add "libvirt" label to dfly30.
2022-11-27 11:45:43 +11:00
Darren Tucker
9775473d84
Rename "os" in matrix to "target".
...
This is in preparation to distinguish this from the host that the runner
runs on in case where they are separate (eg VMs).
2022-11-27 11:20:35 +11:00
Darren Tucker
04fd00ceff
Remove unused self-hosted test targets.
2022-11-27 09:23:04 +11:00
Darren Tucker
c9d9fcad2a
Remove explicit "default" test config argument.
...
Not specifying the test config implicitly selects default args.
2022-11-27 09:16:15 +11:00
Darren Tucker
8926956f22
Add dfly62 test target.
2022-11-11 11:25:37 +11:00
Darren Tucker
ca98d3f8c6
Skip reexec test on OpenSSL 1.1.1 specifically.
...
OpenSSL 1.1.1 has a bug in its RNG that breaks reexec fallback, so skip
that test. See bz#3483 for details.
2022-11-09 20:59:20 +11:00
Darren Tucker
419aa8a312
Shutdown any VM before trying to check out repo.
...
In the case where the previous run did not clean up, the checkout will
fail as it'll leave a stale mount.
2022-11-08 12:42:52 +11:00
Darren Tucker
a32c07cbb7
Run vm startup and shutdown from runner temp dir.
...
Should work even if the github workspace dir is on a stale sshfs mount.
2022-11-08 12:01:33 +11:00
Darren Tucker
2b40a7dfcd
Add valrind-5 test here too.
2022-11-08 11:03:31 +11:00
Darren Tucker
2ea03d1f6d
Update checkout and upload actions.
...
Update actions/checkout and actions/upload-artifact to main branch for
compatibility with node.js v16.
2022-11-08 10:29:32 +11:00
Darren Tucker
4e316ff0f1
Split out rekey test since it runs the longest.
2022-11-08 10:29:32 +11:00
Darren Tucker
f58acaf8c7
Fix merge conflict.
2022-11-07 15:10:59 +11:00
Darren Tucker
162e574102
Branch-specific links for master status badges.
2022-11-07 15:07:33 +11:00
Darren Tucker
e4b7c12ab2
Add CIFuzz status badge.
2022-11-07 14:46:38 +11:00
Darren Tucker
b496b9f831
Do not run CIFuzz on selfhosted tree.
...
We already run it on the regular tree, no need to double up.
2022-11-07 14:45:37 +11:00
Darren Tucker
2138b1c4dd
Whitespace change to trigger CIFuzz workflow.
2022-11-07 14:41:58 +11:00
Darren Tucker
4670b97ef8
Run cifuzz workflow on the actions as regular CI.
2022-11-07 14:34:04 +11:00
David Korczynski
79391e66ce
Add CIFuzz integration
2022-11-07 14:31:45 +11:00
Darren Tucker
e1c6fcc142
Link to branch-specific queries for V_9_1 status.
2022-11-07 12:46:58 +11:00
Darren Tucker
99500df246
Don't run openbsd-compat tests on Cygwin.
...
Add "compat-tests" to the default TEST_TARGET so we can override as
necessary. Override TEST_TARGET for Cygwin as the tests don't currently
compile there.
2022-11-04 17:30:58 +11:00
Darren Tucker
9655217231
Rerun tests on changes to Makefile.in in any dir.
2022-11-03 23:07:50 +11:00
Darren Tucker
73550a218e
Increase selfhosted job timeout.
...
The default job timeout of 360 (6h) is not enough to complete the
regress tests for some of the slow VMs depending on the load on the host.
Increase to 600 (10h).
2022-11-03 13:41:16 +11:00
Darren Tucker
ba05370963
Add tests for OpenSSL 3.0.7 and LibreSSL 3.6.1.
2022-11-03 08:40:06 +11:00
Darren Tucker
edd24101c7
Run compat regress tests too.
2022-11-03 08:17:39 +11:00
Darren Tucker
a75cffc270
Add LibreSSL 3.6.0 to test suite.
...
While there, bump OpenSSL to latest 1.1.1q release.
2022-10-07 03:54:56 +11:00
Darren Tucker
fcc0f0c0e9
Add 9.1 branch to CI status page.
2022-10-06 21:18:16 +11:00
Darren Tucker
ef211eee63
Test commits to all branches of portable.
...
Only test OpenBSD upstream on commits to master since that's what it
tracks.
2022-10-05 06:31:48 +11:00
Darren Tucker
03277a4aa4
Move sftp from valgrind-2 to 3 to rebalance.
2022-08-31 20:26:30 +10:00
Darren Tucker
51b345f177
Add libcrypt-devel to cygwin-release deps.
...
Based on feedback from vinschen at redhat.com.
2022-08-27 21:49:27 +10:00
Darren Tucker
9f81736cf1
Add Windows 2022 test targets.
2022-08-27 09:37:40 +10:00
Darren Tucker
85e1a69243
Add cygwin-release test target.
...
This also moves the cygwin package install from the workflow file to
setup_ci.sh so that we can install different sets of Cygwin packages
for different test configs.
2022-08-26 19:22:25 +10:00
Darren Tucker
6851f4b8c3
Install Cygwin packages based on OS not config.
2022-08-19 17:22:18 +10:00
Darren Tucker
1b470b9036
Fix cygwin conditional steps.
2022-08-19 15:19:19 +10:00
Darren Tucker
fd6ee741ab
Add a bit more debug output.
2022-08-19 15:12:57 +10:00
Darren Tucker
a9305c4c73
Add Cygwin (on windows-2019) test target.
...
In addition to installing the requisite Cygwin packages, we also need to
explicitly invoke "sh" for steps that run other scripts since the runner
environment doesn't understand #! paths.
2022-08-19 15:08:57 +10:00
Darren Tucker
5a5c580b48
Check for perms to run agent-getpeereid test.
...
Ubuntu 22.04 defaults to private home dirs which prevents "nobody"
running ssh-add during the agent-getpeereid test. Check for this and
add the necessary permissions.
2022-08-18 21:36:39 +10:00
Darren Tucker
5e820bf79c
Replace deprecated ubuntu-18.04 runners with 22.04
2022-08-12 14:56:55 +10:00
Darren Tucker
d0c4fa5859
Rerun tests if any .github config file changes.
2022-08-11 14:23:58 +10:00
Darren Tucker
113fe6c77a
Skip hostbased during Valgrind tests.
...
Valgrind doesn't let ssh exec ssh-keysign (because it's setuid) so skip
it during the Valgrind based tests.
See https://bugs.kde.org/show_bug.cgi?id=119404 for a discussion of this
(ironically there the problematic binary was ssh(1) back when it could
still be setuid).
2022-08-11 13:50:05 +10:00
Darren Tucker
4df246ec75
Fix conditional for running hostbased tests.
2022-08-11 10:24:56 +10:00
Darren Tucker
fdbd5bf507
Test hostbased auth on github runners.
2022-08-10 17:37:58 +10:00
Darren Tucker
dba7099ffc
Remove deprecated MacOS 10.15 runners.
2022-07-27 18:40:12 +10:00
Darren Tucker
eb679e2959
Move vmshutdown to first step.
...
If a previous run on a physical runner has failed to clean up, the next
run will fail because it'll try to check out the code to a broken
directory mount. Make cleanup the first step.
2022-07-15 21:31:48 +10:00
Darren Tucker
46b91b70ff
Rename bbone test target to ARM.
2022-07-15 20:25:27 +10:00
Darren Tucker
e75bbc1d88
Capture stderr output from configure.
2022-07-12 14:37:15 +10:00
Darren Tucker
fb2f3a61bf
Move unset to before we set anything.
2022-07-12 12:54:24 +10:00
Darren Tucker
c483a5c0fb
Test against openssl-3.0.5.
2022-07-06 11:52:54 +10:00
Darren Tucker
669a56bcfe
Update sanitizer test targets:
...
- remove clang-sanitize-memory for now. It takes so long that the test
times out.
- add gcc sanitize-address and sanitize-undefined test targets.
2022-07-05 18:35:53 +10:00
Darren Tucker
48cc68b691
Add GCC address sanitizer build/test.
2022-07-05 18:30:10 +10:00
Darren Tucker
55c60bdd39
Move sanitizer logs into regress for collection.
2022-07-05 18:30:10 +10:00
Darren Tucker
7394ed80c4
Add clang sanitizer tests.
2022-07-03 22:53:44 +10:00
Darren Tucker
c71a1442d0
Update OpenSSL tests to the most recent releases.
2022-06-29 18:28:47 +10:00
Darren Tucker
1781f507c1
Test against OpenSSL 1.1.1o and 3.0.3.
2022-05-27 18:19:48 +10:00
Darren Tucker
c53906e0c5
Test against LibreSSL 3.5.3.
2022-05-27 18:18:31 +10:00
Darren Tucker
f38cf74f20
Also retest OpenBSD upstream on .yml changes.
2022-05-06 14:50:18 +10:00
Darren Tucker
217b518e0f
Add ubsan minimal testcase on OpenBSD.
...
As suggested by djm@.
2022-05-06 14:39:34 +10:00
Darren Tucker
4835544d2d
Add Mac OS X 12 test target.
2022-04-30 11:00:02 +10:00
Darren Tucker
97a6a8b8c1
Only run tests when source files change.
...
Also run tests on changes to V_9_0 branch.
2022-04-29 18:34:38 +10:00
Darren Tucker
7bf2eb958f
Add debian-riscv64 test target.
2022-04-26 23:30:59 +10:00
Darren Tucker
3913c93552
Update OpenSSL and LibreSSL versions in tests.
2022-04-25 17:21:24 +10:00
Darren Tucker
d2b888762b
Retire fbsd6 test VM.
...
It's long since out of support, relatively slow (it's i686) and the
compiler has trouble with PIE.
2022-04-16 14:31:13 +10:00
Darren Tucker
3741df98ff
Disable security key on fbsd6 test host.
2022-04-04 23:52:11 +10:00
Darren Tucker
32c12236f2
Specify TEST_SHELL=bash on AIX.
...
The system shells cause the agent-restrict test to fail due to some
quoting so explicitly specify bash until we can get configure to
autmatically work around that.
2022-04-04 15:16:51 +10:00
Darren Tucker
6c49eb5fab
Only run regression tests on slow VMs.
2022-04-01 23:21:40 +11:00
Darren Tucker
f67e479039
Increase test timeout to allow slow VMs to finish
2022-04-01 23:21:06 +11:00
Darren Tucker
5b054d7640
Skip slow tests on (very) slow test targets.
2022-04-01 13:16:47 +11:00
Darren Tucker
e937514920
Add AIX 5.1 test target.
2022-03-28 17:51:03 +11:00
Darren Tucker
4bbe815ba9
Drop leading "v" from release version identifier.
...
It's present in the git tags but not in the release tarball names.
Also drop extra "/" from URL path.
2022-03-26 22:01:31 +11:00
Darren Tucker
f5cdd3b3c2
Use tarballs when testing LibreSSL releases.
...
This means they'll still work when the combination of -portable and
openbsd github repos no longer match.
2022-03-26 16:28:04 +11:00
Darren Tucker
d23efe4b12
Add OpenWRT mips and mipsel test targets.
2022-03-26 08:13:46 +11:00
Darren Tucker
f2ff669347
No unused param warnings for clang-12 and gcc-11.
...
These have too many false positives in -Werror tests on the github CI
since we often provide empty stub functions for functionality not needed
for particular configurations.
2022-02-28 22:21:36 +11:00
Darren Tucker
96558ecd87
Add debian-i386 test target.
2022-02-26 14:10:41 +11:00
Darren Tucker
3016ba4703
Add tests for latest releases of {Libre,Open}SSL.
2022-02-25 11:39:34 +11:00
Darren Tucker
6520c488de
Add Alpine Linux test VM.
2022-02-22 13:08:59 +11:00
Damien Miller
7c0d4ce911
disable agent-restrict test on minix3
...
Minix seems to have a platform-wide limit on the number of
select(2) syscalls that can be concurrently issued. This test
seems to exceed this limit.
Refer to:
https://github.com/Stichting-MINIX-Research-Foundation/minix/blob/R3.3.0/minix/servers/vfs/select.c#L114
https://github.com/Stichting-MINIX-Research-Foundation/minix/blob/R3.3.0/minix/servers/vfs/select.c#L30-L31
2022-02-22 11:14:51 +11:00
Darren Tucker
63bf4f49ed
Add test configs for MUSL C library.
2022-02-18 12:12:21 +11:00
Darren Tucker
41417dbda9
Comment hurd test, the VM is currently broken.
2022-02-17 22:05:29 +11:00
Darren Tucker
766176cfdb
Add cygwin-release test config.
...
This tests the flags used to build the cygwin release binaries.
2022-02-12 10:24:56 +11:00
Darren Tucker
ea13fc830f
Stop testing OpenBSD HEAD on 6.9 and 7.0.
...
HEAD is not guaranteed to work on previous stable branches, and at the
moment is broken due to libfido API changes.
2022-02-11 13:42:36 +11:00
Darren Tucker
3fe6800b60
Move more tests out of valgrind-1 runner.
2022-01-20 00:49:57 +11:00
Darren Tucker
6c435bd499
Stop on first test failure to minimize logs.
2022-01-13 09:01:05 +11:00
Darren Tucker
613a6545fc
libhardended_malloc.so moved into out dir.
2022-01-11 20:56:01 +11:00
Darren Tucker
bbf285164d
Always save config.h as build artifact.
...
Should allow better comparison between failing and succeeding test
platforms.
2022-01-05 11:48:58 +11:00
Darren Tucker
03bd4ed0db
Add OpenBSD 7.0 target. Retire 6.8.
2022-01-05 11:48:58 +11:00
Darren Tucker
3dafd3fe22
Increase timeout for test step.
2021-12-11 08:43:41 +11:00
Darren Tucker
5aefb05cd5
Update the list of tests that don't work on Minix.
...
While there, remove CC (configure will now find clang) and make the test
list easier to update via cut and paste.
2021-12-10 10:27:27 +11:00
Darren Tucker
e665ed2d0c
Switch from LibreSSL 3.4.0 to 3.4.1.
...
The LibreSSL 3.4.0 release has an OPENBSD_BRANCH that points to
"master" and that branch no longer has the files LibreSSL expects
and thus it will no longer build, breaking the test.
2021-11-12 22:57:51 +11:00
Darren Tucker
d575cf4489
kitchensink test target now needs krb5.
2021-10-22 23:27:41 +11:00
Darren Tucker
4ae39cada2
Test both MIT KRB5 and Heimdal.
2021-10-22 22:54:33 +11:00
Darren Tucker
a07664646b
Source configs script so setup_ci can use settings
2021-10-22 14:00:05 +11:00
Darren Tucker
34df52c201
Install libedit and pam based on config flags.
2021-10-22 09:42:14 +11:00
Darren Tucker
8c626cc563
Don't use 'here string", it's not POSIX.
2021-10-21 16:53:39 +11:00
Darren Tucker
086a4b5977
Remove -Werror from compiler package to install.
2021-10-21 15:33:27 +11:00