openssh-portable/regress
djm@openbsd.org 1a7217ac06 upstream: adapt to ssh-sk-client change
OpenBSD-Regress-ID: 40481999a5928d635ab2e5b029e8239c112005ea
2019-12-16 14:20:35 +11:00
..
misc upstream: adapt to ssh-sk-client change 2019-12-16 14:20:35 +11:00
unittests upstream: unbreak tests for recent security key changes 2019-11-25 21:34:20 +11:00
Makefile Add SKIP_LTESTS for skipping specific tests. 2019-09-30 14:11:42 +10:00
README.regress Add SKIP_LTESTS for skipping specific tests. 2019-09-30 14:11:42 +10:00
addrmatch.sh - dtucker@cvs.openbsd.org 2012/05/13 01:42:32 2012-06-30 15:01:22 +10:00
agent-getpeereid.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
agent-pkcs11.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
agent-ptrace.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
agent-timeout.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
agent.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
allow-deny-users.sh upstream: make this use ssh_proxy rather than starting/stopping a 2018-07-13 12:14:38 +10:00
authinfo.sh upstream: make this a bit more portable-friendly 2018-04-10 10:17:56 +10:00
banner.sh upstream commit 2017-05-01 11:59:42 +10:00
broken-pipe.sh upstream commit 2017-05-01 11:59:42 +10:00
brokenkeys.sh upstream commit 2017-05-01 11:59:42 +10:00
cert-file.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
cert-hostkey.sh upstream: it's no longer possible to disable privilege separation 2019-12-16 14:20:35 +11:00
cert-userkey.sh upstream: it's no longer possible to disable privilege separation 2019-12-16 14:20:35 +11:00
cfginclude.sh upstream commit 2016-06-08 11:42:39 +10:00
cfgmatch.sh upstream: Add tests for sshd -T -C with Match. 2019-06-19 12:21:23 +10:00
cfgmatchlisten.sh increase timeout to match cfgmatch.sh 2018-07-13 11:41:33 +10:00
cfgparse.sh upstream: Clean up comment. 2018-06-01 14:23:22 +10:00
check-perm.c Disable tests where fs perms are incorrect 2016-02-23 17:40:16 +11:00
cipher-speed.sh upstream commit 2017-05-01 11:59:42 +10:00
conch-ciphers.sh upstream: Adapt the PuTTY/Conch tests to new key names. 2019-07-05 15:07:55 +10:00
connect-privsep.sh upstream commit 2017-05-01 11:59:42 +10:00
connect-uri.sh upstream commit 2017-10-31 09:08:51 +11:00
connect.sh upstream commit 2017-05-01 11:59:42 +10:00
dhgex.sh upstream: Instead of running sed over the whole log to remove CRs, 2019-10-07 13:08:57 +11:00
dsa_ssh2.prv
dsa_ssh2.pub
dynamic-forward.sh upstream commit 2017-09-26 16:56:15 +10:00
envpass.sh - djm@cvs.openbsd.org 2005/03/04 08:48:46 2005-03-07 18:35:34 +11:00
exit-status.sh upstream commit 2017-05-01 11:59:42 +10:00
forcecommand.sh upstream commit 2017-05-01 11:59:42 +10:00
forward-control.sh upstream: test the correct configuration option name 2018-06-08 00:31:38 +10:00
forwarding.sh upstream: Remove the sleeps and thus races from the forwarding 2019-07-20 22:13:40 +10:00
host-expand.sh upstream commit 2017-05-01 11:59:42 +10:00
hostkey-agent.sh upstream: it's no longer possible to disable privilege separation 2019-12-16 14:20:35 +11:00
hostkey-rotate.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
integrity.sh Remove override disabling DH-GEX. 2019-07-25 20:11:45 +10:00
kextype.sh upstream commit 2015-04-01 10:02:56 +11:00
key-options.sh upstream: add a comment that could have saved me 45 minutes of wild 2018-07-04 23:52:50 +10:00
keygen-change.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
keygen-convert.sh upstream: Only test conversion of key types supported by the 2019-07-23 22:51:22 +10:00
keygen-knownhosts.sh upstream: Adapt to extra default verboisity from ssh-keygen when 2018-06-01 14:24:08 +10:00
keygen-moduli.sh upstream: Skip DH group generation test if binaries don't support 2019-07-23 22:51:22 +10:00
keys-command.sh upstream: Switch keys-command test from rsa to ed25519 since it's 2019-07-25 20:01:19 +10:00
keyscan.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
keytype.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
krl.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
limit-keytype.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
localcommand.sh upstream commit 2017-05-01 11:59:42 +10:00
login-timeout.sh upstream commit 2017-09-18 14:11:22 +10:00
mkdtemp.c Missing unistd.h for regress/mkdtemp.c 2018-08-20 15:57:29 +10:00
modpipe.c Remove execute bit from modpipe.c. 2018-02-15 22:33:21 +11:00
moduli.in upstream commit 2016-09-14 10:57:21 +10:00
multiplex.sh upstream: Add a sleep to allow forwards to come up. 2019-07-05 15:06:04 +10:00
multipubkey.sh upstream: it's no longer possible to disable privilege separation 2019-12-16 14:20:35 +11:00
netcat.c Check if IP_TOS is defined before using. 2019-11-01 13:42:52 +11:00
portnum.sh - dtucker@cvs.openbsd.org 2013/05/17 10:34:30 2013-05-17 20:47:29 +10:00
principals-command.sh upstream: it's no longer possible to disable privilege separation 2019-12-16 14:20:35 +11:00
proto-mismatch.sh upstream commit 2017-05-01 11:59:42 +10:00
proto-version.sh upstream commit 2017-06-08 13:11:11 +10:00
proxy-connect.sh upstream commit 2017-10-05 05:51:46 +11:00
putty-ciphers.sh Provide $OBJ to paths in PuTTY interop tests. 2018-03-23 13:21:14 +11:00
putty-kex.sh Provide $OBJ to paths in PuTTY interop tests. 2018-03-23 13:21:14 +11:00
putty-transfer.sh Provide $OBJ to paths in PuTTY interop tests. 2018-03-23 13:21:14 +11:00
reconfigure.sh upstream commit 2017-05-01 11:59:42 +10:00
reexec.sh upstream commit 2017-09-18 14:11:42 +10:00
rekey.sh upstream: more typos spotted by Karsten Weiss using codespell 2018-04-10 10:17:56 +10:00
rsa_openssh.prv
rsa_openssh.pub
rsa_ssh2.prv
scp-ssh-wrapper.sh upstream: add regression tests for scp for out-of-destination path file 2019-07-19 13:53:27 +10:00
scp-uri.sh upstream commit 2017-12-19 16:19:18 +11:00
scp.sh upstream: add regression tests for scp for out-of-destination path file 2019-07-19 13:53:27 +10:00
setuid-allowed.c Adapt portable to legacy buffer API removal 2018-07-10 19:39:52 +10:00
sftp-badcmds.sh - dtucker@cvs.openbsd.org 2013/05/17 10:26:26 2013-05-17 20:43:13 +10:00
sftp-batch.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
sftp-chroot.sh upstream: Append pid to temp files in /var/run and set a cleanup 2018-11-22 20:58:27 +11:00
sftp-cmds.sh drop old Cygwin considerations 2019-03-28 09:23:46 +11:00
sftp-glob.sh - (djm) [configure.ac regress/agent-getpeereid.sh regress/multiplex.sh] 2011-01-17 16:17:09 +11:00
sftp-perm.sh - (tim) [regress/sftp-perm.sh] We need a shell that understands "! somecmd" 2013-10-24 12:22:49 -07:00
sftp-uri.sh Replace /dev/stdin with "-". 2018-03-25 09:40:46 +11:00
sftp.sh upstream commit 2017-10-31 09:08:51 +11:00
ssh-com-client.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
ssh-com-keygen.sh - markus@cvs.openbsd.org 2004/02/24 17:06:52 2004-02-29 20:33:51 +11:00
ssh-com-sftp.sh - dtucker@cvs.openbsd.org 2013/05/17 04:29:14 2013-05-17 15:32:29 +10:00
ssh-com.sh upstream commit 2017-05-08 11:54:17 +10:00
ssh2putty.sh upstream: Remove workaround for broken 'openssl rsa -text' output 2019-11-25 21:34:20 +11:00
sshcfgparse.sh upstream: Only use DSA key type in tests if binaries support it. 2019-07-24 16:47:58 +10:00
sshd-log-wrapper.sh Fix missed RCSID merges 2017-10-20 13:22:00 +11:00
sshsig.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
stderr-after-eof.sh upstream commit 2017-05-01 11:59:42 +10:00
stderr-data.sh upstream commit 2017-05-01 11:59:42 +10:00
t4.ok upstream commit 2014-12-22 13:21:07 +11:00
t5.ok
t11.ok add missing regress output file 2014-12-22 13:47:07 +11:00
test-exec.sh upstream: test FIDO2/U2F key types; ok markus@ 2019-11-27 11:02:49 +11:00
transfer.sh upstream commit 2017-05-01 11:59:42 +10:00
try-ciphers.sh upstream commit 2017-05-01 11:59:42 +10:00
valgrind-unit.sh Enable leak checks for unit tests with valgrind 2018-07-13 11:41:33 +10:00
yes-head.sh Fix missed RCSID merges 2017-10-20 13:22:00 +11:00

README.regress

Overview.

$ ./configure && make tests

You'll see some progress info. A failure will cause either the make to
abort or the driver script to report a "FATAL" failure.

The test consists of 2 parts. The first is the file-based tests which is
driven by the Makefile, and the second is a set of network or proxycommand
based tests, which are driven by a driver script (test-exec.sh) which is
called multiple times by the Makefile.

Failures in the first part will cause the Makefile to return an error.
Failures in the second part will print a "FATAL" message for the failed
test and continue.

OpenBSD has a system-wide regression test suite. OpenSSH Portable's test
suite is based on OpenBSD's with modifications.


Environment variables.

SKIP_UNIT: Skip unit tests.
SUDO: path to sudo/doas command, if desired. Note that some systems
	(notably systems using PAM) require sudo to execute some tests.
LTESTS: Whitespace separated list of tests (filenames without the .sh
	extension) to run.
SKIP_LTESTS: Whitespace separated list of tests to skip.
OBJ: used by test scripts to access build dir.
TEST_SHELL: shell used for running the test scripts.
TEST_SSH_FAIL_FATAL: set to "yes" to make any failure abort the test
	currently in progress.
TEST_SSH_PORT: TCP port to be used for the listening tests.
TEST_SSH_QUIET: set to "yes" to suppress non-fatal output.
TEST_SSH_SSHD_CONFOPTS: Configuration directives to be added to sshd_config
	before running each test.
TEST_SSH_SSH_CONFOPTS: Configuration directives to be added to
	ssh_config before running each test.
TEST_SSH_TRACE: set to "yes" for verbose output from tests 
TEST_SSH_x: path to "ssh" command under test, where x is one of
	SSH, SSHD, SSHAGENT, SSHADD, SSHKEYGEN, SSHKEYSCAN, SFTP or
	SFTPSERVER
USE_VALGRIND: Run the tests under valgrind memory checker.


Individual tests.

You can run an individual test from the top-level Makefile, eg:
$ make tests LTESTS=agent-timeout

If you need to manipulate the environment more you can invoke test-exec.sh
directly if you set up the path to find the binaries under test and the
test scripts themselves, for example:

$ cd regress
$ PATH=`pwd`/..:$PATH:. TEST_SHELL=/bin/sh sh test-exec.sh `pwd` \
    agent-timeout.sh
ok agent timeout test


Files.

test-exec.sh: the main test driver. Sets environment, creates config files
and keys and runs the specified test.

At the time of writing, the individual tests are:
connect.sh:		simple connect
proxy-connect.sh:	proxy connect
connect-privsep.sh:	proxy connect with privsep
connect-uri.sh:		uri connect
proto-version.sh:	sshd version with different protocol combinations
proto-mismatch.sh:	protocol version mismatch
exit-status.sh:		remote exit status
envpass.sh:		environment passing
transfer.sh:		transfer data
banner.sh:		banner
rekey.sh:		rekey
stderr-data.sh:		stderr data transfer
stderr-after-eof.sh:	stderr data after eof
broken-pipe.sh:		broken pipe test
try-ciphers.sh:		try ciphers
yes-head.sh:		yes pipe head
login-timeout.sh:	connect after login grace timeout
agent.sh:		simple connect via agent
agent-getpeereid.sh:	disallow agent attach from other uid
agent-timeout.sh:	agent timeout test
agent-ptrace.sh:	disallow agent ptrace attach
keyscan.sh:		keyscan
keygen-change.sh:	change passphrase for key
keygen-convert.sh:	convert keys
keygen-moduli.sh:	keygen moduli
key-options.sh:		key options
scp.sh:			scp
scp-uri.sh:		scp-uri
sftp.sh:		basic sftp put/get
sftp-chroot.sh:		sftp in chroot
sftp-cmds.sh:		sftp command
sftp-badcmds.sh:	sftp invalid commands
sftp-batch.sh:		sftp batchfile
sftp-glob.sh:		sftp glob
sftp-perm.sh:		sftp permissions
sftp-uri.sh:		sftp-uri
ssh-com-client.sh:	connect with ssh.com client
ssh-com-keygen.sh:	ssh.com key import
ssh-com-sftp.sh:	basic sftp put/get with ssh.com server
ssh-com.sh:		connect to ssh.com server
reconfigure.sh:		simple connect after reconfigure
dynamic-forward.sh:	dynamic forwarding
forwarding.sh:		local and remote forwarding
multiplex.sh:		connection multiplexing
reexec.sh:		reexec tests
brokenkeys.sh:		broken keys
sshcfgparse.sh:		ssh config parse
cfgparse.sh:		sshd config parse
cfgmatch.sh:		sshd_config match
cfgmatchlisten.sh:	sshd_config matchlisten
addrmatch.sh:		address match
localcommand.sh:	localcommand
forcecommand.sh:	forced command
portnum.sh:		port number parsing
keytype.sh:		login with different key types
kextype.sh:		login with different key exchange algorithms
cert-hostkey.sh		certified host keys
cert-userkey.sh:	certified user keys
host-expand.sh:		expand %h and %n
keys-command.sh:	authorized keys from command
forward-control.sh:	sshd control of local and remote forwarding
integrity.sh:		integrity
krl.sh:			key revocation lists
multipubkey.sh:		multiple pubkey
limit-keytype.sh:	restrict pubkey type
hostkey-agent.sh:	hostkey agent
keygen-knownhosts.sh:	ssh-keygen known_hosts
hostkey-rotate.sh:	hostkey rotate
principals-command.sh:	authorized principals command
cert-file.sh:		ssh with certificates
cfginclude.sh:		config include
allow-deny-users.sh:	AllowUsers/DenyUsers
authinfo.sh:		authinfo


Problems?

Run the failing test with shell tracing (-x) turned on:
$ PATH=`pwd`/..:$PATH:. sh -x test-exec.sh `pwd` agent-timeout.sh

Failed tests can be difficult to diagnose. Suggestions:
- run the individual test via ./test-exec.sh `pwd` [testname]
- set LogLevel to VERBOSE in test-exec.sh and enable syslogging of
  auth.debug (eg to /var/log/authlog).


Known Issues.

- Similarly, if you do not have "scp" in your system's $PATH then the
  multiplex scp tests will fail (since the system's shell startup scripts
  will determine where the shell started by sshd will look for scp).

- Recent GNU coreutils deprecate "head -[n]": this will cause the yes-head
  test to fail.  The old behaviour can be restored by setting (and
  exporting) _POSIX2_VERSION=199209 before running the tests.