Commit Graph

289 Commits

Author SHA1 Message Date
djm@openbsd.org c97520d23d
upstream: in olde rcp/scp protocol mode, when rejecting a path from the
server as not matching the glob that the client sent, log (at debug level)
the received pathname as well as the list of possible expected paths expanded
from the glob. bz2966

OpenBSD-Commit-ID: 0bd8db8a595334ca86bca8f36e23fc0395315765
2023-10-11 16:44:40 +11:00
djm@openbsd.org b6b49130a0
upstream: rename remote_glob() -> sftp_glob() to match other API
OpenBSD-Commit-ID: d9dfb3708d824ec02970a84d96cf5937e0887229
2023-09-11 09:14:02 +10:00
djm@openbsd.org 2de990142a
upstream: the sftp code was one of my first contributions to
OpenSSH and it shows - the function names are terrible.

Rename do_blah() to sftp_blah() to make them less so.

Completely mechanical except for sftp_stat() and sftp_lstat() which
change from returning a pointer to a static variable (error-prone) to
taking a pointer to a caller-provided receiver.

OpenBSD-Commit-ID: eb54d6a72d0bbba4d623e2175cf5cc4c75dc2ba4
2023-09-08 15:59:08 +10:00
djm@openbsd.org 2ee48adb9f
upstream: add defence-in-depth checks for some unreachable integer
overflows reported by Yair Mizrahi @ JFrog; feedback/ok millert@

OpenBSD-Commit-ID: 52af085f4e7ef9f9d8423d8c1840a6a88bda90bd
2023-07-14 15:33:30 +10:00
dtucker@openbsd.org f703757234
upstream: Explicitly ignore return from waitpid here too.
OpenBSD-Commit-ID: eef2403df083c61028969fc679ee370373eacacb
2023-03-31 17:01:01 +11:00
dtucker@openbsd.org 6b73aa2903
upstream: Explictly ignore return codes
where we don't check them.

OpenBSD-Commit-ID: 1ffb03038ba1b6b72667be50cf5e5e396b5f2740
2023-03-31 16:17:46 +11:00
djm@openbsd.org 4f0a676486
upstream: scp: when copying local->remote, check that source file
exists before opening SFTP connection to the server. Based on GHPR#370 ok
dtucker, markus

OpenBSD-Commit-ID: b4dd68e15bfe22ce4fac9960a1066a2b721e54fb
2023-03-27 14:28:00 +11:00
dtucker@openbsd.org 93291bd723
upstream: Check return values of dup2. Spotted by Coverity, ok djm@
OpenBSD-Commit-ID: 19fb1b53072826d00c67df677731d2f6c1dd602b
2023-03-03 14:49:14 +11:00
Damien Miller 4bcc737a35
remove buffer len workaround for NetBSD 4.x
Switching to from pipes to a socketpair for communicating with the
ssh process avoids the (kernel bug?) problem.
2023-01-11 11:45:17 +11:00
Damien Miller f5154d2aac
add back use of pipes in scp.c under USE_PIPES
This matches sftp.c which prefers socketpair but uses pipes on
some older platforms.
2023-01-11 11:44:19 +11:00
millert@openbsd.org eec737b59c
upstream: Switch scp from using pipes to a socketpair for
communication with it's ssh sub-processes.  We no longer need to reserve two
descriptors to ensure that we don't end up using fd 0-2 unexpectedly, that is
handled by sanitise_stdfd() in main(). Based on an original diff from djm@.
OK deraadt@ djm@

OpenBSD-Commit-ID: b80c372faac462471e955ddeab9480d668a2e48d
2023-01-11 11:43:17 +11:00
Damien Miller ab6bb69e25
unbreak scp on NetBSD 4.x
e555d5cad5 effectively increased the default copy buffer size for SFTP
transfers. This caused NetBSD 4.x to hang during the "copy local file to
remote file in place" scp.sh regression test.

This puts back the original 32KB copy buffer size until we can properly
figure out why.

lots of debugging assistance from dtucker@
2023-01-06 19:13:36 +11:00
jmc@openbsd.org 4acfaabfae
upstream: add -X to usage();
OpenBSD-Commit-ID: 1bdc3df7de11d766587b0428318336dbffe4a9d0
2023-01-03 17:53:05 +11:00
djm@openbsd.org e555d5cad5
upstream: add a -X option to both scp(1) and sftp(1) to allow
control over some SFTP protocol knobs: the copy buffer length and
the number of inflight requests, both of which are used during
upload/download.

Previously these could be controlled in sftp(1) using the -b/-R options.
This makes them available in both SFTP protocol clients using the same
option character sequence.

ok dtucker@

OpenBSD-Commit-ID: 27502bffc589776f5da1f31df8cb51abe9a15f1c
2023-01-03 17:53:05 +11:00
djm@openbsd.org a4821a5924
upstream: when scp(1) is using the SFTP protocol for transport (the
default), better match scp/rcp's handling of globs that don't match the
globbed characters but do match literally (e.g. trying to transfer
"foo.[1]").

Previously scp(1) in SFTP mode would not match these pathnames but
legacy scp/rcp mode would.

Reported by Michael Yagliyan in bz3488; ok dtucker@

OpenBSD-Commit-ID: d8a3773f53015ba811fddba7473769a2fd343e11
2022-10-25 08:55:11 +11:00
Darren Tucker 1bdf86725b Remove seed_rng calls from scp, sftp, sftp-server.
These binaries don't use OpenSSL's random functions.  The next step
will be to stop linking them against libcrypto.  ok djm@
2022-07-27 16:22:30 +10:00
djm@openbsd.org 56a0697fe0 upstream: arrange for scp, when in sftp mode, to not ftruncate(3) files
early

previous behavious of unconditionally truncating the destination file
would cause "scp ~/foo localhost:" and "scp localhost:foo ~/" to
delete all the contents of their destination.

spotted by solene@ sthen@, also bz3431; ok dtucker@

OpenBSD-Commit-ID: ca39fdd39e0ec1466b9666f15cbcfddea6aaa179
2022-05-13 17:00:56 +10:00
djm@openbsd.org 16ea8b8583 upstream: don't leak argument list; bz3404, reported by Balu
Gajjala ok dtucker@

OpenBSD-Commit-ID: fddc32d74e5dd5cff1a49ddd6297b0867eae56a6
2022-03-20 19:54:35 +11:00
deraadt@openbsd.org 449bcb8403 upstream: and we go back to testing sftp-scp after the 8.9
release...

OpenBSD-Commit-ID: a80440168258adca543a4607b871327a279c569c
2022-02-24 08:23:13 +11:00
djm@openbsd.org 9699151b03 upstream: revert for imminent OpenSSH release, which wil ship with
scp in RCP mode.

> revision 1.106
> date: 2021/10/15 14:46:46;  author: deraadt;  state: Exp;  lines: +13 -9;  commitid: w5n9B2RE38tFfggl;
> openbsd 7.0 release shipped with the (hopefully last) scp that uses RCP
> protocol for copying.  Let's get back to testing the SFTP protocol.

This will be put back once the OpenSSH release is done.

OpenBSD-Commit-ID: 0c725481a78210aceecff1537322c0b2df03e768
2022-02-10 15:14:24 +11:00
djm@openbsd.org 8cfa73f8a2 upstream: better match legacy scp behaviour: show un-expanded paths
in error messages. Spotted by and ok tb@

OpenBSD-Commit-ID: 866c8ffac5bd7d38ecbfc3357c8adfa58af637b7
2022-02-02 10:38:59 +11:00
djm@openbsd.org ac7c9ec894 upstream: when transferring multiple files in SFTP mode, create the
destination directory if it doesn't already exist to match olde-scp(1)
behaviour. noticed by deraadt@ ok markus@

OpenBSD-Commit-ID: cf44dfa231d4112f697c24ff39d7ecf2e6311407
2022-01-18 10:00:41 +11:00
djm@openbsd.org 63670d4e90 upstream: fix some corner-case bugs in scp sftp-mode handling of
~-prefixed paths; spotted by jsg; feedback jsg & deraadt, ok jsg & markus

OpenBSD-Commit-ID: d1697dbaaa9f0f5649d69be897eab25c7d37c222
2022-01-08 18:38:50 +11:00
deraadt@openbsd.org d4bed54456 upstream: For open/openat, if the flags parameter does not contain
O_CREAT, the 3rd (variadic) mode_t parameter is irrelevant.  Many developers
in the past have passed mode_t (0, 044, 0644, or such), which might lead
future people to copy this broken idiom, and perhaps even believe this
parameter has some meaning or implication or application. Delete them all.
This comes out of a conversation where tb@ noticed that a strange (but
intentional) pledge behaviour is to always knock-out high-bits from mode_t on
a number of system calls as a safety factor, and his bewilderment that this
appeared to be happening against valid modes (at least visually), but no
sorry, they are all irrelevant junk.  They could all be 0xdeafbeef. ok
millert

OpenBSD-Commit-ID: 503d11633497115688c0c6952686524f01f53121
2021-10-28 13:56:59 +11:00
deraadt@openbsd.org 052a9d8494 upstream: switch scp(1) back to sftp protocol.
openbsd 7.0 release shipped with the (hopefully last) scp that uses RCP
protocol for copying.  Let's get back to testing the SFTP protocol.

OpenBSD-Commit-ID: 9eaa35d95fd547b78b0a043b3f518e135f151f30
2021-10-22 22:01:46 +11:00
djm@openbsd.org 7ed1a3117c upstream: fix missing -s in SYNOPSYS and usage() as well as a
capitalisation mistake; spotted by jmc@

OpenBSD-Commit-ID: 0ed8ee085c7503c60578941d8b45f3a61d4c9710
2021-09-21 08:06:09 +10:00
djm@openbsd.org 277d3c6adf upstream: Switch scp back to use the old protocol by default, ahead of
release. We'll wait a little longer for people to pick up sftp-server(8) that
supports the extension that scp needs for ~user paths to continue working in
SFTP protocol mode. Discussed with deraadt@

OpenBSD-Commit-ID: f281f603a705fba317ff076e7b11bcf2df941871
2021-09-20 12:03:17 +10:00
djm@openbsd.org ace19b34cc upstream: better error message for ~user failures when the
sftp-server lacks the expand-path extension; ok deraadt@

OpenBSD-Commit-ID: 9c1d965d389411f7e86f0a445158bf09b8f9e4bc
2021-09-19 17:21:59 +10:00
djm@openbsd.org 6b1238ba97 upstream: make some more scp-in-SFTP mode better match Unix idioms
suggested by deraadt@

OpenBSD-Commit-ID: 0f2439404ed4cf0b0be8bf49a1ee734836e1ac87
2021-09-19 17:21:59 +10:00
djm@openbsd.org e694f8ac44 upstream: allow log_stderr==2 to prefix log messages with argv[0]
use this to make scp's SFTP mode error messages more scp-like

prompted by and ok deraadt@

OpenBSD-Commit-ID: 0e821dbde423fc2280e47414bdc22aaa5b4e0733
2021-09-19 17:21:59 +10:00
djm@openbsd.org ba61123eef upstream: when using SFTP protocol, continue transferring files after a
transfer error occurs. This matches original scp/rcp behaviour. ok dtucker@

OpenBSD-Commit-ID: dfe4558d71dd09707e9b5d6e7d2e53b793da69fa
2021-09-11 10:41:39 +10:00
djm@openbsd.org 73050fa38f upstream: Use the SFTP protocol by default. The original scp/rcp
protocol remains available via the -O flag.

Note that ~user/ prefixed paths in SFTP mode require a protocol extension
that was first shipped in OpenSSH 8.7.

ok deraadt, after baking in snaps for a while without incident

OpenBSD-Commit-ID: 23588976e28c281ff5988da0848cb821fec9213c
2021-09-09 12:35:37 +10:00
naddy@openbsd.org e1371e4f58 upstream: scp: tweak man page and error message for -3 by default
Now that the -3 option is enabled by default, flip the documentation
and error message logic from "requires -3" to "blocked by -R".

ok djm@

OpenBSD-Commit-ID: a872592118444fb3acda5267b2a8c3d4c4252020
2021-08-12 23:03:13 +10:00
naddy@openbsd.org 49f46f6d77 upstream: scp: do not spawn ssh with two -s flags for
remote-to-remote copies

Do not add another "-s" to the argument vector every time an SFTP
connection is initiated.  Instead, introduce a subsystem flag to
do_cmd() and add "-s" when the flag is set.

ok djm@

OpenBSD-Commit-ID: 25df69759f323661d31b2e1e790faa22e27966c1
2021-08-12 23:03:13 +10:00
djm@openbsd.org 391ca67fb9 upstream: Prepare for a future where scp(1) uses the SFTP protocol by
default. Replace recently added -M option to select the protocol with -O
(olde) and -s (SFTP) flags, and label the -s flag with a clear warning that
it will be removed in the near future (so no, don't use it in scripts!).

prompted by/feedback from deraadt@

OpenBSD-Commit-ID: 92ad72cc6f0023c9be9e316d8b30eb6d8d749cfc
2021-08-10 13:37:20 +10:00
djm@openbsd.org bfdd4b722f upstream: make scp -3 the default for remote-to-remote copies. It
provides a much better and more intuitive user experience and doesn't require
exposing credentials to the source host.

thanks naddy@ for catching the missing argument in usage()

"Yes please!" - markus@
"makes a lot of sense" - deraadt@
"the right thing to do" - dtucker@

OpenBSD-Commit-ID: d0d2af5f0965c5192ba5b2fa461c9f9b130e5dd9
2021-08-10 12:47:46 +10:00
djm@openbsd.org 2f7a3b51ce upstream: make scp in SFTP mode try to use relative paths as much
as possible. Previosuly, it would try to make relative and ~/-rooted paths
absolute before requesting transfers.

prompted by and much discussion deraadt@
ok markus@

OpenBSD-Commit-ID: 46639d382ea99546a4914b545fa7b00fa1be5566
2021-08-10 12:47:46 +10:00
djm@openbsd.org 2ab864010e upstream: SFTP protocol extension to allow the server to expand
~-prefixed paths, in particular ~user ones. Allows scp in sftp mode to accept
these paths, like scp in rcp mode does.

prompted by and much discussion deraadt@
ok markus@

OpenBSD-Commit-ID: 7d794def9e4de348e1e777f6030fc9bafdfff392
2021-08-10 12:47:46 +10:00
djm@openbsd.org 41b019ac06 upstream: when scp is in SFTP mode, try to deal better with ~
prefixed paths. ~user paths aren't supported, but ~/ paths will be accepted
and prefixed with the SFTP server starting directory (more to come)

prompted by and discussed with deraadt@
ok markus@

OpenBSD-Commit-ID: 263a071f14555c045fd03132a8fb6cbd983df00d
2021-08-10 12:47:45 +10:00
djm@openbsd.org b4b3f3da6c upstream: on fatal errors, make scp wait for ssh connection before
exiting avoids LogLevel=verbose (or greater) messages from ssh appearing
after scp has returned exited and control has returned to the shell; ok
markus@

(this was originally committed as r1.223 along with unrelated stuff that
I rolled back in r1.224)

OpenBSD-Commit-ID: 1261fd667ad918484889ed3d7aec074f3956a74b
2021-08-10 12:47:45 +10:00
djm@openbsd.org 2ae7771749 upstream: rever r1.223 - I accidentally committed unrelated changes
OpenBSD-Commit-ID: fb73f3865b2647a27dd94db73d6589506a9625f9
2021-08-10 12:47:45 +10:00
djm@openbsd.org 2b67932bb3 upstream: on fatal errors, make scp wait for ssh connection before
exiting avoids LogLevel=verbose (or greater) messages from ssh appearing
after scp has returned exited and control has returned to the shell; ok
markus@

OpenBSD-Commit-ID: ef9dab5ef5ae54a6a4c3b15d380568e94263456c
2021-08-10 12:47:39 +10:00
dtucker@openbsd.org cf27810a64 upstream: Fix prototype mismatch for do_cmd. ok djm@
OpenBSD-Commit-ID: 1c1598bb5237a7ae0be99152f185e0071163714d
2021-08-07 12:11:03 +10:00
djm@openbsd.org 7b1cbcb759 upstream: make scp(1) in SFTP mode follow symlinks like
traditional scp(1) ok markus@

OpenBSD-Commit-ID: 97255e55be37e8e26605e4ba1e69f9781765d231
2021-08-07 10:20:31 +10:00
djm@openbsd.org c677e65365 upstream: make scp(1) in SFTP mode output better match original
scp(1) by suppressing "Retrieving [path]" lines that were emitted to support
the interactive sftp(1) client. ok markus@

OpenBSD-Commit-ID: 06be293df5f156a18f366079be2f33fa68001acc
2021-08-07 10:20:31 +10:00
djm@openbsd.org 318c06bb04 upstream: use sftp_client crossloading to implement scp -3
feedback/ok markus@

OpenBSD-Commit-ID: 7db4c0086cfc12afc9cfb71d4c2fd3c7e9416ee9
2021-08-07 10:20:31 +10:00
djm@openbsd.org e3c0ba0587 upstream: prepare for scp -3 implemented via sftp
OpenBSD-Commit-ID: 194aac0dd87cb175334b71c2a30623a5ad55bb44
2021-08-07 10:20:24 +10:00
dtucker@openbsd.org 911ec64118 upstream: Allow for different (but POSIX compliant) behaviour of
basename(3) and prevent a use-after-free in that case in the new sftp-compat
code.

POSIX allows basename(3) to either return a pointer to static storage
or modify the passed string and return a pointer to that.  OpenBSD does
the former and works as is, but on other platforms "filename" points
into "tmp" which was just freed.  This makes the freeing of tmp
consistent with the other variable in the loop.

Pinpointed by the -portable Valgrind regress test.  ok djm@ deraadt@

OpenBSD-Commit-ID: 750f3c19bd4440e4210e30dd5d7367386e833374
2021-08-04 12:04:33 +10:00
Damien Miller 6df1fecb5d use openbsd-compat glob.h is required 2021-08-04 11:05:11 +10:00
djm@openbsd.org 197e29f1cc upstream: support for using the SFTP protocol for file transfers in
scp, via a new "-M sftp" option. Marked as experimental for now.

Some corner-cases exist, in particular there is no attempt to
provide bug-compatibility with scp's weird "double shell" quoting
rules.

Mostly by Jakub Jelen in GHPR#194 with some tweaks by me. ok markus@
Thanks jmc@ for improving the scp.1 bits.

OpenBSD-Commit-ID: 6ce4c9157ff17b650ace571c9f7793d92874051c
2021-08-03 11:03:09 +10:00