Commit Graph

714 Commits

Author SHA1 Message Date
Milas Bowman f94cb49062
test: fix e2e test for privileged builds (#10873)
We cannot guarantee the exact value of `CapEff` across
environments, and this test has started failing some places,
e.g. Docker Desktop, and now GitHub Actions (likely due to
a kernel upgrade on the runners or similar).

By setting `privileged: true` on the build, we're asking for
the `security.insecure` entitlement on the build. A safe
assumption is that will include `CAP_SYS_ADMIN`, which won't
be present otherwise, so mask the `CapEff` value and check
for that.

It's worth noting that realistically, the build won't even
be able to complete without the correct entitlement, since the
`Dockerfile` uses `RUN --security=insecure`, so this is really
an additional sanity check.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-03 12:23:24 -04:00
Milas Bowman f65fd02383
watch: add tar sync implementation (#10853)
Brought to you by Tilt ❤️ 

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-01 14:39:08 -04:00
Milas Bowman cb17c3c8a6 watch: move sync logic into separate package
Just moving some code around in preparation for an alternative
sync implementation that can do bulk transfers by using `tar`.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-19 12:25:13 +02:00
Sebastiaan van Stijn 4eb43c53fa pkg/compose: RunOneOffContainer: don't use NewStartOptions()
It's no longer used in docker/cli, and doesn't do anything other than
creating an empty struct, so replacing it (as we're planning to
deprecate that function)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 12:13:40 +02:00
Guillaume Lours 150b88ab5d
Merge pull request #10829 from milas/e2e-watch-test-fix
test: watch e2e reliability tweaks
2023-07-19 12:07:10 +02:00
Milas Bowman 1ae191a936 trace: instrument `compose up` at a high-level
* Image pull
* Image build
* Service apply
  * Scale down/up (event)
  * Recreate container (event)
  * Scale up (event)
  * Container start (event)

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-19 11:26:12 +02:00
Nicolas De Loof 47778f8b77 check secret target is an absolute windows path
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-07-19 10:57:22 +02:00
Milas Bowman 636c13f818 build: do not attempt to push unnamed service images
When building, if images are being pushed, ensure that only
named images (i.e. services with a populated `image` field)
are attempted to be pushed.

Services without `image` get an auto-generated name, which
will be a "Docker library" reference since they're in the
format `$project-$service`, which is implicitly the same as
`docker.io/library/$project-$service`. A push for that is
never desirable / will always fail.

The key here is that we cannot overwrite the `<svc>.image`
field when doing builds, as we need to be able to check for
its presence to determine whether a push makes sense.

Fixes #10813.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-19 09:58:37 +02:00
Milas Bowman ddceb1ac9d test: do not run watch e2e tests in parallel
This isn't playing nicely with the GHA CI runner.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-18 18:54:23 -04:00
Milas Bowman d48f28c72c test: skip watch e2e test on macOS for the moment
Fix forthcoming via https://github.com/compose-spec/compose-go/pull/436
which addresses some symlink limitations. These can
actually effect other platforms but are most common
on macOS because the test creates temporary directories,
which are symlinked on macOS.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-18 18:53:26 -04:00
Guillaume Lours 2d16a05afa
only check if a dependency is required when something unexpected happens
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-07-18 23:45:31 +02:00
Guillaume Lours bb94ea034e add support of depends_on.required attribute
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-07-18 23:13:47 +02:00
Milas Bowman 3bc871e64b test: speed up the e2e test suite
Lots of our phony Compose files launch pointless long-lived processes
so we can assert on state. However, this means they often don't respond
well to signals on their own, requiring Compose to timeout and kill
them when doing a `down`.

Add in lots of `init: true` where appropriate so that we don't block
for no reason while running E2E tests all over the place.

Additionally, a couple tests have gotten a cleanup so they don't leave
behind containers. I still want to build this into the framework in
the future, but this is easier for the moment and won't cause any
trouble in the future.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-18 11:08:06 +02:00
Milas Bowman 3dc8734897
watch: add end-to-end test (#10801)
Add an end-to-end test that covers the core watch functionality,
i.e. CRUD on files & directories.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-17 10:47:36 -04:00
Guillaume Lours 8339269e13
Merge pull request #10789 from ndeloof/run_no_deps
Apply no-deps before we select and mutate target service
2023-07-10 15:46:01 +02:00
Guillaume Lours ee6aeed84e
Merge pull request #10700 from ndeloof/attach
support `attach`
2023-07-10 15:17:44 +02:00
Guillaume Lours 7a9dfa4284
Merge pull request #10790 from milas/e2e-process-leak
test: fix process leak in wait e2e test
2023-07-10 15:09:41 +02:00
Milas Bowman 8dea7b5cae test: fix process leak in wait e2e test
* Run `down` before and after test to not leave around containers
* Kill the `wait` process that's waiting on `infinity`
  * NOTE: If the test is actually working, this should exit once
    the `down` happens, but this ensures that we kill everything
    we start

I'd like to generalize more of this into the framework, but this
is a quick fix to prevent filling up CI machines with tons of
processes over time.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-10 08:42:09 -04:00
Nicolas De Loof e6a7694b8d
Apply no-deps before we select and mutate target service
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-07-10 14:37:42 +02:00
Nicolas De Loof 46d936c750 support `attach`
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-07-10 14:34:28 +02:00
Shan Desai fd0e0a2cbd
fix(secrets): file permission value does not comply with spec
closes #10783

Compose Spec mentions that default values for secrets is `0444` aka. world-readable permissions. However, the value was previously set to `0400`. 


Signed-off-by: Shan Desai <shantanoo.desai@gmail.com>
2023-07-07 18:58:21 +02:00
Nicolas De Loof b0af2deb2b when --index is not set select first service container
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-07-07 14:08:24 +02:00
Milas Bowman be22bc735a network: fix random missing network when service has more than one
As part of the fix for #10668, the logic was adjusted so that the
default (highest-priority) network is used in the `ContainerCreate`,
and then the remaining networks are connected via calls to
`NetworkConnect` before starting the container.

Unfortunately, `ServiceConfig::NetworksByPriority` is neither
deterministic nor stable when networks have the same priority.

It's non-deterministic because the order of networks from parsing
YAML is random, since they are loaded into a Go map (which have
random iteration order). Additionally, it's not using a `SortStable`
in `compose-go`, so even if the load order was predictable, it
still might produce different results.

While I look at improving `compose-go` here to prevent this from
tripping us up in the future, this fix looks at _all_ networks for
a service and ignores the "default" one now. Before, it would
always skip the first one in the slice since that _should_ have
been the "default".

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-07 09:18:01 +02:00
Nicolas De Loof b5f5e27597 don't use unitialized cli to setup DryRunClient
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-07-06 14:27:57 +02:00
cui fliter 25ca75db4d fix some comments
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-07-04 11:34:49 +08:00
Guillaume Lours 28301fb1a4 add support of --builder and BUILDX_BUILDER
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-07-03 10:11:18 +02:00
Ulysses Souza edd76bfd70 Add `docker compose wait`
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
2023-06-30 16:07:03 +02:00
Milas Bowman 10b290e682 up: fix race condition on network connect
Engine API only allows at most one network to be connected as
part of the ContainerCreate API request. Compose will pick the
highest priority network.

Afterwards, the remaining networks (if any) are connected before
the container is actually started.

The big change here is that, previously, the highest-priority
network was connected in the create, and then disconnected and
immediately reconnected along with all the others. This was
racy because evidently connecting the container to the network
as part of the create isn't synchronous, so sometimes when Compose
tried to disconnect it, the API would return an error like:
```
container <id> is not connected to the network <network>
```

To avoid needing to disconnect and immediately reconnect, the
network config logic has been refactored to ensure that it sets
up the network config correctly the first time.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-06-29 16:00:55 -04:00
Jes Cok 1a41678c58 fix typos
Signed-off-by: Jes Cok <xigua67damn@gmail.com>
2023-06-27 16:12:25 +02:00
Guillaume Lours 035276e027
watch: add warning when a path is already used by a bind mount volume (#10741)
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-06-26 18:56:04 -04:00
Milas Bowman 061b52da9a ci: build fix for new buildx
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-06-21 11:17:39 +02:00
Milas Bowman 04aa155878 ci: upgrade to buildx v0.11
https://github.com/docker/buildx/releases/tag/v0.11.0

Several `replace` directives have been removed and dependencies
aligned with buildx as needed.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-06-21 11:17:39 +02:00
Guillaume Lours 2d4f8d31fc
Merge pull request #10709 from ndeloof/secret_uid
warn user build.secrets uid,gid,mode are not implemented
2023-06-21 10:19:43 +02:00
Nicolas De Loof a2ce602f6c fix race condition, waiting for containers when one exit
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-06-20 16:17:15 +02:00
Nicolas De Loof c61b8aa5ac introduce run --cap-add to run maintenance commands using service image
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-06-19 14:20:20 +02:00
Nicolas De Loof 7ffe83dc95 don't apply "rebuild" watch strategy by default
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-06-15 15:30:21 +02:00
Nicolas De Loof d20c2551f2 warn user build.secrets uid,gid,mode are not implemented
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-06-14 10:21:20 +02:00
Nicolas De Loof cfe91becc7 use `--progress` to configure progress UI stylet push
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-06-12 16:53:42 +02:00
Milas Bowman e63ab14b1e
ci: merge Go coverage reports before upload (#10666)
Attempting to fix the state of codecov action checks right now,
which are behaving very erratically.

Using the new functionality in Go 1.20 to merge multiple reports,
so now the unit & E2E coverage data reports are stored as artifacts
and then downloaded, merged, and finally uploaded to codecov as a
new job.

Additionally, add a `codecov.yml` config and try to turn down the
aggressiveness of it for CI checks.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-06-08 14:58:21 -04:00
Guillaume Lours 32cf776ecd
Merge pull request #10620 from ndeloof/Building
do not render `Building` when no build is needed
2023-06-08 12:16:46 +02:00
Guillaume Lours 955784c406
Merge pull request #10662 from milas/bump-deps
ci: upgrade to Go 1.20.5 and Moby v24.x
2023-06-08 12:03:23 +02:00
Nicolas De Loof 852c9e80b4 create directory in container using `mkdir -p`
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-06-08 11:28:22 +02:00
Milas Bowman 37850f7955 ci: upgrade to Go 1.20.5 and Moby v24.x
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-06-07 10:41:49 -04:00
Nicolas De Loof 4bf2fe9fed assume we receive logs by lines and don't ignore those without EOL
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-06-07 10:40:37 +02:00
Nicolas De Loof 629c9f62e9 better diagnostic message on network label mismatch
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-06-05 16:54:58 +02:00
Guillaume Lours 7c3fe359b7
Merge pull request #10622 from ndeloof/logs_follow
fix `compose -p x logs -f` detect new services started after command
2023-06-02 09:39:20 +02:00
Nicolas De Loof d2aa15c06e bump buildx and use confutil.ConfigDir
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-06-01 12:09:13 +02:00
Guillaume Lours 6530880361
Merge pull request #10623 from jfly/jfly/tweak-warning-message
Fix typo in warning about existing volume
2023-06-01 08:50:33 +02:00
Nicolas De loof 1bd8a773a7
detect network conflict as name is not guaranteed to be unique (#10612)
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-31 14:46:23 -04:00
Nicolas De Loof fed8ef6b79 forward signal to container
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-31 15:10:11 +02:00