Commit Graph

251 Commits

Author SHA1 Message Date
Guillaume Lours 2d16a05afa
only check if a dependency is required when something unexpected happens
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-07-18 23:45:31 +02:00
Guillaume Lours bb94ea034e add support of depends_on.required attribute
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-07-18 23:13:47 +02:00
Milas Bowman 3bc871e64b test: speed up the e2e test suite
Lots of our phony Compose files launch pointless long-lived processes
so we can assert on state. However, this means they often don't respond
well to signals on their own, requiring Compose to timeout and kill
them when doing a `down`.

Add in lots of `init: true` where appropriate so that we don't block
for no reason while running E2E tests all over the place.

Additionally, a couple tests have gotten a cleanup so they don't leave
behind containers. I still want to build this into the framework in
the future, but this is easier for the moment and won't cause any
trouble in the future.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-18 11:08:06 +02:00
Milas Bowman 3dc8734897
watch: add end-to-end test (#10801)
Add an end-to-end test that covers the core watch functionality,
i.e. CRUD on files & directories.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-17 10:47:36 -04:00
Guillaume Lours 8339269e13
Merge pull request #10789 from ndeloof/run_no_deps
Apply no-deps before we select and mutate target service
2023-07-10 15:46:01 +02:00
Milas Bowman 8dea7b5cae test: fix process leak in wait e2e test
* Run `down` before and after test to not leave around containers
* Kill the `wait` process that's waiting on `infinity`
  * NOTE: If the test is actually working, this should exit once
    the `down` happens, but this ensures that we kill everything
    we start

I'd like to generalize more of this into the framework, but this
is a quick fix to prevent filling up CI machines with tons of
processes over time.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-10 08:42:09 -04:00
Nicolas De Loof e6a7694b8d
Apply no-deps before we select and mutate target service
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-07-10 14:37:42 +02:00
Milas Bowman be22bc735a network: fix random missing network when service has more than one
As part of the fix for #10668, the logic was adjusted so that the
default (highest-priority) network is used in the `ContainerCreate`,
and then the remaining networks are connected via calls to
`NetworkConnect` before starting the container.

Unfortunately, `ServiceConfig::NetworksByPriority` is neither
deterministic nor stable when networks have the same priority.

It's non-deterministic because the order of networks from parsing
YAML is random, since they are loaded into a Go map (which have
random iteration order). Additionally, it's not using a `SortStable`
in `compose-go`, so even if the load order was predictable, it
still might produce different results.

While I look at improving `compose-go` here to prevent this from
tripping us up in the future, this fix looks at _all_ networks for
a service and ignores the "default" one now. Before, it would
always skip the first one in the slice since that _should_ have
been the "default".

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-07 09:18:01 +02:00
Guillaume Lours 28301fb1a4 add support of --builder and BUILDX_BUILDER
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-07-03 10:11:18 +02:00
Ulysses Souza edd76bfd70 Add `docker compose wait`
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
2023-06-30 16:07:03 +02:00
Jes Cok 1a41678c58 fix typos
Signed-off-by: Jes Cok <xigua67damn@gmail.com>
2023-06-27 16:12:25 +02:00
Milas Bowman 061b52da9a ci: build fix for new buildx
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-06-21 11:17:39 +02:00
Milas Bowman e63ab14b1e
ci: merge Go coverage reports before upload (#10666)
Attempting to fix the state of codecov action checks right now,
which are behaving very erratically.

Using the new functionality in Go 1.20 to merge multiple reports,
so now the unit & E2E coverage data reports are stored as artifacts
and then downloaded, merged, and finally uploaded to codecov as a
new job.

Additionally, add a `codecov.yml` config and try to turn down the
aggressiveness of it for CI checks.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-06-08 14:58:21 -04:00
Nicolas De Loof 0d6b99e6f9
e2e test to cover logs -f managing service being added/scaled
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-26 14:51:23 +02:00
Nicolas De Loof 6f6e1635fd compute service hash with a default DeployConfig
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-26 14:15:33 +02:00
Nicolas De Loof 93bd27a0cc introduce ability to select service to be stopped by `compose down`
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-24 17:32:32 +02:00
Nicolas De Loof 9b5a4588f9 introduce --no-path-resolution to skip relative path to be resolved
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-16 13:02:56 +02:00
Nicolas De loof 67455e9f33
fix builkit progressui integration (#10535)
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-08 09:21:41 -04:00
Nicolas De Loof b45ca82791
let user declare build secret target (id)
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-04 10:01:10 +02:00
Milas Bowman 1383ab09ec test: fix E2E tests under Engine v23 / DD 4.19
Some error messages have been tweaked slightly, this adapts the
assertions to work on both Engine v20.10.x and v23.x.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-04-25 08:27:42 +02:00
Nicolas De Loof 981cb2024e
prevent panic using classic builder
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-31 11:59:22 +02:00
Nicolas De Loof c5317496ac
workaround race condition in ContainerList
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-28 12:28:41 +02:00
Milas Bowman d818bf6f34
Merge pull request #10401 from milas/deps-update
ci: upgrade to Go 1.20.2 & bump deps
2023-03-24 11:05:53 -04:00
Milas Bowman cd17c8a950 test: update error message
Validation got improved in `compose-go` so the error message is
slightly different.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-03-24 10:42:43 -04:00
Milas Bowman 36625ed229 test: fix race in e2e build test
This was running two tests in parallel that would build/delete the
same images. Run in serial instead since that's not safe.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-03-24 10:32:55 -04:00
Nicolas De Loof 6c1f06e420 Run classic builder with BuildConfig, not buildx.Options
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-21 15:37:55 +01:00
Nicolas De Loof 88b0d17ff8 use `build` as common API for build scenarios
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-21 15:37:55 +01:00
Nicolas De Loof 754c06886f one off container name use configured Separator for naming
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-15 09:47:43 +01:00
Nicolas De Loof 85ddfde5d6 use go 1.20 -cover support
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-10 16:54:39 +00:00
Guillaume Lours 3cfbac6624
restart only needed services by checking depends_on relations
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-03-01 18:22:40 +01:00
Milas Bowman e31b95c16d test: tweak pause test to try and prevent failures in Windows CI
This test keeps failing with a timeout in Windows. I don't actually
think it should take that long to bring up an nginx container, so
I'm guessing that there's something else going on that's causing
trouble.

Increase the verbosity when running Compose commands: I think this
will generally make E2E test failures easier to diagnose by always
logging the full command that's going to be run and also capturing
stdout.

Add a health check and use `--wait` when launching the fixture for
the pause test. Combined with the verbosity increase, this should
make it easier to understand what's going on here.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-02-27 14:20:20 -05:00
Nicolas De Loof 52478f0c6e wait on service containers as dependencies to be deterministic
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-02-07 16:36:45 +01:00
Guillaume Lours fd353ffa94 add support of privileged attribut in service.build section
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-23 09:32:48 +01:00
Guillaume Lours adf8e75317 cleanup framework.go from uncessary debug logs
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-21 21:28:06 +01:00
Guillaume Lours 6c537cc0f4
Merge pull request #10113 from glours/add-buildx-plugin-e2e
add buildx plugin to e2e configuration directory
2022-12-21 21:21:01 +01:00
Nicolas De Loof 9bd9f1765c check service names based on project, not running containers
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-21 21:11:44 +01:00
Guillaume Lours 5dcadc05d9
debut output for CI
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-21 21:09:49 +01:00
Guillaume Lours c72f161afb
change the way finding the just built compose binary
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-21 19:47:39 +01:00
Guillaume Lours 86a648bd51
e2e tests display Compose version used to run the test
currently the version displayed is the one installed and not the one use for the tests

Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-21 18:38:24 +01:00
Guillaume Lours 935968fe2c
add buildx plugin to e2e configuration directory
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-21 15:27:34 +01:00
Nicolas De Loof 91371fef7a remove flaky TestLocalComposeLogsFollow
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-21 08:45:41 +01:00
Nicolas De Loof 986bc44549 service hash MUST exclude replicas
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-21 08:45:41 +01:00
Nicolas De Loof dacf24374d
dump stdout to help diagnose flaky test
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-20 23:50:58 +01:00
Nicolas De Loof 22d2e83896
don't fail `logs` when driver:none is set
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-20 16:23:47 +01:00
Nicolas De Loof b8bbdcd872 detect dependency failed to start
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-19 16:26:39 +01:00
Nicolas De Loof c1ce53c972 fix regression running pull --ignore-pull-failures
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-19 11:56:03 +01:00
Nicolas De Loof e42673daed only list running containers when --all=false
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-16 21:46:55 +01:00
Nicolas De Loof 5c1484ece6 apply uid/gid when creating secret from environment
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-15 15:54:04 +01:00
Nicolas De Loof 8c39b5b7fd align `--format` flag and UX with docker cli
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-14 22:53:43 +01:00
Nicolas De Loof 0368f19030 distinguish stdout and stderr in `up` logs
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-13 14:42:42 +01:00
Nicolas De Loof 3ee2ab87bb ContainerStart must run sequentially for engine to assing distinct ports within configured range
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-12 16:08:50 +01:00
Nicolas De Loof 804d7163a7 detect required service are gone to stop watching
explicit API to stop the log printer

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-07 21:07:27 +01:00
Sebastiaan van Stijn 87a0a57f70 Cleanup tips from output
The scan tip has been shown for two years, and most users will know
about it by now. Presenting the message also involved checking if the
plugin was installed, and wether or not the message was shown before,
which also caused some overhead, so cleaning up the output a bit.

The corresponding DOCKER_SCAN_SUGGEST environment-variable is also
removed with this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-06 23:29:03 +01:00
Guillaume Lours 95bc6c58b7
check only running containers in after down tests of profiles e2e tests
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-06 23:17:14 +01:00
Milas Bowman 113fb6732d
schema: add support for tmpfs.mode in mount definition (#10031)
See compose-spec/compose-go#325 for the acutal spec change. This
propagates it to the Engine API object and adds an E2E test via
Cucumber 🥒

Fixes #9873.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-12-02 11:21:53 -05:00
Guillaume Lours 707d55c77f add file header and cleanup profiles e2e tests
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-11-30 13:21:16 +01:00
Guillaume Lours 5edd783032 add e2e tests to check profile activation via targeted service
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-11-30 13:21:16 +01:00
Guillaume Lours 6fbef29619 add e2e tests to check no profile usages
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-11-30 13:21:16 +01:00
Guillaume Lours 7fe43a8b4a add e2e tests using explicitly profiles
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-11-30 13:21:16 +01:00
Guillaume Lours 8c1e2af3e1 add e2e tests to check build dependency between services
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-11-15 14:43:53 +01:00
Laura Brehm dd13299ede
Skip flaky test in CI
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-10-20 18:30:27 +02:00
Ulysses Souza 91eae4f035 Add Codecov
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
2022-10-17 15:32:51 +02:00
Laura Brehm 25c4bcef85
Merge pull request #9824 from laurazard/cucumber-test
🥒 Cucumber PoC 🥒
2022-09-27 23:38:44 +02:00
Laura Brehm fffe7fff57
Create new `e2e` module to separate out test dependencies, move cucumber tests
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-09-27 02:13:52 +02:00
Laura Brehm 0a5f4e62e4
Removed tests that were replaced by Cucumber features
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-09-27 02:13:52 +02:00
Laura Brehm 266ab22d53
Rename start cucumber feature
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-09-27 02:13:51 +02:00
Laura Brehm a7476c8eeb
Convert `cascade_stop_test.go` into a cucumber feature `stop.feature`
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-09-27 02:13:51 +02:00
Laura Brehm 15ebff00b1
Cucumber test setup/fixtures
(run with `go test -v -run ^TestCucumberFeatures$ github.com/docker/compose/v2/pkg/e2e/cucumber`)

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-09-27 02:13:47 +02:00
Bartłomiej Klimczak 0d0a02cc6b
add more information when service.platform isn't part of service.build.platforms
Signed-off-by: Bartłomiej Klimczak <bartlomiej.klimczak88@gmail.com>
2022-09-26 20:44:59 +02:00
Laura Brehm 94465d57cc
Merge pull request #9863 from docker/gha-win-mac-runners
Add `merge` GitHub Actions workflow to run tests on Windows and macOS runners
2022-09-21 16:39:27 +02:00
Laura Brehm a1984ca1de
Skip some tests in CI due to flakiness
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-09-20 11:33:31 -04:00
Laura Brehm 118b4f07e5
Increase E2E test timeouts to reduce flakiness
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-09-20 11:33:31 -04:00
Laura Brehm 8714f983ac
Temporarily disable broken E2E tests on Windows
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-09-20 11:33:31 -04:00
Laura Brehm 937fa2dc8f
Add GitHub Action workflow to run tests on Mac/Windows runners
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-09-20 11:33:28 -04:00
Guillaume Lours 0e975262da
keep the platform defined at service level during build if no build platforms provided
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-09-15 08:30:52 +02:00
Milas Bowman 403d691abf small cleanup + godoc
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-09-13 18:29:33 +01:00
Milas Bowman b49b9ffe7e Merge remote-tracking branch 'upstream/v2' into down-image-rm 2022-09-13 18:00:41 +01:00
Milas Bowman 680763f8b7 down: refactor image pruning
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-09-13 17:23:44 +01:00
Lucas Berg 7a8d157871
convert: do not escape $ into $$ when using the --no-interpolate option (#9703)
Signed-off-by: Lucas Berg <root.lucasberg@gmail.com>
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
Co-authored-by: Ulysses Souza <ulyssessouza@gmail.com>
2022-09-08 16:25:23 -04:00
Laura Brehm 88df5ede42
Merge pull request #9797 from laurazard/start-only-services
Only attempt to start specified services on `compose start [services]`
2022-09-08 13:00:20 -04:00
Laura Brehm a7cc406187
Cleanup E2E tests
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-09-08 12:49:42 -04:00
Milas Bowman bc806da712 build: label built images for reliable cleanup on `down`
When running `compose down`, the `--rmi` flag can be passed,
which currently supports two values:
 * `local`: remove any _implicitly-named_ images that Compose
            built
 * `all`  : remove any named images (locally-built or fetched
            from a remote repo)

Removing images in the `local` case can be problematic, as it's
historically been done via a fair amount of inference over the
Compose model. Additionally, when using the "project-model"
(by passing `--project-name` instead of using a Compose file),
we're even more limited: if no containers for the project are
running, there's nothing to derive state from to perform the
inference on.

As a first pass, we started labeling _containers_ with the name
of the locally-built image associated with it (if any) in #9715.
Unfortunately, this still suffers from the aforementioned problems
around using actual state (i.e. the containers might no longer
exist) and meant that when operating in file mode (the default),
things did not behave as expected: the label is not available
in the project since it only exists at runtime.

Now, with these changes, Compose will label any images it builds
with project metadata. Upon cleanup during `down`, the engine
image API is queried for related images and matched up with the
services for the project. As a fallback for images built with
prior versions of Compose, the previous approach is still taken.

See also:
 * https://github.com/docker/compose/issues/9655
 * https://github.com/docker/compose/pull/9715

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-09-07 17:57:29 -04:00
Laura Brehm 361194472e
Cleanup E2E tests
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-09-06 20:28:35 +02:00
Guillaume Lours e016faac33 don't push images at the end of multi-arch build (and simplify e2e tests)
support DOCKER_DEFAULT_PLATFORM when 'compose up --build'
add tests to check behaviour when DOCKER_DEFAULT_PLATFORM is defined

Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-09-02 15:44:55 +02:00
Guillaume Lours 8ed2d8ad07 add a test with multiple service builds using platforms in the same compose file
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-09-02 15:44:55 +02:00
Guillaume Lours 537f023a3b fix panic when using 'compose up --build'
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-09-02 15:44:55 +02:00
Guillaume Lours 8b1b70833e add support of platforms in build section
Signed-off-by: Guillaume Lours <guillaume.lours@docker.com>
2022-09-02 15:44:55 +02:00
Laura Brehm cc4f194295
Add E2E tests for starting/stopping single services
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-08-30 12:55:51 +02:00
Laura Brehm 0d2beddf20
Add E2E tests for `up --exit-code-from`
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-08-27 17:44:08 +02:00
Laura Brehm 235734823e
Pull image regardless of whether it exists locally if `tag=latest`
... and pull policy is `missing` or `if_not_present`

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-08-23 16:57:40 +02:00
Nick Sieger 5723dee316
pull: only skip pull when policy is `missing`/`if_not_present`
Fixes #9773.

Signed-off-by: Nick Sieger <nick@nicksieger.com>
2022-08-22 09:22:20 -05:00
Vedant Koditkar b96b5449e5 Merge branch 'v2' into 8768-avoid-pulling-same-image-multiple-times 2022-08-19 13:08:36 +05:30
Ulysses Souza 3f4f4e5975 Give environment variables precedence back to OS over .env
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
2022-08-18 22:39:21 +02:00
Nick Sieger 09e6b0292a
up/start/run: don't wait for disabled service
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2022-08-17 12:32:10 -05:00
Vedant Koditkar 293cf21c58 Merge branch '8768-avoid-pulling-same-image-multiple-times' of github.com:KoditkarVedant/compose into 8768-avoid-pulling-same-image-multiple-times 2022-08-14 16:52:17 +05:30
Vedant Koditkar 25f4cb2ee6 Use compose to pull image twice
Signed-off-by: Vedant Koditkar <vedant.koditkar@outlook.com>
2022-08-14 16:51:29 +05:30
Vedant Koditkar 817e875cbf
Merge branch 'v2' into 8768-avoid-pulling-same-image-multiple-times 2022-08-13 12:19:49 +05:30
Vedant Koditkar 79ed1290a6 Use alpine:3.13.12 to be unique across the test cases
Signed-off-by: Vedant Koditkar <vedant.koditkar@outlook.com>
2022-08-13 02:59:44 +05:30
Vedant Koditkar 4853ace155 Fix package name
Signed-off-by: Vedant Koditkar <vedant.koditkar@outlook.com>
2022-08-13 02:06:28 +05:30
Vedant Koditkar de49bea774 Try changing package used to avoid any race condition in tests
Signed-off-by: Vedant Koditkar <vedant.koditkar@outlook.com>
2022-08-12 23:06:51 +05:30