Commit Graph

283 Commits

Author SHA1 Message Date
Guillaume Lours 4debb133a7
add support of environment secret during build step
Signed-off-by: Guillaume Lours <guillaume.lours@docker.com>
2022-07-01 11:21:49 +02:00
Milas Bowman 4aa8c4a1e5 e2e: add more start/stop test cases
* Starting a service that's already running
* Stopping a service that's already stopped
* Starting/stopping multiple services (by name) at once

Also renamed a test that was about `up` behavior but was
misleadingly labeled start/stop.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-30 09:52:14 -04:00
Milas Bowman 06649442eb
Merge pull request #9609 from milas/e2e-pause
e2e: split out `pause` tests and add more cases
2022-06-30 08:42:59 -04:00
Guillaume Lours cd940693aa
Merge pull request #9049 from ndeloof/9044
(re)start should not impact one-off containers
2022-06-29 10:43:10 +02:00
Laura Brehm 96b152f705 Update logic to keep `compose stop` and `restart` from affecting one-offs
Also add e2e tests.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-06-29 00:58:02 +02:00
Milas Bowman 8e7b6582d4
Merge pull request #9595 from abhinavnair/replace-ioutil
Replace deprecated ioutil pkg with os & io
2022-06-28 15:08:57 -04:00
Milas Bowman 26014d49a5 e2e: split out `pause` tests and add more cases
Pause/unpause was being partially tested under the start/stop test.
This removes it from that test and adds dedicated pause + unpause
tests.

Note that the tests assert on current behavior, though it's been
noted where that is undesirable due to divergence from the Docker
CLI. Will change the behavior + update tests in a subsequent PR.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-28 15:06:17 -04:00
Laura Brehm 3ccf4349e7
Merge pull request #9572 from laurazard/dont-wait-for-oneshots
Use appropriate dependency condition for one-shot containers when running `compose up --wait`
2022-06-26 14:48:49 +02:00
Laura Brehm 74fd14ec65 Use alpine instead of ubuntu for E2E compose file, no need to include version
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-06-26 14:41:23 +02:00
Abhinav Nair 11f2f2dbc4
Replace deprecated ioutil pkg with os & io
As of Go 1.16, the same functionality is now provided by package io or
package os, and those implementations should be preferred in new code.

So replacing all usage of ioutil pkg with io & os.

Signed-off-by: Abhinav Nair <11939846+abhinavnair@users.noreply.github.com>
2022-06-25 14:06:10 +08:00
Guillaume Lours 2cd9c0df5a
Merge pull request #9587 from milas/build-order-classic
build: respect dependency order for classic builder
2022-06-24 22:24:22 +02:00
Milas Bowman ec0efec839 test: add copyright notice
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-24 16:16:53 -04:00
Milas Bowman 5f89365c8f network: make test mock consistent throughout
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-22 16:34:59 -04:00
Milas Bowman b2cd089bae build: respect dependency order for classic builder
When using the "classic" (non-BuildKit) builder, ensure that
services are iterated in dependency order for a build so that
it's possible to guarantee the presence of a base image that's
been added as a dependency with `depends_on`. This is a very
common pattern when using base images with Compose.

A fix for BuildKit is blocked currently until we can rely on a
newer version of the engine (see docker/compose#9324)[^1].

[^1]: https://github.com/docker/compose/issues/9232#issuecomment-1060389808

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-22 16:24:30 -04:00
Milas Bowman 0f747419b6 networks: prevent issues due to duplicate names
Within the Docker API/engine, networks have unique IDs, and the
name is a friendly label/alias, which notably does NOT have any
guarantees around uniqueness (see moby/moby#18864 [^1]).

During day-to-day interactive/CLI Compose usage, this is rarely
an issue, as Compose itself isn't creating networks concurrently
across goroutines. However, if multiple Compose instances are
executed simultaneously (e.g. as part of a test suite that runs
in parallel), this can easily occur.

When it does happen, it's very confusing for users and resolving
it via the `docker` CLI is not straightforward either [^2].

There's two primary changes here:
 * Pass `CheckDuplicates: true` to the Docker API when creating
   networks to reduce the likelihood of Compose creating duplicates
   in the first place
 * On `down`, list networks using a name filter and then remove
   them all by ID, as the Docker API will return an error if the
   name alias is used and maps to more than one network

Hopefully, this provides a better UX, since the issue should be
less likely to occur, and if it does, it can now be resolved via
standard Compose workflow commands.

[^1]: https://github.com/moby/moby/issues/18864
[^2]: https://stackoverflow.com/questions/63776518/error-2-matches-found-based-on-name-network-nameofservice-default-is-ambiguo

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-22 12:27:30 -04:00
Laura Brehm fd8538f780 Use appropriate dependency condition for one-shot containers when running `compose up --wait`
Also add e2e tests to ensure `compose up --wait` does not get stuck forever waiting for one-shot containers

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-06-18 01:20:40 +02:00
Milas Bowman 7f441eb013 e2e: improve test output on failures
This is mostly marking a bunch of the run methods as helpers so
that the internal assertions they do will show the line number of
the calling test instead.

There's also some small tweaks around the plugin initialization to
help with the output in the event that it fails to make it easier
to debug what went wrong.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-16 17:05:43 -04:00
Guillaume Lours 4270987d7c
Merge pull request #9567 from milas/e2e-cmd
e2e: ensure all Compose cmds standalone compatible
2022-06-16 23:02:14 +02:00
Milas Bowman 3248b7b335 e2e: use Compose for exec test
This was using `docker exec` on Compose containers instead of
`docker compose exec` (and `docker-compose exec` for standalone).

Thanks to @glours for catching!

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-16 16:44:21 -04:00
Milas Bowman 4ea3ba77b2
Merge pull request #9559 from venthur/fix_spell
fixed some spelling mistakes
2022-06-16 15:56:44 -04:00
Milas Bowman 9622395c8a e2e: ensure all Compose cmds standalone compatible
The E2E tests can be run in plugin (`docker compose`) or standalone
(`docker-compose`) mode. Existing logic was in place to ensure that
the helper method is always used, which will invoke the right one
based on how tests are being executed.

However, this logic was too easy to (unintentionally) bypass given
the myriad of ways that commands can be run. The check has been
made stricter and pushed to a lower-level to more aggressively
catch instances.

As a result, a bunch of calls to `RunDockerCmd` are now updated
to be `RunDockerComposeCmd`, which will ensure that the invocation
is correct based on test mode.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-16 15:51:54 -04:00
Guillaume Lours bec4ea4ab0
Merge pull request #9566 from milas/e2e-ps-stdout
e2e: fix spurious `ps` failures
2022-06-16 18:07:22 +02:00
Guillaume Lours 3f10753178
Merge pull request #9563 from milas/e2e-env
e2e: isolate test command env from system env
2022-06-16 18:05:39 +02:00
Milas Bowman 152c2d9a33 e2e: fix spurious `ps` failures
Use the command `stdout` instead of combined `stdout` + `stderr`
for assertions to avoid failures from any CLI logging such as
warnings, which will be on `stderr`.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-16 11:28:29 -04:00
Milas Bowman a261682ca8 e2e: fix per-command env overrides
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-16 11:27:39 -04:00
Milas Bowman ccd87311e8 e2e: always set HOME + USER for cmd env
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-16 09:44:40 -04:00
Nick Sieger 49fb4ca25f
down: fix COMPOSE_REMOVE_ORPHANS env name and add test
Fixes #9562.

Signed-off-by: Nick Sieger <nick@nicksieger.com>
2022-06-16 08:40:40 -05:00
Milas Bowman 3ae6c52e8a e2e: add extra tools needed for ddev test
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-16 09:38:25 -04:00
Milas Bowman 1c41df8f56 e2e: robustness changes for ddev test
The most important change here is to ensure that the correct Compose
standalone binary is used by `ddev`. Since it invokes Compose itself,
we need to ensure that `PATH` is set appropriately such that it finds
the binary we want to test rather than something from the system.

As part of this, the rest of the environment has been isolated, which
should make the test more reliable, and avoids polluting `~/.ddev`
with test artifacts by using a tmpdir as `HOME` for the test instead
of the user's real home folder.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-16 08:59:02 -04:00
Milas Bowman ea8341865d e2e: isolate test command env from system env
When running Docker / Compose commands, do NOT inherit the system
environment to ensure that the tests are reproducible regardless
of host settings.

Additionally, per-command environment overrides are provided to
the command instead of using `os.SetEnv`, as this is not safe when
running tests in parallel (`testing.T::SetEnv` will actually error
if used in this way!)

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-16 08:30:57 -04:00
Milas Bowman d9065050fd e2e: fix subtests and block parallel unsafe tests
The big mechanical change here is to NOT store `t` as a field on
the `CLI` object (which has been renamed as well to fix the odd
capitalization). The way the tests are structured meant that the
"subtests" were using the _parent_ `*testing.T` instance, which
causes various oddities including confusing messages on failure
about a child test causing a panic in the parent.

Additionally, a few tests have been blocked from running in
parallel because they are sharing `compose.yaml` fixtures and
can fail as a result (e.g. due to a port conflict on the host).
I'll fix these in follow-up PRs but want to focus on correctness
for the tests before optimizing them.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-06-15 16:27:30 -04:00
Nicolas De Loof 005fc25823 add support for setting secret from env variable
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-06-15 17:06:48 +02:00
Bastian Venthur 8a5d555de7 fixed some spelling mistakes
that's all :)

Signed-off-by: Bastian Venthur <venthur@debian.org>
2022-06-14 21:33:36 +02:00
Nick Sieger a48f1e8c13
Merge pull request #9547 from docker/nicksieger/e2e-ps-unmarshal
e2e: unmarshal json into container summaries
2022-06-14 10:33:53 -05:00
Guillaume Lours 57975094cc
Merge pull request #9552 from glours/fix-deploy-cpus-issue
Fix deploy cpus issue and support pids limit
2022-06-14 11:41:35 +02:00
Laura Brehm 80b7a8d274 Only start direct dependencies of service on `compose run ...`
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-06-14 07:32:15 +02:00
Guillaume Lours e111b651b3 add deploy.resourses.limits.pids to the managed resources
Signed-off-by: Guillaume Lours <guillaume.lours@docker.com>
2022-06-13 17:21:48 +02:00
Guillaume Lours 638d78516a parse deploy.resources.limit.cpus as float before converting
Signed-off-by: Guillaume Lours <guillaume.lours@docker.com>
2022-06-13 17:21:48 +02:00
Guillaume Lours 8862f95858
Merge pull request #9514 from ulyssessouza/fix-bindmounts
Fix bind mounts when in project volumes definition
2022-06-09 11:15:24 +02:00
Nick Sieger ebb45b400c
e2e: unmarshal json into container summaries
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2022-06-08 15:13:31 -05:00
Nick Sieger c83133f73b
e2e: add test for ps
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2022-06-08 14:05:20 -05:00
Nick Sieger 3599fc8533
mocks: create mocks for compose api.Service
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2022-06-07 17:09:06 -05:00
Ulysses Souza 919f351b4b Fix bind mounts when in project volumes definition
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
2022-06-02 16:37:24 +02:00
Guillaume Lours d5528f3a54
Merge pull request #9520 from milas/bugfix-network-ambiguity
fix: prevent network name ambiguity
2022-06-02 12:44:12 +02:00
Laura Brehm 81182fca53 Add links to container create request.
In v1, links were sent alongside the rest of the container create request, as part of `HostConfig`. In v2, links are usually set on the connect container to network request that happens after the create. However, this only happens if the service has one or more networks defined for it. If the services are configured to use the default bridge network, this request is not made and so links are never configured.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2022-06-02 02:51:25 +02:00
Milas Bowman 7f32f02817 fix: prevent network name ambiguity
`NetworkInspect` will match a network ID by prefix. While rare,
it's possible that users might use a network name that is also
a valid network ID prefix for a pre-existing Docker network.
(In the reported case, the network was named `db`, for example.)

Fixes #9496.

Signed-off-by: Milas Bowman <milas@tilt.dev>
2022-06-01 15:28:42 -04:00
Guillaume Lours 335decceda
Merge pull request #9504 from docker/nicksieger/9427
fix: bring up services with deps with --no-deps
2022-05-31 23:39:19 +02:00
Nick Sieger 28c0fbfdc0
e2e: reproduce bug with links
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2022-05-31 16:32:29 -05:00
Nick Sieger cacff89cef
fix: bring up services with deps with --no-deps
Don't fail on not finding dependent services because they were put in the
disabled slice.

Fixes #9427.

Signed-off-by: Nick Sieger <nick@nicksieger.com>
2022-05-31 16:19:21 -05:00
Guillaume Lours a1b3f95709
add e2e tests to verify env variables priority
Signed-off-by: Guillaume Lours <guillaume.lours@docker.com>
2022-05-30 17:11:49 +02:00