We cannot guarantee the exact value of `CapEff` across
environments, and this test has started failing some places,
e.g. Docker Desktop, and now GitHub Actions (likely due to
a kernel upgrade on the runners or similar).
By setting `privileged: true` on the build, we're asking for
the `security.insecure` entitlement on the build. A safe
assumption is that will include `CAP_SYS_ADMIN`, which won't
be present otherwise, so mask the `CapEff` value and check
for that.
It's worth noting that realistically, the build won't even
be able to complete without the correct entitlement, since the
`Dockerfile` uses `RUN --security=insecure`, so this is really
an additional sanity check.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
When building, if images are being pushed, ensure that only
named images (i.e. services with a populated `image` field)
are attempted to be pushed.
Services without `image` get an auto-generated name, which
will be a "Docker library" reference since they're in the
format `$project-$service`, which is implicitly the same as
`docker.io/library/$project-$service`. A push for that is
never desirable / will always fail.
The key here is that we cannot overwrite the `<svc>.image`
field when doing builds, as we need to be able to check for
its presence to determine whether a push makes sense.
Fixes#10813.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Fix forthcoming via https://github.com/compose-spec/compose-go/pull/436
which addresses some symlink limitations. These can
actually effect other platforms but are most common
on macOS because the test creates temporary directories,
which are symlinked on macOS.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Lots of our phony Compose files launch pointless long-lived processes
so we can assert on state. However, this means they often don't respond
well to signals on their own, requiring Compose to timeout and kill
them when doing a `down`.
Add in lots of `init: true` where appropriate so that we don't block
for no reason while running E2E tests all over the place.
Additionally, a couple tests have gotten a cleanup so they don't leave
behind containers. I still want to build this into the framework in
the future, but this is easier for the moment and won't cause any
trouble in the future.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Add an end-to-end test that covers the core watch functionality,
i.e. CRUD on files & directories.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
* Run `down` before and after test to not leave around containers
* Kill the `wait` process that's waiting on `infinity`
* NOTE: If the test is actually working, this should exit once
the `down` happens, but this ensures that we kill everything
we start
I'd like to generalize more of this into the framework, but this
is a quick fix to prevent filling up CI machines with tons of
processes over time.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
As part of the fix for #10668, the logic was adjusted so that the
default (highest-priority) network is used in the `ContainerCreate`,
and then the remaining networks are connected via calls to
`NetworkConnect` before starting the container.
Unfortunately, `ServiceConfig::NetworksByPriority` is neither
deterministic nor stable when networks have the same priority.
It's non-deterministic because the order of networks from parsing
YAML is random, since they are loaded into a Go map (which have
random iteration order). Additionally, it's not using a `SortStable`
in `compose-go`, so even if the load order was predictable, it
still might produce different results.
While I look at improving `compose-go` here to prevent this from
tripping us up in the future, this fix looks at _all_ networks for
a service and ignores the "default" one now. Before, it would
always skip the first one in the slice since that _should_ have
been the "default".
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Attempting to fix the state of codecov action checks right now,
which are behaving very erratically.
Using the new functionality in Go 1.20 to merge multiple reports,
so now the unit & E2E coverage data reports are stored as artifacts
and then downloaded, merged, and finally uploaded to codecov as a
new job.
Additionally, add a `codecov.yml` config and try to turn down the
aggressiveness of it for CI checks.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Some error messages have been tweaked slightly, this adapts the
assertions to work on both Engine v20.10.x and v23.x.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
This was running two tests in parallel that would build/delete the
same images. Run in serial instead since that's not safe.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
This test keeps failing with a timeout in Windows. I don't actually
think it should take that long to bring up an nginx container, so
I'm guessing that there's something else going on that's causing
trouble.
Increase the verbosity when running Compose commands: I think this
will generally make E2E test failures easier to diagnose by always
logging the full command that's going to be run and also capturing
stdout.
Add a health check and use `--wait` when launching the fixture for
the pause test. Combined with the verbosity increase, this should
make it easier to understand what's going on here.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
currently the version displayed is the one installed and not the one use for the tests
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
The scan tip has been shown for two years, and most users will know
about it by now. Presenting the message also involved checking if the
plugin was installed, and wether or not the message was shown before,
which also caused some overhead, so cleaning up the output a bit.
The corresponding DOCKER_SCAN_SUGGEST environment-variable is also
removed with this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
See compose-spec/compose-go#325 for the acutal spec change. This
propagates it to the Engine API object and adds an E2E test via
Cucumber 🥒Fixes#9873.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
When running `compose down`, the `--rmi` flag can be passed,
which currently supports two values:
* `local`: remove any _implicitly-named_ images that Compose
built
* `all` : remove any named images (locally-built or fetched
from a remote repo)
Removing images in the `local` case can be problematic, as it's
historically been done via a fair amount of inference over the
Compose model. Additionally, when using the "project-model"
(by passing `--project-name` instead of using a Compose file),
we're even more limited: if no containers for the project are
running, there's nothing to derive state from to perform the
inference on.
As a first pass, we started labeling _containers_ with the name
of the locally-built image associated with it (if any) in #9715.
Unfortunately, this still suffers from the aforementioned problems
around using actual state (i.e. the containers might no longer
exist) and meant that when operating in file mode (the default),
things did not behave as expected: the label is not available
in the project since it only exists at runtime.
Now, with these changes, Compose will label any images it builds
with project metadata. Upon cleanup during `down`, the engine
image API is queried for related images and matched up with the
services for the project. As a fallback for images built with
prior versions of Compose, the previous approach is still taken.
See also:
* https://github.com/docker/compose/issues/9655
* https://github.com/docker/compose/pull/9715
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
support DOCKER_DEFAULT_PLATFORM when 'compose up --build'
add tests to check behaviour when DOCKER_DEFAULT_PLATFORM is defined
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
* update dockerfiles to use latest stable syntax
Some Dockerfiles were pinned to a minor release, which meant they
wouldn't be updated to get the latest stable syntax (and fixes),
and one Dockerfile used the "labs" variant to use the HEREDOC syntax,
which has now been promoted to the stable syntax.
* docs: rename Dockerfile
There's no other Dockerfiles in the same path, so the "docs"
prefix was redundant.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This keeps parity with v1, where only the containers explicitly
passed to `up` are torn down when `Ctrl-C` is hit, so any
dependencies that got launched (or orphan containers hanging
around) should not be touched.
Fixes#9696.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
* Starting a service that's already running
* Stopping a service that's already stopped
* Starting/stopping multiple services (by name) at once
Also renamed a test that was about `up` behavior but was
misleadingly labeled start/stop.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Pause/unpause was being partially tested under the start/stop test.
This removes it from that test and adds dedicated pause + unpause
tests.
Note that the tests assert on current behavior, though it's been
noted where that is undesirable due to divergence from the Docker
CLI. Will change the behavior + update tests in a subsequent PR.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
As of Go 1.16, the same functionality is now provided by package io or
package os, and those implementations should be preferred in new code.
So replacing all usage of ioutil pkg with io & os.
Signed-off-by: Abhinav Nair <11939846+abhinavnair@users.noreply.github.com>
When using the "classic" (non-BuildKit) builder, ensure that
services are iterated in dependency order for a build so that
it's possible to guarantee the presence of a base image that's
been added as a dependency with `depends_on`. This is a very
common pattern when using base images with Compose.
A fix for BuildKit is blocked currently until we can rely on a
newer version of the engine (see docker/compose#9324)[^1].
[^1]: https://github.com/docker/compose/issues/9232#issuecomment-1060389808
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Also add e2e tests to ensure `compose up --wait` does not get stuck forever waiting for one-shot containers
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This is mostly marking a bunch of the run methods as helpers so
that the internal assertions they do will show the line number of
the calling test instead.
There's also some small tweaks around the plugin initialization to
help with the output in the event that it fails to make it easier
to debug what went wrong.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
This was using `docker exec` on Compose containers instead of
`docker compose exec` (and `docker-compose exec` for standalone).
Thanks to @glours for catching!
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
The E2E tests can be run in plugin (`docker compose`) or standalone
(`docker-compose`) mode. Existing logic was in place to ensure that
the helper method is always used, which will invoke the right one
based on how tests are being executed.
However, this logic was too easy to (unintentionally) bypass given
the myriad of ways that commands can be run. The check has been
made stricter and pushed to a lower-level to more aggressively
catch instances.
As a result, a bunch of calls to `RunDockerCmd` are now updated
to be `RunDockerComposeCmd`, which will ensure that the invocation
is correct based on test mode.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Use the command `stdout` instead of combined `stdout` + `stderr`
for assertions to avoid failures from any CLI logging such as
warnings, which will be on `stderr`.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Milas Bowman
Guillaume Lours
Nicolas De Loof
Ulysses Souza
Jes Cok
Sebastiaan van Stijn
Laura Brehm
Laura Brehm
Bartłomiej Klimczak
Lucas Berg
Guillaume Lours
Nick Sieger
Vedant Koditkar
Vedant Koditkar
CrazyMax
Sebastiaan van Stijn
Abhinav Nair