Commit Graph

207 Commits

Author SHA1 Message Date
Milas Bowman f94cb49062
test: fix e2e test for privileged builds (#10873)
We cannot guarantee the exact value of `CapEff` across
environments, and this test has started failing some places,
e.g. Docker Desktop, and now GitHub Actions (likely due to
a kernel upgrade on the runners or similar).

By setting `privileged: true` on the build, we're asking for
the `security.insecure` entitlement on the build. A safe
assumption is that will include `CAP_SYS_ADMIN`, which won't
be present otherwise, so mask the `CapEff` value and check
for that.

It's worth noting that realistically, the build won't even
be able to complete without the correct entitlement, since the
`Dockerfile` uses `RUN --security=insecure`, so this is really
an additional sanity check.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-03 12:23:24 -04:00
Milas Bowman f65fd02383
watch: add tar sync implementation (#10853)
Brought to you by Tilt ❤️ 

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-01 14:39:08 -04:00
Guillaume Lours 150b88ab5d
Merge pull request #10829 from milas/e2e-watch-test-fix
test: watch e2e reliability tweaks
2023-07-19 12:07:10 +02:00
Milas Bowman 636c13f818 build: do not attempt to push unnamed service images
When building, if images are being pushed, ensure that only
named images (i.e. services with a populated `image` field)
are attempted to be pushed.

Services without `image` get an auto-generated name, which
will be a "Docker library" reference since they're in the
format `$project-$service`, which is implicitly the same as
`docker.io/library/$project-$service`. A push for that is
never desirable / will always fail.

The key here is that we cannot overwrite the `<svc>.image`
field when doing builds, as we need to be able to check for
its presence to determine whether a push makes sense.

Fixes #10813.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-19 09:58:37 +02:00
Milas Bowman ddceb1ac9d test: do not run watch e2e tests in parallel
This isn't playing nicely with the GHA CI runner.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-18 18:54:23 -04:00
Milas Bowman d48f28c72c test: skip watch e2e test on macOS for the moment
Fix forthcoming via https://github.com/compose-spec/compose-go/pull/436
which addresses some symlink limitations. These can
actually effect other platforms but are most common
on macOS because the test creates temporary directories,
which are symlinked on macOS.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-18 18:53:26 -04:00
Guillaume Lours 2d16a05afa
only check if a dependency is required when something unexpected happens
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-07-18 23:45:31 +02:00
Guillaume Lours bb94ea034e add support of depends_on.required attribute
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-07-18 23:13:47 +02:00
Milas Bowman 3bc871e64b test: speed up the e2e test suite
Lots of our phony Compose files launch pointless long-lived processes
so we can assert on state. However, this means they often don't respond
well to signals on their own, requiring Compose to timeout and kill
them when doing a `down`.

Add in lots of `init: true` where appropriate so that we don't block
for no reason while running E2E tests all over the place.

Additionally, a couple tests have gotten a cleanup so they don't leave
behind containers. I still want to build this into the framework in
the future, but this is easier for the moment and won't cause any
trouble in the future.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-18 11:08:06 +02:00
Milas Bowman 3dc8734897
watch: add end-to-end test (#10801)
Add an end-to-end test that covers the core watch functionality,
i.e. CRUD on files & directories.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-17 10:47:36 -04:00
Guillaume Lours 8339269e13
Merge pull request #10789 from ndeloof/run_no_deps
Apply no-deps before we select and mutate target service
2023-07-10 15:46:01 +02:00
Milas Bowman 8dea7b5cae test: fix process leak in wait e2e test
* Run `down` before and after test to not leave around containers
* Kill the `wait` process that's waiting on `infinity`
  * NOTE: If the test is actually working, this should exit once
    the `down` happens, but this ensures that we kill everything
    we start

I'd like to generalize more of this into the framework, but this
is a quick fix to prevent filling up CI machines with tons of
processes over time.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-10 08:42:09 -04:00
Nicolas De Loof e6a7694b8d
Apply no-deps before we select and mutate target service
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-07-10 14:37:42 +02:00
Milas Bowman be22bc735a network: fix random missing network when service has more than one
As part of the fix for #10668, the logic was adjusted so that the
default (highest-priority) network is used in the `ContainerCreate`,
and then the remaining networks are connected via calls to
`NetworkConnect` before starting the container.

Unfortunately, `ServiceConfig::NetworksByPriority` is neither
deterministic nor stable when networks have the same priority.

It's non-deterministic because the order of networks from parsing
YAML is random, since they are loaded into a Go map (which have
random iteration order). Additionally, it's not using a `SortStable`
in `compose-go`, so even if the load order was predictable, it
still might produce different results.

While I look at improving `compose-go` here to prevent this from
tripping us up in the future, this fix looks at _all_ networks for
a service and ignores the "default" one now. Before, it would
always skip the first one in the slice since that _should_ have
been the "default".

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-07-07 09:18:01 +02:00
Guillaume Lours 28301fb1a4 add support of --builder and BUILDX_BUILDER
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-07-03 10:11:18 +02:00
Ulysses Souza edd76bfd70 Add `docker compose wait`
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
2023-06-30 16:07:03 +02:00
Jes Cok 1a41678c58 fix typos
Signed-off-by: Jes Cok <xigua67damn@gmail.com>
2023-06-27 16:12:25 +02:00
Milas Bowman 061b52da9a ci: build fix for new buildx
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-06-21 11:17:39 +02:00
Milas Bowman e63ab14b1e
ci: merge Go coverage reports before upload (#10666)
Attempting to fix the state of codecov action checks right now,
which are behaving very erratically.

Using the new functionality in Go 1.20 to merge multiple reports,
so now the unit & E2E coverage data reports are stored as artifacts
and then downloaded, merged, and finally uploaded to codecov as a
new job.

Additionally, add a `codecov.yml` config and try to turn down the
aggressiveness of it for CI checks.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-06-08 14:58:21 -04:00
Nicolas De Loof 0d6b99e6f9
e2e test to cover logs -f managing service being added/scaled
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-26 14:51:23 +02:00
Nicolas De Loof 6f6e1635fd compute service hash with a default DeployConfig
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-26 14:15:33 +02:00
Nicolas De Loof 93bd27a0cc introduce ability to select service to be stopped by `compose down`
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-24 17:32:32 +02:00
Nicolas De Loof 9b5a4588f9 introduce --no-path-resolution to skip relative path to be resolved
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-16 13:02:56 +02:00
Nicolas De loof 67455e9f33
fix builkit progressui integration (#10535)
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-08 09:21:41 -04:00
Nicolas De Loof b45ca82791
let user declare build secret target (id)
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-04 10:01:10 +02:00
Milas Bowman 1383ab09ec test: fix E2E tests under Engine v23 / DD 4.19
Some error messages have been tweaked slightly, this adapts the
assertions to work on both Engine v20.10.x and v23.x.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-04-25 08:27:42 +02:00
Nicolas De Loof 981cb2024e
prevent panic using classic builder
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-31 11:59:22 +02:00
Nicolas De Loof c5317496ac
workaround race condition in ContainerList
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-28 12:28:41 +02:00
Milas Bowman d818bf6f34
Merge pull request #10401 from milas/deps-update
ci: upgrade to Go 1.20.2 & bump deps
2023-03-24 11:05:53 -04:00
Milas Bowman cd17c8a950 test: update error message
Validation got improved in `compose-go` so the error message is
slightly different.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-03-24 10:42:43 -04:00
Milas Bowman 36625ed229 test: fix race in e2e build test
This was running two tests in parallel that would build/delete the
same images. Run in serial instead since that's not safe.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-03-24 10:32:55 -04:00
Nicolas De Loof 6c1f06e420 Run classic builder with BuildConfig, not buildx.Options
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-21 15:37:55 +01:00
Nicolas De Loof 88b0d17ff8 use `build` as common API for build scenarios
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-21 15:37:55 +01:00
Nicolas De Loof 754c06886f one off container name use configured Separator for naming
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-15 09:47:43 +01:00
Nicolas De Loof 85ddfde5d6 use go 1.20 -cover support
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-03-10 16:54:39 +00:00
Guillaume Lours 3cfbac6624
restart only needed services by checking depends_on relations
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-03-01 18:22:40 +01:00
Milas Bowman e31b95c16d test: tweak pause test to try and prevent failures in Windows CI
This test keeps failing with a timeout in Windows. I don't actually
think it should take that long to bring up an nginx container, so
I'm guessing that there's something else going on that's causing
trouble.

Increase the verbosity when running Compose commands: I think this
will generally make E2E test failures easier to diagnose by always
logging the full command that's going to be run and also capturing
stdout.

Add a health check and use `--wait` when launching the fixture for
the pause test. Combined with the verbosity increase, this should
make it easier to understand what's going on here.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-02-27 14:20:20 -05:00
Nicolas De Loof 52478f0c6e wait on service containers as dependencies to be deterministic
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-02-07 16:36:45 +01:00
Guillaume Lours fd353ffa94 add support of privileged attribut in service.build section
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-23 09:32:48 +01:00
Guillaume Lours adf8e75317 cleanup framework.go from uncessary debug logs
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-21 21:28:06 +01:00
Guillaume Lours 6c537cc0f4
Merge pull request #10113 from glours/add-buildx-plugin-e2e
add buildx plugin to e2e configuration directory
2022-12-21 21:21:01 +01:00
Nicolas De Loof 9bd9f1765c check service names based on project, not running containers
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-21 21:11:44 +01:00
Guillaume Lours 5dcadc05d9
debut output for CI
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-21 21:09:49 +01:00
Guillaume Lours c72f161afb
change the way finding the just built compose binary
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-21 19:47:39 +01:00
Guillaume Lours 86a648bd51
e2e tests display Compose version used to run the test
currently the version displayed is the one installed and not the one use for the tests

Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-21 18:38:24 +01:00
Guillaume Lours 935968fe2c
add buildx plugin to e2e configuration directory
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2022-12-21 15:27:34 +01:00
Nicolas De Loof 91371fef7a remove flaky TestLocalComposeLogsFollow
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-21 08:45:41 +01:00
Nicolas De Loof 986bc44549 service hash MUST exclude replicas
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-21 08:45:41 +01:00
Nicolas De Loof dacf24374d
dump stdout to help diagnose flaky test
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-20 23:50:58 +01:00
Nicolas De Loof 22d2e83896
don't fail `logs` when driver:none is set
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-20 16:23:47 +01:00