Build{x,kit} support passing in source policies via an (expirimental)
env var.
This change adds those policies to the build request.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Previously, if a long-lived plugin process (such as
an execution of `compose up`) was running and then
detached from a terminal, signalling the parent CLI
process to exit would leave the plugin process behind.
To address this, changes were introduced on the CLI side
(see: https://github.com/docker/cli/pull/4599) to enable
the CLI to notify a running plugin process that it should
exit. This makes it so that, when the parent CLI process
is going to exit, the command context of the plugin
command being executed is cancelled.
This commit takes advantage of these changes by tapping into
the command context's done channel and using it to teardown
on an up.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
The code used an atomic bool to guard channel writes. However, this
failed to synchronize with the call to close(), causing a panic.
Fix the race condition by using a mutex to guard the update to the
bool `stopped` and subsequent channel writes. This ensures atomic
execution of both updates to `stopped` and channel writes, preventing
races between writes and close().
Signed-off-by: horus <horus.li@gmail.com>
Related to:
- https://github.com/compose-spec/compose-spec/pull/435
- https://github.com/moby/moby/pull/45905
Since API v1.44, Moby supports a per-endpoint MAC address and returns a
warning when the container-wide mac_address field is set.
A corresponding field has been added to compose-spec and compose-go, so
we need to leverage it to set the right API field.
This commit is backward-compatible with compose files that still set the
container-wide mac_address field, and older API versions that don't know
about the endpoint-specific MAC address field.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Currently, we publish Compose artifacts following the OCI 1.1
specification, which is still in the RC state.
As a result, not all registries support it yet. Most notably,
AWS ECR will reject certain OCI 1.1-compliant requests with
`405 Method Not Supported` with cryptic `Invalid JSON` errors.
This adds initial support for Compose to generate either an
OCI 1.0 or OCI 1.1 compatible manifest. Notably, the OCI 1.0
manifest will be missing the `application/vnd.docker.compose.project`
artifact type, as that does not exist in that version of the
spec. (Less importantly, it uses an empty `ImageConfig`
instead of the newer `application/vnd.oci.empty.v1+json` media
type for the config.)
Currently, this is not exposed as an option (via CLI flags or
env vars). By default, OCI 1.1 is used unless the registry
domain is `amazonaws.com`, which indicates an ECR registry, so
Compose will instead use OCI 1.0.
Moving forward, we should decide how much we want to expose/
support different OCI versions and investigate if there's a
more generic way to feature probe the registry to avoid
maintaining a hardcoded list of domains, which is both tedious
and insufficient.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
- added clarity with error handling. added test to show issue.
- in manual testing, this fixes the issue and allows watch to run after rebuild
- added cleanup back in
- fixed issue where watch extnet rebuild test would start all containers listed in the fixture
Signed-off-by: kimdcottrell <me@kimdcottrell.com>
From the Go specification [1]:
"1. For a nil slice, the number of iterations is 0."
`len` returns 0 if the slice is nil [2]. Therefore, checking
`len(v) > 0` before a loop is unnecessary.
[1]: https://go.dev/ref/spec#For_range
[2]: https://pkg.go.dev/builtin#len
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Eliminate direct dependency on gopkg.in/yaml.v2
* Add gopkg.in/yaml.v2 as a restricted import
* Add github.com/distribution/distribution as a restricted dependency in favor of distribution/reference which is the subset of functionality that Compose needs
* Remove an unused exclusion
NOTE: This does change the `compose config` output slightly but does NOT change the semantics:
* YAML indentation is slightly different for lists (this is a `v2` / `v3` thing)
* JSON is now "minified" instead of pretty-printed (I think this generally desirable and more consistent with other JSON command outputs)
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
The `alpha watch` command current "attaches" to an already-running
Compose project, so it's necessary to run something like
`docker compose up --wait` first.
Now, we'll do the equivalent of an `up --build` before starting the
watch, so that we know the project is up-to-date and running.
Additionally, unlike an interactive `up`, the services are not stopped
when `watch` exits (e.g. via `Ctrl-C`). This prevents the need to start
from scratch each time the command is run - if some services are already
running and up-to-date, they can be used as-is. A `down` can always be
used to destroy everything, and we can consider introducing a flag like
`--down-on-exit` to `watch` or changing the default.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
The big change here is to pass around an explicit `*BuildOptions` object
as part of Compose operations like `up` & `run` that may or may not do
builds. If the options object is `nil`, no builds whatsoever will be
attempted.
Motivation is to allow for partial rebuilds in the context of an `up`
for watch. This was broken and tricky to accomplish because various parts
of the Compose APIs mutate the `*Project` for convenience in ways that
make it unusable afterwards. (For example, it might set `service.Build = nil`
because it's not going to build that service right _then_. But we might
still want to build it later!)
NOTE: This commit does not actually touch the watch logic. This is all
in preparation to make it possible.
As part of this, a bunch of code moved around and I eliminated a bunch
of partially redundant logic, mostly around multi-platform. Several
edge cases have been addressed as part of this:
* `DOCKER_DEFAULT_PLATFORM` was _overriding_ explicitly set platforms
in some cases, this is no longer true, and it behaves like the Docker
CLI now
* It was possible for Compose to build an image for one platform and
then try to run it for a different platform (and fail)
* Errors are no longer returned if a local image exists but for the
wrong platform - the correct platform will be fetched/built (if
possible).
Because there's a LOT of subtlety and tricky logic here, I've also tried
to add an excessive amount of explanatory comments.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
If running `up` in foreground mode (i.e. not `-d`),
when exiting via `Ctrl-C`, Compose stops all the
services it launched directly as part of that `up`
command.
In one of the E2E tests (`TestUpDependenciesNotStopped`),
this was occasionally flaking because the stop
behavior was racy: the return might not block on
the stop operation because it gets added to the
error group in a goroutine. As a result, it was
possible for no services to get terminated on exit.
There were a few other related pieces here that
I uncovered and tried to fix while stressing this.
For example, the printer could cause a deadlock if
an event was sent to it after it stopped.
Also, an error group wasn't really appropriate here;
each goroutine is a different operation for printing,
signal-handling, etc. If one part fails, we don't
actually want printing to stop, for example. This has
been switched to a `multierror.Group`, which has the
same API but coalesces errors instead of canceling a
context the moment the first one fails and returning
that single error.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
The uuid package in distribution was created as a utility for the distribution
project itself, to cut down external dependencies (see [1][1]).
For compose, this has the reverse effect, as it now brings all the dependencies
of the distribution module with it.
This patch switches to the uuid generation to crypto/rand to produce a random
id. I was considering using a different uuid implementation, or docker's
"stringid.GenerateRandomID", but all of those are doing more than needed,
so keep it simple.
Currently, this change has little effect, because compose also uses the
distribution module for other purposes, but the distribution project is
in the process of moving the "reference" package to a separate module,
in which case we don't want to depend on the distribution module only for
the uuid package.
[1]: 36e34a55ad
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Use unique project name prefixes (some of these tests assert
on output using the project name as a magic string, so could
be impacted by other tests with the same project name prefix)
* Tear down port range project before starting to try and avoid
race conditions with the engine and port assignment
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Nicolas De Loof
Guillaume Lours
Brian Goff
Wes Higbee
Laura Brehm
horus
Albin Kerouanton
Milas Bowman
vyneer
Amit Saha
Ulysses Souza
kimdcottrell
Eng Zer Jun
Matthieu MOREL
Matthew Walowski
Bjorn Neergaard
Sebastiaan van Stijn