Commit Graph

4559 Commits

Author SHA1 Message Date
Bjorn Neergaard 991901f2ef pkg/remote/oci: refer to the manifest as manifest
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-29 05:27:14 +02:00
Guillaume Lours bd74a9260d
Merge pull request #11047 from glours/update-watch-warning-section
update the watch warning message when no services with a develop section
2023-09-27 23:41:53 +02:00
Guillaume Lours 2d971fc97d
update the watch warning message when no services with a develop section
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-09-27 23:28:45 +02:00
Djordje Lukic 78f3361921
ci: enable verbose output for e2e tests (#11045)
It's easier to see that something is happening.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-09-27 16:29:54 +00:00
Nicolas De Loof 44d21280e7 truncate command by default, introduce --no-trunc flag to get the full command
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-09-23 06:34:26 +02:00
Guillaume Lours cd743d17ba
Merge pull request #11036 from docker/dependabot/go_modules/google.golang.org/grpc-1.58.2
build(deps): bump google.golang.org/grpc from 1.58.1 to 1.58.2
2023-09-22 11:23:05 +02:00
dependabot[bot] ff2ff18cdc
build(deps): bump google.golang.org/grpc from 1.58.1 to 1.58.2
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.58.1 to 1.58.2.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.58.1...v1.58.2)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-22 09:10:26 +00:00
Guillaume Lours bd32ed1454
Merge pull request #11025 from ndeloof/SkipResolveEnvironment
config --xx don't need `env_file` being parsed
2023-09-21 12:28:25 +02:00
Nicolas De Loof ab81db5bdb
config --xx don't need `env_file` being parsed
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-09-21 10:47:38 +02:00
Guillaume Lours 52a641bf6d
Merge pull request #11021 from glours/move-watch-to-main-cmd
move watch from alpha to main command
2023-09-21 10:08:23 +02:00
Guillaume Lours 75f5c07d3d
Merge pull request #11023 from glours/fix-watch-e2e-test-cleanup
remove --timeout=0 flag to cleanup function of watch e2e test
2023-09-20 21:43:51 +02:00
Guillaume Lours 61c8be11c0 remove --timeout=0 flag to cleanup function of watch e2e test
compose down command need the watch process to be killed to succeed

Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-09-20 21:32:03 +02:00
Guillaume Lours 6be5f3003a move watch from alpha to main command
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-09-20 18:55:08 +02:00
Guillaume Lours 421a6b0506
Merge pull request #11031 from ndeloof/TestWatch
TestWatch to use new  `develop` section
2023-09-20 18:54:20 +02:00
Nicolas De Loof c34c306cb9
TestWatch to use new `develop` section
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-09-20 18:39:32 +02:00
Nicolas De Loof 5ca35c88be implement publish
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-09-20 18:14:58 +02:00
Nicolas De loof 805541be9d
watch: use official `develop` section (#11026)
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-09-20 11:51:53 -04:00
Guillaume Lours d322ad91e8
Merge pull request #11027 from docker/dependabot/go_modules/github.com/opencontainers/image-spec-1.1.0-rc5
build(deps): bump github.com/opencontainers/image-spec from 1.1.0-rc4 to 1.1.0-rc5
2023-09-20 12:03:40 +02:00
dependabot[bot] 8f489d6d61 build(deps): bump github.com/opencontainers/image-spec
Bumps [github.com/opencontainers/image-spec](https://github.com/opencontainers/image-spec) from 1.1.0-rc4 to 1.1.0-rc5.
- [Release notes](https://github.com/opencontainers/image-spec/releases)
- [Changelog](https://github.com/opencontainers/image-spec/blob/main/RELEASES.md)
- [Commits](https://github.com/opencontainers/image-spec/compare/v1.1.0-rc4...v1.1.0-rc5)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/image-spec
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-20 11:50:03 +02:00
Guillaume Lours a214a715a0
Merge pull request #11017 from docker/dependabot/go_modules/github.com/moby/buildkit-0.12.2
build(deps): bump github.com/moby/buildkit from 0.12.1 to 0.12.2
2023-09-20 11:39:08 +02:00
dependabot[bot] 9ea8fbc69b build(deps): bump github.com/moby/buildkit from 0.12.1 to 0.12.2
Bumps [github.com/moby/buildkit](https://github.com/moby/buildkit) from 0.12.1 to 0.12.2.
- [Release notes](https://github.com/moby/buildkit/releases)
- [Commits](https://github.com/moby/buildkit/compare/v0.12.1...v0.12.2)

---
updated-dependencies:
- dependency-name: github.com/moby/buildkit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-20 11:26:04 +02:00
Guillaume Lours 062fa75534
Merge pull request #11020 from docker/dependabot/go_modules/gotest.tools/v3-3.5.1
build(deps): bump gotest.tools/v3 from 3.5.0 to 3.5.1
2023-09-20 10:53:36 +02:00
dependabot[bot] 76f150e49d
build(deps): bump gotest.tools/v3 from 3.5.0 to 3.5.1
Bumps [gotest.tools/v3](https://github.com/gotestyourself/gotest.tools) from 3.5.0 to 3.5.1.
- [Release notes](https://github.com/gotestyourself/gotest.tools/releases)
- [Commits](https://github.com/gotestyourself/gotest.tools/compare/v3.5.0...v3.5.1)

---
updated-dependencies:
- dependency-name: gotest.tools/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-20 08:21:25 +00:00
Guillaume Lours d45d943e91
Merge pull request #11019 from docker/dependabot/go_modules/google.golang.org/grpc-1.58.1
build(deps): bump google.golang.org/grpc from 1.58.0 to 1.58.1
2023-09-20 10:19:32 +02:00
dependabot[bot] f447c8096f build(deps): bump google.golang.org/grpc from 1.58.0 to 1.58.1
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.58.0 to 1.58.1.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.58.0...v1.58.1)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-20 10:05:44 +02:00
Guillaume Lours 722796ca28
Merge pull request #11016 from docker/dependabot/go_modules/github.com/containerd/containerd-1.7.6
build(deps): bump github.com/containerd/containerd from 1.7.3 to 1.7.6
2023-09-19 21:31:46 +02:00
dependabot[bot] 4587d4bad5
build(deps): bump github.com/containerd/containerd from 1.7.3 to 1.7.6
Bumps [github.com/containerd/containerd](https://github.com/containerd/containerd) from 1.7.3 to 1.7.6.
- [Release notes](https://github.com/containerd/containerd/releases)
- [Changelog](https://github.com/containerd/containerd/blob/main/RELEASES.md)
- [Commits](https://github.com/containerd/containerd/compare/v1.7.3...v1.7.6)

---
updated-dependencies:
- dependency-name: github.com/containerd/containerd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-19 09:41:32 +00:00
Nicolas De Loof a697a0690a introduce pull --missing flag to only pull images not present in cache
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-09-15 08:41:27 +02:00
Nicolas De Loof 8af49ff369 resolve service reference into container based on observed state
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-09-14 15:29:50 +02:00
Nicolas De Loof f6e31dbc6a don't rely on depends_on to resolve volume_from, better use observed state
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-09-14 15:29:50 +02:00
Guillaume Lours e19232e8a3
Merge pull request #11000 from thaJeztah/update_golang_1.21.1
update to go1.21.1
2023-09-14 09:50:50 +02:00
Sebastiaan van Stijn 6d5eb6fde6
update to go1.21.1
go1.21.1 (released 2023-09-06) includes four security fixes to the cmd/go,
crypto/tls, and html/template packages, as well as bug fixes to the compiler,
the go command, the linker, the runtime, and the context, crypto/tls,
encoding/gob, encoding/xml, go/types, net/http, os, and path/filepath packages.
See the Go 1.21.1 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.21.1+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.21.0...go1.21.1

From the security mailing:

[security] Go 1.21.1 and Go 1.20.8 are released

Hello gophers,

We have just released Go versions 1.21.1 and 1.20.8, minor point releases.

These minor releases include 4 security fixes following the security policy:

- cmd/go: go.mod toolchain directive allows arbitrary execution
  The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
  execute scripts and binaries relative to the root of the module when the "go"
  command was executed within the module. This applies to modules downloaded using
  the "go" command from the module proxy, as well as modules downloaded directly
  using VCS software.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.

- html/template: improper handling of HTML-like comments within script contexts
  The html/template package did not properly handle HMTL-like "<!--" and "-->"
  comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
  cause the template parser to improperly interpret the contents of <script>
  contexts, causing actions to be improperly escaped. This could be leveraged to
  perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.

- html/template: improper handling of special tags within script contexts
  The html/template package did not apply the proper rules for handling occurrences
  of "<script", "<!--", and "</script" within JS literals in <script> contexts.
  This may cause the template parser to improperly consider script contexts to be
  terminated early, causing actions to be improperly escaped. This could be
  leveraged to perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.

- crypto/tls: panic when processing post-handshake message on QUIC connections
  Processing an incomplete post-handshake message for a QUIC connection caused a panic.

  Thanks to Marten Seemann for reporting this issue.

  This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-14 09:29:39 +02:00
Guillaume Lours 9d7e0ad6cb correct scale error messages formatting
Co-authored-by: Milas Bowman <devnull@milas.dev>
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-09-13 11:15:14 +02:00
Guillaume Lours 1a98a70b8a add scale command
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-09-13 11:15:14 +02:00
Milas Bowman 19bbb12fac
ci: tweak restricted imports in linter (#10992)
* Eliminate direct dependency on gopkg.in/yaml.v2
* Add gopkg.in/yaml.v2 as a restricted import
* Add github.com/distribution/distribution as a restricted dependency in favor of distribution/reference which is the subset of functionality that Compose needs
* Remove an unused exclusion

NOTE: This does change the `compose config` output slightly but does NOT  change the semantics:
* YAML indentation is slightly different for lists (this is a `v2` / `v3` thing)
* JSON is now "minified" instead of pretty-printed (I think this generally desirable and more consistent with other JSON command outputs)

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-09-11 15:53:19 +00:00
Milas Bowman 7a13457853
deps: upgrade Moby to v24.0.6 and gRPC to v1.58.0 (#10991)
There are numerous transitive dependency upgrades as a result here.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-09-11 11:37:42 -04:00
Milas Bowman 13115468d5
cli: fix `--build` flag for `create` (#10982)
I missed this during a refactor and there wasn't test coverage.
Instead of adding more heavy-weight integration tests, I tried
to use `gomock` here to assert on the options objects after CLI
flag parsing. I think with a few more helpers, this could be a
good way to get a lot more combinations covered without adding
a ton of slow E2E tests.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-09-08 11:35:57 -04:00
Rory e1aa4f779b
otel: add args & flags to cli traces (#10974)
Signed-off-by: rvigus <roryvigus@gmail.com>
2023-09-07 16:04:36 -04:00
Milas Bowman d7b0b2bd7d
watch: build & launch the project at start (#10957)
The `alpha watch` command current "attaches" to an already-running
Compose project, so it's necessary to run something like
`docker compose up --wait` first.

Now, we'll do the equivalent of an `up --build` before starting the
watch, so that we know the project is up-to-date and running.

Additionally, unlike an interactive `up`, the services are not stopped
when `watch` exits (e.g. via `Ctrl-C`). This prevents the need to start
from scratch each time the command is run - if some services are already
running and up-to-date, they can be used as-is. A `down` can always be
used to destroy everything, and we can consider introducing a flag like
`--down-on-exit` to `watch` or changing the default.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-09-07 13:27:23 -04:00
Nicolas De Loof e0f39ebbef pull OCI remote resource
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-09-07 07:27:32 +02:00
Nicolas De Loof c9d54f09cf introduce publish (alpha) command
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-09-07 07:27:32 +02:00
Guillaume Lours aeb835a525
Merge pull request #10966 from kumarlokesh/patch-1
doc: updated README.md to remove broken link
2023-09-04 21:47:05 +02:00
Lokesh Kumar 52e54ef910
doc: updated README.md to remove broken link
- Removed the broken link in README.md file about `backward compatibility`.
- The section 
`# About update and backward compatibility` was removed in an earlier PR (https://github.com/docker/compose/pull/10889/files).

Signed-off-by: Lokesh Kumar <lkumar94@gmail.com>
2023-09-04 20:57:41 +02:00
Guillaume Lours 203bce883c
Merge pull request #10954 from thaJeztah/swap_reference
migrate to github.com/distribution/reference
2023-09-04 17:46:16 +02:00
Sebastiaan van Stijn f4f2e934ac
migrate to github.com/distribution/reference
The "reference" package was moved to a separate module, which was extracted
from b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-04 14:36:36 +02:00
Nicolas De Loof 32c3d0a3ff Enable service explicitly requested to be restarted
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-09-01 12:01:02 +02:00
Milas Bowman 1fdbcb6255 build: pass BuildOptions around explicitly & fix multi-platform issues
The big change here is to pass around an explicit `*BuildOptions` object
as part of Compose operations like `up` & `run` that may or may not do
builds. If the options object is `nil`, no builds whatsoever will be
attempted.

Motivation is to allow for partial rebuilds in the context of an `up`
for watch. This was broken and tricky to accomplish because various parts
of the Compose APIs mutate the `*Project` for convenience in ways that
make it unusable afterwards. (For example, it might set `service.Build = nil`
because it's not going to build that service right _then_. But we might
still want to build it later!)

NOTE: This commit does not actually touch the watch logic. This is all
      in preparation to make it possible.

As part of this, a bunch of code moved around and I eliminated a bunch
of partially redundant logic, mostly around multi-platform. Several
edge cases have been addressed as part of this:
 * `DOCKER_DEFAULT_PLATFORM` was _overriding_ explicitly set platforms
   in some cases, this is no longer true, and it behaves like the Docker
   CLI now
 * It was possible for Compose to build an image for one platform and
   then try to run it for a different platform (and fail)
 * Errors are no longer returned if a local image exists but for the
   wrong platform - the correct platform will be fetched/built (if
   possible).

Because there's a LOT of subtlety and tricky logic here, I've also tried
to add an excessive amount of explanatory comments.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-09-01 08:32:56 +02:00
Milas Bowman 407a0d5b53
up: fix various race/deadlock conditions on exit (#10934)
If running `up` in foreground mode (i.e. not `-d`),
when exiting via `Ctrl-C`, Compose stops all the
services it launched directly as part of that `up`
command.

In one of the E2E tests (`TestUpDependenciesNotStopped`),
this was occasionally flaking because the stop
behavior was racy: the return might not block on
the stop operation because it gets added to the
error group in a goroutine. As a result, it was
possible for no services to get terminated on exit.

There were a few other related pieces here that
I uncovered and tried to fix while stressing this.
For example, the printer could cause a deadlock if
an event was sent to it after it stopped.

Also, an error group wasn't really appropriate here;
each goroutine is a different operation for printing,
signal-handling, etc. If one part fails, we don't
actually want printing to stop, for example. This has
been switched to a `multierror.Group`, which has the
same API but coalesces errors instead of canceling a
context the moment the first one fails and returning
that single error.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-31 10:47:14 -04:00
Milas Bowman d0dfb848df
Merge pull request #10953 from thaJeztah/drop_uuid
pkg/api: replace uuid for basic random id
2023-08-31 10:43:34 -04:00
Sebastiaan van Stijn 8caa6f1f3e
pkg/api: replace uuid for basic random id
The uuid package in distribution was created as a utility for the distribution
project itself, to cut down external dependencies (see [1][1]).

For compose, this has the reverse effect, as it now brings all the dependencies
of the distribution module with it.

This patch switches to the uuid generation to crypto/rand to produce a random
id. I was considering using a different uuid implementation, or docker's
"stringid.GenerateRandomID", but all of those are doing more than needed,
so keep it simple.

Currently, this change has little effect, because compose also uses the
distribution module for other purposes, but the distribution project is
in the process of moving the "reference" package to a separate module,
in which case we don't want to depend on the distribution module only for
the uuid package.

[1]: 36e34a55ad

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-31 10:13:32 +02:00