From f039c8b43cae5fab4b8a22004ae6ab56d4d698b8 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Thu, 20 Oct 2016 17:39:55 -0700 Subject: [PATCH 01/45] Update release process document to account for recent changes. Signed-off-by: Joffrey F --- project/RELEASE-PROCESS.md | 62 ++++++++++++++++++++------------------ 1 file changed, 33 insertions(+), 29 deletions(-) diff --git a/project/RELEASE-PROCESS.md b/project/RELEASE-PROCESS.md index 930af15a8..c1834f2fb 100644 --- a/project/RELEASE-PROCESS.md +++ b/project/RELEASE-PROCESS.md @@ -20,18 +20,30 @@ release. As part of this script you'll be asked to: -1. Update the version in `docs/install.md` and `compose/__init__.py`. +1. Update the version in `compose/__init__.py` and `script/run/run.sh`. - If the next release will be an RC, append `rcN`, e.g. `1.4.0rc1`. + If the next release will be an RC, append `-rcN`, e.g. `1.4.0-rc1`. 2. Write release notes in `CHANGES.md`. - Almost every feature enhancement should be mentioned, with the most visible/exciting ones first. Use descriptive sentences and give context where appropriate. + Almost every feature enhancement should be mentioned, with the most + visible/exciting ones first. Use descriptive sentences and give context + where appropriate. - Bug fixes are worth mentioning if it's likely that they've affected lots of people, or if they were regressions in the previous version. + Bug fixes are worth mentioning if it's likely that they've affected lots + of people, or if they were regressions in the previous version. Improvements to the code are not worth mentioning. +3. Create a new repository on [bintray](https://bintray.com/docker-compose). + The name has to match the name of the branch (e.g. `bump-1.9.0`) and the + type should be "Generic". Other fields can be left blank. + +4. Check that the `vnext-compose` branch on + [the docs repo](https://github.com/docker/docker.github.io/) has + documentation for all the new additions in the upcoming release, and create + a PR there for what needs to be amended. + ## When a PR is merged into master that we want in the release @@ -55,8 +67,8 @@ Check out the bump branch and run the `build-binaries` script When prompted build the non-linux binaries and test them. -1. Download the osx binary from Bintray. Make sure that the latest build has - finished, otherwise you'll be downloading an old binary. +1. Download the osx binary from Bintray. Make sure that the latest Travis + build has finished, otherwise you'll be downloading an old binary. https://dl.bintray.com/docker-compose/$BRANCH_NAME/ @@ -67,22 +79,24 @@ When prompted build the non-linux binaries and test them. 3. Draft a release from the tag on GitHub (the script will open the window for you) - In the "Tag version" dropdown, select the tag you just pushed. + The tag will only be present on Github when you run the `push-release` + script in step 7, but you can pre-fill it at that point. -4. Paste in installation instructions and release notes. Here's an example - change the Compose version and Docker version as appropriate: +4. Paste in installation instructions and release notes. Here's an example - + change the Compose version and Docker version as appropriate: - Firstly, note that Compose 1.5.0 requires Docker 1.8.0 or later. + If you're a Mac or Windows user, the best way to install Compose and keep it up-to-date is **[Docker for Mac and Windows](https://www.docker.com/products/docker)**. - Secondly, if you're a Mac user, the **[Docker Toolbox](https://www.docker.com/toolbox)** will install Compose 1.5.0 for you, alongside the latest versions of the Docker Engine, Machine and Kitematic. + Note that Compose 1.9.0 requires Docker Engine 1.10.0 or later for version 2 of the Compose File format, and Docker Engine 1.9.1 or later for version 1. Docker for Mac and Windows will automatically install the latest version of Docker Engine for you. - Otherwise, you can use the usual commands to install/upgrade. Either download the binary: + Alternatively, you can use the usual commands to install or upgrade Compose: - curl -L https://github.com/docker/compose/releases/download/1.5.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose - chmod +x /usr/local/bin/docker-compose + ``` + curl -L https://github.com/docker/compose/releases/download/1.9.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose + chmod +x /usr/local/bin/docker-compose + ``` - Or install the PyPi package: - - pip install -U docker-compose==1.5.0 + See the [install docs](https://docs.docker.com/compose/install/) for more install options and instructions. Here's what's new: @@ -99,6 +113,8 @@ When prompted build the non-linux binaries and test them. ./script/release/push-release +8. Merge the bump PR. + 8. Publish the release on GitHub. 9. Check that all the binaries download (following the install instructions) and run. @@ -107,19 +123,7 @@ When prompted build the non-linux binaries and test them. ## If it’s a stable release (not an RC) -1. Merge the bump PR. - -2. Make sure `origin/release` is updated locally: - - git fetch origin - -3. Update the `docs` branch on the upstream repo: - - git push git@github.com:docker/compose.git origin/release:docs - -4. Let the docs team know that it’s been updated so they can publish it. - -5. Close the release’s milestone. +1. Close the release’s milestone. ## If it’s a minor release (1.x.0), rather than a patch release (1.x.y) From 3d8dc6f47a1484b0b6f8e3bd5cbf6115524d3892 Mon Sep 17 00:00:00 2001 From: Michal Zdrojewski Date: Mon, 24 Oct 2016 15:05:01 +0100 Subject: [PATCH 02/45] fix(docs): updated documentation links Signed-off-by: Michal Zdrojewski --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 93550f5ac..5cf69b05c 100644 --- a/README.md +++ b/README.md @@ -6,11 +6,11 @@ Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a Compose file to configure your application's services. Then, using a single command, you create and start all the services from your configuration. To learn more about all the features of Compose -see [the list of features](https://github.com/docker/compose/blob/release/docs/overview.md#features). +see [the list of features](https://github.com/docker/docker.github.io/blob/master/compose/overview.md#features). Compose is great for development, testing, and staging environments, as well as CI workflows. You can learn more about each case in -[Common Use Cases](https://github.com/docker/compose/blob/release/docs/overview.md#common-use-cases). +[Common Use Cases](https://github.com/docker/docker.github.io/blob/master/compose/overview.md#common-use-cases). Using Compose is basically a three-step process. @@ -35,7 +35,7 @@ A `docker-compose.yml` looks like this: image: redis For more information about the Compose file, see the -[Compose file reference](https://github.com/docker/compose/blob/release/docs/compose-file.md) +[Compose file reference](https://github.com/docker/docker.github.io/blob/master/compose/compose-file.md) Compose has commands for managing the whole lifecycle of your application: From 2c24bc3a083e138898cfe619a9cbf414f6df12d8 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 10:55:04 -0700 Subject: [PATCH 03/45] Add missing config schema to docker-compose.spec Signed-off-by: Joffrey F --- docker-compose.spec | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docker-compose.spec b/docker-compose.spec index 3a165dd67..e57d6b7a1 100644 --- a/docker-compose.spec +++ b/docker-compose.spec @@ -27,6 +27,11 @@ exe = EXE(pyz, 'compose/config/config_schema_v2.0.json', 'DATA' ), + ( + 'compose/config/config_schema_v2.1.json', + 'compose/config/config_schema_v2.1.json', + 'DATA' + ), ( 'compose/GITSHA', 'compose/GITSHA', From ea68be3441ef2c0c100b8bac9499fa4180106eb5 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 11:36:44 -0700 Subject: [PATCH 04/45] Do not print Swarm mode warning when connecting to a UCP server Signed-off-by: Joffrey F --- compose/project.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/compose/project.py b/compose/project.py index f85e285f3..60647fe95 100644 --- a/compose/project.py +++ b/compose/project.py @@ -538,6 +538,10 @@ def get_volumes_from(project, service_dict): def warn_for_swarm_mode(client): info = client.info() if info.get('Swarm', {}).get('LocalNodeState') == 'active': + if info.get('ServerVersion', '').startswith('ucp'): + # UCP does multi-node scheduling with traditional Compose files. + return + log.warn( "The Docker Engine you're using is running in swarm mode.\n\n" "Compose does not use swarm mode to deploy services to multiple nodes in a swarm. " From 43e29b41c04dfa00fee294f57ccd2e5d1a80f99a Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 11:51:45 -0700 Subject: [PATCH 05/45] Fix schema divergence - add missing fields to compose 2.1 schema Signed-off-by: Joffrey F --- compose/config/config_schema_v2.1.json | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/compose/config/config_schema_v2.1.json b/compose/config/config_schema_v2.1.json index f30b90543..3561a8cf2 100644 --- a/compose/config/config_schema_v2.1.json +++ b/compose/config/config_schema_v2.1.json @@ -140,6 +140,7 @@ "mac_address": {"type": "string"}, "mem_limit": {"type": ["number", "string"]}, "memswap_limit": {"type": ["number", "string"]}, + "mem_swappiness": {"type": "integer"}, "network_mode": {"type": "string"}, "networks": { @@ -168,6 +169,14 @@ } ] }, + "oom_score_adj": {"type": "integer", "minimum": -1000, "maximum": 1000}, + "group_add": { + "type": "array", + "items": { + "type": ["string", "number"] + }, + "uniqueItems": true + }, "pid": {"type": ["string", "null"]}, "ports": { @@ -248,6 +257,7 @@ }, "additionalProperties": false }, + "internal": {"type": "boolean"}, "enable_ipv6": {"type": "boolean"}, "labels": {"$ref": "#/definitions/list_or_dict"} }, From d2fb146913a1204b805ceb05e3d2b1b1a1006675 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 12:12:19 -0700 Subject: [PATCH 06/45] Replace "which" calls with the portable find_executable function Signed-off-by: Joffrey F --- compose/cli/errors.py | 6 +++--- compose/cli/main.py | 6 +++--- tests/unit/cli/errors_test.py | 6 +++--- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/compose/cli/errors.py b/compose/cli/errors.py index 4fdec08a8..5b9770955 100644 --- a/compose/cli/errors.py +++ b/compose/cli/errors.py @@ -4,6 +4,7 @@ from __future__ import unicode_literals import contextlib import logging import socket +from distutils.spawn import find_executable from textwrap import dedent from docker.errors import APIError @@ -13,7 +14,6 @@ from requests.exceptions import SSLError from requests.packages.urllib3.exceptions import ReadTimeoutError from ..const import API_VERSION_TO_ENGINE_VERSION -from .utils import call_silently from .utils import is_docker_for_mac_installed from .utils import is_mac from .utils import is_ubuntu @@ -90,11 +90,11 @@ def exit_with_error(msg): def get_conn_error_message(url): - if call_silently(['which', 'docker']) != 0: + if find_executable('docker') is None: return docker_not_found_msg("Couldn't connect to Docker daemon.") if is_docker_for_mac_installed(): return conn_error_docker_for_mac - if call_silently(['which', 'docker-machine']) == 0: + if find_executable('docker-machine') is not None: return conn_error_docker_machine return conn_error_generic.format(url=url) diff --git a/compose/cli/main.py b/compose/cli/main.py index 58b95c2f6..08e58e372 100644 --- a/compose/cli/main.py +++ b/compose/cli/main.py @@ -10,6 +10,7 @@ import pipes import re import subprocess import sys +from distutils.spawn import find_executable from inspect import getdoc from operator import attrgetter @@ -1063,9 +1064,8 @@ def exit_if(condition, message, exit_code): def call_docker(args): - try: - executable_path = subprocess.check_output(["which", "docker"]).strip() - except subprocess.CalledProcessError: + executable_path = find_executable('docker') + if not executable_path: raise UserError(errors.docker_not_found_msg("Couldn't find `docker` binary.")) args = [executable_path] + args diff --git a/tests/unit/cli/errors_test.py b/tests/unit/cli/errors_test.py index 1d454a081..a7b57562f 100644 --- a/tests/unit/cli/errors_test.py +++ b/tests/unit/cli/errors_test.py @@ -16,9 +16,9 @@ def mock_logging(): yield mock_log -def patch_call_silently(side_effect): +def patch_find_executable(side_effect): return mock.patch( - 'compose.cli.errors.call_silently', + 'compose.cli.errors.find_executable', autospec=True, side_effect=side_effect) @@ -27,7 +27,7 @@ class TestHandleConnectionErrors(object): def test_generic_connection_error(self, mock_logging): with pytest.raises(errors.ConnectionError): - with patch_call_silently([0, 1]): + with patch_find_executable(['/bin/docker', None]): with handle_connection_errors(mock.Mock()): raise ConnectionError() From 60d005b055119ce976265eaf34e4daa6483ead58 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 13:58:45 -0700 Subject: [PATCH 07/45] Improve robustness of a couple integration tests with occasional failures Signed-off-by: Joffrey F --- tests/integration/project_test.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/integration/project_test.py b/tests/integration/project_test.py index e3c34af9c..3afefb5ab 100644 --- a/tests/integration/project_test.py +++ b/tests/integration/project_test.py @@ -826,9 +826,9 @@ class ProjectTest(DockerClientTestCase): name='composetest', config_data=config_data ) - project.up() + project.up(detached=True) - service_container = project.get_service('web').containers()[0] + service_container = project.get_service('web').containers(stopped=True)[0] ipam_config = service_container.inspect().get( 'NetworkSettings', {} ).get( @@ -857,8 +857,8 @@ class ProjectTest(DockerClientTestCase): name='composetest', config_data=config_data ) - project.up() - service_container = project.get_service('web').containers()[0] + project.up(detached=True) + service_container = project.get_service('web').containers(stopped=True)[0] assert service_container.inspect()['HostConfig']['Isolation'] == 'default' @v2_1_only() From 99343fd76cc632d30ff2132ea715728317e10250 Mon Sep 17 00:00:00 2001 From: Albin Kerouanton Date: Tue, 25 Oct 2016 11:06:39 +0200 Subject: [PATCH 08/45] Fix path of the parent dir of COMPOSE_FILE Signed-off-by: Albin Kerouanton --- script/run/run.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/script/run/run.sh b/script/run/run.sh index 6205747af..5872b081a 100755 --- a/script/run/run.sh +++ b/script/run/run.sh @@ -35,7 +35,7 @@ if [ "$(pwd)" != '/' ]; then VOLUMES="-v $(pwd):$(pwd)" fi if [ -n "$COMPOSE_FILE" ]; then - compose_dir=$(dirname $COMPOSE_FILE) + compose_dir=$(realpath $(dirname $COMPOSE_FILE)) fi # TODO: also check --file argument if [ -n "$compose_dir" ]; then From 4871523d5e113e1f16a4ce50533d9b6f0bb80237 Mon Sep 17 00:00:00 2001 From: Mike Dougherty Date: Fri, 22 Apr 2016 12:56:07 -0700 Subject: [PATCH 09/45] Update Jenkinsfile to perform existing jenkins tasks Signed-off-by: Mike Dougherty --- Jenkinsfile | 84 +++++++++++++++++++++++++++++++++++++++++++++---- script/test/all | 3 +- 2 files changed, 80 insertions(+), 7 deletions(-) diff --git a/Jenkinsfile b/Jenkinsfile index fa29520b5..5de9a3fb1 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -1,8 +1,80 @@ -// Only run on Linux atm -wrappedNode(label: 'docker') { - deleteDir() - stage "checkout" - checkout scm +#!groovy - documentationChecker("docs") +def image + +def checkDocs = { -> + wrappedNode(label: 'linux') { + deleteDir(); checkout(scm) + documentationChecker("docs") + } } + +def buildImage = { -> + wrappedNode(label: "linux && !zfs") { + stage("build image") { + deleteDir(); checkout(scm) + def imageName = "dockerbuildbot/compose:${gitCommit()}" + image = docker.image(imageName) + try { + image.pull() + } catch (Exception exc) { + image = docker.build(imageName, ".") + image.push() + } + } + } +} + +def runTests = { Map settings -> + def dockerVersions = settings.get("dockerVersions", null) + def pythonVersions = settings.get("pythonVersions", null) + + if (!pythonVersions) { + throw new Exception("Need Python versions to test. e.g.: `runTests(pythonVersions: 'py27,py34')`") + } + if (!dockerVersions) { + throw new Exception("Need Docker versions to test. e.g.: `runTests(dockerVersions: 'all')`") + } + + { -> + wrappedNode(label: "linux && !zfs") { + stage("test python=${pythonVersions} / docker=${dockerVersions}") { + deleteDir(); checkout(scm) + def storageDriver = sh(script: 'docker info | awk -F \': \' \'$1 == "Storage Driver" { print $2; exit }\'', returnStdout: true).trim() + echo "Using local system's storage driver: ${storageDriver}" + sh """docker run \\ + -t \\ + --rm \\ + --privileged \\ + --volume="\$(pwd)/.git:/code/.git" \\ + --volume="/var/run/docker.sock:/var/run/docker.sock" \\ + -e "TAG=${image.id}" \\ + -e "STORAGE_DRIVER=${storageDriver}" \\ + -e "DOCKER_VERSIONS=${dockerVersions}" \\ + -e "BUILD_NUMBER=\$BUILD_TAG" \\ + -e "PY_TEST_VERSIONS=${pythonVersions}" \\ + --entrypoint="script/ci" \\ + ${image.id} \\ + --verbose + """ + } + } + } +} + +def buildAndTest = { -> + buildImage() + // TODO: break this out into meaningful "DOCKER_VERSIONS" values instead of all + parallel( + failFast: true, + all_py27: runTests(pythonVersions: "py27", dockerVersions: "all"), + all_py34: runTests(pythonVersions: "py34", dockerVersions: "all"), + ) +} + + +parallel( + failFast: false, + docs: checkDocs, + test: buildAndTest +) diff --git a/script/test/all b/script/test/all index 08bf16188..7151a75e1 100755 --- a/script/test/all +++ b/script/test/all @@ -24,6 +24,7 @@ fi BUILD_NUMBER=${BUILD_NUMBER-$USER} +PY_TEST_VERSIONS=${PY_TEST_VERSIONS:-py27,py34} for version in $DOCKER_VERSIONS; do >&2 echo "Running tests against Docker $version" @@ -58,6 +59,6 @@ for version in $DOCKER_VERSIONS; do --env="DOCKER_VERSION=$version" \ --entrypoint="tox" \ "$TAG" \ - -e py27,py34 -- "$@" + -e "$PY_TEST_VERSIONS" -- "$@" done From 046144e8f40de571b0d9c0029329e20712ca4736 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Thu, 27 Oct 2016 12:13:32 -0700 Subject: [PATCH 10/45] Bump docker-py version to include latest patch Signed-off-by: Joffrey F --- requirements.txt | 2 +- setup.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/requirements.txt b/requirements.txt index 474efbdf3..e72a88e78 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,7 +1,7 @@ PyYAML==3.11 backports.ssl-match-hostname==3.5.0.1; python_version < '3' cached-property==1.2.0 -docker-py==1.10.4 +docker-py==1.10.5 dockerpty==0.4.1 docopt==0.6.1 enum34==1.0.4; python_version < '3.4' diff --git a/setup.py b/setup.py index 442bc80c9..19ff5c6ae 100644 --- a/setup.py +++ b/setup.py @@ -34,7 +34,7 @@ install_requires = [ 'requests >= 2.6.1, != 2.11.0, < 2.12', 'texttable >= 0.8.1, < 0.9', 'websocket-client >= 0.32.0, < 1.0', - 'docker-py >= 1.10.4, < 2.0', + 'docker-py >= 1.10.5, < 2.0', 'dockerpty >= 0.4.1, < 0.5', 'six >= 1.3.0, < 2', 'jsonschema >= 2.5.1, < 3', From ba43d08fbdc1e7ec25978cc9068c3a9a7be91aab Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Tue, 1 Nov 2016 11:09:29 -0700 Subject: [PATCH 11/45] Add whitelisted driver option added by the overlay driver to avoid breakage Signed-off-by: Joffrey F --- compose/network.py | 21 +++++++++++++++++ tests/unit/network_test.py | 47 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+) create mode 100644 tests/unit/network_test.py diff --git a/compose/network.py b/compose/network.py index e581a4fe6..8e38401ca 100644 --- a/compose/network.py +++ b/compose/network.py @@ -12,6 +12,10 @@ from .config import ConfigurationError log = logging.getLogger(__name__) +OPTS_EXCEPTIONS = [ + 'com.docker.network.driver.overlay.vxlanid_list', +] + class Network(object): def __init__(self, client, project, name, driver=None, driver_opts=None, @@ -113,6 +117,23 @@ def create_ipam_config_from_dict(ipam_dict): ) +def check_remote_network_config(remote, local): + if local.driver and remote['Driver'] != local.driver: + raise ConfigurationError( + 'Network "{}" needs to be recreated - driver has changed' + .format(local.full_name) + ) + local_opts = local.driver_opts or {} + for k in set.union(set(remote['Options'].keys()), set(local_opts.keys())): + if k in OPTS_EXCEPTIONS: + continue + if remote['Options'].get(k) != local_opts.get(k): + raise ConfigurationError( + 'Network "{}" needs to be recreated - options have changed' + .format(local.full_name) + ) + + def build_networks(name, config_data, client): network_config = config_data.networks or {} networks = { diff --git a/tests/unit/network_test.py b/tests/unit/network_test.py new file mode 100644 index 000000000..4720b0530 --- /dev/null +++ b/tests/unit/network_test.py @@ -0,0 +1,47 @@ +from __future__ import absolute_import +from __future__ import unicode_literals + +import pytest + +from .. import unittest +from compose.config import ConfigurationError +from compose.network import check_remote_network_config +from compose.network import Network + + +class NetworkTest(unittest.TestCase): + def test_check_remote_network_config_success(self): + options = {'com.docker.network.driver.foo': 'bar'} + net = Network( + None, 'compose_test', 'net1', 'bridge', + options + ) + check_remote_network_config( + {'Driver': 'bridge', 'Options': options}, net + ) + + def test_check_remote_network_config_whitelist(self): + options = {'com.docker.network.driver.foo': 'bar'} + remote_options = { + 'com.docker.network.driver.overlay.vxlanid_list': '257', + 'com.docker.network.driver.foo': 'bar' + } + net = Network( + None, 'compose_test', 'net1', 'overlay', + options + ) + check_remote_network_config( + {'Driver': 'overlay', 'Options': remote_options}, net + ) + + def test_check_remote_network_config_driver_mismatch(self): + net = Network(None, 'compose_test', 'net1', 'overlay') + with pytest.raises(ConfigurationError): + check_remote_network_config({'Driver': 'bridge', 'Options': {}}, net) + + def test_check_remote_network_config_options_mismatch(self): + net = Network(None, 'compose_test', 'net1', 'overlay') + with pytest.raises(ConfigurationError): + check_remote_network_config({'Driver': 'overlay', 'Options': { + 'com.docker.network.driver.foo': 'baz' + }}, net) From 7a430dbe96ad01831e21eab7eaebf26fdaa9249c Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 2 Nov 2016 16:43:29 -0700 Subject: [PATCH 12/45] Updated docker-py dependency to latest version Signed-off-by: Joffrey F --- requirements.txt | 2 +- setup.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/requirements.txt b/requirements.txt index e72a88e78..933146c72 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,7 +1,7 @@ PyYAML==3.11 backports.ssl-match-hostname==3.5.0.1; python_version < '3' cached-property==1.2.0 -docker-py==1.10.5 +docker-py==1.10.6 dockerpty==0.4.1 docopt==0.6.1 enum34==1.0.4; python_version < '3.4' diff --git a/setup.py b/setup.py index 19ff5c6ae..672ea80e5 100644 --- a/setup.py +++ b/setup.py @@ -34,7 +34,7 @@ install_requires = [ 'requests >= 2.6.1, != 2.11.0, < 2.12', 'texttable >= 0.8.1, < 0.9', 'websocket-client >= 0.32.0, < 1.0', - 'docker-py >= 1.10.5, < 2.0', + 'docker-py >= 1.10.6, < 2.0', 'dockerpty >= 0.4.1, < 0.5', 'six >= 1.3.0, < 2', 'jsonschema >= 2.5.1, < 3', From da1508051dcc7f106f891078b688bc31db7e43c5 Mon Sep 17 00:00:00 2001 From: Mike Dougherty Date: Thu, 3 Nov 2016 13:59:56 -0700 Subject: [PATCH 13/45] Remove docs checker from Jenkinsfile and use cleanWorkspace option on wrappedNode Signed-off-by: Mike Dougherty --- Jenkinsfile | 34 +++++++++------------------------- 1 file changed, 9 insertions(+), 25 deletions(-) diff --git a/Jenkinsfile b/Jenkinsfile index 5de9a3fb1..e2f86daad 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -2,17 +2,10 @@ def image -def checkDocs = { -> - wrappedNode(label: 'linux') { - deleteDir(); checkout(scm) - documentationChecker("docs") - } -} - def buildImage = { -> - wrappedNode(label: "linux && !zfs") { + wrappedNode(label: "linux && !zfs", cleanWorkspace: true) { stage("build image") { - deleteDir(); checkout(scm) + checkout(scm) def imageName = "dockerbuildbot/compose:${gitCommit()}" image = docker.image(imageName) try { @@ -37,9 +30,9 @@ def runTests = { Map settings -> } { -> - wrappedNode(label: "linux && !zfs") { + wrappedNode(label: "linux && !zfs", cleanWorkspace: true) { stage("test python=${pythonVersions} / docker=${dockerVersions}") { - deleteDir(); checkout(scm) + checkout(scm) def storageDriver = sh(script: 'docker info | awk -F \': \' \'$1 == "Storage Driver" { print $2; exit }\'', returnStdout: true).trim() echo "Using local system's storage driver: ${storageDriver}" sh """docker run \\ @@ -62,19 +55,10 @@ def runTests = { Map settings -> } } -def buildAndTest = { -> - buildImage() - // TODO: break this out into meaningful "DOCKER_VERSIONS" values instead of all - parallel( - failFast: true, - all_py27: runTests(pythonVersions: "py27", dockerVersions: "all"), - all_py34: runTests(pythonVersions: "py34", dockerVersions: "all"), - ) -} - - +buildImage() +// TODO: break this out into meaningful "DOCKER_VERSIONS" values instead of all parallel( - failFast: false, - docs: checkDocs, - test: buildAndTest + failFast: true, + all_py27: runTests(pythonVersions: "py27", dockerVersions: "all"), + all_py34: runTests(pythonVersions: "py34", dockerVersions: "all"), ) From 10417eebd70f028f57e56ef9a04d8ed51abdad99 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Thu, 3 Nov 2016 16:28:44 -0700 Subject: [PATCH 14/45] Fix logging dict merging Signed-off-by: Joffrey F --- compose/config/config.py | 2 +- tests/unit/config/config_test.py | 45 ++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+), 1 deletion(-) diff --git a/compose/config/config.py b/compose/config/config.py index 437ed3892..9d23b34da 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -771,7 +771,7 @@ def merge_service_dicts(base, override, version): for field in ['dns', 'dns_search', 'env_file', 'tmpfs']: md.merge_field(field, merge_list_or_string) - md.merge_field('logging', merge_logging) + md.merge_field('logging', merge_logging, default={}) for field in set(ALLOWED_KEYS) - set(md): md.merge_scalar(field) diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index d205e2825..66ae01474 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -1657,6 +1657,51 @@ class ConfigTest(unittest.TestCase): } } + def test_merge_logging_v2_no_base(self): + base = { + 'image': 'alpine:edge' + } + override = { + 'logging': { + 'driver': 'json-file', + 'options': { + 'frequency': '2000' + } + } + } + actual = config.merge_service_dicts(base, override, V2_0) + assert actual == { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'json-file', + 'options': { + 'frequency': '2000' + } + } + } + + def test_merge_logging_v2_no_override(self): + base = { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'syslog', + 'options': { + 'frequency': '2000' + } + } + } + override = {} + actual = config.merge_service_dicts(base, override, V2_0) + assert actual == { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'syslog', + 'options': { + 'frequency': '2000' + } + } + } + def test_external_volume_config(self): config_details = build_config_details({ 'version': '2', From 4aa7d15d9771fadd22e8c1ff952526fdd6a67d77 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Fri, 4 Nov 2016 10:51:14 -0700 Subject: [PATCH 15/45] Call check_remote_network_config from Network.ensure Signed-off-by: Joffrey F --- compose/network.py | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/compose/network.py b/compose/network.py index 8e38401ca..06935a46b 100644 --- a/compose/network.py +++ b/compose/network.py @@ -53,14 +53,7 @@ class Network(object): try: data = self.inspect() - if self.driver and data['Driver'] != self.driver: - raise ConfigurationError( - 'Network "{}" needs to be recreated - driver has changed' - .format(self.full_name)) - if data['Options'] != (self.driver_opts or {}): - raise ConfigurationError( - 'Network "{}" needs to be recreated - options have changed' - .format(self.full_name)) + check_remote_network_config(data, self) except NotFound: driver_name = 'the default driver' if self.driver: From ba249e51796e6b35ed13e5f03716a9520a2dacfc Mon Sep 17 00:00:00 2001 From: Aanand Prasad Date: Wed, 9 Nov 2016 15:10:02 +0000 Subject: [PATCH 16/45] Test that values in 'environment' override env files Signed-off-by: Aanand Prasad --- tests/unit/config/config_test.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index 66ae01474..51c5e226b 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -2493,6 +2493,15 @@ class EnvTest(unittest.TestCase): {'ONE': '2', 'TWO': '1', 'THREE': '3', 'FOO': 'bar'}, ) + def test_environment_overrides_env_file(self): + self.assertEqual( + resolve_environment({ + 'environment': {'FOO': 'baz'}, + 'env_file': ['tests/fixtures/env/one.env'], + }), + {'ONE': '2', 'TWO': '1', 'THREE': '3', 'FOO': 'baz'}, + ) + def test_resolve_environment_with_multiple_env_files(self): service_dict = { 'env_file': [ From 91620ae97bbffd247e2f78132255c0fe97910c4f Mon Sep 17 00:00:00 2001 From: Jari Takkala Date: Fri, 29 Jul 2016 06:46:42 -0400 Subject: [PATCH 17/45] Add sysctl option support when creating service Closes #3765 Signed-off-by: Jari Takkala --- compose/config/config.py | 16 +++++++++++----- compose/config/config_schema_v2.1.json | 1 + compose/service.py | 2 ++ 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/compose/config/config.py b/compose/config/config.py index 9d23b34da..57039e693 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -83,6 +83,7 @@ DOCKER_CONFIG_KEYS = [ 'shm_size', 'stdin_open', 'stop_signal', + 'sysctls', 'tty', 'user', 'volume_driver', @@ -629,6 +630,9 @@ def process_service(service_config): if 'extra_hosts' in service_dict: service_dict['extra_hosts'] = parse_extra_hosts(service_dict['extra_hosts']) + if 'sysctls' in service_dict: + service_dict['sysctls'] = build_string_dict(parse_sysctls(service_dict['sysctls'])) + for field in ['dns', 'dns_search', 'tmpfs']: if field in service_dict: service_dict[field] = to_list(service_dict[field]) @@ -757,6 +761,7 @@ def merge_service_dicts(base, override, version): md.merge_mapping('labels', parse_labels) md.merge_mapping('ulimits', parse_ulimits) md.merge_mapping('networks', parse_networks) + md.merge_mapping('sysctls', parse_sysctls) md.merge_sequence('links', ServiceLink.parse) for field in ['volumes', 'devices']: @@ -831,11 +836,11 @@ def merge_environment(base, override): return env -def split_label(label): - if '=' in label: - return label.split('=', 1) +def split_kv(kvpair): + if '=' in kvpair: + return kvpair.split('=', 1) else: - return label, '' + return kvpair, '' def parse_dict_or_list(split_func, type_name, arguments): @@ -856,8 +861,9 @@ def parse_dict_or_list(split_func, type_name, arguments): parse_build_arguments = functools.partial(parse_dict_or_list, split_env, 'build arguments') parse_environment = functools.partial(parse_dict_or_list, split_env, 'environment') -parse_labels = functools.partial(parse_dict_or_list, split_label, 'labels') +parse_labels = functools.partial(parse_dict_or_list, split_kv, 'labels') parse_networks = functools.partial(parse_dict_or_list, lambda k: (k, None), 'networks') +parse_sysctls = functools.partial(parse_dict_or_list, split_kv, 'sysctls') def parse_ulimits(ulimits): diff --git a/compose/config/config_schema_v2.1.json b/compose/config/config_schema_v2.1.json index 3561a8cf2..fc95f2cba 100644 --- a/compose/config/config_schema_v2.1.json +++ b/compose/config/config_schema_v2.1.json @@ -193,6 +193,7 @@ "restart": {"type": "string"}, "security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "shm_size": {"type": ["number", "string"]}, + "sysctls": {"$ref": "#/definitions/list_or_dict"}, "stdin_open": {"type": "boolean"}, "stop_signal": {"type": "string"}, "tmpfs": {"$ref": "#/definitions/string_or_list"}, diff --git a/compose/service.py b/compose/service.py index 760d29a7b..c917aac45 100644 --- a/compose/service.py +++ b/compose/service.py @@ -62,6 +62,7 @@ DOCKER_START_KEYS = [ 'restart', 'security_opt', 'shm_size', + 'sysctls', 'volumes_from', ] @@ -707,6 +708,7 @@ class Service(object): cgroup_parent=options.get('cgroup_parent'), cpu_quota=options.get('cpu_quota'), shm_size=options.get('shm_size'), + sysctls=options.get('sysctls'), tmpfs=options.get('tmpfs'), oom_score_adj=options.get('oom_score_adj'), mem_swappiness=options.get('mem_swappiness'), From 0291d9ade5574ff31aecc1a9f263f1ec21a901cf Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 14 Nov 2016 17:23:25 -0800 Subject: [PATCH 18/45] Limit testing pool to Ubuntu hosts to avoid errors with dind not starting properly. Signed-off-by: Joffrey F --- Jenkinsfile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Jenkinsfile b/Jenkinsfile index e2f86daad..51136b1f7 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -3,7 +3,7 @@ def image def buildImage = { -> - wrappedNode(label: "linux && !zfs", cleanWorkspace: true) { + wrappedNode(label: "ubuntu && !zfs", cleanWorkspace: true) { stage("build image") { checkout(scm) def imageName = "dockerbuildbot/compose:${gitCommit()}" @@ -30,7 +30,7 @@ def runTests = { Map settings -> } { -> - wrappedNode(label: "linux && !zfs", cleanWorkspace: true) { + wrappedNode(label: "ubuntu && !zfs", cleanWorkspace: true) { stage("test python=${pythonVersions} / docker=${dockerVersions}") { checkout(scm) def storageDriver = sh(script: 'docker info | awk -F \': \' \'$1 == "Storage Driver" { print $2; exit }\'', returnStdout: true).trim() From efb4ed1b9e130ca5ac54f6e0fb23ce68c3689c1f Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 14 Nov 2016 18:03:26 -0800 Subject: [PATCH 19/45] Handle new pull failures behavior in Engine 1.13 Signed-off-by: Joffrey F --- compose/service.py | 6 +++--- tests/acceptance/cli_test.py | 7 ++++--- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/compose/service.py b/compose/service.py index 760d29a7b..ad4267062 100644 --- a/compose/service.py +++ b/compose/service.py @@ -10,6 +10,7 @@ from operator import attrgetter import enum import six from docker.errors import APIError +from docker.errors import NotFound from docker.utils import LogConfig from docker.utils.ports import build_port_bindings from docker.utils.ports import split_port @@ -829,12 +830,11 @@ class Service(object): repo, tag, separator = parse_repository_tag(self.options['image']) tag = tag or 'latest' log.info('Pulling %s (%s%s%s)...' % (self.name, repo, separator, tag)) - output = self.client.pull(repo, tag=tag, stream=True) - try: + output = self.client.pull(repo, tag=tag, stream=True) return progress_stream.get_digest_from_pull( stream_output(output, sys.stdout)) - except StreamOutputError as e: + except (StreamOutputError, NotFound) as e: if not ignore_pull_failures: raise else: diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index a7cd78f18..f153bd95b 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -330,12 +330,13 @@ class CLITestCase(DockerClientTestCase): def test_pull_with_ignore_pull_failures(self): result = self.dispatch([ '-f', 'ignore-pull-failures.yml', - 'pull', '--ignore-pull-failures']) + 'pull', '--ignore-pull-failures'] + ) assert 'Pulling simple (busybox:latest)...' in result.stderr assert 'Pulling another (nonexisting-image:latest)...' in result.stderr - assert 'Error: image library/nonexisting-image' in result.stderr - assert 'not found' in result.stderr + assert ('repository nonexisting-image not found' in result.stderr or + 'image library/nonexisting-image:latest not found' in result.stderr) def test_build_plain(self): self.base_dir = 'tests/fixtures/simple-dockerfile' From 7f60ff5ae6b320dc0b38f8f2bcc850eca95f49ae Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 9 Nov 2016 15:38:09 -0800 Subject: [PATCH 20/45] Avoid breaking when remote driver options are null. Signed-off-by: Joffrey F --- compose/network.py | 7 ++++--- tests/unit/network_test.py | 10 +++++++++- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/compose/network.py b/compose/network.py index 06935a46b..3b57cb94e 100644 --- a/compose/network.py +++ b/compose/network.py @@ -111,16 +111,17 @@ def create_ipam_config_from_dict(ipam_dict): def check_remote_network_config(remote, local): - if local.driver and remote['Driver'] != local.driver: + if local.driver and remote.get('Driver') != local.driver: raise ConfigurationError( 'Network "{}" needs to be recreated - driver has changed' .format(local.full_name) ) local_opts = local.driver_opts or {} - for k in set.union(set(remote['Options'].keys()), set(local_opts.keys())): + remote_opts = remote.get('Options') or {} + for k in set.union(set(remote_opts.keys()), set(local_opts.keys())): if k in OPTS_EXCEPTIONS: continue - if remote['Options'].get(k) != local_opts.get(k): + if remote_opts.get(k) != local_opts.get(k): raise ConfigurationError( 'Network "{}" needs to be recreated - options have changed' .format(local.full_name) diff --git a/tests/unit/network_test.py b/tests/unit/network_test.py index 4720b0530..12d06f415 100644 --- a/tests/unit/network_test.py +++ b/tests/unit/network_test.py @@ -37,7 +37,9 @@ class NetworkTest(unittest.TestCase): def test_check_remote_network_config_driver_mismatch(self): net = Network(None, 'compose_test', 'net1', 'overlay') with pytest.raises(ConfigurationError): - check_remote_network_config({'Driver': 'bridge', 'Options': {}}, net) + check_remote_network_config( + {'Driver': 'bridge', 'Options': {}}, net + ) def test_check_remote_network_config_options_mismatch(self): net = Network(None, 'compose_test', 'net1', 'overlay') @@ -45,3 +47,9 @@ class NetworkTest(unittest.TestCase): check_remote_network_config({'Driver': 'overlay', 'Options': { 'com.docker.network.driver.foo': 'baz' }}, net) + + def test_check_remote_network_config_null_remote(self): + net = Network(None, 'compose_test', 'net1', 'overlay') + check_remote_network_config( + {'Driver': 'overlay', 'Options': None}, net + ) From d717c88b6e81f5eb0769bc0670a6b78de842b2ce Mon Sep 17 00:00:00 2001 From: Aanand Prasad Date: Wed, 9 Nov 2016 17:16:36 +0000 Subject: [PATCH 21/45] Support version 3.0 of the Compose file format Signed-off-by: Aanand Prasad --- compose/config/config.py | 13 +- compose/config/config_schema_v3.0.json | 378 ++++++++++++++++++++++ compose/config/errors.py | 4 +- compose/config/serialize.py | 3 +- compose/const.py | 3 + tests/acceptance/cli_test.py | 55 ++++ tests/fixtures/v3-full/docker-compose.yml | 37 +++ tests/unit/config/config_test.py | 6 + 8 files changed, 491 insertions(+), 8 deletions(-) create mode 100644 compose/config/config_schema_v3.0.json create mode 100644 tests/fixtures/v3-full/docker-compose.yml diff --git a/compose/config/config.py b/compose/config/config.py index 9d23b34da..fb77436d1 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -15,6 +15,7 @@ from cached_property import cached_property from ..const import COMPOSEFILE_V1 as V1 from ..const import COMPOSEFILE_V2_0 as V2_0 from ..const import COMPOSEFILE_V2_1 as V2_1 +from ..const import COMPOSEFILE_V3_0 as V3_0 from ..utils import build_string_dict from ..utils import splitdrive from .environment import env_vars_from_file @@ -175,7 +176,10 @@ class ConfigFile(namedtuple('_ConfigFile', 'filename config')): if version == '2': version = V2_0 - if version not in (V2_0, V2_1): + if version == '3': + version = V3_0 + + if version not in (V2_0, V2_1, V3_0): raise ConfigurationError( 'Version in "{}" is unsupported. {}' .format(self.filename, VERSION_EXPLANATION)) @@ -433,7 +437,7 @@ def process_config_file(config_file, environment, service_name=None): 'service', environment) - if config_file.version in (V2_0, V2_1): + if config_file.version in (V2_0, V2_1, V3_0): processed_config = dict(config_file.config) processed_config['services'] = services processed_config['volumes'] = interpolate_config_section( @@ -446,9 +450,10 @@ def process_config_file(config_file, environment, service_name=None): config_file.get_networks(), 'network', environment) - - if config_file.version == V1: + elif config_file.version == V1: processed_config = services + else: + raise Exception("Unsupported version: {}".format(repr(config_file.version))) config_file = config_file._replace(config=processed_config) validate_against_config_schema(config_file) diff --git a/compose/config/config_schema_v3.0.json b/compose/config/config_schema_v3.0.json new file mode 100644 index 000000000..9ac31b1f8 --- /dev/null +++ b/compose/config/config_schema_v3.0.json @@ -0,0 +1,378 @@ +{ + "$schema": "http://json-schema.org/draft-04/schema#", + "id": "config_schema_v3.0.json", + "type": "object", + "required": ["version"], + + "properties": { + "version": { + "type": "string" + }, + + "services": { + "id": "#/properties/services", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/service" + } + }, + "additionalProperties": false + }, + + "networks": { + "id": "#/properties/networks", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/network" + } + } + }, + + "volumes": { + "id": "#/properties/volumes", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/volume" + } + }, + "additionalProperties": false + } + }, + + "additionalProperties": false, + + "definitions": { + + "service": { + "id": "#/definitions/service", + "type": "object", + + "properties": { + "deploy": {"$ref": "#/definitions/deployment"}, + "build": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + "properties": { + "context": {"type": "string"}, + "dockerfile": {"type": "string"}, + "args": {"$ref": "#/definitions/list_or_dict"} + }, + "additionalProperties": false + } + ] + }, + "cap_add": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cap_drop": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cgroup_parent": {"type": "string"}, + "command": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "container_name": {"type": "string"}, + "depends_on": {"$ref": "#/definitions/list_of_strings"}, + "devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "dns": {"$ref": "#/definitions/string_or_list"}, + "dns_search": {"$ref": "#/definitions/string_or_list"}, + "domainname": {"type": "string"}, + "entrypoint": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "env_file": {"$ref": "#/definitions/string_or_list"}, + "environment": {"$ref": "#/definitions/list_or_dict"}, + + "expose": { + "type": "array", + "items": { + "type": ["string", "number"], + "format": "expose" + }, + "uniqueItems": true + }, + + "external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "extra_hosts": {"$ref": "#/definitions/list_or_dict"}, + "healthcheck": {"$ref": "#/definitions/healthcheck"}, + "hostname": {"type": "string"}, + "image": {"type": "string"}, + "ipc": {"type": "string"}, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + + "logging": { + "type": "object", + + "properties": { + "driver": {"type": "string"}, + "options": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number", "null"]} + } + } + }, + "additionalProperties": false + }, + + "mac_address": {"type": "string"}, + "network_mode": {"type": "string"}, + + "networks": { + "oneOf": [ + {"$ref": "#/definitions/list_of_strings"}, + { + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "oneOf": [ + { + "type": "object", + "properties": { + "aliases": {"$ref": "#/definitions/list_of_strings"}, + "ipv4_address": {"type": "string"}, + "ipv6_address": {"type": "string"} + }, + "additionalProperties": false + }, + {"type": "null"} + ] + } + }, + "additionalProperties": false + } + ] + }, + "pid": {"type": ["string", "null"]}, + + "ports": { + "type": "array", + "items": { + "type": ["string", "number"], + "format": "ports" + }, + "uniqueItems": true + }, + + "privileged": {"type": "boolean"}, + "read_only": {"type": "boolean"}, + "restart": {"type": "string"}, + "security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "shm_size": {"type": ["number", "string"]}, + "stdin_open": {"type": "boolean"}, + "stop_signal": {"type": "string"}, + "stop_grace_period": {"type": "string", "format": "duration"}, + "tmpfs": {"$ref": "#/definitions/string_or_list"}, + "tty": {"type": "boolean"}, + "ulimits": { + "type": "object", + "patternProperties": { + "^[a-z]+$": { + "oneOf": [ + {"type": "integer"}, + { + "type":"object", + "properties": { + "hard": {"type": "integer"}, + "soft": {"type": "integer"} + }, + "required": ["soft", "hard"], + "additionalProperties": false + } + ] + } + } + }, + "user": {"type": "string"}, + "volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "working_dir": {"type": "string"} + }, + "additionalProperties": false + }, + + "healthcheck": { + "id": "#/definitions/healthcheck", + "type": ["object", "null"], + "properties": { + "interval": {"type":"string"}, + "timeout": {"type":"string"}, + "retries": {"type": "number"}, + "command": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + } + }, + "additionalProperties": false + }, + "deployment": { + "id": "#/definitions/deployment", + "type": ["object", "null"], + "properties": { + "mode": {"type": "string"}, + "replicas": {"type": "integer"}, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "update_config": { + "type": "object", + "properties": { + "parallelism": {"type": "integer"}, + "delay": {"type": "string", "format": "duration"}, + "failure_action": {"type": "string"}, + "monitor": {"type": "string", "format": "duration"}, + "max_failure_ratio": {"type": "number"} + }, + "additionalProperties": false + }, + "resources": { + "type": "object", + "properties": { + "limits": {"$ref": "#/definitions/resource"}, + "reservations": {"$ref": "#/definitions/resource"} + } + }, + "restart_policy": { + "type": "object", + "properties": { + "condition": {"type": "string"}, + "delay": {"type": "string", "format": "duration"}, + "max_attempts": {"type": "integer"}, + "window": {"type": "string", "format": "duration"} + }, + "additionalProperties": false + }, + "placement": { + "type": "object", + "properties": { + "constraints": {"type": "array", "items": {"type": "string"}} + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + + "resource": { + "id": "#/definitions/resource", + "type": "object", + "properties": { + "cpus": {"type": "string"}, + "memory": {"type": "string"} + }, + "additionaProperties": false + }, + + "network": { + "id": "#/definitions/network", + "type": ["object", "null"], + "properties": { + "driver": {"type": "string"}, + "driver_opts": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number"]} + } + }, + "ipam": { + "type": "object", + "properties": { + "driver": {"type": "string"}, + "config": { + "type": "array", + "items": { + "type": "object", + "properties": { + "subnet": {"type": "string"} + }, + "additionalProperties": false + } + } + }, + "additionalProperties": false + }, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": {"type": "string"} + }, + "additionalProperties": false + }, + "labels": {"$ref": "#/definitions/list_or_dict"} + }, + "additionalProperties": false + }, + + "volume": { + "id": "#/definitions/volume", + "type": ["object", "null"], + "properties": { + "driver": {"type": "string"}, + "driver_opts": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number"]} + } + }, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": {"type": "string"} + } + } + }, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "additionalProperties": false + }, + + "string_or_list": { + "oneOf": [ + {"type": "string"}, + {"$ref": "#/definitions/list_of_strings"} + ] + }, + + "list_of_strings": { + "type": "array", + "items": {"type": "string"}, + "uniqueItems": true + }, + + "list_or_dict": { + "oneOf": [ + { + "type": "object", + "patternProperties": { + ".+": { + "type": ["string", "number", "null"] + } + }, + "additionalProperties": false + }, + {"type": "array", "items": {"type": "string"}, "uniqueItems": true} + ] + }, + + "constraints": { + "service": { + "id": "#/definitions/constraints/service", + "anyOf": [ + {"required": ["build"]}, + {"required": ["image"]} + ], + "properties": { + "build": { + "required": ["context"] + } + } + } + } + } +} diff --git a/compose/config/errors.py b/compose/config/errors.py index d14cbbdd0..16ed01b86 100644 --- a/compose/config/errors.py +++ b/compose/config/errors.py @@ -3,8 +3,8 @@ from __future__ import unicode_literals VERSION_EXPLANATION = ( - 'You might be seeing this error because you\'re using the wrong Compose ' - 'file version. Either specify a version of "2" (or "2.0") and place your ' + 'You might be seeing this error because you\'re using the wrong Compose file version. ' + 'Either specify a supported version ("2.0", "2.1", "3.0") and place your ' 'service definitions under the `services` key, or omit the `version` key ' 'and place your service definitions at the root of the file to use ' 'version 1.\nFor more on the Compose file format versions, see ' diff --git a/compose/config/serialize.py b/compose/config/serialize.py index 95b1387fc..768f3d473 100644 --- a/compose/config/serialize.py +++ b/compose/config/serialize.py @@ -6,7 +6,6 @@ import yaml from compose.config import types from compose.config.config import V1 -from compose.config.config import V2_0 from compose.config.config import V2_1 @@ -34,7 +33,7 @@ def denormalize_config(config): del net_conf['external_name'] version = config.version - if version not in (V2_0, V2_1): + if version == V1: version = V2_1 return { diff --git a/compose/const.py b/compose/const.py index e7b1ae97a..3da398551 100644 --- a/compose/const.py +++ b/compose/const.py @@ -17,15 +17,18 @@ LABEL_CONFIG_HASH = 'com.docker.compose.config-hash' COMPOSEFILE_V1 = '1' COMPOSEFILE_V2_0 = '2.0' COMPOSEFILE_V2_1 = '2.1' +COMPOSEFILE_V3_0 = '3.0' API_VERSIONS = { COMPOSEFILE_V1: '1.21', COMPOSEFILE_V2_0: '1.22', COMPOSEFILE_V2_1: '1.24', + COMPOSEFILE_V3_0: '1.24', } API_VERSION_TO_ENGINE_VERSION = { API_VERSIONS[COMPOSEFILE_V1]: '1.9.0', API_VERSIONS[COMPOSEFILE_V2_0]: '1.10.0', API_VERSIONS[COMPOSEFILE_V2_1]: '1.12.0', + API_VERSIONS[COMPOSEFILE_V3_0]: '1.12.0', } diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index f153bd95b..e9a416919 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -285,6 +285,61 @@ class CLITestCase(DockerClientTestCase): 'volumes': {}, } + def test_config_v3(self): + self.base_dir = 'tests/fixtures/v3-full' + result = self.dispatch(['config']) + + assert yaml.load(result.stdout) == { + 'version': '3.0', + 'networks': {}, + 'volumes': {}, + 'services': { + 'web': { + 'image': 'busybox', + 'deploy': { + 'mode': 'replicated', + 'replicas': 6, + 'labels': ['FOO=BAR'], + 'update_config': { + 'parallelism': 3, + 'delay': '10s', + 'failure_action': 'continue', + 'monitor': '60s', + 'max_failure_ratio': 0.3, + }, + 'resources': { + 'limits': { + 'cpus': '0.001', + 'memory': '50M', + }, + 'reservations': { + 'cpus': '0.0001', + 'memory': '20M', + }, + }, + 'restart_policy': { + 'condition': 'on_failure', + 'delay': '5s', + 'max_attempts': 3, + 'window': '120s', + }, + 'placement': { + 'constraints': ['node=foo'], + }, + }, + + 'healthcheck': { + 'command': 'cat /etc/passwd', + 'interval': '10s', + 'timeout': '1s', + 'retries': 5, + }, + + 'stop_grace_period': '20s', + }, + }, + } + def test_ps(self): self.project.get_service('simple').create_container() result = self.dispatch(['ps']) diff --git a/tests/fixtures/v3-full/docker-compose.yml b/tests/fixtures/v3-full/docker-compose.yml new file mode 100644 index 000000000..1187dd7b2 --- /dev/null +++ b/tests/fixtures/v3-full/docker-compose.yml @@ -0,0 +1,37 @@ +version: "3" +services: + web: + image: busybox + + deploy: + mode: replicated + replicas: 6 + labels: [FOO=BAR] + update_config: + parallelism: 3 + delay: 10s + failure_action: continue + monitor: 60s + max_failure_ratio: 0.3 + resources: + limits: + cpus: '0.001' + memory: 50M + reservations: + cpus: '0.0001' + memory: 20M + restart_policy: + condition: on_failure + delay: 5s + max_attempts: 3 + window: 120s + placement: + constraints: [node=foo] + + healthcheck: + command: cat /etc/passwd + interval: 10s + timeout: 1s + retries: 5 + + stop_grace_period: 20s diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index 51c5e226b..114145e1e 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -18,6 +18,7 @@ from compose.config.config import resolve_environment from compose.config.config import V1 from compose.config.config import V2_0 from compose.config.config import V2_1 +from compose.config.config import V3_0 from compose.config.environment import Environment from compose.config.errors import ConfigurationError from compose.config.errors import VERSION_EXPLANATION @@ -156,9 +157,14 @@ class ConfigTest(unittest.TestCase): for version in ['2', '2.0']: cfg = config.load(build_config_details({'version': version})) assert cfg.version == V2_0 + cfg = config.load(build_config_details({'version': '2.1'})) assert cfg.version == V2_1 + for version in ['3', '3.0']: + cfg = config.load(build_config_details({'version': version})) + assert cfg.version == V3_0 + def test_v1_file_version(self): cfg = config.load(build_config_details({'web': {'image': 'busybox'}})) assert cfg.version == V1 From f75ef6862fe20f378c9702cda652deb34a49f946 Mon Sep 17 00:00:00 2001 From: Aanand Prasad Date: Thu, 10 Nov 2016 17:30:46 +0000 Subject: [PATCH 22/45] Warn if any services use 'deploy' Signed-off-by: Aanand Prasad --- compose/config/config.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/compose/config/config.py b/compose/config/config.py index fb77436d1..940b9eb06 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -330,6 +330,14 @@ def load(config_details): for service_dict in service_dicts: match_named_volumes(service_dict, volumes) + services_using_deploy = [s for s in service_dicts if s.get('deploy')] + if services_using_deploy: + log.warn( + "Some services ({}) use the 'deploy' key, which will be ignored. " + "Compose does not support deploy configuration - use the experimental " + "`docker deploy` command to deploy to a swarm." + .format(", ".join(sorted(s['name'] for s in services_using_deploy)))) + return Config(main_file.version, service_dicts, volumes, networks) From 6cac48c0564f7f6b4e4d91055f50ec8c86505dd9 Mon Sep 17 00:00:00 2001 From: Daniel Nephin Date: Thu, 10 Nov 2016 17:38:19 -0500 Subject: [PATCH 23/45] Add a vendored and modified pytimeparse Signed-off-by: Daniel Nephin --- compose/timeparse.py | 96 ++++++++++++++++++++++++++++++++++++ tests/unit/timeparse_test.py | 52 +++++++++++++++++++ 2 files changed, 148 insertions(+) create mode 100644 compose/timeparse.py create mode 100644 tests/unit/timeparse_test.py diff --git a/compose/timeparse.py b/compose/timeparse.py new file mode 100644 index 000000000..16ef8a6dc --- /dev/null +++ b/compose/timeparse.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +''' +timeparse.py +(c) Will Roberts 1 February, 2014 + +This is a vendored and modified copy of: +github.com/wroberts/pytimeparse @ cc0550d + +It has been modified to mimic the behaviour of +https://golang.org/pkg/time/#ParseDuration +''' +# MIT LICENSE +# +# Permission is hereby granted, free of charge, to any person +# obtaining a copy of this software and associated documentation files +# (the "Software"), to deal in the Software without restriction, +# including without limitation the rights to use, copy, modify, merge, +# publish, distribute, sublicense, and/or sell copies of the Software, +# and to permit persons to whom the Software is furnished to do so, +# subject to the following conditions: +# +# The above copyright notice and this permission notice shall be +# included in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS +# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN +# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. +from __future__ import absolute_import +from __future__ import unicode_literals + +import re + +HOURS = r'(?P[\d.]+)h' +MINS = r'(?P[\d.]+)m' +SECS = r'(?P[\d.]+)s' +MILLI = r'(?P[\d.]+)ms' +MICRO = r'(?P[\d.]+)(?:us|µs)' +NANO = r'(?P[\d.]+)ns' + + +def opt(x): + return r'(?:{x})?'.format(x=x) + + +TIMEFORMAT = r'{HOURS}{MINS}{SECS}{MILLI}{MICRO}{NANO}'.format( + HOURS=opt(HOURS), + MINS=opt(MINS), + SECS=opt(SECS), + MILLI=opt(MILLI), + MICRO=opt(MICRO), + NANO=opt(NANO), +) + +MULTIPLIERS = dict([ + ('hours', 60 * 60), + ('mins', 60), + ('secs', 1), + ('milli', 1.0 / 1000), + ('micro', 1.0 / 1000.0 / 1000), + ('nano', 1.0 / 1000.0 / 1000.0 / 1000.0), +]) + + +def timeparse(sval): + """Parse a time expression, returning it as a number of seconds. If + possible, the return value will be an `int`; if this is not + possible, the return will be a `float`. Returns `None` if a time + expression cannot be parsed from the given string. + + Arguments: + - `sval`: the string value to parse + + >>> timeparse('1m24s') + 84 + >>> timeparse('1.2 minutes') + 72 + >>> timeparse('1.2 seconds') + 1.2 + """ + match = re.match(r'\s*' + TIMEFORMAT + r'\s*$', sval, re.I) + if not match or not match.group(0).strip(): + return + + mdict = match.groupdict() + return sum( + MULTIPLIERS[k] * cast(v) for (k, v) in mdict.items() if v is not None) + + +def cast(value): + return int(value, 10) if value.isdigit() else float(value) diff --git a/tests/unit/timeparse_test.py b/tests/unit/timeparse_test.py new file mode 100644 index 000000000..e9fe6c24c --- /dev/null +++ b/tests/unit/timeparse_test.py @@ -0,0 +1,52 @@ +from __future__ import absolute_import +from __future__ import unicode_literals + +from compose import timeparse + + +def test_milli(): + assert timeparse.timeparse('5ms') == 0.005 + + +def test_milli_float(): + assert timeparse.timeparse('50.5ms') == 0.0505 + + +def test_second_milli(): + assert timeparse.timeparse('200s5ms') == 200.005 + + +def test_second_milli_micro(): + assert timeparse.timeparse('200s5ms10us') == 200.00501 + + +def test_second(): + assert timeparse.timeparse('200s') == 200 + + +def test_second_as_float(): + assert timeparse.timeparse('20.5s') == 20.5 + + +def test_minute(): + assert timeparse.timeparse('32m') == 1920 + + +def test_hour_minute(): + assert timeparse.timeparse('2h32m') == 9120 + + +def test_minute_as_float(): + assert timeparse.timeparse('1.5m') == 90 + + +def test_hour_minute_second(): + assert timeparse.timeparse('5h34m56s') == 20096 + + +def test_invalid_with_space(): + assert timeparse.timeparse('5h 34m 56s') is None + + +def test_invalid_with_comma(): + assert timeparse.timeparse('5h,34m,56s') is None From 079c95c3401adb0f837e6b4e54132cdd41eada68 Mon Sep 17 00:00:00 2001 From: Daniel Nephin Date: Thu, 10 Nov 2016 17:47:05 -0500 Subject: [PATCH 24/45] Use stop grace period for container stop. Signed-off-by: Daniel Nephin --- compose/cli/main.py | 14 +++++++++----- compose/parallel.py | 4 ---- compose/project.py | 20 ++++++++++++++++---- compose/service.py | 23 ++++++++++++++++------- tests/unit/timeparse_test.py | 4 ++++ 5 files changed, 45 insertions(+), 20 deletions(-) diff --git a/compose/cli/main.py b/compose/cli/main.py index 08e58e372..cf53f6aa4 100644 --- a/compose/cli/main.py +++ b/compose/cli/main.py @@ -24,7 +24,6 @@ from ..config import ConfigurationError from ..config import parse_environment from ..config.environment import Environment from ..config.serialize import serialize_config -from ..const import DEFAULT_TIMEOUT from ..const import IS_WINDOWS_PLATFORM from ..errors import StreamParseError from ..progress_stream import StreamOutputError @@ -726,7 +725,7 @@ class TopLevelCommand(object): -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. (default: 10) """ - timeout = int(options.get('--timeout') or DEFAULT_TIMEOUT) + timeout = timeout_from_opts(options) for s in options['SERVICE=NUM']: if '=' not in s: @@ -760,7 +759,7 @@ class TopLevelCommand(object): -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. (default: 10) """ - timeout = int(options.get('--timeout') or DEFAULT_TIMEOUT) + timeout = timeout_from_opts(options) self.project.stop(service_names=options['SERVICE'], timeout=timeout) def restart(self, options): @@ -773,7 +772,7 @@ class TopLevelCommand(object): -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. (default: 10) """ - timeout = int(options.get('--timeout') or DEFAULT_TIMEOUT) + timeout = timeout_from_opts(options) containers = self.project.restart(service_names=options['SERVICE'], timeout=timeout) exit_if(not containers, 'No containers to restart', 1) @@ -831,7 +830,7 @@ class TopLevelCommand(object): start_deps = not options['--no-deps'] cascade_stop = options['--abort-on-container-exit'] service_names = options['SERVICE'] - timeout = int(options.get('--timeout') or DEFAULT_TIMEOUT) + timeout = timeout_from_opts(options) remove_orphans = options['--remove-orphans'] detached = options.get('-d') @@ -896,6 +895,11 @@ def convergence_strategy_from_opts(options): return ConvergenceStrategy.changed +def timeout_from_opts(options): + timeout = options.get('--timeout') + return None if timeout is None else int(timeout) + + def image_type_from_opt(flag, value): if not value: return ImageType.none diff --git a/compose/parallel.py b/compose/parallel.py index 7ac66b37a..267188728 100644 --- a/compose/parallel.py +++ b/compose/parallel.py @@ -248,7 +248,3 @@ def parallel_unpause(containers, options): def parallel_kill(containers, options): parallel_operation(containers, 'kill', options, 'Killing') - - -def parallel_restart(containers, options): - parallel_operation(containers, 'restart', options, 'Restarting') diff --git a/compose/project.py b/compose/project.py index 60647fe95..eef2f3b85 100644 --- a/compose/project.py +++ b/compose/project.py @@ -14,7 +14,6 @@ from .config import ConfigurationError from .config.config import V1 from .config.sort_services import get_container_name_from_network_mode from .config.sort_services import get_service_name_from_network_mode -from .const import DEFAULT_TIMEOUT from .const import IMAGE_EVENTS from .const import LABEL_ONE_OFF from .const import LABEL_PROJECT @@ -250,7 +249,7 @@ class Project(object): parallel.parallel_execute( containers, - operator.methodcaller('stop', **options), + self.build_container_operation_with_timeout_func('stop', options), operator.attrgetter('name'), 'Stopping', get_deps) @@ -291,7 +290,12 @@ class Project(object): def restart(self, service_names=None, **options): containers = self.containers(service_names, stopped=True) - parallel.parallel_restart(containers, options) + + parallel.parallel_execute( + containers, + self.build_container_operation_with_timeout_func('restart', options), + operator.attrgetter('name'), + 'Restarting') return containers def build(self, service_names=None, no_cache=False, pull=False, force_rm=False): @@ -365,7 +369,7 @@ class Project(object): start_deps=True, strategy=ConvergenceStrategy.changed, do_build=BuildAction.none, - timeout=DEFAULT_TIMEOUT, + timeout=None, detached=False, remove_orphans=False): @@ -506,6 +510,14 @@ class Project(object): dep_services.append(service) return acc + dep_services + def build_container_operation_with_timeout_func(self, operation, options): + def container_operation_with_timeout(container): + if options.get('timeout') is None: + service = self.get_service(container.service) + options['timeout'] = service.stop_timeout(None) + return getattr(container, operation)(**options) + return container_operation_with_timeout + def get_volumes_from(project, service_dict): volumes_from = service_dict.pop('volumes_from', None) diff --git a/compose/service.py b/compose/service.py index ad4267062..39737694d 100644 --- a/compose/service.py +++ b/compose/service.py @@ -17,6 +17,7 @@ from docker.utils.ports import split_port from . import __version__ from . import progress_stream +from . import timeparse from .config import DOCKER_CONFIG_KEYS from .config import merge_environment from .config.types import VolumeSpec @@ -169,7 +170,7 @@ class Service(object): self.start_container_if_stopped(c, **options) return containers - def scale(self, desired_num, timeout=DEFAULT_TIMEOUT): + def scale(self, desired_num, timeout=None): """ Adjusts the number of containers to the specified number and ensures they are running. @@ -196,7 +197,7 @@ class Service(object): return container def stop_and_remove(container): - container.stop(timeout=timeout) + container.stop(timeout=self.stop_timeout(timeout)) container.remove() running_containers = self.containers(stopped=False) @@ -374,7 +375,7 @@ class Service(object): def execute_convergence_plan(self, plan, - timeout=DEFAULT_TIMEOUT, + timeout=None, detached=False, start=True): (action, containers) = plan @@ -421,7 +422,7 @@ class Service(object): def recreate_container( self, container, - timeout=DEFAULT_TIMEOUT, + timeout=None, attach_logs=False, start_new_container=True): """Recreate a container. @@ -432,7 +433,7 @@ class Service(object): """ log.info("Recreating %s" % container.name) - container.stop(timeout=timeout) + container.stop(timeout=self.stop_timeout(timeout)) container.rename_to_tmp_name() new_container = self.create_container( previous_container=container, @@ -446,6 +447,14 @@ class Service(object): container.remove() return new_container + def stop_timeout(self, timeout): + if timeout is not None: + return timeout + timeout = timeparse.timeparse(self.options.get('stop_grace_period') or '') + if timeout is not None: + return timeout + return DEFAULT_TIMEOUT + def start_container_if_stopped(self, container, attach_logs=False, quiet=False): if not container.is_running: if not quiet: @@ -483,10 +492,10 @@ class Service(object): link_local_ips=netdefs.get('link_local_ips', None), ) - def remove_duplicate_containers(self, timeout=DEFAULT_TIMEOUT): + def remove_duplicate_containers(self, timeout=None): for c in self.duplicate_containers(): log.info('Removing %s' % c.name) - c.stop(timeout=timeout) + c.stop(timeout=self.stop_timeout(timeout)) c.remove() def duplicate_containers(self): diff --git a/tests/unit/timeparse_test.py b/tests/unit/timeparse_test.py index e9fe6c24c..9915932c3 100644 --- a/tests/unit/timeparse_test.py +++ b/tests/unit/timeparse_test.py @@ -50,3 +50,7 @@ def test_invalid_with_space(): def test_invalid_with_comma(): assert timeparse.timeparse('5h,34m,56s') is None + + +def test_invalid_with_empty_string(): + assert timeparse.timeparse('') is None From b93211881b913091501a76d062f033754a653740 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 16 Nov 2016 13:35:29 -0800 Subject: [PATCH 25/45] Changelog update Signed-off-by: Joffrey F --- CHANGELOG.md | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8ec7d5b57..174878909 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,106 @@ Change log ========== +1.9.0 (2016-11-16) +----------------- + +**Breaking changes** + +- When using Compose with Docker Toolbox/Machine on Windows, volume paths are + no longer converted from `C:\Users` to `/c/Users`-style by default. To + re-enable this conversion so that your volumes keep working, set the + environment variable `COMPOSE_CONVERT_WINDOWS_PATHS=1`. Users of + Docker for Windows are not affected and do not need to set the variable. + +New Features + +- Interactive mode for `docker-compose run` and `docker-compose exec` is + now supported on Windows platforms. Please note that the `docker` binary + is required to be present on the system for this feature to work. + +- Introduced version 2.1 of the `docker-compose.yml` specification. This + version requires to be used with Docker Engine 1.12 or above. + - Added support for setting volume labels and network labels in + `docker-compose.yml`. + - Added support for the `isolation` parameter in service definitions. + - Added support for link-local IPs in the service networks definitions. + - Added support for shell-style inline defaults in variable interpolation. + The supported forms are `${FOO-default}` (fall back if FOO is unset) and + `${FOO:-default}` (fall back if FOO is unset or empty). + +- Added support for the `group_add` and `oom_score_adj` parameters in + service definitions. + +- Added support for the `internal` and `enable_ipv6` parameters in network + definitions. + +- Compose now defaults to using the `npipe` protocol on Windows. + +- Overriding a `logging` configuration will now properly merge the `options` + mappings if the `driver` values do not conflict. + +Bug Fixes + +- Fixed several bugs related to `npipe` protocol support on Windows. + +- Fixed an issue with Windows paths being incorrectly converted when + using Docker on Windows Server. + +- Fixed a bug where an empty `restart` value would sometimes result in an + exception being raised. + +- Fixed an issue where service logs containing unicode characters would + sometimes cause an error to occur. + +- Fixed a bug where unicode values in environment variables would sometimes + raise a unicode exception when retrieved. + +- Fixed an issue where Compose would incorrectly detect a configuration + mismatch for overlay networks. + + +1.8.1 (2016-09-22) +----------------- + +Bug Fixes + +- Fixed a bug where users using a credentials store were not able + to access their private images. + +- Fixed a bug where users using identity tokens to authenticate + were not able to access their private images. + +- Fixed a bug where an `HttpHeaders` entry in the docker configuration + file would cause Compose to crash when trying to build an image. + +- Fixed a few bugs related to the handling of Windows paths in volume + binding declarations. + +- Fixed a bug where Compose would sometimes crash while trying to + read a streaming response from the engine. + +- Fixed an issue where Compose would crash when encountering an API error + while streaming container logs. + +- Fixed an issue where Compose would erroneously try to output logs from + drivers not handled by the Engine's API. + +- Fixed a bug where options from the `docker-machine config` command would + not be properly interpreted by Compose. + +- Fixed a bug where the connection to the Docker Engine would + sometimes fail when running a large number of services simultaneously. + +- Fixed an issue where Compose would sometimes print a misleading + suggestion message when running the `bundle` command. + +- Fixed a bug where connection errors would not be handled properly by + Compose during the project initialization phase. + +- Fixed a bug where a misleading error would appear when encountering + a connection timeout. + + 1.8.0 (2016-06-14) ----------------- From 716a6baa59c62266af0bb7628ec88586f470140c Mon Sep 17 00:00:00 2001 From: Aanand Prasad Date: Mon, 14 Nov 2016 18:31:38 +0000 Subject: [PATCH 26/45] Implement 'healthcheck' option Signed-off-by: Aanand Prasad --- compose/config/config.py | 29 +++++++++++ compose/config/config_schema_v3.0.json | 5 +- compose/service.py | 4 +- compose/utils.py | 16 ++++++ requirements.txt | 2 +- tests/acceptance/cli_test.py | 50 +++++++++++++++++-- tests/fixtures/healthcheck/docker-compose.yml | 24 +++++++++ tests/fixtures/v3-full/docker-compose.yml | 2 +- tests/unit/config/config_test.py | 49 ++++++++++++++++++ 9 files changed, 172 insertions(+), 9 deletions(-) create mode 100644 tests/fixtures/healthcheck/docker-compose.yml diff --git a/compose/config/config.py b/compose/config/config.py index 940b9eb06..e5a37e841 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -17,6 +17,7 @@ from ..const import COMPOSEFILE_V2_0 as V2_0 from ..const import COMPOSEFILE_V2_1 as V2_1 from ..const import COMPOSEFILE_V3_0 as V3_0 from ..utils import build_string_dict +from ..utils import parse_nanoseconds_int from ..utils import splitdrive from .environment import env_vars_from_file from .environment import Environment @@ -65,6 +66,7 @@ DOCKER_CONFIG_KEYS = [ 'extra_hosts', 'group_add', 'hostname', + 'healthcheck', 'image', 'ipc', 'labels', @@ -642,6 +644,10 @@ def process_service(service_config): if 'extra_hosts' in service_dict: service_dict['extra_hosts'] = parse_extra_hosts(service_dict['extra_hosts']) + if 'healthcheck' in service_dict: + service_dict['healthcheck'] = process_healthcheck( + service_dict['healthcheck'], service_config.name) + for field in ['dns', 'dns_search', 'tmpfs']: if field in service_dict: service_dict[field] = to_list(service_dict[field]) @@ -649,6 +655,29 @@ def process_service(service_config): return service_dict +def process_healthcheck(raw, service_name): + hc = {} + + if raw.get('disable'): + if len(raw) > 1: + raise ConfigurationError( + 'Service "{}" defines an invalid healthcheck: ' + '"disable: true" cannot be combined with other options' + .format(service_name)) + hc['test'] = ['NONE'] + elif 'test' in raw: + hc['test'] = raw['test'] + + if 'interval' in raw: + hc['interval'] = parse_nanoseconds_int(raw['interval']) + if 'timeout' in raw: + hc['timeout'] = parse_nanoseconds_int(raw['timeout']) + if 'retries' in raw: + hc['retries'] = raw['retries'] + + return hc + + def finalize_service(service_config, service_names, version, environment): service_dict = dict(service_config.config) diff --git a/compose/config/config_schema_v3.0.json b/compose/config/config_schema_v3.0.json index 9ac31b1f8..4edd8dd44 100644 --- a/compose/config/config_schema_v3.0.json +++ b/compose/config/config_schema_v3.0.json @@ -205,12 +205,13 @@ "interval": {"type":"string"}, "timeout": {"type":"string"}, "retries": {"type": "number"}, - "command": { + "test": { "oneOf": [ {"type": "string"}, {"type": "array", "items": {"type": "string"}} ] - } + }, + "disable": {"type": "boolean"} }, "additionalProperties": false }, diff --git a/compose/service.py b/compose/service.py index 39737694d..cf52d4893 100644 --- a/compose/service.py +++ b/compose/service.py @@ -17,7 +17,6 @@ from docker.utils.ports import split_port from . import __version__ from . import progress_stream -from . import timeparse from .config import DOCKER_CONFIG_KEYS from .config import merge_environment from .config.types import VolumeSpec @@ -35,6 +34,7 @@ from .parallel import parallel_start from .progress_stream import stream_output from .progress_stream import StreamOutputError from .utils import json_hash +from .utils import parse_seconds_float log = logging.getLogger(__name__) @@ -450,7 +450,7 @@ class Service(object): def stop_timeout(self, timeout): if timeout is not None: return timeout - timeout = timeparse.timeparse(self.options.get('stop_grace_period') or '') + timeout = parse_seconds_float(self.options.get('stop_grace_period')) if timeout is not None: return timeout return DEFAULT_TIMEOUT diff --git a/compose/utils.py b/compose/utils.py index 8f05e3081..b8bdf732f 100644 --- a/compose/utils.py +++ b/compose/utils.py @@ -11,6 +11,7 @@ import ntpath import six from .errors import StreamParseError +from .timeparse import timeparse json_decoder = json.JSONDecoder() @@ -107,6 +108,21 @@ def microseconds_from_time_nano(time_nano): return int(time_nano % 1000000000 / 1000) +def nanoseconds_from_time_seconds(time_seconds): + return time_seconds * 1000000000 + + +def parse_seconds_float(value): + return timeparse(value or '') + + +def parse_nanoseconds_int(value): + parsed = timeparse(value or '') + if parsed is None: + return None + return int(parsed * 1000000000) + + def build_string_dict(source_dict): return dict((k, str(v if v is not None else '')) for k, v in source_dict.items()) diff --git a/requirements.txt b/requirements.txt index 933146c72..63469799d 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,11 +1,11 @@ PyYAML==3.11 backports.ssl-match-hostname==3.5.0.1; python_version < '3' cached-property==1.2.0 -docker-py==1.10.6 dockerpty==0.4.1 docopt==0.6.1 enum34==1.0.4; python_version < '3.4' functools32==3.2.3.post2; python_version < '3.2' +git+https://github.com/docker/docker-py.git@2ff7371ae7703033f981e1b137a3be0caf7a4f9c#egg=docker-py ipaddress==1.0.16 jsonschema==2.5.1 pypiwin32==219; sys_platform == 'win32' diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index e9a416919..97518f4f0 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -21,6 +21,7 @@ from .. import mock from compose.cli.command import get_project from compose.container import Container from compose.project import OneOffFilter +from compose.utils import nanoseconds_from_time_seconds from tests.integration.testcases import DockerClientTestCase from tests.integration.testcases import get_links from tests.integration.testcases import pull_busybox @@ -329,9 +330,9 @@ class CLITestCase(DockerClientTestCase): }, 'healthcheck': { - 'command': 'cat /etc/passwd', - 'interval': '10s', - 'timeout': '1s', + 'test': 'cat /etc/passwd', + 'interval': 10000000000, + 'timeout': 1000000000, 'retries': 5, }, @@ -925,6 +926,49 @@ class CLITestCase(DockerClientTestCase): assert foo_container.get('HostConfig.NetworkMode') == \ 'container:{}'.format(bar_container.id) + def test_up_with_healthcheck(self): + def wait_on_health_status(container, status): + def condition(): + container.inspect() + return container.get('State.Health.Status') == status + + return wait_on_condition(condition, delay=0.5) + + self.base_dir = 'tests/fixtures/healthcheck' + self.dispatch(['up', '-d'], None) + + passes = self.project.get_service('passes') + passes_container = passes.containers()[0] + + assert passes_container.get('Config.Healthcheck') == { + "Test": ["CMD-SHELL", "/bin/true"], + "Interval": nanoseconds_from_time_seconds(1), + "Timeout": nanoseconds_from_time_seconds(30*60), + "Retries": 1, + } + + wait_on_health_status(passes_container, 'healthy') + + fails = self.project.get_service('fails') + fails_container = fails.containers()[0] + + assert fails_container.get('Config.Healthcheck') == { + "Test": ["CMD", "/bin/false"], + "Interval": nanoseconds_from_time_seconds(2.5), + "Retries": 2, + } + + wait_on_health_status(fails_container, 'unhealthy') + + disabled = self.project.get_service('disabled') + disabled_container = disabled.containers()[0] + + assert disabled_container.get('Config.Healthcheck') == { + "Test": ["NONE"], + } + + assert 'Health' not in disabled_container.get('State') + def test_up_with_no_deps(self): self.base_dir = 'tests/fixtures/links-composefile' self.dispatch(['up', '-d', '--no-deps', 'web'], None) diff --git a/tests/fixtures/healthcheck/docker-compose.yml b/tests/fixtures/healthcheck/docker-compose.yml new file mode 100644 index 000000000..2c45b8d8c --- /dev/null +++ b/tests/fixtures/healthcheck/docker-compose.yml @@ -0,0 +1,24 @@ +version: "3" +services: + passes: + image: busybox + command: top + healthcheck: + test: "/bin/true" + interval: 1s + timeout: 30m + retries: 1 + + fails: + image: busybox + command: top + healthcheck: + test: ["CMD", "/bin/false"] + interval: 2.5s + retries: 2 + + disabled: + image: busybox + command: top + healthcheck: + disable: true diff --git a/tests/fixtures/v3-full/docker-compose.yml b/tests/fixtures/v3-full/docker-compose.yml index 1187dd7b2..b4d1b6422 100644 --- a/tests/fixtures/v3-full/docker-compose.yml +++ b/tests/fixtures/v3-full/docker-compose.yml @@ -29,7 +29,7 @@ services: constraints: [node=foo] healthcheck: - command: cat /etc/passwd + test: cat /etc/passwd interval: 10s timeout: 1s retries: 5 diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index 114145e1e..f7df3aee4 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -24,6 +24,7 @@ from compose.config.errors import ConfigurationError from compose.config.errors import VERSION_EXPLANATION from compose.config.types import VolumeSpec from compose.const import IS_WINDOWS_PLATFORM +from compose.utils import nanoseconds_from_time_seconds from tests import mock from tests import unittest @@ -3171,6 +3172,54 @@ class BuildPathTest(unittest.TestCase): assert 'build path' in exc.exconly() +class HealthcheckTest(unittest.TestCase): + def test_healthcheck(self): + service_dict = make_service_dict( + 'test', + {'healthcheck': { + 'test': ['CMD', 'true'], + 'interval': '1s', + 'timeout': '1m', + 'retries': 3, + }}, + '.', + ) + + assert service_dict['healthcheck'] == { + 'test': ['CMD', 'true'], + 'interval': nanoseconds_from_time_seconds(1), + 'timeout': nanoseconds_from_time_seconds(60), + 'retries': 3, + } + + def test_disable(self): + service_dict = make_service_dict( + 'test', + {'healthcheck': { + 'disable': True, + }}, + '.', + ) + + assert service_dict['healthcheck'] == { + 'test': ['NONE'], + } + + def test_disable_with_other_config_is_invalid(self): + with pytest.raises(ConfigurationError) as excinfo: + make_service_dict( + 'invalid-healthcheck', + {'healthcheck': { + 'disable': True, + 'interval': '1s', + }}, + '.', + ) + + assert 'invalid-healthcheck' in excinfo.exconly() + assert 'disable' in excinfo.exconly() + + class GetDefaultConfigFilesTestCase(unittest.TestCase): files = [ From c26a2afaf36443fbc4ea813e34027bdb05ded0e1 Mon Sep 17 00:00:00 2001 From: Daniel Nephin Date: Mon, 21 Nov 2016 16:34:49 -0500 Subject: [PATCH 27/45] Update messages about docker stack deploy. Signed-off-by: Daniel Nephin --- compose/config/config.py | 4 ++-- compose/project.py | 4 +--- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/compose/config/config.py b/compose/config/config.py index 940b9eb06..5215b361f 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -334,8 +334,8 @@ def load(config_details): if services_using_deploy: log.warn( "Some services ({}) use the 'deploy' key, which will be ignored. " - "Compose does not support deploy configuration - use the experimental " - "`docker deploy` command to deploy to a swarm." + "Compose does not support deploy configuration - use " + "`docker stack deploy` to deploy to a swarm." .format(", ".join(sorted(s['name'] for s in services_using_deploy)))) return Config(main_file.version, service_dicts, volumes, networks) diff --git a/compose/project.py b/compose/project.py index eef2f3b85..0178bbab0 100644 --- a/compose/project.py +++ b/compose/project.py @@ -559,9 +559,7 @@ def warn_for_swarm_mode(client): "Compose does not use swarm mode to deploy services to multiple nodes in a swarm. " "All containers will be scheduled on the current node.\n\n" "To deploy your application across the swarm, " - "use the bundle feature of the Docker experimental build.\n\n" - "More info:\n" - "https://docs.docker.com/compose/bundles\n" + "use `docker stack deploy`.\n" ) From 024b5dd6da4c96822ea84e31cedcb88a8949a245 Mon Sep 17 00:00:00 2001 From: Thomas Grainger Date: Tue, 22 Nov 2016 11:14:56 +0000 Subject: [PATCH 28/45] case PyPI correctly Signed-off-by: Thomas Grainger --- CHANGELOG.md | 2 +- script/release/push-release | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 174878909..9780df98a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -814,7 +814,7 @@ Fig has been renamed to Docker Compose, or just Compose for short. This has seve - The command you type is now `docker-compose`, not `fig`. - You should rename your fig.yml to docker-compose.yml. -- If you’re installing via PyPi, the package is now `docker-compose`, so install it with `pip install docker-compose`. +- If you’re installing via PyPI, the package is now `docker-compose`, so install it with `pip install docker-compose`. Besides that, there’s a lot of new stuff in this release: diff --git a/script/release/push-release b/script/release/push-release index 33d0d7772..d5ae3de9d 100755 --- a/script/release/push-release +++ b/script/release/push-release @@ -54,7 +54,7 @@ git push $GITHUB_REPO $VERSION echo "Uploading the docker image" docker push docker/compose:$VERSION -echo "Uploading sdist to pypi" +echo "Uploading sdist to PyPI" pandoc -f markdown -t rst README.md -o README.rst sed -i -e 's/logo.png?raw=true/https:\/\/github.com\/docker\/compose\/raw\/master\/logo.png?raw=true/' README.rst ./script/build/write-git-sha From dc9184a90fd22b13265162c4bc8e04c9867a768d Mon Sep 17 00:00:00 2001 From: Ian Campbell Date: Fri, 25 Nov 2016 10:12:22 +0000 Subject: [PATCH 29/45] progress_stream: Avoid undefined ANSI escape codes The ANSI escape codes \e[0A (cursor up 0 lines) and \e[0B (cursor down 0 lines) are not well defined and are treated differently by different terminals. In particular xterm treats 0 as a missing parameter and therefore defaults to 1, whereas rxvt-unicode treats these escapes as a request to move 0 lines. However the use of these codes is unnecessary and were really just hiding the fact that we were not correctly computing diff when adding a new line. Having added the new line to the ids map and output the corresponding \n the correct diff would be 1 and not 0 (which xterm interprets as 1) as currently. Rather than changing the hardcoded 0 to a 1 pull the diff calculation out and always do it since it produces the correct answer in both cases. This fixes similar corruption when compose is pulling an image to that seen with `docker pull` and rxvt-unicode (and likely other terminals in that family) seen in docker/docker#28111. This is the same as the fix made to Docker's pkg/jsonmessage in https://github.com/docker/docker/pull/28238 (and I have shamelessly ripped off most of this commit message from there). Signed-off-by: Ian Campbell --- compose/progress_stream.py | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/compose/progress_stream.py b/compose/progress_stream.py index a0f5601f1..5314f89fd 100644 --- a/compose/progress_stream.py +++ b/compose/progress_stream.py @@ -32,12 +32,11 @@ def stream_output(output, stream): if not image_id: continue - if image_id in lines: - diff = len(lines) - lines[image_id] - else: + if image_id not in lines: lines[image_id] = len(lines) stream.write("\n") - diff = 0 + + diff = len(lines) - lines[image_id] # move cursor up `diff` rows stream.write("%c[%dA" % (27, diff)) From 6aacf5142750654a9b1f165223478fab3a424eeb Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Fri, 2 Dec 2016 16:10:15 -0800 Subject: [PATCH 30/45] Win32 interactive run - Connect container to networks before starting Signed-off-by: Joffrey F --- compose/cli/main.py | 1 + 1 file changed, 1 insertion(+) diff --git a/compose/cli/main.py b/compose/cli/main.py index cf53f6aa4..c25ccbfa4 100644 --- a/compose/cli/main.py +++ b/compose/cli/main.py @@ -988,6 +988,7 @@ def run_one_off_container(container_options, project, service, options): try: try: if IS_WINDOWS_PLATFORM: + service.connect_container_to_networks(container) exit_code = call_docker(["start", "--attach", "--interactive", container.id]) else: operation = RunOperation( From 62f8b1402e1bbe726cf83ea204077754385fe53a Mon Sep 17 00:00:00 2001 From: Yong Wen Chua Date: Mon, 5 Dec 2016 14:25:56 +0800 Subject: [PATCH 31/45] Implement `userns_mode` HostConfig for services Fixes #3349 This allows the key `userns_mode` to be used in service definitions. Since `userns_mode` requires API version > 1.23, this is only available in 2.1 and 3.0 versions of compose file Signed-off-by: Yong Wen Chua --- compose/config/config.py | 1 + compose/config/config_schema_v2.1.json | 1 + compose/config/config_schema_v3.0.json | 1 + compose/service.py | 4 +++- tests/integration/service_test.py | 13 +++++++++++++ 5 files changed, 19 insertions(+), 1 deletion(-) diff --git a/compose/config/config.py b/compose/config/config.py index 5215b361f..b83b12bff 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -86,6 +86,7 @@ DOCKER_CONFIG_KEYS = [ 'stop_signal', 'tty', 'user', + 'userns_mode', 'volume_driver', 'volumes', 'volumes_from', diff --git a/compose/config/config_schema_v2.1.json b/compose/config/config_schema_v2.1.json index 3561a8cf2..b5d614d6e 100644 --- a/compose/config/config_schema_v2.1.json +++ b/compose/config/config_schema_v2.1.json @@ -217,6 +217,7 @@ } }, "user": {"type": "string"}, + "userns_mode": {"type": "string"}, "volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "volume_driver": {"type": "string"}, "volumes_from": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, diff --git a/compose/config/config_schema_v3.0.json b/compose/config/config_schema_v3.0.json index 9ac31b1f8..297c60be0 100644 --- a/compose/config/config_schema_v3.0.json +++ b/compose/config/config_schema_v3.0.json @@ -192,6 +192,7 @@ } }, "user": {"type": "string"}, + "userns_mode": {"type": "string"}, "volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "working_dir": {"type": "string"} }, diff --git a/compose/service.py b/compose/service.py index 39737694d..6a62067b8 100644 --- a/compose/service.py +++ b/compose/service.py @@ -64,6 +64,7 @@ DOCKER_START_KEYS = [ 'restart', 'security_opt', 'shm_size', + 'userns_mode', 'volumes_from', ] @@ -720,7 +721,8 @@ class Service(object): tmpfs=options.get('tmpfs'), oom_score_adj=options.get('oom_score_adj'), mem_swappiness=options.get('mem_swappiness'), - group_add=options.get('group_add') + group_add=options.get('group_add'), + userns_mode=options.get('userns_mode') ) # TODO: Add as an argument to create_host_config once it's supported diff --git a/tests/integration/service_test.py b/tests/integration/service_test.py index a5ca81ee3..09758eee9 100644 --- a/tests/integration/service_test.py +++ b/tests/integration/service_test.py @@ -30,6 +30,7 @@ from compose.service import ConvergencePlan from compose.service import ConvergenceStrategy from compose.service import NetworkMode from compose.service import Service +from tests.integration.testcases import v2_1_only from tests.integration.testcases import v2_only @@ -842,6 +843,18 @@ class ServiceTest(DockerClientTestCase): container = create_and_start_container(service) self.assertEqual(container.get('HostConfig.PidMode'), 'host') + @v2_1_only() + def test_userns_mode_none_defined(self): + service = self.create_service('web', userns_mode=None) + container = create_and_start_container(service) + self.assertEqual(container.get('HostConfig.UsernsMode'), '') + + @v2_1_only() + def test_userns_mode_host(self): + service = self.create_service('web', userns_mode='host') + container = create_and_start_container(service) + self.assertEqual(container.get('HostConfig.UsernsMode'), 'host') + def test_dns_no_value(self): service = self.create_service('web') container = create_and_start_container(service) From 4d0575355c1a0201ed7fc11063a4efbca6971a96 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Tue, 6 Dec 2016 17:14:05 -0800 Subject: [PATCH 32/45] Bump master version to 1.10.0dev Signed-off-by: Joffrey F --- compose/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/compose/__init__.py b/compose/__init__.py index 6e6106527..6f05b282f 100644 --- a/compose/__init__.py +++ b/compose/__init__.py @@ -1,4 +1,4 @@ from __future__ import absolute_import from __future__ import unicode_literals -__version__ = '1.9.0dev' +__version__ = '1.10.0dev' From e04a12b5ca9fb30d152ab6a83ae8c0ec15ac85b0 Mon Sep 17 00:00:00 2001 From: Daniel Nephin Date: Tue, 6 Dec 2016 17:01:35 -0500 Subject: [PATCH 33/45] Increase minimum version for v3. Signed-off-by: Daniel Nephin --- compose/const.py | 4 ++-- tests/acceptance/cli_test.py | 2 ++ tests/integration/testcases.py | 23 +++++++++++------------ 3 files changed, 15 insertions(+), 14 deletions(-) diff --git a/compose/const.py b/compose/const.py index 3da398551..ca8d7fe51 100644 --- a/compose/const.py +++ b/compose/const.py @@ -23,12 +23,12 @@ API_VERSIONS = { COMPOSEFILE_V1: '1.21', COMPOSEFILE_V2_0: '1.22', COMPOSEFILE_V2_1: '1.24', - COMPOSEFILE_V3_0: '1.24', + COMPOSEFILE_V3_0: '1.25', } API_VERSION_TO_ENGINE_VERSION = { API_VERSIONS[COMPOSEFILE_V1]: '1.9.0', API_VERSIONS[COMPOSEFILE_V2_0]: '1.10.0', API_VERSIONS[COMPOSEFILE_V2_1]: '1.12.0', - API_VERSIONS[COMPOSEFILE_V3_0]: '1.12.0', + API_VERSIONS[COMPOSEFILE_V3_0]: '1.13.0', } diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index e9a416919..0d5d50588 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -26,6 +26,7 @@ from tests.integration.testcases import get_links from tests.integration.testcases import pull_busybox from tests.integration.testcases import v2_1_only from tests.integration.testcases import v2_only +from tests.integration.testcases import v3_only ProcessResult = namedtuple('ProcessResult', 'stdout stderr') @@ -285,6 +286,7 @@ class CLITestCase(DockerClientTestCase): 'volumes': {}, } + @v3_only() def test_config_v3(self): self.base_dir = 'tests/fixtures/v3-full' result = self.dispatch(['config']) diff --git a/tests/integration/testcases.py b/tests/integration/testcases.py index c7743fb83..f6bc402bf 100644 --- a/tests/integration/testcases.py +++ b/tests/integration/testcases.py @@ -45,11 +45,11 @@ def engine_max_version(): return V2_1 -def v2_only(): +def build_version_required_decorator(ignored_versions): def decorator(f): @functools.wraps(f) def wrapper(self, *args, **kwargs): - if engine_max_version() == V1: + if engine_max_version() in ignored_versions: skip("Engine version is too low") return return f(self, *args, **kwargs) @@ -58,17 +58,16 @@ def v2_only(): return decorator -def v2_1_only(): - def decorator(f): - @functools.wraps(f) - def wrapper(self, *args, **kwargs): - if engine_max_version() in (V1, V2_0): - skip('Engine version is too low') - return - return f(self, *args, **kwargs) - return wrapper +def v2_only(): + return build_version_required_decorator((V1,)) - return decorator + +def v2_1_only(): + return build_version_required_decorator((V1, V2_0)) + + +def v3_only(): + return build_version_required_decorator((V1, V2_0, V2_1)) class DockerClientTestCase(unittest.TestCase): From 04e5925a230d07a6ffd855b4c3812f5a6b54a523 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 5 Dec 2016 17:42:07 -0800 Subject: [PATCH 34/45] Use docker SDK 2.0 Signed-off-by: Joffrey F --- compose/cli/docker_client.py | 4 ++-- compose/network.py | 8 ++++---- compose/service.py | 2 +- requirements.txt | 2 +- setup.py | 2 +- tests/unit/bundle_test.py | 2 +- tests/unit/cli_test.py | 4 ++-- tests/unit/container_test.py | 2 +- tests/unit/project_test.py | 2 +- tests/unit/service_test.py | 10 +++++----- tests/unit/volume_test.py | 2 +- 11 files changed, 20 insertions(+), 20 deletions(-) diff --git a/compose/cli/docker_client.py b/compose/cli/docker_client.py index b196d3036..018d24513 100644 --- a/compose/cli/docker_client.py +++ b/compose/cli/docker_client.py @@ -3,7 +3,7 @@ from __future__ import unicode_literals import logging -from docker import Client +from docker import APIClient from docker.errors import TLSParameterError from docker.tls import TLSConfig from docker.utils import kwargs_from_env @@ -71,4 +71,4 @@ def docker_client(environment, version=None, tls_config=None, host=None, kwargs['user_agent'] = generate_user_agent() - return Client(**kwargs) + return APIClient(**kwargs) diff --git a/compose/network.py b/compose/network.py index 3b57cb94e..8a29b73e2 100644 --- a/compose/network.py +++ b/compose/network.py @@ -4,8 +4,8 @@ from __future__ import unicode_literals import logging from docker.errors import NotFound -from docker.utils import create_ipam_config -from docker.utils import create_ipam_pool +from docker.types import IPAMConfig +from docker.types import IPAMPool from .config import ConfigurationError @@ -96,10 +96,10 @@ def create_ipam_config_from_dict(ipam_dict): if not ipam_dict: return None - return create_ipam_config( + return IPAMConfig( driver=ipam_dict.get('driver'), pool_configs=[ - create_ipam_pool( + IPAMPool( subnet=config.get('subnet'), iprange=config.get('ip_range'), gateway=config.get('gateway'), diff --git a/compose/service.py b/compose/service.py index cf52d4893..c32788fe2 100644 --- a/compose/service.py +++ b/compose/service.py @@ -11,7 +11,7 @@ import enum import six from docker.errors import APIError from docker.errors import NotFound -from docker.utils import LogConfig +from docker.types import LogConfig from docker.utils.ports import build_port_bindings from docker.utils.ports import split_port diff --git a/requirements.txt b/requirements.txt index 63469799d..8f6fe1695 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,11 +1,11 @@ PyYAML==3.11 backports.ssl-match-hostname==3.5.0.1; python_version < '3' cached-property==1.2.0 +docker==2.0.0 dockerpty==0.4.1 docopt==0.6.1 enum34==1.0.4; python_version < '3.4' functools32==3.2.3.post2; python_version < '3.2' -git+https://github.com/docker/docker-py.git@2ff7371ae7703033f981e1b137a3be0caf7a4f9c#egg=docker-py ipaddress==1.0.16 jsonschema==2.5.1 pypiwin32==219; sys_platform == 'win32' diff --git a/setup.py b/setup.py index 672ea80e5..9dba21670 100644 --- a/setup.py +++ b/setup.py @@ -34,7 +34,7 @@ install_requires = [ 'requests >= 2.6.1, != 2.11.0, < 2.12', 'texttable >= 0.8.1, < 0.9', 'websocket-client >= 0.32.0, < 1.0', - 'docker-py >= 1.10.6, < 2.0', + 'docker >= 2.0.0, < 3.0', 'dockerpty >= 0.4.1, < 0.5', 'six >= 1.3.0, < 2', 'jsonschema >= 2.5.1, < 3', diff --git a/tests/unit/bundle_test.py b/tests/unit/bundle_test.py index 223b3b07a..a279cab05 100644 --- a/tests/unit/bundle_test.py +++ b/tests/unit/bundle_test.py @@ -15,7 +15,7 @@ from compose.config.config import Config def mock_service(): return mock.create_autospec( service.Service, - client=mock.create_autospec(docker.Client), + client=mock.create_autospec(docker.APIClient), options={}) diff --git a/tests/unit/cli_test.py b/tests/unit/cli_test.py index 2c90b29b7..f9b60bff5 100644 --- a/tests/unit/cli_test.py +++ b/tests/unit/cli_test.py @@ -97,7 +97,7 @@ class CLITestCase(unittest.TestCase): @mock.patch('compose.cli.main.RunOperation', autospec=True) @mock.patch('compose.cli.main.PseudoTerminal', autospec=True) def test_run_interactive_passes_logs_false(self, mock_pseudo_terminal, mock_run_operation): - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) project = Project.from_config( name='composetest', client=mock_client, @@ -128,7 +128,7 @@ class CLITestCase(unittest.TestCase): assert call_kwargs['logs'] is False def test_run_service_with_restart_always(self): - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) project = Project.from_config( name='composetest', diff --git a/tests/unit/container_test.py b/tests/unit/container_test.py index 62e3aa2cf..04f430162 100644 --- a/tests/unit/container_test.py +++ b/tests/unit/container_test.py @@ -98,7 +98,7 @@ class ContainerTest(unittest.TestCase): self.assertEqual(container.name_without_project, "custom_name_of_container") def test_inspect_if_not_inspected(self): - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) container = Container(mock_client, dict(Id="the_id")) container.inspect_if_not_inspected() diff --git a/tests/unit/project_test.py b/tests/unit/project_test.py index 9569adc90..9a12438f2 100644 --- a/tests/unit/project_test.py +++ b/tests/unit/project_test.py @@ -19,7 +19,7 @@ from compose.service import Service class ProjectTest(unittest.TestCase): def setUp(self): - self.mock_client = mock.create_autospec(docker.Client) + self.mock_client = mock.create_autospec(docker.APIClient) def test_from_config(self): config = Config( diff --git a/tests/unit/service_test.py b/tests/unit/service_test.py index 1d5aa10fb..2d5b17619 100644 --- a/tests/unit/service_test.py +++ b/tests/unit/service_test.py @@ -34,7 +34,7 @@ from compose.service import warn_on_masked_volume class ServiceTest(unittest.TestCase): def setUp(self): - self.mock_client = mock.create_autospec(docker.Client) + self.mock_client = mock.create_autospec(docker.APIClient) def test_containers(self): service = Service('db', self.mock_client, 'myproject', image='foo') @@ -666,7 +666,7 @@ class ServiceTest(unittest.TestCase): class TestServiceNetwork(object): def test_connect_container_to_networks_short_aliase_exists(self): - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) service = Service( 'db', mock_client, @@ -751,7 +751,7 @@ class NetTestCase(unittest.TestCase): def test_network_mode_service(self): container_id = 'bbbb' service_name = 'web' - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) mock_client.containers.return_value = [ {'Id': container_id, 'Name': container_id, 'Image': 'abcd'}, ] @@ -765,7 +765,7 @@ class NetTestCase(unittest.TestCase): def test_network_mode_service_no_containers(self): service_name = 'web' - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) mock_client.containers.return_value = [] service = Service(name=service_name, client=mock_client) @@ -783,7 +783,7 @@ def build_mount(destination, source, mode='rw'): class ServiceVolumesTest(unittest.TestCase): def setUp(self): - self.mock_client = mock.create_autospec(docker.Client) + self.mock_client = mock.create_autospec(docker.APIClient) def test_build_volume_binding(self): binding = build_volume_binding(VolumeSpec.parse('/outside:/inside', True)) diff --git a/tests/unit/volume_test.py b/tests/unit/volume_test.py index d7ad07928..24829192a 100644 --- a/tests/unit/volume_test.py +++ b/tests/unit/volume_test.py @@ -10,7 +10,7 @@ from tests import mock @pytest.fixture def mock_client(): - return mock.create_autospec(docker.Client) + return mock.create_autospec(docker.APIClient) class TestVolume(object): From fb165d9c1505e2505f637cb2cede77e4d0731884 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Fri, 16 Dec 2016 12:21:59 -0800 Subject: [PATCH 35/45] Add v3_only marker to healthcheck test Signed-off-by: Joffrey F --- tests/acceptance/cli_test.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index 856b8f93c..2d0ce7155 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -316,8 +316,8 @@ class CLITestCase(DockerClientTestCase): 'memory': '50M', }, 'reservations': { - 'cpus': '0.0001', - 'memory': '20M', + 'cpus': '0.0001', + 'memory': '20M', }, }, 'restart_policy': { @@ -928,6 +928,7 @@ class CLITestCase(DockerClientTestCase): assert foo_container.get('HostConfig.NetworkMode') == \ 'container:{}'.format(bar_container.id) + @v3_only() def test_up_with_healthcheck(self): def wait_on_health_status(container, status): def condition(): @@ -945,7 +946,7 @@ class CLITestCase(DockerClientTestCase): assert passes_container.get('Config.Healthcheck') == { "Test": ["CMD-SHELL", "/bin/true"], "Interval": nanoseconds_from_time_seconds(1), - "Timeout": nanoseconds_from_time_seconds(30*60), + "Timeout": nanoseconds_from_time_seconds(30 * 60), "Retries": 1, } From e736151ee497a65df4e7e271ebd059731e250760 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 5 Dec 2016 16:11:21 -0800 Subject: [PATCH 36/45] Make created networks attachable for file format >=2.1 Signed-off-by: Joffrey F --- compose/network.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/compose/network.py b/compose/network.py index 8a29b73e2..eb76e2920 100644 --- a/compose/network.py +++ b/compose/network.py @@ -6,6 +6,7 @@ import logging from docker.errors import NotFound from docker.types import IPAMConfig from docker.types import IPAMPool +from docker.utils import version_gte from .config import ConfigurationError @@ -72,6 +73,7 @@ class Network(object): internal=self.internal, enable_ipv6=self.enable_ipv6, labels=self.labels, + attachable=version_gte(self.client._version, '1.24') or None ) def remove(self): From eb6441c8e337a1e797adee26b77c0c03465375a2 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 19 Dec 2016 20:35:09 -0800 Subject: [PATCH 37/45] Add sysctls option to 3.0 schema Signed-off-by: Joffrey F --- compose/config/config_schema_v3.0.json | 1 + 1 file changed, 1 insertion(+) diff --git a/compose/config/config_schema_v3.0.json b/compose/config/config_schema_v3.0.json index 8b8ceac07..1f93347f9 100644 --- a/compose/config/config_schema_v3.0.json +++ b/compose/config/config_schema_v3.0.json @@ -167,6 +167,7 @@ "restart": {"type": "string"}, "security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "shm_size": {"type": ["number", "string"]}, + "sysctls": {"$ref": "#/definitions/list_or_dict"}, "stdin_open": {"type": "boolean"}, "stop_signal": {"type": "string"}, "stop_grace_period": {"type": "string", "format": "duration"}, From 346802715dd3869c2a33c5d361ec8a56fa3352c5 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 16:27:47 -0700 Subject: [PATCH 38/45] Use colorama to enable colored output on Windows Signed-off-by: Joffrey F --- compose/cli/colors.py | 4 ++++ requirements.txt | 1 + setup.py | 1 + 3 files changed, 6 insertions(+) diff --git a/compose/cli/colors.py b/compose/cli/colors.py index 3c18886f8..6677a376a 100644 --- a/compose/cli/colors.py +++ b/compose/cli/colors.py @@ -1,5 +1,8 @@ from __future__ import absolute_import from __future__ import unicode_literals + +import colorama + NAMES = [ 'grey', 'red', @@ -30,6 +33,7 @@ def make_color_fn(code): return lambda s: ansi_color(code, s) +colorama.init() for (name, code) in get_pairs(): globals()[name] = make_color_fn(code) diff --git a/requirements.txt b/requirements.txt index 8f6fe1695..bae5d9ea1 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,6 +1,7 @@ PyYAML==3.11 backports.ssl-match-hostname==3.5.0.1; python_version < '3' cached-property==1.2.0 +colorama==0.3.7 docker==2.0.0 dockerpty==0.4.1 docopt==0.6.1 diff --git a/setup.py b/setup.py index 9dba21670..8b4cf709e 100644 --- a/setup.py +++ b/setup.py @@ -29,6 +29,7 @@ def find_version(*file_paths): install_requires = [ 'cached-property >= 1.2.0, < 2', + 'colorama >= 0.3.7, < 0.4', 'docopt >= 0.6.1, < 0.7', 'PyYAML >= 3.10, < 4', 'requests >= 2.6.1, != 2.11.0, < 2.12', From ba47fb99ba7670fc8435af0a958a3ed01188711a Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 5 Dec 2016 18:21:10 -0800 Subject: [PATCH 39/45] Add default labels to networks and volumes created by Compose Signed-off-by: Joffrey F --- compose/const.py | 2 ++ compose/network.py | 18 ++++++++++++++++-- compose/volume.py | 16 +++++++++++++++- tests/acceptance/cli_test.py | 8 ++++---- tests/integration/network_test.py | 17 +++++++++++++++++ tests/integration/project_test.py | 7 ++++--- tests/integration/volume_test.py | 10 ++++++++++ 7 files changed, 68 insertions(+), 10 deletions(-) create mode 100644 tests/integration/network_test.py diff --git a/compose/const.py b/compose/const.py index ca8d7fe51..1b1be5c76 100644 --- a/compose/const.py +++ b/compose/const.py @@ -11,7 +11,9 @@ LABEL_CONTAINER_NUMBER = 'com.docker.compose.container-number' LABEL_ONE_OFF = 'com.docker.compose.oneoff' LABEL_PROJECT = 'com.docker.compose.project' LABEL_SERVICE = 'com.docker.compose.service' +LABEL_NETWORK = 'com.docker.compose.network' LABEL_VERSION = 'com.docker.compose.version' +LABEL_VOLUME = 'com.docker.compose.volume' LABEL_CONFIG_HASH = 'com.docker.compose.config-hash' COMPOSEFILE_V1 = '1' diff --git a/compose/network.py b/compose/network.py index eb76e2920..d98f68d2f 100644 --- a/compose/network.py +++ b/compose/network.py @@ -7,8 +7,11 @@ from docker.errors import NotFound from docker.types import IPAMConfig from docker.types import IPAMPool from docker.utils import version_gte +from docker.utils import version_lt from .config import ConfigurationError +from .const import LABEL_NETWORK +from .const import LABEL_PROJECT log = logging.getLogger(__name__) @@ -72,8 +75,8 @@ class Network(object): ipam=self.ipam, internal=self.internal, enable_ipv6=self.enable_ipv6, - labels=self.labels, - attachable=version_gte(self.client._version, '1.24') or None + labels=self._labels, + attachable=version_gte(self.client._version, '1.24') or None, ) def remove(self): @@ -93,6 +96,17 @@ class Network(object): return self.external_name return '{0}_{1}'.format(self.project, self.name) + @property + def _labels(self): + if version_lt(self.client._version, '1.23'): + return None + labels = self.labels.copy() if self.labels else {} + labels.update({ + LABEL_PROJECT: self.project, + LABEL_NETWORK: self.name, + }) + return labels + def create_ipam_config_from_dict(ipam_dict): if not ipam_dict: diff --git a/compose/volume.py b/compose/volume.py index 1fd1d51c9..ab6a88fac 100644 --- a/compose/volume.py +++ b/compose/volume.py @@ -4,8 +4,11 @@ from __future__ import unicode_literals import logging from docker.errors import NotFound +from docker.utils import version_lt from .config import ConfigurationError +from .const import LABEL_PROJECT +from .const import LABEL_VOLUME log = logging.getLogger(__name__) @@ -23,7 +26,7 @@ class Volume(object): def create(self): return self.client.create_volume( - self.full_name, self.driver, self.driver_opts, labels=self.labels + self.full_name, self.driver, self.driver_opts, labels=self._labels ) def remove(self): @@ -53,6 +56,17 @@ class Volume(object): return self.external_name return '{0}_{1}'.format(self.project, self.name) + @property + def _labels(self): + if version_lt(self.client._version, '1.23'): + return None + labels = self.labels.copy() if self.labels else {} + labels.update({ + LABEL_PROJECT: self.project, + LABEL_VOLUME: self.name, + }) + return labels + class ProjectVolumes(object): diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index 2d0ce7155..b9766226d 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -850,8 +850,8 @@ class CLITestCase(DockerClientTestCase): ] assert [n['Name'] for n in networks] == [network_with_label] - - assert networks[0]['Labels'] == {'label_key': 'label_val'} + assert 'label_key' in networks[0]['Labels'] + assert networks[0]['Labels']['label_key'] == 'label_val' @v2_1_only() def test_up_with_volume_labels(self): @@ -870,8 +870,8 @@ class CLITestCase(DockerClientTestCase): ] assert [v['Name'] for v in volumes] == [volume_with_label] - - assert volumes[0]['Labels'] == {'label_key': 'label_val'} + assert 'label_key' in volumes[0]['Labels'] + assert volumes[0]['Labels']['label_key'] == 'label_val' @v2_only() def test_up_no_services(self): diff --git a/tests/integration/network_test.py b/tests/integration/network_test.py new file mode 100644 index 000000000..2ff610fbf --- /dev/null +++ b/tests/integration/network_test.py @@ -0,0 +1,17 @@ +from __future__ import absolute_import +from __future__ import unicode_literals + +from .testcases import DockerClientTestCase +from compose.const import LABEL_NETWORK +from compose.const import LABEL_PROJECT +from compose.network import Network + + +class NetworkTest(DockerClientTestCase): + def test_network_default_labels(self): + net = Network(self.client, 'composetest', 'foonet') + net.ensure() + net_data = net.inspect() + labels = net_data['Labels'] + assert labels[LABEL_NETWORK] == net.name + assert labels[LABEL_PROJECT] == net.project diff --git a/tests/integration/project_test.py b/tests/integration/project_test.py index 3afefb5ab..de0732393 100644 --- a/tests/integration/project_test.py +++ b/tests/integration/project_test.py @@ -942,8 +942,8 @@ class ProjectTest(DockerClientTestCase): ] assert [n['Name'] for n in networks] == ['composetest_{}'.format(network_name)] - - assert networks[0]['Labels'] == {'label_key': 'label_val'} + assert 'label_key' in networks[0]['Labels'] + assert networks[0]['Labels']['label_key'] == 'label_val' @v2_only() def test_project_up_volumes(self): @@ -1009,7 +1009,8 @@ class ProjectTest(DockerClientTestCase): assert [v['Name'] for v in volumes] == ['composetest_{}'.format(volume_name)] - assert volumes[0]['Labels'] == {'label_key': 'label_val'} + assert 'label_key' in volumes[0]['Labels'] + assert volumes[0]['Labels']['label_key'] == 'label_val' @v2_only() def test_project_up_logging_with_multiple_files(self): diff --git a/tests/integration/volume_test.py b/tests/integration/volume_test.py index a75250ac7..add169623 100644 --- a/tests/integration/volume_test.py +++ b/tests/integration/volume_test.py @@ -4,6 +4,8 @@ from __future__ import unicode_literals from docker.errors import DockerException from .testcases import DockerClientTestCase +from compose.const import LABEL_PROJECT +from compose.const import LABEL_VOLUME from compose.volume import Volume @@ -94,3 +96,11 @@ class VolumeTest(DockerClientTestCase): assert vol.exists() is False vol.create() assert vol.exists() is True + + def test_volume_default_labels(self): + vol = self.create_volume('volume01') + vol.create() + vol_data = vol.inspect() + labels = vol_data['Labels'] + assert labels[LABEL_VOLUME] == vol.name + assert labels[LABEL_PROJECT] == vol.project From 0edfe08bf017841a1a2624b98e4a05bb11fc6d4f Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Tue, 13 Dec 2016 16:54:41 -0800 Subject: [PATCH 40/45] Add healthchecks to 2.1 schema. Update depends_on Signed-off-by: Joffrey F --- compose/config/config_schema_v2.1.json | 42 +++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/compose/config/config_schema_v2.1.json b/compose/config/config_schema_v2.1.json index 3057f2dee..5ab9f71fb 100644 --- a/compose/config/config_schema_v2.1.json +++ b/compose/config/config_schema_v2.1.json @@ -77,7 +77,28 @@ "cpu_shares": {"type": ["number", "string"]}, "cpu_quota": {"type": ["number", "string"]}, "cpuset": {"type": "string"}, - "depends_on": {"$ref": "#/definitions/list_of_strings"}, + "depends_on": { + "oneOf": [ + {"$ref": "#/definitions/list_of_strings"}, + { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "type": "object", + "additionalProperties": false, + "properties": { + "condition": { + "type": "string", + "enum": ["service_started", "service_healthy"] + } + }, + "required": ["condition"] + } + } + } + ] + }, "devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "dns": {"$ref": "#/definitions/string_or_list"}, "dns_search": {"$ref": "#/definitions/string_or_list"}, @@ -120,6 +141,7 @@ "external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "extra_hosts": {"$ref": "#/definitions/list_or_dict"}, + "healthcheck": {"$ref": "#/definitions/healthcheck"}, "hostname": {"type": "string"}, "image": {"type": "string"}, "ipc": {"type": "string"}, @@ -231,6 +253,24 @@ "additionalProperties": false }, + "healthcheck": { + "id": "#/definitions/healthcheck", + "type": "object", + "additionalProperties": false, + "properties": { + "disabled": {"type": "boolean"}, + "interval": {"type": "string"}, + "retries": {"type": "number"}, + "test": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "timeout": {"type": "string"} + } + }, + "network": { "id": "#/definitions/network", "type": "object", From f6edd610f36ec9f6ffdd16df970f3d6cefb1169d Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 14 Dec 2016 15:39:42 -0800 Subject: [PATCH 41/45] Add 3.0 schema to docker-compose.spec Signed-off-by: Joffrey F --- docker-compose.spec | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docker-compose.spec b/docker-compose.spec index e57d6b7a1..ec5a2039c 100644 --- a/docker-compose.spec +++ b/docker-compose.spec @@ -32,6 +32,11 @@ exe = EXE(pyz, 'compose/config/config_schema_v2.1.json', 'DATA' ), + ( + 'compose/config/config_schema_v3.0.json', + 'compose/config/config_schema_v3.0.json', + 'DATA' + ), ( 'compose/GITSHA', 'compose/GITSHA', From 04394b1d0a82f4507edc8863c4ab9cf64944f6d9 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 19 Dec 2016 20:20:03 -0800 Subject: [PATCH 42/45] Expand depends_on to allow different conditions (service_start, service_healthy) Rework "up" and "start" to wait on conditional state of dependent services Add integration tests Signed-off-by: Joffrey F --- compose/config/config.py | 12 +++- compose/config/validation.py | 8 ++- compose/errors.py | 21 ++++++ compose/parallel.py | 7 +- compose/project.py | 12 +++- compose/service.py | 59 ++++++++++++++-- tests/acceptance/cli_test.py | 2 +- tests/integration/project_test.py | 114 ++++++++++++++++++++++++++++++ tests/unit/config/config_test.py | 9 ++- tests/unit/parallel_test.py | 2 +- 10 files changed, 228 insertions(+), 18 deletions(-) diff --git a/compose/config/config.py b/compose/config/config.py index dccd11e0d..73a340172 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -649,6 +649,8 @@ def process_service(service_config): if 'sysctls' in service_dict: service_dict['sysctls'] = build_string_dict(parse_sysctls(service_dict['sysctls'])) + service_dict = process_depends_on(service_dict) + for field in ['dns', 'dns_search', 'tmpfs']: if field in service_dict: service_dict[field] = to_list(service_dict[field]) @@ -658,6 +660,14 @@ def process_service(service_config): return service_dict +def process_depends_on(service_dict): + if 'depends_on' in service_dict and not isinstance(service_dict['depends_on'], dict): + service_dict['depends_on'] = dict([ + (svc, {'condition': 'service_started'}) for svc in service_dict['depends_on'] + ]) + return service_dict + + def process_healthcheck(service_dict, service_name): if 'healthcheck' not in service_dict: return service_dict @@ -665,7 +675,7 @@ def process_healthcheck(service_dict, service_name): hc = {} raw = service_dict['healthcheck'] - if raw.get('disable'): + if raw.get('disable') or raw.get('disabled'): if len(raw) > 1: raise ConfigurationError( 'Service "{}" defines an invalid healthcheck: ' diff --git a/compose/config/validation.py b/compose/config/validation.py index 7452e9849..3f23f0a7a 100644 --- a/compose/config/validation.py +++ b/compose/config/validation.py @@ -180,11 +180,13 @@ def validate_links(service_config, service_names): def validate_depends_on(service_config, service_names): - for dependency in service_config.config.get('depends_on', []): + deps = service_config.config.get('depends_on', {}) + for dependency in deps.keys(): if dependency not in service_names: raise ConfigurationError( "Service '{s.name}' depends on service '{dep}' which is " - "undefined.".format(s=service_config, dep=dependency)) + "undefined.".format(s=service_config, dep=dependency) + ) def get_unsupported_config_msg(path, error_key): @@ -201,7 +203,7 @@ def anglicize_json_type(json_type): def is_service_dict_schema(schema_id): - return schema_id in ('config_schema_v1.json', '#/properties/services') + return schema_id in ('config_schema_v1.json', '#/properties/services') def handle_error_for_schema_with_id(error, path): diff --git a/compose/errors.py b/compose/errors.py index 376cc5558..415b41e7f 100644 --- a/compose/errors.py +++ b/compose/errors.py @@ -10,3 +10,24 @@ class OperationFailedError(Exception): class StreamParseError(RuntimeError): def __init__(self, reason): self.msg = reason + + +class HealthCheckException(Exception): + def __init__(self, reason): + self.msg = reason + + +class HealthCheckFailed(HealthCheckException): + def __init__(self, container_id): + super(HealthCheckFailed, self).__init__( + 'Container "{}" is unhealthy.'.format(container_id) + ) + + +class NoHealthCheckConfigured(HealthCheckException): + def __init__(self, service_name): + super(NoHealthCheckConfigured, self).__init__( + 'Service "{}" is missing a healthcheck configuration'.format( + service_name + ) + ) diff --git a/compose/parallel.py b/compose/parallel.py index 267188728..b2654dcfd 100644 --- a/compose/parallel.py +++ b/compose/parallel.py @@ -165,13 +165,14 @@ def feed_queue(objects, func, get_deps, results, state): for obj in pending: deps = get_deps(obj) - if any(dep in state.failed for dep in deps): + if any(dep[0] in state.failed for dep in deps): log.debug('{} has upstream errors - not processing'.format(obj)) results.put((obj, None, UpstreamError())) state.failed.add(obj) elif all( - dep not in objects or dep in state.finished - for dep in deps + dep not in objects or ( + dep in state.finished and (not ready_check or ready_check(dep)) + ) for dep, ready_check in deps ): log.debug('Starting producer thread for {}'.format(obj)) t = Thread(target=producer, args=(obj, func, results)) diff --git a/compose/project.py b/compose/project.py index 0178bbab0..d99ef7c93 100644 --- a/compose/project.py +++ b/compose/project.py @@ -227,7 +227,10 @@ class Project(object): services = self.get_services(service_names) def get_deps(service): - return {self.get_service(dep) for dep in service.get_dependency_names()} + return { + (self.get_service(dep), config) + for dep, config in service.get_dependency_configs().items() + } parallel.parallel_execute( services, @@ -243,7 +246,7 @@ class Project(object): def get_deps(container): # actually returning inversed dependencies - return {other for other in containers + return {(other, None) for other in containers if container.service in self.get_service(other.service).get_dependency_names()} @@ -394,7 +397,10 @@ class Project(object): ) def get_deps(service): - return {self.get_service(dep) for dep in service.get_dependency_names()} + return { + (self.get_service(dep), config) + for dep, config in service.get_dependency_configs().items() + } results, errors = parallel.parallel_execute( services, diff --git a/compose/service.py b/compose/service.py index e3862b6ea..1338f1540 100644 --- a/compose/service.py +++ b/compose/service.py @@ -28,6 +28,8 @@ from .const import LABEL_PROJECT from .const import LABEL_SERVICE from .const import LABEL_VERSION from .container import Container +from .errors import HealthCheckFailed +from .errors import NoHealthCheckConfigured from .errors import OperationFailedError from .parallel import parallel_execute from .parallel import parallel_start @@ -69,6 +71,9 @@ DOCKER_START_KEYS = [ 'volumes_from', ] +CONDITION_STARTED = 'service_started' +CONDITION_HEALTHY = 'service_healthy' + class BuildError(Exception): def __init__(self, service, reason): @@ -533,10 +538,38 @@ class Service(object): def get_dependency_names(self): net_name = self.network_mode.service_name - return (self.get_linked_service_names() + - self.get_volumes_from_names() + - ([net_name] if net_name else []) + - self.options.get('depends_on', [])) + return ( + self.get_linked_service_names() + + self.get_volumes_from_names() + + ([net_name] if net_name else []) + + list(self.options.get('depends_on', {}).keys()) + ) + + def get_dependency_configs(self): + net_name = self.network_mode.service_name + configs = dict( + [(name, None) for name in self.get_linked_service_names()] + ) + configs.update(dict( + [(name, None) for name in self.get_volumes_from_names()] + )) + configs.update({net_name: None} if net_name else {}) + configs.update(self.options.get('depends_on', {})) + for svc, config in self.options.get('depends_on', {}).items(): + if config['condition'] == CONDITION_STARTED: + configs[svc] = lambda s: True + elif config['condition'] == CONDITION_HEALTHY: + configs[svc] = lambda s: s.is_healthy() + else: + # The config schema already prevents this, but it might be + # bypassed if Compose is called programmatically. + raise ValueError( + 'depends_on condition "{}" is invalid.'.format( + config['condition'] + ) + ) + + return configs def get_linked_service_names(self): return [service.name for (service, _) in self.links] @@ -871,6 +904,24 @@ class Service(object): else: log.error(six.text_type(e)) + def is_healthy(self): + """ Check that all containers for this service report healthy. + Returns false if at least one healthcheck is pending. + If an unhealthy container is detected, raise a HealthCheckFailed + exception. + """ + result = True + for ctnr in self.containers(): + ctnr.inspect() + status = ctnr.get('State.Health.Status') + if status is None: + raise NoHealthCheckConfigured(self.name) + elif status == 'starting': + result = False + elif status == 'unhealthy': + raise HealthCheckFailed(ctnr.short_id) + return result + def short_id_alias_exists(container, network): aliases = container.get( diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index b9766226d..77d578404 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -928,7 +928,7 @@ class CLITestCase(DockerClientTestCase): assert foo_container.get('HostConfig.NetworkMode') == \ 'container:{}'.format(bar_container.id) - @v3_only() + @v2_1_only() def test_up_with_healthcheck(self): def wait_on_health_status(container, status): def condition(): diff --git a/tests/integration/project_test.py b/tests/integration/project_test.py index de0732393..855974de1 100644 --- a/tests/integration/project_test.py +++ b/tests/integration/project_test.py @@ -19,6 +19,8 @@ from compose.config.types import VolumeSpec from compose.const import LABEL_PROJECT from compose.const import LABEL_SERVICE from compose.container import Container +from compose.errors import HealthCheckFailed +from compose.errors import NoHealthCheckConfigured from compose.project import Project from compose.project import ProjectError from compose.service import ConvergenceStrategy @@ -1375,3 +1377,115 @@ class ProjectTest(DockerClientTestCase): ctnr for ctnr in project._labeled_containers() if ctnr.labels.get(LABEL_SERVICE) == 'service1' ]) == 0 + + @v2_1_only() + def test_project_up_healthy_dependency(self): + config_dict = { + 'version': '2.1', + 'services': { + 'svc1': { + 'image': 'busybox:latest', + 'command': 'top', + 'healthcheck': { + 'test': 'exit 0', + 'retries': 1, + 'timeout': '10s', + 'interval': '0.1s' + }, + }, + 'svc2': { + 'image': 'busybox:latest', + 'command': 'top', + 'depends_on': { + 'svc1': {'condition': 'service_healthy'}, + } + } + } + } + config_data = build_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + project.up() + containers = project.containers() + assert len(containers) == 2 + + svc1 = project.get_service('svc1') + svc2 = project.get_service('svc2') + assert 'svc1' in svc2.get_dependency_names() + assert svc1.is_healthy() + + @v2_1_only() + def test_project_up_unhealthy_dependency(self): + config_dict = { + 'version': '2.1', + 'services': { + 'svc1': { + 'image': 'busybox:latest', + 'command': 'top', + 'healthcheck': { + 'test': 'exit 1', + 'retries': 1, + 'timeout': '10s', + 'interval': '0.1s' + }, + }, + 'svc2': { + 'image': 'busybox:latest', + 'command': 'top', + 'depends_on': { + 'svc1': {'condition': 'service_healthy'}, + } + } + } + } + config_data = build_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + with pytest.raises(HealthCheckFailed): + project.up() + containers = project.containers() + assert len(containers) == 1 + + svc1 = project.get_service('svc1') + svc2 = project.get_service('svc2') + assert 'svc1' in svc2.get_dependency_names() + with pytest.raises(HealthCheckFailed): + svc1.is_healthy() + + @v2_1_only() + def test_project_up_no_healthcheck_dependency(self): + config_dict = { + 'version': '2.1', + 'services': { + 'svc1': { + 'image': 'busybox:latest', + 'command': 'top', + 'healthcheck': { + 'disabled': True + }, + }, + 'svc2': { + 'image': 'busybox:latest', + 'command': 'top', + 'depends_on': { + 'svc1': {'condition': 'service_healthy'}, + } + } + } + } + config_data = build_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + with pytest.raises(NoHealthCheckConfigured): + project.up() + containers = project.containers() + assert len(containers) == 1 + + svc1 = project.get_service('svc1') + svc2 = project.get_service('svc2') + assert 'svc1' in svc2.get_dependency_names() + with pytest.raises(NoHealthCheckConfigured): + svc1.is_healthy() diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index f7df3aee4..7a68333ff 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -920,7 +920,10 @@ class ConfigTest(unittest.TestCase): 'build': {'context': os.path.abspath('/')}, 'image': 'example/web', 'volumes': [VolumeSpec.parse('/home/user/project:/code')], - 'depends_on': ['db', 'other'], + 'depends_on': { + 'db': {'condition': 'container_start'}, + 'other': {'condition': 'container_start'}, + }, }, { 'name': 'db', @@ -3055,7 +3058,9 @@ class ExtendsTest(unittest.TestCase): image: example """) services = load_from_filename(str(tmpdir.join('docker-compose.yml'))) - assert service_sort(services)[2]['depends_on'] == ['other'] + assert service_sort(services)[2]['depends_on'] == { + 'other': {'condition': 'container_start'} + } @pytest.mark.xfail(IS_WINDOWS_PLATFORM, reason='paths use slash') diff --git a/tests/unit/parallel_test.py b/tests/unit/parallel_test.py index 479c0f1d3..2a50b7189 100644 --- a/tests/unit/parallel_test.py +++ b/tests/unit/parallel_test.py @@ -25,7 +25,7 @@ deps = { def get_deps(obj): - return deps[obj] + return [(dep, None) for dep in deps[obj]] def test_parallel_execute(): From bef230853012d78e7ab58de65b653839401be974 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 19 Dec 2016 20:39:16 -0800 Subject: [PATCH 43/45] Fix condition name in config tests Signed-off-by: Joffrey F --- tests/acceptance/cli_test.py | 2 +- tests/unit/config/config_test.py | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index 77d578404..b9766226d 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -928,7 +928,7 @@ class CLITestCase(DockerClientTestCase): assert foo_container.get('HostConfig.NetworkMode') == \ 'container:{}'.format(bar_container.id) - @v2_1_only() + @v3_only() def test_up_with_healthcheck(self): def wait_on_health_status(container, status): def condition(): diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index 7a68333ff..31a888ed0 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -921,8 +921,8 @@ class ConfigTest(unittest.TestCase): 'image': 'example/web', 'volumes': [VolumeSpec.parse('/home/user/project:/code')], 'depends_on': { - 'db': {'condition': 'container_start'}, - 'other': {'condition': 'container_start'}, + 'db': {'condition': 'service_started'}, + 'other': {'condition': 'service_started'}, }, }, { @@ -3059,7 +3059,7 @@ class ExtendsTest(unittest.TestCase): """) services = load_from_filename(str(tmpdir.join('docker-compose.yml'))) assert service_sort(services)[2]['depends_on'] == { - 'other': {'condition': 'container_start'} + 'other': {'condition': 'service_started'} } From f90618fc43471355702c703909cc2d92813498f2 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 4 Jan 2017 13:14:23 -0800 Subject: [PATCH 44/45] Unify healthcheck spec definition in v2 and v3 Signed-off-by: Joffrey F --- compose/config/config.py | 2 +- compose/config/config_schema_v2.1.json | 2 +- compose/config/config_schema_v3.0.json | 12 ++++++------ tests/integration/project_test.py | 2 +- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/compose/config/config.py b/compose/config/config.py index 73a340172..fd935591f 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -675,7 +675,7 @@ def process_healthcheck(service_dict, service_name): hc = {} raw = service_dict['healthcheck'] - if raw.get('disable') or raw.get('disabled'): + if raw.get('disable'): if len(raw) > 1: raise ConfigurationError( 'Service "{}" defines an invalid healthcheck: ' diff --git a/compose/config/config_schema_v2.1.json b/compose/config/config_schema_v2.1.json index 5ab9f71fb..d0d5233a5 100644 --- a/compose/config/config_schema_v2.1.json +++ b/compose/config/config_schema_v2.1.json @@ -258,7 +258,7 @@ "type": "object", "additionalProperties": false, "properties": { - "disabled": {"type": "boolean"}, + "disable": {"type": "boolean"}, "interval": {"type": "string"}, "retries": {"type": "number"}, "test": { diff --git a/compose/config/config_schema_v3.0.json b/compose/config/config_schema_v3.0.json index 1f93347f9..8d075d47a 100644 --- a/compose/config/config_schema_v3.0.json +++ b/compose/config/config_schema_v3.0.json @@ -202,10 +202,11 @@ "healthcheck": { "id": "#/definitions/healthcheck", - "type": ["object", "null"], + "type": "object", + "additionalProperties": false, "properties": { - "interval": {"type":"string"}, - "timeout": {"type":"string"}, + "disable": {"type": "boolean"}, + "interval": {"type": "string"}, "retries": {"type": "number"}, "test": { "oneOf": [ @@ -213,9 +214,8 @@ {"type": "array", "items": {"type": "string"}} ] }, - "disable": {"type": "boolean"} - }, - "additionalProperties": false + "timeout": {"type": "string"} + } }, "deployment": { "id": "#/definitions/deployment", diff --git a/tests/integration/project_test.py b/tests/integration/project_test.py index 855974de1..c5e3cf50f 100644 --- a/tests/integration/project_test.py +++ b/tests/integration/project_test.py @@ -1463,7 +1463,7 @@ class ProjectTest(DockerClientTestCase): 'image': 'busybox:latest', 'command': 'top', 'healthcheck': { - 'disabled': True + 'disable': True }, }, 'svc2': { From ecff6f1a9a6f56eeaff378da814dafde9dd53b02 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 4 Jan 2017 13:47:13 -0800 Subject: [PATCH 45/45] Bump 1.10.0-rc1 Signed-off-by: Joffrey F --- CHANGELOG.md | 40 ++++++++++++++++++++++++++++++++++++++++ compose/__init__.py | 2 +- script/run/run.sh | 2 +- 3 files changed, 42 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9780df98a..3653a4343 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,46 @@ Change log ========== +1.10.0 (2017-01-18) +------------------- + +### New Features + +#### Compose file version 3.0 + +- Introduced version 3.0 of the `docker-compose.yml` specification. This + version requires to be used with Docker Engine 1.13 or above and is + specifically designed to work with the `docker stack` commands. + + - Added support for the `stop_grace_period` option in service definitions. + +#### Compose file version 2.1 and up + +- Healthcheck configuration can now be done in the service definition using + the `healthcheck` parameter + +- Containers dependencies can now be set up to wait on positive healthchecks + when declared using `depends_on`. See the documentation for the updated + syntax. + **Note:** This feature will not be ported to version 3 Compose files. + +- Added support for the `sysctls` parameter in service definitions + +- Added support for the `userns_mode` parameter in service definitions + +- Compose now adds identifying labels to networks and volumes it creates + +### Bugfixes + +- Colored output now works properly on Windows. + +- Fixed a bug where docker-compose run would fail to set up link aliases + in interactive mode on Windows. + +- Networks created by Compose are now always made attachable + (Compose files v2.1 and up). + + 1.9.0 (2016-11-16) ----------------- diff --git a/compose/__init__.py b/compose/__init__.py index 6f05b282f..f1bad0e1a 100644 --- a/compose/__init__.py +++ b/compose/__init__.py @@ -1,4 +1,4 @@ from __future__ import absolute_import from __future__ import unicode_literals -__version__ = '1.10.0dev' +__version__ = '1.10.0-rc1' diff --git a/script/run/run.sh b/script/run/run.sh index 5872b081a..5354b5f0d 100755 --- a/script/run/run.sh +++ b/script/run/run.sh @@ -15,7 +15,7 @@ set -e -VERSION="1.8.0" +VERSION="1.10.0-rc1" IMAGE="docker/compose:$VERSION"