Commit Graph

578 Commits

Author SHA1 Message Date
Daniel Nephin a92d86308f Rename ServiceLoader to ServiceExtendsResolver
ServiceLoader has evolved to be not really all that related to "loading" a
service. It's responsibility is more to do with handling the `extends`
field, which is only part of loading.  The class and its primary method
(make_service_dict()) were renamed to better reflect their responsibility.

As part of that change process_container_options() was removed from
make_service_dict() and renamed to process_service().  It contains logic for
handling the non-extends options.

This change allows us to remove the hacks from testcase.py and only call
the functions we need to format a service dict correctly for integration tests.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-11 10:58:07 -05:00
Daniel Nephin 98ad5a05e4 Validate additional files before merging them.
Consolidates all the top level config handling into `process_config_file` which
is now used for both files and merge sources.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-11 10:56:15 -05:00
Aanand Prasad 7466d14826 Merge pull request #2363 from dnephin/pr-2261
Rebase of PR 2261
2015-11-11 15:48:28 +00:00
Daniel Nephin c006c6ea09 Merge pull request #2341 from dnephin/dont_create_default_network_sometimes
Only create the default network if at least one service needs it
2015-11-11 10:48:08 -05:00
Daniel Nephin 7365a398b3 Update doc wording for ulimits.
and move tests to the correct module

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-10 15:03:20 -05:00
Kevin Greene 22d90d2180 Added ulimits functionality to docker compose
Signed-off-by: Kevin Greene <kevin@spantree.net>
2015-11-10 14:07:04 -05:00
Daniel Nephin a8ac6e6f93 Add a warning when the host volume config is being ignored.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-10 13:48:53 -05:00
Adrian Budau c5c36d8b00 Added --force-rm to compose build.
It's a flag passed to docker build that removes the intermediate
containers left behind on fail builds.

Signed-off-by: Adrian Budau <budau.adi@gmail.com>
2015-11-10 13:05:04 -05:00
Daniel Nephin 45724fc667 Only create the default network if at least one service needs it.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-10 12:43:55 -05:00
Daniel Nephin 3474bb6cf5 Cleanup workaround in testcase.py
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-10 12:43:55 -05:00
Aanand Prasad 0017f43d7f Merge pull request #2339 from dnephin/recreate_deps_on_create
Recreate dependents when a dependency is created
2015-11-10 17:36:23 +00:00
Daniel Nephin c8fa0a902a Merge pull request #2331 from dnephin/flush_log_buffer_when_printing_logs
Flush the log stream after writing each line
2015-11-10 12:34:42 -05:00
Aanand Prasad 917b885a44 Merge pull request #2334 from dnephin/dont_set_hostname
Don't set the hostname to the service name with networking.
2015-11-10 17:34:02 +00:00
Daniel Nephin b09cf86dd6 Merge pull request #2326 from dnephin/unicode_decode_error
Unicode decode error
2015-11-10 12:29:52 -05:00
Joffrey F 774d852f93 Merge pull request #2335 from dnephin/volumes_refactoring
Some small refactoring around volumes
2015-11-09 14:05:52 -08:00
Joffrey F a1e140f5a3 Update service config_dict computation to include volumes_from mode
Ensure config_hash is updated when volumes_from mode is changed, and
service is recreated on next up as a result.

Signed-off-by: Joffrey F <joffrey@docker.com>
2015-11-09 13:07:26 -08:00
Daniel Nephin 7c2a16234f Recreate dependents when a dependency is created (not just when it's recreated).
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-06 12:29:52 -05:00
Daniel Nephin ec22d98377 Use VolumeSpec instead of re-parsing the volume string.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-05 18:11:56 -05:00
Daniel Nephin 0e19c92e82 Make working_dir consistent in the config package.
- make it a positional arg, since it's required
- make it the first argument for all functions that require it
- remove an unnecessary one-line function that was only called in one place

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-05 18:11:56 -05:00
Daniel Nephin 3456002aef Don't set the hostname to the service name with networking.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-05 17:50:32 -05:00
Daniel Nephin d32bb8efee Fix #1549 - flush after each line of logs.
Includes some refactoring of log_printer_test to support checking for flush(), and so that each test calls the unit-under-test directly, instead of through a helper function.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-05 15:33:42 -05:00
Daniel Nephin 26c7dd3712 Handle non-utf8 unicode without raising an error.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-05 12:56:36 -05:00
Daniel Nephin 8ff960afd1 Fix service recreate when image changes to build.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-05 10:27:11 -05:00
Daniel Nephin ce322047a0 Move config hash tests to service_test.py
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-05 10:27:11 -05:00
Daniel Nephin 3d9e3d0877 Remove service.start_container()
It has been an unnecessary wrapper around container.start() for a little while now, so we can call it directly.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-03 10:17:09 -05:00
Daniel Nephin 45635f7097 Move cli tests to a new testing package.
These cli tests are now a different kind of that that run the compose binary. They are not the same as integration tests that test some internal interface.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-02 19:28:58 -05:00
Daniel Nephin 8057bb3fcc Update cli tests to use subprocess.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-02 17:35:00 -05:00
Daniel Nephin 1b5b407619 Fix networking tests to work with new API in engine rc4 (https://github.com/docker/docker/pull/17536)
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-11-02 13:40:56 -05:00
Daniel Nephin c439e056a0 Merge pull request #2288 from dnephin/warning_about_vars
Add console color to warnings and errors so they are more obvious
2015-10-30 10:58:00 -04:00
Joffrey F 072e7687ae Integration test for run command with networking enabled
Signed-off-by: Joffrey F <joffrey@docker.com>
2015-10-30 14:15:47 +01:00
Daniel Nephin 841ed4ed21 Remove the duplicate 'Warning' prefix now that the logger adds the prefix.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-29 14:18:28 -04:00
Daniel Nephin d836973a04 Use colors when logging warnings or errors, so they are more obvious.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-29 14:06:50 -04:00
Mazz Mosley 5dc14f3925 Handle non-ascii chars in volume directories
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-29 12:05:51 +00:00
Daniel Nephin a71d9af522 Disable a test against docker 1.8.3 because it fails due to a bug in docker engine.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-28 17:54:38 -04:00
Daniel Nephin b6b9b002e5 Merge pull request #2254 from mnowster/1756-dont-display-output-from-previous-runs
1756 dont display output from previous runs
2015-10-27 18:45:47 -04:00
Mazz Mosley bee063c07d Fix tests
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-27 17:12:18 +00:00
Mazz Mosley 7603ebea9b Attach to a container's log_stream before they're started
So we're not displaying output of all previous logs for a container, we attach,
if possible, to a container before the container is started.

LogPrinter checks if a container has a log_stream already attached and
print from that rather than always attempting to attach one itself.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-27 17:12:18 +00:00
Daniel Nephin 7878d38dee Fix running one-off containers with --x-networking by disabling linking to self.
docker create fails if networking and links are used together.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-27 11:11:15 -04:00
mnowster 5d60fbe31e Merge pull request #1659 from viranch/hostip-random-port
Allow scaling services that have port binding of the form `host_ip::container_port`
2015-10-23 17:11:28 +01:00
Daniel Nephin ae47435425 Fix unicode in environment variables for python2.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-22 12:12:43 -04:00
Daniel Nephin f7100b2ef3 Change version check from engine version to api version.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-21 15:40:50 -04:00
Daniel Nephin 0fed5e6864 Use inspect network to query for an existing network.
And more tests for get_network()

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-21 13:05:14 -04:00
mnowster fb69391889 Merge pull request #2233 from dnephin/some_unit_test_cleanup
Cleanup some unit tests and whitespace
2015-10-21 10:39:28 +01:00
Daniel Nephin b1f8ed84a3 Cleanup some unit tests and whitespace.
Remove some unnecessary newlines.
Remove a unittest that was attempting to test behaviour that was removed a while ago, so isn't testing anything.
Updated some unit tests to use mocks instead of a custom fake.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-20 14:23:56 -04:00
Daniel Nephin 3f0e083585 Force windows drives to be lowercase.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-20 11:39:06 -04:00
mnowster 5ed9f9bdf7 Merge pull request #2206 from dnephin/fix_extends_from_second_file
extends must be copied from override file
2015-10-20 10:22:58 +01:00
Daniel Nephin 3725c3ab7e Merge pull request #2225 from mnowster/2221-blank-env-vars
2221 allow empty environment keys
2015-10-19 14:06:31 -04:00
Daniel Nephin 938d49cbdc Fixes #2205 - extends must be copied from override file.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-19 13:41:58 -04:00
Daniel Nephin 937e087c6c Fixes #2203 - properly validate files when multiple files are used.
Remove the single-use decorators so the functionality can be used directly as a function.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-19 12:39:45 -04:00
Mazz Mosley 08add665e9 Environment keys can contain empty values
Environment keys that contain no value, get populated with values taken
from the environment not from the build phase but from running the command `up`.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-19 16:32:12 +01:00
Viranch Mehta 258c8bc54d Fix specifies_host_port() to handle port binding with host IP but no host port
Signed-off-by: Viranch Mehta <viranch.mehta@gmail.com>
2015-10-18 00:40:51 +05:30
Daniel Nephin 26dc0b785b Give the user a better error message (without a stack trace) when there is a yaml error.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-16 15:05:06 -04:00
Daniel Nephin 0409b287ad Merge pull request #2191 from dnephin/networking
Networking
2015-10-14 12:31:26 -04:00
mnowster 6b68a84b9b Merge pull request #2019 from mrfuxi/less-verbose-extends
Allow to extend service using shorthand notation. Closes #1989
2015-10-14 17:13:09 +01:00
Daniel Nephin e2f792c4f4 If -x-networking is used, set the correct API version.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-14 12:09:50 -04:00
Aanand Prasad d5f5eb1924 Enable use of Docker networking with the --x-networking flag
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
2015-10-14 11:19:43 -04:00
Daniel Nephin b9c76dfabd Merge pull request #2181 from mnowster/2128-split-drive-not-handling-relative-paths
2128 split drive not handling relative paths
2015-10-14 11:04:42 -04:00
Mazz Mosley ca36628a0e Test cgroup_parent option is being sent.
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-14 15:02:42 +01:00
Mohit Soni f4efa29377 Added support for cgroup_parent
This change adds cgroup-parent support to compose project. It allows
each service to specify a 'cgroup_parent' option.

Signed-off-by: Mohit Soni <mosoni@paypal.com>
2015-10-14 15:02:42 +01:00
Karol Duleba c7ffbf97c8 Extend oneOf error handling. Issue #1989
Signed-off-by: Karol Duleba <mr.fuxi@gmail.com>
2015-10-14 14:31:35 +01:00
Mazz Mosley c1d5ecaafe Workaround splitdrive limitations
splitdrive doesn't handle relative paths, so if volume_path contains
a relative path, we handle that differently and manually set drive to ''.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-13 17:27:25 +01:00
Mazz Mosley 8efc39e616 Improve boolean warning message.
Including examples of more boolean types, eg yes/N as it's not
always immediately clear that they are treated as booleans.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-07 14:59:08 +01:00
Aanand Prasad 627f1acdc4 Merge pull request #2126 from mnowster/1188-support-ro-options-in-volumes-from
1188 support ro options in volumes from
2015-10-07 11:57:14 +01:00
Aanand Prasad 60c9225d22 Merge pull request #2140 from dnephin/fix_patch_stdout
Reduce the scope of sys.stdout patching
2015-10-07 11:43:25 +01:00
Mazz Mosley f9028703f4 Pick the first container
Rather than inefficiently looping through all the containers that a
service has and overriding each volumes_from value, pick the first
one and return that.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-07 11:27:13 +01:00
Mazz Mosley 467c731869 address PR feedback
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-07 11:27:13 +01:00
Aanand Prasad fda615008c Merge pull request #2142 from dnephin/fix_error_cant_connect
Fix error when compose can't connect to docker
2015-10-07 11:22:32 +01:00
Daniel Nephin 15d0c60a73 Fix split buffer with inconsistently delimited json objects.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-06 10:43:07 -04:00
Daniel Nephin 3661e8bc74 Fix build against the swarm cluster by joining buffered output before parsing json.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-06 10:37:05 -04:00
Vincent Demeester 5b55a08846 Add support for ro option in volumes_from
Fixes #1188

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-10-06 13:12:55 +01:00
Daniel Nephin fbaea58fc1 Fix #2133 - fix call to get_client()
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-05 15:57:49 -04:00
Daniel Nephin e230142a25 Reduce the scope of sys.stdout patching.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-03 01:24:28 -04:00
Mazz Mosley bef5c2238e Skip a test for now
This needs resolving outside of this PR, as it is a much bigger piece
of work.

https://github.com/docker/compose/issues/2128

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-02 15:29:26 +01:00
Mazz Mosley af8032a5f4 Fix incorrect test name
I'd written relative, when I meant absolute. D'oh.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-02 12:01:56 +01:00
Mazz Mosley 2276326d7e volume path compatibility with engine
An expanded windows path of c:\shiny\thing:\shiny:rw
needs to be changed to be linux style path, including the drive,
like /c/shiny/thing /shiny
to be mounted successfully by the engine.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-02 12:01:56 +01:00
Mazz Mosley f4cd5b1d45 Handle windows volume paths
When a relative path is expanded and we're on a windows platform,
it expands to include the drive, eg C:\ , which was causing a ConfigError
as we split on ":" in parse_volume_spec and that was giving too many parts.

Use os.path.splitdrive instead of manually calculating the drive.

This should help us deal with windows drives as part of the volume
path better than us doing it manually.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-01 16:06:57 +01:00
Aanand Prasad e38334efbd Don't expand volume names
Only expand volume host paths if they begin with a dot.

This is a breaking change. The deprecation warning preparing users for
this change has been removed.

Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
2015-10-01 14:48:38 +01:00
Karol Duleba 91b227d133 Allow to extend service using shorthand notation. Closes #1989
Signed-off-by: Karol Duleba <mr.fuxi@gmail.com>
2015-09-24 18:01:47 +01:00
Daniel Nephin 78c0734cbd Disable some tests in windows for now.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-23 10:42:57 -04:00
Daniel Nephin c37a0c38a2 Fix a test case that assumes busybox image id.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-22 12:28:59 -04:00
mnowster b9252aa48e Merge pull request #2032 from villlem/master
Flag to skip all pull errors when pulling images
2015-09-21 13:41:08 +01:00
Aanand Prasad 18dbe1b1c0 Merge pull request #2051 from dnephin/extend_compose_files
Extend compose files by allowing multiple files
2015-09-21 12:34:22 +02:00
Vojta Orgon c9083e21c8 Flag to skip all pull errors when pulling images.
Signed-off-by: Vojta Orgon <villlem@gmail.com>
2015-09-21 11:59:23 +02:00
Aanand Prasad 3e58003bd3 Merge pull request #2041 from ggtools/master
Add new --pull option in build.
2015-09-18 16:49:05 +01:00
Daniel Nephin be0611bf3e Cleanup get_default_config_files tests.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:13:50 -04:00
Daniel Nephin 39ae85db8a Support a default docker-compose.override.yml for overrides
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:13:50 -04:00
Daniel Nephin 89be7f1fa7 Unit tests for multiple files
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:13:50 -04:00
Daniel Nephin 831276f531 Move config_test to the correct package name.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:13:50 -04:00
Daniel Nephin c0c9a7c1e4 Update integration tests for multiple file support
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:13:50 -04:00
Daniel Nephin 10b3188214 Support multiple config files
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:13:50 -04:00
Daniel Nephin 258d0fa0c6 Remove some functions from Command class
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:13:50 -04:00
Mazz Mosley 39ba2c5a7c Fix leaky tests
It was mocking self.client but relying on the call to
utils.create_host_config which was not mocked. So now that function
has moved to also be on self.client we need to redefine the test
boundary, up to where we would call docker-py, not the result of
docker-py.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-17 17:11:00 +01:00
Mazz Mosley 6f6c04b5c9 Test what we are sending, not what we get
This is a unit test and we are mocking the client. The method to get
the create_config_host now lives on the client, so we mock that too.

So we can test to the boundary that the method is called with the
arguments we expect.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-17 17:11:00 +01:00
Mazz Mosley 42c890796b Use docker.client.create_host_config
create_host_config from docker.utils will be deprecated so that
the new create_host_config has access to the _version so
we can ensure that network_mode only gets set to 'default' by
default if the version is high enough and won't explode.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-17 17:11:00 +01:00
Christophe Labouisse 39786d4da7 Add new --pull option in build.
Signed-off-by: Christophe Labouisse <christophe@labouisse.org>
2015-09-15 20:19:17 +02:00
Aanand Prasad 1dabf1c366 Merge pull request #1878 from mnowster/change-default-log-driver-value
Change default `log_driver` value
2015-09-15 17:23:00 +01:00
Mazz Mosley 2f45649611 Handle invalid log_driver
Now docker-py isn't hardcoding a list of valid log_drivers, we
can expect an APIError in response rather than a ValueError
if we send an invalid log_driver.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-15 15:47:39 +01:00
Mazz Mosley 4b2fd7699b Relax constraints on key naming for environment
One of the use cases is swarm requires at least : character, so going
from conservative to relaxed.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-15 15:33:30 +01:00
Mazz Mosley 8caeffe27e Log a warning when boolean is found in `environment`
We're going to warn people that allowing a boolean in the environment is
being deprecated, so in a future release we can disallow it. This is to
ensure boolean variables are quoted in strings to ensure they don't get
mis-parsed by YML.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-15 15:33:30 +01:00
Mazz Mosley a594a2ccc2 Disallow booleans in environment
When users were putting true/false/yes/no in the environment key,
the YML parser was converting them into True/False, rather than leaving
them as a string.

This change will force people to put them in quotes, thus ensuring
that the value gets passed through as intended.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-15 15:33:30 +01:00