Commit Graph

397 Commits

Author SHA1 Message Date
Nicolas De Loof 74de423cc3
reuse ECS logConsumer to implement formatted compose log output
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-17 11:28:08 +01:00
Nicolas De Loof 976eacd198
pass json encoded compose file to docker-compose to prevent yaml format mismatch
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-17 11:02:31 +01:00
Nicolas De loof a05df9490b
Merge pull request #913 from docker/gpu_spec
Add support for gpu devices reservation
2020-11-16 17:36:04 +01:00
Nicolas De Loof cb808dec31
Don't mess with exiting .aws/credentials
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-16 14:51:05 +01:00
Nicolas De Loof efd0f2b9ec
Add support for gpu devices reservation
Adopt https://github.com/compose-spec/compose-spec/pull/100

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-16 14:24:25 +01:00
Nicolas De Loof 8438eee612
Use reservations if user has not set limits
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-12 10:57:34 +01:00
Guillaume Tardif 949f0ce62e
Merge pull request #897 from docker/volume_conflict
Don't panic on bind mount
2020-11-10 17:09:53 +01:00
Nicolas De Loof 8e847bfd9a
Include volume name in PolicyName to disambiguate
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-10 15:47:16 +01:00
Nicolas De Loof 06fd442a53
Don't panic on bind mount
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-10 15:47:04 +01:00
Nicolas De Loof 71ecbda48f
Publish on s3 if payload is > API limit
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-10 11:03:00 +01:00
Nicolas De Loof 10a384d35b
Pass region to create s3 bucket into
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-10 08:40:02 +01:00
Nicolas De Loof 3f184f7552
store cloudformation template on s3 to workaround API limit
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-10 08:40:02 +01:00
Guillaume Tardif 5c04e84e49
Merge pull request #882 from docker/yaml
Marshall cloudformation template as yaml
2020-11-06 13:51:12 +01:00
Nicolas De Loof e99a8b54f8
Detect changeset creation failure when there's no changet
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-05 13:39:52 +01:00
Nicolas De Loof cb02622318
Marshall cloudformation template as yaml
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-05 11:01:32 +01:00
Guillaume Tardif 8cc438c9a5 Pin ECS secrets sidecar image & add makefile targets
Signed-off-by: Guillaume Tardif <guillaume.tardif@docker.com>
2020-11-04 10:30:46 +01:00
Nicolas De loof 084a04fdd9
Merge pull request #866 from docker/token
DescribeSubnets uses pagination
2020-11-03 11:46:21 +01:00
Nicolas De loof f66123b34a
Merge pull request #846 from docker/ecs_context 2020-11-03 11:38:06 +01:00
Nicolas De Loof af7aebf8cf
Make newcomer experience smooth
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-03 10:49:28 +01:00
Nicolas De Loof a0addb7dcb
DescribeSubnets uses pagination
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-11-03 10:45:40 +01:00
Nicolas De loof 7018477407
Merge pull request #859 from flaviostutz/env-sort
Sorting env pair array to avoid creating different CloudFormation templates between calls
2020-11-03 06:34:32 +01:00
Guillaume Tardif 3b6bfdd5b1 Display ecs status in CamelCase in order to have homogeneous output between backends.
Signed-off-by: Guillaume Tardif <guillaume.tardif@docker.com>
2020-11-02 13:08:54 +01:00
flaviostutz 49f2283646 sorting env pair array to avoid creating different cloudformation templates between calls
Signed-off-by: flaviostutz <flaviostutz@gmail.com>
2020-10-31 18:06:18 -03:00
Nicolas De loof dc8a432352
Merge pull request #857 from docker/ecs_pin_search_sidecar_image
Pinned and publish docker/ecs-searchdomain-sidecar:1.0
2020-10-31 10:40:32 +01:00
aiordache 3066a1cdad cleanup
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-30 18:50:56 +01:00
Guillaume Tardif 3cfb252ff0 Pinned and publish docker/ecs-searchdomain-sidecar:1.0
Signed-off-by: Guillaume Tardif <guillaume.tardif@docker.com>
2020-10-30 18:35:28 +01:00
aiordache de0be8650e Remove region from contexts and cleanup
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-30 17:51:16 +01:00
flaviostutz 25c5a8cd72 fixing empty vpcId
Signed-off-by: flaviostutz <flaviostutz@gmail.com>
2020-10-30 10:49:04 -03:00
aiordache 879afa85c0 fix lint issues
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-29 11:33:56 +01:00
aiordache 2fdc93786e Fix region from environment
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-29 10:51:57 +01:00
aiordache 53efa312c8 Refactor context create options
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-29 10:25:23 +01:00
aiordache e44d0b922a cleanup
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-29 10:10:00 +01:00
aiordache 902b660de1 revisit context creation
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-29 10:10:00 +01:00
aiordache 2dcd1a5826 refactor ecs context creation
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-29 10:10:00 +01:00
Gergely Imreh d6461df474 cloudformation: fix typo in generated ingress description
Signed-off-by: Gergely Imreh <imrehg@gmail.com>
2020-10-28 22:44:20 +00:00
Guillaume Tardif c1504d484f Docs: adding docs README, moving ecs docs in docs folder, removing obsolete cli docs
Signed-off-by: Guillaume Tardif <guillaume.tardif@docker.com>
2020-10-28 10:44:42 +01:00
Guillaume Tardif 0d197401a2 Fix Dockerfile syntax in ecs sidecars introduced by https://github.com/docker/compose-cli/pull/799
Signed-off-by: Guillaume Tardif <guillaume.tardif@docker.com>
2020-10-26 16:42:51 +01:00
Guillaume Tardif 382827241d
Merge pull request #799 from docker/aci_etchosts
minimalist container image to setup /etc/hosts on ACI
2020-10-26 15:58:19 +01:00
aiordache 78931b3efa Enable cross zone load balancing
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-22 15:19:00 +02:00
Nicolas De loof 34a7466571
Merge pull request #817 from docker/scalescale
Configure autoscaling parameters
2020-10-21 10:56:28 +02:00
Nicolas De Loof 553865f294
minimalist container image to setup /etc/hosts on ACI
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-21 10:25:05 +02:00
Nicolas De loof ed69f38b44
Merge pull request #813 from docker/ecsVolumeService
CLI command to manage ECS volumes
2020-10-20 16:58:13 +02:00
Nicolas De loof d5ed2bdecf
Merge pull request #818 from docker/ecs_event_status
Remove status truncate on wait
2020-10-20 16:57:53 +02:00
Nicolas De Loof de96a0c1d0
CLI command to manage ECS volumes
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-20 16:52:50 +02:00
Nicolas De loof 75a5a0f205
Merge pull request #812 from docker/barbarian
Introduce use of EFS Access Point to mount filesystems as volumes
2020-10-20 16:23:50 +02:00
aiordache 4add3a1662 Remove status truncate from wait and calculate max length in the progress writer based on the terminal width
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-20 16:18:08 +02:00
Nicolas De loof 1c211550fc
Merge pull request #696 from docker/ec2_machine
Allow user to set ami/machine by Deploy.Placement.Constraint
2020-10-20 11:19:29 +02:00
Nicolas De Loof a3221341da
Configure autoscaling parameters
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-20 11:05:44 +02:00
Nicolas De Loof 6fa30284d9
Introduce use of EFS Access Point to mount filesystems as volumes
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-20 10:52:33 +02:00
Nicolas De loof dfd660d401
Merge pull request #735 from docker/ecs_secret_stdin
Add password file argument to secret create (handles '-' as stdin read)
2020-10-20 10:31:54 +02:00
aiordache 8f31ad59be Refactor secrets
- create secrets from files
  - update Secret structure

Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-19 11:33:55 +02:00
flaviostutz a85f0042d6 fix "autoscaling" typo
Signed-off-by: flaviostutz <flaviostutz@gmail.com>
2020-10-18 21:14:23 -03:00
Nicolas De loof 31a0cf08d7
Merge pull request #779 from docker/ulimits
add support for ulimits.nofile
2020-10-16 16:12:02 +02:00
Nicolas De Loof 10b8fabaae
Allow user to set ami/machine by Deploy.Placement.Constraint
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-16 14:19:51 +02:00
Nicolas De loof d55f8e4ee1
Merge pull request #795 from docker/kirikou
Store external cluster as metadata
2020-10-16 13:59:53 +02:00
Nicolas De Loof 4760ae334d
Store external cluster as metadata
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-16 13:42:20 +02:00
Anca Iordache 05295b5e8a
Merge pull request #765 from docker/ecs_default_profile
Populate ~/.aws/config(credentials) on ecs context create
2020-10-16 11:33:03 +02:00
aiordache c2af0a136a Populate ~/.aws/config(credentials) on ecs context create
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-15 20:26:38 +02:00
Nicolas De loof 2f64376f84
Merge pull request #786 from docker/feat_prune 2020-10-14 18:19:21 +02:00
Guillaume Tardif 7cf7b00584 Add ResourceService definition and ACI NOOP implementation
Signed-off-by: Guillaume Tardif <guillaume.tardif@docker.com>
2020-10-14 15:40:24 +02:00
Nicolas De Loof 47344608b8
add support for ulimits.nofile
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-14 12:08:48 +02:00
Nicolas De Loof 8379bf467e
Let user pass EFS create option by driver_opts
fixes #781

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-14 11:52:03 +02:00
Nicolas De Loof a87300d1df
set filesystem name to make it more visible on AWS web console
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-13 15:08:40 +02:00
Nicolas De Loof 5d5765173e
Create volume for compose app
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-13 14:18:01 +02:00
Nicolas De Loof e903326e1a
Check external filesystems exist on compose up
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-13 10:43:01 +02:00
Nicolas De loof 45bb05ee19
Merge pull request #764 from docker/test_with_mock
More unit tests with aws-sdk behind an interface + mocks
2020-10-12 21:21:38 +02:00
Nicolas De Loof 432ce79e2c
More unig tests with aws-sdk behind an interface + mocks
Fix use of existing SecurityGroup

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-12 16:51:05 +02:00
Ulysses Souza 10632b008b Add version to UserAgent on ACI
This also refactors the Version insertion to make it global

Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
2020-10-12 09:15:32 +02:00
aiordache 265dd197c3 Add detach flag to `compose up`
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-07 15:51:43 +02:00
Nicolas De Loof e8507893ee
restore status in addition to reason as progress message
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-06 14:05:08 +02:00
Nicolas De Loof 481e069b2e
NLB don't support securityGroups
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-06 10:55:45 +02:00
aiordache c15d05f7f4 refactor task failure detection
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-05 19:37:13 +02:00
aiordache eadcb3be32 Error check stack delete
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-05 16:55:38 +02:00
aiordache d3effd2ead Detect task failures
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-05 16:55:35 +02:00
Nicolas De loof cc81a2546c
Merge pull request #719 from aiordache/ecs_norm_vol_names
Normalize volume name
2020-10-05 11:14:49 +02:00
Nicolas De loof 408ab6d43a
Merge pull request #688 from docker/resolv
Configure /etc/resolv.conf before container start by an init container
2020-10-05 10:56:16 +02:00
Nicolas De loof 10824fe7c1
Merge pull request #718 from docker/capacity_provider_shutdown
delete CapacityProvider before we invoke DeleteStack
2020-10-05 09:26:17 +02:00
Nicolas De Loof eab140ee71
delete CapacityProvider before we invoke DeleteStack
this is a workaround for CloudFormation issue to manage CapacityProvider <-> Cluster reverse dependency

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-10-02 18:22:13 +02:00
aiordache 21f5cf8b40 normalize volume name
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-02 15:50:25 +02:00
Nicolas De loof 7b9390d91f
Merge pull request #701 from docker/autoscale 2020-10-02 15:01:51 +02:00
aiordache cf906951bf Normalize volume names
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-10-02 14:54:23 +02:00
Nicolas De Loof 20a8f01269
Don't set securityGroup name, as all compose apps will create a `DefaultNetwork`
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-30 15:26:12 +02:00
Nicolas De Loof 757b9bb221
Introduce x-aws-autoscale to support CPU based autoscaling
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-30 14:59:25 +02:00
Nicolas De Loof 1444d68685
Introduce a sidecard to setup resolv.conf search domain
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-29 12:23:27 +02:00
Nicolas De Loof b56e17cf27
Sidecar image to append resolvconf search directive
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-29 12:08:42 +02:00
Nicolas De Loof d5e0ec7aa6
`parse` to return awsResources then convert into CF template
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-29 08:31:11 +02:00
Nicolas De Loof 7034254911
introduce awsResources to replace CloudFormation parameters
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-24 19:34:41 +02:00
Nicolas De Loof b30cd649b9
Dispatch methods into more specific files
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-24 10:32:46 +02:00
Nicolas De Loof ed5d774180
createPolicies to rely on project/service not converted aws model
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-24 08:27:32 +02:00
Nicolas De loof 767ed0c20d
Merge pull request #633 from docker/ec2 2020-09-24 08:26:25 +02:00
Nicolas De Loof 6a226ace9d
disable LoadBalancer healthCheck
compose-spec only support command-based health check.
we need to investigate introducing URI-based health check support

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-23 16:05:29 +02:00
Nicolas De Loof 6f0ba3bac5
architecture document
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-23 12:19:34 +02:00
Nicolas De Loof d57a39958b
Compute machine type matching reservations
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-23 12:18:42 +02:00
Nicolas De Loof f2430afa06
Fix CPU limit computation targetting EC2
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-23 10:56:32 +02:00
Nicolas De Loof 2cfaf69546
Distinguish limits for Fargate and EC2
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-23 09:40:07 +02:00
Nicolas De Loof 109ba96743
Run on EC2 when a service require GPU
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-09-23 09:31:16 +02:00
Chris Crone d154c41586 Update copyright
Signed-off-by: Chris Crone <christopher.crone@docker.com>
2020-09-22 12:13:00 +02:00
Guillaume Tardif 01ea2488a2
Merge pull request #630 from docker/cli_metrics_failures
Cli metrics failures
2020-09-18 18:26:54 +02:00
Nicolas De loof 57c14e70e1
Merge pull request #628 from docker/machine
Guess AWS machine type based on service resources reservations
2020-09-18 17:05:54 +02:00
Nicolas De loof b22ebd61c4
minor error improvement to match AWS docs
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

Co-authored-by: Chris Crone <christopher.crone@docker.com>
2020-09-18 16:44:17 +02:00