Use latest Go minor release. Note: this release included fixes for
several CVEs, but they do not impact Compose.
Small errors have been fixed to keep the linter happy.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
This was running two tests in parallel that would build/delete the
same images. Run in serial instead since that's not safe.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Was getting segfaults with multiple services using
`x-develop` and `watch` at the same time. Turns out
the Moby path matcher lazily initializes the regex
pattern internally the first time it's used, so it's
not goroutine-safe.
Change here is to not use a global instance for the
ephemeral path matcher, but a per-watcher instance.
Additionally, the data race detector caught a couple
other issues that were easy enough to fix:
* Use the lock that's used elsewhere for convergence
before manipulating
* Eliminate concurrent map access when triggering
rebuilds
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
This approach mimics Tilt's behavior[^1]:
1. At sync time, `stat` the path on host
2. If the path does not exist -> `rm` from container
3. If the path exists -> sync to container
By handling things this way, we're always syncing based on the true
state, regardless of what's happened in the interim. For example, a
common pattern in POSIX tools is to create a file and then rename it
over an existing file. Based on timing, this could be a sync, delete,
sync (every file gets seen & processed) OR a delete, sync (by the
the time we process the event, the "temp" file is already gone, so
we just delete it from the container, where it never existed, but
that's fine since we deletes are idempotent thanks to the `-f` flag
on `rm`).
Additionally, when syncing, if the `stat` call shows it's for a
directory, we ignore it. Otherwise, duplicate, nested copies of the
entire path could get synced in. (On some OSes, an event for the
directory gets dispatched when a file inside of it is modified. In
practice, I think we might want this pushed further down in the
watching code, but since we're already `stat`ing the paths here now,
it's a good place to handle it.)
Lastly, there's some very light changes to the text when it does a
full rebuild that will list out the (merged) set of paths that
triggered it. We can continue to improve the output, but this is
really helpful for understanding why it's rebuilding.
[^1]: db7f887b06/internal/controllers/core/liveupdate/reconciler.go (L911)
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
This test keeps failing with a timeout in Windows. I don't actually
think it should take that long to bring up an nginx container, so
I'm guessing that there's something else going on that's causing
trouble.
Increase the verbosity when running Compose commands: I think this
will generally make E2E test failures easier to diagnose by always
logging the full command that's going to be run and also capturing
stdout.
Add a health check and use `--wait` when launching the fixture for
the pause test. Combined with the verbosity increase, this should
make it easier to understand what's going on here.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Big change here is to import the ephemeral ignore set from Tilt.
The `.git` directory is also ignored for now: this restriction
should probably be lifted and made configurable in the future,
but it's not generally important to watch and triggers a LOT of
events (e.g. Git creates `index.lock` files that will appear and
disappear rapidly as terminals/IDEs/etc interact with Git, even
for read-only operations).
The Tilt-provided ephemeral file set has been slowly devised over
time based on temporary files that can cause trouble. We can also
look at a more robust/configurable solution here in the future,
but thse provide a reasonable out-of-the-box configuration for
the moment.
There's also some small tweaks to the output to add missing
newlines in a few edge cases and such.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Currently the emacs ignore patterns include `**/.#*` (lock files), but
doesn't include `**/#*#` (autosave files;
https://www.emacswiki.org/emacs/AutoSave, not to be confused with
`**/*~` backup files which are ignored.)
Add autosave files.
When creating files in Go, the stdlib will create (and then rapidly
delete) files ending with `-go-tmp-umask` to determine the umask
to use for permission purposes.
This can cause trouble with Live Update because the files tend to
vanish underneath it, for example.
Fixes#5117.
`exportloopref` - detects captures of loop variable without
re-assignment
NOTE: There can be false negatives with this linter to avoid being
overly strict and annoying!
Also upgraded `golangci-lint` to latest (v1.43.0 published 2021-11-03).
I've been running the test suite (`./internal/engine` in particular)
with `-count X` a lot recently to catch timing-related test failures.
After running enough times, the tests start failing due to too many
open files, so I did an audit and found a few places where files or
readers weren't always being closed.
* lint: add make lintfix and run it
Fixes all errors like:
```
File is not `goimports`-ed with -local github.com/tilt-dev/tilt (goimports)
```
* git: change to use TrimSuffix
* build: remove unnecessary cast
`WalkDir` is new in Go 1.16 and avoids calling `os.Lstat` on
every visited file and directory. In most cases, we don't need
that info, so this will help reduce I/O when listing files,
which can be helpful for particularly big monorepos.
Unused code linter isn't particularly smart about platform build
tags, so since this func is only used by the "naive" (non-macOS)
file watcher, it needs to live with that or it gets flagged as
dead code when linting on macOS.
In most places in Tilt, we try to use absolute paths everywhere.
So this makes things more consistent with the rest of Tilt, and lets us be
a bit more flexible in how we handle subdirs and parent dirs in ignores.
Fixes https://github.com/tilt-dev/tilt/issues/3740
Here's our new watch strategy on Darwin in a nutshell:
1. Create an fsevents stream for events "since" the last event ID that
we saw globally.
2. Add a path that we want to watch
3. Add that path to a map of paths that we're watching _directly_.
4. Restart the event stream to pick up the new path.
5. Ignore all events for all watches until we've seen a `HistoryDone`
event.
6. Ignore the first `ItemCreated` event for paths we're watching that
are also directories
7. Otherwise, forward along all events.
Without this I get an exception when building multiple images in a
compose run.
```
fatal error: concurrent map writes
goroutine 16 [running]:
github.com/docker/compose/v2/pkg/compose.(*composeService).build.func1({0x2cba4e0, 0xc00019b2c0}, {0xc000233150?, 0xd?})
github.com/docker/compose/v2/pkg/compose/build.go:95 +0x652
github.com/docker/compose/v2/pkg/compose.(*graphTraversal).run.func1()
github.com/docker/compose/v2/pkg/compose/dependencies.go:127 +0x63
golang.org/x/sync/errgroup.(*Group).Go.func1()
golang.org/x/sync@v0.1.0/errgroup/errgroup.go:75 +0x64
created by golang.org/x/sync/errgroup.(*Group).Go
golang.org/x/sync@v0.1.0/errgroup/errgroup.go:72 +0xa5
...
```
I'm not sure why the map is currently shared, but with this patch
applied my docker-compose build run works.
Signed-off-by: Edward Muller <emuller@fastly.com>
Guillaume Lours
Nicolas De Loof
Milas Bowman
Luis Rascao
Laura Brehm
Josh Wilson
Milas Bowman
Iggy Jackson
Nick Santos
Matt Landis
Maia McCormick
Dan Miller
Nick Santos
Ken Sipe
Nick Sieger
Dan Miller
Dan Bentley
Maia McCormick
Dan Miller
Ben Moss
maxcleme
Laura Brehm
Tom Collingwood
Edward Muller