Attempting to fix the state of codecov action checks right now,
which are behaving very erratically.
Using the new functionality in Go 1.20 to merge multiple reports,
so now the unit & E2E coverage data reports are stored as artifacts
and then downloaded, merged, and finally uploaded to codecov as a
new job.
Additionally, add a `codecov.yml` config and try to turn down the
aggressiveness of it for CI checks.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
This test keeps failing with a timeout in Windows. I don't actually
think it should take that long to bring up an nginx container, so
I'm guessing that there's something else going on that's causing
trouble.
Increase the verbosity when running Compose commands: I think this
will generally make E2E test failures easier to diagnose by always
logging the full command that's going to be run and also capturing
stdout.
Add a health check and use `--wait` when launching the fixture for
the pause test. Combined with the verbosity increase, this should
make it easier to understand what's going on here.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
currently the version displayed is the one installed and not the one use for the tests
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
See compose-spec/compose-go#325 for the acutal spec change. This
propagates it to the Engine API object and adds an E2E test via
Cucumber 🥒Fixes#9873.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
As of Go 1.16, the same functionality is now provided by package io or
package os, and those implementations should be preferred in new code.
So replacing all usage of ioutil pkg with io & os.
Signed-off-by: Abhinav Nair <11939846+abhinavnair@users.noreply.github.com>
This is mostly marking a bunch of the run methods as helpers so
that the internal assertions they do will show the line number of
the calling test instead.
There's also some small tweaks around the plugin initialization to
help with the output in the event that it fails to make it easier
to debug what went wrong.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
The E2E tests can be run in plugin (`docker compose`) or standalone
(`docker-compose`) mode. Existing logic was in place to ensure that
the helper method is always used, which will invoke the right one
based on how tests are being executed.
However, this logic was too easy to (unintentionally) bypass given
the myriad of ways that commands can be run. The check has been
made stricter and pushed to a lower-level to more aggressively
catch instances.
As a result, a bunch of calls to `RunDockerCmd` are now updated
to be `RunDockerComposeCmd`, which will ensure that the invocation
is correct based on test mode.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
The most important change here is to ensure that the correct Compose
standalone binary is used by `ddev`. Since it invokes Compose itself,
we need to ensure that `PATH` is set appropriately such that it finds
the binary we want to test rather than something from the system.
As part of this, the rest of the environment has been isolated, which
should make the test more reliable, and avoids polluting `~/.ddev`
with test artifacts by using a tmpdir as `HOME` for the test instead
of the user's real home folder.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
When running Docker / Compose commands, do NOT inherit the system
environment to ensure that the tests are reproducible regardless
of host settings.
Additionally, per-command environment overrides are provided to
the command instead of using `os.SetEnv`, as this is not safe when
running tests in parallel (`testing.T::SetEnv` will actually error
if used in this way!)
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
The big mechanical change here is to NOT store `t` as a field on
the `CLI` object (which has been renamed as well to fix the odd
capitalization). The way the tests are structured meant that the
"subtests" were using the _parent_ `*testing.T` instance, which
causes various oddities including confusing messages on failure
about a child test causing a panic in the parent.
Additionally, a few tests have been blocked from running in
parallel because they are sharing `compose.yaml` fixtures and
can fail as a result (e.g. due to a port conflict on the host).
I'll fix these in follow-up PRs but want to focus on correctness
for the tests before optimizing them.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
