Commit Graph

466 Commits

Author SHA1 Message Date
Kir Kolyshkin 053e15c001 tests/checkpoint: show full log lazy pages cpt
Sometimes the test fails without any clear diagnostics:

> not ok 16 checkpoint --lazy-pages and restore
> (in test file tests/integration/checkpoint.bats, line 191)
>  `[ "$out" = "0000000 000000 0000001" ]' failed
> ...
> criu failed: type NOTIFY errno 3\nlog file: work-dir/dump.log

We look for and print errors via grep, but in the above case
there are nothing that is denoted error in the log.

So, let's show the damn log in its entirely (note it is only shown
if test fails).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-03-08 15:04:40 -08:00
Kir Kolyshkin 59ad417c14 Merge pull request #2833 from kolyshkin/curl-retry
curl: add --retry 5
2021-03-05 20:00:53 -08:00
Daniel Dao 8c7ece1e6d fs2: fallback to setting io.weight if io.bfq.weight
if bfq is not loaded, then io.bfq.weight is not available. io.weight
should always be available and is the next best equivalent thing.

Signed-off-by: Daniel Dao <dqminh89@gmail.com>
2021-03-05 13:55:36 +00:00
Mrunal Patel 3742bfe891 Merge pull request #2824 from Iceber/fix-integration-tests-readme
fix integration tests README.md
2021-03-04 15:40:24 -08:00
Kir Kolyshkin e618a6d5ee curl: add --retry 5
Sometimes github gives up 5xx errors, for example:

> panic: getImages error exit status 1 (output: curl: (22) The requested URL returned error: 502
> Failed to get https://github.com/docker-library/busybox/raw/dist-i386/stable/glibc/busybox.tar.xz

This is unfortunate but temporary, and adding --retry should handle
at least some cases, improving CI stability.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-03-04 15:37:59 -08:00
Kir Kolyshkin db025aba75 libct: criuSwrk: only iterate over CriuOpts if debug is set
In case log level is less than debug, this code does nothing,
so let's add a condition and skip it entirely.

Add a test case to make sure this code path is hit.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-03-04 08:58:42 -08:00
Akihiro Suda d56a9c67ac Merge pull request #2812 from kolyshkin/memory-tight 2021-02-26 10:52:17 +09:00
Iceber Gu 71ca643213 fix integration tests README.md
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2021-02-26 02:23:44 +08:00
Aleksa Sarai c153261830 merge branch 'pr-2786'
Daniel Dao (1):
  Do not convert blkio weight value using blkio->io conversion scheme

LGTMs: @kolyshkin @AkihiroSuda @cyphar
Closes #2786
2021-02-24 18:29:27 +11:00
Kir Kolyshkin 65116710dd Merge pull request #2757 from kolyshkin/test-int-cleanups
tests/int: cleanups
2021-02-23 20:50:17 -08:00
Daniel Dao c3ffd2ef81 Do not convert blkio weight value using blkio->io conversion scheme
bfq weight controller (i.e. io.bfq.weight if present) is still using the
same bfq weight scheme (i.e 1->1000, see [1].) Unfortunately the
documentation for this was wrong, and only fixed recently [2].

Therefore, if we map blkio weight to io.bfq.weight, there's no need to
do any conversion. Otherwise, we will try to write invalid value which
results in error such as:

```
time="2021-02-03T14:55:30Z" level=error msg="container_linux.go:367: starting container process caused: process_linux.go:495: container init caused: process_linux.go:458: setting cgroup config for procHooks process caused: failed to write \"7475\": write /sys/fs/cgroup/runc-cgroups-integration-test/test-cgroup/io.bfq.weight: numerical result out of range"
```

[1] https://github.com/torvalds/linux/blob/master/Documentation/block/bfq-iosched.rst
[2] https://github.com/torvalds/linux/commit/65752aef0a407e1ef17ec78a7fc31ba4e0b360f9

Signed-off-by: Daniel Dao <dqminh89@gmail.com>
2021-02-23 19:46:16 -08:00
Kir Kolyshkin 5d0ffbf9c8 runc start/run: report OOM
In some cases, container init fails to start because it is killed by
the kernel OOM killer. The errors returned by runc in such cases are
semi-random and rather cryptic. Below are a few examples.

On cgroup v1 + systemd cgroup driver:

> process_linux.go:348: copying bootstrap data to pipe caused: write init-p: broken pipe

> process_linux.go:352: getting the final child's pid from pipe caused: EOF

On cgroup v2:

> process_linux.go:495: container init caused: read init-p: connection reset by peer

> process_linux.go:484: writing syncT 'resume' caused: write init-p: broken pipe

This commits adds the OOM method to cgroup managers, which tells whether
the container was OOM-killed. In case that has happened, the original error
is discarded (unless --debug is set), and the new OOM error is reported
instead:

> ERRO[0000] container_linux.go:367: starting container process caused: container init was OOM-killed (memory limit too low?)

Also, fix the rootless test cases that are failing because they expect
an error in the first line, and we have an additional warning now:

> unable to get oom kill count" error="no directory specified for memory.oom_control

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-23 16:15:33 -08:00
Kir Kolyshkin 65c2d3c2b6 tests/int/update: add test case for PR #592
Add integration test coverage for code initially added by commit
d8b8f76c4f ("Fix problem when update memory and swap memory",
2016-04-05).

This is in addition to existing unit test,
TestMemorySetSwapSmallerThanMemory.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-23 16:11:55 -08:00
Adrian Reber 051646a3b0 tests: test nested bind mount restore
This adds a checkpoint/restore test to ensure that nested bind mount
mountpoints are correctly re-created.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-02-10 08:55:57 +01:00
Kir Kolyshkin 41670e21f0 tests/int: rework/simplify setup and teardown
1. Get rid of fixed ROOT, *_BUNDLE, and CONSOLE_SOCKET dirs.
   Now they are temporary directories created in setup_bundle.

2. Automate containers cleanup: instead of having to specify all
   containers to be removed, list and destroy everything (which is
   now possible since every test case has its own unique root).

3. Randomize cgroup paths so two tests running in parallel won't
   use the same cgroup.

Now it's theoretically possible to execute tests in parallel.
Practically it's not possible yet because bats uses GNU parallel,
which do not provide a terminal for whatever it executes, and
many runc tests (all those that run containers with terminal:
true) needs a tty. This may possibly be addressed later.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-09 20:05:54 -08:00
Kir Kolyshkin c34a9b1025 tests/int/hooks.bats: don't use DEBIAN_BUNDLE
1. Do not depend on $DEBIAN_BUNDLE, instead use the current directory.

2. Simplify setup() by calling teardown().

3. In teardown(), check that LIBPATH is set.

4. Move global variables into setup.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-08 12:49:04 -08:00
Kir Kolyshkin e40a369ea8 tests/int/list.bats: don't use $BUSYBOX_BUNDLE
Use the value of $(pwd) as we cd to bundle dir before running a test.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-08 12:47:27 -08:00
Kir Kolyshkin 985546b445 tests/int: BATS_TMPDIR -> BATS_RUN_TMPDIR
It's a little known fact, but BATS_TMPDIR is just /tmp, and it is not
being cleaned by bats after the test.

BATS_RUN_TMPDIR (available since bats 1.2.1) is $BATS_TMPDIR/bats-run-$$
(i.e. it is per bats run and includes PID), and it is cleaned up after
the test (unless --no-tmpdir-cleanup is given).

Use BATS_RUN_TMPDIR for better isolation and cleanup.

[v2: check for BATS_RUN_TMPDIR, effectively requiring bats >= 1.2.1]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-08 12:46:58 -08:00
Kir Kolyshkin 85d5fea468 tests/int: stop reusing HELLO_BUNDLE for alt root
Overloading $HELLO_BUNDLE with a second purpose of having an alternative
runc root is confusing.

Instead, let's create a temp directory in setup() and clean it up in
teardown().

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-08 12:45:44 -08:00
Kir Kolyshkin 76532fac03 tests/int/events: rm unneeded eval
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-08 12:45:44 -08:00
Kir Kolyshkin 497661406e tests/int: use wait_for_container where appropriate
... instead of more generic "retry", to simplify code.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-08 12:45:44 -08:00
Kir Kolyshkin 4d6ffa39a0 tests/int/helpers: reimplement wait_for_container
... by using retry, to avoid code duplication.

While at it, make it check the number of arguments and error out on
invalid usage.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-08 12:45:44 -08:00
Kir Kolyshkin e7052dcd83 tests/int/spec.bats: don't use HELLO_BUNDLE
All tests are run with cd to bundle directory, so
let's just use $(pwd).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-08 12:45:44 -08:00
Kir Kolyshkin 0cfc2e327d tests/int: rm teardown_running_container_inroot
It is only used in a couple of places and is easy to replace
with ROOT=xxx teardown_running_container.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-08 12:45:44 -08:00
Kir Kolyshkin 78f0e4b277 tests/int: rm wait_for_container_inroot
Since commit abd0515816 it is no longed used.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-08 12:45:44 -08:00
Kir Kolyshkin 64d5702f02 tests/int: don't depend on BUSYBOX_BUNDLE var
All the tests are run with cd to bundle directory, so
it does not make ense to explicitly specify it.

This relies on the feature of functions like set_cgroups_path
or update_config to use current directory by default.

In those cases where we need to change the directory
(runc create/run -b), save the current one and use it later.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-08 12:45:44 -08:00
Vasiliy Ulyanov efb8552b05 tests/int: add device access test
The test verifies if the device file can be queried using
'access(dev_name, F_OK)' when the permissions are set to 'rw'. The call
does not explicitly trigger read or write access but should succeed.

Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>
2021-02-08 14:14:14 +01:00
Kir Kolyshkin c348b98203 tests/rootless.sh: fix/ignore shellcheck warnings
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-02 16:03:52 -08:00
Kir Kolyshkin 1143759338 tests/rootless.sh: use set -e -u -o pipefail
Currently, set -e ("exit on error") is only set before running tests.
The bad consequence is the script may fail to set up test environment
but will still run the tests.

Use "set -e -u -o pipefail" for the whole script to catch potential
issues with test setup and/or the script itself.

A few caveats:

1. We have to set RUNC_USE_SYSTEMD to an empty value if it is unset, so
   that the subsequent checks like [ -z "$RUNC_USE_SYSTEMD" ] won't
   trigger a "unbound variable" error. Same for ROOTLESS_TESTPATH.

2. Functions that have code like [ -f $file ] && do_something towards
   the end may return 1 in case $file does not exist (as test aka [
   was the last operation performed, and its exit code is returned.
   This is why we had to add return 0.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-02 16:03:52 -08:00
Kir Kolyshkin 5ab0588432 ci: untangle getting test images
This simplifies and optimizes getting container images used for tests.

Currently, we have three different ways of getting images:

1. (for hello-world) the image is in this repo under tests/integration/testdata.

2. (for busybox) download it from github (the repo that is used for
   preparing official Docker image) using curl.

3. (for debian) download from Docker hub, using skopeo and umoci.

To further complicate things, we have to do this downloading in multiple
scenarios (at least 4): locally, in github CI, from Dockefile, inside a
Vagrant VM. For each scenario, we have to install skopeo and umoci, and
those two are not yet universally available for all the distros that we
use.

Yet another complication is those images are used for tests/integration
(bats-driven tests) as well as for libcontainer/integration (go tests).
The tests in libcontainer/integration rely on busybox being available
from /busybox, and the bats tests just download the images to a
temporary location during every run.

It is also hard to support CI for other architectures, because all
the machinery for preparing images is so complicated.

This commit is an attempt to simplify and optimize getting images,
mostly by getting rid of skopeo and umoci dependencies, but also
by moving the download logic into one small shell script, which
is used from all the places.

Benefits:

 - images (if not present) are only downloaded once;
 - same images are used for both kind of tests (go and bats);
 - same images are used for local and inside-docker tests
   (because source directory is mounted into container);
 - the download logic is located within 1 simple shell script.

[v2: fix eval; more doc to get-images; print URL if curl failed]
[v3: use "slim" debian, twice as small]
[v4: fix not using $image in setup_bundle]
[v5: don't remove TESTDATA from helpers.bash]
[v6: add i386 support]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-02 16:03:29 -08:00
Kir Kolyshkin bc175ba423 tests/helpers.bash: rm GOPATH
It is not used anywhere since commit 30601efa81.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-02-02 11:47:06 -08:00
Aleksa Sarai a4f2b2b0a8 merge branch 'pr-2769'
Kir Kolyshkin (2):
  ci: test with golang 1.16-rc1
  tests/int/spec.bats: fix for go 1.16

LGTMs: @AkihiroSuda @cyphar
Closes #2769
2021-02-01 11:05:49 +11:00
Kir Kolyshkin 164e0adb58 tests/int/spec.bats: fix for go 1.16
go 1.16 has GO111MODULE=on by default, and since runtime-spec/schema
does not come with go.mod, it complains:

> runtime-spec/schema/validate.go:10:2: no required module provides package github.com/xeipuuv/gojsonschema: working directory is not part of a module

Use GO111MODULE=auto as a workaround.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-01-28 13:13:56 -08:00
Aleksa Sarai 8bd19cd5f8 tests: add seccomp -ENOSYS integration test
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
2021-01-28 23:11:25 +11:00
Aleksa Sarai 346f87fec1 merge branch 'pr-2715'
Kir Kolyshkin (3):
  tests/int/mounts.bats: cleanup
  tests/int/mount.bats: reformat
  runc run: resolve tmpfs mount dest in container scope

LGTMs: @AkihiroSuda @cyphar
Closes #2715
2021-01-27 21:50:11 +11:00
Mrunal Patel 2adbc66b71 Merge pull request #2727 from kolyshkin/cpuset
libc/cg: convert r.CPU.Cpus/Mems to systemd props
2021-01-20 21:25:52 -08:00
Kir Kolyshkin a21e57d7f6 tests/int/hooks.bats: skip earlier
This particular test downloads and unpacks debian from setup(),
but then in many cases the actual test is skipped because it
requires root and no_systemd.

Let's skip it earlier.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-01-14 18:11:05 -08:00
Kir Kolyshkin 03b512e511 libc/cg: convert r.CPU.Cpus/Mems to systemd props
Support for systemd properties AllowedCPUs and AllowedMemoryNodes
was added by commit 13afa58d0e, but only for unified resources
of systemd v2 driver.

This adds support for Cpu.Cpus and Cpu.Mems resources to
both systemd v1 and v2 cgroup drivers.

An integration test is added to check that the settings work.

[v2: check for systemd version]
[v3: same in the test]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-01-13 19:28:54 -08:00
Kir Kolyshkin 5de00ad9d4 tests/int/multi-arch.bash: fix for busybox
Due to https://github.com/docker-library/busybox/pull/94 the URLs
to get busybox has changed, so the current URLs give HTTP 404.

Fix the URLs accordingly.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-01-11 17:41:28 -08:00
Kir Kolyshkin 637f82d6b8 runc run: resolve tmpfs mount dest in container scope
In case a tmpfs mount path contains absolute symlinks, runc errors out
because those symlinks are resolved in the host (rather than container)
filesystem scope.

The fix is similar to that for bind mounts -- resolve the destination
in container rootfs scope using securejoin, and use the resolved path.

A simple integration test case is added to prevent future regressions.

Fixes https://github.com/opencontainers/runc/issues/2683.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-01-06 14:57:02 -08:00
Kir Kolyshkin d64c3afe1b tests/int/mount.bats: reformat
Easier to read mounts. No functional change.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-01-06 14:57:02 -08:00
Kir Kolyshkin a2c9866e54 tests/int/mounts.bats: cleanup
1. Make sure that containers created are stopped and removed.
This is done by using a predefined test_busybox container name,
which is getting removed in teardown_busybox.

2. Fix space at EOL.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-01-06 14:57:02 -08:00
Kir Kolyshkin 1125ae7835 tests/events.bats: unify duplicated code
There are three tests that only differ in delays (and a few other
minor details line whitespace formatting and number of retries).

Let's unify those so maintaining this code will be easier.

This also implicitly increases the number of retries from 3 to 10
for the non-default --interval tests.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2021-01-05 20:26:03 -08:00
Kir Kolyshkin 27268b1a7e tests/int/cwd: add test case for cwd not owned by runc
The test setup is complicated, as we want to run it as non-root,
but we need to create a directory owned by a particular user,
and root is needed for chown. To do that, we hijack enable_idmap.

Now, there is no ideal place to clean up AUX_DIR. A function
cleanup is called after the test to do so. Note that it won't
be called if the test fails.

This was verified to fail before and pass after the
previous commit.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-12-18 13:23:27 -08:00
Kir Kolyshkin 8bd3b8786b test: add case for GH #2086
Commit 5e0e67d7 ("fix permission denied") modified some code but
did not provide a test case.

This is a test case that was tested to fail before and succeed after
the above commit.

For more details, see https://github.com/opencontainers/runc/pull/2086

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-12-18 12:56:48 -08:00
Kir Kolyshkin fce8dd4d66 tests/int/tty.bats: increase timeout
Just saw the timeout being hit on CI (Fedora 33 vagrant VM on Travis),
so let's increase it.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-27 14:15:32 -08:00
Sebastiaan van Stijn d9010b0e2d integration: update README to link to bats-core
We switched to bats-core in commit d4bc7c10ec,
so updating the instructions to match.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-17 17:46:04 +01:00
Mrunal Patel 27227a9358 Merge pull request #2669 from kolyshkin/systemd-unified
Initial v2 resources.unified systemd support
2020-11-06 12:31:31 -08:00
Kir Kolyshkin c013be563c libct/cg/sd/v2: support memory.* / Memory* unified
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:05:00 -08:00
Kir Kolyshkin 13afa58d0e libct/cg/sd/v2: support cpuset.* / Allowed*
* cpuset.cpus -> AllowedCPUs
 * cpuset.mems -> AllowedMemoryNodes

No test for cgroup v2 resources.unified override, as this requires a
separate test case, and all the unified resources are handled uniformly
so there's little sense to test all parameters.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:04:57 -08:00