Those tests were added by commit 8d180e96 ("Add support for Linux
Network Devices"), apparently by copy-pasting the test cases which
call simple_cr (all four of them).
While different simple_cr tests make sense as they cover different
code paths in runc and/or check for various regression, the same
variations with netdevice do not make sense, as having a net device
is orthogonal to e.g. bind mount, --debug, or cgroupns.
Remove those.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Some of runc integration tests may do something that I would not like
when running those on my development laptop. Examples include
- changing the root mount propagation [1];
- replacing /root/runc [2];
- changing the file in /etc (see checkpoint.bats).
Yet it is totally fine to do all that in a throwaway CI environment,
or inside a Docker container.
Introduce a mechanism to skip specific "unsafe" tests unless an
environment variable, RUNC_ALLOW_UNSAFE_TESTS, is set. Use it
from a specific checkpoint/restore test which modifies
/etc/criu/default.conf.
[1]: https://github.com/opencontainers/runc/pull/5200
[2]: https://github.com/opencontainers/runc/pull/5207
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In our bats tests, runc itself is a wrapper which calls bats run helper,
so using "run runc" is wrong as it results in calling run helper twice.
Fixes: 8d180e965
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is a bit opinionated, but some comments in integration tests do not
really help to understand the nature of the tests being performed by
stating something very obvious, like
# run busybox detached
runc run -d busybox
To make things worse, these not-so-helpful messages are being
copy/pasted over and over, and that is the main reason to remove them.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Implement support for passing Linux Network Devices to the container
network namespace.
The network device is passed during the creation of the container,
before the process is started.
It implements the logic defined in the OCI runtime specification.
Signed-off-by: Antonio Ojea <aojea@google.com>
Since v3.14, CRIU always restores processes into a time namespace to
prevent backward jumps of monotonic and boottime clocks. This change
updates the container configuration to ensure that `runc exec` launches
new processes within the container's time namespace.
Fixes#2610
Signed-off-by: Andrei Vagin <avagin@gmail.com>
1. A few tests use "criu check --feature" to check for a specific
feature. Let's generalize it.
2. Fix "checkpoint --pre-dump and restore" test to require memory
tracking (which is missing on ARM).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Since commit c77aaa3f the tail of criu.log is printed by runc, so
there's no need to do the same thing in tests.
Related to 3711, 3816.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Apparently, bash with set -e deliberately ignores non-zero return codes
from ! cmd, unless this is the last command. The workaround is to either
use "! cmd || false', "or run ! cmd". Choose the latter, and require
bash-core 1.5.0 (since this is when "run !" was added), replacing the
older check.
Alas I only learned this recently from the bash-core documentation.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
When doing a lazy checkpoint/restore, we should not restore into the
same cgroup, otherwise there is a race which result in occasional
killing of the restored container (GH #2760, #2924).
The fix is to use --manage-cgroup-mode=ignore, which allows to restore
into a different cgroup.
Note that since cgroupsPath is not set in config.json, the cgroup is
derived from the container name, so calling set_cgroups_path is not
needed.
For the previous (unsuccessful) attempt to fix this, as well as detailed
(and apparently correct) analysis, see commit 36fe3cc28c.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This test checks that the container is restored into a different cgroup.
To do so, a user should
- use --manage-cgroups-mode ignore on both checkpoint and restore;
- change the cgroupsPath value in config.json before restoring.
The test does some checks to ensure that its logic is correct, and that
after the restore the old (original) cgroup does not exist, the new one
exists, and the container's init is in that new cgroup.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Strictly speaking, == is for [[ only, not for [ / test,
and, unlike =, the right side is a pattern.
To avoid confusion, use =. In cases where we compare with empty string,
use -z instead.
Keep using [[ in some cases since it does not require quoting the left
and right side of comparison (I trust shellcheck on that one).
This should have no effect (other than the code being a tad more
strict).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Audit all checks for non-empty variables (i.e. ' -z ', ' -n ',
' != ""' and '= ""'), and fix those cases where a variable might be
unset. Those variables (that might not be set) are
- RUNC_USE_SYSTEMD
- BATS_RUN_TMPDIR
- AUX_UID
- AUX_DIR
- SD_PARENT_NAME
- REL_PARENT_PATH
- ROOT
- HAVE_CRIU
- ROOTLESS_FEATURES
- and a few test-specific or file-specific variables
This should allow us to enable set -u.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This was introduced in an initial commit, back in the day when criu was
a highly experimental thing. Today it's not; most users who need it have
it packaged by their distro vendor.
The usual way to run a binary is to look it up in directories listed in
$PATH. This is flexible enough and allows for multiple scenarios (custom
binaries, extra binaries, etc.). This is the way criu should be run.
Make --criu a hidden option (thus removing it from help). Remove the
option from man pages, integration tests, etc. Remove all traces of
CriuPath from data structures.
Add a warning that --criu is ignored and will be removed.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
runc resolves symlink before doing bind mount. So
we should save original path while formatting CriuReq for
dump and restore.
"checkpoint: resolve symlink for external bind mount" is merged as
da22625f6986f0ef196eaa1f8bb6adce098f0fb7(PR 2902) previously. And reverted
in commit 70fdc0573dced3464e9c31d674559f77c1de3973(PR 3043) duo to behavior changes
caused by commit 0ca91f44f1664da834bc61115a849b56d22f595f(Fixes: CVE-2021-30465)
Signed-off-by: Liu Hua <weldonliu@tencent.com>
runc resolves symlink before doing bind mount. So
we should save original path while formatting CriuReq for
checkpoint.
Signed-off-by: Liu Hua <weldonliu@tencent.com>
`--parent-path` needs to be relative path of `--image-path`,
and points to previous checkpoint directory.
Signed-off-by: Liu Hua <weldonliu@tencent.com>
Criu creates symlink named "parent" under image-path while doing
interactive checkpoint, without checking wether symlink points to
right place. This patch corrects related test case.
Signed-off-by: Liu Hua <weldonliu@tencent.com>
"checkpoint --lazy-pages and restore" test sometimes fails on restore
in our CI on Fedora 33 when systemd cgroup driver is used:
> (00.076104) Error (compel/src/lib/infect.c:1513): Task 48521 is in unexpected state: f7f
> (00.076122) Error (compel/src/lib/infect.c:1520): Task stopped with 15: Terminated
> ...
> (00.078246) Error (criu/cr-restore.c:2483): Restoring FAILED.
I think what happens is
1. The test runs runc checkpoint in lazy-pages mode in background.
2. The test runs criu lazy-pages in background.
3. The test runs runc restore.
Now, all three are working in together: criu restore restores, criu
lazy-pages listens for page faults on a uffd and fetch missing pages
from runc checkpoint, who serves those pages.
At some point criu lazy-pages decides to fetch the rest of the pages,
and once it's done it exits, and runc checkpoint, as there are no more
pages to serve, exits too.
At the end of runc checkpoint the container is removed (see "defer
destroy(container)" in checkpoint.go. This involves a call to
cgroupManager.Destroy, which, in case systemd manager is used,
calls stopUnit, which makes systemd to not just remove the unit,
but also send SIGTERM to its processes, if there are any.
As the container is being restored into the same systemd unit,
sometimes this results in sending SIGTERM to a process which
criu restores, and thus restoring fails.
The remedy here is to change the name of systemd unit to which the
container is restored.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In check_pipes, make sure we
- close all fds we opened in setup_pipes;
- check that runc stderr is empty (and fail if it's not).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The full log is very long and it did not gave us any additional clues.
This reverts commit 053e15c001.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Sometimes the test fails without any clear diagnostics:
> not ok 16 checkpoint --lazy-pages and restore
> (in test file tests/integration/checkpoint.bats, line 191)
> `[ "$out" = "0000000 000000 0000001" ]' failed
> ...
> criu failed: type NOTIFY errno 3\nlog file: work-dir/dump.log
We look for and print errors via grep, but in the above case
there are nothing that is denoted error in the log.
So, let's show the damn log in its entirely (note it is only shown
if test fails).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In case log level is less than debug, this code does nothing,
so let's add a condition and skip it entirely.
Add a test case to make sure this code path is hit.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This adds a checkpoint/restore test to ensure that nested bind mount
mountpoints are correctly re-created.
Signed-off-by: Adrian Reber <areber@redhat.com>
1. Get rid of fixed ROOT, *_BUNDLE, and CONSOLE_SOCKET dirs.
Now they are temporary directories created in setup_bundle.
2. Automate containers cleanup: instead of having to specify all
containers to be removed, list and destroy everything (which is
now possible since every test case has its own unique root).
3. Randomize cgroup paths so two tests running in parallel won't
use the same cgroup.
Now it's theoretically possible to execute tests in parallel.
Practically it's not possible yet because bats uses GNU parallel,
which do not provide a terminal for whatever it executes, and
many runc tests (all those that run containers with terminal:
true) needs a tty. This may possibly be addressed later.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Bats' run should only be used when we want to check both the command
output and its non-zero exit status.
Otherwise, we can rely on implicit exit code check (as the tests are
run with set -e), or use if, etc.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Various bats tests use various types of indentation (about half
is tabs, the rest is 2 spaces, 4 spaces, etc.).
Let's bring it to one style (tabs) using recently added
shfmt support for bats files (see [1]).
This commit is brought to you by
shfmt -ln bats -w tests/integration/*.bats
[1] https://github.com/mvdan/sh/issues/600
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
With some grep|awk help, found a few places where the containers
supposed to be removed in teardown() weren't.
To find container names:
grep -E '^[[:space:]]*\<_*_*runc\>.*\<(create|run|restore)\>'
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
For test cases where we used pipes for container's stdin/stdout/stderr,
stderr was redirected to the same pipe as stdout, which practically
means it is lost.
These redirects to fd is needed not because we check that container is
working by writing to its stdin and reading from stdout (see
check_pipes), but also because bats redirects test stdout/stderr to a
file, which makes c/r impossible (as the file is outside of container).
This is why we can't just do something like `2>stderr.log`, and have
to do what is done in this commit.
Introduce and use another pipe for stdout, to be used for both runc run
and runc restore, so it will be shown in case of errors.
Since its handling is somewhat complicated and is used from 4 places
(2 for run, 2 for restore), separate it into a helper functions.
NOTE the code assumes that runc exits with non-zero exit code in case
there is anything that needs to be shown to a user from runc's stderr.
While at it, add error checking to runc run calls.
Hopefully, this will help debug those rare checkpoint failures in CI.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Commit 417f5ff40 added a code to close fds and kill processes, which
should have helped in an event of test case failure.
In fact, each test case is executed in a subshell, so
- any variables set there can't reach teardown();
- all the fds are closed (as the process is gone).
Now, I am not sure about the processes, but the code being removed
has never worked anyway, so it does not make sense to keep it.
Normally, those are waited for. In case of a test case failure,
well, the subsequent cases might fail, too.
Fixes: 417f5ff40d
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This fixes the following issue with the test:
not ok 12 checkpoint --lazy-pages and restore
# (in test file tests/integration/checkpoint.bats, line 202)
# `[ $ret -eq 0 ]' failed
...
# grep: ./work-dir/restore.log: No such file or directory
One might think that `--work-path ./image-dir` is a mistake, but it's
not, since `criu lazy-pages -D ./image-dir` creates a socket in
./image-dir, and then `criu restore --lazy-pages` expects a socket in
the workdir.
Fixes: e232a71a3d
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Ignore the shellcheck warnings like this one:
> In tty.bats line 32:
> update_config '(.. | select(.[]? == "sh")) += ["-c", "stat -c %u:%g $(tty) | tr : \\\\n"]'
> ^-- SC2016: Expressions don't expand in single quotes, use double quotes for that.
While at it, fix some minor whitespace issues in tty.bats.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Ignore warnings like this:
> In checkpoint.bats line 169:
> PIDS_TO_KILL=($cpt_pid)
> ^------^ SC2206: Quote to prevent word splitting/globbing, or split robustly with mapfile or read -a.
Since in all the cases we deal with either pids or fds, and they don't
have spaces.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Those are pretty simple to allow shellcheck to fix these, so
this commit is courtesy of
> shellcheck -i SC2086 -i SC2006 -f diff *.bats > fix.diff
> patch -p1 < fix.diff
repeated 3 times ;)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
TL;DR: this allows to show logs from failed runc restore.
Bats scripts are run with `set -e`. This is well known and obvious,
and yet there are a few errors with respect to that, including a few
"gems" by yours truly.
1. bats scripts are run with `set -e`, meaning that `[ $? -eq 0 ]` is
useless since the execution won't ever reach this line in case of
non-zero exit code from a preceding command. So, remove all such
checks, they are useless and misleading.
2. bats scripts are run with `set -e`, meaning that `ret=$?` is useless
since the execution won't ever reach this line in case of non-zero
exit code from a preceding command.
In particular, the code that calls runc restore needs to save the exit
code, show the errors in the log, and only when check the exit code and
fail if it's non-zero. It can not use `run` (or `runc` which uses `run`)
because of shell redirection that we need to set up.
The solution, implemented in this patch, is to use code like this:
```bash
ret=0
__runc ... || ret=$?
show_logs
[ $ret -eq 0 ]
```
In case __runc exits with non-zero exit code, `ret=$?` is executed, and
it always succeeds, so we won't fail just yet and have a chance to show
logs before checking the value of $ret.
In case __runc succeeds, `ret=$?` is never executed, so $ret will still
be zero (this is the reason why it needs to be set explicitly).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
sean