1. A few tests use "criu check --feature" to check for a specific
feature. Let's generalize it.
2. Fix "checkpoint --pre-dump and restore" test to require memory
tracking (which is missing on ARM).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This handles a corner case when joining a container having all
the processes running exclusively on isolated CPU cores to force
the kernel to schedule runc process on the first CPU core within the
cgroups cpuset.
The introduction of the kernel commit
46a87b3851f0d6eb05e6d83d5c5a30df0eca8f76 has affected this deterministic
scheduling behavior by distributing tasks across CPU cores within the
cgroups cpuset. Some intensive real-time application are relying on this
deterministic behavior and use the first CPU core to run a slow thread
while other CPU cores are fully used by real-time threads with SCHED_FIFO
policy. Such applications prevents runc process from joining a container
when the runc process is randomly scheduled on a CPU core owned by a
real-time thread.
Introduces isolated CPU affinity transition OCI runtime annotation
org.opencontainers.runc.exec.isolated-cpu-affinity-transition to restore
the behavior during runc exec.
Fix issue with kernel >= 6.2 not resetting CPU affinity for container processes.
Signed-off-by: Cédric Clerget <cedric.clerget@gmail.com>
Now that runc-dmz is opt-in, we no longer need to try to detect whether
SELinux would cause issues for us. We can also remove the
special-purpose build-tag we added.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
It takes some time for the kernel to kill the process (and remove its
PID from cgroup.procs). To ensure we don't have flakes from reading
cgroup.procs right after the kill, check and wait for processes to
actually be gone.
Fixes: 4163
Reported-by: lifubang@acmcoder.com
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This expression is specific to GNU awk (gawk), so if someone has other version
of awk installed, this won't work and it's not easy to see why.
Explicitly requiring gawk here is better.
Revert "tests/int/helpers: gawk -> awk"
This reverts commit 4e65118d02.
Reported-by: lifubang <lifubang@acmcoder.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Previously, all of our userns tests worked around the remapping issue by
creating the paths that runc would attempt to create (like /proc).
However, this isn't really accurate to how real userns containers are
created, so it's much better to actually remap the rootfs.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
The container manager like containerd-shim can't use cgroup.kill feature or
freeze all the processes in cgroup to terminate the exec init process.
It's unsafe to call kill(2) since the pid can be recycled. It's good to
provide the pidfd of init process through the pidfd-socket. It's similar to
the console-socket. With the pidfd, the container manager like containerd-shim
can send the signal to target process safely.
And for the standard init process, we can have polling support to get
exit event instead of blocking on wait4.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Commit e1584831b6 did two modifications to check_cgroup_value():
1. It now skips the test if the file is not found.
2. If the comparison failed, a second comparison, with value divided by 1000,
is performed.
These modifications were only needed for cpu.burst, but instead were done
in a generic function used from many cgroup tests. As a result, we can
no longer be sure about the test coverage (item 1) and the check being
correct (item 2) anymore. In fact, part of "update cgroup cpu limits"
test is currently skipped on CentOS 7 and 8 because of item 1.
To fix:
- replace item 1 with a new "cgroups_cpu_burst" argument for "requires",
and move the test to a separate case;
- replace item 2 with a local change in check_cpu_burst.
Fixes: e1584831b6
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.
This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.
The patch adds the support/control for CFS bandwidth burst.
runtime-spec: https://github.com/opencontainers/runtime-spec/pull/1120
Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
These are not exhaustive, but at least confirm that the feature is not
obviously broken (we correctly set the time offsets).
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
We read output from the following files if they exists:
- cpu.pressure
- memory.pressure
- io.pressure
Each are in format:
```
some avg10=0.00 avg60=0.00 avg300=0.00 total=0
full avg10=0.00 avg60=0.00 avg300=0.00 total=0
```
Signed-off-by: Daniel Dao <dqminh89@gmail.com>
Co-authored-by: Sandor Szücs <sandor.szuecs@zalando.de>
Co-authored-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Rename CGROUP_PATH to CGROUP_V2_PATH so it is more clear that it can
only be used for CGROUP_V2, and to resolve ambiguity with CGROUP_PATH
variable used in tests/rootless.sh.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Apparently, bash with set -e deliberately ignores non-zero return codes
from ! cmd, unless this is the last command. The workaround is to either
use "! cmd || false', "or run ! cmd". Choose the latter, and require
bash-core 1.5.0 (since this is when "run !" was added), replacing the
older check.
Alas I only learned this recently from the bash-core documentation.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This test checks that the container is restored into a different cgroup.
To do so, a user should
- use --manage-cgroups-mode ignore on both checkpoint and restore;
- change the cgroupsPath value in config.json before restoring.
The test does some checks to ensure that its logic is correct, and that
after the restore the old (original) cgroup does not exist, the new one
exists, and the container's init is in that new cgroup.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Add a test case for an issue fixed by the previous commit.
The env should has more than 8 core CPU to meet the test requirement.
Signed-off-by: Chengen, Du <chengen.du@canonical.com>
We use awk in other 9 or so places, and here it's gawk.
Since this is on Linux, most probably awk is gawk.
So s/gawk/awk/.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The situation when /sys/fs/cgroup/unified is not present normal and
should not result in anything on stderr. Suppress it.
Fixes: cc15b887a0
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Currently only amd64 and arm64v8 tarball have been checked in testdata,
while busybox bundle is downloaded on fly, and supports multiple architectures.
To enable integration tests for more architectures, the hello world
bundle is replaced by busybox one.
Signed-off-by: Shengjing Zhu <zhsj@debian.org>
The variable $ROOTLESS, as set by helpers.bash and used in many places,
provides the same value as $EUID which is always set by bash. Since we
are using bash, we can rely on $EUID being omnipresent.
Modify all uses accordingly, and since the value is known to be a
number, omit the quoting.
Similarly, replace all uses of $(id -u) to $EUID.
Do some trivial cleanups along the way, such as
- simplify some if A; then B; to A && B;
- do not use [[ instead of [ where not necessary.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This makes it work similar to all the other variables we use as binary
flags.
The new 'shellcheck disable' is due to a bug in shellcheck (basically,
it does not track the scope of variables or execution order, assuming
everything is executed as soon as it is seen).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Strictly speaking, == is for [[ only, not for [ / test,
and, unlike =, the right side is a pattern.
To avoid confusion, use =. In cases where we compare with empty string,
use -z instead.
Keep using [[ in some cases since it does not require quoting the left
and right side of comparison (I trust shellcheck on that one).
This should have no effect (other than the code being a tad more
strict).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is a way to prevent the code doing something really bad when a
variable it uses is not set. Good to have since it helps to catch some
logical errors etc.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Audit all checks for non-empty variables (i.e. ' -z ', ' -n ',
' != ""' and '= ""'), and fix those cases where a variable might be
unset. Those variables (that might not be set) are
- RUNC_USE_SYSTEMD
- BATS_RUN_TMPDIR
- AUX_UID
- AUX_DIR
- SD_PARENT_NAME
- REL_PARENT_PATH
- ROOT
- HAVE_CRIU
- ROOTLESS_FEATURES
- and a few test-specific or file-specific variables
This should allow us to enable set -u.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Some tests (those in help.bats and version.bats) do not use setup_bundle
(as they do not need to start any containers), and thus they do not set
$ROOT. As a consequence, these tests now call "runc --root /state" which
is not nice.
Make adding --root conditional (only if $ROOT is set).
Amazingly, this change breaks help.bats tests under rootless, because
"sudo rootless" does not change the value of XDG_RUNTIME_DIR which still
points to root-owned directory, and as a result we have this:
> runc foo -h (status=1):
> the path in $XDG_RUNTIME_DIR must be writable by the user
> time="2022-02-08T07:04:57Z" level=error msg="mkdir /run/user/0/runc: permission denied"
This could be fixed by adding proper $ROOT, but it's easier just to skip
those tests under non-root.
NOTE that version.bats is not broken because -v is handled by urfave/cli
very early, so app.Before function is not run.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Older bash versions treats variable as unset if nothing has been
assigned to it. Here is an example from CentOS 7 system:
[kir@localhost ~]$ bash -u -c 'x() { local args=(); echo "${args[@]}"; }; x'
bash: args[@]: unbound variable
[kir@localhost ~]$ echo $BASH_VERSION
4.2.46(2)-release
Rewrite to work around this.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. This valid warning is reported by shellcheck v0.8.0:
In tests/integration/helpers.bash line 38:
KERNEL_MINOR="${KERNEL_VERSION#$KERNEL_MAJOR.}"
^-----------^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.
Did you mean:
KERNEL_MINOR="${KERNEL_VERSION#"$KERNEL_MAJOR".}"
Fix this.
2. These (invalid) warnings are also reported by the new version:
In tests/integration/events.bats line 13:
@test "events --stats" {
^-- SC2030 (info): Modification of status is local (to subshell caused by @bats test).
In tests/integration/events.bats line 41:
[ "$status" -eq 0 ]
^-----^ SC2031 (info): status was modified in a subshell. That change might be lost.
Basically, this is happening because shellcheck do not really track
the call tree and/or local variables. This is a known (and reported)
deficiency, and the alternative to disabling these warnings is moving
the code around, which is worse due to more changes in git history.
So we have to silence/disable these.
3. Update shellcheck to 0.8.0.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This was introduced in an initial commit, back in the day when criu was
a highly experimental thing. Today it's not; most users who need it have
it packaged by their distro vendor.
The usual way to run a binary is to look it up in directories listed in
$PATH. This is flexible enough and allows for multiple scenarios (custom
binaries, extra binaries, etc.). This is the way criu should be run.
Make --criu a hidden option (thus removing it from help). Remove the
option from man pages, integration tests, etc. Remove all traces of
CriuPath from data structures.
Add a warning that --criu is ignored and will be removed.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Using "$@" instead of $1 in update_config() allows us to use it from
hooks.bats, where jq is used with more options than usual.
We need to disable SC2016 as otherwise shellcheck sees $something inside
single quotes and think we are losing the shell expansion (we are not).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This can be used to specify a different runc binary, for example:
sudo -E RUNC=$PWD/runc.mine tests/integration/cwd.bats
A different (but compatible enough) runtime also works:
sudo -E RUNC=/usr/local/bin/crun tests/integration/cwd.bats
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Some test directories are created using mktemp -d, and so they have
permissions set to 0700 and are thus inaccessible to a user inside
userns. This was workarounded for $ROOT in userns.bats before.
Now, when we have updated Cirrus CI config to use Fedora 35 (rather than
34), userns tests fail:
> runc run failed: unable to start container process: error during
> container init: error preparing rootfs: mount
> /tmp/bats-run-4pCERd/runc.f66gCC/bundle/rootfs:/tmp/bats-run-4pCERd/runc.f66gCC/bundle/rootfs,
> flags: 0x5000: permission denied
Fedora 34 image used kernel v5.11, while Fedora 35 has v5.15.
Apparently, the newer kernel also checks that the parent directories
are accessible by the user before doing mount.
Move the old workaround from userns.bats to helpers.bats, drop the r bit
(not needed), and add $BATS_RUN_TMPDIR (also created by mktemp -d) to
fix userns.bats test failures under Fedora 35.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Check that runc run and runc exec put the process on the same cgroups v2
when using hybrid mode.
Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Add functional test to check seccomp notify end-to-end. This test uses the
sample seccomp agent from the contrib/cmd folder.
Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Rodrigo Campos <rodrigo@kinvolk.io>
Co-authored-by: Rodrigo Campos <rodrigo@kinvolk.io>
Add a test case for an issue fixed by the previous commit.
Unfortunately, this is somewhat complicated as there's no easy way to
create a transient unit, so a binary, sd-helper, had to be added. On top
of that, an ability to create a parent/pod cgroup is added to
helpers.bash, which might be useful for future integration tests.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
... and add the file to be checked by shellcheck.
The warnings fixed are:
In tests/integration/helpers.bash line 10:
INTEGRATION_ROOT=$(dirname "$(readlink -f "$BASH_SOURCE")")
^----------^ SC2128: Expanding an array without an index only gives the first element.
In tests/integration/helpers.bash line 22:
TESTDATA="${INTEGRATION_ROOT}/testdata"
^------^ SC2034: TESTDATA appears unused. Verify use (or export if used externally).
In tests/integration/helpers.bash line 42:
echo "runc $@ (status=$status):" >&2
^-- SC2145: Argument mixes string and array. Use * or separate argument.
^-----^ SC2154: status is referenced but not assigned.
In tests/integration/helpers.bash line 43:
echo "$output" >&2
^-----^ SC2154: output is referenced but not assigned.
In tests/integration/helpers.bash line 77:
| .linux.gidMappings += [{"hostID": '"$(($ROOTLESS_GIDMAP_START + 10))"', "containerID": 1, "size": 20}]
^--------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
In tests/integration/helpers.bash line 78:
| .linux.gidMappings += [{"hostID": '"$(($ROOTLESS_GIDMAP_START + 100))"', "containerID": 1000, "size": '"$(($ROOTLESS_GIDMAP_LENGTH - 1000))"'}]'
^--------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
^---------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
In tests/integration/helpers.bash line 125:
base_path=$(gawk '$(NF-2) == "cgroup" && $NF ~ /\<'${g}'\>/ { print $5; exit }' /proc/self/mountinfo)
^--^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
base_path=$(gawk '$(NF-2) == "cgroup" && $NF ~ /\<'"${g}"'\>/ { print $5; exit }' /proc/self/mountinfo)
In tests/integration/helpers.bash line 127:
eval CGROUP_${g^^}_BASE_PATH="${base_path}"
^----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
eval CGROUP_"${g^^}"_BASE_PATH="${base_path}"
In tests/integration/helpers.bash line 229:
if [ "x$CGROUP_UNIFIED" = "xyes" ]; then
^----------------^ SC2268: Avoid x-prefix in comparisons as it no longer serves a purpose.
Did you mean:
if [ "$CGROUP_UNIFIED" = "yes" ]; then
In tests/integration/helpers.bash line 234:
eval cgroup=\$${var}${REL_CGROUPS_PATH}
^----^ SC2086: Double quote to prevent globbing and word splitting.
^-----------------^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
eval cgroup=\$"${var}""${REL_CGROUPS_PATH}"
In tests/integration/helpers.bash line 236:
cat $cgroup/$source
^-----^ SC2086: Double quote to prevent globbing and word splitting.
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
cat "$cgroup"/"$source"
In tests/integration/helpers.bash line 242:
current="$(get_cgroup_value $1)"
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
current="$(get_cgroup_value "$1")"
In tests/integration/helpers.bash line 245:
echo "current" $current "!?" "$expected"
^------^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
echo "current" "$current" "!?" "$expected"
In tests/integration/helpers.bash line 257:
[ $(id -u) != "0" ] && user="--user"
^------^ SC2046: Quote this to prevent word splitting.
In tests/integration/helpers.bash line 259:
current=$(systemctl show $user --property $source $SD_UNIT_NAME | awk -F= '{print $2}')
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
current=$(systemctl show $user --property "$source" $SD_UNIT_NAME | awk -F= '{print $2}')
In tests/integration/helpers.bash line 261:
[ "$current" = "$expected" ] || [ -n "$expected2" -a "$current" = "$expected2" ]
^-- SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
In tests/integration/helpers.bash line 309:
check_cgroup_value "cpu.weight" $weight
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
check_cgroup_value "cpu.weight" "$weight"
In tests/integration/helpers.bash line 310:
check_systemd_value "CPUWeight" $weight
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
check_systemd_value "CPUWeight" "$weight"
In tests/integration/helpers.bash line 383:
if [ $CGROUP_UNIFIED = "no" -a ! -e "${CGROUP_MEMORY_BASE_PATH}/memory.memsw.limit_in_bytes" ]; then
^-- SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
In tests/integration/helpers.bash line 412:
local cpu_count=$(grep -c '^processor' /proc/cpuinfo)
^-------^ SC2155: Declare and assign separately to avoid masking return values.
In tests/integration/helpers.bash line 450:
sleep $delay
^----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
sleep "$delay"
In tests/integration/helpers.bash line 453:
echo "Command \"$@\" failed $attempts times. Output: $output"
^-- SC2145: Argument mixes string and array. Use * or separate argument.
In tests/integration/helpers.bash line 471:
runc state $1
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
runc state "$1"
In tests/integration/helpers.bash line 472:
if [ $2 == "checkpointed" ]; then
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
if [ "$2" == "checkpointed" ]; then
In tests/integration/helpers.bash line 484:
mkdir $dir
^--^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
mkdir "$dir"
In tests/integration/helpers.bash line 497:
kill -9 $(cat "$dir/pid")
^---------------^ SC2046: Quote this to prevent word splitting.
In tests/integration/helpers.bash line 508:
export ROOT=$(mktemp -d "$BATS_RUN_TMPDIR/runc.XXXXXX")
^--^ SC2155: Declare and assign separately to avoid masking return values.
In tests/integration/helpers.bash line 512:
cd "$ROOT/bundle"
^---------------^ SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Did you mean:
cd "$ROOT/bundle" || exit
In tests/integration/helpers.bash line 535:
cd "$INTEGRATION_ROOT"
^--------------------^ SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Did you mean:
cd "$INTEGRATION_ROOT" || exit
For more information:
https://www.shellcheck.net/wiki/SC2145 -- Argument mixes string and array. ...
https://www.shellcheck.net/wiki/SC2034 -- TESTDATA appears unused. Verify u...
https://www.shellcheck.net/wiki/SC2046 -- Quote this to prevent word splitt...
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It is not used since PR 2757, as all tests are run with cd to bundle
directory.
runc_spec argument count checking is removed since otherwise shellcheck
complains:
> SC2120: runc_spec references arguments, but none are ever passed.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Kir Kolyshkin