Add functional test to check seccomp notify end-to-end. This test uses the
sample seccomp agent from the contrib/cmd folder.
Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Rodrigo Campos <rodrigo@kinvolk.io>
Co-authored-by: Rodrigo Campos <rodrigo@kinvolk.io>
Add a test case for an issue fixed by the previous commit.
Unfortunately, this is somewhat complicated as there's no easy way to
create a transient unit, so a binary, sd-helper, had to be added. On top
of that, an ability to create a parent/pod cgroup is added to
helpers.bash, which might be useful for future integration tests.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
... and add the file to be checked by shellcheck.
The warnings fixed are:
In tests/integration/helpers.bash line 10:
INTEGRATION_ROOT=$(dirname "$(readlink -f "$BASH_SOURCE")")
^----------^ SC2128: Expanding an array without an index only gives the first element.
In tests/integration/helpers.bash line 22:
TESTDATA="${INTEGRATION_ROOT}/testdata"
^------^ SC2034: TESTDATA appears unused. Verify use (or export if used externally).
In tests/integration/helpers.bash line 42:
echo "runc $@ (status=$status):" >&2
^-- SC2145: Argument mixes string and array. Use * or separate argument.
^-----^ SC2154: status is referenced but not assigned.
In tests/integration/helpers.bash line 43:
echo "$output" >&2
^-----^ SC2154: output is referenced but not assigned.
In tests/integration/helpers.bash line 77:
| .linux.gidMappings += [{"hostID": '"$(($ROOTLESS_GIDMAP_START + 10))"', "containerID": 1, "size": 20}]
^--------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
In tests/integration/helpers.bash line 78:
| .linux.gidMappings += [{"hostID": '"$(($ROOTLESS_GIDMAP_START + 100))"', "containerID": 1000, "size": '"$(($ROOTLESS_GIDMAP_LENGTH - 1000))"'}]'
^--------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
^---------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
In tests/integration/helpers.bash line 125:
base_path=$(gawk '$(NF-2) == "cgroup" && $NF ~ /\<'${g}'\>/ { print $5; exit }' /proc/self/mountinfo)
^--^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
base_path=$(gawk '$(NF-2) == "cgroup" && $NF ~ /\<'"${g}"'\>/ { print $5; exit }' /proc/self/mountinfo)
In tests/integration/helpers.bash line 127:
eval CGROUP_${g^^}_BASE_PATH="${base_path}"
^----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
eval CGROUP_"${g^^}"_BASE_PATH="${base_path}"
In tests/integration/helpers.bash line 229:
if [ "x$CGROUP_UNIFIED" = "xyes" ]; then
^----------------^ SC2268: Avoid x-prefix in comparisons as it no longer serves a purpose.
Did you mean:
if [ "$CGROUP_UNIFIED" = "yes" ]; then
In tests/integration/helpers.bash line 234:
eval cgroup=\$${var}${REL_CGROUPS_PATH}
^----^ SC2086: Double quote to prevent globbing and word splitting.
^-----------------^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
eval cgroup=\$"${var}""${REL_CGROUPS_PATH}"
In tests/integration/helpers.bash line 236:
cat $cgroup/$source
^-----^ SC2086: Double quote to prevent globbing and word splitting.
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
cat "$cgroup"/"$source"
In tests/integration/helpers.bash line 242:
current="$(get_cgroup_value $1)"
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
current="$(get_cgroup_value "$1")"
In tests/integration/helpers.bash line 245:
echo "current" $current "!?" "$expected"
^------^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
echo "current" "$current" "!?" "$expected"
In tests/integration/helpers.bash line 257:
[ $(id -u) != "0" ] && user="--user"
^------^ SC2046: Quote this to prevent word splitting.
In tests/integration/helpers.bash line 259:
current=$(systemctl show $user --property $source $SD_UNIT_NAME | awk -F= '{print $2}')
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
current=$(systemctl show $user --property "$source" $SD_UNIT_NAME | awk -F= '{print $2}')
In tests/integration/helpers.bash line 261:
[ "$current" = "$expected" ] || [ -n "$expected2" -a "$current" = "$expected2" ]
^-- SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
In tests/integration/helpers.bash line 309:
check_cgroup_value "cpu.weight" $weight
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
check_cgroup_value "cpu.weight" "$weight"
In tests/integration/helpers.bash line 310:
check_systemd_value "CPUWeight" $weight
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
check_systemd_value "CPUWeight" "$weight"
In tests/integration/helpers.bash line 383:
if [ $CGROUP_UNIFIED = "no" -a ! -e "${CGROUP_MEMORY_BASE_PATH}/memory.memsw.limit_in_bytes" ]; then
^-- SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
In tests/integration/helpers.bash line 412:
local cpu_count=$(grep -c '^processor' /proc/cpuinfo)
^-------^ SC2155: Declare and assign separately to avoid masking return values.
In tests/integration/helpers.bash line 450:
sleep $delay
^----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
sleep "$delay"
In tests/integration/helpers.bash line 453:
echo "Command \"$@\" failed $attempts times. Output: $output"
^-- SC2145: Argument mixes string and array. Use * or separate argument.
In tests/integration/helpers.bash line 471:
runc state $1
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
runc state "$1"
In tests/integration/helpers.bash line 472:
if [ $2 == "checkpointed" ]; then
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
if [ "$2" == "checkpointed" ]; then
In tests/integration/helpers.bash line 484:
mkdir $dir
^--^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
mkdir "$dir"
In tests/integration/helpers.bash line 497:
kill -9 $(cat "$dir/pid")
^---------------^ SC2046: Quote this to prevent word splitting.
In tests/integration/helpers.bash line 508:
export ROOT=$(mktemp -d "$BATS_RUN_TMPDIR/runc.XXXXXX")
^--^ SC2155: Declare and assign separately to avoid masking return values.
In tests/integration/helpers.bash line 512:
cd "$ROOT/bundle"
^---------------^ SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Did you mean:
cd "$ROOT/bundle" || exit
In tests/integration/helpers.bash line 535:
cd "$INTEGRATION_ROOT"
^--------------------^ SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Did you mean:
cd "$INTEGRATION_ROOT" || exit
For more information:
https://www.shellcheck.net/wiki/SC2145 -- Argument mixes string and array. ...
https://www.shellcheck.net/wiki/SC2034 -- TESTDATA appears unused. Verify u...
https://www.shellcheck.net/wiki/SC2046 -- Quote this to prevent word splitt...
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It is not used since PR 2757, as all tests are run with cd to bundle
directory.
runc_spec argument count checking is removed since otherwise shellcheck
complains:
> SC2120: runc_spec references arguments, but none are ever passed.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Before this commit, "require rootless_cgroup" feature check required "cgroup"
to be present in ROOTLESS_FEATURES. The idea of the requirement, though, is
to ensure that rootless runc can manage cgroups.
In case of systemd + cgroup v2, rootless runc can manage cgroups,
thanks to systemd delegation, so modify the feature check accordingly.
Next, convert (simplify) some of the existing users to the modified check.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. In case of cgroup v2 + systemd + rootless, get the list of controllers from
the own cgroup, rather than from the root one (as systemd cgroup delegation
is required in this case, and it might not be set or fully working).
2. Use "requires cgroups_<controller>" in tests that need those
controllers.
Tested on Fedora 31 (which does not support delegation of cpuset controller).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This was added by commit ca4f427af, together with set_resources_limit,
and was needed for the latter, since sed was used to update resources.
Now when we switched to jq in commit 79fe41d3c1, this kludge is no
longer needed.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is somewhat radical approach to deal with kernel memory.
Per-cgroup kernel memory limiting was always problematic. A few
examples:
- older kernels had bugs and were even oopsing sometimes (best example
is RHEL7 kernel);
- kernel is unable to reclaim the kernel memory so once the limit is
hit a cgroup is toasted;
- some kernel memory allocations don't allow failing.
In addition to that,
- users don't have a clue about how to set kernel memory limits
(as the concept is much more complicated than e.g. [user] memory);
- different kernels might have different kernel memory usage,
which is sort of unexpected;
- cgroup v2 do not have a [dedicated] kmem limit knob, and thus
runc silently ignores kernel memory limits for v2;
- kernel v5.4 made cgroup v1 kmem.limit obsoleted (see
https://github.com/torvalds/linux/commit/0158115f702b).
In view of all this, and as the runtime-spec lists memory.kernel
and memory.kernelTCP as OPTIONAL, let's ignore kernel memory
limits (for cgroup v1, same as we're already doing for v2).
This should result in less bugs and better user experience.
The only bad side effect from it might be that stat can show kernel
memory usage as 0 (since the accounting is not enabled).
[v2: add a warning in specconv that limits are ignored]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Commit 41670e21f0 added some randomization to cgroup paths
and (if systemd cgroup driver is used) systemd unit names,
but the randomization was per bats instance, not per test.
Fix this by refactoring init_cgroups_path/set_cgroups_path
(moving variable/random part to set_cgroups_path).
NOTE though that the randomization is only performed for those tests
that explicitly call set_cgroups_path.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Helper function init_cgroup_paths sets two sets of cgroup path variables
for cgroup v1 case (below XXX is cgroup controller name, e.g. MEMORY):
1. CGROUP_XXX_BASE_PATH -- path to XXX controller mount point
(e.g. CGROUP_MEMORY_BASE_PATH=/sys/fs/cgroup/memory);
2. CGROUP_XXX -- path to the particular container XXX controller cgroup
(e.g. CGROUP_MEMORY=/sys/fs/cgroup/memory/runc-cgroups-integration-test/test-cgroup).
The second set of variables is mostly used by check_cgroup_value(),
with only two exceptions:
- CGROUP_CPU in @test "update rt period and runtime";
- few CGROUP_XXX in @test "runc delete --force in cgroupv1 with
subcgroups".
Remove these variables, as their values are not used much
and are easy to get (as can be seen in modified test cases).
While at it, mark some variables as local.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Get rid of fixed ROOT, *_BUNDLE, and CONSOLE_SOCKET dirs.
Now they are temporary directories created in setup_bundle.
2. Automate containers cleanup: instead of having to specify all
containers to be removed, list and destroy everything (which is
now possible since every test case has its own unique root).
3. Randomize cgroup paths so two tests running in parallel won't
use the same cgroup.
Now it's theoretically possible to execute tests in parallel.
Practically it's not possible yet because bats uses GNU parallel,
which do not provide a terminal for whatever it executes, and
many runc tests (all those that run containers with terminal:
true) needs a tty. This may possibly be addressed later.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It's a little known fact, but BATS_TMPDIR is just /tmp, and it is not
being cleaned by bats after the test.
BATS_RUN_TMPDIR (available since bats 1.2.1) is $BATS_TMPDIR/bats-run-$$
(i.e. it is per bats run and includes PID), and it is cleaned up after
the test (unless --no-tmpdir-cleanup is given).
Use BATS_RUN_TMPDIR for better isolation and cleanup.
[v2: check for BATS_RUN_TMPDIR, effectively requiring bats >= 1.2.1]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
... by using retry, to avoid code duplication.
While at it, make it check the number of arguments and error out on
invalid usage.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It is only used in a couple of places and is easy to replace
with ROOT=xxx teardown_running_container.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This simplifies and optimizes getting container images used for tests.
Currently, we have three different ways of getting images:
1. (for hello-world) the image is in this repo under tests/integration/testdata.
2. (for busybox) download it from github (the repo that is used for
preparing official Docker image) using curl.
3. (for debian) download from Docker hub, using skopeo and umoci.
To further complicate things, we have to do this downloading in multiple
scenarios (at least 4): locally, in github CI, from Dockefile, inside a
Vagrant VM. For each scenario, we have to install skopeo and umoci, and
those two are not yet universally available for all the distros that we
use.
Yet another complication is those images are used for tests/integration
(bats-driven tests) as well as for libcontainer/integration (go tests).
The tests in libcontainer/integration rely on busybox being available
from /busybox, and the bats tests just download the images to a
temporary location during every run.
It is also hard to support CI for other architectures, because all
the machinery for preparing images is so complicated.
This commit is an attempt to simplify and optimize getting images,
mostly by getting rid of skopeo and umoci dependencies, but also
by moving the download logic into one small shell script, which
is used from all the places.
Benefits:
- images (if not present) are only downloaded once;
- same images are used for both kind of tests (go and bats);
- same images are used for local and inside-docker tests
(because source directory is mounted into container);
- the download logic is located within 1 simple shell script.
[v2: fix eval; more doc to get-images; print URL if curl failed]
[v3: use "slim" debian, twice as small]
[v4: fix not using $image in setup_bundle]
[v5: don't remove TESTDATA from helpers.bash]
[v6: add i386 support]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
* cpuset.cpus -> AllowedCPUs
* cpuset.mems -> AllowedMemoryNodes
No test for cgroup v2 resources.unified override, as this requires a
separate test case, and all the unified resources are handled uniformly
so there's little sense to test all parameters.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Bats' run should only be used when we want to check both the command
output and its non-zero exit status.
Otherwise, we can rely on implicit exit code check (as the tests are
run with set -e), or use if, etc.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In its current form, it is complicated, unreliable, and error prone.
Using runc delete -f will kill and remove any container, running or not,
and it won't error if a container with a given name does not exist.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
I played with shfmt options (-bn, -ci, -sr) a bit trying to minimize
the patch generated (and also because I don't have a strong preference
on these matters), and it appears to be that the patch size is about the
same nevertheless, so I chose no options.
This commit is brought to you by
shfmt -ln bash -w man/*.sh script/*.sh tests/*.sh tests/integration/*.bash
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The purpose of the spec validation test is to make sure
that config.json generated by runc spec is valid. Therefore,
it does not depend on ROOTLESS_FEATURES.
Taking that into account, and given the fact that this test
involves cloning repos and building some code, it makes sense
to not run it 4 times for various rootless features.
Time it takes to execute spec.bats in rootless mode has improved.
Before:
real 0m21.286s
user 0m37.837s
sys 0m5.745s
After:
real 0m13.162s
user 0m30.814s
sys 0m4.050s
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This patch adds a test based on real world usage of runc hooks
(libnvidia-container). We verify that mounting a library inside
a container and running ldconfig succeeds.
Signed-off-by: Renaud Gaubert <rgaubert@nvidia.com>
For some reason, runc systemd drivers (both v1 and v2) never set
systemd unit property named `CPUQuotaPeriod` (known as
`CPUQuotaPeriodUSec` on dbus and in `systemctl show` output).
Set it, and add a check to all the integration tests. The check is less
than trivial because, when not set, the value is shown as "infinity" but
when set to the same (default) value, shown as "100ms", so in case we
expect 100ms (period = 100000 us), we have to _also_ check for
"infinity".
[v2: add systemd version checks since CPUQuotaPeriod requires v242+]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In case swap cgroup control is not available, the "event oom" test gives
the following error:
> # not ok 30 events oom
> # (in test file tests/integration/events.bats, line 134)
> # `[ "$status" -eq 0 ]' failed
> # <....>
> # runc run -d --console-socket /tmp/console.sock test_busybox (status=1):
> # time="2020-05-29T02:10:20Z" level=warning msg="signal: killed"
> # time="2020-05-29T02:10:20Z" level=error msg="container_linux.go:353: starting container process caused: process_linux.go:437: container init caused: process_linux.go:403: setting cgroup config for procHooks process caused: failed to write \"33554432\" to \"/sys/fs/cgroup/memory/test_busybox/memory.memsw.limit_in_bytes\": open /sys/fs/cgroup/memory/test_busybox/memory.memsw.limit_in_bytes: permission denied"
When I try to run the test without setting the swap limit, the shell
process is still getting killed, but the test hangs. I am not sure what
the reason is, but realistically this test is hard to perform without
the swap limit, so let's require cgroup swap for it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Such containers should remain paused after the update. This has
historically been true, but this helps ensure that the systemd cgroup
changes (freezing the container during SetUnitProperties) don't break
this behaviour.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
Introduce a special case for `testcontainer` to test
for container that is not present (checkpointed), use it.
Fix one place where testcontainer was not used.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Add `cgroups_v1` and `cgroups_v2` options to `requires`.
2. Modify `check_cgroup_value` to be able to work with v2.
3. Split `test "update"` into two:
- (1) testing cgroupv1-only cpu shares and cfs
- (2) testing limits that are more or less common
between v1 and v2: memory/swap, pids, cpusets.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Consolidate all the cgroup-related initialization code to
a single place, init_cgroup_paths(), so we can see which
variables are set.
2. Lazily call init_cgroup_paths() from all places that require it.
3. Don't set globals KMEM and RT_PERIOD.
4. Slightly clarlify variables naming:
- use OCI_CGROUPS_PATH for cgroupsPath in config.json
- use REL_CGROUPS_PATH for relative cgroups path
5. Do not hardcode the list of cgroup subsystems -- get it from
/proc/cgroup.
6. Preliminary support for cgroupv2 unified hierarchy (not yet working).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Consolidate two implementations of check_cgroup_value()
into one, putting it into helpers.
Remove the first parameter, deducing the variable to get
the path from by the parameter name.
This should help in future cgroupv2 support.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Mauricio Vásquez