The passing run (with the fix) looks like this:
----
delete.bats
✓ runc delete removes failed systemd unit [4556]
runc spec (status=0):
runc run -d --console-socket /tmp/bats-run-B08vu1/runc.lbQwU5/tty/sock test-failed-unit (status=0):
Warning: The unit file, source configuration file or drop-ins of runc-cgroups-integration-test-12869.scope changed on disk. Run 'systemctl daemon-reload' to reload units.
× runc-cgroups-integration-test-12869.scope - libcontainer container integration-test-12869
Loaded: loaded (/run/systemd/transient/runc-cgroups-integration-test-12869.scope; transient)
Transient: yes
Drop-In: /run/systemd/transient/runc-cgroups-integration-test-12869.scope.d
└─50-DevicePolicy.conf, 50-DeviceAllow.conf
Active: failed (Result: timeout) since Tue 2023-06-13 14:41:38 PDT; 751ms ago
Duration: 2.144s
CPU: 8ms
Jun 13 14:41:34 kir-rhat systemd[1]: Started runc-cgroups-integration-test-12869.scope - libcontainer container integration-test-12869.
Jun 13 14:41:37 kir-rhat systemd[1]: runc-cgroups-integration-test-12869.scope: Scope reached runtime time limit. Stopping.
Jun 13 14:41:38 kir-rhat systemd[1]: runc-cgroups-integration-test-12869.scope: Stopping timed out. Killing.
Jun 13 14:41:38 kir-rhat systemd[1]: runc-cgroups-integration-test-12869.scope: Killing process 1107438 (sleep) with signal SIGKILL.
Jun 13 14:41:38 kir-rhat systemd[1]: runc-cgroups-integration-test-12869.scope: Failed with result 'timeout'.
runc delete test-failed-unit (status=0):
Unit runc-cgroups-integration-test-12869.scope could not be found.
----
Before the fix, the test was failing like this:
----
delete.bats
✗ runc delete removes failed systemd unit
(in test file tests/integration/delete.bats, line 194)
`run -4 systemctl status "$SD_UNIT_NAME"' failed, expected exit code 4, got 3
....
----
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
There is no such thing as linux.resources.memorySwap (the mem+swap is
set as linux.resources.memory.swap).
As it is not used in this test anyway, remove it.
Fixes: 4929c05ad1
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
When a directory already exists (or after a container is restarted) the
perms of the directory being mounted to were being used even when a
different permission is set on the tmpfs mount options.
This prepends the original directory perms to the mount options.
If the perms were already set in the mount opts then those perms will
win.
This eliminates the need to perform a chmod after mount entirely.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Rename CGROUP_PATH to CGROUP_V2_PATH so it is more clear that it can
only be used for CGROUP_V2, and to resolve ambiguity with CGROUP_PATH
variable used in tests/rootless.sh.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Filter out rdma controller since systemd is unable to delegate it.
Similar to commits 05272718f4 and 601cf5825f.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
As of previous commit, this is implied in a particular scenario. In
fact, this is the one and only scenario that justifies the use of -a.
Drop the option from the documentation. For backward compatibility, do
recognize it, and retain the feature of ignoring the "container is
stopped" error when set.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is roughly the same as TestPIDHostInitProcessWait in libct/int,
except that here we use separate processes to create and to kill a
container, so the processes inside a container are not children of "runc kill", and
also we hit different codepaths (nonChildProcess.signal rather than
initProcess.signal).
One other thing is, rootless is also tested.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It seems that set -x was temporarily added as a debug measure, but
slipped into the final commit.
Remove it, for the sake of test logs brevity.
Fixes: 9f656dbb11
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This test is occasionally failing on CS9.
The test case always takes about 7 seconds on my laptop (decreasing
memory, using a different memory eater in shell etc. doesn't help).
Increase the number of iterations from 10 to 30 to make sure we don't
see any flakes.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Apparently, bash with set -e deliberately ignores non-zero return codes
from ! cmd, unless this is the last command. The workaround is to either
use "! cmd || false', "or run ! cmd". Choose the latter, and require
bash-core 1.5.0 (since this is when "run !" was added), replacing the
older check.
Alas I only learned this recently from the bash-core documentation.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Systemd v252 (available in CentOS Stream 9 in our CI) added support
for setting cpu.idle (see [1]). The way it works is:
- if CPUWeight == 0, cpu.idle is set to 1;
- if CPUWeight != 0, cpu.idle is set to 0.
This commit implements setting cpu.idle in systemd cgroup driver via a
unit property. In case CPUIdle is set to non-zero value, the driver sets
adds CPUWeight=0 property, which will result in systemd setting cpu.idle
to 1.
Unfortunately, there's no way to set cpu.idle to 0 without also changing
the CPUWeight value, so the driver doesn't do anything if CPUIdle is
explicitly set to 0. This case is handled by the fs driver which is
always used as a followup to setting systemd unit properties.
Also, handle cpu.idle set via unified map. In case it is set to non-zero
value, add CPUWeight=0 property, and ignore cpu.weight (otherwise we'll
get two different CPUWeight properties set).
Add a unit test for new values in unified map, and an integration test case.
[1] https://github.com/systemd/systemd/pull/23299
[2] https://github.com/opencontainers/runc/issues/3786
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Values other than 1 or 0 are ignored by the kernel,
see sched_group_set_idle() in kernel/sched/fair.c
If the added test case ever fails, it means that the kernel now accepts
values other than 0 or 1, and runc needs to adopt to that.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Systemd v252 (available in CentOS Stream 9 in our CI) added support
for setting cpu.idle (see [1]). The way it works is:
- if CPUWeight == 0, cpu.idle is set to 1;
- if CPUWeight != 0, cpu.idle is set to 0.
This behavior breaks the existing test case, as described in [2].
To fix, skip the last check in the test case in case a newer systemd
is used.
Fixes: #3786
[1] https://github.com/systemd/systemd/pull/23299
[2] https://github.com/opencontainers/runc/issues/3786
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Commit d08bc0c1b3 ("runc run: warn on non-empty cgroup") introduced
a warning when a container is started in a non-empty cgroup. Such
configuration has lots of issues.
In addition to that, such configuration is not possible at all when
using the systemd cgroup driver.
As planned, let's promote this warning to an error, and fix the test
case accordingly.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This fixes a bogus failure in "ro cgroup" test cases when running as
rootless.
The test finds the following mount that is not read-only:
> cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0
This happens because:
1. runc spec --rootless adds an rbind /sys mounts, so we have all the
/sys/fs/cgroup/XXX mounts inside the container;
2. Those /sys/fs/cgroup/XXX mounts are shadowed by the /sys/fs/cgroup
tmpfs mount created by mountCgroupV1().
This means that this mount is shadowed, inaccessible, and it can not be
unshadowed, thus it should not be checked.
The fix is to check whether the directory exists, to exclude such
shadowed mounts.
NOTE that item 2 comes from commit ff692f289b60e19b3079cb; before it, we
had the whole hierarchy of host /sys/fs/cgroup visible (though not
writable -- because rootless) from inside of any rootless container.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Fix the following issues with the "ro /sys/fs/cgroup" test:
1. Disable bogus SC2016 warning from shellcheck.
2. Split the test into two -- with and without cgroupns. This is done
because not all systems have cgroupns available (so the "+cgroupns"
test will be skipped).
3. This splitting resulted in a few more bogus shellcheck warnings that
we have to suppress -- due to a known bug in shellcheck (see [1]).
4. s/mount/mounts/ in the test name, because in case of cgroup v1 there
are multiple mounts.
[1] https://github.com/koalaman/shellcheck/issues/2431
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is a test case for issue reported as #3715. In short, even if a
(non-root) user that the container is run as does not have execute
permission bit set for the executable, it should still work in case runc
has the CAP_DAC_OVERRIDE capability set.
Note that since the upstream golang is also broken (see [1]), this test
will fail for Go 1.20 and 1.20.1 (fix is in Go 1.20.2 as per [2]).
[1] https://go.dev/issue/58552
[2] https://go-review.googlesource.com/c/go/+/469956
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Commit 3291d66b98 introduced a check for /proc and /sys, making sure
the destination (dest) is a directory (and not e.g. a symlink).
Later, a hunk from commit 0ca91f44f switched from using filepath.Join
to SecureJoin for dest. As SecureJoin follows and resolves symlinks,
the check whether dest is a symlink no longer works.
To fix, do the check without/before using SecureJoin.
Add integration tests to make sure we won't regress.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It was found that rootless runc makes `/sys/fs/cgroup` writable in following conditons:
1. when runc is executed inside the user namespace, and the config.json does not specify the cgroup namespace to be unshared
(e.g.., `(docker|podman|nerdctl) run --cgroupns=host`, with Rootless Docker/Podman/nerdctl)
2. or, when runc is executed outside the user namespace, and `/sys` is mounted with `rbind, ro`
(e.g., `runc spec --rootless`; this condition is very rare)
A container may gain the write access to user-owned cgroup hierarchy `/sys/fs/cgroup/user.slice/...` on the host.
Other users's cgroup hierarchies are not affected.
To fix the issue, this commit does:
1. Remount `/sys/fs/cgroup` to apply `MS_RDONLY` when it is being bind-mounted
2. Mask `/sys/fs/cgroup` when the bind source is unavailable
Fix CVE-2023-25809 (GHSA-m8cg-xc2p-r3fc)
Co-authored-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
A couple of test cases in delete.bats check that a particular cgroup
exists (or doesn't exist) using find. This is now resulting in errors
like these:
find: ‘/sys/fs/cgroup/blkio/azsec’: Permission denied
find: ‘/sys/fs/cgroup/blkio/azsec_clamav’: Permission denied
find: ‘/sys/fs/cgroup/cpu,cpuacct/azsec’: Permission denied
find: ‘/sys/fs/cgroup/cpu,cpuacct/azsec_clamav’: Permission denied
find: ‘/sys/fs/cgroup/memory/azsec’: Permission denied
find: ‘/sys/fs/cgroup/memory/azsec_clamav’: Permission denied
leading to test case failures.
Apparently, GHA runs something else on a test box, so we get this.
To fix, ignore non-zero exit code from find, and redirect its stderr
to /dev/null.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
When doing a lazy checkpoint/restore, we should not restore into the
same cgroup, otherwise there is a race which result in occasional
killing of the restored container (GH #2760, #2924).
The fix is to use --manage-cgroup-mode=ignore, which allows to restore
into a different cgroup.
Note that since cgroupsPath is not set in config.json, the cgroup is
derived from the container name, so calling set_cgroups_path is not
needed.
For the previous (unsuccessful) attempt to fix this, as well as detailed
(and apparently correct) analysis, see commit 36fe3cc28c.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This test checks that the container is restored into a different cgroup.
To do so, a user should
- use --manage-cgroups-mode ignore on both checkpoint and restore;
- change the cgroupsPath value in config.json before restoring.
The test does some checks to ensure that its logic is correct, and that
after the restore the old (original) cgroup does not exist, the new one
exists, and the container's init is in that new cgroup.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This test (initially added by commit 58ea21daef and later amended in
commit 26dc55ef1a) currently has two major deficiencies:
1. All possible flag combinations, and their respective numeric values,
have to be explicitly listed. Currently we support 3 flags, so
there is only 2^3 - 1 = 7 combinations, but adding more flags will
become increasingly difficult (for example, 5 flags will result in
31 combinations).
2. The test requires kernel 4.17 (for SECCOMP_FILTER_FLAG_SPEC_ALLOW),
and not doing any tests when running on an older kernel. This, too,
will make it more difficult to add extra flags in the future.
Both issues can be solved by using runc features which now prints all
known and supported runc flags. We still have to hardcode the numeric
values of all flags, but most of the other work is coded now.
In particular:
* The test only uses supported flags, meaning it can be used with
older kernels, removing the limitation (2) above.
* The test calculates the powerset (all possible combinations) of
flags and their numeric values. This makes it easier to add more
flags, removing the limitation (1) above.
* The test will fail (in flags_value) if any new flags will be added
to runc but the test itself is not amended.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
If no seccomps flags are set in OCI runtime spec (not even the empty
set), set SPEC_ALLOW as the default (if it's supported).
Otherwise, use the flags as they are set (that includes no flags for
empty seccomp.Flags array).
This mimics the crun behavior, and makes runc seccomp performance on par
with crun.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is aimed at solving the problem of cgroup v2 memory controller
behavior which is not compatible with that of cgroup v1.
In cgroup v1, if the new memory limit being set is lower than the
current usage, setting the new limit fails.
In cgroup v2, same operation succeeds, and the container is OOM killed.
Introduce a new setting, memory.checkBeforeUpdate, and use it to mimic
cgroup v1 behavior.
Note that this is not 100% reliable because of TOCTOU, but this is the
best we can do.
Add some test cases.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
On ARM, mkdirat(2) is used instead of mkdir(2), thus the seccomp rules
needs to be amended accordingly.
This is a change similar to one in commit e119db7a23, but but it
evaded the test case added in commit 58ea21dae as it took a long time to
merge, and we don't have ARM CI.
Fixes: 58ea21dae ("seccomp: add support for flags")
Reported-by: Ryan Phillips <rphillips@redhat.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Add a test case for an issue fixed by the previous commit.
The env should has more than 8 core CPU to meet the test requirement.
Signed-off-by: Chengen, Du <chengen.du@canonical.com>
We use awk in other 9 or so places, and here it's gawk.
Since this is on Linux, most probably awk is gawk.
So s/gawk/awk/.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The situation when /sys/fs/cgroup/unified is not present normal and
should not result in anything on stderr. Suppress it.
Fixes: cc15b887a0
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Add a debug print of seccomp flags value, so the test can check
those (without using something like strace, that is).
Amend the flags setting test with the numeric values expected, and the
logic to check those.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
A regression reported for runc v1.1.3 says that "runc exec -t" fails
after doing "systemctl daemon-reload":
> exec failed: unable to start container process: open /dev/pts/0: operation not permitted: unknown
Apparently, with commit 7219387eb7 we are no longer adding
"DeviceAllow=char-pts rwm" rule (because os.Stat("char-pts") returns
ENOENT).
The bug can only be seen after "systemctl daemon-reload" because runc
also applies the same rules manually (by writing to devices.allow for
cgroup v1), and apparently reloading systemd leads to re-applying the
rules that systemd has (thus removing the char-pts access).
The fix is to do os.Stat only for "/dev" paths.
Also, emit a warning that the path was skipped. Since the original idea
was to emit less warnings, demote the level to debug.
Note this also fixes the issue of not adding "m" permission for block-*
and char-* devices.
A test case is added, which reliably fails before the fix
on both cgroup v1 and v2.
Fixes: https://github.com/opencontainers/runc/issues/3551
Fixes: 7219387eb7
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Recently released codespell 2.2 adds some more false positives,
such as:
./Makefile:78: ro ==> to, row, rob, rod, roe, rot
./Makefile:88: ro ==> to, row, rob, rod, roe, rot
./notify_socket.go:51: ro ==> to, row, rob, rod, roe, rot
./LICENSE:128: complies ==> compiles
./go.sum:59: BU ==> BY
./types/features/features.go:17: ro ==> to, row, rob, rod, roe, rot
./libcontainer/rootfs_linux.go:52: ro ==> to, row, rob, rod, roe, rot
./libcontainer/rootfs_linux.go:166: ro ==> to, row, rob, rod, roe, rot
....
./tests/integration/cgroup_delegation.bats:38: inh ==> in
...
To fix:
- exclude go.sum;
- add ro and complies to the list of ignored words;
- s/inh/inherit in cgroup_delegation.bats.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Kir Kolyshkin