Containerd pre-creates userns and netns before calling runc, which
results in the current code not working when SELinux is enabled,
resulting in the following error:
> runc create failed: unable to start container process: error during
container init: error mounting "mqueue" to rootfs at "/dev/mqueue":
setxattr /path/to/rootfs/dev/mqueue: operation not permitted
The solution is to become root in the user namespace right after
we join it.
Fixes#4466.
Co-authored-by: Wei Fu <fuweid89@gmail.com>
Co-authored-by: Kir Kolyshkin <kolyshkin@gmail.com>
Co-authored-by: Aleksa Sarai <cyphar@cyphar.com>
Signed-off-by: lifubang <lifubang@acmcoder.com>
Since Go 1.19, the same functionality is there in os/exec package.
As we require go 1.22 now, there's no need to have this.
This basically reverts commit 9258eac0 ("libct/start: use execabs for
newuidmap lookup").
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This reverts commit 65a1074c75.
We needed [1] because when we removed the bindfd logic in [2] we had not
yet moved the binary cloning logic to Go and thus it was necessary to
increase the memory limit in CI because the clone was happening after
joining the cgroup. However, [3] finally moved that code to Go and thus
the cloning is now done outside of the container's cgroup and thus is no
longer accounted as part of the container's memory usage at any point.
Now we can properly support running a simple container with lower memory
usage as we did before.
[1]: commit 65a1074c75 ("increase memory.max in cgroups.bats")
[2]: commit b999376fb2 ("nsenter: cloned_binary: remove bindfd logic entirely")
[3]: commit 0e9a3358f8 ("nsexec: migrate memfd /proc/self/exe logic to Go code")
Signed-off-by: lfbzhm <lifubang@acmcoder.com>
[cyphar: fixed commit messages]
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
There is a race situation when we are opening a file, if there is a
small fd was closed at that time, maybe it will be reused by safeExe.
Because of Go stdlib fds shuffling bug, if the fd of safeExe is too
small, go stdlib will dup3 it to another fd, or dup3 a other fd to this
fd, then it will cause the fd type cmd.Path refers to a random path,
and it can lead to an error "permission denied" when starting the process.
Please see #4294 and <https://github.com/golang/go/issues/61751>.
So we should not use the original fd of safeExe, but use the fd after
shuffled by Go stdlib. Because Go stdlib will guarantee this fd refers to
the correct file.
Signed-off-by: lfbzhm <lifubang@acmcoder.com>
Commit b999376fb2 ("nsenter: cloned_binary: remove bindfd logic
entirely") removed the read-only bind-mount logic from our cloned binary
code because it wasn't really safe because a container with
CAP_SYS_ADMIN could remove the MS_RDONLY bit and get write access to
/proc/self/exe (even with user namespaces this could've been an issue
because it's not clear if the flags are locked).
However, copying a binary does seem to have a minor performance impact.
The only way to have no performance impact would be for the kernel to
block these write attempts, but barring that we could try to reduce the
overhead by coming up with a mount that cannot have it's read-only bits
cleared.
The "simplest" solution is to create a temporary overlayfs using
fsopen(2) which uses the directory where runc exists as a lowerdir,
ensuring that the container cannot access the underlying file -- and we
don't have to do any copies.
While fsopen(2) is not free because mount namespace cloning is usually
expensive (and so it seems like the difference would be marginal), some
basic performance testing seems to indicate there is a ~60% improvement
doing it this way and that it has effectively no overhead even when
compared to just using /proc/self/exe directly:
% hyperfine --warmup 50 \
> "./runc-noclone run -b bundle ctr" \
> "./runc-overlayfs run -b bundle ctr" \
> "./runc-memfd run -b bundle ctr"
Benchmark 1: ./runc-noclone run -b bundle ctr
Time (mean ± σ): 13.7 ms ± 0.9 ms [User: 6.0 ms, System: 10.9 ms]
Range (min … max): 11.3 ms … 16.1 ms 184 runs
Benchmark 2: ./runc-overlayfs run -b bundle ctr
Time (mean ± σ): 13.9 ms ± 0.9 ms [User: 6.2 ms, System: 10.8 ms]
Range (min … max): 11.8 ms … 16.0 ms 180 runs
Benchmark 3: ./runc-memfd run -b bundle ctr
Time (mean ± σ): 22.6 ms ± 1.3 ms [User: 5.7 ms, System: 20.7 ms]
Range (min … max): 19.9 ms … 26.5 ms 114 runs
Summary
./runc-noclone run -b bundle ctr ran
1.01 ± 0.09 times faster than ./runc-overlayfs run -b bundle ctr
1.65 ± 0.15 times faster than ./runc-memfd run -b bundle ctr
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
This test case is frequently hanging recently. Might be caused
by a recent kernel update from 5.14.0-427.33.1.el9_4.x86_64 to
5.14.0-427.37.1.el9_4.x86_64.
Could not reproduce locally.
Let's skip it for now.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The issue quoted is now fixed, so add some information about the fixed
kernel version, and remove links to older discussions about idmapped
mounts security.
We can actually remove all of it for now, but let's keep it. Change
the skip message to say which kernel is required.
Amends commit b460dc39.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This initWaiter logic was introduced by commit 4ecff8d9, but since the logic of
/proc/self/exe was moved out of runc init in commit 0e9a335, this
seems unnecessary to have initWaiter.
Remove it.
This essentially reverts commit 4ecff8d9.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
While we did set +x when "sealing" regular temporary files, the "is
executable" checks were done before then and would thus fail, causing
the fallback to not work properly.
So just set +x after we create the file. We already have a O_RDWR handle
open when we do the chmod so we won't get permission issues when writing
to the file.
Fixes: e089db3b4a ("dmz: add fallbacks to handle noexec for O_TMPFILE and mktemp()")
Signed-off-by: lifubang <lifubang@acmcoder.com>
We are not really interested in the capabilities of the current process,
so there is no need to load those.
This results in some performance improvement since now the capability
package don't have to parse /proc/self/status.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The userns package was moved to the moby/sys/userns module
at commit 3778ae603c.
This patch deprecates the old location, and adds it as an alias
for the moby/sys/userns package.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Let's point to the relevant README directly in the systemd unit file,
as it is hard to find in the whole nine yards of the runc repo.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The example of starting memfd-bind via systemd in README did not work
for me (Fedora 40, systemd 255):
# systemctl status memfd-bind@/usr/bin/runc
Invalid unit name "memfd-bind@/usr/bin/runc" escaped as "memfd-bind@-usr-bin-runc" (maybe you should use systemd-escape?).
○ memfd-bind@-usr-bin-runc.service
Loaded: bad-setting (Reason: Unit memfd-bind@-usr-bin-runc.service has a bad unit file setting.)
Active: inactive (dead)
Docs: https://github.com/opencontainers/runc
So, let's use systemd-escape -p ("path") in the README example,
and use %f in the systemd unit file to prepend the slash to the
filename.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
When preparing to mount container root, we need to make its parent mount
private (i.e. disable propagation), otherwise the new in-container
mounts are leaked to the host.
To find a parent mount, we use to read mountinfo and find the longest
entry which can be a parent of the container root directory.
Unfortunately, due to kernel bug in all Linux kernels older than v5.8
(see [1], [2]), sometimes mountinfo can't be read in its entirety. In
this case, getParentMount may occasionally return a wrong parent mount.
As a result, we do not change the mount propagation to private, and
container mounts are leaked.
Alas, we can not fix the kernel, and reading mountinfo a few times to
ensure its consistency (like it's done in, say, Kubernetes) does not
look like a good solution for performance reasons.
Fortunately, we don't need mountinfo. Let's just traverse the directory
tree, trying to remount it private until we find a mount point (any
error other than EINVAL means we just found it).
Fixes issue 2404.
[1]: https://github.com/kolyshkin/procfs-test
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9f6c61f96f2d97cbb5f
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
lifubang