This update includes a few breaking API changes that I needed to get in
before an actual runc release depends on it, so that we don't need to
deal with compatibility shims for them (or bumping the SOVERSION).
From a Go API perspective, there were no major changes -- though this
bump did also require a bump to github.com/cyphar/filepath-securejoin
because one of the wrapped APIs changed from int to uint64 as a flag
argument type. Again, better to get this done before we really depend on
this in a public way.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
(Based on commit d47bf88349de01e444be8f78ab8c96dae7020b75.)
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
Found out that these changes were backported to release-1.4 (PR 5040)
and made its way into runc v1.4.0, but were missing from its CHANGELOG.
Add the item to v1.4.0 changelog.
Same as commit e232a54 in main branch.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The intention of commit 531e29e192 ("script/lib.sh: set GOARM=5 for
armel, GOARM=6 for armhf") was to properly support older ARM platforms
with our release builds.
However, we have never been able to support ARMv6 for our builds because
we use the Debian compiler to build the libseccomp we statically compile
into our binaries and (as per the now-deleted comment itself) Debian
treats armhf as being ARMv7 so the final binaries we produced were
always only ever compatible with ARMv7+.
This was a bit of an oddity before but when building libpathrs for
releases we will need to use Rust which makes the target more explicit
(and while it does support armhf, we are using the Debian-packaged Rust
cross-compiler and thus are in the same dilemma with what Debian
considers "armhf" to be).
All-in-all, it's better to just bite the bullet and just follow Debian
here properly.
Fixes: 531e29e192 ("script/lib.sh: set GOARM=5 for armel, GOARM=6 for armhf")
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
(cherry picked from commit 51ae8de054)
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
The main update is actually in github.com/opencontainers/cgroups, but we
need to also update runtime-spec to a newer pre-release version to get
the updates from there as well.
In short, the behaviour change is now that "0" is treated as a valid
value to set in "pids.max", "-1" means "max" and unset/nil means "do
nothing". As described in the opencontainers/cgroups PR, this change is
actually backwards compatible because our internal state.json stores
PidsLimit, and that entry is marked as "omitempty". So, an old runc
would omit PidsLimit=0 in state.json, and this will be parsed by a new
runc as being "nil" -- and both would treat this case as "do not set
anything".
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
(cherry picked from commit 3b75374cc7)
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
Now that we've updated to golang.org/x/sys@v0.37.0, CPUSet has a Fill
helper that does the equivalent to our underflow trick to make setting
all CPUs efficient.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
(cherry picked from commit 93f9a392cf)
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
This was a notable change in v1.4.0-rc.1 but this was not sufficiently
well-signposted in our changelog.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
This package was marked deprecated in commit 9b60a93cf3
("libcontainer/userns: migrate to github.com/moby/sys/userns"), which
was included in runc 1.2. Users have thus had a year to migrate to
github.com/moby/sys/userns and it's okay for us to remove this wrapper
package.
(Cherry-pick of commit e4f99b5c95b8f49434452edff82e73547c7a8252.)
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
libcontainer/user was marked as deprecated in d9ea71bf96 ("deprecate
libcontainer/user") and users have had plenty of time to migrate to
github.com/moby/sys/user.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
In certain deployments, it's possible for runc to be spawned by a
process with a restrictive cpumask (such as from a systemd unit with
CPUAffinity=... configured) which will be inherited by runc and thus the
container process by default.
The cpuset cgroup used to reconfigure the cpumask automatically for
joining processes, but kcommit da019032819a ("sched: Enforce user
requested affinity") changed this behaviour in Linux 6.2.
The solution is to try to emulate the expected behaviour by resetting
our cpumask to correspond with the configured cpuset (in the case of
"runc exec", if the user did not configure an alternative one). Normally
we would have to parse /proc/stat and /sys/fs/cgroup, but luckily
sched_setaffinity(2) will transparently convert an all-set cpumask (even
if it has more entries than the number of CPUs on the system) to the
correct value for our usecase.
For some reason, in our CI it seems that rootless --systemd-cgroup
results in the cpuset (presumably temporarily?) being configured such
that sched_setaffinity(2) will allow the full set of CPUs. For this
particular case, all we care about is that it is different to the
original set, so include some special-casing (but we should probably
investigate this further...).
Reported-by: ningmingxiao <ning.mingxiao@zte.com.cn>
Reported-by: Martin Sivak <msivak@redhat.com>
Reported-by: Peter Hunt <pehunt@redhat.com>
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
As per
- https://github.com/opencontainers/runtime-spec/pull/1253
- https://github.com/opencontainers/runtime-spec/pull/1261
CPU affinity can be set in two ways:
1. When creating/starting a container, in config.json's
Process.ExecCPUAffinity, which is when applied to all execs.
2. When running an exec, in process.json's CPUAffinity, which
applied to a given exec and overrides the value from (1).
Add some basic tests.
Note that older kernels (RHEL8, Ubuntu 20.04) change CPU affinity of a
process to that of a container's cgroup, as soon as it is moved to that
cgroup, while newer kernels (Ubuntu 24.04, Fedora 41) don't do that.
Because of the above,
- it's impossible to really test initial CPU affinity without adding
debug logging to libcontainer/nsenter;
- for older kernels, there can be a brief moment when exec's affinity
is different than either initial or final affinity being set;
- exec's final CPU affinity, if not specified, can be different
depending on the kernel, therefore we don't test it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Commit 770728e1 added Scheduler field into both Config and Process,
but forgot to add a mechanism to actually use Process.Scheduler.
As a result, runc exec does not set Process.Scheduler ever.
Fix it, and a test case (which fails before the fix).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Commit bfbd0305b added IOPriority field into both Config and Process,
but forgot to add a mechanism to actually use Process.IOPriority.
As a result, runc exec does not set Process.IOPriority ever.
Fix it, and a test case (which fails before the fix).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This addresses the following TODO in the code (added back in 2015
by commit 845fc65e5):
> // TODO: fix libcontainer's API to better support uid/gid in a typesafe way.
Historically, libcontainer internally uses strings for user, group, and
additional (aka supplementary) groups.
Yet, runc receives those credentials as part of runtime-spec's process,
which uses integers for all of them (see [1], [2]).
What happens next is:
1. runc start/run/exec converts those credentials to strings (a User
string containing "UID:GID", and a []string for additional GIDs) and
passes those onto runc init.
2. runc init converts them back to int, in the most complicated way
possible (parsing container's /etc/passwd and /etc/group).
All this conversion and, especially, parsing is totally unnecessary,
but is performed on every container exec (and start).
The only benefit of all this is, a libcontainer user could use user and
group names instead of numeric IDs (but runc itself is not using this
feature, and we don't know if there are any other users of this).
Let's remove this back and forth translation, hopefully increasing
runc exec performance.
The only remaining need to parse /etc/passwd is to set HOME environment
variable for a specified UID, in case $HOME is not explicitly set in
process.Env. This can now be done right in prepareEnv, which simplifies
the code flow a lot. Alas, we can not use standard os/user.LookupId, as
it could cache host's /etc/passwd or the current user (even with the
osusergo tag).
PS Note that the structures being changed (initConfig and Process) are
never saved to disk as JSON by runc, so there is no compatibility issue
for runc users.
Still, this is a breaking change in libcontainer, but we never promised
that libcontainer API will be stable (and there's a special package
that can handle it -- github.com/moby/sys/user). Reflect this in
CHANGELOG.
For 3998.
[1]: https://github.com/opencontainers/runtime-spec/blob/v1.0.2/config.md#posix-platform-user
[2]: https://github.com/opencontainers/runtime-spec/blob/v1.0.2/specs-go/config.go#L86
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Make CommandHook.Command a pointer, which reduces the amount of data
being copied when using hooks, and allows to modify command hooks.
2. Add SetDefaultEnv, which is to be used by the next commit.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Remove changes that are already reflected in v1.1.13 changelog:
- rlimit_nofile fix;
- rt_period vs rt_runtime fix;
- gpg vs keyboxd fix;
- nsexec debug log fix;
- fips faking;
- vagrant Fedora 39 bump;
- golangci-lint bump;
- x/net bump;
- centos stream 8 removal;
- codespell ci fixes.
Compact some of the entries that are related (e.g. about actuated-ci).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Commit b6967fa84c moved the functionality of managing cgroup devices
into a separate package, and decoupled libcontainer/cgroups from it.
Yet, some software (e.g. cadvisor) may need to use libcontainer package,
which imports libcontainer/cgroups/devices, thus making it impossible to
use libcontainer without bringing in cgroup/devices dependency.
In fact, we only need to manage devices in runc binary, so move the
import to main.go.
The need to import libct/cg/dev in order to manage devices is already
documented in libcontainer/cgroups, but let's
- update that documentation;
- add a similar note to libcontainer/cgroups/systemd;
- add a note to libct README.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
[ cyphar: restructuring and removal of outdated or incorrect info ]
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
We will almost certainly need to eventually rework nsenter to:
1. Figure out a way to make pthread_self() not break after nsenter runs
(probably not possible, because the core issue is likely that we are
ignoring the rules of signal-safety(7)); or
2. Do an other re-exec of /proc/self/exe to execute the Go half of
"runc init" -- after we've done the nsenter setup. This would reset
all of the process state and ensure we have a clean glibc state for
Go, but it would make runc slower...
For now, just block Go 1.22 builds to avoid having broken runcs floating
around until we resolve the issue. It seems possible for musl to also
have an issue, but it appears to work and so for now just block glibc
builds.
Note that this will only block builds for anything that uses nsenter --
so users of our (internal) libcontainer libraries should be fine. Only
users that are starting containers using nsenter to actually start
containers will see the error (which is precisely what we want).
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
This field reports swap-only usage. For cgroupv1, `Usage` and `Failcnt`
are set by subtracting memory usage from memory+swap usage. For cgroupv2,
`Usage`, `Limit`, and `MaxUsage` are set. This commit also export `MaxUsage`
of memory under cgroupv2 mode, using `memory.peak` introduced in kernel 5.19.
Signed-off-by: Heran Yang <heran55@126.com>
This aligns v2 usage calculations more closely with v1.
Current node-level reporting for v1 vs v2 on the same
machine under similar load may differ by ~250-750Mi.
Also return usage as combined swap + memory usage, aligned
with v1 and non-root v2 cgroups.
`mem_cgroup_usage` in the kernel counts NR_FILE_PAGES
+ NR_ANON_MAPPED + `nr_swap_pages` (if swap enabled) [^0].
Using total - free results in higher "usage" numbers.
This is likely due to various types of reclaimable
memory technically counted as in use (e.g. inactive anon).
See also https://github.com/kubernetes/kubernetes/issues/118916 for more context
[^0]: https://github.com/torvalds/linux/blob/06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5/mm/memcontrol.c#L3673-L3680
Signed-off-by: Alexander Eldeib <alexeldeib@gmail.com>
Aleksa Sarai