We recently switched VERSION to be read from env vars (#4270). This
broke several projects, as they were building runc and using a `VERSION`
env var for, e.g. the containerd version.
When fixing that in #4370, we discussed to consider doing the same for
these variables too
(https://github.com/opencontainers/runc/pull/4370#pullrequestreview-2240030944).
Let's stop reading them from env vars, as it is very easy to do it by
mistake (e.g. compile runc and define a COMMIT env var, not to override
the commit shown in `runc --version`) and users that want can still
override them if they want to. For example, with:
make EXTRA_BUILDTAGS=runc_nodmz
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
Add this new make variable so users can specify build information
without modifying the runc version nor the source code.
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
The following commands are moved from `contrib/cmd` to `tests/cmd`:
- fs-idmap
- pidfd-kill
- recvtty
- remap-rootfs
- sd-helper
- seccompagent
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
This reverts commit 9d9273c926.
This commit broke the build for several other projects (see comments
here: https://github.com/opencontainers/runc/pull/4270, after the merge)
and we don't really need this to be able to set the version without
changing the file.
With this commit reverted, we can still run:
make VERSION="1.2.3"
and it just works. It doesn't take it from an env variable, but that is
what broke all the other projects (VERSION is just too generic as an env
var, especially for a project like runc that is embedded in many
others).
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
* Simple error correction of a spelling mistake which was
introduced at commit b8f75f3
Signed-off-by: Sjoerd van Leent <sjoerd.van.leent@alliander.com>
Previously, all of our userns tests worked around the remapping issue by
creating the paths that runc would attempt to create (like /proc).
However, this isn't really accurate to how real userns containers are
created, so it's much better to actually remap the rootfs.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
The container manager like containerd-shim can't use cgroup.kill feature or
freeze all the processes in cgroup to terminate the exec init process.
It's unsafe to call kill(2) since the pid can be recycled. It's good to
provide the pidfd of init process through the pidfd-socket. It's similar to
the console-socket. With the pidfd, the container manager like containerd-shim
can send the signal to target process safely.
And for the standard init process, we can have polling support to get
exit event instead of blocking on wait4.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
All the targets in the Makefile we have are phony (as we mostly rely on
go to figure out dependencies and whether to rebuild something), and
they have to be marked as such. We do that at the end of the file, and
the list is pretty long.
Instead, let's just add .PHONY before each target. That way it is easier
to spot any omissions.
Alternative solutions:
- add ".PHONY: %"; it won't work as wildcards are not recongized in
this context;
- add "MAKEFLAGS += --always-make".
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Every `make` now produces something like this:
make[1]: Entering directory '/home/kir/go/src/github.com/opencontainers/runc'
readelf -h runc
Machine: Advanced Micro Devices X86-64
Flags: 0x0
readelf -h libcontainer/dmz/runc-dmz
Machine: Advanced Micro Devices X86-64
Flags: 0x0
runc-dmz architecture matches runc binary.
make[1]: Leaving directory '/home/kir/go/src/github.com/opencontainers/runc'
That is a bit too much. Let's make it less verbose.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This really isn't ideal but it can be used to avoid the largest issues
with the memfd-based runc binary protection. There are several caveats
with using this tool, see the help page for the new binary for details.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
The idea is to remove the need for cloning the entire runc binary by
replacing the final execve() call of the container process with an
execve() call to a clone of a small C binary which just does an execve()
of its arguments.
This provides similar protection against CVE-2019-5736 but without
requiring a >10MB binary copy for each "runc init". When compiled with
musl, runc-dmz is 13kB (though unfortunately with glibc, it is 1.1MB
which is still quite large).
It should be noted that there is still a window where the container
processes could get access to the host runc binary, but because we set
ourselves as non-dumpable the container would need CAP_SYS_PTRACE (which
is not enabled by default in Docker) in order to get around the
proc_fd_access_allowed() checks. In addition, since Linux 4.10[1] the
kernel blocks access entirely for user namespaced containers in this
scenario. For those cases we cannot use runc-dmz, but most containers
won't have this issue.
This new runc-dmz binary can be opted out of at compile time by setting
the "runc_nodmz" buildtag, and at runtime by setting the RUNC_DMZ=legacy
environment variable. In both cases, runc will fall back to the classic
/proc/self/exe-based cloning trick. If /proc/self/exe is already a
sealed memfd (namely if the user is using contrib/cmd/memfd-bind to
create a persistent sealed memfd for runc), neither runc-dmz nor
/proc/self/exe cloning will be used because they are not necessary.
[1]: https://github.com/torvalds/linux/commit/bfedb589252c01fa505ac9f6f2a3d5d68d707ef4
Co-authored-by: lifubang <lifubang@acmcoder.com>
Signed-off-by: lifubang <lifubang@acmcoder.com>
[cyphar: address various review nits]
[cyphar: fix runc-dmz cross-compilation]
[cyphar: embed runc-dmz into runc binary and clone in Go code]
[cyphar: make runc-dmz optional, with fallback to /proc/self/exe cloning]
[cyphar: do not use runc-dmz when the container has certain privs]
Co-authored-by: Aleksa Sarai <cyphar@cyphar.com>
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
We need these to match the Makefile detection of the right gcc for
runc-dmz, as well as making sure that everything builds properly for our
cross-i386 tests. While we're at it, add x86 to the list of build
targets for release builds (presumably nobody will use it, but since we
do test builds of this anyway it probably won't hurt).
In addition, clean up the handling of the native architecture build by
treating it the same as any other build (ensuring that building runc
from a different platform will work the same way regardless of the
native architecture). In practice, the build works the same way as
before.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
These checks ensure that all of the keys in the runc.keyring list are
actually the keys of the specified user and that the users themselves
are actually maintainers.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
... as a way to maybe catch some CHANGELOG.md bugs at the last moment.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 54cfb25d69)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Previously (see commit 91fa032da4) we found a few issues
using this check, but apparently the CHANGELOG.md is in UTF-8, and
the recently added quote is breaking this, so remove.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 7b3ac330f7)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Bump shfmt to v3.5.1. Release notes:
https://github.com/mvdan/sh/releases
2. Since shfmt v3.5.0, specifying -l bash (or -l bats) is no longer
necessary. Therefore, we can use shfmt to find all the files.
Add .editorconfig to ignore vendor subdirectory.
3. Use shfmt docker image, so that we don't have to install anything
explicitly. This greatly simplifies the shfmt CI job. Add
localshfmt target so developers can still use a local shfmt binary
when necessary.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is a forward-port of commit 91fa032da4 ("ci: add basic checks for
CHANGELOG.md"), plus whatever changes were made in release-1.1 branch
(up to v1.1.3).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This removes the runc dependency on cpuguy83/md2man and
russross/blackfriday, which saves more than 400 KB (more than 300 KB
once stripped) from the binary.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It doesn't matter whether static or dynamic linking is used, runc
always needs libcontainer/nsenter, which is written in C and thus
requires cgo. Same is true for libcontainer/integration.
In addition, contrib/pkg/seccompagent also needs cgo (if seccomp build
tag is set), as it need to be linked against libseccomp C library.
By default, cgo is disabled when cross-compiling, meaning that
CGO_ENABLED=1 has to be set explicitly in such cases.
In all other cases (e.g. other contrib binaries) we do not need cgo.
Remove CGO_ENABLED=1 from GO_BUILD_STATIC (as it does not have anything
to do with static linking), and add it to all targets that require it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Set to empty value by default.
2. Assume Linux (remove GOOS check, since we do not support other OSes).
3. Instead of using a "not-supported" list, use a "supported" list
(as Go release notes usually say which platforms are supported).
As of today, -buildmode=pie is supported for:
* linux/386, linux/amd64, linux/arm, linux/arm64, and linux/ppc64le
(since Go 1.6, see https://tip.golang.org/doc/go1.6#compiler)
* linux/s390x (since Go 1.7, which adds the initial port)
* linux/riscv64 (since Go 1.16, see
https://tip.golang.org/doc/go1.16#riscv)
NOTE this does not mean we support these architectures; it is merely
a way to see if -buildmode=pie can be used.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
LDFLAGS_COMMON are used from two places, so it makes sense to dedup.
LDFLAGS_STATIC is a preparation for the next commit.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Now the only remaining file that needs shellcheck warnings to be fixed
is bash-completion. Note that in Makefile's TODO.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
... and add this file to shellcheck target in Makefile.
These:
In script/check-config.sh line 27:
kernelMinor="${kernelVersion#$kernelMajor.}"
^----------^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.
Did you mean:
kernelMinor="${kernelVersion#"$kernelMajor".}"
In script/check-config.sh line 103:
source /etc/os-release 2>/dev/null || /bin/true
^-------------^ SC1091 (info): Not following: /etc/os-release was not specified as input (see shellcheck -x).
In script/check-config.sh line 267:
NET_CLS_CGROUP $netprio
^------^ SC2206 (warning): Quote to prevent word splitting/globbing, or split robustly with mapfile or read -a.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
My GPG keys are not available inside the container, so it makes little
sense to try to sign the binaries inside the container's release.sh. The
solution is to split things into separate build and sign stages, with
signing ocurring after the in-Docker build.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
This implements cross-build for "make release", moving the build into a
container. This way we can support arm, arm64, ppc, and whatnot.
* script/seccomp.sh: separate out of script/release.sh, amend to support
cross-compile and save needed environment variables to a file.
* Dockerfile: add installing libseccomp from source, as this is needed
for release builds.
* script/release.sh: amend to support more architectures in addition to
the native build. Additional arches can be added by specifying
"-a <arch>" argument (can be specified multiple times), or
"make RELEASE_ARGS="-a arm64" release" if called via make.
All supported architectures can be enabled via "make releaseall".
* Makefile: move "release" target to "localrelease", add "release" and
"releaseall" targets to build via the Dockerfile. This is done because
most distros (including Fedora and openSUSE) lack cross-glibc, which is
needed to cross-compile libseccomp.
* Makefile: remove 'cross' and 'localcross' targets, as this is now done
by the release script.
* .github/workflows/validate.yum: amend the release CI job to cross-build
for supported architectures, remove cross job.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
There is no need to have a static version of recvtty and/or sd-helper
binary.
This speeds up script/release.sh a bit.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. The seccompagent target it built in the same way as others in contrib,
so there is no need to have a separate rule.
2. Mark seccompagent as phony, because it is (it rarely happens, but I
actually just had an issue because this was absent).
3. Add seccompagent binary to clean target.
Fixes: e21a9ee81
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Implement sample seccomp agent. It's also used in integration tests in
the following commit.
Instructions how to use it in contrib/cmd/seccompagent/README.md
Signed-off-by: Alban Crequy <alban@kinvolk.io>
Signed-off-by: Rodrigo Campos <rodrigo@kinvolk.io>
Co-authored-by: Rodrigo Campos <rodrigo@kinvolk.io>
Add a test case for an issue fixed by the previous commit.
Unfortunately, this is somewhat complicated as there's no easy way to
create a transient unit, so a binary, sd-helper, had to be added. On top
of that, an ability to create a parent/pod cgroup is added to
helpers.bash, which might be useful for future integration tests.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
... and add the file to be checked by shellcheck.
The warnings fixed are:
In tests/integration/helpers.bash line 10:
INTEGRATION_ROOT=$(dirname "$(readlink -f "$BASH_SOURCE")")
^----------^ SC2128: Expanding an array without an index only gives the first element.
In tests/integration/helpers.bash line 22:
TESTDATA="${INTEGRATION_ROOT}/testdata"
^------^ SC2034: TESTDATA appears unused. Verify use (or export if used externally).
In tests/integration/helpers.bash line 42:
echo "runc $@ (status=$status):" >&2
^-- SC2145: Argument mixes string and array. Use * or separate argument.
^-----^ SC2154: status is referenced but not assigned.
In tests/integration/helpers.bash line 43:
echo "$output" >&2
^-----^ SC2154: output is referenced but not assigned.
In tests/integration/helpers.bash line 77:
| .linux.gidMappings += [{"hostID": '"$(($ROOTLESS_GIDMAP_START + 10))"', "containerID": 1, "size": 20}]
^--------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
In tests/integration/helpers.bash line 78:
| .linux.gidMappings += [{"hostID": '"$(($ROOTLESS_GIDMAP_START + 100))"', "containerID": 1000, "size": '"$(($ROOTLESS_GIDMAP_LENGTH - 1000))"'}]'
^--------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
^---------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
In tests/integration/helpers.bash line 125:
base_path=$(gawk '$(NF-2) == "cgroup" && $NF ~ /\<'${g}'\>/ { print $5; exit }' /proc/self/mountinfo)
^--^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
base_path=$(gawk '$(NF-2) == "cgroup" && $NF ~ /\<'"${g}"'\>/ { print $5; exit }' /proc/self/mountinfo)
In tests/integration/helpers.bash line 127:
eval CGROUP_${g^^}_BASE_PATH="${base_path}"
^----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
eval CGROUP_"${g^^}"_BASE_PATH="${base_path}"
In tests/integration/helpers.bash line 229:
if [ "x$CGROUP_UNIFIED" = "xyes" ]; then
^----------------^ SC2268: Avoid x-prefix in comparisons as it no longer serves a purpose.
Did you mean:
if [ "$CGROUP_UNIFIED" = "yes" ]; then
In tests/integration/helpers.bash line 234:
eval cgroup=\$${var}${REL_CGROUPS_PATH}
^----^ SC2086: Double quote to prevent globbing and word splitting.
^-----------------^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
eval cgroup=\$"${var}""${REL_CGROUPS_PATH}"
In tests/integration/helpers.bash line 236:
cat $cgroup/$source
^-----^ SC2086: Double quote to prevent globbing and word splitting.
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
cat "$cgroup"/"$source"
In tests/integration/helpers.bash line 242:
current="$(get_cgroup_value $1)"
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
current="$(get_cgroup_value "$1")"
In tests/integration/helpers.bash line 245:
echo "current" $current "!?" "$expected"
^------^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
echo "current" "$current" "!?" "$expected"
In tests/integration/helpers.bash line 257:
[ $(id -u) != "0" ] && user="--user"
^------^ SC2046: Quote this to prevent word splitting.
In tests/integration/helpers.bash line 259:
current=$(systemctl show $user --property $source $SD_UNIT_NAME | awk -F= '{print $2}')
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
current=$(systemctl show $user --property "$source" $SD_UNIT_NAME | awk -F= '{print $2}')
In tests/integration/helpers.bash line 261:
[ "$current" = "$expected" ] || [ -n "$expected2" -a "$current" = "$expected2" ]
^-- SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
In tests/integration/helpers.bash line 309:
check_cgroup_value "cpu.weight" $weight
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
check_cgroup_value "cpu.weight" "$weight"
In tests/integration/helpers.bash line 310:
check_systemd_value "CPUWeight" $weight
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
check_systemd_value "CPUWeight" "$weight"
In tests/integration/helpers.bash line 383:
if [ $CGROUP_UNIFIED = "no" -a ! -e "${CGROUP_MEMORY_BASE_PATH}/memory.memsw.limit_in_bytes" ]; then
^-- SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
In tests/integration/helpers.bash line 412:
local cpu_count=$(grep -c '^processor' /proc/cpuinfo)
^-------^ SC2155: Declare and assign separately to avoid masking return values.
In tests/integration/helpers.bash line 450:
sleep $delay
^----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
sleep "$delay"
In tests/integration/helpers.bash line 453:
echo "Command \"$@\" failed $attempts times. Output: $output"
^-- SC2145: Argument mixes string and array. Use * or separate argument.
In tests/integration/helpers.bash line 471:
runc state $1
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
runc state "$1"
In tests/integration/helpers.bash line 472:
if [ $2 == "checkpointed" ]; then
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
if [ "$2" == "checkpointed" ]; then
In tests/integration/helpers.bash line 484:
mkdir $dir
^--^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
mkdir "$dir"
In tests/integration/helpers.bash line 497:
kill -9 $(cat "$dir/pid")
^---------------^ SC2046: Quote this to prevent word splitting.
In tests/integration/helpers.bash line 508:
export ROOT=$(mktemp -d "$BATS_RUN_TMPDIR/runc.XXXXXX")
^--^ SC2155: Declare and assign separately to avoid masking return values.
In tests/integration/helpers.bash line 512:
cd "$ROOT/bundle"
^---------------^ SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Did you mean:
cd "$ROOT/bundle" || exit
In tests/integration/helpers.bash line 535:
cd "$INTEGRATION_ROOT"
^--------------------^ SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Did you mean:
cd "$INTEGRATION_ROOT" || exit
For more information:
https://www.shellcheck.net/wiki/SC2145 -- Argument mixes string and array. ...
https://www.shellcheck.net/wiki/SC2034 -- TESTDATA appears unused. Verify u...
https://www.shellcheck.net/wiki/SC2046 -- Quote this to prevent word splitt...
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This patch
* drops the default `-w` flag for `make static`, which helps with
debugging the static runc binary;
* adds `EXTRA_LDFLAGS="-w -s"` to `script/release.sh` to disable DWARF
generation and symbol table for the release runc binary;
* adds strip in `script/release.sh` for a further size-optimized release
runc binary.
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Use "git describe --dirty --long" instead of "git rev-parse". As a
result, the commit ID will contain the closest tag, the number of commits
since the tag, and the (abbreviated) git commit sha (see example below).
NOTE that this tag is still unique and can be used instead of bare sha
for all git commands.
Example output of "runc -v | grep commit".
Before:
commit: 4d87573871
After:
commit: v1.0.0-rc95-9-g6f55d074
This means that
- the closest tag is v1.0.0-rc95
- there were 9 commits after the tag
- the abbreviated sha is 6f55d074
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is required for environments/build systems where a specific
go version / command needs to be used.
Signed-off-by: Enrico Weigelt, metux IT consult <info@metux.net>
This reverts commit d0cbef576f.
Dockre/Moby still builds runc with Go 1.13, so we should still support
Go 1.13.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Rodrigo Campos