Every `make` now produces something like this:
make[1]: Entering directory '/home/kir/go/src/github.com/opencontainers/runc'
readelf -h runc
Machine: Advanced Micro Devices X86-64
Flags: 0x0
readelf -h libcontainer/dmz/runc-dmz
Machine: Advanced Micro Devices X86-64
Flags: 0x0
runc-dmz architecture matches runc binary.
make[1]: Leaving directory '/home/kir/go/src/github.com/opencontainers/runc'
That is a bit too much. Let's make it less verbose.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This really isn't ideal but it can be used to avoid the largest issues
with the memfd-based runc binary protection. There are several caveats
with using this tool, see the help page for the new binary for details.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
The idea is to remove the need for cloning the entire runc binary by
replacing the final execve() call of the container process with an
execve() call to a clone of a small C binary which just does an execve()
of its arguments.
This provides similar protection against CVE-2019-5736 but without
requiring a >10MB binary copy for each "runc init". When compiled with
musl, runc-dmz is 13kB (though unfortunately with glibc, it is 1.1MB
which is still quite large).
It should be noted that there is still a window where the container
processes could get access to the host runc binary, but because we set
ourselves as non-dumpable the container would need CAP_SYS_PTRACE (which
is not enabled by default in Docker) in order to get around the
proc_fd_access_allowed() checks. In addition, since Linux 4.10[1] the
kernel blocks access entirely for user namespaced containers in this
scenario. For those cases we cannot use runc-dmz, but most containers
won't have this issue.
This new runc-dmz binary can be opted out of at compile time by setting
the "runc_nodmz" buildtag, and at runtime by setting the RUNC_DMZ=legacy
environment variable. In both cases, runc will fall back to the classic
/proc/self/exe-based cloning trick. If /proc/self/exe is already a
sealed memfd (namely if the user is using contrib/cmd/memfd-bind to
create a persistent sealed memfd for runc), neither runc-dmz nor
/proc/self/exe cloning will be used because they are not necessary.
[1]: https://github.com/torvalds/linux/commit/bfedb589252c01fa505ac9f6f2a3d5d68d707ef4
Co-authored-by: lifubang <lifubang@acmcoder.com>
Signed-off-by: lifubang <lifubang@acmcoder.com>
[cyphar: address various review nits]
[cyphar: fix runc-dmz cross-compilation]
[cyphar: embed runc-dmz into runc binary and clone in Go code]
[cyphar: make runc-dmz optional, with fallback to /proc/self/exe cloning]
[cyphar: do not use runc-dmz when the container has certain privs]
Co-authored-by: Aleksa Sarai <cyphar@cyphar.com>
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
We need these to match the Makefile detection of the right gcc for
runc-dmz, as well as making sure that everything builds properly for our
cross-i386 tests. While we're at it, add x86 to the list of build
targets for release builds (presumably nobody will use it, but since we
do test builds of this anyway it probably won't hurt).
In addition, clean up the handling of the native architecture build by
treating it the same as any other build (ensuring that building runc
from a different platform will work the same way regardless of the
native architecture). In practice, the build works the same way as
before.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
These checks ensure that all of the keys in the runc.keyring list are
actually the keys of the specified user and that the users themselves
are actually maintainers.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
... as a way to maybe catch some CHANGELOG.md bugs at the last moment.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 54cfb25d69)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Previously (see commit 91fa032da4) we found a few issues
using this check, but apparently the CHANGELOG.md is in UTF-8, and
the recently added quote is breaking this, so remove.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 7b3ac330f7)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Bump shfmt to v3.5.1. Release notes:
https://github.com/mvdan/sh/releases
2. Since shfmt v3.5.0, specifying -l bash (or -l bats) is no longer
necessary. Therefore, we can use shfmt to find all the files.
Add .editorconfig to ignore vendor subdirectory.
3. Use shfmt docker image, so that we don't have to install anything
explicitly. This greatly simplifies the shfmt CI job. Add
localshfmt target so developers can still use a local shfmt binary
when necessary.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is a forward-port of commit 91fa032da4 ("ci: add basic checks for
CHANGELOG.md"), plus whatever changes were made in release-1.1 branch
(up to v1.1.3).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This removes the runc dependency on cpuguy83/md2man and
russross/blackfriday, which saves more than 400 KB (more than 300 KB
once stripped) from the binary.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It doesn't matter whether static or dynamic linking is used, runc
always needs libcontainer/nsenter, which is written in C and thus
requires cgo. Same is true for libcontainer/integration.
In addition, contrib/pkg/seccompagent also needs cgo (if seccomp build
tag is set), as it need to be linked against libseccomp C library.
By default, cgo is disabled when cross-compiling, meaning that
CGO_ENABLED=1 has to be set explicitly in such cases.
In all other cases (e.g. other contrib binaries) we do not need cgo.
Remove CGO_ENABLED=1 from GO_BUILD_STATIC (as it does not have anything
to do with static linking), and add it to all targets that require it.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Set to empty value by default.
2. Assume Linux (remove GOOS check, since we do not support other OSes).
3. Instead of using a "not-supported" list, use a "supported" list
(as Go release notes usually say which platforms are supported).
As of today, -buildmode=pie is supported for:
* linux/386, linux/amd64, linux/arm, linux/arm64, and linux/ppc64le
(since Go 1.6, see https://tip.golang.org/doc/go1.6#compiler)
* linux/s390x (since Go 1.7, which adds the initial port)
* linux/riscv64 (since Go 1.16, see
https://tip.golang.org/doc/go1.16#riscv)
NOTE this does not mean we support these architectures; it is merely
a way to see if -buildmode=pie can be used.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
LDFLAGS_COMMON are used from two places, so it makes sense to dedup.
LDFLAGS_STATIC is a preparation for the next commit.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Now the only remaining file that needs shellcheck warnings to be fixed
is bash-completion. Note that in Makefile's TODO.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
... and add this file to shellcheck target in Makefile.
These:
In script/check-config.sh line 27:
kernelMinor="${kernelVersion#$kernelMajor.}"
^----------^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.
Did you mean:
kernelMinor="${kernelVersion#"$kernelMajor".}"
In script/check-config.sh line 103:
source /etc/os-release 2>/dev/null || /bin/true
^-------------^ SC1091 (info): Not following: /etc/os-release was not specified as input (see shellcheck -x).
In script/check-config.sh line 267:
NET_CLS_CGROUP $netprio
^------^ SC2206 (warning): Quote to prevent word splitting/globbing, or split robustly with mapfile or read -a.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
My GPG keys are not available inside the container, so it makes little
sense to try to sign the binaries inside the container's release.sh. The
solution is to split things into separate build and sign stages, with
signing ocurring after the in-Docker build.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
This implements cross-build for "make release", moving the build into a
container. This way we can support arm, arm64, ppc, and whatnot.
* script/seccomp.sh: separate out of script/release.sh, amend to support
cross-compile and save needed environment variables to a file.
* Dockerfile: add installing libseccomp from source, as this is needed
for release builds.
* script/release.sh: amend to support more architectures in addition to
the native build. Additional arches can be added by specifying
"-a <arch>" argument (can be specified multiple times), or
"make RELEASE_ARGS="-a arm64" release" if called via make.
All supported architectures can be enabled via "make releaseall".
* Makefile: move "release" target to "localrelease", add "release" and
"releaseall" targets to build via the Dockerfile. This is done because
most distros (including Fedora and openSUSE) lack cross-glibc, which is
needed to cross-compile libseccomp.
* Makefile: remove 'cross' and 'localcross' targets, as this is now done
by the release script.
* .github/workflows/validate.yum: amend the release CI job to cross-build
for supported architectures, remove cross job.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
There is no need to have a static version of recvtty and/or sd-helper
binary.
This speeds up script/release.sh a bit.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. The seccompagent target it built in the same way as others in contrib,
so there is no need to have a separate rule.
2. Mark seccompagent as phony, because it is (it rarely happens, but I
actually just had an issue because this was absent).
3. Add seccompagent binary to clean target.
Fixes: e21a9ee81
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Implement sample seccomp agent. It's also used in integration tests in
the following commit.
Instructions how to use it in contrib/cmd/seccompagent/README.md
Signed-off-by: Alban Crequy <alban@kinvolk.io>
Signed-off-by: Rodrigo Campos <rodrigo@kinvolk.io>
Co-authored-by: Rodrigo Campos <rodrigo@kinvolk.io>
Add a test case for an issue fixed by the previous commit.
Unfortunately, this is somewhat complicated as there's no easy way to
create a transient unit, so a binary, sd-helper, had to be added. On top
of that, an ability to create a parent/pod cgroup is added to
helpers.bash, which might be useful for future integration tests.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
... and add the file to be checked by shellcheck.
The warnings fixed are:
In tests/integration/helpers.bash line 10:
INTEGRATION_ROOT=$(dirname "$(readlink -f "$BASH_SOURCE")")
^----------^ SC2128: Expanding an array without an index only gives the first element.
In tests/integration/helpers.bash line 22:
TESTDATA="${INTEGRATION_ROOT}/testdata"
^------^ SC2034: TESTDATA appears unused. Verify use (or export if used externally).
In tests/integration/helpers.bash line 42:
echo "runc $@ (status=$status):" >&2
^-- SC2145: Argument mixes string and array. Use * or separate argument.
^-----^ SC2154: status is referenced but not assigned.
In tests/integration/helpers.bash line 43:
echo "$output" >&2
^-----^ SC2154: output is referenced but not assigned.
In tests/integration/helpers.bash line 77:
| .linux.gidMappings += [{"hostID": '"$(($ROOTLESS_GIDMAP_START + 10))"', "containerID": 1, "size": 20}]
^--------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
In tests/integration/helpers.bash line 78:
| .linux.gidMappings += [{"hostID": '"$(($ROOTLESS_GIDMAP_START + 100))"', "containerID": 1000, "size": '"$(($ROOTLESS_GIDMAP_LENGTH - 1000))"'}]'
^--------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
^---------------------^ SC2004: $/${} is unnecessary on arithmetic variables.
In tests/integration/helpers.bash line 125:
base_path=$(gawk '$(NF-2) == "cgroup" && $NF ~ /\<'${g}'\>/ { print $5; exit }' /proc/self/mountinfo)
^--^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
base_path=$(gawk '$(NF-2) == "cgroup" && $NF ~ /\<'"${g}"'\>/ { print $5; exit }' /proc/self/mountinfo)
In tests/integration/helpers.bash line 127:
eval CGROUP_${g^^}_BASE_PATH="${base_path}"
^----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
eval CGROUP_"${g^^}"_BASE_PATH="${base_path}"
In tests/integration/helpers.bash line 229:
if [ "x$CGROUP_UNIFIED" = "xyes" ]; then
^----------------^ SC2268: Avoid x-prefix in comparisons as it no longer serves a purpose.
Did you mean:
if [ "$CGROUP_UNIFIED" = "yes" ]; then
In tests/integration/helpers.bash line 234:
eval cgroup=\$${var}${REL_CGROUPS_PATH}
^----^ SC2086: Double quote to prevent globbing and word splitting.
^-----------------^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
eval cgroup=\$"${var}""${REL_CGROUPS_PATH}"
In tests/integration/helpers.bash line 236:
cat $cgroup/$source
^-----^ SC2086: Double quote to prevent globbing and word splitting.
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
cat "$cgroup"/"$source"
In tests/integration/helpers.bash line 242:
current="$(get_cgroup_value $1)"
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
current="$(get_cgroup_value "$1")"
In tests/integration/helpers.bash line 245:
echo "current" $current "!?" "$expected"
^------^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
echo "current" "$current" "!?" "$expected"
In tests/integration/helpers.bash line 257:
[ $(id -u) != "0" ] && user="--user"
^------^ SC2046: Quote this to prevent word splitting.
In tests/integration/helpers.bash line 259:
current=$(systemctl show $user --property $source $SD_UNIT_NAME | awk -F= '{print $2}')
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
current=$(systemctl show $user --property "$source" $SD_UNIT_NAME | awk -F= '{print $2}')
In tests/integration/helpers.bash line 261:
[ "$current" = "$expected" ] || [ -n "$expected2" -a "$current" = "$expected2" ]
^-- SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
In tests/integration/helpers.bash line 309:
check_cgroup_value "cpu.weight" $weight
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
check_cgroup_value "cpu.weight" "$weight"
In tests/integration/helpers.bash line 310:
check_systemd_value "CPUWeight" $weight
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
check_systemd_value "CPUWeight" "$weight"
In tests/integration/helpers.bash line 383:
if [ $CGROUP_UNIFIED = "no" -a ! -e "${CGROUP_MEMORY_BASE_PATH}/memory.memsw.limit_in_bytes" ]; then
^-- SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
In tests/integration/helpers.bash line 412:
local cpu_count=$(grep -c '^processor' /proc/cpuinfo)
^-------^ SC2155: Declare and assign separately to avoid masking return values.
In tests/integration/helpers.bash line 450:
sleep $delay
^----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
sleep "$delay"
In tests/integration/helpers.bash line 453:
echo "Command \"$@\" failed $attempts times. Output: $output"
^-- SC2145: Argument mixes string and array. Use * or separate argument.
In tests/integration/helpers.bash line 471:
runc state $1
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
runc state "$1"
In tests/integration/helpers.bash line 472:
if [ $2 == "checkpointed" ]; then
^-- SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
if [ "$2" == "checkpointed" ]; then
In tests/integration/helpers.bash line 484:
mkdir $dir
^--^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
mkdir "$dir"
In tests/integration/helpers.bash line 497:
kill -9 $(cat "$dir/pid")
^---------------^ SC2046: Quote this to prevent word splitting.
In tests/integration/helpers.bash line 508:
export ROOT=$(mktemp -d "$BATS_RUN_TMPDIR/runc.XXXXXX")
^--^ SC2155: Declare and assign separately to avoid masking return values.
In tests/integration/helpers.bash line 512:
cd "$ROOT/bundle"
^---------------^ SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Did you mean:
cd "$ROOT/bundle" || exit
In tests/integration/helpers.bash line 535:
cd "$INTEGRATION_ROOT"
^--------------------^ SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Did you mean:
cd "$INTEGRATION_ROOT" || exit
For more information:
https://www.shellcheck.net/wiki/SC2145 -- Argument mixes string and array. ...
https://www.shellcheck.net/wiki/SC2034 -- TESTDATA appears unused. Verify u...
https://www.shellcheck.net/wiki/SC2046 -- Quote this to prevent word splitt...
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This patch
* drops the default `-w` flag for `make static`, which helps with
debugging the static runc binary;
* adds `EXTRA_LDFLAGS="-w -s"` to `script/release.sh` to disable DWARF
generation and symbol table for the release runc binary;
* adds strip in `script/release.sh` for a further size-optimized release
runc binary.
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Use "git describe --dirty --long" instead of "git rev-parse". As a
result, the commit ID will contain the closest tag, the number of commits
since the tag, and the (abbreviated) git commit sha (see example below).
NOTE that this tag is still unique and can be used instead of bare sha
for all git commands.
Example output of "runc -v | grep commit".
Before:
commit: 4d87573871
After:
commit: v1.0.0-rc95-9-g6f55d074
This means that
- the closest tag is v1.0.0-rc95
- there were 9 commits after the tag
- the abbreviated sha is 6f55d074
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is required for environments/build systems where a specific
go version / command needs to be used.
Signed-off-by: Enrico Weigelt, metux IT consult <info@metux.net>
This reverts commit d0cbef576f.
Dockre/Moby still builds runc with Go 1.13, so we should still support
Go 1.13.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
libseccomp is LGPL, meaning if we statically link it, we have to include
the source code of the library.
Amend "make release" to download and build libseccomp, build runc
against it, and include its sources into the release directory.
The only caveat is I found no way to stop go build from using the
stock (distro-provided) libseccomp.a, so the script checks that
the stock libseccomp.a is not available, and aborts otherwise.
While at it:
- enable shellcheck for script/release.sh
- remove libseccomp installation from the gha job
- add dependecies needed for libseccomp build to the gha job
[v2: also include libseccomp .asc file]
[v3: rebase]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Apparently, scripts/validate-c is not working in CI (or maybe
maintainers ignored the failures from it) -- current C code
gets some changes if we run indent on it.
This commit fixes this, simplifying things along the way.
In particular:
1. Remove "validate" make target, add "cfmt" target that just runs
indent on all *.c files in the repository (NOTE that *.h files
are not included, as before).
This may help a contributor to fix their code -- they just need
to run "make cfmt" now instead of running "make validate" and
copy-pasting the indent command and options from the hint.
2. Split GHA validate/misc into validate/release and validate/cfmt.
The latter checks that the sources are not changed after "make cfmt".
3. Adds a few more options to indent. This was mostly motivated by
trying to save the existing formatting, minimizing the amount of
changes indent produces.
The new options are:
* -il0: sets the offset for goto labels to 0 (currently all labels
but one are not indented -- let's keep it that way);
* -ppi2: sets the indentation for nested preprocessor directives
to 2 spaces (same as it is done in "SYS_memfd_create" defines);
* -cp1: sets the indentation between #else / #endif and the
following comment to 1 space.
4. Reformat the code using the new indent options.
5. Remove the now-unused script/{.validate,validate-c}.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Since go 1.14, mod=vendor is used automatically. Since go 1.16 is now
released, and minimally supported go version is 1.15.
As per commit fbeed5228, remove the go 1.13 workaround.
Fix README to require go 1.14.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This simplifies and optimizes getting container images used for tests.
Currently, we have three different ways of getting images:
1. (for hello-world) the image is in this repo under tests/integration/testdata.
2. (for busybox) download it from github (the repo that is used for
preparing official Docker image) using curl.
3. (for debian) download from Docker hub, using skopeo and umoci.
To further complicate things, we have to do this downloading in multiple
scenarios (at least 4): locally, in github CI, from Dockefile, inside a
Vagrant VM. For each scenario, we have to install skopeo and umoci, and
those two are not yet universally available for all the distros that we
use.
Yet another complication is those images are used for tests/integration
(bats-driven tests) as well as for libcontainer/integration (go tests).
The tests in libcontainer/integration rely on busybox being available
from /busybox, and the bats tests just download the images to a
temporary location during every run.
It is also hard to support CI for other architectures, because all
the machinery for preparing images is so complicated.
This commit is an attempt to simplify and optimize getting images,
mostly by getting rid of skopeo and umoci dependencies, but also
by moving the download logic into one small shell script, which
is used from all the places.
Benefits:
- images (if not present) are only downloaded once;
- same images are used for both kind of tests (go and bats);
- same images are used for local and inside-docker tests
(because source directory is mounted into container);
- the download logic is located within 1 simple shell script.
[v2: fix eval; more doc to get-images; print URL if curl failed]
[v3: use "slim" debian, twice as small]
[v4: fix not using $image in setup_bundle]
[v5: don't remove TESTDATA from helpers.bash]
[v6: add i386 support]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The apparmor tag was introduced in a01ed80 (2014) to make cgo dependency
on libapparmor optional.
However, the cgo dependency was removed in db093f6 (2017), so it is no
longer meaningful to keep apparmor build tag.
Close#2704
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
This serves two purposes:
1. A developer can now run `make shellcheck` to show any issues with
shell or bats files formatting (this requires a recent version of
shfmt, which I think is out of scope for Makefile).
2. Exclude shellcheck from travis ci (will be re-added as a GH action
by the next commit).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Kir Kolyshkin