Apparently, scripts/validate-c is not working in CI (or maybe
maintainers ignored the failures from it) -- current C code
gets some changes if we run indent on it.
This commit fixes this, simplifying things along the way.
In particular:
1. Remove "validate" make target, add "cfmt" target that just runs
indent on all *.c files in the repository (NOTE that *.h files
are not included, as before).
This may help a contributor to fix their code -- they just need
to run "make cfmt" now instead of running "make validate" and
copy-pasting the indent command and options from the hint.
2. Split GHA validate/misc into validate/release and validate/cfmt.
The latter checks that the sources are not changed after "make cfmt".
3. Adds a few more options to indent. This was mostly motivated by
trying to save the existing formatting, minimizing the amount of
changes indent produces.
The new options are:
* -il0: sets the offset for goto labels to 0 (currently all labels
but one are not indented -- let's keep it that way);
* -ppi2: sets the indentation for nested preprocessor directives
to 2 spaces (same as it is done in "SYS_memfd_create" defines);
* -cp1: sets the indentation between #else / #endif and the
following comment to 1 space.
4. Reformat the code using the new indent options.
5. Remove the now-unused script/{.validate,validate-c}.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Sometimes github gives up 5xx errors, for example:
> panic: getImages error exit status 1 (output: curl: (22) The requested URL returned error: 502
> Failed to get https://github.com/docker-library/busybox/raw/dist-i386/stable/glibc/busybox.tar.xz
This is unfortunate but temporary, and adding --retry should handle
at least some cases, improving CI stability.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This simplifies and optimizes getting container images used for tests.
Currently, we have three different ways of getting images:
1. (for hello-world) the image is in this repo under tests/integration/testdata.
2. (for busybox) download it from github (the repo that is used for
preparing official Docker image) using curl.
3. (for debian) download from Docker hub, using skopeo and umoci.
To further complicate things, we have to do this downloading in multiple
scenarios (at least 4): locally, in github CI, from Dockefile, inside a
Vagrant VM. For each scenario, we have to install skopeo and umoci, and
those two are not yet universally available for all the distros that we
use.
Yet another complication is those images are used for tests/integration
(bats-driven tests) as well as for libcontainer/integration (go tests).
The tests in libcontainer/integration rely on busybox being available
from /busybox, and the bats tests just download the images to a
temporary location during every run.
It is also hard to support CI for other architectures, because all
the machinery for preparing images is so complicated.
This commit is an attempt to simplify and optimize getting images,
mostly by getting rid of skopeo and umoci dependencies, but also
by moving the download logic into one small shell script, which
is used from all the places.
Benefits:
- images (if not present) are only downloaded once;
- same images are used for both kind of tests (go and bats);
- same images are used for local and inside-docker tests
(because source directory is mounted into container);
- the download logic is located within 1 simple shell script.
[v2: fix eval; more doc to get-images; print URL if curl failed]
[v3: use "slim" debian, twice as small]
[v4: fix not using $image in setup_bundle]
[v5: don't remove TESTDATA from helpers.bash]
[v6: add i386 support]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Having -EPERM is the default was a fairly significant mistake from a
future-proofing standpoint in that it makes any new syscall return a
non-ignorable error (from glibc's point of view). We need to correct
this now because faccessat2(2) is something glibc critically needs to
have support for, but they're blocked on container runtimes because we
return -EPERM unconditionally (leading to confusion in glibc). This is
also a problem we're probably going to keep running into in the future.
Unfortunately there are several issues which stop us from having a clean
solution to this problem:
1. libseccomp has several limitations which require us to emulate
behaviour we want:
a. We cannot do logic based on syscall number, meaning we cannot
specify a "largest known syscall number";
b. libseccomp doesn't know in which kernel version a syscall was
added, and has no API for "minimum kernel version" so we cannot
simply ask libseccomp to generate sane -ENOSYS rules for us.
c. Additional seccomp rules for the same syscall are not treated as
distinct rules -- if rules overlap, seccomp will merge them. This
means we cannot add per-syscall -EPERM fallbacks;
d. There is no inverse operation for SCMP_CMP_MASKED_EQ;
e. libseccomp does not allow you to specify multiple rules for a
single argument, making it impossible to invert OR rules for
arguments.
2. The runtime-spec does not have any way of specifying:
a. The errno for the default action;
b. The minimum kernel version or "newest syscall at time of profile
creation"; nor
c. Which syscalls were intentionally excluded from the allow list
(weird syscalls that are no longer used were excluded entirely,
but Docker et al expect those syscalls to get EPERM not ENOSYS).
3. Certain syscalls should not return -ENOSYS (especially only for
certain argument combinations) because this could also trigger glibc
confusion. This means we have to return -EPERM for certain syscalls
but not as a global default.
4. There is not an obvious (and reasonable) upper limit to syscall
numbers, so we cannot create a set of rules for each syscall above
the largest syscall number in libseccomp. This means we must handle
inverse rules as described below.
5. Any syscall can be specified multiple times, which can make
generation of hotfix rules much harder.
As a result, we have to work around all of these things by coming up
with a heuristic to stop the bleeding. In the future we could hopefully
improve the situation in the runtime-spec and libseccomp.
The solution applied here is to prepend a "stub" filter which returns
-ENOSYS if the requested syscall has a larger syscall number than any
syscall mentioned in the filter. The reason for this specific rule is
that syscall numbers are (roughly) allocated sequentially and thus newer
syscalls will (usually) have a larger syscall number -- thus causing our
filters to produce -ENOSYS if the filter was written before the syscall
existed.
Sadly this is not a perfect solution because syscalls can be added
out-of-order and the syscall table can contain holes for several
releases. Unfortuntely we do not have a nicer solution at the moment
because there is no library which provides information about which Linux
version a syscall was introduced in. Until that exists, this workaround
will have to be good enough.
The above behaviour only happens if the default action is a blocking
action (in other words it is not SCMP_ACT_LOG or SCMP_ACT_ALLOW). If the
default action is permissive then we don't do any patching.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
The fedora job usually completes within 15-16 minutes,
and the centos-7 one needs 7-8 minutes.
Set the timeouts to twice those values.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Github Actions execution environments lack tty, which is needed for
integration tests. We have used script utility before to provide a fake
tty, but it appears to be buggy on CentOS 7.
Since we use ssh anyway to access Vagant VM, add -tt to forcefully
allocate a tty.
Apparently, neither "TERM=xterm" nor "stty size" is needed for
any tests to work, so remove those, too.
Finally, unit tests do not need a terminal, so remove the "script"
from those.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This check (for commit subject being short) only make sense for pull
requests, and it fails on pushes to master (i.e. on merges).
Make sure it's only run for pull requests.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It currently takes close to 2 minutes to build docker image
(which is currently only used for cross compilation).
Let's see if caching docker layers will help.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Note that `make ci` in Travis included `make test`, but we're already
doing that elsewhere (see .github/workflows/test.yml).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In here we have to use Docker, as Ubuntu does not support
all the architectures we're compile-testing here.
Since this is the only step that is using Docker,
there is no sense to separate `make runcimage` from
the rest of it. In case we'll have to use Docker image
more, it will make sense to do so.
While at it, ditch script/tmpmount (added by commit 1735ad788f),
because
- it required root (because mount);
- it is probably no longer needed.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is Ubuntu 20.04 so cgroup v1 testing only :(
The upside is, it's pretty fast (currently all the tests finish
within 5 minutes).
rootless tests take longer time so we run them in parallel
with non-rootless ones. Also, to balance the time, we run
unit tests when rootless is not set. IOW, 'rootless' in job
name means it's only rootless integration tests, and no rootless
means it's unit tests, integration tests with fs driver, and
integration tests with systemd driver.
Note, `script` is used to run integration tests to provide a tty
(which by default is not available on gh actions).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Note that
1. No golang caching is required, since we do not build any go code,
and it does not look like it make sense to cache a single binary
we donwload from github.
2. Use a problem matcher, so the PRs will be annotated with
shellcheck warnings.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Kir Kolyshkin