Commit Graph

4785 Commits

Author SHA1 Message Date
Sebastiaan van Stijn 677baf22d2 libcontainer: isolate libcontainer/devices
Move the Device-related types to libcontainer/devices, so that
the package can be used in isolation. Aliases have been created
in libcontainer/configs for backward compatibility.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-12-01 11:11:21 +01:00
Mrunal Patel 2b92c25130 Merge pull request #2690 from kolyshkin/gh-act-int
ci: add integration / unit tests to github actions
2020-11-30 13:27:39 -08:00
Mrunal Patel 7cfb3dcfa2 Merge pull request #2692 from kolyshkin/test-tty-nits
tests/int/tty.bats: increase timeout
2020-11-29 17:42:11 -08:00
Kir Kolyshkin c1ef0cf62c ci: add integration+unit tests to github actions
This is Ubuntu 20.04 so cgroup v1 testing only :(
The upside is, it's pretty fast (currently all the tests finish
within 5 minutes).

rootless tests take longer time so we run them in parallel
with non-rootless ones. Also, to balance the time, we run
unit tests when rootless is not set. IOW, 'rootless' in job
name means it's only rootless integration tests, and no rootless
means it's unit tests, integration tests with fs driver, and
integration tests with systemd driver.

Note, `script` is used to run integration tests to provide a tty
(which by default is not available on gh actions).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-27 21:35:02 -08:00
Kir Kolyshkin fce8dd4d66 tests/int/tty.bats: increase timeout
Just saw the timeout being hit on CI (Fedora 33 vagrant VM on Travis),
so let's increase it.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-27 14:15:32 -08:00
Mrunal Patel 06b737bb15 Merge pull request #2687 from kolyshkin/more-github-actions
More GitHub actions
2020-11-23 21:02:51 -08:00
Akihiro Suda d15ffff5df Merge pull request #2681 from thaJeztah/fix_bats_link
integration: update README to link to bats-core
2020-11-24 02:02:03 +09:00
Akihiro Suda 0b11e29a87 Merge pull request #2684 from thaJeztah/bump_ebpf
vendor: github.com/cilium/ebpf v0.2.0
2020-11-23 03:44:12 +09:00
Akihiro Suda 69d84cb12d Merge pull request #2688 from kolyshkin/criu-3.15
Dockerfile: bump criu to 3.15
2020-11-23 03:42:07 +09:00
Kir Kolyshkin c6ed18542d ci: add shellcheck to github actions
Note that

1. No golang caching is required, since we do not build any go code,
   and it does not look like it make sense to cache a single binary
   we donwload from github.

2. Use a problem matcher, so the PRs will be annotated with
   shellcheck warnings.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-20 10:35:56 -08:00
Kir Kolyshkin 27835a9e5e Makefile: move shellcheck out of validate
This serves two purposes:

1. A developer can now run `make shellcheck` to show  any issues    with
   shell or bats files formatting (this requires a recent version of
   shfmt, which I think is out of scope for Makefile).

2. Exclude shellcheck from travis ci (will be re-added as a GH action
   by the next commit).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-20 10:35:56 -08:00
Kir Kolyshkin 33bda24ac3 ci: move verify-deps from travis to github actions
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-20 10:35:56 -08:00
Kir Kolyshkin c60f23b335 ci: add shfmt to github actions
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-20 10:35:56 -08:00
Kir Kolyshkin 717a73b397 ci: renamed golangci-lint to validate
...so we can add more validation-type jobs.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-20 10:35:56 -08:00
Kir Kolyshkin 06b204e5c2 Makefile: move shfmt out of validate, add -w
This serves two purposes:

1. A developer can now run `make shfmt` to show and fix any issues
   with shell or bats files formatting (this requires a recent version
   of shfmt, which I think is out of scope for Makefile).

2. Exclude shfmt check from travis ci (will be re-added as a GH action
   by the next commit).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-20 10:35:56 -08:00
Kir Kolyshkin 7856c34018 Dockerfile: bump criu to 3.15
Changes: see https://criu.org/Download/criu/3.15

Note in Vagrant.centos7 we use Adrian's repo [1] which already have criu
3.15, and in Vagrant.fedora33 we use stock fedora which also already has
criu 3.15.

[1] https://copr.fedorainfracloud.org/coprs/adrian/criu-el7/packages/

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-20 10:35:32 -08:00
Sebastiaan van Stijn ee1bdb80af vendor: github.com/cilium/ebpf v0.2.0
full diff: https://github.com/cilium/ebpf/compare/v0.1.0...v0.2.0

- btf: add go-fuzz targets
- btf: avoid Type copy in FuncProto.walk
- btf: check err in loadSpecFromVmlinux
- btf: handle type name flavours
- CI: test on 5.9 kernel
- cmd/bpf2go: output ELF .o next to the .go file
- link: add AttachSkLookup
- Remove two unused functions
- Support LSM attach
- use buffered I/O to cut down on read syscalls
- Various doc link fixes

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-20 13:56:46 +01:00
Akihiro Suda b69070a5fc Merge pull request #2686 from kolyshkin/fix-kubic-repo
Dockefile: fix path to skopeo repo
2020-11-20 17:55:11 +09:00
Kir Kolyshkin f0d5e83966 Dockefile: fix path to skopeo repo
The current URL now gives 404. I looked in there and found that
apparently Debian_Unstable becomes Debian_10. Fix the URLs accordingly.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-19 18:19:01 -08:00
Akihiro Suda 689513cc09 Merge pull request #2643 from xiaochenshen/rdt-cmt-check
libcontainer/intelrdt: fix CMT feature check
2020-11-18 01:56:22 +09:00
Sebastiaan van Stijn d9010b0e2d integration: update README to link to bats-core
We switched to bats-core in commit d4bc7c10ec,
so updating the instructions to match.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-17 17:46:04 +01:00
Kir Kolyshkin 636f23dd21 Merge pull request #2674 from AkihiroSuda/graduate-cgroup2-experimental
Move cgroup v2 out of experimental
2020-11-10 15:16:03 -08:00
Michael Crosby f4440f9b89 Merge pull request #2665 from AkihiroSuda/fedora33
CI: update Fedora to 33
2020-11-10 10:47:32 -05:00
Michael Crosby 104c493a9b Merge pull request #2676 from KentaTada/modify-doc-of-passthrough
docs: terminals: modify the example of Pass-Through mode.
2020-11-10 10:46:54 -05:00
Michael Crosby 295e4e0772 Merge pull request #2677 from xiaochenshen/rdt-rename-cat-mba-flags
libcontainer/intelrdt: rename CAT and MBA enabled flags
2020-11-10 10:46:13 -05:00
Akihiro Suda 3f2f06dfe1 Move cgroup v2 out of experimental
After a lot of refactoring, our cgroup v1 and v2 drivers now have same level of implementation quality,
so we can move the v2 driver out of experimental.

Close #2663

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-11-10 17:07:06 +09:00
Xiaochen Shen f62ad4a0de libcontainer/intelrdt: rename CAT and MBA enabled flags
Rename CAT and MBA enabled flags to be consistent with others.
No functional change.

Signed-off-by: Xiaochen Shen <xiaochen.shen@intel.com>
2020-11-10 15:32:01 +08:00
Mrunal Patel 30233a7aea Merge pull request #2675 from AkihiroSuda/vendor-20201109
update vendor
2020-11-09 21:26:49 -08:00
Xiaochen Shen 620f4c5c88 libcontainer/intelrdt: fix CMT feature check
Intel RDT sub-features can be selectively disabled or enabled by kernel
command line. See "rdt=" option details in kernel document:
https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt

But Cache Monitoring Technology (CMT) feature is not correctly checked
in init() and getCMTNumaNodeStats() now. If CMT is disabled by kernel
command line (e.g., rdt=!cmt,mbmtotal,mbmlocal,l3cat,mba) while hardware
supports CMT, we may get following error when getting Intel RDT stats:
  runc run c1
  runc events c1
  ERRO[0005] container_linux.go:200: getting container's Intel RDT stats
  caused: open /sys/fs/resctrl/c1/mon_data/mon_L3_00/llc_occupancy: no
  such file or directory

Fix CMT feature check in init() and GetStats() call paths.

Signed-off-by: Xiaochen Shen <xiaochen.shen@intel.com>
2020-11-10 10:36:20 +08:00
Akihiro Suda fb59e6f726 Merge pull request #2583 from adrianreber/join-more-namespaces
restore: tell CRIU to use existing namespaces
2020-11-10 00:19:40 +09:00
Kenta Tada 896da0b95e docs: terminals: modify the example of Pass-Through mode.
This commit removes the unnecessary ampersand.
Especially, it causes the error of "ambiguous redirect" when use bash.

Signed-off-by: Kenta Tada <Kenta.Tada@sony.com>
2020-11-09 17:10:52 +09:00
Akihiro Suda 4690064f05 update vendor
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-11-09 15:14:23 +09:00
Mrunal Patel 27227a9358 Merge pull request #2669 from kolyshkin/systemd-unified
Initial v2 resources.unified systemd support
2020-11-06 12:31:31 -08:00
Akihiro Suda e8498d3ad5 Merge pull request #2672 from giuseppe/fix-remount-readonly
linux: fix remount readonly in a user namespace
2020-11-06 20:40:46 +09:00
Akihiro Suda 9403afd75c CI: Fedora 33: print kernel version, systemd version, and rootfs type
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-11-06 20:30:04 +09:00
Akihiro Suda 0a097615c0 CI: update Fedora to 33
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-11-06 20:30:01 +09:00
Giuseppe Scrivano 41aa764010 linux: drop MS_REC for readonly remount
it has no effect.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-11-06 11:36:50 +01:00
Giuseppe Scrivano a4e6955e31 linux: fix remount readonly in a user namespace
if we are remounting root read only when in a user namespace, make
sure the existing flags (e.g. MS_NOEXEC, MS_NODEV) are maintained
otherwise the mount fails with EPERM.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-11-06 11:35:40 +01:00
Akihiro Suda 8591c3364e Merge pull request #2658 from kolyshkin/tests-ex-run
tests/integration: rm excessive run use; fix exec --preserve-fds test
2020-11-06 13:56:21 +09:00
Mrunal Patel 60b2a47b17 Merge pull request #2671 from kolyshkin/shfmt-pin
ci: pin shfmt to v3.2.0
2020-11-05 16:39:18 -08:00
Kir Kolyshkin 2e968a833f libct/cg/sd/v2: "support" (ignore) memory.oom.group
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:05:02 -08:00
Kir Kolyshkin c013be563c libct/cg/sd/v2: support memory.* / Memory* unified
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:05:00 -08:00
Kir Kolyshkin 13afa58d0e libct/cg/sd/v2: support cpuset.* / Allowed*
* cpuset.cpus -> AllowedCPUs
 * cpuset.mems -> AllowedMemoryNodes

No test for cgroup v2 resources.unified override, as this requires a
separate test case, and all the unified resources are handled uniformly
so there's little sense to test all parameters.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:04:57 -08:00
Kir Kolyshkin 5be8b97aec libct/cg/sd/v2: support cpu.weight / CPUWeight
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:04:46 -08:00
Kir Kolyshkin 390a98f3fd runc update: support unified resources
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:04:41 -08:00
Kir Kolyshkin ab80eb32d2 libct/cg/sd/v2: support cpu.max unified resource
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:04:37 -08:00
Kir Kolyshkin 7f24098d9b tests/int: move check_cpu* to helpers
To be used by the following commit.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:04:32 -08:00
Kir Kolyshkin fd5226d0eb libct/cg/sd: add defCPUQuotaPeriod
Un-inline defCPUQuotaPeriod constant. To be used by the next commit.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:04:27 -08:00
Kir Kolyshkin 0cb8bf67a3 Initial v2 resources.unified systemd support
In case systemd is used as cgroups manager, and a user sets some
resources using unified resource map (as per [1]), systemd is not
aware of any parameters, so there will be a discrepancy between
the cgroupfs state and systemd unit state.

Let's try to fix that by converting known unified resources to systemd
properties.

Currently, this is only implemented for pids.max as a POC.

Some other parameters (that might or might not have systemd unit
property equivalents) are:

$ ls -l | grep w-
-rw-r--r--. 1 root root 0 Oct 10 13:57 cgroup.freeze
-rw-r--r--. 1 root root 0 Oct 10 13:57 cgroup.max.depth
-rw-r--r--. 1 root root 0 Oct 10 13:57 cgroup.max.descendants
-rw-r--r--. 1 root root 0 Oct 10 13:57 cgroup.procs
-rw-r--r--. 1 root root 0 Oct 21 09:43 cgroup.subtree_control
-rw-r--r--. 1 root root 0 Oct 10 13:57 cgroup.threads
-rw-r--r--. 1 root root 0 Oct 10 13:57 cgroup.type
-rw-r--r--. 1 root root 0 Oct 22 10:30 cpu.max
-rw-r--r--. 1 root root 0 Oct 10 13:57 cpu.pressure
-rw-r--r--. 1 root root 0 Oct 22 10:30 cpuset.cpus
-rw-r--r--. 1 root root 0 Oct 22 10:30 cpuset.cpus.partition
-rw-r--r--. 1 root root 0 Oct 22 10:30 cpuset.mems
-rw-r--r--. 1 root root 0 Oct 22 10:30 cpu.weight
-rw-r--r--. 1 root root 0 Oct 22 10:30 cpu.weight.nice
-rw-r--r--. 1 root root 0 Oct 22 10:30 hugetlb.1GB.max
-rw-r--r--. 1 root root 0 Oct 22 10:30 hugetlb.1GB.rsvd.max
-rw-r--r--. 1 root root 0 Oct 22 10:30 hugetlb.2MB.max
-rw-r--r--. 1 root root 0 Oct 22 10:30 hugetlb.2MB.rsvd.max
-rw-r--r--. 1 root root 0 Oct 22 10:30 io.bfq.weight
-rw-r--r--. 1 root root 0 Oct 22 10:30 io.latency
-rw-r--r--. 1 root root 0 Oct 22 10:30 io.max
-rw-r--r--. 1 root root 0 Oct 10 13:57 io.pressure
-rw-r--r--. 1 root root 0 Oct 22 10:30 io.weight
-rw-r--r--. 1 root root 0 Oct 10 13:57 memory.high
-rw-r--r--. 1 root root 0 Oct 10 13:57 memory.low
-rw-r--r--. 1 root root 0 Oct 10 13:57 memory.max
-rw-r--r--. 1 root root 0 Oct 10 13:57 memory.min
-rw-r--r--. 1 root root 0 Oct 10 13:57 memory.oom.group
-rw-r--r--. 1 root root 0 Oct 10 13:57 memory.pressure
-rw-r--r--. 1 root root 0 Oct 10 13:57 memory.swap.high
-rw-r--r--. 1 root root 0 Oct 10 13:57 memory.swap.max

Surely, it is a manual conversion for every such case...

[1] https://github.com/opencontainers/runtime-spec/pull/1040

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:04:19 -08:00
Kir Kolyshkin ed54837690 tests/int/update.bats: add checks for runc status
Those were missing for some reason, so we did not fail the test in case
"runc update" returned an error.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-05 16:03:21 -08:00