Commit b999376fb2 ("nsenter: cloned_binary: remove bindfd logic
entirely") removed the read-only bind-mount logic from our cloned binary
code because it wasn't really safe because a container with
CAP_SYS_ADMIN could remove the MS_RDONLY bit and get write access to
/proc/self/exe (even with user namespaces this could've been an issue
because it's not clear if the flags are locked).
However, copying a binary does seem to have a minor performance impact.
The only way to have no performance impact would be for the kernel to
block these write attempts, but barring that we could try to reduce the
overhead by coming up with a mount that cannot have it's read-only bits
cleared.
The "simplest" solution is to create a temporary overlayfs using
fsopen(2) which uses the directory where runc exists as a lowerdir,
ensuring that the container cannot access the underlying file -- and we
don't have to do any copies.
While fsopen(2) is not free because mount namespace cloning is usually
expensive (and so it seems like the difference would be marginal), some
basic performance testing seems to indicate there is a ~60% improvement
doing it this way and that it has effectively no overhead even when
compared to just using /proc/self/exe directly:
% hyperfine --warmup 50 \
> "./runc-noclone run -b bundle ctr" \
> "./runc-overlayfs run -b bundle ctr" \
> "./runc-memfd run -b bundle ctr"
Benchmark 1: ./runc-noclone run -b bundle ctr
Time (mean ± σ): 13.7 ms ± 0.9 ms [User: 6.0 ms, System: 10.9 ms]
Range (min … max): 11.3 ms … 16.1 ms 184 runs
Benchmark 2: ./runc-overlayfs run -b bundle ctr
Time (mean ± σ): 13.9 ms ± 0.9 ms [User: 6.2 ms, System: 10.8 ms]
Range (min … max): 11.8 ms … 16.0 ms 180 runs
Benchmark 3: ./runc-memfd run -b bundle ctr
Time (mean ± σ): 22.6 ms ± 1.3 ms [User: 5.7 ms, System: 20.7 ms]
Range (min … max): 19.9 ms … 26.5 ms 114 runs
Summary
./runc-noclone run -b bundle ctr ran
1.01 ± 0.09 times faster than ./runc-overlayfs run -b bundle ctr
1.65 ± 0.15 times faster than ./runc-memfd run -b bundle ctr
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
runc Integration Tests
Integration tests provide end-to-end testing of runc.
Note that integration tests do not replace unit tests.
As a rule of thumb, code should be tested thoroughly with unit tests. Integration tests on the other hand are meant to test a specific feature end to end.
Integration tests are written in bash using the bats (Bash Automated Testing System) framework.
Running integration tests
The easiest way to run integration tests is with Docker:
$ make integration
Alternatively, you can run integration tests directly on your host through make:
$ sudo make localintegration
Or you can just run them directly using bats
$ sudo bats tests/integration
To run a single test bucket:
$ make integration TESTPATH="/checkpoint.bats"
To run them on your host, you need to set up a development environment plus bats (Bash Automated Testing System).
For example:
$ cd ~/go/src/github.com
$ git clone https://github.com/bats-core/bats-core.git
$ cd bats-core
$ ./install.sh /usr/local
Note
: There are known issues running the integration tests using devicemapper as a storage driver, make sure that your docker daemon is using aufs if you want to successfully run the integration tests.
Writing integration tests
helper functions are provided in order to facilitate writing tests.
#!/usr/bin/env bats
# This will load the helpers.
load helpers
# setup is called at the beginning of every test.
function setup() {
setup_busybox
}
# teardown is called at the end of every test.
function teardown() {
teardown_bundle
}
@test "this is a simple test" {
runc run containerid
# "The runc macro" automatically populates $status, $output and $lines.
# Please refer to bats documentation to find out more.
[ "$status" -eq 0 ]
# check expected output
[[ "${output}" == *"Hello"* ]]
}