Kir Kolyshkin
5ab0588432
This simplifies and optimizes getting container images used for tests. Currently, we have three different ways of getting images: 1. (for hello-world) the image is in this repo under tests/integration/testdata. 2. (for busybox) download it from github (the repo that is used for preparing official Docker image) using curl. 3. (for debian) download from Docker hub, using skopeo and umoci. To further complicate things, we have to do this downloading in multiple scenarios (at least 4): locally, in github CI, from Dockefile, inside a Vagrant VM. For each scenario, we have to install skopeo and umoci, and those two are not yet universally available for all the distros that we use. Yet another complication is those images are used for tests/integration (bats-driven tests) as well as for libcontainer/integration (go tests). The tests in libcontainer/integration rely on busybox being available from /busybox, and the bats tests just download the images to a temporary location during every run. It is also hard to support CI for other architectures, because all the machinery for preparing images is so complicated. This commit is an attempt to simplify and optimize getting images, mostly by getting rid of skopeo and umoci dependencies, but also by moving the download logic into one small shell script, which is used from all the places. Benefits: - images (if not present) are only downloaded once; - same images are used for both kind of tests (go and bats); - same images are used for local and inside-docker tests (because source directory is mounted into container); - the download logic is located within 1 simple shell script. [v2: fix eval; more doc to get-images; print URL if curl failed] [v3: use "slim" debian, twice as small] [v4: fix not using $image in setup_bundle] [v5: don't remove TESTDATA from helpers.bash] [v6: add i386 support] Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
runc Integration Tests
Integration tests provide end-to-end testing of runc.
Note that integration tests do not replace unit tests.
As a rule of thumb, code should be tested thoroughly with unit tests. Integration tests on the other hand are meant to test a specific feature end to end.
Integration tests are written in bash using the bats (Bash Automated Testing System) framework.
Running integration tests
The easiest way to run integration tests is with Docker:
$ make integration
Alternatively, you can run integration tests directly on your host through make:
$ sudo make localintegration
Or you can just run them directly using bats
$ sudo bats tests/integration
To run a single test bucket:
$ make integration TESTPATH="/checkpoint.bats"
To run them on your host, you need to set up a development environment plus bats (Bash Automated Testing System).
For example:
$ cd ~/go/src/github.com
$ git clone https://github.com/bats-core/bats-core.git
$ cd bats-core
$ ./install.sh /usr/local
Note
: There are known issues running the integration tests using devicemapper as a storage driver, make sure that your docker daemon is using aufs if you want to successfully run the integration tests.
Writing integration tests
[helper functions] (https://github.com/opencontainers/runc/blob/master/tests/integration/helpers.bash) are provided in order to facilitate writing tests.
#!/usr/bin/env bats
# This will load the helpers.
load helpers
# setup is called at the beginning of every test.
function setup() {
# see functions teardown_hello and setup_hello in helpers.bash, used to
# create a pristine environment for running your tests
teardown_hello
setup_hello
}
# teardown is called at the end of every test.
function teardown() {
teardown_hello
}
@test "this is a simple test" {
runc run containerid
# "The runc macro" automatically populates $status, $output and $lines.
# Please refer to bats documentation to find out more.
[ "$status" -eq 0 ]
# check expected output
[[ "${output}" == *"Hello"* ]]
}