Move libcontainer into subdirectory

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

libcontainer/CONTRIBUTING.md

@@ -0,0 +1,257 @@
+# The libcontainer Contributors' Guide
+
+Want to hack on libcontainer? Awesome! Here are instructions to get you
+started. They are probably not perfect, please let us know if anything
+feels wrong or incomplete.
+
+## Reporting Issues
+
+When reporting [issues](https://github.com/docker/libcontainer/issues)
+on GitHub please include your host OS (Ubuntu 12.04, Fedora 19, etc),
+the output of `uname -a`. Please include the steps required to reproduce
+the problem if possible and applicable.
+This information will help us review and fix your issue faster.
+
+## Development Environment
+
+### Requirements
+
+For best results, use a Linux development environment.
+The following packages are required to compile libcontainer natively.
+
+- Golang 1.3
+- GCC
+- git
+- cgutils
+
+You can develop on OSX, but you are limited to Dockerfile-based builds only.
+
+### Building libcontainer from Dockerfile
+
+    make all
+
+This is the easiest way of building libcontainer.
+As this build is done using Docker, you can even run this from [OSX](https://github.com/boot2docker/boot2docker)
+
+### Testing changes with "nsinit"
+
+    make sh
+
+This will create an container that runs `nsinit exec sh` on a busybox rootfs with the configuration from ['minimal.json'](https://github.com/docker/libcontainer/blob/master/sample_configs/minimal.json).
+Like the previous command, you can run this on OSX too!
+
+### Building libcontainer directly
+
+> Note: You should add the `vendor` directory to your GOPATH to use the vendored libraries
+
+    ./update-vendor.sh
+    go get -d ./...
+    make direct-build
+    # Run the tests
+    make direct-test-short | egrep --color 'FAIL|$'
+    # Run all the test
+    make direct-test | egrep --color 'FAIL|$'
+
+### Testing Changes with "nsinit" directly
+
+To test a change:
+
+    # Install nsinit
+    make direct-install
+
+    # Optional, add a docker0 bridge
+    ip link add docker0 type bridge
+    ifconfig docker0 172.17.0.1/16 up
+
+    mkdir testfs
+    curl -sSL https://github.com/jpetazzo/docker-busybox/raw/buildroot-2014.02/rootfs.tar | tar -xC testfs
+    cd testfs
+    cp <your-sample-config.json> container.json
+    nsinit exec sh
+
+## Contribution Guidelines
+
+### Pull requests are always welcome
+
+We are always thrilled to receive pull requests, and do our best to
+process them as fast as possible. Not sure if that typo is worth a pull
+request? Do it! We will appreciate it.
+
+If your pull request is not accepted on the first try, don't be
+discouraged! If there's a problem with the implementation, hopefully you
+received feedback on what to improve.
+
+We're trying very hard to keep libcontainer lean and focused. We don't want it
+to do everything for everybody. This means that we might decide against
+incorporating a new feature. However, there might be a way to implement
+that feature *on top of* libcontainer.
+
+### Discuss your design on the mailing list
+
+We recommend discussing your plans [on the mailing
+list](https://groups.google.com/forum/?fromgroups#!forum/libcontainer)
+before starting to code - especially for more ambitious contributions.
+This gives other contributors a chance to point you in the right
+direction, give feedback on your design, and maybe point out if someone
+else is working on the same thing.
+
+### Create issues...
+
+Any significant improvement should be documented as [a GitHub
+issue](https://github.com/docker/libcontainer/issues) before anybody
+starts working on it.
+
+### ...but check for existing issues first!
+
+Please take a moment to check that an issue doesn't already exist
+documenting your bug report or improvement proposal. If it does, it
+never hurts to add a quick "+1" or "I have this problem too". This will
+help prioritize the most common problems and requests.
+
+### Conventions
+
+Fork the repo and make changes on your fork in a feature branch:
+
+- If it's a bugfix branch, name it XXX-something where XXX is the number of the
+  issue
+- If it's a feature branch, create an enhancement issue to announce your
+  intentions, and name it XXX-something where XXX is the number of the issue.
+
+Submit unit tests for your changes.  Go has a great test framework built in; use
+it! Take a look at existing tests for inspiration. Run the full test suite on
+your branch before submitting a pull request.
+
+Update the documentation when creating or modifying features. Test
+your documentation changes for clarity, concision, and correctness, as
+well as a clean documentation build. See ``docs/README.md`` for more
+information on building the docs and how docs get released.
+
+Write clean code. Universally formatted code promotes ease of writing, reading,
+and maintenance. Always run `gofmt -s -w file.go` on each changed file before
+committing your changes. Most editors have plugins that do this automatically.
+
+Pull requests descriptions should be as clear as possible and include a
+reference to all the issues that they address.
+
+Pull requests must not contain commits from other users or branches.
+
+Commit messages must start with a capitalized and short summary (max. 50
+chars) written in the imperative, followed by an optional, more detailed
+explanatory text which is separated from the summary by an empty line.
+
+Code review comments may be added to your pull request. Discuss, then make the
+suggested modifications and push additional commits to your feature branch. Be
+sure to post a comment after pushing. The new commits will show up in the pull
+request automatically, but the reviewers will not be notified unless you
+comment.
+
+Before the pull request is merged, make sure that you squash your commits into
+logical units of work using `git rebase -i` and `git push -f`. After every
+commit the test suite should be passing. Include documentation changes in the
+same commit so that a revert would remove all traces of the feature or fix.
+
+Commits that fix or close an issue should include a reference like `Closes #XXX`
+or `Fixes #XXX`, which will automatically close the issue when merged.
+
+### Testing
+
+Make sure you include suitable tests, preferably unit tests, in your pull request
+and that all the tests pass.
+
+*Instructions for running tests to be added.*
+
+### Merge approval
+
+libcontainer maintainers use LGTM (looks good to me) in comments on the code review
+to indicate acceptance.
+
+A change requires LGTMs from at lease two maintainers. One of those must come from
+a maintainer of the component affected. For example, if a change affects `netlink/`
+and `security`, it needs at least one LGTM from a maintainer of each. Maintainers
+only need one LGTM as presumably they LGTM their own change.
+
+For more details see [MAINTAINERS.md](MAINTAINERS.md)
+
+### Sign your work
+
+The sign-off is a simple line at the end of the explanation for the
+patch, which certifies that you wrote it or otherwise have the right to
+pass it on as an open-source patch.  The rules are pretty simple: if you
+can certify the below (from
+[developercertificate.org](http://developercertificate.org/)):
+
+```
+Developer Certificate of Origin
+Version 1.1
+
+Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
+660 York Street, Suite 102,
+San Francisco, CA 94110 USA
+
+Everyone is permitted to copy and distribute verbatim copies of this
+license document, but changing it is not allowed.
+
+
+Developer's Certificate of Origin 1.1
+
+By making a contribution to this project, I certify that:
+
+(a) The contribution was created in whole or in part by me and I
+    have the right to submit it under the open source license
+    indicated in the file; or
+
+(b) The contribution is based upon previous work that, to the best
+    of my knowledge, is covered under an appropriate open source
+    license and I have the right under that license to submit that
+    work with modifications, whether created in whole or in part
+    by me, under the same open source license (unless I am
+    permitted to submit under a different license), as indicated
+    in the file; or
+
+(c) The contribution was provided directly to me by some other
+    person who certified (a), (b) or (c) and I have not modified
+    it.
+
+(d) I understand and agree that this project and the contribution
+    are public and that a record of the contribution (including all
+    personal information I submit with it, including my sign-off) is
+    maintained indefinitely and may be redistributed consistent with
+    this project or the open source license(s) involved.
+```
+
+then you just add a line to every git commit message:
+
+    Docker-DCO-1.1-Signed-off-by: Joe Smith <joe.smith@email.com> (github: github_handle)
+
+using your real name (sorry, no pseudonyms or anonymous contributions.)
+
+One way to automate this, is customise your get ``commit.template`` by adding
+a ``prepare-commit-msg`` hook to your libcontainer checkout:
+
+```
+curl -o .git/hooks/prepare-commit-msg https://raw.githubusercontent.com/docker/docker/master/contrib/prepare-commit-msg.hook && chmod +x .git/hooks/prepare-commit-msg
+```
+
+* Note: the above script expects to find your GitHub user name in ``git config --get github.user``
+
+#### Small patch exception
+
+There are several exceptions to the signing requirement. Currently these are:
+
+* Your patch fixes spelling or grammar errors.
+* Your patch is a single line change to documentation contained in the
+  `docs` directory.
+* Your patch fixes Markdown formatting or syntax errors in the
+  documentation contained in the `docs` directory.
+
+If you have any questions, please refer to the FAQ in the [docs](to be written)
+
+### How can I become a maintainer?
+
+* Step 1: learn the component inside out
+* Step 2: make yourself useful by contributing code, bugfixes, support etc.
+* Step 3: volunteer on the irc channel (#libcontainer@freenode)
+
+Don't forget: being a maintainer is a time investment. Make sure you will have time to make yourself available.
+You don't have to be a maintainer to make a difference on the project!
+

libcontainer/Dockerfile

@@ -0,0 +1,25 @@
+FROM golang:1.4
+
+RUN echo "deb http://ftp.us.debian.org/debian testing main contrib" >> /etc/apt/sources.list
+RUN apt-get update && apt-get install -y iptables criu=1.5.2-1 && rm -rf /var/lib/apt/lists/*
+
+RUN go get golang.org/x/tools/cmd/cover
+
+ENV GOPATH $GOPATH:/go/src/github.com/docker/libcontainer/vendor
+RUN go get github.com/docker/docker/pkg/term
+
+# setup a playground for us to spawn containers in
+RUN mkdir /busybox && \
+    curl -sSL 'https://github.com/jpetazzo/docker-busybox/raw/buildroot-2014.11/rootfs.tar' | tar -xC /busybox
+
+RUN curl -sSL https://raw.githubusercontent.com/docker/docker/master/hack/dind -o /dind && \
+    chmod +x /dind
+
+COPY . /go/src/github.com/docker/libcontainer
+WORKDIR /go/src/github.com/docker/libcontainer
+RUN cp sample_configs/minimal.json /busybox/container.json
+
+RUN make direct-install
+
+ENTRYPOINT ["/dind"]
+CMD ["make", "direct-test"]

libcontainer/LICENSE

@@ -0,0 +1,191 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   Copyright 2014 Docker, Inc.
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

libcontainer/MAINTAINERS

@@ -0,0 +1,7 @@
+Michael Crosby <michael@docker.com> (@crosbymichael)
+Rohit Jnagal <jnagal@google.com> (@rjnagal)
+Victor Marmol <vmarmol@google.com> (@vmarmol)
+Mrunal Patel <mpatel@redhat.com> (@mrunalp)
+Alexandr Morozov <lk4d4@docker.com> (@LK4D4)
+Daniel, Dao Quang Minh <dqminh89@gmail.com> (@dqminh)
+update-vendor.sh: Tianon Gravi <admwiggin@gmail.com> (@tianon)

libcontainer/MAINTAINERS_GUIDE.md

@@ -0,0 +1,99 @@
+# The libcontainer Maintainers' Guide
+
+## Introduction
+
+Dear maintainer. Thank you for investing the time and energy to help
+make libcontainer as useful as possible. Maintaining a project is difficult,
+sometimes unrewarding work.  Sure, you will get to contribute cool
+features to the project. But most of your time will be spent reviewing,
+cleaning up, documenting, answering questions, justifying design
+decisions - while everyone has all the fun! But remember - the quality
+of the maintainers work is what distinguishes the good projects from the
+great.  So please be proud of your work, even the unglamourous parts,
+and encourage a culture of appreciation and respect for *every* aspect
+of improving the project - not just the hot new features.
+
+This document is a manual for maintainers old and new. It explains what
+is expected of maintainers, how they should work, and what tools are
+available to them.
+
+This is a living document - if you see something out of date or missing,
+speak up!
+
+## What are a maintainer's responsibility?
+
+It is every maintainer's responsibility to:
+
+* 1) Expose a clear roadmap for improving their component.
+* 2) Deliver prompt feedback and decisions on pull requests.
+* 3) Be available to anyone with questions, bug reports, criticism etc.
+  on their component. This includes IRC, GitHub requests and the mailing
+  list.
+* 4) Make sure their component respects the philosophy, design and
+  roadmap of the project.
+
+## How are decisions made?
+
+Short answer: with pull requests to the libcontainer repository.
+
+libcontainer is an open-source project with an open design philosophy. This
+means that the repository is the source of truth for EVERY aspect of the
+project, including its philosophy, design, roadmap and APIs. *If it's
+part of the project, it's in the repo. It's in the repo, it's part of
+the project.*
+
+As a result, all decisions can be expressed as changes to the
+repository. An implementation change is a change to the source code. An
+API change is a change to the API specification. A philosophy change is
+a change to the philosophy manifesto. And so on.
+
+All decisions affecting libcontainer, big and small, follow the same 3 steps:
+
+* Step 1: Open a pull request. Anyone can do this.
+
+* Step 2: Discuss the pull request. Anyone can do this.
+
+* Step 3: Accept (`LGTM`) or refuse a pull request. The relevant maintainers do 
+this (see below "Who decides what?")
+
+
+## Who decides what?
+
+All decisions are pull requests, and the relevant maintainers make
+decisions by accepting or refusing the pull request. Review and acceptance
+by anyone is denoted by adding a comment in the pull request: `LGTM`. 
+However, only currently listed `MAINTAINERS` are counted towards the required
+two LGTMs.
+
+libcontainer follows the timeless, highly efficient and totally unfair system
+known as [Benevolent dictator for life](http://en.wikipedia.org/wiki/Benevolent_Dictator_for_Life), with Michael Crosby in the role of BDFL.
+This means that all decisions are made by default by Michael. Since making
+every decision himself would be highly un-scalable, in practice decisions
+are spread across multiple maintainers.
+
+The relevant maintainers for a pull request can be worked out in two steps:
+
+* Step 1: Determine the subdirectories affected by the pull request. This
+  might be `netlink/` and `security/`, or any other part of the repo.
+
+* Step 2: Find the `MAINTAINERS` file which affects this directory. If the
+  directory itself does not have a `MAINTAINERS` file, work your way up
+  the repo hierarchy until you find one.
+
+### I'm a maintainer, and I'm going on holiday
+
+Please let your co-maintainers and other contributors know by raising a pull
+request that comments out your `MAINTAINERS` file entry using a `#`.
+
+### I'm a maintainer, should I make pull requests too?
+
+Yes. Nobody should ever push to master directly. All changes should be
+made through a pull request.
+
+### Who assigns maintainers?
+
+Michael has final `LGTM` approval for all pull requests to `MAINTAINERS` files.
+
+### How is this process changed?
+
+Just like everything else: by making a pull request :)

libcontainer/Makefile

@@ -0,0 +1,33 @@
+
+all:
+	docker build -t dockercore/libcontainer .
+
+test: 
+	# we need NET_ADMIN for the netlink tests and SYS_ADMIN for mounting
+	docker run --rm -it --privileged dockercore/libcontainer
+
+sh:
+	docker run --rm -it --privileged -w /busybox dockercore/libcontainer nsinit exec sh
+
+GO_PACKAGES = $(shell find . -not \( -wholename ./vendor -prune -o -wholename ./.git -prune \) -name '*.go' -print0 | xargs -0n1 dirname | sort -u)
+
+direct-test:
+	go test $(TEST_TAGS) -cover -v $(GO_PACKAGES)
+
+direct-test-short:
+	go test $(TEST_TAGS) -cover -test.short -v $(GO_PACKAGES)
+
+direct-build:
+	go build -v $(GO_PACKAGES)
+
+direct-install:
+	go install -v $(GO_PACKAGES)
+
+local:
+	go test -v
+
+validate:
+	hack/validate.sh
+
+binary: all
+	docker run --rm --privileged -v $(CURDIR)/bundles:/go/bin dockercore/libcontainer make direct-install

libcontainer/NOTICE

@@ -0,0 +1,16 @@
+libcontainer
+Copyright 2012-2015 Docker, Inc.
+
+This product includes software developed at Docker, Inc. (http://www.docker.com).
+
+The following is courtesy of our legal counsel:
+
+
+Use and transfer of Docker may be subject to certain restrictions by the
+United States and other governments.  
+It is your responsibility to ensure that your use and/or transfer does not
+violate applicable laws. 
+
+For more information, please see http://www.bis.doc.gov
+
+See also http://www.apache.org/dev/crypto.html and/or seek legal counsel.

libcontainer/PRINCIPLES.md

@@ -0,0 +1,19 @@
+# libcontainer Principles
+
+In the design and development of libcontainer we try to follow these principles:
+
+(Work in progress)
+
+* Don't try to replace every tool. Instead, be an ingredient to improve them.
+* Less code is better.
+* Fewer components are better. Do you really need to add one more class?
+* 50 lines of straightforward, readable code is better than 10 lines of magic that nobody can understand.
+* Don't do later what you can do now. "//TODO: refactor" is not acceptable in new code.
+* When hesitating between two options, choose the one that is easier to reverse.
+* "No" is temporary; "Yes" is forever. If you're not sure about a new feature, say no. You can change your mind later.
+* Containers must be portable to the greatest possible number of machines. Be suspicious of any change which makes machines less interchangeable.
+* The fewer moving parts in a container, the better.
+* Don't merge it unless you document it.
+* Don't document it unless you can keep it up-to-date.
+* Don't merge it unless you test it!
+* Everyone's problem is slightly different. Focus on the part that is the same for everyone, and solve that.

libcontainer/README.md

@@ -0,0 +1,218 @@
+## libcontainer - reference implementation for containers [![Build Status](https://jenkins.dockerproject.org/buildStatus/icon?job=Libcontainer%20Master)](https://jenkins.dockerproject.org/job/Libcontainer%20Master/)
+
+Libcontainer provides a native Go implementation for creating containers
+with namespaces, cgroups, capabilities, and filesystem access controls.
+It allows you to manage the lifecycle of the container performing additional operations
+after the container is created.
+
+
+#### Container
+A container is a self contained execution environment that shares the kernel of the
+host system and which is (optionally) isolated from other containers in the system.
+
+#### Using libcontainer
+
+To create a container you first have to initialize an instance of a factory
+that will handle the creation and initialization for a container.
+
+Because containers are spawned in a two step process you will need to provide
+arguments to a binary that will be executed as the init process for the container.
+To use the current binary that is spawning the containers and acting as the parent
+you can use `os.Args[0]` and we have a command called `init` setup.
+
+```go
+root, err := libcontainer.New("/var/lib/container", libcontainer.InitArgs(os.Args[0], "init"))
+if err != nil {
+    log.Fatal(err)
+}
+```
+
+Once you have an instance of the factory created we can create a configuration
+struct describing how the container is to be created.  A sample would look similar to this:
+
+```go
+config := &configs.Config{
+    Rootfs: rootfs,
+    Capabilities: []string{
+        "CHOWN",
+        "DAC_OVERRIDE",
+        "FSETID",
+        "FOWNER",
+        "MKNOD",
+        "NET_RAW",
+        "SETGID",
+        "SETUID",
+        "SETFCAP",
+        "SETPCAP",
+        "NET_BIND_SERVICE",
+        "SYS_CHROOT",
+        "KILL",
+        "AUDIT_WRITE",
+    },
+    Namespaces: configs.Namespaces([]configs.Namespace{
+        {Type: configs.NEWNS},
+        {Type: configs.NEWUTS},
+        {Type: configs.NEWIPC},
+        {Type: configs.NEWPID},
+        {Type: configs.NEWNET},
+    }),
+    Cgroups: &configs.Cgroup{
+        Name:            "test-container",
+        Parent:          "system",
+        AllowAllDevices: false,
+        AllowedDevices:  configs.DefaultAllowedDevices,
+    },
+
+    Devices:  configs.DefaultAutoCreatedDevices,
+    Hostname: "testing",
+    Networks: []*configs.Network{
+        {
+            Type:    "loopback",
+            Address: "127.0.0.1/0",
+            Gateway: "localhost",
+        },
+    },
+    Rlimits: []configs.Rlimit{
+        {
+            Type: syscall.RLIMIT_NOFILE,
+            Hard: uint64(1024),
+            Soft: uint64(1024),
+        },
+    },
+}
+```
+
+Once you have the configuration populated you can create a container:
+
+```go
+container, err := root.Create("container-id", config)
+```
+
+To spawn bash as the initial process inside the container and have the
+processes pid returned in order to wait, signal, or kill the process:
+
+```go
+process := &libcontainer.Process{
+    Args:   []string{"/bin/bash"},
+    Env:    []string{"PATH=/bin"},
+    User:   "daemon",
+    Stdin:  os.Stdin,
+    Stdout: os.Stdout,
+    Stderr: os.Stderr,
+}
+
+err := container.Start(process)
+if err != nil {
+    log.Fatal(err)
+}
+
+// wait for the process to finish.
+status, err := process.Wait()
+if err != nil {
+    log.Fatal(err)
+}
+
+// destroy the container.
+container.Destroy()
+```
+
+Additional ways to interact with a running container are:
+
+```go
+// return all the pids for all processes running inside the container.
+processes, err := container.Processes()
+
+// get detailed cpu, memory, io, and network statistics for the container and
+// it's processes.
+stats, err := container.Stats()
+
+
+// pause all processes inside the container.
+container.Pause()
+
+// resume all paused processes.
+container.Resume()
+```
+
+
+#### nsinit
+
+`nsinit` is a cli application which demonstrates the use of libcontainer.
+It is able to spawn new containers or join existing containers.  A root
+filesystem must be provided for use along with a container configuration file.
+
+To build `nsinit`, run `make binary`. It will save the binary into
+`bundles/nsinit`.
+
+To use `nsinit`, cd into a Linux rootfs and copy a `container.json` file into
+the directory with your specified configuration. Environment, networking,
+and different capabilities for the container are specified in this file.
+The configuration is used for each process executed inside the container.
+
+See the `sample_configs` folder for examples of what the container configuration should look like.
+
+To execute `/bin/bash` in the current directory as a container just run the following **as root**:
+```bash
+nsinit exec --tty /bin/bash
+```
+
+If you wish to spawn another process inside the container while your
+current bash session is running, run the same command again to
+get another bash shell (or change the command).  If the original
+process (PID 1) dies, all other processes spawned inside the container
+will be killed and the namespace will be removed.
+
+You can identify if a process is running in a container by
+looking to see if `state.json` is in the root of the directory.
+
+You may also specify an alternate root place where
+the `container.json` file is read and where the `state.json` file will be saved.
+
+
+#### Checkpoint & Restore
+
+libcontainer now integrates [CRIU](http://criu.org/) for checkpointing and restoring containers.
+This let's you save the state of a process running inside a container to disk, and then restore
+that state into a new process, on the same machine or on another machine.
+
+`criu` version 1.5.2 or higher is required to use checkpoint and restore.
+If you don't already  have `criu` installed, you can build it from source, following the
+[online instructions](http://criu.org/Installation). `criu` is also installed in the docker image
+generated when building libcontainer with docker.
+
+To try an example with `nsinit`, open two terminals to the same busybox directory.
+In the first terminal, run a command like this one:
+```bash
+nsinit exec -- sh -c 'i=0; while true; do echo $i; i=$(expr $i + 1); sleep 1; done'
+```
+
+You should see logs printing to the terminal every second. Now, in the second terminal, run:
+```bash
+nsinit checkpoint --image-path=/tmp/criu
+```
+
+The logs in your first terminal will stop and the process will exit. Finally, in the second
+terminal, run the restore command:
+```bash
+nsinit restore --image-path=/tmp/criu
+```
+
+The process will resume counting where it left off and printing to the new terminal window.
+
+
+#### Future
+See the [roadmap](ROADMAP.md).
+
+## Copyright and license
+
+Code and documentation copyright 2014 Docker, inc. Code released under the Apache 2.0 license.
+Docs released under Creative commons.
+
+## Hacking on libcontainer
+
+First of all, please familiarise yourself with the [libcontainer Principles](PRINCIPLES.md).
+
+If you're a *contributor* or aspiring contributor, you should read the [Contributors' Guide](CONTRIBUTING.md).
+
+If you're a *maintainer* or aspiring maintainer, you should read the [Maintainers' Guide](MAINTAINERS_GUIDE.md) and
+"How can I become a maintainer?" in the Contributors' Guide.

libcontainer/ROADMAP.md

@@ -0,0 +1,20 @@
+# libcontainer: what's next?
+
+This document is a high-level overview of where we want to take libcontainer next.
+It is a curated selection of planned improvements which are either important, difficult, or both.
+
+For a more complete view of planned and requested improvements, see [the Github issues](https://github.com/docker/libcontainer/issues).
+
+To suggest changes to the roadmap, including additions, please write the change as if it were already in effect, and make a pull request.
+
+## Broader kernel support
+
+Our goal is to make libcontainer run everywhere, but currently libcontainer requires Linux version 3.8 or higher. If you’re deploying new machines for the purpose of running libcontainer, this is a fairly easy requirement to meet. However, if you’re adding libcontainer to an existing deployment, you may not have the flexibility to update and patch the kernel.
+
+## Cross-architecture support
+
+Our goal is to make libcontainer run everywhere. Recently libcontainer has
+expanded from its initial support for x86_64 systems to include POWER (ppc64
+little and big endian variants), IBM System z (s390x 64-bit), and ARM. We plan
+to continue expanding architecture support such that libcontainer containers
+can be created and used on more architectures.

libcontainer/SPEC.md

@@ -0,0 +1,334 @@
+## Container Specification - v1
+
+This is the standard configuration for version 1 containers.  It includes
+namespaces, standard filesystem setup, a default Linux capability set, and
+information about resource reservations.  It also has information about any 
+populated environment settings for the processes running inside a container.
+
+Along with the configuration of how a container is created the standard also
+discusses actions that can be performed on a container to manage and inspect
+information about the processes running inside.
+
+The v1 profile is meant to be able to accommodate the majority of applications
+with a strong security configuration.
+
+### System Requirements and Compatibility
+
+Minimum requirements:
+* Kernel version - 3.10 recommended 2.6.2x minimum(with backported patches)
+* Mounted cgroups with each subsystem in its own hierarchy
+
+
+### Namespaces
+
+|     Flag      | Enabled | 
+| ------------  | ------- |
+| CLONE_NEWPID  |    1    |
+| CLONE_NEWUTS  |    1    |
+| CLONE_NEWIPC  |    1    |
+| CLONE_NEWNET  |    1    |
+| CLONE_NEWNS   |    1    |
+| CLONE_NEWUSER |    1    |
+
+Namespaces are created for the container via the `clone` syscall.  
+
+
+### Filesystem
+
+A root filesystem must be provided to a container for execution.  The container
+will use this root filesystem (rootfs) to jail and spawn processes inside where
+the binaries and system libraries are local to that directory.  Any binaries
+to be executed must be contained within this rootfs.
+
+Mounts that happen inside the container are automatically cleaned up when the
+container exits as the mount namespace is destroyed and the kernel will 
+unmount all the mounts that were setup within that namespace.
+
+For a container to execute properly there are certain filesystems that 
+are required to be mounted within the rootfs that the runtime will setup.
+
+|     Path    |  Type  |                  Flags                 |                 Data                     |
+| ----------- | ------ | -------------------------------------- | ---------------------------------------- |
+| /proc       | proc   | MS_NOEXEC,MS_NOSUID,MS_NODEV           |                                          |
+| /dev        | tmpfs  | MS_NOEXEC,MS_STRICTATIME               | mode=755                                 |
+| /dev/shm    | tmpfs  | MS_NOEXEC,MS_NOSUID,MS_NODEV           | mode=1777,size=65536k                    |
+| /dev/mqueue | mqueue | MS_NOEXEC,MS_NOSUID,MS_NODEV           |                                          |
+| /dev/pts    | devpts | MS_NOEXEC,MS_NOSUID                    | newinstance,ptmxmode=0666,mode=620,gid=5 |
+| /sys        | sysfs  | MS_NOEXEC,MS_NOSUID,MS_NODEV,MS_RDONLY |                                          |
+
+
+After a container's filesystems are mounted within the newly created 
+mount namespace `/dev` will need to be populated with a set of device nodes.
+It is expected that a rootfs does not need to have any device nodes specified
+for `/dev` witin the rootfs as the container will setup the correct devices
+that are required for executing a container's process.
+
+|      Path    | Mode |   Access   |
+| ------------ | ---- | ---------- |
+| /dev/null    | 0666 |  rwm       |
+| /dev/zero    | 0666 |  rwm       |
+| /dev/full    | 0666 |  rwm       |
+| /dev/tty     | 0666 |  rwm       |
+| /dev/random  | 0666 |  rwm       |
+| /dev/urandom | 0666 |  rwm       |
+| /dev/fuse    | 0666 |  rwm       |
+
+
+**ptmx**
+`/dev/ptmx` will need to be a symlink to the host's `/dev/ptmx` within
+the container.  
+
+The use of a pseudo TTY is optional within a container and it should support both.
+If a pseudo is provided to the container `/dev/console` will need to be 
+setup by binding the console in `/dev/` after it has been populated and mounted
+in tmpfs.
+
+|      Source     | Destination  | UID GID | Mode | Type |
+| --------------- | ------------ | ------- | ---- | ---- |
+| *pty host path* | /dev/console | 0 0     | 0600 | bind | 
+
+
+After `/dev/null` has been setup we check for any external links between
+the container's io, STDIN, STDOUT, STDERR.  If the container's io is pointing
+to `/dev/null` outside the container we close and `dup2` the the `/dev/null` 
+that is local to the container's rootfs.
+
+
+After the container has `/proc` mounted a few standard symlinks are setup 
+within `/dev/` for the io.
+
+|    Source    | Destination |
+| ------------ | ----------- |
+| /proc/1/fd   | /dev/fd     |
+| /proc/1/fd/0 | /dev/stdin  |
+| /proc/1/fd/1 | /dev/stdout |
+| /proc/1/fd/2 | /dev/stderr |
+
+A `pivot_root` is used to change the root for the process, effectively 
+jailing the process inside the rootfs.
+
+```c
+put_old = mkdir(...);
+pivot_root(rootfs, put_old);
+chdir("/");
+unmount(put_old, MS_DETACH);
+rmdir(put_old);
+```
+
+For container's running with a rootfs inside `ramfs` a `MS_MOVE` combined
+with a `chroot` is required as `pivot_root` is not supported in `ramfs`.
+
+```c
+mount(rootfs, "/", NULL, MS_MOVE, NULL);
+chroot(".");
+chdir("/");
+```
+
+The `umask` is set back to `0022` after the filesystem setup has been completed.
+
+### Resources
+
+Cgroups are used to handle resource allocation for containers.  This includes
+system resources like cpu, memory, and device access.
+
+| Subsystem  | Enabled |
+| ---------- | ------- |
+| devices    | 1       |
+| memory     | 1       |
+| cpu        | 1       |
+| cpuacct    | 1       |
+| cpuset     | 1       |
+| blkio      | 1       |
+| perf_event | 1       |
+| freezer    | 1       |
+| hugetlb    | 1       |
+
+
+All cgroup subsystem are joined so that statistics can be collected from
+each of the subsystems.  Freezer does not expose any stats but is joined
+so that containers can be paused and resumed.
+
+The parent process of the container's init must place the init pid inside
+the correct cgroups before the initialization begins.  This is done so
+that no processes or threads escape the cgroups.  This sync is 
+done via a pipe ( specified in the runtime section below ) that the container's
+init process will block waiting for the parent to finish setup.
+
+### Security 
+
+The standard set of Linux capabilities that are set in a container
+provide a good default for security and flexibility for the applications.
+
+
+|     Capability       | Enabled |
+| -------------------- | ------- |
+| CAP_NET_RAW          | 1       |
+| CAP_NET_BIND_SERVICE | 1       |
+| CAP_AUDIT_READ       | 1       |
+| CAP_AUDIT_WRITE      | 1       |
+| CAP_DAC_OVERRIDE     | 1       |
+| CAP_SETFCAP          | 1       |
+| CAP_SETPCAP          | 1       |
+| CAP_SETGID           | 1       |
+| CAP_SETUID           | 1       |
+| CAP_MKNOD            | 1       |
+| CAP_CHOWN            | 1       |
+| CAP_FOWNER           | 1       |
+| CAP_FSETID           | 1       |
+| CAP_KILL             | 1       |
+| CAP_SYS_CHROOT       | 1       |
+| CAP_NET_BROADCAST    | 0       |
+| CAP_SYS_MODULE       | 0       |
+| CAP_SYS_RAWIO        | 0       |
+| CAP_SYS_PACCT        | 0       |
+| CAP_SYS_ADMIN        | 0       |
+| CAP_SYS_NICE         | 0       |
+| CAP_SYS_RESOURCE     | 0       |
+| CAP_SYS_TIME         | 0       |
+| CAP_SYS_TTY_CONFIG   | 0       |
+| CAP_AUDIT_CONTROL    | 0       |
+| CAP_MAC_OVERRIDE     | 0       |
+| CAP_MAC_ADMIN        | 0       |
+| CAP_NET_ADMIN        | 0       |
+| CAP_SYSLOG           | 0       |
+| CAP_DAC_READ_SEARCH  | 0       |
+| CAP_LINUX_IMMUTABLE  | 0       |
+| CAP_IPC_LOCK         | 0       |
+| CAP_IPC_OWNER        | 0       |
+| CAP_SYS_PTRACE       | 0       |
+| CAP_SYS_BOOT         | 0       |
+| CAP_LEASE            | 0       |
+| CAP_WAKE_ALARM       | 0       |
+| CAP_BLOCK_SUSPE      | 0       |
+
+
+Additional security layers like [apparmor](https://wiki.ubuntu.com/AppArmor)
+and [selinux](http://selinuxproject.org/page/Main_Page) can be used with
+the containers.  A container should support setting an apparmor profile or 
+selinux process and mount labels if provided in the configuration.  
+
+Standard apparmor profile:
+```c
+#include <tunables/global>
+profile <profile_name> flags=(attach_disconnected,mediate_deleted) {
+  #include <abstractions/base>
+  network,
+  capability,
+  file,
+  umount,
+
+  deny @{PROC}/sys/fs/** wklx,
+  deny @{PROC}/sysrq-trigger rwklx,
+  deny @{PROC}/mem rwklx,
+  deny @{PROC}/kmem rwklx,
+  deny @{PROC}/sys/kernel/[^s][^h][^m]* wklx,
+  deny @{PROC}/sys/kernel/*/** wklx,
+
+  deny mount,
+
+  deny /sys/[^f]*/** wklx,
+  deny /sys/f[^s]*/** wklx,
+  deny /sys/fs/[^c]*/** wklx,
+  deny /sys/fs/c[^g]*/** wklx,
+  deny /sys/fs/cg[^r]*/** wklx,
+  deny /sys/firmware/efi/efivars/** rwklx,
+  deny /sys/kernel/security/** rwklx,
+}
+```
+
+*TODO: seccomp work is being done to find a good default config*
+
+### Runtime and Init Process
+
+During container creation the parent process needs to talk to the container's init 
+process and have a form of synchronization.  This is accomplished by creating
+a pipe that is passed to the container's init.  When the init process first spawns 
+it will block on its side of the pipe until the parent closes its side.  This
+allows the parent to have time to set the new process inside a cgroup hierarchy 
+and/or write any uid/gid mappings required for user namespaces.  
+The pipe is passed to the init process via FD 3.
+
+The application consuming libcontainer should be compiled statically.  libcontainer
+does not define any init process and the arguments provided are used to `exec` the
+process inside the application.  There should be no long running init within the 
+container spec.
+
+If a pseudo tty is provided to a container it will open and `dup2` the console
+as the container's STDIN, STDOUT, STDERR as well as mounting the console
+as `/dev/console`.
+
+An extra set of mounts are provided to a container and setup for use.  A container's
+rootfs can contain some non portable files inside that can cause side effects during
+execution of a process.  These files are usually created and populated with the container
+specific information via the runtime.  
+
+**Extra runtime files:**
+* /etc/hosts 
+* /etc/resolv.conf
+* /etc/hostname
+* /etc/localtime
+
+
+#### Defaults
+
+There are a few defaults that can be overridden by users, but in their omission
+these apply to processes within a container.
+
+|       Type          |             Value              |
+| ------------------- | ------------------------------ |
+| Parent Death Signal | SIGKILL                        | 
+| UID                 | 0                              |
+| GID                 | 0                              |
+| GROUPS              | 0, NULL                        |
+| CWD                 | "/"                            |
+| $HOME               | Current user's home dir or "/" |
+| Readonly rootfs     | false                          |
+| Pseudo TTY          | false                          |
+
+
+## Actions
+
+After a container is created there is a standard set of actions that can
+be done to the container.  These actions are part of the public API for 
+a container.
+
+|     Action     |                         Description                                |
+| -------------- | ------------------------------------------------------------------ |
+| Get processes  | Return all the pids for processes running inside a container       | 
+| Get Stats      | Return resource statistics for the container as a whole            |
+| Wait           | Wait waits on the container's init process ( pid 1 )               |
+| Wait Process   | Wait on any of the container's processes returning the exit status | 
+| Destroy        | Kill the container's init process and remove any filesystem state  |
+| Signal         | Send a signal to the container's init process                      |
+| Signal Process | Send a signal to any of the container's processes                  |
+| Pause          | Pause all processes inside the container                           |
+| Resume         | Resume all processes inside the container if paused                |
+| Exec           | Execute a new process inside of the container  ( requires setns )  |
+| Set            | Setup configs of the container after it's created                  |
+
+### Execute a new process inside of a running container.
+
+User can execute a new process inside of a running container. Any binaries to be
+executed must be accessible within the container's rootfs.
+
+The started process will run inside the container's rootfs. Any changes
+made by the process to the container's filesystem will persist after the
+process finished executing.
+
+The started process will join all the container's existing namespaces. When the
+container is paused, the process will also be paused and will resume when
+the container is unpaused.  The started process will only run when the container's
+primary process (PID 1) is running, and will not be restarted when the container
+is restarted.
+
+#### Planned additions
+
+The started process will have its own cgroups nested inside the container's
+cgroups. This is used for process tracking and optionally resource allocation
+handling for the new process. Freezer cgroup is required, the rest of the cgroups
+are optional. The process executor must place its pid inside the correct
+cgroups before starting the process. This is done so that no child processes or
+threads can escape the cgroups.
+
+When the process is stopped, the process executor will try (in a best-effort way)
+to stop all its children and remove the sub-cgroups.

libcontainer/apparmor/apparmor.go

@@ -0,0 +1,36 @@
+// +build apparmor,linux
+
+package apparmor
+
+// #cgo LDFLAGS: -lapparmor
+// #include <sys/apparmor.h>
+// #include <stdlib.h>
+import "C"
+import (
+	"io/ioutil"
+	"os"
+	"unsafe"
+)
+
+func IsEnabled() bool {
+	if _, err := os.Stat("/sys/kernel/security/apparmor"); err == nil && os.Getenv("container") == "" {
+		if _, err = os.Stat("/sbin/apparmor_parser"); err == nil {
+			buf, err := ioutil.ReadFile("/sys/module/apparmor/parameters/enabled")
+			return err == nil && len(buf) > 1 && buf[0] == 'Y'
+		}
+	}
+	return false
+}
+
+func ApplyProfile(name string) error {
+	if name == "" {
+		return nil
+	}
+	cName := C.CString(name)
+	defer C.free(unsafe.Pointer(cName))
+
+	if _, err := C.aa_change_onexec(cName); err != nil {
+		return err
+	}
+	return nil
+}

libcontainer/apparmor/apparmor_disabled.go

@@ -0,0 +1,11 @@
+// +build !apparmor !linux
+
+package apparmor
+
+func IsEnabled() bool {
+	return false
+}
+
+func ApplyProfile(name string) error {
+	return nil
+}

libcontainer/apparmor/gen.go

@@ -0,0 +1,83 @@
+// +build linux
+
+package apparmor
+
+import (
+	"io"
+	"os"
+	"text/template"
+)
+
+type data struct {
+	Name         string
+	Imports      []string
+	InnerImports []string
+}
+
+const baseTemplate = `
+{{range $value := .Imports}}
+{{$value}}
+{{end}}
+
+profile {{.Name}} flags=(attach_disconnected,mediate_deleted) {
+{{range $value := .InnerImports}}
+  {{$value}}
+{{end}}
+
+  network,
+  capability,
+  file,
+  umount,
+
+  deny @{PROC}/sys/fs/** wklx,
+  deny @{PROC}/sysrq-trigger rwklx,
+  deny @{PROC}/mem rwklx,
+  deny @{PROC}/kmem rwklx,
+  deny @{PROC}/sys/kernel/[^s][^h][^m]* wklx,
+  deny @{PROC}/sys/kernel/*/** wklx,
+
+  deny mount,
+
+  deny /sys/[^f]*/** wklx,
+  deny /sys/f[^s]*/** wklx,
+  deny /sys/fs/[^c]*/** wklx,
+  deny /sys/fs/c[^g]*/** wklx,
+  deny /sys/fs/cg[^r]*/** wklx,
+  deny /sys/firmware/efi/efivars/** rwklx,
+  deny /sys/kernel/security/** rwklx,
+}
+`
+
+func generateProfile(out io.Writer) error {
+	compiled, err := template.New("apparmor_profile").Parse(baseTemplate)
+	if err != nil {
+		return err
+	}
+	data := &data{
+		Name: "docker-default",
+	}
+	if tunablesExists() {
+		data.Imports = append(data.Imports, "#include <tunables/global>")
+	} else {
+		data.Imports = append(data.Imports, "@{PROC}=/proc/")
+	}
+	if abstractionsExists() {
+		data.InnerImports = append(data.InnerImports, "#include <abstractions/base>")
+	}
+	if err := compiled.Execute(out, data); err != nil {
+		return err
+	}
+	return nil
+}
+
+// check if the tunables/global exist
+func tunablesExists() bool {
+	_, err := os.Stat("/etc/apparmor.d/tunables/global")
+	return err == nil
+}
+
+// check if abstractions/base exist
+func abstractionsExists() bool {
+	_, err := os.Stat("/etc/apparmor.d/abstractions/base")
+	return err == nil
+}

libcontainer/apparmor/setup.go

@@ -0,0 +1,46 @@
+// +build linux
+
+package apparmor
+
+import (
+	"fmt"
+	"os"
+	"os/exec"
+	"path"
+)
+
+const (
+	DefaultProfilePath = "/etc/apparmor.d/docker"
+)
+
+func InstallDefaultProfile() error {
+	if !IsEnabled() {
+		return nil
+	}
+
+	// Make sure /etc/apparmor.d exists
+	if err := os.MkdirAll(path.Dir(DefaultProfilePath), 0755); err != nil {
+		return err
+	}
+
+	f, err := os.OpenFile(DefaultProfilePath, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)
+	if err != nil {
+		return err
+	}
+	if err := generateProfile(f); err != nil {
+		f.Close()
+		return err
+	}
+	f.Close()
+
+	cmd := exec.Command("/sbin/apparmor_parser", "-r", "-W", "docker")
+	// to use the parser directly we have to make sure we are in the correct
+	// dir with the profile
+	cmd.Dir = "/etc/apparmor.d"
+
+	output, err := cmd.CombinedOutput()
+	if err != nil {
+		return fmt.Errorf("Error loading docker apparmor profile: %s (%s)", err, output)
+	}
+	return nil
+}

libcontainer/capabilities_linux.go

@@ -0,0 +1,91 @@
+// +build linux
+
+package libcontainer
+
+import (
+	"fmt"
+	"os"
+
+	"github.com/syndtr/gocapability/capability"
+)
+
+const allCapabilityTypes = capability.CAPS | capability.BOUNDS
+
+var capabilityList = map[string]capability.Cap{
+	"SETPCAP":          capability.CAP_SETPCAP,
+	"SYS_MODULE":       capability.CAP_SYS_MODULE,
+	"SYS_RAWIO":        capability.CAP_SYS_RAWIO,
+	"SYS_PACCT":        capability.CAP_SYS_PACCT,
+	"SYS_ADMIN":        capability.CAP_SYS_ADMIN,
+	"SYS_NICE":         capability.CAP_SYS_NICE,
+	"SYS_RESOURCE":     capability.CAP_SYS_RESOURCE,
+	"SYS_TIME":         capability.CAP_SYS_TIME,
+	"SYS_TTY_CONFIG":   capability.CAP_SYS_TTY_CONFIG,
+	"MKNOD":            capability.CAP_MKNOD,
+	"AUDIT_WRITE":      capability.CAP_AUDIT_WRITE,
+	"AUDIT_CONTROL":    capability.CAP_AUDIT_CONTROL,
+	"MAC_OVERRIDE":     capability.CAP_MAC_OVERRIDE,
+	"MAC_ADMIN":        capability.CAP_MAC_ADMIN,
+	"NET_ADMIN":        capability.CAP_NET_ADMIN,
+	"SYSLOG":           capability.CAP_SYSLOG,
+	"CHOWN":            capability.CAP_CHOWN,
+	"NET_RAW":          capability.CAP_NET_RAW,
+	"DAC_OVERRIDE":     capability.CAP_DAC_OVERRIDE,
+	"FOWNER":           capability.CAP_FOWNER,
+	"DAC_READ_SEARCH":  capability.CAP_DAC_READ_SEARCH,
+	"FSETID":           capability.CAP_FSETID,
+	"KILL":             capability.CAP_KILL,
+	"SETGID":           capability.CAP_SETGID,
+	"SETUID":           capability.CAP_SETUID,
+	"LINUX_IMMUTABLE":  capability.CAP_LINUX_IMMUTABLE,
+	"NET_BIND_SERVICE": capability.CAP_NET_BIND_SERVICE,
+	"NET_BROADCAST":    capability.CAP_NET_BROADCAST,
+	"IPC_LOCK":         capability.CAP_IPC_LOCK,
+	"IPC_OWNER":        capability.CAP_IPC_OWNER,
+	"SYS_CHROOT":       capability.CAP_SYS_CHROOT,
+	"SYS_PTRACE":       capability.CAP_SYS_PTRACE,
+	"SYS_BOOT":         capability.CAP_SYS_BOOT,
+	"LEASE":            capability.CAP_LEASE,
+	"SETFCAP":          capability.CAP_SETFCAP,
+	"WAKE_ALARM":       capability.CAP_WAKE_ALARM,
+	"BLOCK_SUSPEND":    capability.CAP_BLOCK_SUSPEND,
+	"AUDIT_READ":       capability.CAP_AUDIT_READ,
+}
+
+func newCapWhitelist(caps []string) (*whitelist, error) {
+	l := []capability.Cap{}
+	for _, c := range caps {
+		v, ok := capabilityList[c]
+		if !ok {
+			return nil, fmt.Errorf("unknown capability %q", c)
+		}
+		l = append(l, v)
+	}
+	pid, err := capability.NewPid(os.Getpid())
+	if err != nil {
+		return nil, err
+	}
+	return &whitelist{
+		keep: l,
+		pid:  pid,
+	}, nil
+}
+
+type whitelist struct {
+	pid  capability.Capabilities
+	keep []capability.Cap
+}
+
+// dropBoundingSet drops the capability bounding set to those specified in the whitelist.
+func (w *whitelist) dropBoundingSet() error {
+	w.pid.Clear(capability.BOUNDS)
+	w.pid.Set(capability.BOUNDS, w.keep...)
+	return w.pid.Apply(capability.BOUNDS)
+}
+
+// drop drops all capabilities for the current process except those specified in the whitelist.
+func (w *whitelist) drop() error {
+	w.pid.Clear(allCapabilityTypes)
+	w.pid.Set(allCapabilityTypes, w.keep...)
+	return w.pid.Apply(allCapabilityTypes)
+}

libcontainer/cgroups/cgroups.go

@@ -0,0 +1,61 @@
+// +build linux
+
+package cgroups
+
+import (
+	"fmt"
+
+	"github.com/docker/libcontainer/configs"
+)
+
+type Manager interface {
+	// Apply cgroup configuration to the process with the specified pid
+	Apply(pid int) error
+
+	// Returns the PIDs inside the cgroup set
+	GetPids() ([]int, error)
+
+	// Returns statistics for the cgroup set
+	GetStats() (*Stats, error)
+
+	// Toggles the freezer cgroup according with specified state
+	Freeze(state configs.FreezerState) error
+
+	// Destroys the cgroup set
+	Destroy() error
+
+	// NewCgroupManager() and LoadCgroupManager() require following attributes:
+	// 	Paths   map[string]string
+	// 	Cgroups *cgroups.Cgroup
+	// Paths maps cgroup subsystem to path at which it is mounted.
+	// Cgroups specifies specific cgroup settings for the various subsystems
+
+	// Returns cgroup paths to save in a state file and to be able to
+	// restore the object later.
+	GetPaths() map[string]string
+
+	// Set the cgroup as configured.
+	Set(container *configs.Config) error
+}
+
+type NotFoundError struct {
+	Subsystem string
+}
+
+func (e *NotFoundError) Error() string {
+	return fmt.Sprintf("mountpoint for %s not found", e.Subsystem)
+}
+
+func NewNotFoundError(sub string) error {
+	return &NotFoundError{
+		Subsystem: sub,
+	}
+}
+
+func IsNotFound(err error) bool {
+	if err == nil {
+		return false
+	}
+	_, ok := err.(*NotFoundError)
+	return ok
+}

libcontainer/cgroups/cgroups_test.go

@@ -0,0 +1,29 @@
+// +build linux
+
+package cgroups
+
+import (
+	"bytes"
+	"testing"
+)
+
+const (
+	cgroupsContents = `11:hugetlb:/
+10:perf_event:/
+9:blkio:/
+8:net_cls:/
+7:freezer:/
+6:devices:/
+5:memory:/
+4:cpuacct,cpu:/
+3:cpuset:/
+2:name=systemd:/user.slice/user-1000.slice/session-16.scope`
+)
+
+func TestParseCgroups(t *testing.T) {
+	r := bytes.NewBuffer([]byte(cgroupsContents))
+	_, err := ParseCgroupFile("blkio", r)
+	if err != nil {
+		t.Fatal(err)
+	}
+}

libcontainer/cgroups/cgroups_unsupported.go

@@ -0,0 +1,3 @@
+// +build !linux
+
+package cgroups

libcontainer/cgroups/fs/apply_raw.go

@@ -0,0 +1,329 @@
+// +build linux
+
+package fs
+
+import (
+	"fmt"
+	"io"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strconv"
+	"sync"
+
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+var (
+	subsystems = map[string]subsystem{
+		"devices":    &DevicesGroup{},
+		"memory":     &MemoryGroup{},
+		"cpu":        &CpuGroup{},
+		"cpuset":     &CpusetGroup{},
+		"cpuacct":    &CpuacctGroup{},
+		"blkio":      &BlkioGroup{},
+		"hugetlb":    &HugetlbGroup{},
+		"net_cls":    &NetClsGroup{},
+		"net_prio":   &NetPrioGroup{},
+		"perf_event": &PerfEventGroup{},
+		"freezer":    &FreezerGroup{},
+	}
+	CgroupProcesses  = "cgroup.procs"
+	HugePageSizes, _ = cgroups.GetHugePageSize()
+)
+
+type subsystem interface {
+	// Returns the stats, as 'stats', corresponding to the cgroup under 'path'.
+	GetStats(path string, stats *cgroups.Stats) error
+	// Removes the cgroup represented by 'data'.
+	Remove(*data) error
+	// Creates and joins the cgroup represented by data.
+	Apply(*data) error
+	// Set the cgroup represented by cgroup.
+	Set(path string, cgroup *configs.Cgroup) error
+}
+
+type Manager struct {
+	mu      sync.Mutex
+	Cgroups *configs.Cgroup
+	Paths   map[string]string
+}
+
+// The absolute path to the root of the cgroup hierarchies.
+var cgroupRootLock sync.Mutex
+var cgroupRoot string
+
+// Gets the cgroupRoot.
+func getCgroupRoot() (string, error) {
+	cgroupRootLock.Lock()
+	defer cgroupRootLock.Unlock()
+
+	if cgroupRoot != "" {
+		return cgroupRoot, nil
+	}
+
+	root, err := cgroups.FindCgroupMountpointDir()
+	if err != nil {
+		return "", err
+	}
+
+	if _, err := os.Stat(root); err != nil {
+		return "", err
+	}
+
+	cgroupRoot = root
+	return cgroupRoot, nil
+}
+
+type data struct {
+	root   string
+	cgroup string
+	c      *configs.Cgroup
+	pid    int
+}
+
+func (m *Manager) Apply(pid int) error {
+	if m.Cgroups == nil {
+		return nil
+	}
+
+	var c = m.Cgroups
+
+	d, err := getCgroupData(m.Cgroups, pid)
+	if err != nil {
+		return err
+	}
+
+	paths := make(map[string]string)
+	defer func() {
+		if err != nil {
+			cgroups.RemovePaths(paths)
+		}
+	}()
+	for name, sys := range subsystems {
+		if err := sys.Apply(d); err != nil {
+			return err
+		}
+		// TODO: Apply should, ideally, be reentrant or be broken up into a separate
+		// create and join phase so that the cgroup hierarchy for a container can be
+		// created then join consists of writing the process pids to cgroup.procs
+		p, err := d.path(name)
+		if err != nil {
+			if cgroups.IsNotFound(err) {
+				continue
+			}
+			return err
+		}
+		paths[name] = p
+	}
+	m.Paths = paths
+
+	if paths["cpu"] != "" {
+		if err := CheckCpushares(paths["cpu"], c.CpuShares); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func (m *Manager) Destroy() error {
+	m.mu.Lock()
+	defer m.mu.Unlock()
+	if err := cgroups.RemovePaths(m.Paths); err != nil {
+		return err
+	}
+	m.Paths = make(map[string]string)
+	return nil
+}
+
+func (m *Manager) GetPaths() map[string]string {
+	m.mu.Lock()
+	paths := m.Paths
+	m.mu.Unlock()
+	return paths
+}
+
+func (m *Manager) GetStats() (*cgroups.Stats, error) {
+	m.mu.Lock()
+	defer m.mu.Unlock()
+	stats := cgroups.NewStats()
+	for name, path := range m.Paths {
+		sys, ok := subsystems[name]
+		if !ok || !cgroups.PathExists(path) {
+			continue
+		}
+		if err := sys.GetStats(path, stats); err != nil {
+			return nil, err
+		}
+	}
+
+	return stats, nil
+}
+
+func (m *Manager) Set(container *configs.Config) error {
+	for name, path := range m.Paths {
+		sys, ok := subsystems[name]
+		if !ok || !cgroups.PathExists(path) {
+			continue
+		}
+		if err := sys.Set(path, container.Cgroups); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+// Freeze toggles the container's freezer cgroup depending on the state
+// provided
+func (m *Manager) Freeze(state configs.FreezerState) error {
+	d, err := getCgroupData(m.Cgroups, 0)
+	if err != nil {
+		return err
+	}
+
+	dir, err := d.path("freezer")
+	if err != nil {
+		return err
+	}
+
+	prevState := m.Cgroups.Freezer
+	m.Cgroups.Freezer = state
+
+	freezer := subsystems["freezer"]
+	err = freezer.Set(dir, m.Cgroups)
+	if err != nil {
+		m.Cgroups.Freezer = prevState
+		return err
+	}
+
+	return nil
+}
+
+func (m *Manager) GetPids() ([]int, error) {
+	d, err := getCgroupData(m.Cgroups, 0)
+	if err != nil {
+		return nil, err
+	}
+
+	dir, err := d.path("devices")
+	if err != nil {
+		return nil, err
+	}
+
+	return cgroups.ReadProcsFile(dir)
+}
+
+func getCgroupData(c *configs.Cgroup, pid int) (*data, error) {
+	root, err := getCgroupRoot()
+	if err != nil {
+		return nil, err
+	}
+
+	cgroup := c.Name
+	if c.Parent != "" {
+		cgroup = filepath.Join(c.Parent, cgroup)
+	}
+
+	return &data{
+		root:   root,
+		cgroup: cgroup,
+		c:      c,
+		pid:    pid,
+	}, nil
+}
+
+func (raw *data) parent(subsystem, mountpoint string) (string, error) {
+	initPath, err := cgroups.GetInitCgroupDir(subsystem)
+	if err != nil {
+		return "", err
+	}
+	return filepath.Join(mountpoint, initPath), nil
+}
+
+func (raw *data) path(subsystem string) (string, error) {
+	mnt, err := cgroups.FindCgroupMountpoint(subsystem)
+	// If we didn't mount the subsystem, there is no point we make the path.
+	if err != nil {
+		return "", err
+	}
+
+	// If the cgroup name/path is absolute do not look relative to the cgroup of the init process.
+	if filepath.IsAbs(raw.cgroup) {
+		return filepath.Join(raw.root, subsystem, raw.cgroup), nil
+	}
+
+	parent, err := raw.parent(subsystem, mnt)
+	if err != nil {
+		return "", err
+	}
+
+	return filepath.Join(parent, raw.cgroup), nil
+}
+
+func (raw *data) join(subsystem string) (string, error) {
+	path, err := raw.path(subsystem)
+	if err != nil {
+		return "", err
+	}
+	if err := os.MkdirAll(path, 0755); err != nil && !os.IsExist(err) {
+		return "", err
+	}
+	if err := writeFile(path, CgroupProcesses, strconv.Itoa(raw.pid)); err != nil {
+		return "", err
+	}
+	return path, nil
+}
+
+func writeFile(dir, file, data string) error {
+	// Normally dir should not be empty, one case is that cgroup subsystem
+	// is not mounted, we will get empty dir, and we want it fail here.
+	if dir == "" {
+		return fmt.Errorf("no such directory for %s.", file)
+	}
+	return ioutil.WriteFile(filepath.Join(dir, file), []byte(data), 0700)
+}
+
+func readFile(dir, file string) (string, error) {
+	data, err := ioutil.ReadFile(filepath.Join(dir, file))
+	return string(data), err
+}
+
+func removePath(p string, err error) error {
+	if err != nil {
+		return err
+	}
+	if p != "" {
+		return os.RemoveAll(p)
+	}
+	return nil
+}
+
+func CheckCpushares(path string, c int64) error {
+	var cpuShares int64
+
+	if c == 0 {
+		return nil
+	}
+
+	fd, err := os.Open(filepath.Join(path, "cpu.shares"))
+	if err != nil {
+		return err
+	}
+	defer fd.Close()
+
+	_, err = fmt.Fscanf(fd, "%d", &cpuShares)
+	if err != nil && err != io.EOF {
+		return err
+	}
+
+	if c > cpuShares {
+		return fmt.Errorf("The maximum allowed cpu-shares is %d", cpuShares)
+	} else if c < cpuShares {
+		return fmt.Errorf("The minimum allowed cpu-shares is %d", cpuShares)
+	}
+
+	return nil
+}

libcontainer/cgroups/fs/blkio.go

@@ -0,0 +1,230 @@
+// +build linux
+
+package fs
+
+import (
+	"bufio"
+	"fmt"
+	"os"
+	"path/filepath"
+	"strconv"
+	"strings"
+
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type BlkioGroup struct {
+}
+
+func (s *BlkioGroup) Apply(d *data) error {
+	dir, err := d.join("blkio")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	if err := s.Set(dir, d.c); err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func (s *BlkioGroup) Set(path string, cgroup *configs.Cgroup) error {
+	if cgroup.BlkioWeight != 0 {
+		if err := writeFile(path, "blkio.weight", strconv.FormatInt(cgroup.BlkioWeight, 10)); err != nil {
+			return err
+		}
+	}
+
+	if cgroup.BlkioWeightDevice != "" {
+		if err := writeFile(path, "blkio.weight_device", cgroup.BlkioWeightDevice); err != nil {
+			return err
+		}
+	}
+	if cgroup.BlkioThrottleReadBpsDevice != "" {
+		if err := writeFile(path, "blkio.throttle.read_bps_device", cgroup.BlkioThrottleReadBpsDevice); err != nil {
+			return err
+		}
+	}
+	if cgroup.BlkioThrottleWriteBpsDevice != "" {
+		if err := writeFile(path, "blkio.throttle.write_bps_device", cgroup.BlkioThrottleWriteBpsDevice); err != nil {
+			return err
+		}
+	}
+	if cgroup.BlkioThrottleReadIOpsDevice != "" {
+		if err := writeFile(path, "blkio.throttle.read_iops_device", cgroup.BlkioThrottleReadIOpsDevice); err != nil {
+			return err
+		}
+	}
+	if cgroup.BlkioThrottleWriteIOpsDevice != "" {
+		if err := writeFile(path, "blkio.throttle.write_iops_device", cgroup.BlkioThrottleWriteIOpsDevice); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func (s *BlkioGroup) Remove(d *data) error {
+	return removePath(d.path("blkio"))
+}
+
+/*
+examples:
+
+    blkio.sectors
+    8:0 6792
+
+    blkio.io_service_bytes
+    8:0 Read 1282048
+    8:0 Write 2195456
+    8:0 Sync 2195456
+    8:0 Async 1282048
+    8:0 Total 3477504
+    Total 3477504
+
+    blkio.io_serviced
+    8:0 Read 124
+    8:0 Write 104
+    8:0 Sync 104
+    8:0 Async 124
+    8:0 Total 228
+    Total 228
+
+    blkio.io_queued
+    8:0 Read 0
+    8:0 Write 0
+    8:0 Sync 0
+    8:0 Async 0
+    8:0 Total 0
+    Total 0
+*/
+
+func splitBlkioStatLine(r rune) bool {
+	return r == ' ' || r == ':'
+}
+
+func getBlkioStat(path string) ([]cgroups.BlkioStatEntry, error) {
+	var blkioStats []cgroups.BlkioStatEntry
+	f, err := os.Open(path)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return blkioStats, nil
+		}
+		return nil, err
+	}
+	defer f.Close()
+
+	sc := bufio.NewScanner(f)
+	for sc.Scan() {
+		// format: dev type amount
+		fields := strings.FieldsFunc(sc.Text(), splitBlkioStatLine)
+		if len(fields) < 3 {
+			if len(fields) == 2 && fields[0] == "Total" {
+				// skip total line
+				continue
+			} else {
+				return nil, fmt.Errorf("Invalid line found while parsing %s: %s", path, sc.Text())
+			}
+		}
+
+		v, err := strconv.ParseUint(fields[0], 10, 64)
+		if err != nil {
+			return nil, err
+		}
+		major := v
+
+		v, err = strconv.ParseUint(fields[1], 10, 64)
+		if err != nil {
+			return nil, err
+		}
+		minor := v
+
+		op := ""
+		valueField := 2
+		if len(fields) == 4 {
+			op = fields[2]
+			valueField = 3
+		}
+		v, err = strconv.ParseUint(fields[valueField], 10, 64)
+		if err != nil {
+			return nil, err
+		}
+		blkioStats = append(blkioStats, cgroups.BlkioStatEntry{Major: major, Minor: minor, Op: op, Value: v})
+	}
+
+	return blkioStats, nil
+}
+
+func (s *BlkioGroup) GetStats(path string, stats *cgroups.Stats) error {
+	// Try to read CFQ stats available on all CFQ enabled kernels first
+	if blkioStats, err := getBlkioStat(filepath.Join(path, "blkio.io_serviced_recursive")); err == nil && blkioStats != nil {
+		return getCFQStats(path, stats)
+	}
+	return getStats(path, stats) // Use generic stats as fallback
+}
+
+func getCFQStats(path string, stats *cgroups.Stats) error {
+	var blkioStats []cgroups.BlkioStatEntry
+	var err error
+
+	if blkioStats, err = getBlkioStat(filepath.Join(path, "blkio.sectors_recursive")); err != nil {
+		return err
+	}
+	stats.BlkioStats.SectorsRecursive = blkioStats
+
+	if blkioStats, err = getBlkioStat(filepath.Join(path, "blkio.io_service_bytes_recursive")); err != nil {
+		return err
+	}
+	stats.BlkioStats.IoServiceBytesRecursive = blkioStats
+
+	if blkioStats, err = getBlkioStat(filepath.Join(path, "blkio.io_serviced_recursive")); err != nil {
+		return err
+	}
+	stats.BlkioStats.IoServicedRecursive = blkioStats
+
+	if blkioStats, err = getBlkioStat(filepath.Join(path, "blkio.io_queued_recursive")); err != nil {
+		return err
+	}
+	stats.BlkioStats.IoQueuedRecursive = blkioStats
+
+	if blkioStats, err = getBlkioStat(filepath.Join(path, "blkio.io_service_time_recursive")); err != nil {
+		return err
+	}
+	stats.BlkioStats.IoServiceTimeRecursive = blkioStats
+
+	if blkioStats, err = getBlkioStat(filepath.Join(path, "blkio.io_wait_time_recursive")); err != nil {
+		return err
+	}
+	stats.BlkioStats.IoWaitTimeRecursive = blkioStats
+
+	if blkioStats, err = getBlkioStat(filepath.Join(path, "blkio.io_merged_recursive")); err != nil {
+		return err
+	}
+	stats.BlkioStats.IoMergedRecursive = blkioStats
+
+	if blkioStats, err = getBlkioStat(filepath.Join(path, "blkio.time_recursive")); err != nil {
+		return err
+	}
+	stats.BlkioStats.IoTimeRecursive = blkioStats
+
+	return nil
+}
+
+func getStats(path string, stats *cgroups.Stats) error {
+	var blkioStats []cgroups.BlkioStatEntry
+	var err error
+
+	if blkioStats, err = getBlkioStat(filepath.Join(path, "blkio.throttle.io_service_bytes")); err != nil {
+		return err
+	}
+	stats.BlkioStats.IoServiceBytesRecursive = blkioStats
+
+	if blkioStats, err = getBlkioStat(filepath.Join(path, "blkio.throttle.io_serviced")); err != nil {
+		return err
+	}
+	stats.BlkioStats.IoServicedRecursive = blkioStats
+
+	return nil
+}

libcontainer/cgroups/fs/blkio_test.go

@@ -0,0 +1,570 @@
+// +build linux
+
+package fs
+
+import (
+	"strconv"
+	"testing"
+
+	"github.com/docker/libcontainer/cgroups"
+)
+
+const (
+	sectorsRecursiveContents      = `8:0 1024`
+	serviceBytesRecursiveContents = `8:0 Read 100
+8:0 Write 200
+8:0 Sync 300
+8:0 Async 500
+8:0 Total 500
+Total 500`
+	servicedRecursiveContents = `8:0 Read 10
+8:0 Write 40
+8:0 Sync 20
+8:0 Async 30
+8:0 Total 50
+Total 50`
+	queuedRecursiveContents = `8:0 Read 1
+8:0 Write 4
+8:0 Sync 2
+8:0 Async 3
+8:0 Total 5
+Total 5`
+	serviceTimeRecursiveContents = `8:0 Read 173959
+8:0 Write 0
+8:0 Sync 0
+8:0 Async 173959
+8:0 Total 17395
+Total 17395`
+	waitTimeRecursiveContents = `8:0 Read 15571
+8:0 Write 0
+8:0 Sync 0
+8:0 Async 15571
+8:0 Total 15571`
+	mergedRecursiveContents = `8:0 Read 5
+8:0 Write 10
+8:0 Sync 0
+8:0 Async 0
+8:0 Total 15
+Total 15`
+	timeRecursiveContents = `8:0 8`
+	throttleServiceBytes  = `8:0 Read 11030528
+8:0 Write 23
+8:0 Sync 42
+8:0 Async 11030528
+8:0 Total 11030528
+252:0 Read 11030528
+252:0 Write 23
+252:0 Sync 42
+252:0 Async 11030528
+252:0 Total 11030528
+Total 22061056`
+	throttleServiced = `8:0 Read 164
+8:0 Write 23
+8:0 Sync 42
+8:0 Async 164
+8:0 Total 164
+252:0 Read 164
+252:0 Write 23
+252:0 Sync 42
+252:0 Async 164
+252:0 Total 164
+Total 328`
+	throttleBefore = `8:0 1024`
+	throttleAfter  = `8:0 2048`
+)
+
+func appendBlkioStatEntry(blkioStatEntries *[]cgroups.BlkioStatEntry, major, minor, value uint64, op string) {
+	*blkioStatEntries = append(*blkioStatEntries, cgroups.BlkioStatEntry{Major: major, Minor: minor, Value: value, Op: op})
+}
+
+func TestBlkioSetWeight(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+
+	const (
+		weightBefore = 100
+		weightAfter  = 200
+	)
+
+	helper.writeFileContents(map[string]string{
+		"blkio.weight": strconv.Itoa(weightBefore),
+	})
+
+	helper.CgroupData.c.BlkioWeight = weightAfter
+	blkio := &BlkioGroup{}
+	if err := blkio.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamUint(helper.CgroupPath, "blkio.weight")
+	if err != nil {
+		t.Fatalf("Failed to parse blkio.weight - %s", err)
+	}
+
+	if value != weightAfter {
+		t.Fatal("Got the wrong value, set blkio.weight failed.")
+	}
+}
+
+func TestBlkioSetWeightDevice(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+
+	const (
+		weightDeviceBefore = "8:0 400"
+		weightDeviceAfter  = "8:0 500"
+	)
+
+	helper.writeFileContents(map[string]string{
+		"blkio.weight_device": weightDeviceBefore,
+	})
+
+	helper.CgroupData.c.BlkioWeightDevice = weightDeviceAfter
+	blkio := &BlkioGroup{}
+	if err := blkio.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamString(helper.CgroupPath, "blkio.weight_device")
+	if err != nil {
+		t.Fatalf("Failed to parse blkio.weight_device - %s", err)
+	}
+
+	if value != weightDeviceAfter {
+		t.Fatal("Got the wrong value, set blkio.weight_device failed.")
+	}
+}
+
+func TestBlkioStats(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_service_bytes_recursive": serviceBytesRecursiveContents,
+		"blkio.io_serviced_recursive":      servicedRecursiveContents,
+		"blkio.io_queued_recursive":        queuedRecursiveContents,
+		"blkio.io_service_time_recursive":  serviceTimeRecursiveContents,
+		"blkio.io_wait_time_recursive":     waitTimeRecursiveContents,
+		"blkio.io_merged_recursive":        mergedRecursiveContents,
+		"blkio.time_recursive":             timeRecursiveContents,
+		"blkio.sectors_recursive":          sectorsRecursiveContents,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// Verify expected stats.
+	expectedStats := cgroups.BlkioStats{}
+	appendBlkioStatEntry(&expectedStats.SectorsRecursive, 8, 0, 1024, "")
+
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 8, 0, 100, "Read")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 8, 0, 200, "Write")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 8, 0, 300, "Sync")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 8, 0, 500, "Async")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 8, 0, 500, "Total")
+
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 8, 0, 10, "Read")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 8, 0, 40, "Write")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 8, 0, 20, "Sync")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 8, 0, 30, "Async")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 8, 0, 50, "Total")
+
+	appendBlkioStatEntry(&expectedStats.IoQueuedRecursive, 8, 0, 1, "Read")
+	appendBlkioStatEntry(&expectedStats.IoQueuedRecursive, 8, 0, 4, "Write")
+	appendBlkioStatEntry(&expectedStats.IoQueuedRecursive, 8, 0, 2, "Sync")
+	appendBlkioStatEntry(&expectedStats.IoQueuedRecursive, 8, 0, 3, "Async")
+	appendBlkioStatEntry(&expectedStats.IoQueuedRecursive, 8, 0, 5, "Total")
+
+	appendBlkioStatEntry(&expectedStats.IoServiceTimeRecursive, 8, 0, 173959, "Read")
+	appendBlkioStatEntry(&expectedStats.IoServiceTimeRecursive, 8, 0, 0, "Write")
+	appendBlkioStatEntry(&expectedStats.IoServiceTimeRecursive, 8, 0, 0, "Sync")
+	appendBlkioStatEntry(&expectedStats.IoServiceTimeRecursive, 8, 0, 173959, "Async")
+	appendBlkioStatEntry(&expectedStats.IoServiceTimeRecursive, 8, 0, 17395, "Total")
+
+	appendBlkioStatEntry(&expectedStats.IoWaitTimeRecursive, 8, 0, 15571, "Read")
+	appendBlkioStatEntry(&expectedStats.IoWaitTimeRecursive, 8, 0, 0, "Write")
+	appendBlkioStatEntry(&expectedStats.IoWaitTimeRecursive, 8, 0, 0, "Sync")
+	appendBlkioStatEntry(&expectedStats.IoWaitTimeRecursive, 8, 0, 15571, "Async")
+	appendBlkioStatEntry(&expectedStats.IoWaitTimeRecursive, 8, 0, 15571, "Total")
+
+	appendBlkioStatEntry(&expectedStats.IoMergedRecursive, 8, 0, 5, "Read")
+	appendBlkioStatEntry(&expectedStats.IoMergedRecursive, 8, 0, 10, "Write")
+	appendBlkioStatEntry(&expectedStats.IoMergedRecursive, 8, 0, 0, "Sync")
+	appendBlkioStatEntry(&expectedStats.IoMergedRecursive, 8, 0, 0, "Async")
+	appendBlkioStatEntry(&expectedStats.IoMergedRecursive, 8, 0, 15, "Total")
+
+	appendBlkioStatEntry(&expectedStats.IoTimeRecursive, 8, 0, 8, "")
+
+	expectBlkioStatsEquals(t, expectedStats, actualStats.BlkioStats)
+}
+
+func TestBlkioStatsNoSectorsFile(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_service_bytes_recursive": serviceBytesRecursiveContents,
+		"blkio.io_serviced_recursive":      servicedRecursiveContents,
+		"blkio.io_queued_recursive":        queuedRecursiveContents,
+		"blkio.io_service_time_recursive":  serviceTimeRecursiveContents,
+		"blkio.io_wait_time_recursive":     waitTimeRecursiveContents,
+		"blkio.io_merged_recursive":        mergedRecursiveContents,
+		"blkio.time_recursive":             timeRecursiveContents,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatalf("Failed unexpectedly: %s", err)
+	}
+}
+
+func TestBlkioStatsNoServiceBytesFile(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_serviced_recursive":     servicedRecursiveContents,
+		"blkio.io_queued_recursive":       queuedRecursiveContents,
+		"blkio.sectors_recursive":         sectorsRecursiveContents,
+		"blkio.io_service_time_recursive": serviceTimeRecursiveContents,
+		"blkio.io_wait_time_recursive":    waitTimeRecursiveContents,
+		"blkio.io_merged_recursive":       mergedRecursiveContents,
+		"blkio.time_recursive":            timeRecursiveContents,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatalf("Failed unexpectedly: %s", err)
+	}
+}
+
+func TestBlkioStatsNoServicedFile(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_service_bytes_recursive": serviceBytesRecursiveContents,
+		"blkio.io_queued_recursive":        queuedRecursiveContents,
+		"blkio.sectors_recursive":          sectorsRecursiveContents,
+		"blkio.io_service_time_recursive":  serviceTimeRecursiveContents,
+		"blkio.io_wait_time_recursive":     waitTimeRecursiveContents,
+		"blkio.io_merged_recursive":        mergedRecursiveContents,
+		"blkio.time_recursive":             timeRecursiveContents,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatalf("Failed unexpectedly: %s", err)
+	}
+}
+
+func TestBlkioStatsNoQueuedFile(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_service_bytes_recursive": serviceBytesRecursiveContents,
+		"blkio.io_serviced_recursive":      servicedRecursiveContents,
+		"blkio.sectors_recursive":          sectorsRecursiveContents,
+		"blkio.io_service_time_recursive":  serviceTimeRecursiveContents,
+		"blkio.io_wait_time_recursive":     waitTimeRecursiveContents,
+		"blkio.io_merged_recursive":        mergedRecursiveContents,
+		"blkio.time_recursive":             timeRecursiveContents,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatalf("Failed unexpectedly: %s", err)
+	}
+}
+
+func TestBlkioStatsNoServiceTimeFile(t *testing.T) {
+	if testing.Short() {
+		t.Skip("skipping test in short mode.")
+	}
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_service_bytes_recursive": serviceBytesRecursiveContents,
+		"blkio.io_serviced_recursive":      servicedRecursiveContents,
+		"blkio.io_queued_recursive":        queuedRecursiveContents,
+		"blkio.io_wait_time_recursive":     waitTimeRecursiveContents,
+		"blkio.io_merged_recursive":        mergedRecursiveContents,
+		"blkio.time_recursive":             timeRecursiveContents,
+		"blkio.sectors_recursive":          sectorsRecursiveContents,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatalf("Failed unexpectedly: %s", err)
+	}
+}
+
+func TestBlkioStatsNoWaitTimeFile(t *testing.T) {
+	if testing.Short() {
+		t.Skip("skipping test in short mode.")
+	}
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_service_bytes_recursive": serviceBytesRecursiveContents,
+		"blkio.io_serviced_recursive":      servicedRecursiveContents,
+		"blkio.io_queued_recursive":        queuedRecursiveContents,
+		"blkio.io_service_time_recursive":  serviceTimeRecursiveContents,
+		"blkio.io_merged_recursive":        mergedRecursiveContents,
+		"blkio.time_recursive":             timeRecursiveContents,
+		"blkio.sectors_recursive":          sectorsRecursiveContents,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatalf("Failed unexpectedly: %s", err)
+	}
+}
+
+func TestBlkioStatsNoMergedFile(t *testing.T) {
+	if testing.Short() {
+		t.Skip("skipping test in short mode.")
+	}
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_service_bytes_recursive": serviceBytesRecursiveContents,
+		"blkio.io_serviced_recursive":      servicedRecursiveContents,
+		"blkio.io_queued_recursive":        queuedRecursiveContents,
+		"blkio.io_service_time_recursive":  serviceTimeRecursiveContents,
+		"blkio.io_wait_time_recursive":     waitTimeRecursiveContents,
+		"blkio.time_recursive":             timeRecursiveContents,
+		"blkio.sectors_recursive":          sectorsRecursiveContents,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatalf("Failed unexpectedly: %s", err)
+	}
+}
+
+func TestBlkioStatsNoTimeFile(t *testing.T) {
+	if testing.Short() {
+		t.Skip("skipping test in short mode.")
+	}
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_service_bytes_recursive": serviceBytesRecursiveContents,
+		"blkio.io_serviced_recursive":      servicedRecursiveContents,
+		"blkio.io_queued_recursive":        queuedRecursiveContents,
+		"blkio.io_service_time_recursive":  serviceTimeRecursiveContents,
+		"blkio.io_wait_time_recursive":     waitTimeRecursiveContents,
+		"blkio.io_merged_recursive":        mergedRecursiveContents,
+		"blkio.sectors_recursive":          sectorsRecursiveContents,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatalf("Failed unexpectedly: %s", err)
+	}
+}
+
+func TestBlkioStatsUnexpectedNumberOfFields(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_service_bytes_recursive": "8:0 Read 100 100",
+		"blkio.io_serviced_recursive":      servicedRecursiveContents,
+		"blkio.io_queued_recursive":        queuedRecursiveContents,
+		"blkio.sectors_recursive":          sectorsRecursiveContents,
+		"blkio.io_service_time_recursive":  serviceTimeRecursiveContents,
+		"blkio.io_wait_time_recursive":     waitTimeRecursiveContents,
+		"blkio.io_merged_recursive":        mergedRecursiveContents,
+		"blkio.time_recursive":             timeRecursiveContents,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected to fail, but did not")
+	}
+}
+
+func TestBlkioStatsUnexpectedFieldType(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_service_bytes_recursive": "8:0 Read Write",
+		"blkio.io_serviced_recursive":      servicedRecursiveContents,
+		"blkio.io_queued_recursive":        queuedRecursiveContents,
+		"blkio.sectors_recursive":          sectorsRecursiveContents,
+		"blkio.io_service_time_recursive":  serviceTimeRecursiveContents,
+		"blkio.io_wait_time_recursive":     waitTimeRecursiveContents,
+		"blkio.io_merged_recursive":        mergedRecursiveContents,
+		"blkio.time_recursive":             timeRecursiveContents,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected to fail, but did not")
+	}
+}
+
+func TestNonCFQBlkioStats(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"blkio.io_service_bytes_recursive": "",
+		"blkio.io_serviced_recursive":      "",
+		"blkio.io_queued_recursive":        "",
+		"blkio.sectors_recursive":          "",
+		"blkio.io_service_time_recursive":  "",
+		"blkio.io_wait_time_recursive":     "",
+		"blkio.io_merged_recursive":        "",
+		"blkio.time_recursive":             "",
+		"blkio.throttle.io_service_bytes":  throttleServiceBytes,
+		"blkio.throttle.io_serviced":       throttleServiced,
+	})
+
+	blkio := &BlkioGroup{}
+	actualStats := *cgroups.NewStats()
+	err := blkio.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// Verify expected stats.
+	expectedStats := cgroups.BlkioStats{}
+
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 8, 0, 11030528, "Read")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 8, 0, 23, "Write")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 8, 0, 42, "Sync")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 8, 0, 11030528, "Async")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 8, 0, 11030528, "Total")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 252, 0, 11030528, "Read")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 252, 0, 23, "Write")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 252, 0, 42, "Sync")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 252, 0, 11030528, "Async")
+	appendBlkioStatEntry(&expectedStats.IoServiceBytesRecursive, 252, 0, 11030528, "Total")
+
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 8, 0, 164, "Read")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 8, 0, 23, "Write")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 8, 0, 42, "Sync")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 8, 0, 164, "Async")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 8, 0, 164, "Total")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 252, 0, 164, "Read")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 252, 0, 23, "Write")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 252, 0, 42, "Sync")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 252, 0, 164, "Async")
+	appendBlkioStatEntry(&expectedStats.IoServicedRecursive, 252, 0, 164, "Total")
+
+	expectBlkioStatsEquals(t, expectedStats, actualStats.BlkioStats)
+}
+
+func TestBlkioSetThrottleReadBpsDevice(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+
+	helper.writeFileContents(map[string]string{
+		"blkio.throttle.read_bps_device": throttleBefore,
+	})
+
+	helper.CgroupData.c.BlkioThrottleReadBpsDevice = throttleAfter
+	blkio := &BlkioGroup{}
+	if err := blkio.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamString(helper.CgroupPath, "blkio.throttle.read_bps_device")
+	if err != nil {
+		t.Fatalf("Failed to parse blkio.throttle.read_bps_device - %s", err)
+	}
+
+	if value != throttleAfter {
+		t.Fatal("Got the wrong value, set blkio.throttle.read_bps_device failed.")
+	}
+}
+func TestBlkioSetThrottleWriteBpsDevice(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+
+	helper.writeFileContents(map[string]string{
+		"blkio.throttle.write_bps_device": throttleBefore,
+	})
+
+	helper.CgroupData.c.BlkioThrottleWriteBpsDevice = throttleAfter
+	blkio := &BlkioGroup{}
+	if err := blkio.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamString(helper.CgroupPath, "blkio.throttle.write_bps_device")
+	if err != nil {
+		t.Fatalf("Failed to parse blkio.throttle.write_bps_device - %s", err)
+	}
+
+	if value != throttleAfter {
+		t.Fatal("Got the wrong value, set blkio.throttle.write_bps_device failed.")
+	}
+}
+func TestBlkioSetThrottleReadIOpsDevice(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+
+	helper.writeFileContents(map[string]string{
+		"blkio.throttle.read_iops_device": throttleBefore,
+	})
+
+	helper.CgroupData.c.BlkioThrottleReadIOpsDevice = throttleAfter
+	blkio := &BlkioGroup{}
+	if err := blkio.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamString(helper.CgroupPath, "blkio.throttle.read_iops_device")
+	if err != nil {
+		t.Fatalf("Failed to parse blkio.throttle.read_iops_device - %s", err)
+	}
+
+	if value != throttleAfter {
+		t.Fatal("Got the wrong value, set blkio.throttle.read_iops_device failed.")
+	}
+}
+func TestBlkioSetThrottleWriteIOpsDevice(t *testing.T) {
+	helper := NewCgroupTestUtil("blkio", t)
+	defer helper.cleanup()
+
+	helper.writeFileContents(map[string]string{
+		"blkio.throttle.write_iops_device": throttleBefore,
+	})
+
+	helper.CgroupData.c.BlkioThrottleWriteIOpsDevice = throttleAfter
+	blkio := &BlkioGroup{}
+	if err := blkio.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamString(helper.CgroupPath, "blkio.throttle.write_iops_device")
+	if err != nil {
+		t.Fatalf("Failed to parse blkio.throttle.write_iops_device - %s", err)
+	}
+
+	if value != throttleAfter {
+		t.Fatal("Got the wrong value, set blkio.throttle.write_iops_device failed.")
+	}
+}

libcontainer/cgroups/fs/cpu.go

@@ -0,0 +1,95 @@
+// +build linux
+
+package fs
+
+import (
+	"bufio"
+	"os"
+	"path/filepath"
+	"strconv"
+
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type CpuGroup struct {
+}
+
+func (s *CpuGroup) Apply(d *data) error {
+	// We always want to join the cpu group, to allow fair cpu scheduling
+	// on a container basis
+	dir, err := d.join("cpu")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	if err := s.Set(dir, d.c); err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func (s *CpuGroup) Set(path string, cgroup *configs.Cgroup) error {
+	if cgroup.CpuShares != 0 {
+		if err := writeFile(path, "cpu.shares", strconv.FormatInt(cgroup.CpuShares, 10)); err != nil {
+			return err
+		}
+	}
+	if cgroup.CpuPeriod != 0 {
+		if err := writeFile(path, "cpu.cfs_period_us", strconv.FormatInt(cgroup.CpuPeriod, 10)); err != nil {
+			return err
+		}
+	}
+	if cgroup.CpuQuota != 0 {
+		if err := writeFile(path, "cpu.cfs_quota_us", strconv.FormatInt(cgroup.CpuQuota, 10)); err != nil {
+			return err
+		}
+	}
+	if cgroup.CpuRtPeriod != 0 {
+		if err := writeFile(path, "cpu.rt_period_us", strconv.FormatInt(cgroup.CpuRtPeriod, 10)); err != nil {
+			return err
+		}
+	}
+	if cgroup.CpuRtRuntime != 0 {
+		if err := writeFile(path, "cpu.rt_runtime_us", strconv.FormatInt(cgroup.CpuRtRuntime, 10)); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func (s *CpuGroup) Remove(d *data) error {
+	return removePath(d.path("cpu"))
+}
+
+func (s *CpuGroup) GetStats(path string, stats *cgroups.Stats) error {
+	f, err := os.Open(filepath.Join(path, "cpu.stat"))
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil
+		}
+		return err
+	}
+	defer f.Close()
+
+	sc := bufio.NewScanner(f)
+	for sc.Scan() {
+		t, v, err := getCgroupParamKeyValue(sc.Text())
+		if err != nil {
+			return err
+		}
+		switch t {
+		case "nr_periods":
+			stats.CpuStats.ThrottlingData.Periods = v
+
+		case "nr_throttled":
+			stats.CpuStats.ThrottlingData.ThrottledPeriods = v
+
+		case "throttled_time":
+			stats.CpuStats.ThrottlingData.ThrottledTime = v
+		}
+	}
+	return nil
+}

libcontainer/cgroups/fs/cpu_test.go

@@ -0,0 +1,163 @@
+// +build linux
+
+package fs
+
+import (
+	"fmt"
+	"strconv"
+	"testing"
+
+	"github.com/docker/libcontainer/cgroups"
+)
+
+func TestCpuSetShares(t *testing.T) {
+	helper := NewCgroupTestUtil("cpu", t)
+	defer helper.cleanup()
+
+	const (
+		sharesBefore = 1024
+		sharesAfter  = 512
+	)
+
+	helper.writeFileContents(map[string]string{
+		"cpu.shares": strconv.Itoa(sharesBefore),
+	})
+
+	helper.CgroupData.c.CpuShares = sharesAfter
+	cpu := &CpuGroup{}
+	if err := cpu.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamUint(helper.CgroupPath, "cpu.shares")
+	if err != nil {
+		t.Fatalf("Failed to parse cpu.shares - %s", err)
+	}
+
+	if value != sharesAfter {
+		t.Fatal("Got the wrong value, set cpu.shares failed.")
+	}
+}
+
+func TestCpuSetBandWidth(t *testing.T) {
+	helper := NewCgroupTestUtil("cpu", t)
+	defer helper.cleanup()
+
+	const (
+		quotaBefore     = 8000
+		quotaAfter      = 5000
+		periodBefore    = 10000
+		periodAfter     = 7000
+		rtRuntimeBefore = 8000
+		rtRuntimeAfter  = 5000
+		rtPeriodBefore  = 10000
+		rtPeriodAfter   = 7000
+	)
+
+	helper.writeFileContents(map[string]string{
+		"cpu.cfs_quota_us":  strconv.Itoa(quotaBefore),
+		"cpu.cfs_period_us": strconv.Itoa(periodBefore),
+		"cpu.rt_runtime_us": strconv.Itoa(rtRuntimeBefore),
+		"cpu.rt_period_us":  strconv.Itoa(rtPeriodBefore),
+	})
+
+	helper.CgroupData.c.CpuQuota = quotaAfter
+	helper.CgroupData.c.CpuPeriod = periodAfter
+	helper.CgroupData.c.CpuRtRuntime = rtRuntimeAfter
+	helper.CgroupData.c.CpuRtPeriod = rtPeriodAfter
+	cpu := &CpuGroup{}
+	if err := cpu.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	quota, err := getCgroupParamUint(helper.CgroupPath, "cpu.cfs_quota_us")
+	if err != nil {
+		t.Fatalf("Failed to parse cpu.cfs_quota_us - %s", err)
+	}
+	if quota != quotaAfter {
+		t.Fatal("Got the wrong value, set cpu.cfs_quota_us failed.")
+	}
+
+	period, err := getCgroupParamUint(helper.CgroupPath, "cpu.cfs_period_us")
+	if err != nil {
+		t.Fatalf("Failed to parse cpu.cfs_period_us - %s", err)
+	}
+	if period != periodAfter {
+		t.Fatal("Got the wrong value, set cpu.cfs_period_us failed.")
+	}
+	rtRuntime, err := getCgroupParamUint(helper.CgroupPath, "cpu.rt_runtime_us")
+	if err != nil {
+		t.Fatalf("Failed to parse cpu.rt_runtime_us - %s", err)
+	}
+	if rtRuntime != rtRuntimeAfter {
+		t.Fatal("Got the wrong value, set cpu.rt_runtime_us failed.")
+	}
+	rtPeriod, err := getCgroupParamUint(helper.CgroupPath, "cpu.rt_period_us")
+	if err != nil {
+		t.Fatalf("Failed to parse cpu.rt_period_us - %s", err)
+	}
+	if rtPeriod != rtPeriodAfter {
+		t.Fatal("Got the wrong value, set cpu.rt_period_us failed.")
+	}
+}
+
+func TestCpuStats(t *testing.T) {
+	helper := NewCgroupTestUtil("cpu", t)
+	defer helper.cleanup()
+
+	const (
+		kNrPeriods     = 2000
+		kNrThrottled   = 200
+		kThrottledTime = uint64(18446744073709551615)
+	)
+
+	cpuStatContent := fmt.Sprintf("nr_periods %d\n nr_throttled %d\n throttled_time %d\n",
+		kNrPeriods, kNrThrottled, kThrottledTime)
+	helper.writeFileContents(map[string]string{
+		"cpu.stat": cpuStatContent,
+	})
+
+	cpu := &CpuGroup{}
+	actualStats := *cgroups.NewStats()
+	err := cpu.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	expectedStats := cgroups.ThrottlingData{
+		Periods:          kNrPeriods,
+		ThrottledPeriods: kNrThrottled,
+		ThrottledTime:    kThrottledTime}
+
+	expectThrottlingDataEquals(t, expectedStats, actualStats.CpuStats.ThrottlingData)
+}
+
+func TestNoCpuStatFile(t *testing.T) {
+	helper := NewCgroupTestUtil("cpu", t)
+	defer helper.cleanup()
+
+	cpu := &CpuGroup{}
+	actualStats := *cgroups.NewStats()
+	err := cpu.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatal("Expected not to fail, but did")
+	}
+}
+
+func TestInvalidCpuStat(t *testing.T) {
+	helper := NewCgroupTestUtil("cpu", t)
+	defer helper.cleanup()
+	cpuStatContent := `nr_periods 2000
+	nr_throttled 200
+	throttled_time fortytwo`
+	helper.writeFileContents(map[string]string{
+		"cpu.stat": cpuStatContent,
+	})
+
+	cpu := &CpuGroup{}
+	actualStats := *cgroups.NewStats()
+	err := cpu.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected failed stat parsing.")
+	}
+}

libcontainer/cgroups/fs/cpuacct.go

@@ -0,0 +1,117 @@
+// +build linux
+
+package fs
+
+import (
+	"fmt"
+	"io/ioutil"
+	"path/filepath"
+	"strconv"
+	"strings"
+
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+	"github.com/docker/libcontainer/system"
+)
+
+const (
+	cgroupCpuacctStat   = "cpuacct.stat"
+	nanosecondsInSecond = 1000000000
+)
+
+var clockTicks = uint64(system.GetClockTicks())
+
+type CpuacctGroup struct {
+}
+
+func (s *CpuacctGroup) Apply(d *data) error {
+	// we just want to join this group even though we don't set anything
+	if _, err := d.join("cpuacct"); err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	return nil
+}
+
+func (s *CpuacctGroup) Set(path string, cgroup *configs.Cgroup) error {
+	return nil
+}
+
+func (s *CpuacctGroup) Remove(d *data) error {
+	return removePath(d.path("cpuacct"))
+}
+
+func (s *CpuacctGroup) GetStats(path string, stats *cgroups.Stats) error {
+	userModeUsage, kernelModeUsage, err := getCpuUsageBreakdown(path)
+	if err != nil {
+		return err
+	}
+
+	totalUsage, err := getCgroupParamUint(path, "cpuacct.usage")
+	if err != nil {
+		return err
+	}
+
+	percpuUsage, err := getPercpuUsage(path)
+	if err != nil {
+		return err
+	}
+
+	stats.CpuStats.CpuUsage.TotalUsage = totalUsage
+	stats.CpuStats.CpuUsage.PercpuUsage = percpuUsage
+	stats.CpuStats.CpuUsage.UsageInUsermode = userModeUsage
+	stats.CpuStats.CpuUsage.UsageInKernelmode = kernelModeUsage
+	return nil
+}
+
+// Returns user and kernel usage breakdown in nanoseconds.
+func getCpuUsageBreakdown(path string) (uint64, uint64, error) {
+	userModeUsage := uint64(0)
+	kernelModeUsage := uint64(0)
+	const (
+		userField   = "user"
+		systemField = "system"
+	)
+
+	// Expected format:
+	// user <usage in ticks>
+	// system <usage in ticks>
+	data, err := ioutil.ReadFile(filepath.Join(path, cgroupCpuacctStat))
+	if err != nil {
+		return 0, 0, err
+	}
+	fields := strings.Fields(string(data))
+	if len(fields) != 4 {
+		return 0, 0, fmt.Errorf("failure - %s is expected to have 4 fields", filepath.Join(path, cgroupCpuacctStat))
+	}
+	if fields[0] != userField {
+		return 0, 0, fmt.Errorf("unexpected field %q in %q, expected %q", fields[0], cgroupCpuacctStat, userField)
+	}
+	if fields[2] != systemField {
+		return 0, 0, fmt.Errorf("unexpected field %q in %q, expected %q", fields[2], cgroupCpuacctStat, systemField)
+	}
+	if userModeUsage, err = strconv.ParseUint(fields[1], 10, 64); err != nil {
+		return 0, 0, err
+	}
+	if kernelModeUsage, err = strconv.ParseUint(fields[3], 10, 64); err != nil {
+		return 0, 0, err
+	}
+
+	return (userModeUsage * nanosecondsInSecond) / clockTicks, (kernelModeUsage * nanosecondsInSecond) / clockTicks, nil
+}
+
+func getPercpuUsage(path string) ([]uint64, error) {
+	percpuUsage := []uint64{}
+	data, err := ioutil.ReadFile(filepath.Join(path, "cpuacct.usage_percpu"))
+	if err != nil {
+		return percpuUsage, err
+	}
+	for _, value := range strings.Fields(string(data)) {
+		value, err := strconv.ParseUint(value, 10, 64)
+		if err != nil {
+			return percpuUsage, fmt.Errorf("Unable to convert param value to uint64: %s", err)
+		}
+		percpuUsage = append(percpuUsage, value)
+	}
+	return percpuUsage, nil
+}

libcontainer/cgroups/fs/cpuset.go

@@ -0,0 +1,135 @@
+// +build linux
+
+package fs
+
+import (
+	"bytes"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strconv"
+
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type CpusetGroup struct {
+}
+
+func (s *CpusetGroup) Apply(d *data) error {
+	dir, err := d.path("cpuset")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+	return s.ApplyDir(dir, d.c, d.pid)
+}
+
+func (s *CpusetGroup) Set(path string, cgroup *configs.Cgroup) error {
+	if cgroup.CpusetCpus != "" {
+		if err := writeFile(path, "cpuset.cpus", cgroup.CpusetCpus); err != nil {
+			return err
+		}
+	}
+	if cgroup.CpusetMems != "" {
+		if err := writeFile(path, "cpuset.mems", cgroup.CpusetMems); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func (s *CpusetGroup) Remove(d *data) error {
+	return removePath(d.path("cpuset"))
+}
+
+func (s *CpusetGroup) GetStats(path string, stats *cgroups.Stats) error {
+	return nil
+}
+
+func (s *CpusetGroup) ApplyDir(dir string, cgroup *configs.Cgroup, pid int) error {
+	// This might happen if we have no cpuset cgroup mounted.
+	// Just do nothing and don't fail.
+	if dir == "" {
+		return nil
+	}
+	root, err := getCgroupRoot()
+	if err != nil {
+		return err
+	}
+	if err := s.ensureParent(dir, root); err != nil {
+		return err
+	}
+	// because we are not using d.join we need to place the pid into the procs file
+	// unlike the other subsystems
+	if err := writeFile(dir, "cgroup.procs", strconv.Itoa(pid)); err != nil {
+		return err
+	}
+
+	// the default values inherit from parent cgroup are already set in
+	// s.ensureParent, cover these if we have our own
+	if err := s.Set(dir, cgroup); err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func (s *CpusetGroup) getSubsystemSettings(parent string) (cpus []byte, mems []byte, err error) {
+	if cpus, err = ioutil.ReadFile(filepath.Join(parent, "cpuset.cpus")); err != nil {
+		return
+	}
+	if mems, err = ioutil.ReadFile(filepath.Join(parent, "cpuset.mems")); err != nil {
+		return
+	}
+	return cpus, mems, nil
+}
+
+// ensureParent makes sure that the parent directory of current is created
+// and populated with the proper cpus and mems files copied from
+// it's parent.
+func (s *CpusetGroup) ensureParent(current, root string) error {
+	parent := filepath.Dir(current)
+	if filepath.Clean(parent) == root {
+		return nil
+	}
+	if err := s.ensureParent(parent, root); err != nil {
+		return err
+	}
+	if err := os.MkdirAll(current, 0755); err != nil && !os.IsExist(err) {
+		return err
+	}
+	return s.copyIfNeeded(current, parent)
+}
+
+// copyIfNeeded copies the cpuset.cpus and cpuset.mems from the parent
+// directory to the current directory if the file's contents are 0
+func (s *CpusetGroup) copyIfNeeded(current, parent string) error {
+	var (
+		err                      error
+		currentCpus, currentMems []byte
+		parentCpus, parentMems   []byte
+	)
+
+	if currentCpus, currentMems, err = s.getSubsystemSettings(current); err != nil {
+		return err
+	}
+	if parentCpus, parentMems, err = s.getSubsystemSettings(parent); err != nil {
+		return err
+	}
+
+	if s.isEmpty(currentCpus) {
+		if err := writeFile(current, "cpuset.cpus", string(parentCpus)); err != nil {
+			return err
+		}
+	}
+	if s.isEmpty(currentMems) {
+		if err := writeFile(current, "cpuset.mems", string(parentMems)); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func (s *CpusetGroup) isEmpty(b []byte) bool {
+	return len(bytes.Trim(b, "\n")) == 0
+}

libcontainer/cgroups/fs/cpuset_test.go

@@ -0,0 +1,65 @@
+// +build linux
+
+package fs
+
+import (
+	"testing"
+)
+
+func TestCpusetSetCpus(t *testing.T) {
+	helper := NewCgroupTestUtil("cpuset", t)
+	defer helper.cleanup()
+
+	const (
+		cpusBefore = "0"
+		cpusAfter  = "1-3"
+	)
+
+	helper.writeFileContents(map[string]string{
+		"cpuset.cpus": cpusBefore,
+	})
+
+	helper.CgroupData.c.CpusetCpus = cpusAfter
+	cpuset := &CpusetGroup{}
+	if err := cpuset.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamString(helper.CgroupPath, "cpuset.cpus")
+	if err != nil {
+		t.Fatalf("Failed to parse cpuset.cpus - %s", err)
+	}
+
+	if value != cpusAfter {
+		t.Fatal("Got the wrong value, set cpuset.cpus failed.")
+	}
+}
+
+func TestCpusetSetMems(t *testing.T) {
+	helper := NewCgroupTestUtil("cpuset", t)
+	defer helper.cleanup()
+
+	const (
+		memsBefore = "0"
+		memsAfter  = "1"
+	)
+
+	helper.writeFileContents(map[string]string{
+		"cpuset.mems": memsBefore,
+	})
+
+	helper.CgroupData.c.CpusetMems = memsAfter
+	cpuset := &CpusetGroup{}
+	if err := cpuset.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamString(helper.CgroupPath, "cpuset.mems")
+	if err != nil {
+		t.Fatalf("Failed to parse cpuset.mems - %s", err)
+	}
+
+	if value != memsAfter {
+		t.Fatal("Got the wrong value, set cpuset.mems failed.")
+	}
+}

libcontainer/cgroups/fs/devices.go

@@ -0,0 +1,61 @@
+// +build linux
+
+package fs
+
+import (
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type DevicesGroup struct {
+}
+
+func (s *DevicesGroup) Apply(d *data) error {
+	dir, err := d.join("devices")
+	if err != nil {
+		// We will return error even it's `not found` error, devices
+		// cgroup is hard requirement for container's security.
+		return err
+	}
+
+	if err := s.Set(dir, d.c); err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func (s *DevicesGroup) Set(path string, cgroup *configs.Cgroup) error {
+	if !cgroup.AllowAllDevices {
+		if err := writeFile(path, "devices.deny", "a"); err != nil {
+			return err
+		}
+
+		for _, dev := range cgroup.AllowedDevices {
+			if err := writeFile(path, "devices.allow", dev.CgroupString()); err != nil {
+				return err
+			}
+		}
+		return nil
+	}
+
+	if err := writeFile(path, "devices.allow", "a"); err != nil {
+		return err
+	}
+
+	for _, dev := range cgroup.DeniedDevices {
+		if err := writeFile(path, "devices.deny", dev.CgroupString()); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func (s *DevicesGroup) Remove(d *data) error {
+	return removePath(d.path("devices"))
+}
+
+func (s *DevicesGroup) GetStats(path string, stats *cgroups.Stats) error {
+	return nil
+}

libcontainer/cgroups/fs/devices_test.go

@@ -0,0 +1,84 @@
+// +build linux
+
+package fs
+
+import (
+	"testing"
+
+	"github.com/docker/libcontainer/configs"
+)
+
+var (
+	allowedDevices = []*configs.Device{
+		{
+			Path:        "/dev/zero",
+			Type:        'c',
+			Major:       1,
+			Minor:       5,
+			Permissions: "rwm",
+			FileMode:    0666,
+		},
+	}
+	allowedList   = "c 1:5 rwm"
+	deniedDevices = []*configs.Device{
+		{
+			Path:        "/dev/null",
+			Type:        'c',
+			Major:       1,
+			Minor:       3,
+			Permissions: "rwm",
+			FileMode:    0666,
+		},
+	}
+	deniedList = "c 1:3 rwm"
+)
+
+func TestDevicesSetAllow(t *testing.T) {
+	helper := NewCgroupTestUtil("devices", t)
+	defer helper.cleanup()
+
+	helper.writeFileContents(map[string]string{
+		"devices.deny": "a",
+	})
+
+	helper.CgroupData.c.AllowAllDevices = false
+	helper.CgroupData.c.AllowedDevices = allowedDevices
+	devices := &DevicesGroup{}
+	if err := devices.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamString(helper.CgroupPath, "devices.allow")
+	if err != nil {
+		t.Fatalf("Failed to parse devices.allow - %s", err)
+	}
+
+	if value != allowedList {
+		t.Fatal("Got the wrong value, set devices.allow failed.")
+	}
+}
+
+func TestDevicesSetDeny(t *testing.T) {
+	helper := NewCgroupTestUtil("devices", t)
+	defer helper.cleanup()
+
+	helper.writeFileContents(map[string]string{
+		"devices.allow": "a",
+	})
+
+	helper.CgroupData.c.AllowAllDevices = true
+	helper.CgroupData.c.DeniedDevices = deniedDevices
+	devices := &DevicesGroup{}
+	if err := devices.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamString(helper.CgroupPath, "devices.deny")
+	if err != nil {
+		t.Fatalf("Failed to parse devices.deny - %s", err)
+	}
+
+	if value != deniedList {
+		t.Fatal("Got the wrong value, set devices.deny failed.")
+	}
+}

libcontainer/cgroups/fs/freezer.go

@@ -0,0 +1,62 @@
+// +build linux
+
+package fs
+
+import (
+	"fmt"
+	"strings"
+	"time"
+
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type FreezerGroup struct {
+}
+
+func (s *FreezerGroup) Apply(d *data) error {
+	dir, err := d.join("freezer")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	if err := s.Set(dir, d.c); err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func (s *FreezerGroup) Set(path string, cgroup *configs.Cgroup) error {
+	switch cgroup.Freezer {
+	case configs.Frozen, configs.Thawed:
+		if err := writeFile(path, "freezer.state", string(cgroup.Freezer)); err != nil {
+			return err
+		}
+
+		for {
+			state, err := readFile(path, "freezer.state")
+			if err != nil {
+				return err
+			}
+			if strings.TrimSpace(state) == string(cgroup.Freezer) {
+				break
+			}
+			time.Sleep(1 * time.Millisecond)
+		}
+	case configs.Undefined:
+		return nil
+	default:
+		return fmt.Errorf("Invalid argument '%s' to freezer.state", string(cgroup.Freezer))
+	}
+
+	return nil
+}
+
+func (s *FreezerGroup) Remove(d *data) error {
+	return removePath(d.path("freezer"))
+}
+
+func (s *FreezerGroup) GetStats(path string, stats *cgroups.Stats) error {
+	return nil
+}

libcontainer/cgroups/fs/freezer_test.go

@@ -0,0 +1,45 @@
+package fs
+
+import (
+	"testing"
+
+	"github.com/docker/libcontainer/configs"
+)
+
+func TestFreezerSetState(t *testing.T) {
+	helper := NewCgroupTestUtil("freezer", t)
+	defer helper.cleanup()
+
+	helper.writeFileContents(map[string]string{
+		"freezer.state": string(configs.Frozen),
+	})
+
+	helper.CgroupData.c.Freezer = configs.Thawed
+	freezer := &FreezerGroup{}
+	if err := freezer.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamString(helper.CgroupPath, "freezer.state")
+	if err != nil {
+		t.Fatalf("Failed to parse freezer.state - %s", err)
+	}
+	if value != string(configs.Thawed) {
+		t.Fatal("Got the wrong value, set freezer.state failed.")
+	}
+}
+
+func TestFreezerSetInvalidState(t *testing.T) {
+	helper := NewCgroupTestUtil("freezer", t)
+	defer helper.cleanup()
+
+	const (
+		invalidArg configs.FreezerState = "Invalid"
+	)
+
+	helper.CgroupData.c.Freezer = invalidArg
+	freezer := &FreezerGroup{}
+	if err := freezer.Set(helper.CgroupPath, helper.CgroupData.c); err == nil {
+		t.Fatal("Failed to return invalid argument error")
+	}
+}

libcontainer/cgroups/fs/fs_unsupported.go

@@ -0,0 +1,3 @@
+// +build !linux
+
+package fs

libcontainer/cgroups/fs/hugetlb.go

@@ -0,0 +1,72 @@
+// +build linux
+
+package fs
+
+import (
+	"fmt"
+	"strconv"
+	"strings"
+
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type HugetlbGroup struct {
+}
+
+func (s *HugetlbGroup) Apply(d *data) error {
+	dir, err := d.join("hugetlb")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	if err := s.Set(dir, d.c); err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func (s *HugetlbGroup) Set(path string, cgroup *configs.Cgroup) error {
+	for _, hugetlb := range cgroup.HugetlbLimit {
+		if err := writeFile(path, strings.Join([]string{"hugetlb", hugetlb.Pagesize, "limit_in_bytes"}, "."), strconv.Itoa(hugetlb.Limit)); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func (s *HugetlbGroup) Remove(d *data) error {
+	return removePath(d.path("hugetlb"))
+}
+
+func (s *HugetlbGroup) GetStats(path string, stats *cgroups.Stats) error {
+	hugetlbStats := cgroups.HugetlbStats{}
+	for _, pageSize := range HugePageSizes {
+		usage := strings.Join([]string{"hugetlb", pageSize, "usage_in_bytes"}, ".")
+		value, err := getCgroupParamUint(path, usage)
+		if err != nil {
+			return fmt.Errorf("failed to parse %s - %v", usage, err)
+		}
+		hugetlbStats.Usage = value
+
+		maxUsage := strings.Join([]string{"hugetlb", pageSize, "max_usage_in_bytes"}, ".")
+		value, err = getCgroupParamUint(path, maxUsage)
+		if err != nil {
+			return fmt.Errorf("failed to parse %s - %v", maxUsage, err)
+		}
+		hugetlbStats.MaxUsage = value
+
+		failcnt := strings.Join([]string{"hugetlb", pageSize, "failcnt"}, ".")
+		value, err = getCgroupParamUint(path, failcnt)
+		if err != nil {
+			return fmt.Errorf("failed to parse %s - %v", failcnt, err)
+		}
+		hugetlbStats.Failcnt = value
+
+		stats.HugetlbStats[pageSize] = hugetlbStats
+	}
+
+	return nil
+}

libcontainer/cgroups/fs/hugetlb_test.go

@@ -0,0 +1,138 @@
+package fs
+
+import (
+	"strconv"
+	"strings"
+	"testing"
+
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+const (
+	hugetlbUsageContents    = "128\n"
+	hugetlbMaxUsageContents = "256\n"
+	hugetlbFailcnt          = "100\n"
+)
+
+var (
+	hugePageSize, _ = cgroups.GetHugePageSize()
+	usage           = strings.Join([]string{"hugetlb", hugePageSize[0], "usage_in_bytes"}, ".")
+	limit           = strings.Join([]string{"hugetlb", hugePageSize[0], "limit_in_bytes"}, ".")
+	maxUsage        = strings.Join([]string{"hugetlb", hugePageSize[0], "max_usage_in_bytes"}, ".")
+	failcnt         = strings.Join([]string{"hugetlb", hugePageSize[0], "failcnt"}, ".")
+)
+
+func TestHugetlbSetHugetlb(t *testing.T) {
+	helper := NewCgroupTestUtil("hugetlb", t)
+	defer helper.cleanup()
+
+	const (
+		hugetlbBefore = 256
+		hugetlbAfter  = 512
+	)
+
+	helper.writeFileContents(map[string]string{
+		limit: strconv.Itoa(hugetlbBefore),
+	})
+
+	helper.CgroupData.c.HugetlbLimit = []*configs.HugepageLimit{
+		{
+			Pagesize: hugePageSize[0],
+			Limit:    hugetlbAfter,
+		},
+	}
+	hugetlb := &HugetlbGroup{}
+	if err := hugetlb.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamUint(helper.CgroupPath, limit)
+	if err != nil {
+		t.Fatalf("Failed to parse %s - %s", limit, err)
+	}
+	if value != hugetlbAfter {
+		t.Fatalf("Set hugetlb.limit_in_bytes failed. Expected: %v, Got: %v", hugetlbAfter, value)
+	}
+}
+
+func TestHugetlbStats(t *testing.T) {
+	helper := NewCgroupTestUtil("hugetlb", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		usage:    hugetlbUsageContents,
+		maxUsage: hugetlbMaxUsageContents,
+		failcnt:  hugetlbFailcnt,
+	})
+
+	hugetlb := &HugetlbGroup{}
+	actualStats := *cgroups.NewStats()
+	err := hugetlb.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatal(err)
+	}
+	expectedStats := cgroups.HugetlbStats{Usage: 128, MaxUsage: 256, Failcnt: 100}
+	expectHugetlbStatEquals(t, expectedStats, actualStats.HugetlbStats[hugePageSize[0]])
+}
+
+func TestHugetlbStatsNoUsageFile(t *testing.T) {
+	helper := NewCgroupTestUtil("hugetlb", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		maxUsage: hugetlbMaxUsageContents,
+	})
+
+	hugetlb := &HugetlbGroup{}
+	actualStats := *cgroups.NewStats()
+	err := hugetlb.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected failure")
+	}
+}
+
+func TestHugetlbStatsNoMaxUsageFile(t *testing.T) {
+	helper := NewCgroupTestUtil("hugetlb", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		usage: hugetlbUsageContents,
+	})
+
+	hugetlb := &HugetlbGroup{}
+	actualStats := *cgroups.NewStats()
+	err := hugetlb.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected failure")
+	}
+}
+
+func TestHugetlbStatsBadUsageFile(t *testing.T) {
+	helper := NewCgroupTestUtil("hugetlb", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		usage:    "bad",
+		maxUsage: hugetlbMaxUsageContents,
+	})
+
+	hugetlb := &HugetlbGroup{}
+	actualStats := *cgroups.NewStats()
+	err := hugetlb.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected failure")
+	}
+}
+
+func TestHugetlbStatsBadMaxUsageFile(t *testing.T) {
+	helper := NewCgroupTestUtil("hugetlb", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		usage:    hugetlbUsageContents,
+		maxUsage: "bad",
+	})
+
+	hugetlb := &HugetlbGroup{}
+	actualStats := *cgroups.NewStats()
+	err := hugetlb.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected failure")
+	}
+}

libcontainer/cgroups/fs/memory.go

@@ -0,0 +1,164 @@
+// +build linux
+
+package fs
+
+import (
+	"bufio"
+	"fmt"
+	"os"
+	"path/filepath"
+	"strconv"
+	"strings"
+
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type MemoryGroup struct {
+}
+
+func (s *MemoryGroup) Apply(d *data) error {
+	path, err := d.path("memory")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+	if err := os.MkdirAll(path, 0755); err != nil && !os.IsExist(err) {
+		return err
+	}
+	if err := s.Set(path, d.c); err != nil {
+		return err
+	}
+
+	// We need to join memory cgroup after set memory limits, because
+	// kmem.limit_in_bytes can only be set when the cgroup is empty.
+	_, err = d.join("memory")
+	if err != nil {
+		return err
+	}
+	defer func() {
+		if err != nil {
+			os.RemoveAll(path)
+		}
+	}()
+
+	return nil
+}
+
+func (s *MemoryGroup) Set(path string, cgroup *configs.Cgroup) error {
+	if cgroup.Memory != 0 {
+		if err := writeFile(path, "memory.limit_in_bytes", strconv.FormatInt(cgroup.Memory, 10)); err != nil {
+			return err
+		}
+	}
+	if cgroup.MemoryReservation != 0 {
+		if err := writeFile(path, "memory.soft_limit_in_bytes", strconv.FormatInt(cgroup.MemoryReservation, 10)); err != nil {
+			return err
+		}
+	}
+	if cgroup.MemorySwap > 0 {
+		if err := writeFile(path, "memory.memsw.limit_in_bytes", strconv.FormatInt(cgroup.MemorySwap, 10)); err != nil {
+			return err
+		}
+	}
+	if cgroup.KernelMemory > 0 {
+		if err := writeFile(path, "memory.kmem.limit_in_bytes", strconv.FormatInt(cgroup.KernelMemory, 10)); err != nil {
+			return err
+		}
+	}
+
+	if cgroup.OomKillDisable {
+		if err := writeFile(path, "memory.oom_control", "1"); err != nil {
+			return err
+		}
+	}
+	if cgroup.MemorySwappiness >= 0 && cgroup.MemorySwappiness <= 100 {
+		if err := writeFile(path, "memory.swappiness", strconv.FormatInt(cgroup.MemorySwappiness, 10)); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func (s *MemoryGroup) Remove(d *data) error {
+	return removePath(d.path("memory"))
+}
+
+func (s *MemoryGroup) GetStats(path string, stats *cgroups.Stats) error {
+	// Set stats from memory.stat.
+	statsFile, err := os.Open(filepath.Join(path, "memory.stat"))
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil
+		}
+		return err
+	}
+	defer statsFile.Close()
+
+	sc := bufio.NewScanner(statsFile)
+	for sc.Scan() {
+		t, v, err := getCgroupParamKeyValue(sc.Text())
+		if err != nil {
+			return fmt.Errorf("failed to parse memory.stat (%q) - %v", sc.Text(), err)
+		}
+		stats.MemoryStats.Stats[t] = v
+	}
+	stats.MemoryStats.Cache = stats.MemoryStats.Stats["cache"]
+
+	memoryUsage, err := getMemoryData(path, "")
+	if err != nil {
+		return err
+	}
+	stats.MemoryStats.Usage = memoryUsage
+	swapUsage, err := getMemoryData(path, "memsw")
+	if err != nil {
+		return err
+	}
+	stats.MemoryStats.SwapUsage = swapUsage
+	kernelUsage, err := getMemoryData(path, "kmem")
+	if err != nil {
+		return err
+	}
+	stats.MemoryStats.KernelUsage = kernelUsage
+
+	return nil
+}
+
+func getMemoryData(path, name string) (cgroups.MemoryData, error) {
+	memoryData := cgroups.MemoryData{}
+
+	moduleName := "memory"
+	if name != "" {
+		moduleName = strings.Join([]string{"memory", name}, ".")
+	}
+	usage := strings.Join([]string{moduleName, "usage_in_bytes"}, ".")
+	maxUsage := strings.Join([]string{moduleName, "max_usage_in_bytes"}, ".")
+	failcnt := strings.Join([]string{moduleName, "failcnt"}, ".")
+
+	value, err := getCgroupParamUint(path, usage)
+	if err != nil {
+		if moduleName != "memory" && os.IsNotExist(err) {
+			return cgroups.MemoryData{}, nil
+		}
+		return cgroups.MemoryData{}, fmt.Errorf("failed to parse %s - %v", usage, err)
+	}
+	memoryData.Usage = value
+	value, err = getCgroupParamUint(path, maxUsage)
+	if err != nil {
+		if moduleName != "memory" && os.IsNotExist(err) {
+			return cgroups.MemoryData{}, nil
+		}
+		return cgroups.MemoryData{}, fmt.Errorf("failed to parse %s - %v", maxUsage, err)
+	}
+	memoryData.MaxUsage = value
+	value, err = getCgroupParamUint(path, failcnt)
+	if err != nil {
+		if moduleName != "memory" && os.IsNotExist(err) {
+			return cgroups.MemoryData{}, nil
+		}
+		return cgroups.MemoryData{}, fmt.Errorf("failed to parse %s - %v", failcnt, err)
+	}
+	memoryData.Failcnt = value
+
+	return memoryData, nil
+}

libcontainer/cgroups/fs/memory_test.go

@@ -0,0 +1,294 @@
+// +build linux
+
+package fs
+
+import (
+	"strconv"
+	"testing"
+
+	"github.com/docker/libcontainer/cgroups"
+)
+
+const (
+	memoryStatContents = `cache 512
+rss 1024`
+	memoryUsageContents    = "2048\n"
+	memoryMaxUsageContents = "4096\n"
+	memoryFailcnt          = "100\n"
+)
+
+func TestMemorySetMemory(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+
+	const (
+		memoryBefore      = 314572800 // 300M
+		memoryAfter       = 524288000 // 500M
+		reservationBefore = 209715200 // 200M
+		reservationAfter  = 314572800 // 300M
+	)
+
+	helper.writeFileContents(map[string]string{
+		"memory.limit_in_bytes":      strconv.Itoa(memoryBefore),
+		"memory.soft_limit_in_bytes": strconv.Itoa(reservationBefore),
+	})
+
+	helper.CgroupData.c.Memory = memoryAfter
+	helper.CgroupData.c.MemoryReservation = reservationAfter
+	memory := &MemoryGroup{}
+	if err := memory.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamUint(helper.CgroupPath, "memory.limit_in_bytes")
+	if err != nil {
+		t.Fatalf("Failed to parse memory.limit_in_bytes - %s", err)
+	}
+	if value != memoryAfter {
+		t.Fatal("Got the wrong value, set memory.limit_in_bytes failed.")
+	}
+
+	value, err = getCgroupParamUint(helper.CgroupPath, "memory.soft_limit_in_bytes")
+	if err != nil {
+		t.Fatalf("Failed to parse memory.soft_limit_in_bytes - %s", err)
+	}
+	if value != reservationAfter {
+		t.Fatal("Got the wrong value, set memory.soft_limit_in_bytes failed.")
+	}
+}
+
+func TestMemorySetMemoryswap(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+
+	const (
+		memoryswapBefore = 314572800 // 300M
+		memoryswapAfter  = 524288000 // 500M
+	)
+
+	helper.writeFileContents(map[string]string{
+		"memory.memsw.limit_in_bytes": strconv.Itoa(memoryswapBefore),
+	})
+
+	helper.CgroupData.c.MemorySwap = memoryswapAfter
+	memory := &MemoryGroup{}
+	if err := memory.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamUint(helper.CgroupPath, "memory.memsw.limit_in_bytes")
+	if err != nil {
+		t.Fatalf("Failed to parse memory.memsw.limit_in_bytes - %s", err)
+	}
+	if value != memoryswapAfter {
+		t.Fatal("Got the wrong value, set memory.memsw.limit_in_bytes failed.")
+	}
+}
+
+func TestMemorySetKernelMemory(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+
+	const (
+		kernelMemoryBefore = 314572800 // 300M
+		kernelMemoryAfter  = 524288000 // 500M
+	)
+
+	helper.writeFileContents(map[string]string{
+		"memory.kmem.limit_in_bytes": strconv.Itoa(kernelMemoryBefore),
+	})
+
+	helper.CgroupData.c.KernelMemory = kernelMemoryAfter
+	memory := &MemoryGroup{}
+	if err := memory.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamUint(helper.CgroupPath, "memory.kmem.limit_in_bytes")
+	if err != nil {
+		t.Fatalf("Failed to parse memory.kmem.limit_in_bytes - %s", err)
+	}
+	if value != kernelMemoryAfter {
+		t.Fatal("Got the wrong value, set memory.kmem.limit_in_bytes failed.")
+	}
+}
+
+func TestMemorySetMemorySwappinessDefault(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+
+	const (
+		swappinessBefore = 60 //deafult is 60
+		swappinessAfter  = 0
+	)
+
+	helper.writeFileContents(map[string]string{
+		"memory.swappiness": strconv.Itoa(swappinessBefore),
+	})
+
+	helper.CgroupData.c.Memory = swappinessAfter
+	memory := &MemoryGroup{}
+	if err := memory.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamUint(helper.CgroupPath, "memory.swappiness")
+	if err != nil {
+		t.Fatalf("Failed to parse memory.swappiness - %s", err)
+	}
+	if value != swappinessAfter {
+		t.Fatal("Got the wrong value, set memory.swappiness failed.")
+	}
+}
+
+func TestMemoryStats(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"memory.stat":                     memoryStatContents,
+		"memory.usage_in_bytes":           memoryUsageContents,
+		"memory.max_usage_in_bytes":       memoryMaxUsageContents,
+		"memory.failcnt":                  memoryFailcnt,
+		"memory.memsw.usage_in_bytes":     memoryUsageContents,
+		"memory.memsw.max_usage_in_bytes": memoryMaxUsageContents,
+		"memory.memsw.failcnt":            memoryFailcnt,
+		"memory.kmem.usage_in_bytes":      memoryUsageContents,
+		"memory.kmem.max_usage_in_bytes":  memoryMaxUsageContents,
+		"memory.kmem.failcnt":             memoryFailcnt,
+	})
+
+	memory := &MemoryGroup{}
+	actualStats := *cgroups.NewStats()
+	err := memory.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatal(err)
+	}
+	expectedStats := cgroups.MemoryStats{Cache: 512, Usage: cgroups.MemoryData{Usage: 2048, MaxUsage: 4096, Failcnt: 100}, SwapUsage: cgroups.MemoryData{Usage: 2048, MaxUsage: 4096, Failcnt: 100}, KernelUsage: cgroups.MemoryData{Usage: 2048, MaxUsage: 4096, Failcnt: 100}, Stats: map[string]uint64{"cache": 512, "rss": 1024}}
+	expectMemoryStatEquals(t, expectedStats, actualStats.MemoryStats)
+}
+
+func TestMemoryStatsNoStatFile(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"memory.usage_in_bytes":     memoryUsageContents,
+		"memory.max_usage_in_bytes": memoryMaxUsageContents,
+	})
+
+	memory := &MemoryGroup{}
+	actualStats := *cgroups.NewStats()
+	err := memory.GetStats(helper.CgroupPath, &actualStats)
+	if err != nil {
+		t.Fatal(err)
+	}
+}
+
+func TestMemoryStatsNoUsageFile(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"memory.stat":               memoryStatContents,
+		"memory.max_usage_in_bytes": memoryMaxUsageContents,
+	})
+
+	memory := &MemoryGroup{}
+	actualStats := *cgroups.NewStats()
+	err := memory.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected failure")
+	}
+}
+
+func TestMemoryStatsNoMaxUsageFile(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"memory.stat":           memoryStatContents,
+		"memory.usage_in_bytes": memoryUsageContents,
+	})
+
+	memory := &MemoryGroup{}
+	actualStats := *cgroups.NewStats()
+	err := memory.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected failure")
+	}
+}
+
+func TestMemoryStatsBadStatFile(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"memory.stat":               "rss rss",
+		"memory.usage_in_bytes":     memoryUsageContents,
+		"memory.max_usage_in_bytes": memoryMaxUsageContents,
+	})
+
+	memory := &MemoryGroup{}
+	actualStats := *cgroups.NewStats()
+	err := memory.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected failure")
+	}
+}
+
+func TestMemoryStatsBadUsageFile(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"memory.stat":               memoryStatContents,
+		"memory.usage_in_bytes":     "bad",
+		"memory.max_usage_in_bytes": memoryMaxUsageContents,
+	})
+
+	memory := &MemoryGroup{}
+	actualStats := *cgroups.NewStats()
+	err := memory.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected failure")
+	}
+}
+
+func TestMemoryStatsBadMaxUsageFile(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+	helper.writeFileContents(map[string]string{
+		"memory.stat":               memoryStatContents,
+		"memory.usage_in_bytes":     memoryUsageContents,
+		"memory.max_usage_in_bytes": "bad",
+	})
+
+	memory := &MemoryGroup{}
+	actualStats := *cgroups.NewStats()
+	err := memory.GetStats(helper.CgroupPath, &actualStats)
+	if err == nil {
+		t.Fatal("Expected failure")
+	}
+}
+
+func TestMemorySetOomControl(t *testing.T) {
+	helper := NewCgroupTestUtil("memory", t)
+	defer helper.cleanup()
+
+	const (
+		oom_kill_disable = 1 // disable oom killer, default is 0
+	)
+
+	helper.writeFileContents(map[string]string{
+		"memory.oom_control": strconv.Itoa(oom_kill_disable),
+	})
+
+	memory := &MemoryGroup{}
+	if err := memory.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamUint(helper.CgroupPath, "memory.oom_control")
+	if err != nil {
+		t.Fatalf("Failed to parse memory.oom_control - %s", err)
+	}
+
+	if value != oom_kill_disable {
+		t.Fatalf("Got the wrong value, set memory.oom_control failed.")
+	}
+}

libcontainer/cgroups/fs/net_cls.go

@@ -0,0 +1,40 @@
+package fs
+
+import (
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type NetClsGroup struct {
+}
+
+func (s *NetClsGroup) Apply(d *data) error {
+	dir, err := d.join("net_cls")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	if err := s.Set(dir, d.c); err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func (s *NetClsGroup) Set(path string, cgroup *configs.Cgroup) error {
+	if cgroup.NetClsClassid != "" {
+		if err := writeFile(path, "net_cls.classid", cgroup.NetClsClassid); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func (s *NetClsGroup) Remove(d *data) error {
+	return removePath(d.path("net_cls"))
+}
+
+func (s *NetClsGroup) GetStats(path string, stats *cgroups.Stats) error {
+	return nil
+}

libcontainer/cgroups/fs/net_cls_test.go

@@ -0,0 +1,36 @@
+package fs
+
+import (
+	"testing"
+)
+
+const (
+	classidBefore = "0x100002"
+	classidAfter  = "0x100001"
+)
+
+func TestNetClsSetClassid(t *testing.T) {
+	helper := NewCgroupTestUtil("net_cls", t)
+	defer helper.cleanup()
+
+	helper.writeFileContents(map[string]string{
+		"net_cls.classid": classidBefore,
+	})
+
+	helper.CgroupData.c.NetClsClassid = classidAfter
+	netcls := &NetClsGroup{}
+	if err := netcls.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	// As we are in mock environment, we can't get correct value of classid from
+	// net_cls.classid.
+	// So. we just judge if we successfully write classid into file
+	value, err := getCgroupParamString(helper.CgroupPath, "net_cls.classid")
+	if err != nil {
+		t.Fatalf("Failed to parse net_cls.classid - %s", err)
+	}
+	if value != classidAfter {
+		t.Fatal("Got the wrong value, set net_cls.classid failed.")
+	}
+}

libcontainer/cgroups/fs/net_prio.go

@@ -0,0 +1,40 @@
+package fs
+
+import (
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type NetPrioGroup struct {
+}
+
+func (s *NetPrioGroup) Apply(d *data) error {
+	dir, err := d.join("net_prio")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	if err := s.Set(dir, d.c); err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func (s *NetPrioGroup) Set(path string, cgroup *configs.Cgroup) error {
+	for _, prioMap := range cgroup.NetPrioIfpriomap {
+		if err := writeFile(path, "net_prio.ifpriomap", prioMap.CgroupString()); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func (s *NetPrioGroup) Remove(d *data) error {
+	return removePath(d.path("net_prio"))
+}
+
+func (s *NetPrioGroup) GetStats(path string, stats *cgroups.Stats) error {
+	return nil
+}

libcontainer/cgroups/fs/net_prio_test.go

@@ -0,0 +1,36 @@
+package fs
+
+import (
+	"strings"
+	"testing"
+
+	"github.com/docker/libcontainer/configs"
+)
+
+var (
+	prioMap = []*configs.IfPrioMap{
+		{
+			Interface: "test",
+			Priority:  5,
+		},
+	}
+)
+
+func TestNetPrioSetIfPrio(t *testing.T) {
+	helper := NewCgroupTestUtil("net_prio", t)
+	defer helper.cleanup()
+
+	helper.CgroupData.c.NetPrioIfpriomap = prioMap
+	netPrio := &NetPrioGroup{}
+	if err := netPrio.Set(helper.CgroupPath, helper.CgroupData.c); err != nil {
+		t.Fatal(err)
+	}
+
+	value, err := getCgroupParamString(helper.CgroupPath, "net_prio.ifpriomap")
+	if err != nil {
+		t.Fatalf("Failed to parse net_prio.ifpriomap - %s", err)
+	}
+	if !strings.Contains(value, "test 5") {
+		t.Fatal("Got the wrong value, set net_prio.ifpriomap failed.")
+	}
+}

libcontainer/cgroups/fs/perf_event.go

@@ -0,0 +1,31 @@
+// +build linux
+
+package fs
+
+import (
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type PerfEventGroup struct {
+}
+
+func (s *PerfEventGroup) Apply(d *data) error {
+	// we just want to join this group even though we don't set anything
+	if _, err := d.join("perf_event"); err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+	return nil
+}
+
+func (s *PerfEventGroup) Set(path string, cgroup *configs.Cgroup) error {
+	return nil
+}
+
+func (s *PerfEventGroup) Remove(d *data) error {
+	return removePath(d.path("perf_event"))
+}
+
+func (s *PerfEventGroup) GetStats(path string, stats *cgroups.Stats) error {
+	return nil
+}

libcontainer/cgroups/fs/stats_util_test.go

@@ -0,0 +1,113 @@
+// +build linux
+
+package fs
+
+import (
+	"fmt"
+	"testing"
+
+	"github.com/Sirupsen/logrus"
+	"github.com/docker/libcontainer/cgroups"
+)
+
+func blkioStatEntryEquals(expected, actual []cgroups.BlkioStatEntry) error {
+	if len(expected) != len(actual) {
+		return fmt.Errorf("blkioStatEntries length do not match")
+	}
+	for i, expValue := range expected {
+		actValue := actual[i]
+		if expValue != actValue {
+			return fmt.Errorf("Expected blkio stat entry %v but found %v", expValue, actValue)
+		}
+	}
+	return nil
+}
+
+func expectBlkioStatsEquals(t *testing.T, expected, actual cgroups.BlkioStats) {
+	if err := blkioStatEntryEquals(expected.IoServiceBytesRecursive, actual.IoServiceBytesRecursive); err != nil {
+		logrus.Printf("blkio IoServiceBytesRecursive do not match - %s\n", err)
+		t.Fail()
+	}
+
+	if err := blkioStatEntryEquals(expected.IoServicedRecursive, actual.IoServicedRecursive); err != nil {
+		logrus.Printf("blkio IoServicedRecursive do not match - %s\n", err)
+		t.Fail()
+	}
+
+	if err := blkioStatEntryEquals(expected.IoQueuedRecursive, actual.IoQueuedRecursive); err != nil {
+		logrus.Printf("blkio IoQueuedRecursive do not match - %s\n", err)
+		t.Fail()
+	}
+
+	if err := blkioStatEntryEquals(expected.SectorsRecursive, actual.SectorsRecursive); err != nil {
+		logrus.Printf("blkio SectorsRecursive do not match - %s\n", err)
+		t.Fail()
+	}
+
+	if err := blkioStatEntryEquals(expected.IoServiceTimeRecursive, actual.IoServiceTimeRecursive); err != nil {
+		logrus.Printf("blkio IoServiceTimeRecursive do not match - %s\n", err)
+		t.Fail()
+	}
+
+	if err := blkioStatEntryEquals(expected.IoWaitTimeRecursive, actual.IoWaitTimeRecursive); err != nil {
+		logrus.Printf("blkio IoWaitTimeRecursive do not match - %s\n", err)
+		t.Fail()
+	}
+
+	if err := blkioStatEntryEquals(expected.IoMergedRecursive, actual.IoMergedRecursive); err != nil {
+		logrus.Printf("blkio IoMergedRecursive do not match - %v vs %v\n", expected.IoMergedRecursive, actual.IoMergedRecursive)
+		t.Fail()
+	}
+
+	if err := blkioStatEntryEquals(expected.IoTimeRecursive, actual.IoTimeRecursive); err != nil {
+		logrus.Printf("blkio IoTimeRecursive do not match - %s\n", err)
+		t.Fail()
+	}
+}
+
+func expectThrottlingDataEquals(t *testing.T, expected, actual cgroups.ThrottlingData) {
+	if expected != actual {
+		logrus.Printf("Expected throttling data %v but found %v\n", expected, actual)
+		t.Fail()
+	}
+}
+
+func expectHugetlbStatEquals(t *testing.T, expected, actual cgroups.HugetlbStats) {
+	if expected != actual {
+		logrus.Printf("Expected hugetlb stats %v but found %v\n", expected, actual)
+		t.Fail()
+	}
+}
+
+func expectMemoryStatEquals(t *testing.T, expected, actual cgroups.MemoryStats) {
+	expectMemoryDataEquals(t, expected.Usage, actual.Usage)
+	expectMemoryDataEquals(t, expected.SwapUsage, actual.SwapUsage)
+	expectMemoryDataEquals(t, expected.KernelUsage, actual.KernelUsage)
+
+	for key, expValue := range expected.Stats {
+		actValue, ok := actual.Stats[key]
+		if !ok {
+			logrus.Printf("Expected memory stat key %s not found\n", key)
+			t.Fail()
+		}
+		if expValue != actValue {
+			logrus.Printf("Expected memory stat value %d but found %d\n", expValue, actValue)
+			t.Fail()
+		}
+	}
+}
+
+func expectMemoryDataEquals(t *testing.T, expected, actual cgroups.MemoryData) {
+	if expected.Usage != actual.Usage {
+		logrus.Printf("Expected memory usage %d but found %d\n", expected.Usage, actual.Usage)
+		t.Fail()
+	}
+	if expected.MaxUsage != actual.MaxUsage {
+		logrus.Printf("Expected memory max usage %d but found %d\n", expected.MaxUsage, actual.MaxUsage)
+		t.Fail()
+	}
+	if expected.Failcnt != actual.Failcnt {
+		logrus.Printf("Expected memory failcnt %d but found %d\n", expected.Failcnt, actual.Failcnt)
+		t.Fail()
+	}
+}

libcontainer/cgroups/fs/util_test.go

@@ -0,0 +1,66 @@
+// +build linux
+
+/*
+Utility for testing cgroup operations.
+
+Creates a mock of the cgroup filesystem for the duration of the test.
+*/
+package fs
+
+import (
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/docker/libcontainer/configs"
+)
+
+type cgroupTestUtil struct {
+	// data to use in tests.
+	CgroupData *data
+
+	// Path to the mock cgroup directory.
+	CgroupPath string
+
+	// Temporary directory to store mock cgroup filesystem.
+	tempDir string
+	t       *testing.T
+}
+
+// Creates a new test util for the specified subsystem
+func NewCgroupTestUtil(subsystem string, t *testing.T) *cgroupTestUtil {
+	d := &data{
+		c: &configs.Cgroup{},
+	}
+	tempDir, err := ioutil.TempDir("", "cgroup_test")
+	if err != nil {
+		t.Fatal(err)
+	}
+	d.root = tempDir
+	testCgroupPath := filepath.Join(d.root, subsystem)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// Ensure the full mock cgroup path exists.
+	err = os.MkdirAll(testCgroupPath, 0755)
+	if err != nil {
+		t.Fatal(err)
+	}
+	return &cgroupTestUtil{CgroupData: d, CgroupPath: testCgroupPath, tempDir: tempDir, t: t}
+}
+
+func (c *cgroupTestUtil) cleanup() {
+	os.RemoveAll(c.tempDir)
+}
+
+// Write the specified contents on the mock of the specified cgroup files.
+func (c *cgroupTestUtil) writeFileContents(fileContents map[string]string) {
+	for file, contents := range fileContents {
+		err := writeFile(c.CgroupPath, file, contents)
+		if err != nil {
+			c.t.Fatal(err)
+		}
+	}
+}

libcontainer/cgroups/fs/utils.go

@@ -0,0 +1,74 @@
+// +build linux
+
+package fs
+
+import (
+	"errors"
+	"fmt"
+	"io/ioutil"
+	"path/filepath"
+	"strconv"
+	"strings"
+)
+
+var (
+	ErrNotSupportStat = errors.New("stats are not supported for subsystem")
+	ErrNotValidFormat = errors.New("line is not a valid key value format")
+)
+
+// Saturates negative values at zero and returns a uint64.
+// Due to kernel bugs, some of the memory cgroup stats can be negative.
+func parseUint(s string, base, bitSize int) (uint64, error) {
+	value, err := strconv.ParseUint(s, base, bitSize)
+	if err != nil {
+		intValue, intErr := strconv.ParseInt(s, base, bitSize)
+		// 1. Handle negative values greater than MinInt64 (and)
+		// 2. Handle negative values lesser than MinInt64
+		if intErr == nil && intValue < 0 {
+			return 0, nil
+		} else if intErr != nil && intErr.(*strconv.NumError).Err == strconv.ErrRange && intValue < 0 {
+			return 0, nil
+		}
+
+		return value, err
+	}
+
+	return value, nil
+}
+
+// Parses a cgroup param and returns as name, value
+//  i.e. "io_service_bytes 1234" will return as io_service_bytes, 1234
+func getCgroupParamKeyValue(t string) (string, uint64, error) {
+	parts := strings.Fields(t)
+	switch len(parts) {
+	case 2:
+		value, err := parseUint(parts[1], 10, 64)
+		if err != nil {
+			return "", 0, fmt.Errorf("Unable to convert param value (%q) to uint64: %v", parts[1], err)
+		}
+
+		return parts[0], value, nil
+	default:
+		return "", 0, ErrNotValidFormat
+	}
+}
+
+// Gets a single uint64 value from the specified cgroup file.
+func getCgroupParamUint(cgroupPath, cgroupFile string) (uint64, error) {
+	contents, err := ioutil.ReadFile(filepath.Join(cgroupPath, cgroupFile))
+	if err != nil {
+		return 0, err
+	}
+
+	return parseUint(strings.TrimSpace(string(contents)), 10, 64)
+}
+
+// Gets a string value from the specified cgroup file
+func getCgroupParamString(cgroupPath, cgroupFile string) (string, error) {
+	contents, err := ioutil.ReadFile(filepath.Join(cgroupPath, cgroupFile))
+	if err != nil {
+		return "", err
+	}
+
+	return strings.TrimSpace(string(contents)), nil
+}

libcontainer/cgroups/fs/utils_test.go

@@ -0,0 +1,97 @@
+// +build linux
+
+package fs
+
+import (
+	"io/ioutil"
+	"math"
+	"os"
+	"path/filepath"
+	"strconv"
+	"testing"
+)
+
+const (
+	cgroupFile  = "cgroup.file"
+	floatValue  = 2048.0
+	floatString = "2048"
+)
+
+func TestGetCgroupParamsInt(t *testing.T) {
+	// Setup tempdir.
+	tempDir, err := ioutil.TempDir("", "cgroup_utils_test")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.RemoveAll(tempDir)
+	tempFile := filepath.Join(tempDir, cgroupFile)
+
+	// Success.
+	err = ioutil.WriteFile(tempFile, []byte(floatString), 0755)
+	if err != nil {
+		t.Fatal(err)
+	}
+	value, err := getCgroupParamUint(tempDir, cgroupFile)
+	if err != nil {
+		t.Fatal(err)
+	} else if value != floatValue {
+		t.Fatalf("Expected %d to equal %f", value, floatValue)
+	}
+
+	// Success with new line.
+	err = ioutil.WriteFile(tempFile, []byte(floatString+"\n"), 0755)
+	if err != nil {
+		t.Fatal(err)
+	}
+	value, err = getCgroupParamUint(tempDir, cgroupFile)
+	if err != nil {
+		t.Fatal(err)
+	} else if value != floatValue {
+		t.Fatalf("Expected %d to equal %f", value, floatValue)
+	}
+
+	// Success with negative values
+	err = ioutil.WriteFile(tempFile, []byte("-12345"), 0755)
+	if err != nil {
+		t.Fatal(err)
+	}
+	value, err = getCgroupParamUint(tempDir, cgroupFile)
+	if err != nil {
+		t.Fatal(err)
+	} else if value != 0 {
+		t.Fatalf("Expected %d to equal %d", value, 0)
+	}
+
+	// Success with negative values lesser than min int64
+	s := strconv.FormatFloat(math.MinInt64, 'f', -1, 64)
+	err = ioutil.WriteFile(tempFile, []byte(s), 0755)
+	if err != nil {
+		t.Fatal(err)
+	}
+	value, err = getCgroupParamUint(tempDir, cgroupFile)
+	if err != nil {
+		t.Fatal(err)
+	} else if value != 0 {
+		t.Fatalf("Expected %d to equal %d", value, 0)
+	}
+
+	// Not a float.
+	err = ioutil.WriteFile(tempFile, []byte("not-a-float"), 0755)
+	if err != nil {
+		t.Fatal(err)
+	}
+	_, err = getCgroupParamUint(tempDir, cgroupFile)
+	if err == nil {
+		t.Fatal("Expecting error, got none")
+	}
+
+	// Unknown file.
+	err = os.Remove(tempFile)
+	if err != nil {
+		t.Fatal(err)
+	}
+	_, err = getCgroupParamUint(tempDir, cgroupFile)
+	if err == nil {
+		t.Fatal("Expecting error, got none")
+	}
+}

libcontainer/cgroups/stats.go

@@ -0,0 +1,92 @@
+// +build linux
+
+package cgroups
+
+type ThrottlingData struct {
+	// Number of periods with throttling active
+	Periods uint64 `json:"periods,omitempty"`
+	// Number of periods when the container hit its throttling limit.
+	ThrottledPeriods uint64 `json:"throttled_periods,omitempty"`
+	// Aggregate time the container was throttled for in nanoseconds.
+	ThrottledTime uint64 `json:"throttled_time,omitempty"`
+}
+
+// All CPU stats are aggregate since container inception.
+type CpuUsage struct {
+	// Total CPU time consumed.
+	// Units: nanoseconds.
+	TotalUsage uint64 `json:"total_usage,omitempty"`
+	// Total CPU time consumed per core.
+	// Units: nanoseconds.
+	PercpuUsage []uint64 `json:"percpu_usage,omitempty"`
+	// Time spent by tasks of the cgroup in kernel mode.
+	// Units: nanoseconds.
+	UsageInKernelmode uint64 `json:"usage_in_kernelmode"`
+	// Time spent by tasks of the cgroup in user mode.
+	// Units: nanoseconds.
+	UsageInUsermode uint64 `json:"usage_in_usermode"`
+}
+
+type CpuStats struct {
+	CpuUsage       CpuUsage       `json:"cpu_usage,omitempty"`
+	ThrottlingData ThrottlingData `json:"throttling_data,omitempty"`
+}
+
+type MemoryData struct {
+	Usage    uint64 `json:"usage,omitempty"`
+	MaxUsage uint64 `json:"max_usage,omitempty"`
+	Failcnt  uint64 `json:"failcnt"`
+}
+type MemoryStats struct {
+	// memory used for cache
+	Cache uint64 `json:"cache,omitempty"`
+	// usage of memory
+	Usage MemoryData `json:"usage,omitempty"`
+	// usage of memory + swap
+	SwapUsage MemoryData `json:"swap_usage,omitempty"`
+	// usafe of kernel memory
+	KernelUsage MemoryData        `json:"kernel_usage,omitempty"`
+	Stats       map[string]uint64 `json:"stats,omitempty"`
+}
+
+type BlkioStatEntry struct {
+	Major uint64 `json:"major,omitempty"`
+	Minor uint64 `json:"minor,omitempty"`
+	Op    string `json:"op,omitempty"`
+	Value uint64 `json:"value,omitempty"`
+}
+
+type BlkioStats struct {
+	// number of bytes tranferred to and from the block device
+	IoServiceBytesRecursive []BlkioStatEntry `json:"io_service_bytes_recursive,omitempty"`
+	IoServicedRecursive     []BlkioStatEntry `json:"io_serviced_recursive,omitempty"`
+	IoQueuedRecursive       []BlkioStatEntry `json:"io_queue_recursive,omitempty"`
+	IoServiceTimeRecursive  []BlkioStatEntry `json:"io_service_time_recursive,omitempty"`
+	IoWaitTimeRecursive     []BlkioStatEntry `json:"io_wait_time_recursive,omitempty"`
+	IoMergedRecursive       []BlkioStatEntry `json:"io_merged_recursive,omitempty"`
+	IoTimeRecursive         []BlkioStatEntry `json:"io_time_recursive,omitempty"`
+	SectorsRecursive        []BlkioStatEntry `json:"sectors_recursive,omitempty"`
+}
+
+type HugetlbStats struct {
+	// current res_counter usage for hugetlb
+	Usage uint64 `json:"usage,omitempty"`
+	// maximum usage ever recorded.
+	MaxUsage uint64 `json:"max_usage,omitempty"`
+	// number of times htgetlb usage allocation failure.
+	Failcnt uint64 `json:"failcnt"`
+}
+
+type Stats struct {
+	CpuStats    CpuStats    `json:"cpu_stats,omitempty"`
+	MemoryStats MemoryStats `json:"memory_stats,omitempty"`
+	BlkioStats  BlkioStats  `json:"blkio_stats,omitempty"`
+	// the map is in the format "size of hugepage: stats of the hugepage"
+	HugetlbStats map[string]HugetlbStats `json:"hugetlb_stats,omitempty"`
+}
+
+func NewStats() *Stats {
+	memoryStats := MemoryStats{Stats: make(map[string]uint64)}
+	hugetlbStats := make(map[string]HugetlbStats)
+	return &Stats{MemoryStats: memoryStats, HugetlbStats: hugetlbStats}
+}

libcontainer/cgroups/systemd/apply_nosystemd.go

@@ -0,0 +1,51 @@
+// +build !linux
+
+package systemd
+
+import (
+	"fmt"
+
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type Manager struct {
+	Cgroups *configs.Cgroup
+	Paths   map[string]string
+}
+
+func UseSystemd() bool {
+	return false
+}
+
+func (m *Manager) Apply(pid int) error {
+	return fmt.Errorf("Systemd not supported")
+}
+
+func (m *Manager) GetPids() ([]int, error) {
+	return nil, fmt.Errorf("Systemd not supported")
+}
+
+func (m *Manager) Destroy() error {
+	return fmt.Errorf("Systemd not supported")
+}
+
+func (m *Manager) GetPaths() map[string]string {
+	return nil
+}
+
+func (m *Manager) GetStats() (*cgroups.Stats, error) {
+	return nil, fmt.Errorf("Systemd not supported")
+}
+
+func (m *Manager) Set(container *configs.Config) error {
+	return nil, fmt.Errorf("Systemd not supported")
+}
+
+func (m *Manager) Freeze(state configs.FreezerState) error {
+	return fmt.Errorf("Systemd not supported")
+}
+
+func Freeze(c *configs.Cgroup, state configs.FreezerState) error {
+	return fmt.Errorf("Systemd not supported")
+}

libcontainer/cgroups/systemd/apply_systemd.go

@@ -0,0 +1,578 @@
+// +build linux
+
+package systemd
+
+import (
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+
+	systemd "github.com/coreos/go-systemd/dbus"
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/cgroups/fs"
+	"github.com/docker/libcontainer/configs"
+	"github.com/godbus/dbus"
+)
+
+type Manager struct {
+	mu      sync.Mutex
+	Cgroups *configs.Cgroup
+	Paths   map[string]string
+}
+
+type subsystem interface {
+	// Returns the stats, as 'stats', corresponding to the cgroup under 'path'.
+	GetStats(path string, stats *cgroups.Stats) error
+	// Set the cgroup represented by cgroup.
+	Set(path string, cgroup *configs.Cgroup) error
+}
+
+var subsystems = map[string]subsystem{
+	"devices":    &fs.DevicesGroup{},
+	"memory":     &fs.MemoryGroup{},
+	"cpu":        &fs.CpuGroup{},
+	"cpuset":     &fs.CpusetGroup{},
+	"cpuacct":    &fs.CpuacctGroup{},
+	"blkio":      &fs.BlkioGroup{},
+	"hugetlb":    &fs.HugetlbGroup{},
+	"perf_event": &fs.PerfEventGroup{},
+	"freezer":    &fs.FreezerGroup{},
+	"net_prio":   &fs.NetPrioGroup{},
+	"net_cls":    &fs.NetClsGroup{},
+}
+
+const (
+	testScopeWait = 4
+)
+
+var (
+	connLock                        sync.Mutex
+	theConn                         *systemd.Conn
+	hasStartTransientUnit           bool
+	hasTransientDefaultDependencies bool
+)
+
+func newProp(name string, units interface{}) systemd.Property {
+	return systemd.Property{
+		Name:  name,
+		Value: dbus.MakeVariant(units),
+	}
+}
+
+func UseSystemd() bool {
+	s, err := os.Stat("/run/systemd/system")
+	if err != nil || !s.IsDir() {
+		return false
+	}
+
+	connLock.Lock()
+	defer connLock.Unlock()
+
+	if theConn == nil {
+		var err error
+		theConn, err = systemd.New()
+		if err != nil {
+			return false
+		}
+
+		// Assume we have StartTransientUnit
+		hasStartTransientUnit = true
+
+		// But if we get UnknownMethod error we don't
+		if _, err := theConn.StartTransientUnit("test.scope", "invalid"); err != nil {
+			if dbusError, ok := err.(dbus.Error); ok {
+				if dbusError.Name == "org.freedesktop.DBus.Error.UnknownMethod" {
+					hasStartTransientUnit = false
+					return hasStartTransientUnit
+				}
+			}
+		}
+
+		// Ensure the scope name we use doesn't exist. Use the Pid to
+		// avoid collisions between multiple libcontainer users on a
+		// single host.
+		scope := fmt.Sprintf("libcontainer-%d-systemd-test-default-dependencies.scope", os.Getpid())
+		testScopeExists := true
+		for i := 0; i <= testScopeWait; i++ {
+			if _, err := theConn.StopUnit(scope, "replace"); err != nil {
+				if dbusError, ok := err.(dbus.Error); ok {
+					if strings.Contains(dbusError.Name, "org.freedesktop.systemd1.NoSuchUnit") {
+						testScopeExists = false
+						break
+					}
+				}
+			}
+			time.Sleep(time.Millisecond)
+		}
+
+		// Bail out if we can't kill this scope without testing for DefaultDependencies
+		if testScopeExists {
+			return hasStartTransientUnit
+		}
+
+		// Assume StartTransientUnit on a scope allows DefaultDependencies
+		hasTransientDefaultDependencies = true
+		ddf := newProp("DefaultDependencies", false)
+		if _, err := theConn.StartTransientUnit(scope, "replace", ddf); err != nil {
+			if dbusError, ok := err.(dbus.Error); ok {
+				if strings.Contains(dbusError.Name, "org.freedesktop.DBus.Error.PropertyReadOnly") {
+					hasTransientDefaultDependencies = false
+				}
+			}
+		}
+
+		// Not critical because of the stop unit logic above.
+		theConn.StopUnit(scope, "replace")
+	}
+	return hasStartTransientUnit
+}
+
+func getIfaceForUnit(unitName string) string {
+	if strings.HasSuffix(unitName, ".scope") {
+		return "Scope"
+	}
+	if strings.HasSuffix(unitName, ".service") {
+		return "Service"
+	}
+	return "Unit"
+}
+
+func (m *Manager) Apply(pid int) error {
+	var (
+		c          = m.Cgroups
+		unitName   = getUnitName(c)
+		slice      = "system.slice"
+		properties []systemd.Property
+	)
+
+	if c.Slice != "" {
+		slice = c.Slice
+	}
+
+	properties = append(properties,
+		systemd.PropSlice(slice),
+		systemd.PropDescription("docker container "+c.Name),
+		newProp("PIDs", []uint32{uint32(pid)}),
+	)
+
+	// Always enable accounting, this gets us the same behaviour as the fs implementation,
+	// plus the kernel has some problems with joining the memory cgroup at a later time.
+	properties = append(properties,
+		newProp("MemoryAccounting", true),
+		newProp("CPUAccounting", true),
+		newProp("BlockIOAccounting", true))
+
+	if hasTransientDefaultDependencies {
+		properties = append(properties,
+			newProp("DefaultDependencies", false))
+	}
+
+	if c.Memory != 0 {
+		properties = append(properties,
+			newProp("MemoryLimit", uint64(c.Memory)))
+	}
+	// TODO: MemoryReservation and MemorySwap not available in systemd
+
+	if c.CpuShares != 0 {
+		properties = append(properties,
+			newProp("CPUShares", uint64(c.CpuShares)))
+	}
+
+	if c.BlkioWeight != 0 {
+		properties = append(properties,
+			newProp("BlockIOWeight", uint64(c.BlkioWeight)))
+	}
+
+	// We need to set kernel memory before processes join cgroup because
+	// kmem.limit_in_bytes can only be set when the cgroup is empty.
+	// And swap memory limit needs to be set after memory limit, only
+	// memory limit is handled by systemd, so it's kind of ugly here.
+	if c.KernelMemory > 0 {
+		if err := setKernelMemory(c); err != nil {
+			return err
+		}
+	}
+
+	if _, err := theConn.StartTransientUnit(unitName, "replace", properties...); err != nil {
+		return err
+	}
+
+	if err := joinDevices(c, pid); err != nil {
+		return err
+	}
+
+	// TODO: CpuQuota and CpuPeriod not available in systemd
+	// we need to manually join the cpu.cfs_quota_us and cpu.cfs_period_us
+	if err := joinCpu(c, pid); err != nil {
+		return err
+	}
+
+	if err := joinMemory(c, pid); err != nil {
+		return err
+	}
+
+	// we need to manually join the freezer, net_cls, net_prio and cpuset cgroup in systemd
+	// because it does not currently support it via the dbus api.
+	if err := joinFreezer(c, pid); err != nil {
+		return err
+	}
+
+	if err := joinNetPrio(c, pid); err != nil {
+		return err
+	}
+	if err := joinNetCls(c, pid); err != nil {
+		return err
+	}
+
+	if err := joinCpuset(c, pid); err != nil {
+		return err
+	}
+
+	if err := joinHugetlb(c, pid); err != nil {
+		return err
+	}
+	// FIXME: Systemd does have `BlockIODeviceWeight` property, but we got problem
+	// using that (at least on systemd 208, see https://github.com/docker/libcontainer/pull/354),
+	// so use fs work around for now.
+	if err := joinBlkio(c, pid); err != nil {
+		return err
+	}
+
+	paths := make(map[string]string)
+	for sysname := range subsystems {
+		subsystemPath, err := getSubsystemPath(m.Cgroups, sysname)
+		if err != nil {
+			// Don't fail if a cgroup hierarchy was not found, just skip this subsystem
+			if cgroups.IsNotFound(err) {
+				continue
+			}
+			return err
+		}
+		paths[sysname] = subsystemPath
+	}
+	m.Paths = paths
+
+	if paths["cpu"] != "" {
+		if err := fs.CheckCpushares(paths["cpu"], c.CpuShares); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func (m *Manager) Destroy() error {
+	m.mu.Lock()
+	defer m.mu.Unlock()
+	theConn.StopUnit(getUnitName(m.Cgroups), "replace")
+	if err := cgroups.RemovePaths(m.Paths); err != nil {
+		return err
+	}
+	m.Paths = make(map[string]string)
+	return nil
+}
+
+func (m *Manager) GetPaths() map[string]string {
+	m.mu.Lock()
+	paths := m.Paths
+	m.mu.Unlock()
+	return paths
+}
+
+func writeFile(dir, file, data string) error {
+	// Normally dir should not be empty, one case is that cgroup subsystem
+	// is not mounted, we will get empty dir, and we want it fail here.
+	if dir == "" {
+		return fmt.Errorf("no such directory for %s.", file)
+	}
+	return ioutil.WriteFile(filepath.Join(dir, file), []byte(data), 0700)
+}
+
+func join(c *configs.Cgroup, subsystem string, pid int) (string, error) {
+	path, err := getSubsystemPath(c, subsystem)
+	if err != nil {
+		return "", err
+	}
+	if err := os.MkdirAll(path, 0755); err != nil && !os.IsExist(err) {
+		return "", err
+	}
+	if err := writeFile(path, "cgroup.procs", strconv.Itoa(pid)); err != nil {
+		return "", err
+	}
+
+	return path, nil
+}
+
+func joinCpu(c *configs.Cgroup, pid int) error {
+	path, err := getSubsystemPath(c, "cpu")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+	if c.CpuQuota != 0 {
+		if err = writeFile(path, "cpu.cfs_quota_us", strconv.FormatInt(c.CpuQuota, 10)); err != nil {
+			return err
+		}
+	}
+	if c.CpuPeriod != 0 {
+		if err = writeFile(path, "cpu.cfs_period_us", strconv.FormatInt(c.CpuPeriod, 10)); err != nil {
+			return err
+		}
+	}
+	if c.CpuRtPeriod != 0 {
+		if err = writeFile(path, "cpu.rt_period_us", strconv.FormatInt(c.CpuRtPeriod, 10)); err != nil {
+			return err
+		}
+	}
+	if c.CpuRtRuntime != 0 {
+		if err = writeFile(path, "cpu.rt_runtime_us", strconv.FormatInt(c.CpuRtRuntime, 10)); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func joinFreezer(c *configs.Cgroup, pid int) error {
+	path, err := join(c, "freezer", pid)
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	freezer := subsystems["freezer"]
+	return freezer.Set(path, c)
+}
+
+func joinNetPrio(c *configs.Cgroup, pid int) error {
+	path, err := join(c, "net_prio", pid)
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+	netPrio := subsystems["net_prio"]
+
+	return netPrio.Set(path, c)
+}
+
+func joinNetCls(c *configs.Cgroup, pid int) error {
+	path, err := join(c, "net_cls", pid)
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+	netcls := subsystems["net_cls"]
+
+	return netcls.Set(path, c)
+}
+
+func getSubsystemPath(c *configs.Cgroup, subsystem string) (string, error) {
+	mountpoint, err := cgroups.FindCgroupMountpoint(subsystem)
+	if err != nil {
+		return "", err
+	}
+
+	initPath, err := cgroups.GetInitCgroupDir(subsystem)
+	if err != nil {
+		return "", err
+	}
+
+	slice := "system.slice"
+	if c.Slice != "" {
+		slice = c.Slice
+	}
+
+	return filepath.Join(mountpoint, initPath, slice, getUnitName(c)), nil
+}
+
+func (m *Manager) Freeze(state configs.FreezerState) error {
+	path, err := getSubsystemPath(m.Cgroups, "freezer")
+	if err != nil {
+		return err
+	}
+
+	prevState := m.Cgroups.Freezer
+	m.Cgroups.Freezer = state
+
+	freezer := subsystems["freezer"]
+	err = freezer.Set(path, m.Cgroups)
+	if err != nil {
+		m.Cgroups.Freezer = prevState
+		return err
+	}
+
+	return nil
+}
+
+func (m *Manager) GetPids() ([]int, error) {
+	path, err := getSubsystemPath(m.Cgroups, "cpu")
+	if err != nil {
+		return nil, err
+	}
+
+	return cgroups.ReadProcsFile(path)
+}
+
+func (m *Manager) GetStats() (*cgroups.Stats, error) {
+	m.mu.Lock()
+	defer m.mu.Unlock()
+	stats := cgroups.NewStats()
+	for name, path := range m.Paths {
+		sys, ok := subsystems[name]
+		if !ok || !cgroups.PathExists(path) {
+			continue
+		}
+		if err := sys.GetStats(path, stats); err != nil {
+			return nil, err
+		}
+	}
+
+	return stats, nil
+}
+
+func (m *Manager) Set(container *configs.Config) error {
+	for name, path := range m.Paths {
+		sys, ok := subsystems[name]
+		if !ok || !cgroups.PathExists(path) {
+			continue
+		}
+		if err := sys.Set(path, container.Cgroups); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func getUnitName(c *configs.Cgroup) string {
+	return fmt.Sprintf("%s-%s.scope", c.Parent, c.Name)
+}
+
+// Atm we can't use the systemd device support because of two missing things:
+// * Support for wildcards to allow mknod on any device
+// * Support for wildcards to allow /dev/pts support
+//
+// The second is available in more recent systemd as "char-pts", but not in e.g. v208 which is
+// in wide use. When both these are available we will be able to switch, but need to keep the old
+// implementation for backwards compat.
+//
+// Note: we can't use systemd to set up the initial limits, and then change the cgroup
+// because systemd will re-write the device settings if it needs to re-apply the cgroup context.
+// This happens at least for v208 when any sibling unit is started.
+func joinDevices(c *configs.Cgroup, pid int) error {
+	path, err := join(c, "devices", pid)
+	// Even if it's `not found` error, we'll return err because devices cgroup
+	// is hard requirement for container security.
+	if err != nil {
+		return err
+	}
+
+	devices := subsystems["devices"]
+	return devices.Set(path, c)
+}
+
+func setKernelMemory(c *configs.Cgroup) error {
+	path, err := getSubsystemPath(c, "memory")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	if err := os.MkdirAll(path, 0755); err != nil && !os.IsExist(err) {
+		return err
+	}
+
+	if c.KernelMemory > 0 {
+		err = writeFile(path, "memory.kmem.limit_in_bytes", strconv.FormatInt(c.KernelMemory, 10))
+		if err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func joinMemory(c *configs.Cgroup, pid int) error {
+	path, err := getSubsystemPath(c, "memory")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	// -1 disables memoryswap
+	if c.MemorySwap > 0 {
+		err = writeFile(path, "memory.memsw.limit_in_bytes", strconv.FormatInt(c.MemorySwap, 10))
+		if err != nil {
+			return err
+		}
+	}
+
+	if c.MemorySwappiness >= 0 && c.MemorySwappiness <= 100 {
+		err = writeFile(path, "memory.swappiness", strconv.FormatInt(c.MemorySwappiness, 10))
+		if err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+// systemd does not atm set up the cpuset controller, so we must manually
+// join it. Additionally that is a very finicky controller where each
+// level must have a full setup as the default for a new directory is "no cpus"
+func joinCpuset(c *configs.Cgroup, pid int) error {
+	path, err := getSubsystemPath(c, "cpuset")
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	s := &fs.CpusetGroup{}
+
+	return s.ApplyDir(path, c, pid)
+}
+
+// `BlockIODeviceWeight` property of systemd does not work properly, and systemd
+// expects device path instead of major minor numbers, which is also confusing
+// for users. So we use fs work around for now.
+func joinBlkio(c *configs.Cgroup, pid int) error {
+	path, err := getSubsystemPath(c, "blkio")
+	if err != nil {
+		return err
+	}
+	if c.BlkioWeightDevice != "" {
+		if err := writeFile(path, "blkio.weight_device", c.BlkioWeightDevice); err != nil {
+			return err
+		}
+	}
+	if c.BlkioThrottleReadBpsDevice != "" {
+		if err := writeFile(path, "blkio.throttle.read_bps_device", c.BlkioThrottleReadBpsDevice); err != nil {
+			return err
+		}
+	}
+	if c.BlkioThrottleWriteBpsDevice != "" {
+		if err := writeFile(path, "blkio.throttle.write_bps_device", c.BlkioThrottleWriteBpsDevice); err != nil {
+			return err
+		}
+	}
+	if c.BlkioThrottleReadIOpsDevice != "" {
+		if err := writeFile(path, "blkio.throttle.read_iops_device", c.BlkioThrottleReadIOpsDevice); err != nil {
+			return err
+		}
+	}
+	if c.BlkioThrottleWriteIOpsDevice != "" {
+		if err := writeFile(path, "blkio.throttle.write_iops_device", c.BlkioThrottleWriteIOpsDevice); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func joinHugetlb(c *configs.Cgroup, pid int) error {
+	path, err := join(c, "hugetlb", pid)
+	if err != nil && !cgroups.IsNotFound(err) {
+		return err
+	}
+
+	hugetlb := subsystems["hugetlb"]
+	return hugetlb.Set(path, c)
+}

libcontainer/cgroups/utils.go

@@ -0,0 +1,264 @@
+// +build linux
+
+package cgroups
+
+import (
+	"bufio"
+	"fmt"
+	"io"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strconv"
+	"strings"
+	"time"
+
+	"github.com/docker/docker/pkg/mount"
+	"github.com/docker/docker/pkg/units"
+)
+
+// https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt
+func FindCgroupMountpoint(subsystem string) (string, error) {
+	f, err := os.Open("/proc/self/mountinfo")
+	if err != nil {
+		return "", err
+	}
+	scanner := bufio.NewScanner(f)
+	for scanner.Scan() {
+		txt := scanner.Text()
+		fields := strings.Split(txt, " ")
+		for _, opt := range strings.Split(fields[len(fields)-1], ",") {
+			if opt == subsystem {
+				return fields[4], nil
+			}
+		}
+	}
+	if err := scanner.Err(); err != nil {
+		return "", err
+	}
+
+	return "", NewNotFoundError(subsystem)
+}
+
+func FindCgroupMountpointDir() (string, error) {
+	mounts, err := mount.GetMounts()
+	if err != nil {
+		return "", err
+	}
+
+	for _, mount := range mounts {
+		if mount.Fstype == "cgroup" {
+			return filepath.Dir(mount.Mountpoint), nil
+		}
+	}
+
+	return "", NewNotFoundError("cgroup")
+}
+
+type Mount struct {
+	Mountpoint string
+	Subsystems []string
+}
+
+func (m Mount) GetThisCgroupDir() (string, error) {
+	if len(m.Subsystems) == 0 {
+		return "", fmt.Errorf("no subsystem for mount")
+	}
+
+	return GetThisCgroupDir(m.Subsystems[0])
+}
+
+func GetCgroupMounts() ([]Mount, error) {
+	mounts, err := mount.GetMounts()
+	if err != nil {
+		return nil, err
+	}
+
+	all, err := GetAllSubsystems()
+	if err != nil {
+		return nil, err
+	}
+
+	allMap := make(map[string]bool)
+	for _, s := range all {
+		allMap[s] = true
+	}
+
+	res := []Mount{}
+	for _, mount := range mounts {
+		if mount.Fstype == "cgroup" {
+			m := Mount{Mountpoint: mount.Mountpoint}
+
+			for _, opt := range strings.Split(mount.VfsOpts, ",") {
+				if strings.HasPrefix(opt, "name=") {
+					m.Subsystems = append(m.Subsystems, opt)
+				}
+				if allMap[opt] {
+					m.Subsystems = append(m.Subsystems, opt)
+				}
+			}
+			res = append(res, m)
+		}
+	}
+	return res, nil
+}
+
+// Returns all the cgroup subsystems supported by the kernel
+func GetAllSubsystems() ([]string, error) {
+	f, err := os.Open("/proc/cgroups")
+	if err != nil {
+		return nil, err
+	}
+	defer f.Close()
+
+	subsystems := []string{}
+
+	s := bufio.NewScanner(f)
+	for s.Scan() {
+		if err := s.Err(); err != nil {
+			return nil, err
+		}
+		text := s.Text()
+		if text[0] != '#' {
+			parts := strings.Fields(text)
+			if len(parts) >= 4 && parts[3] != "0" {
+				subsystems = append(subsystems, parts[0])
+			}
+		}
+	}
+	return subsystems, nil
+}
+
+// Returns the relative path to the cgroup docker is running in.
+func GetThisCgroupDir(subsystem string) (string, error) {
+	f, err := os.Open("/proc/self/cgroup")
+	if err != nil {
+		return "", err
+	}
+	defer f.Close()
+
+	return ParseCgroupFile(subsystem, f)
+}
+
+func GetInitCgroupDir(subsystem string) (string, error) {
+	f, err := os.Open("/proc/1/cgroup")
+	if err != nil {
+		return "", err
+	}
+	defer f.Close()
+
+	return ParseCgroupFile(subsystem, f)
+}
+
+func ReadProcsFile(dir string) ([]int, error) {
+	f, err := os.Open(filepath.Join(dir, "cgroup.procs"))
+	if err != nil {
+		return nil, err
+	}
+	defer f.Close()
+
+	var (
+		s   = bufio.NewScanner(f)
+		out = []int{}
+	)
+
+	for s.Scan() {
+		if t := s.Text(); t != "" {
+			pid, err := strconv.Atoi(t)
+			if err != nil {
+				return nil, err
+			}
+			out = append(out, pid)
+		}
+	}
+	return out, nil
+}
+
+func ParseCgroupFile(subsystem string, r io.Reader) (string, error) {
+	s := bufio.NewScanner(r)
+
+	for s.Scan() {
+		if err := s.Err(); err != nil {
+			return "", err
+		}
+
+		text := s.Text()
+		parts := strings.Split(text, ":")
+
+		for _, subs := range strings.Split(parts[1], ",") {
+			if subs == subsystem {
+				return parts[2], nil
+			}
+		}
+	}
+
+	return "", NewNotFoundError(subsystem)
+}
+
+func PathExists(path string) bool {
+	if _, err := os.Stat(path); err != nil {
+		return false
+	}
+	return true
+}
+
+func EnterPid(cgroupPaths map[string]string, pid int) error {
+	for _, path := range cgroupPaths {
+		if PathExists(path) {
+			if err := ioutil.WriteFile(filepath.Join(path, "cgroup.procs"),
+				[]byte(strconv.Itoa(pid)), 0700); err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}
+
+// RemovePaths iterates over the provided paths removing them.
+// We trying to remove all paths five times with increasing delay between tries.
+// If after all there are not removed cgroups - appropriate error will be
+// returned.
+func RemovePaths(paths map[string]string) (err error) {
+	delay := 10 * time.Millisecond
+	for i := 0; i < 5; i++ {
+		if i != 0 {
+			time.Sleep(delay)
+			delay *= 2
+		}
+		for s, p := range paths {
+			os.RemoveAll(p)
+			// TODO: here probably should be logging
+			_, err := os.Stat(p)
+			// We need this strange way of checking cgroups existence because
+			// RemoveAll almost always returns error, even on already removed
+			// cgroups
+			if os.IsNotExist(err) {
+				delete(paths, s)
+			}
+		}
+		if len(paths) == 0 {
+			return nil
+		}
+	}
+	return fmt.Errorf("Failed to remove paths: %s", paths)
+}
+
+func GetHugePageSize() ([]string, error) {
+	var pageSizes []string
+	sizeList := []string{"B", "kB", "MB", "GB", "TB", "PB"}
+	files, err := ioutil.ReadDir("/sys/kernel/mm/hugepages")
+	if err != nil {
+		return pageSizes, err
+	}
+	for _, st := range files {
+		nameArray := strings.Split(st.Name(), "-")
+		pageSize, err := units.RAMInBytes(nameArray[1])
+		if err != nil {
+			return []string{}, err
+		}
+		sizeString := units.CustomSize("%g%s", float64(pageSize), 1024.0, sizeList)
+		pageSizes = append(pageSizes, sizeString)
+	}
+
+	return pageSizes, nil
+}

libcontainer/configs/cgroup.go

@@ -0,0 +1,98 @@
+package configs
+
+type FreezerState string
+
+const (
+	Undefined FreezerState = ""
+	Frozen    FreezerState = "FROZEN"
+	Thawed    FreezerState = "THAWED"
+)
+
+// TODO Windows: This can be factored out in the future as Cgroups are not
+// supported on the Windows platform.
+
+type Cgroup struct {
+	Name string `json:"name"`
+
+	// name of parent cgroup or slice
+	Parent string `json:"parent"`
+
+	// If this is true allow access to any kind of device within the container.  If false, allow access only to devices explicitly listed in the allowed_devices list.
+	AllowAllDevices bool `json:"allow_all_devices"`
+
+	AllowedDevices []*Device `json:"allowed_devices"`
+
+	DeniedDevices []*Device `json:"denied_devices"`
+
+	// Memory limit (in bytes)
+	Memory int64 `json:"memory"`
+
+	// Memory reservation or soft_limit (in bytes)
+	MemoryReservation int64 `json:"memory_reservation"`
+
+	// Total memory usage (memory + swap); set `-1' to disable swap
+	MemorySwap int64 `json:"memory_swap"`
+
+	// Kernel memory limit (in bytes)
+	KernelMemory int64 `json:"kernel_memory"`
+
+	// CPU shares (relative weight vs. other containers)
+	CpuShares int64 `json:"cpu_shares"`
+
+	// CPU hardcap limit (in usecs). Allowed cpu time in a given period.
+	CpuQuota int64 `json:"cpu_quota"`
+
+	// CPU period to be used for hardcapping (in usecs). 0 to use system default.
+	CpuPeriod int64 `json:"cpu_period"`
+
+	// How many time CPU will use in realtime scheduling (in usecs).
+	CpuRtRuntime int64 `json:"cpu_quota"`
+
+	// CPU period to be used for realtime scheduling (in usecs).
+	CpuRtPeriod int64 `json:"cpu_period"`
+
+	// CPU to use
+	CpusetCpus string `json:"cpuset_cpus"`
+
+	// MEM to use
+	CpusetMems string `json:"cpuset_mems"`
+
+	// IO read rate limit per cgroup per device, bytes per second.
+	BlkioThrottleReadBpsDevice string `json:"blkio_throttle_read_bps_device"`
+
+	// IO write rate limit per cgroup per divice, bytes per second.
+	BlkioThrottleWriteBpsDevice string `json:"blkio_throttle_write_bps_device"`
+
+	// IO read rate limit per cgroup per device, IO per second.
+	BlkioThrottleReadIOpsDevice string `json:"blkio_throttle_read_iops_device"`
+
+	// IO write rate limit per cgroup per device, IO per second.
+	BlkioThrottleWriteIOpsDevice string `json:"blkio_throttle_write_iops_device"`
+
+	// Specifies per cgroup weight, range is from 10 to 1000.
+	BlkioWeight int64 `json:"blkio_weight"`
+
+	// Weight per cgroup per device, can override BlkioWeight.
+	BlkioWeightDevice string `json:"blkio_weight_device"`
+
+	// set the freeze value for the process
+	Freezer FreezerState `json:"freezer"`
+
+	// Hugetlb limit (in bytes)
+	HugetlbLimit []*HugepageLimit `json:"hugetlb_limit"`
+
+	// Parent slice to use for systemd TODO: remove in favor or parent
+	Slice string `json:"slice"`
+
+	// Whether to disable OOM Killer
+	OomKillDisable bool `json:"oom_kill_disable"`
+
+	// Tuning swappiness behaviour per cgroup
+	MemorySwappiness int64 `json:"memory_swappiness"`
+
+	// Set priority of network traffic for container
+	NetPrioIfpriomap []*IfPrioMap `json:"net_prio_ifpriomap"`
+
+	// Set class identifier for container's network packets
+	NetClsClassid string `json:"net_cls_classid"`
+}

libcontainer/configs/config.go

@@ -0,0 +1,146 @@
+package configs
+
+type Rlimit struct {
+	Type int    `json:"type"`
+	Hard uint64 `json:"hard"`
+	Soft uint64 `json:"soft"`
+}
+
+// IDMap represents UID/GID Mappings for User Namespaces.
+type IDMap struct {
+	ContainerID int `json:"container_id"`
+	HostID      int `json:"host_id"`
+	Size        int `json:"size"`
+}
+
+type Seccomp struct {
+	Syscalls []*Syscall `json:"syscalls"`
+}
+
+type Action int
+
+const (
+	Kill Action = iota - 3
+	Trap
+	Allow
+)
+
+type Operator int
+
+const (
+	EqualTo Operator = iota
+	NotEqualTo
+	GreatherThan
+	LessThan
+	MaskEqualTo
+)
+
+type Arg struct {
+	Index int      `json:"index"`
+	Value uint32   `json:"value"`
+	Op    Operator `json:"op"`
+}
+
+type Syscall struct {
+	Value  int    `json:"value"`
+	Action Action `json:"action"`
+	Args   []*Arg `json:"args"`
+}
+
+// TODO Windows. Many of these fields should be factored out into those parts
+// which are common across platforms, and those which are platform specific.
+
+// Config defines configuration options for executing a process inside a contained environment.
+type Config struct {
+	// NoPivotRoot will use MS_MOVE and a chroot to jail the process into the container's rootfs
+	// This is a common option when the container is running in ramdisk
+	NoPivotRoot bool `json:"no_pivot_root"`
+
+	// ParentDeathSignal specifies the signal that is sent to the container's process in the case
+	// that the parent process dies.
+	ParentDeathSignal int `json:"parent_death_signal"`
+
+	// PivotDir allows a custom directory inside the container's root filesystem to be used as pivot, when NoPivotRoot is not set.
+	// When a custom PivotDir not set, a temporary dir inside the root filesystem will be used. The pivot dir needs to be writeable.
+	// This is required when using read only root filesystems. In these cases, a read/writeable path can be (bind) mounted somewhere inside the root filesystem to act as pivot.
+	PivotDir string `json:"pivot_dir"`
+
+	// Path to a directory containing the container's root filesystem.
+	Rootfs string `json:"rootfs"`
+
+	// Readonlyfs will remount the container's rootfs as readonly where only externally mounted
+	// bind mounts are writtable.
+	Readonlyfs bool `json:"readonlyfs"`
+
+	// Privatefs will mount the container's rootfs as private where mount points from the parent will not propogate
+	Privatefs bool `json:"privatefs"`
+
+	// Mounts specify additional source and destination paths that will be mounted inside the container's
+	// rootfs and mount namespace if specified
+	Mounts []*Mount `json:"mounts"`
+
+	// The device nodes that should be automatically created within the container upon container start.  Note, make sure that the node is marked as allowed in the cgroup as well!
+	Devices []*Device `json:"devices"`
+
+	MountLabel string `json:"mount_label"`
+
+	// Hostname optionally sets the container's hostname if provided
+	Hostname string `json:"hostname"`
+
+	// Namespaces specifies the container's namespaces that it should setup when cloning the init process
+	// If a namespace is not provided that namespace is shared from the container's parent process
+	Namespaces Namespaces `json:"namespaces"`
+
+	// Capabilities specify the capabilities to keep when executing the process inside the container
+	// All capbilities not specified will be dropped from the processes capability mask
+	Capabilities []string `json:"capabilities"`
+
+	// Networks specifies the container's network setup to be created
+	Networks []*Network `json:"networks"`
+
+	// Routes can be specified to create entries in the route table as the container is started
+	Routes []*Route `json:"routes"`
+
+	// Cgroups specifies specific cgroup settings for the various subsystems that the container is
+	// placed into to limit the resources the container has available
+	Cgroups *Cgroup `json:"cgroups"`
+
+	// AppArmorProfile specifies the profile to apply to the process running in the container and is
+	// change at the time the process is execed
+	AppArmorProfile string `json:"apparmor_profile"`
+
+	// ProcessLabel specifies the label to apply to the process running in the container.  It is
+	// commonly used by selinux
+	ProcessLabel string `json:"process_label"`
+
+	// Rlimits specifies the resource limits, such as max open files, to set in the container
+	// If Rlimits are not set, the container will inherit rlimits from the parent process
+	Rlimits []Rlimit `json:"rlimits"`
+
+	// AdditionalGroups specifies the gids that should be added to supplementary groups
+	// in addition to those that the user belongs to.
+	AdditionalGroups []string `json:"additional_groups"`
+
+	// UidMappings is an array of User ID mappings for User Namespaces
+	UidMappings []IDMap `json:"uid_mappings"`
+
+	// GidMappings is an array of Group ID mappings for User Namespaces
+	GidMappings []IDMap `json:"gid_mappings"`
+
+	// MaskPaths specifies paths within the container's rootfs to mask over with a bind
+	// mount pointing to /dev/null as to prevent reads of the file.
+	MaskPaths []string `json:"mask_paths"`
+
+	// ReadonlyPaths specifies paths within the container's rootfs to remount as read-only
+	// so that these files prevent any writes.
+	ReadonlyPaths []string `json:"readonly_paths"`
+
+	// SystemProperties is a map of properties and their values. It is the equivalent of using
+	// sysctl -w my.property.name value in Linux.
+	SystemProperties map[string]string `json:"system_properties"`
+
+	// Seccomp allows actions to be taken whenever a syscall is made within the container.
+	// By default, all syscalls are allowed with actions to allow, trap, kill, or return an errno
+	// can be specified on a per syscall basis.
+	Seccomp *Seccomp `json:"seccomp"`
+}

libcontainer/configs/config_test.go

@@ -0,0 +1,249 @@
+package configs
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+// Checks whether the expected capability is specified in the capabilities.
+func contains(expected string, values []string) bool {
+	for _, v := range values {
+		if v == expected {
+			return true
+		}
+	}
+	return false
+}
+
+func containsDevice(expected *Device, values []*Device) bool {
+	for _, d := range values {
+		if d.Path == expected.Path &&
+			d.Permissions == expected.Permissions &&
+			d.FileMode == expected.FileMode &&
+			d.Major == expected.Major &&
+			d.Minor == expected.Minor &&
+			d.Type == expected.Type {
+			return true
+		}
+	}
+	return false
+}
+
+func loadConfig(name string) (*Config, error) {
+	f, err := os.Open(filepath.Join("../sample_configs", name))
+	if err != nil {
+		return nil, err
+	}
+	defer f.Close()
+
+	var container *Config
+	if err := json.NewDecoder(f).Decode(&container); err != nil {
+		return nil, err
+	}
+
+	// Check that a config doesn't contain extra fields
+	var configMap, abstractMap map[string]interface{}
+
+	if _, err := f.Seek(0, 0); err != nil {
+		return nil, err
+	}
+
+	if err := json.NewDecoder(f).Decode(&abstractMap); err != nil {
+		return nil, err
+	}
+
+	configData, err := json.Marshal(&container)
+	if err != nil {
+		return nil, err
+	}
+
+	if err := json.Unmarshal(configData, &configMap); err != nil {
+		return nil, err
+	}
+
+	for k := range configMap {
+		delete(abstractMap, k)
+	}
+
+	if len(abstractMap) != 0 {
+		return nil, fmt.Errorf("unknown fields: %s", abstractMap)
+	}
+
+	return container, nil
+}
+
+func TestConfigJsonFormat(t *testing.T) {
+	container, err := loadConfig("attach_to_bridge.json")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if container.Hostname != "koye" {
+		t.Log("hostname is not set")
+		t.Fail()
+	}
+
+	if !container.Namespaces.Contains(NEWNET) {
+		t.Log("namespaces should contain NEWNET")
+		t.Fail()
+	}
+
+	if container.Namespaces.Contains(NEWUSER) {
+		t.Log("namespaces should not contain NEWUSER")
+		t.Fail()
+	}
+
+	if contains("SYS_ADMIN", container.Capabilities) {
+		t.Log("SYS_ADMIN should not be enabled in capabilities mask")
+		t.Fail()
+	}
+
+	if !contains("MKNOD", container.Capabilities) {
+		t.Log("MKNOD should be enabled in capabilities mask")
+		t.Fail()
+	}
+
+	if !contains("SYS_CHROOT", container.Capabilities) {
+		t.Log("capabilities mask should contain SYS_CHROOT")
+		t.Fail()
+	}
+
+	for _, n := range container.Networks {
+		if n.Type == "veth" {
+			if n.Bridge != "docker0" {
+				t.Logf("veth bridge should be docker0 but received %q", n.Bridge)
+				t.Fail()
+			}
+
+			if n.Address != "172.17.0.101/16" {
+				t.Logf("veth address should be 172.17.0.101/61 but received %q", n.Address)
+				t.Fail()
+			}
+
+			if n.Gateway != "172.17.42.1" {
+				t.Logf("veth gateway should be 172.17.42.1 but received %q", n.Gateway)
+				t.Fail()
+			}
+
+			if n.Mtu != 1500 {
+				t.Logf("veth mtu should be 1500 but received %d", n.Mtu)
+				t.Fail()
+			}
+
+			break
+		}
+	}
+	for _, d := range DefaultSimpleDevices {
+		if !containsDevice(d, container.Devices) {
+			t.Logf("expected device configuration for %s", d.Path)
+			t.Fail()
+		}
+	}
+}
+
+func TestApparmorProfile(t *testing.T) {
+	container, err := loadConfig("apparmor.json")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if container.AppArmorProfile != "docker-default" {
+		t.Fatalf("expected apparmor profile to be docker-default but received %q", container.AppArmorProfile)
+	}
+}
+
+func TestSelinuxLabels(t *testing.T) {
+	container, err := loadConfig("selinux.json")
+	if err != nil {
+		t.Fatal(err)
+	}
+	label := "system_u:system_r:svirt_lxc_net_t:s0:c164,c475"
+
+	if container.ProcessLabel != label {
+		t.Fatalf("expected process label %q but received %q", label, container.ProcessLabel)
+	}
+	if container.MountLabel != label {
+		t.Fatalf("expected mount label %q but received %q", label, container.MountLabel)
+	}
+}
+
+func TestRemoveNamespace(t *testing.T) {
+	ns := Namespaces{
+		{Type: NEWNET},
+	}
+	if !ns.Remove(NEWNET) {
+		t.Fatal("NEWNET was not removed")
+	}
+	if len(ns) != 0 {
+		t.Fatalf("namespaces should have 0 items but reports %d", len(ns))
+	}
+}
+
+func TestHostUIDNoUSERNS(t *testing.T) {
+	config := &Config{
+		Namespaces: Namespaces{},
+	}
+	uid, err := config.HostUID()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if uid != 0 {
+		t.Fatalf("expected uid 0 with no USERNS but received %d", uid)
+	}
+}
+
+func TestHostUIDWithUSERNS(t *testing.T) {
+	config := &Config{
+		Namespaces: Namespaces{{Type: NEWUSER}},
+		UidMappings: []IDMap{
+			{
+				ContainerID: 0,
+				HostID:      1000,
+				Size:        1,
+			},
+		},
+	}
+	uid, err := config.HostUID()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if uid != 1000 {
+		t.Fatalf("expected uid 1000 with no USERNS but received %d", uid)
+	}
+}
+
+func TestHostGIDNoUSERNS(t *testing.T) {
+	config := &Config{
+		Namespaces: Namespaces{},
+	}
+	uid, err := config.HostGID()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if uid != 0 {
+		t.Fatalf("expected gid 0 with no USERNS but received %d", uid)
+	}
+}
+
+func TestHostGIDWithUSERNS(t *testing.T) {
+	config := &Config{
+		Namespaces: Namespaces{{Type: NEWUSER}},
+		GidMappings: []IDMap{
+			{
+				ContainerID: 0,
+				HostID:      1000,
+				Size:        1,
+			},
+		},
+	}
+	uid, err := config.HostGID()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if uid != 1000 {
+		t.Fatalf("expected gid 1000 with no USERNS but received %d", uid)
+	}
+}

libcontainer/configs/config_unix.go

@@ -0,0 +1,51 @@
+// +build freebsd linux
+
+package configs
+
+import "fmt"
+
+// Gets the root uid for the process on host which could be non-zero
+// when user namespaces are enabled.
+func (c Config) HostUID() (int, error) {
+	if c.Namespaces.Contains(NEWUSER) {
+		if c.UidMappings == nil {
+			return -1, fmt.Errorf("User namespaces enabled, but no user mappings found.")
+		}
+		id, found := c.hostIDFromMapping(0, c.UidMappings)
+		if !found {
+			return -1, fmt.Errorf("User namespaces enabled, but no root user mapping found.")
+		}
+		return id, nil
+	}
+	// Return default root uid 0
+	return 0, nil
+}
+
+// Gets the root uid for the process on host which could be non-zero
+// when user namespaces are enabled.
+func (c Config) HostGID() (int, error) {
+	if c.Namespaces.Contains(NEWUSER) {
+		if c.GidMappings == nil {
+			return -1, fmt.Errorf("User namespaces enabled, but no gid mappings found.")
+		}
+		id, found := c.hostIDFromMapping(0, c.GidMappings)
+		if !found {
+			return -1, fmt.Errorf("User namespaces enabled, but no root user mapping found.")
+		}
+		return id, nil
+	}
+	// Return default root uid 0
+	return 0, nil
+}
+
+// Utility function that gets a host ID for a container ID from user namespace map
+// if that ID is present in the map.
+func (c Config) hostIDFromMapping(containerID int, uMap []IDMap) (int, bool) {
+	for _, m := range uMap {
+		if (containerID >= m.ContainerID) && (containerID <= (m.ContainerID + m.Size - 1)) {
+			hostID := m.HostID + (containerID - m.ContainerID)
+			return hostID, true
+		}
+	}
+	return -1, false
+}

libcontainer/configs/device.go

@@ -0,0 +1,54 @@
+package configs
+
+import (
+	"fmt"
+	"os"
+)
+
+const (
+	Wildcard = -1
+)
+
+// TODO Windows: This can be factored out in the future
+
+type Device struct {
+	// Device type, block, char, etc.
+	Type rune `json:"type"`
+
+	// Path to the device.
+	Path string `json:"path"`
+
+	// Major is the device's major number.
+	Major int64 `json:"major"`
+
+	// Minor is the device's minor number.
+	Minor int64 `json:"minor"`
+
+	// Cgroup permissions format, rwm.
+	Permissions string `json:"permissions"`
+
+	// FileMode permission bits for the device.
+	FileMode os.FileMode `json:"file_mode"`
+
+	// Uid of the device.
+	Uid uint32 `json:"uid"`
+
+	// Gid of the device.
+	Gid uint32 `json:"gid"`
+}
+
+func (d *Device) CgroupString() string {
+	return fmt.Sprintf("%c %s:%s %s", d.Type, deviceNumberString(d.Major), deviceNumberString(d.Minor), d.Permissions)
+}
+
+func (d *Device) Mkdev() int {
+	return int((d.Major << 8) | (d.Minor & 0xff) | ((d.Minor & 0xfff00) << 12))
+}
+
+// deviceNumberString converts the device number to a string return result.
+func deviceNumberString(number int64) string {
+	if number == Wildcard {
+		return "*"
+	}
+	return fmt.Sprint(number)
+}

libcontainer/configs/device_defaults.go

@@ -0,0 +1,139 @@
+// +build linux freebsd
+
+package configs
+
+var (
+	// These are devices that are to be both allowed and created.
+	DefaultSimpleDevices = []*Device{
+		// /dev/null and zero
+		{
+			Path:        "/dev/null",
+			Type:        'c',
+			Major:       1,
+			Minor:       3,
+			Permissions: "rwm",
+			FileMode:    0666,
+		},
+		{
+			Path:        "/dev/zero",
+			Type:        'c',
+			Major:       1,
+			Minor:       5,
+			Permissions: "rwm",
+			FileMode:    0666,
+		},
+
+		{
+			Path:        "/dev/full",
+			Type:        'c',
+			Major:       1,
+			Minor:       7,
+			Permissions: "rwm",
+			FileMode:    0666,
+		},
+
+		// consoles and ttys
+		{
+			Path:        "/dev/tty",
+			Type:        'c',
+			Major:       5,
+			Minor:       0,
+			Permissions: "rwm",
+			FileMode:    0666,
+		},
+
+		// /dev/urandom,/dev/random
+		{
+			Path:        "/dev/urandom",
+			Type:        'c',
+			Major:       1,
+			Minor:       9,
+			Permissions: "rwm",
+			FileMode:    0666,
+		},
+		{
+			Path:        "/dev/random",
+			Type:        'c',
+			Major:       1,
+			Minor:       8,
+			Permissions: "rwm",
+			FileMode:    0666,
+		},
+	}
+	DefaultAllowedDevices = append([]*Device{
+		// allow mknod for any device
+		{
+			Type:        'c',
+			Major:       Wildcard,
+			Minor:       Wildcard,
+			Permissions: "m",
+		},
+		{
+			Type:        'b',
+			Major:       Wildcard,
+			Minor:       Wildcard,
+			Permissions: "m",
+		},
+
+		{
+			Path:        "/dev/console",
+			Type:        'c',
+			Major:       5,
+			Minor:       1,
+			Permissions: "rwm",
+		},
+		{
+			Path:        "/dev/tty0",
+			Type:        'c',
+			Major:       4,
+			Minor:       0,
+			Permissions: "rwm",
+		},
+		{
+			Path:        "/dev/tty1",
+			Type:        'c',
+			Major:       4,
+			Minor:       1,
+			Permissions: "rwm",
+		},
+		// /dev/pts/ - pts namespaces are "coming soon"
+		{
+			Path:        "",
+			Type:        'c',
+			Major:       136,
+			Minor:       Wildcard,
+			Permissions: "rwm",
+		},
+		{
+			Path:        "",
+			Type:        'c',
+			Major:       5,
+			Minor:       2,
+			Permissions: "rwm",
+		},
+
+		// tuntap
+		{
+			Path:        "",
+			Type:        'c',
+			Major:       10,
+			Minor:       200,
+			Permissions: "rwm",
+		},
+	}, DefaultSimpleDevices...)
+	DefaultAutoCreatedDevices = append([]*Device{
+		{
+			// /dev/fuse is created but not allowed.
+			// This is to allow java to work.  Because java
+			// Insists on there being a /dev/fuse
+			// https://github.com/docker/docker/issues/514
+			// https://github.com/docker/docker/issues/2393
+			//
+			Path:        "/dev/fuse",
+			Type:        'c',
+			Major:       10,
+			Minor:       229,
+			Permissions: "rwm",
+		},
+	}, DefaultSimpleDevices...)
+)

libcontainer/configs/hugepage_limit.go

@@ -0,0 +1,9 @@
+package configs
+
+type HugepageLimit struct {
+	// which type of hugepage to limit.
+	Pagesize string `json:"page_size"`
+
+	// usage limit for hugepage.
+	Limit int `json:"limit"`
+}

libcontainer/configs/interface_priority_map.go

@@ -0,0 +1,14 @@
+package configs
+
+import (
+	"fmt"
+)
+
+type IfPrioMap struct {
+	Interface string `json:"interface"`
+	Priority  int64  `json:"priority"`
+}
+
+func (i *IfPrioMap) CgroupString() string {
+	return fmt.Sprintf("%s %d", i.Interface, i.Priority)
+}

libcontainer/configs/mount.go

@@ -0,0 +1,34 @@
+package configs
+
+type Mount struct {
+	// Source path for the mount.
+	Source string `json:"source"`
+
+	// Destination path for the mount inside the container.
+	Destination string `json:"destination"`
+
+	// Device the mount is for.
+	Device string `json:"device"`
+
+	// Mount flags.
+	Flags int `json:"flags"`
+
+	// Mount data applied to the mount.
+	Data string `json:"data"`
+
+	// Relabel source if set, "z" indicates shared, "Z" indicates unshared.
+	Relabel string `json:"relabel"`
+
+	// Optional Command to be run before Source is mounted.
+	PremountCmds []Command `json:"premount_cmds"`
+
+	// Optional Command to be run after Source is mounted.
+	PostmountCmds []Command `json:"postmount_cmds"`
+}
+
+type Command struct {
+	Path string   `json:"path"`
+	Args []string `json:"args"`
+	Env  []string `json:"env"`
+	Dir  string   `json:"dir"`
+}

libcontainer/configs/namespaces.go

@@ -0,0 +1,5 @@
+package configs
+
+type NamespaceType string
+
+type Namespaces []Namespace

libcontainer/configs/namespaces_syscall.go

@@ -0,0 +1,31 @@
+// +build linux
+
+package configs
+
+import "syscall"
+
+func (n *Namespace) Syscall() int {
+	return namespaceInfo[n.Type]
+}
+
+var namespaceInfo = map[NamespaceType]int{
+	NEWNET:  syscall.CLONE_NEWNET,
+	NEWNS:   syscall.CLONE_NEWNS,
+	NEWUSER: syscall.CLONE_NEWUSER,
+	NEWIPC:  syscall.CLONE_NEWIPC,
+	NEWUTS:  syscall.CLONE_NEWUTS,
+	NEWPID:  syscall.CLONE_NEWPID,
+}
+
+// CloneFlags parses the container's Namespaces options to set the correct
+// flags on clone, unshare. This functions returns flags only for new namespaces.
+func (n *Namespaces) CloneFlags() uintptr {
+	var flag int
+	for _, v := range *n {
+		if v.Path != "" {
+			continue
+		}
+		flag |= namespaceInfo[v.Type]
+	}
+	return uintptr(flag)
+}

libcontainer/configs/namespaces_syscall_unsupported.go

@@ -0,0 +1,15 @@
+// +build !linux,!windows
+
+package configs
+
+func (n *Namespace) Syscall() int {
+	panic("No namespace syscall support")
+	return 0
+}
+
+// CloneFlags parses the container's Namespaces options to set the correct
+// flags on clone, unshare. This functions returns flags only for new namespaces.
+func (n *Namespaces) CloneFlags() uintptr {
+	panic("No namespace syscall support")
+	return uintptr(0)
+}

libcontainer/configs/namespaces_unix.go

@@ -0,0 +1,89 @@
+// +build linux freebsd
+
+package configs
+
+import "fmt"
+
+const (
+	NEWNET  NamespaceType = "NEWNET"
+	NEWPID  NamespaceType = "NEWPID"
+	NEWNS   NamespaceType = "NEWNS"
+	NEWUTS  NamespaceType = "NEWUTS"
+	NEWIPC  NamespaceType = "NEWIPC"
+	NEWUSER NamespaceType = "NEWUSER"
+)
+
+func NamespaceTypes() []NamespaceType {
+	return []NamespaceType{
+		NEWNET,
+		NEWPID,
+		NEWNS,
+		NEWUTS,
+		NEWIPC,
+		NEWUSER,
+	}
+}
+
+// Namespace defines configuration for each namespace.  It specifies an
+// alternate path that is able to be joined via setns.
+type Namespace struct {
+	Type NamespaceType `json:"type"`
+	Path string        `json:"path"`
+}
+
+func (n *Namespace) GetPath(pid int) string {
+	if n.Path != "" {
+		return n.Path
+	}
+	return fmt.Sprintf("/proc/%d/ns/%s", pid, n.file())
+}
+
+func (n *Namespace) file() string {
+	file := ""
+	switch n.Type {
+	case NEWNET:
+		file = "net"
+	case NEWNS:
+		file = "mnt"
+	case NEWPID:
+		file = "pid"
+	case NEWIPC:
+		file = "ipc"
+	case NEWUSER:
+		file = "user"
+	case NEWUTS:
+		file = "uts"
+	}
+	return file
+}
+
+func (n *Namespaces) Remove(t NamespaceType) bool {
+	i := n.index(t)
+	if i == -1 {
+		return false
+	}
+	*n = append((*n)[:i], (*n)[i+1:]...)
+	return true
+}
+
+func (n *Namespaces) Add(t NamespaceType, path string) {
+	i := n.index(t)
+	if i == -1 {
+		*n = append(*n, Namespace{Type: t, Path: path})
+		return
+	}
+	(*n)[i].Path = path
+}
+
+func (n *Namespaces) index(t NamespaceType) int {
+	for i, ns := range *n {
+		if ns.Type == t {
+			return i
+		}
+	}
+	return -1
+}
+
+func (n *Namespaces) Contains(t NamespaceType) bool {
+	return n.index(t) != -1
+}

libcontainer/configs/namespaces_windows.go

@@ -0,0 +1,6 @@
+package configs
+
+// Namespace defines configuration for each namespace.  It specifies an
+// alternate path that is able to be joined via setns.
+type Namespace struct {
+}

libcontainer/configs/network.go

@@ -0,0 +1,72 @@
+package configs
+
+// Network defines configuration for a container's networking stack
+//
+// The network configuration can be omitted from a container causing the
+// container to be setup with the host's networking stack
+type Network struct {
+	// Type sets the networks type, commonly veth and loopback
+	Type string `json:"type"`
+
+	// Name of the network interface
+	Name string `json:"name"`
+
+	// The bridge to use.
+	Bridge string `json:"bridge"`
+
+	// MacAddress contains the MAC address to set on the network interface
+	MacAddress string `json:"mac_address"`
+
+	// Address contains the IPv4 and mask to set on the network interface
+	Address string `json:"address"`
+
+	// Gateway sets the gateway address that is used as the default for the interface
+	Gateway string `json:"gateway"`
+
+	// IPv6Address contains the IPv6 and mask to set on the network interface
+	IPv6Address string `json:"ipv6_address"`
+
+	// IPv6Gateway sets the ipv6 gateway address that is used as the default for the interface
+	IPv6Gateway string `json:"ipv6_gateway"`
+
+	// Mtu sets the mtu value for the interface and will be mirrored on both the host and
+	// container's interfaces if a pair is created, specifically in the case of type veth
+	// Note: This does not apply to loopback interfaces.
+	Mtu int `json:"mtu"`
+
+	// TxQueueLen sets the tx_queuelen value for the interface and will be mirrored on both the host and
+	// container's interfaces if a pair is created, specifically in the case of type veth
+	// Note: This does not apply to loopback interfaces.
+	TxQueueLen int `json:"txqueuelen"`
+
+	// HostInterfaceName is a unique name of a veth pair that resides on in the host interface of the
+	// container.
+	HostInterfaceName string `json:"host_interface_name"`
+
+	// HairpinMode specifies if hairpin NAT should be enabled on the virtual interface
+	// bridge port in the case of type veth
+	// Note: This is unsupported on some systems.
+	// Note: This does not apply to loopback interfaces.
+	HairpinMode bool `json:"hairpin_mode"`
+}
+
+// Routes can be specified to create entries in the route table as the container is started
+//
+// All of destination, source, and gateway should be either IPv4 or IPv6.
+// One of the three options must be present, and omitted entries will use their
+// IP family default for the route table.  For IPv4 for example, setting the
+// gateway to 1.2.3.4 and the interface to eth0 will set up a standard
+// destination of 0.0.0.0(or *) when viewed in the route table.
+type Route struct {
+	// Sets the destination and mask, should be a CIDR.  Accepts IPv4 and IPv6
+	Destination string `json:"destination"`
+
+	// Sets the source and mask, should be a CIDR.  Accepts IPv4 and IPv6
+	Source string `json:"source"`
+
+	// Sets the gateway.  Accepts IPv4 and IPv6
+	Gateway string `json:"gateway"`
+
+	// The device to set this route up for, for example: eth0
+	InterfaceName string `json:"interface_name"`
+}

libcontainer/configs/validate/config.go

@@ -0,0 +1,93 @@
+package validate
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+
+	"github.com/docker/libcontainer/configs"
+)
+
+type Validator interface {
+	Validate(*configs.Config) error
+}
+
+func New() Validator {
+	return &ConfigValidator{}
+}
+
+type ConfigValidator struct {
+}
+
+func (v *ConfigValidator) Validate(config *configs.Config) error {
+	if err := v.rootfs(config); err != nil {
+		return err
+	}
+	if err := v.network(config); err != nil {
+		return err
+	}
+	if err := v.hostname(config); err != nil {
+		return err
+	}
+	if err := v.security(config); err != nil {
+		return err
+	}
+	if err := v.usernamespace(config); err != nil {
+		return err
+	}
+	return nil
+}
+
+// rootfs validates the the rootfs is an absolute path and is not a symlink
+// to the container's root filesystem.
+func (v *ConfigValidator) rootfs(config *configs.Config) error {
+	cleaned, err := filepath.Abs(config.Rootfs)
+	if err != nil {
+		return err
+	}
+	if cleaned, err = filepath.EvalSymlinks(cleaned); err != nil {
+		return err
+	}
+	if config.Rootfs != cleaned {
+		return fmt.Errorf("%s is not an absolute path or is a symlink", config.Rootfs)
+	}
+	return nil
+}
+
+func (v *ConfigValidator) network(config *configs.Config) error {
+	if !config.Namespaces.Contains(configs.NEWNET) {
+		if len(config.Networks) > 0 || len(config.Routes) > 0 {
+			return fmt.Errorf("unable to apply network settings without a private NET namespace")
+		}
+	}
+	return nil
+}
+
+func (v *ConfigValidator) hostname(config *configs.Config) error {
+	if config.Hostname != "" && !config.Namespaces.Contains(configs.NEWUTS) {
+		return fmt.Errorf("unable to set hostname without a private UTS namespace")
+	}
+	return nil
+}
+
+func (v *ConfigValidator) security(config *configs.Config) error {
+	// restrict sys without mount namespace
+	if (len(config.MaskPaths) > 0 || len(config.ReadonlyPaths) > 0) &&
+		!config.Namespaces.Contains(configs.NEWNS) {
+		return fmt.Errorf("unable to restrict sys entries without a private MNT namespace")
+	}
+	return nil
+}
+
+func (v *ConfigValidator) usernamespace(config *configs.Config) error {
+	if config.Namespaces.Contains(configs.NEWUSER) {
+		if _, err := os.Stat("/proc/self/ns/user"); os.IsNotExist(err) {
+			return fmt.Errorf("USER namespaces aren't enabled in the kernel")
+		}
+	} else {
+		if config.UidMappings != nil || config.GidMappings != nil {
+			return fmt.Errorf("User namespace mappings specified, but USER namespace isn't enabled in the config")
+		}
+	}
+	return nil
+}

libcontainer/console.go

@@ -0,0 +1,15 @@
+package libcontainer
+
+import "io"
+
+// Console represents a pseudo TTY.
+type Console interface {
+	io.ReadWriter
+	io.Closer
+
+	// Path returns the filesystem path to the slave side of the pty.
+	Path() string
+
+	// Fd returns the fd for the master of the pty.
+	Fd() uintptr
+}

libcontainer/console_freebsd.go

@@ -0,0 +1,13 @@
+// +build freebsd
+
+package libcontainer
+
+import (
+	"errors"
+)
+
+// newConsole returns an initalized console that can be used within a container by copying bytes
+// from the master side to the slave that is attached as the tty for the container's init process.
+func newConsole(uid, gid int) (Console, error) {
+	return nil, errors.New("libcontainer console is not supported on FreeBSD")
+}

libcontainer/console_linux.go

@@ -0,0 +1,145 @@
+package libcontainer
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"syscall"
+	"unsafe"
+
+	"github.com/docker/libcontainer/label"
+)
+
+// newConsole returns an initalized console that can be used within a container by copying bytes
+// from the master side to the slave that is attached as the tty for the container's init process.
+func newConsole(uid, gid int) (Console, error) {
+	master, err := os.OpenFile("/dev/ptmx", syscall.O_RDWR|syscall.O_NOCTTY|syscall.O_CLOEXEC, 0)
+	if err != nil {
+		return nil, err
+	}
+	console, err := ptsname(master)
+	if err != nil {
+		return nil, err
+	}
+	if err := unlockpt(master); err != nil {
+		return nil, err
+	}
+	if err := os.Chmod(console, 0600); err != nil {
+		return nil, err
+	}
+	if err := os.Chown(console, uid, gid); err != nil {
+		return nil, err
+	}
+	return &linuxConsole{
+		slavePath: console,
+		master:    master,
+	}, nil
+}
+
+// newConsoleFromPath is an internal function returning an initialized console for use inside
+// a container's MNT namespace.
+func newConsoleFromPath(slavePath string) *linuxConsole {
+	return &linuxConsole{
+		slavePath: slavePath,
+	}
+}
+
+// linuxConsole is a linux psuedo TTY for use within a container.
+type linuxConsole struct {
+	master    *os.File
+	slavePath string
+}
+
+func (c *linuxConsole) Fd() uintptr {
+	return c.master.Fd()
+}
+
+func (c *linuxConsole) Path() string {
+	return c.slavePath
+}
+
+func (c *linuxConsole) Read(b []byte) (int, error) {
+	return c.master.Read(b)
+}
+
+func (c *linuxConsole) Write(b []byte) (int, error) {
+	return c.master.Write(b)
+}
+
+func (c *linuxConsole) Close() error {
+	if m := c.master; m != nil {
+		return m.Close()
+	}
+	return nil
+}
+
+// mount initializes the console inside the rootfs mounting with the specified mount label
+// and applying the correct ownership of the console.
+func (c *linuxConsole) mount(rootfs, mountLabel string, uid, gid int) error {
+	oldMask := syscall.Umask(0000)
+	defer syscall.Umask(oldMask)
+	if err := label.SetFileLabel(c.slavePath, mountLabel); err != nil {
+		return err
+	}
+	dest := filepath.Join(rootfs, "/dev/console")
+	f, err := os.Create(dest)
+	if err != nil && !os.IsExist(err) {
+		return err
+	}
+	if f != nil {
+		f.Close()
+	}
+	return syscall.Mount(c.slavePath, dest, "bind", syscall.MS_BIND, "")
+}
+
+// dupStdio opens the slavePath for the console and dups the fds to the current
+// processes stdio, fd 0,1,2.
+func (c *linuxConsole) dupStdio() error {
+	slave, err := c.open(syscall.O_RDWR)
+	if err != nil {
+		return err
+	}
+	fd := int(slave.Fd())
+	for _, i := range []int{0, 1, 2} {
+		if err := syscall.Dup3(fd, i, 0); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+// open is a clone of os.OpenFile without the O_CLOEXEC used to open the pty slave.
+func (c *linuxConsole) open(flag int) (*os.File, error) {
+	r, e := syscall.Open(c.slavePath, flag, 0)
+	if e != nil {
+		return nil, &os.PathError{
+			Op:   "open",
+			Path: c.slavePath,
+			Err:  e,
+		}
+	}
+	return os.NewFile(uintptr(r), c.slavePath), nil
+}
+
+func ioctl(fd uintptr, flag, data uintptr) error {
+	if _, _, err := syscall.Syscall(syscall.SYS_IOCTL, fd, flag, data); err != 0 {
+		return err
+	}
+	return nil
+}
+
+// unlockpt unlocks the slave pseudoterminal device corresponding to the master pseudoterminal referred to by f.
+// unlockpt should be called before opening the slave side of a pty.
+func unlockpt(f *os.File) error {
+	var u int32
+	return ioctl(f.Fd(), syscall.TIOCSPTLCK, uintptr(unsafe.Pointer(&u)))
+}
+
+// ptsname retrieves the name of the first available pts for the given master.
+func ptsname(f *os.File) (string, error) {
+	var n int32
+	if err := ioctl(f.Fd(), syscall.TIOCGPTN, uintptr(unsafe.Pointer(&n))); err != nil {
+		return "", err
+	}
+	return fmt.Sprintf("/dev/pts/%d", n), nil
+}

libcontainer/console_windows.go

@@ -0,0 +1,30 @@
+package libcontainer
+
+// newConsole returns an initalized console that can be used within a container
+func newConsole(uid, gid int) (Console, error) {
+	return &windowsConsole{}, nil
+}
+
+// windowsConsole is a Windows psuedo TTY for use within a container.
+type windowsConsole struct {
+}
+
+func (c *windowsConsole) Fd() uintptr {
+	return 0
+}
+
+func (c *windowsConsole) Path() string {
+	return ""
+}
+
+func (c *windowsConsole) Read(b []byte) (int, error) {
+	return 0, nil
+}
+
+func (c *windowsConsole) Write(b []byte) (int, error) {
+	return 0, nil
+}
+
+func (c *windowsConsole) Close() error {
+	return nil
+}

libcontainer/container.go

@@ -0,0 +1,162 @@
+// Libcontainer provides a native Go implementation for creating containers
+// with namespaces, cgroups, capabilities, and filesystem access controls.
+// It allows you to manage the lifecycle of the container performing additional operations
+// after the container is created.
+package libcontainer
+
+import (
+	"github.com/docker/libcontainer/configs"
+)
+
+// The status of a container.
+type Status int
+
+const (
+	// The container exists and is running.
+	Running Status = iota + 1
+
+	// The container exists, it is in the process of being paused.
+	Pausing
+
+	// The container exists, but all its processes are paused.
+	Paused
+
+	// The container exists, but its state is saved on disk
+	Checkpointed
+
+	// The container does not exist.
+	Destroyed
+)
+
+// State represents a running container's state
+type State struct {
+	// ID is the container ID.
+	ID string `json:"id"`
+
+	// InitProcessPid is the init process id in the parent namespace.
+	InitProcessPid int `json:"init_process_pid"`
+
+	// InitProcessStartTime is the init process start time.
+	InitProcessStartTime string `json:"init_process_start"`
+
+	// Path to all the cgroups setup for a container. Key is cgroup subsystem name
+	// with the value as the path.
+	CgroupPaths map[string]string `json:"cgroup_paths"`
+
+	// NamespacePaths are filepaths to the container's namespaces. Key is the namespace type
+	// with the value as the path.
+	NamespacePaths map[configs.NamespaceType]string `json:"namespace_paths"`
+
+	// Config is the container's configuration.
+	Config configs.Config `json:"config"`
+
+	// Container's standard descriptors (std{in,out,err}), needed for checkpoint and restore
+	ExternalDescriptors []string `json:"external_descriptors,omitempty"`
+}
+
+// A libcontainer container object.
+//
+// Each container is thread-safe within the same process. Since a container can
+// be destroyed by a separate process, any function may return that the container
+// was not found.
+type Container interface {
+	// Returns the ID of the container
+	ID() string
+
+	// Returns the current status of the container.
+	//
+	// errors:
+	// ContainerDestroyed - Container no longer exists,
+	// Systemerror - System error.
+	Status() (Status, error)
+
+	// State returns the current container's state information.
+	//
+	// errors:
+	// Systemerror - System error.
+	State() (*State, error)
+
+	// Returns the current config of the container.
+	Config() configs.Config
+
+	// Returns the PIDs inside this container. The PIDs are in the namespace of the calling process.
+	//
+	// errors:
+	// ContainerDestroyed - Container no longer exists,
+	// Systemerror - System error.
+	//
+	// Some of the returned PIDs may no longer refer to processes in the Container, unless
+	// the Container state is PAUSED in which case every PID in the slice is valid.
+	Processes() ([]int, error)
+
+	// Returns statistics for the container.
+	//
+	// errors:
+	// ContainerDestroyed - Container no longer exists,
+	// Systemerror - System error.
+	Stats() (*Stats, error)
+
+	// Set cgroup resources of container as configured
+	//
+	// We can use this to change resources when containers are running.
+	//
+	// errors:
+	// Systemerror - System error.
+	Set(config configs.Config) error
+
+	// Start a process inside the container. Returns error if process fails to
+	// start. You can track process lifecycle with passed Process structure.
+	//
+	// errors:
+	// ContainerDestroyed - Container no longer exists,
+	// ConfigInvalid - config is invalid,
+	// ContainerPaused - Container is paused,
+	// Systemerror - System error.
+	Start(process *Process) (err error)
+
+	// Checkpoint checkpoints the running container's state to disk using the criu(8) utility.
+	//
+	// errors:
+	// Systemerror - System error.
+	Checkpoint(criuOpts *CriuOpts) error
+
+	// Restore restores the checkpointed container to a running state using the criu(8) utiity.
+	//
+	// errors:
+	// Systemerror - System error.
+	Restore(process *Process, criuOpts *CriuOpts) error
+
+	// Destroys the container after killing all running processes.
+	//
+	// Any event registrations are removed before the container is destroyed.
+	// No error is returned if the container is already destroyed.
+	//
+	// errors:
+	// Systemerror - System error.
+	Destroy() error
+
+	// If the Container state is RUNNING or PAUSING, sets the Container state to PAUSING and pauses
+	// the execution of any user processes. Asynchronously, when the container finished being paused the
+	// state is changed to PAUSED.
+	// If the Container state is PAUSED, do nothing.
+	//
+	// errors:
+	// ContainerDestroyed - Container no longer exists,
+	// Systemerror - System error.
+	Pause() error
+
+	// If the Container state is PAUSED, resumes the execution of any user processes in the
+	// Container before setting the Container state to RUNNING.
+	// If the Container state is RUNNING, do nothing.
+	//
+	// errors:
+	// ContainerDestroyed - Container no longer exists,
+	// Systemerror - System error.
+	Resume() error
+
+	// NotifyOOM returns a read-only channel signaling when the container receives an OOM notification.
+	//
+	// errors:
+	// Systemerror - System error.
+	NotifyOOM() (<-chan struct{}, error)
+}

libcontainer/container_linux.go

@@ -0,0 +1,779 @@
+// +build linux
+
+package libcontainer
+
+import (
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strings"
+	"sync"
+	"syscall"
+
+	"github.com/Sirupsen/logrus"
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+	"github.com/docker/libcontainer/criurpc"
+	"github.com/golang/protobuf/proto"
+)
+
+const stdioFdCount = 3
+
+type linuxContainer struct {
+	id            string
+	root          string
+	config        *configs.Config
+	cgroupManager cgroups.Manager
+	initPath      string
+	initArgs      []string
+	initProcess   parentProcess
+	criuPath      string
+	m             sync.Mutex
+}
+
+// ID returns the container's unique ID
+func (c *linuxContainer) ID() string {
+	return c.id
+}
+
+// Config returns the container's configuration
+func (c *linuxContainer) Config() configs.Config {
+	return *c.config
+}
+
+func (c *linuxContainer) Status() (Status, error) {
+	c.m.Lock()
+	defer c.m.Unlock()
+	return c.currentStatus()
+}
+
+func (c *linuxContainer) State() (*State, error) {
+	c.m.Lock()
+	defer c.m.Unlock()
+	return c.currentState()
+}
+
+func (c *linuxContainer) Processes() ([]int, error) {
+	pids, err := c.cgroupManager.GetPids()
+	if err != nil {
+		return nil, newSystemError(err)
+	}
+	return pids, nil
+}
+
+func (c *linuxContainer) Stats() (*Stats, error) {
+	var (
+		err   error
+		stats = &Stats{}
+	)
+	if stats.CgroupStats, err = c.cgroupManager.GetStats(); err != nil {
+		return stats, newSystemError(err)
+	}
+	for _, iface := range c.config.Networks {
+		switch iface.Type {
+		case "veth":
+			istats, err := getNetworkInterfaceStats(iface.HostInterfaceName)
+			if err != nil {
+				return stats, newSystemError(err)
+			}
+			stats.Interfaces = append(stats.Interfaces, istats)
+		}
+	}
+	return stats, nil
+}
+
+func (c *linuxContainer) Set(config configs.Config) error {
+	c.m.Lock()
+	defer c.m.Unlock()
+	c.config = &config
+	return c.cgroupManager.Set(c.config)
+}
+
+func (c *linuxContainer) Start(process *Process) error {
+	c.m.Lock()
+	defer c.m.Unlock()
+	status, err := c.currentStatus()
+	if err != nil {
+		return err
+	}
+	doInit := status == Destroyed
+	parent, err := c.newParentProcess(process, doInit)
+	if err != nil {
+		return newSystemError(err)
+	}
+	if err := parent.start(); err != nil {
+		// terminate the process to ensure that it properly is reaped.
+		if err := parent.terminate(); err != nil {
+			logrus.Warn(err)
+		}
+		return newSystemError(err)
+	}
+	process.ops = parent
+	if doInit {
+		c.updateState(parent)
+	}
+	return nil
+}
+
+func (c *linuxContainer) newParentProcess(p *Process, doInit bool) (parentProcess, error) {
+	parentPipe, childPipe, err := newPipe()
+	if err != nil {
+		return nil, newSystemError(err)
+	}
+	cmd, err := c.commandTemplate(p, childPipe)
+	if err != nil {
+		return nil, newSystemError(err)
+	}
+	if !doInit {
+		return c.newSetnsProcess(p, cmd, parentPipe, childPipe), nil
+	}
+	return c.newInitProcess(p, cmd, parentPipe, childPipe)
+}
+
+func (c *linuxContainer) commandTemplate(p *Process, childPipe *os.File) (*exec.Cmd, error) {
+	cmd := &exec.Cmd{
+		Path: c.initPath,
+		Args: c.initArgs,
+	}
+	cmd.Stdin = p.Stdin
+	cmd.Stdout = p.Stdout
+	cmd.Stderr = p.Stderr
+	cmd.Dir = c.config.Rootfs
+	if cmd.SysProcAttr == nil {
+		cmd.SysProcAttr = &syscall.SysProcAttr{}
+	}
+	cmd.ExtraFiles = append(p.ExtraFiles, childPipe)
+	cmd.Env = append(cmd.Env, fmt.Sprintf("_LIBCONTAINER_INITPIPE=%d", stdioFdCount+len(cmd.ExtraFiles)-1))
+	// NOTE: when running a container with no PID namespace and the parent process spawning the container is
+	// PID1 the pdeathsig is being delivered to the container's init process by the kernel for some reason
+	// even with the parent still running.
+	if c.config.ParentDeathSignal > 0 {
+		cmd.SysProcAttr.Pdeathsig = syscall.Signal(c.config.ParentDeathSignal)
+	}
+	return cmd, nil
+}
+
+func (c *linuxContainer) newInitProcess(p *Process, cmd *exec.Cmd, parentPipe, childPipe *os.File) (*initProcess, error) {
+	t := "_LIBCONTAINER_INITTYPE=standard"
+	cloneFlags := c.config.Namespaces.CloneFlags()
+	if cloneFlags&syscall.CLONE_NEWUSER != 0 {
+		if err := c.addUidGidMappings(cmd.SysProcAttr); err != nil {
+			// user mappings are not supported
+			return nil, err
+		}
+		// Default to root user when user namespaces are enabled.
+		if cmd.SysProcAttr.Credential == nil {
+			cmd.SysProcAttr.Credential = &syscall.Credential{}
+		}
+	}
+	cmd.Env = append(cmd.Env, t)
+	cmd.SysProcAttr.Cloneflags = cloneFlags
+	return &initProcess{
+		cmd:        cmd,
+		childPipe:  childPipe,
+		parentPipe: parentPipe,
+		manager:    c.cgroupManager,
+		config:     c.newInitConfig(p),
+	}, nil
+}
+
+func (c *linuxContainer) newSetnsProcess(p *Process, cmd *exec.Cmd, parentPipe, childPipe *os.File) *setnsProcess {
+	cmd.Env = append(cmd.Env,
+		fmt.Sprintf("_LIBCONTAINER_INITPID=%d", c.initProcess.pid()),
+		"_LIBCONTAINER_INITTYPE=setns",
+	)
+	if p.consolePath != "" {
+		cmd.Env = append(cmd.Env, "_LIBCONTAINER_CONSOLE_PATH="+p.consolePath)
+	}
+	// TODO: set on container for process management
+	return &setnsProcess{
+		cmd:         cmd,
+		cgroupPaths: c.cgroupManager.GetPaths(),
+		childPipe:   childPipe,
+		parentPipe:  parentPipe,
+		config:      c.newInitConfig(p),
+	}
+}
+
+func (c *linuxContainer) newInitConfig(process *Process) *initConfig {
+	return &initConfig{
+		Config:           c.config,
+		Args:             process.Args,
+		Env:              process.Env,
+		User:             process.User,
+		Cwd:              process.Cwd,
+		Console:          process.consolePath,
+		Capabilities:     process.Capabilities,
+		PassedFilesCount: len(process.ExtraFiles),
+	}
+}
+
+func newPipe() (parent *os.File, child *os.File, err error) {
+	fds, err := syscall.Socketpair(syscall.AF_LOCAL, syscall.SOCK_STREAM|syscall.SOCK_CLOEXEC, 0)
+	if err != nil {
+		return nil, nil, err
+	}
+	return os.NewFile(uintptr(fds[1]), "parent"), os.NewFile(uintptr(fds[0]), "child"), nil
+}
+
+func (c *linuxContainer) Destroy() error {
+	c.m.Lock()
+	defer c.m.Unlock()
+	status, err := c.currentStatus()
+	if err != nil {
+		return err
+	}
+	if status != Destroyed {
+		return newGenericError(fmt.Errorf("container is not destroyed"), ContainerNotStopped)
+	}
+	if !c.config.Namespaces.Contains(configs.NEWPID) {
+		if err := killCgroupProcesses(c.cgroupManager); err != nil {
+			logrus.Warn(err)
+		}
+	}
+	err = c.cgroupManager.Destroy()
+	if rerr := os.RemoveAll(c.root); err == nil {
+		err = rerr
+	}
+	c.initProcess = nil
+	return err
+}
+
+func (c *linuxContainer) Pause() error {
+	c.m.Lock()
+	defer c.m.Unlock()
+	return c.cgroupManager.Freeze(configs.Frozen)
+}
+
+func (c *linuxContainer) Resume() error {
+	c.m.Lock()
+	defer c.m.Unlock()
+	return c.cgroupManager.Freeze(configs.Thawed)
+}
+
+func (c *linuxContainer) NotifyOOM() (<-chan struct{}, error) {
+	return notifyOnOOM(c.cgroupManager.GetPaths())
+}
+
+// XXX debug support, remove when debugging done.
+func addArgsFromEnv(evar string, args *[]string) {
+	if e := os.Getenv(evar); e != "" {
+		for _, f := range strings.Fields(e) {
+			*args = append(*args, f)
+		}
+	}
+	fmt.Printf(">>> criu %v\n", *args)
+}
+
+func (c *linuxContainer) checkCriuVersion() error {
+	var x, y, z int
+
+	out, err := exec.Command(c.criuPath, "-V").Output()
+	if err != nil {
+		return err
+	}
+
+	n, err := fmt.Sscanf(string(out), "Version: %d.%d.%d\n", &x, &y, &z) // 1.5.2
+	if err != nil {
+		n, err = fmt.Sscanf(string(out), "Version: %d.%d\n", &x, &y) // 1.6
+	}
+	if n < 2 || err != nil {
+		return fmt.Errorf("Unable to parse the CRIU version: %s %d %s", out, n, err)
+	}
+
+	if x*10000+y*100+z < 10502 {
+		return fmt.Errorf("CRIU version must be 1.5.2 or higher")
+	}
+
+	return nil
+}
+
+const descriptors_filename = "descriptors.json"
+
+func (c *linuxContainer) Checkpoint(criuOpts *CriuOpts) error {
+	c.m.Lock()
+	defer c.m.Unlock()
+
+	if err := c.checkCriuVersion(); err != nil {
+		return err
+	}
+
+	if criuOpts.ImagesDirectory == "" {
+		criuOpts.ImagesDirectory = filepath.Join(c.root, "criu.image")
+	}
+
+	// Since a container can be C/R'ed multiple times,
+	// the checkpoint directory may already exist.
+	if err := os.Mkdir(criuOpts.ImagesDirectory, 0755); err != nil && !os.IsExist(err) {
+		return err
+	}
+
+	if criuOpts.WorkDirectory == "" {
+		criuOpts.WorkDirectory = filepath.Join(c.root, "criu.work")
+	}
+
+	if err := os.Mkdir(criuOpts.WorkDirectory, 0755); err != nil && !os.IsExist(err) {
+		return err
+	}
+
+	workDir, err := os.Open(criuOpts.WorkDirectory)
+	if err != nil {
+		return err
+	}
+	defer workDir.Close()
+
+	imageDir, err := os.Open(criuOpts.ImagesDirectory)
+	if err != nil {
+		return err
+	}
+	defer imageDir.Close()
+
+	rpcOpts := criurpc.CriuOpts{
+		ImagesDirFd:    proto.Int32(int32(imageDir.Fd())),
+		WorkDirFd:      proto.Int32(int32(workDir.Fd())),
+		LogLevel:       proto.Int32(4),
+		LogFile:        proto.String("dump.log"),
+		Root:           proto.String(c.config.Rootfs),
+		ManageCgroups:  proto.Bool(true),
+		NotifyScripts:  proto.Bool(true),
+		Pid:            proto.Int32(int32(c.initProcess.pid())),
+		ShellJob:       proto.Bool(criuOpts.ShellJob),
+		LeaveRunning:   proto.Bool(criuOpts.LeaveRunning),
+		TcpEstablished: proto.Bool(criuOpts.TcpEstablished),
+		ExtUnixSk:      proto.Bool(criuOpts.ExternalUnixConnections),
+	}
+
+	// append optional criu opts, e.g., page-server and port
+	if criuOpts.PageServer.Address != "" && criuOpts.PageServer.Port != 0 {
+		rpcOpts.Ps = &criurpc.CriuPageServerInfo{
+			Address: proto.String(criuOpts.PageServer.Address),
+			Port:    proto.Int32(criuOpts.PageServer.Port),
+		}
+	}
+
+	t := criurpc.CriuReqType_DUMP
+	req := criurpc.CriuReq{
+		Type: &t,
+		Opts: &rpcOpts,
+	}
+
+	for _, m := range c.config.Mounts {
+		if m.Device == "bind" {
+			mountDest := m.Destination
+			if strings.HasPrefix(mountDest, c.config.Rootfs) {
+				mountDest = mountDest[len(c.config.Rootfs):]
+			}
+
+			extMnt := new(criurpc.ExtMountMap)
+			extMnt.Key = proto.String(mountDest)
+			extMnt.Val = proto.String(mountDest)
+			req.Opts.ExtMnt = append(req.Opts.ExtMnt, extMnt)
+		}
+	}
+
+	// Write the FD info to a file in the image directory
+
+	fdsJSON, err := json.Marshal(c.initProcess.externalDescriptors())
+	if err != nil {
+		return err
+	}
+
+	err = ioutil.WriteFile(filepath.Join(criuOpts.ImagesDirectory, descriptors_filename), fdsJSON, 0655)
+	if err != nil {
+		return err
+	}
+
+	err = c.criuSwrk(nil, &req, criuOpts)
+	if err != nil {
+		return err
+	}
+	return nil
+}
+
+func (c *linuxContainer) Restore(process *Process, criuOpts *CriuOpts) error {
+	c.m.Lock()
+	defer c.m.Unlock()
+
+	if err := c.checkCriuVersion(); err != nil {
+		return err
+	}
+
+	if criuOpts.WorkDirectory == "" {
+		criuOpts.WorkDirectory = filepath.Join(c.root, "criu.work")
+	}
+	// Since a container can be C/R'ed multiple times,
+	// the work directory may already exist.
+	if err := os.Mkdir(criuOpts.WorkDirectory, 0655); err != nil && !os.IsExist(err) {
+		return err
+	}
+
+	workDir, err := os.Open(criuOpts.WorkDirectory)
+	if err != nil {
+		return err
+	}
+	defer workDir.Close()
+
+	if criuOpts.ImagesDirectory == "" {
+		criuOpts.ImagesDirectory = filepath.Join(c.root, "criu.image")
+	}
+	imageDir, err := os.Open(criuOpts.ImagesDirectory)
+	if err != nil {
+		return err
+	}
+	defer imageDir.Close()
+
+	// CRIU has a few requirements for a root directory:
+	// * it must be a mount point
+	// * its parent must not be overmounted
+	// c.config.Rootfs is bind-mounted to a temporary directory
+	// to satisfy these requirements.
+	root := filepath.Join(c.root, "criu-root")
+	if err := os.Mkdir(root, 0755); err != nil {
+		return err
+	}
+	defer os.Remove(root)
+
+	root, err = filepath.EvalSymlinks(root)
+	if err != nil {
+		return err
+	}
+
+	err = syscall.Mount(c.config.Rootfs, root, "", syscall.MS_BIND|syscall.MS_REC, "")
+	if err != nil {
+		return err
+	}
+	defer syscall.Unmount(root, syscall.MNT_DETACH)
+
+	t := criurpc.CriuReqType_RESTORE
+	req := criurpc.CriuReq{
+		Type: &t,
+		Opts: &criurpc.CriuOpts{
+			ImagesDirFd:    proto.Int32(int32(imageDir.Fd())),
+			WorkDirFd:      proto.Int32(int32(workDir.Fd())),
+			EvasiveDevices: proto.Bool(true),
+			LogLevel:       proto.Int32(4),
+			LogFile:        proto.String("restore.log"),
+			RstSibling:     proto.Bool(true),
+			Root:           proto.String(root),
+			ManageCgroups:  proto.Bool(true),
+			NotifyScripts:  proto.Bool(true),
+			ShellJob:       proto.Bool(criuOpts.ShellJob),
+			ExtUnixSk:      proto.Bool(criuOpts.ExternalUnixConnections),
+			TcpEstablished: proto.Bool(criuOpts.TcpEstablished),
+		},
+	}
+	for _, m := range c.config.Mounts {
+		if m.Device == "bind" {
+			mountDest := m.Destination
+			if strings.HasPrefix(mountDest, c.config.Rootfs) {
+				mountDest = mountDest[len(c.config.Rootfs):]
+			}
+
+			extMnt := new(criurpc.ExtMountMap)
+			extMnt.Key = proto.String(mountDest)
+			extMnt.Val = proto.String(m.Source)
+			req.Opts.ExtMnt = append(req.Opts.ExtMnt, extMnt)
+		}
+	}
+	for _, iface := range c.config.Networks {
+		switch iface.Type {
+		case "veth":
+			veth := new(criurpc.CriuVethPair)
+			veth.IfOut = proto.String(iface.HostInterfaceName)
+			veth.IfIn = proto.String(iface.Name)
+			req.Opts.Veths = append(req.Opts.Veths, veth)
+			break
+		case "loopback":
+			break
+		}
+	}
+
+	var (
+		fds    []string
+		fdJSON []byte
+	)
+
+	if fdJSON, err = ioutil.ReadFile(filepath.Join(criuOpts.ImagesDirectory, descriptors_filename)); err != nil {
+		return err
+	}
+
+	if err = json.Unmarshal(fdJSON, &fds); err != nil {
+		return err
+	}
+
+	for i := range fds {
+		if s := fds[i]; strings.Contains(s, "pipe:") {
+			inheritFd := new(criurpc.InheritFd)
+			inheritFd.Key = proto.String(s)
+			inheritFd.Fd = proto.Int32(int32(i))
+			req.Opts.InheritFd = append(req.Opts.InheritFd, inheritFd)
+		}
+	}
+
+	err = c.criuSwrk(process, &req, criuOpts)
+	if err != nil {
+		return err
+	}
+	return nil
+}
+
+func (c *linuxContainer) criuSwrk(process *Process, req *criurpc.CriuReq, opts *CriuOpts) error {
+	fds, err := syscall.Socketpair(syscall.AF_LOCAL, syscall.SOCK_SEQPACKET|syscall.SOCK_CLOEXEC, 0)
+	if err != nil {
+		return err
+	}
+
+	criuClient := os.NewFile(uintptr(fds[0]), "criu-transport-client")
+	criuServer := os.NewFile(uintptr(fds[1]), "criu-transport-server")
+	defer criuClient.Close()
+	defer criuServer.Close()
+
+	args := []string{"swrk", "3"}
+	cmd := exec.Command(c.criuPath, args...)
+	if process != nil {
+		cmd.Stdin = process.Stdin
+		cmd.Stdout = process.Stdout
+		cmd.Stderr = process.Stderr
+	}
+	cmd.ExtraFiles = append(cmd.ExtraFiles, criuServer)
+
+	if err := cmd.Start(); err != nil {
+		return err
+	}
+	criuServer.Close()
+
+	defer func() {
+		criuClient.Close()
+		_, err := cmd.Process.Wait()
+		if err != nil {
+			return
+		}
+	}()
+
+	var extFds []string
+	if process != nil {
+		extFds, err = getPipeFds(cmd.Process.Pid)
+		if err != nil {
+			return err
+		}
+	}
+
+	data, err := proto.Marshal(req)
+	if err != nil {
+		return err
+	}
+	_, err = criuClient.Write(data)
+	if err != nil {
+		return err
+	}
+
+	buf := make([]byte, 10*4096)
+	for true {
+		n, err := criuClient.Read(buf)
+		if err != nil {
+			return err
+		}
+		if n == 0 {
+			return fmt.Errorf("unexpected EOF")
+		}
+		if n == len(buf) {
+			return fmt.Errorf("buffer is too small")
+		}
+
+		resp := new(criurpc.CriuResp)
+		err = proto.Unmarshal(buf[:n], resp)
+		if err != nil {
+			return err
+		}
+		if !resp.GetSuccess() {
+			return fmt.Errorf("criu failed: type %s errno %d", req.GetType().String(), resp.GetCrErrno())
+		}
+
+		t := resp.GetType()
+		switch {
+		case t == criurpc.CriuReqType_NOTIFY:
+			if err := c.criuNotifications(resp, process, opts, extFds); err != nil {
+				return err
+			}
+			t = criurpc.CriuReqType_NOTIFY
+			req = &criurpc.CriuReq{
+				Type:          &t,
+				NotifySuccess: proto.Bool(true),
+			}
+			data, err = proto.Marshal(req)
+			if err != nil {
+				return err
+			}
+			n, err = criuClient.Write(data)
+			if err != nil {
+				return err
+			}
+			continue
+		case t == criurpc.CriuReqType_RESTORE:
+		case t == criurpc.CriuReqType_DUMP:
+			break
+		default:
+			return fmt.Errorf("unable to parse the response %s", resp.String())
+		}
+
+		break
+	}
+
+	// cmd.Wait() waits cmd.goroutines which are used for proxying file descriptors.
+	// Here we want to wait only the CRIU process.
+	st, err := cmd.Process.Wait()
+	if err != nil {
+		return err
+	}
+	if !st.Success() {
+		return fmt.Errorf("criu failed: %s", st.String())
+	}
+	return nil
+}
+
+// block any external network activity
+func lockNetwork(config *configs.Config) error {
+	for _, config := range config.Networks {
+		strategy, err := getStrategy(config.Type)
+		if err != nil {
+			return err
+		}
+
+		if err := strategy.detach(config); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func unlockNetwork(config *configs.Config) error {
+	for _, config := range config.Networks {
+		strategy, err := getStrategy(config.Type)
+		if err != nil {
+			return err
+		}
+		if err = strategy.attach(config); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func (c *linuxContainer) criuNotifications(resp *criurpc.CriuResp, process *Process, opts *CriuOpts, fds []string) error {
+	notify := resp.GetNotify()
+	if notify == nil {
+		return fmt.Errorf("invalid response: %s", resp.String())
+	}
+
+	switch {
+	case notify.GetScript() == "post-dump":
+		if !opts.LeaveRunning {
+			f, err := os.Create(filepath.Join(c.root, "checkpoint"))
+			if err != nil {
+				return err
+			}
+			f.Close()
+		}
+		break
+
+	case notify.GetScript() == "network-unlock":
+		if err := unlockNetwork(c.config); err != nil {
+			return err
+		}
+		break
+
+	case notify.GetScript() == "network-lock":
+		if err := lockNetwork(c.config); err != nil {
+			return err
+		}
+		break
+
+	case notify.GetScript() == "post-restore":
+		pid := notify.GetPid()
+		r, err := newRestoredProcess(int(pid), fds)
+		if err != nil {
+			return err
+		}
+
+		// TODO: crosbymichael restore previous process information by saving the init process information in
+		// the container's state file or separate process state files.
+		if err := c.updateState(r); err != nil {
+			return err
+		}
+		process.ops = r
+		break
+	}
+
+	return nil
+}
+
+func (c *linuxContainer) updateState(process parentProcess) error {
+	c.initProcess = process
+	state, err := c.currentState()
+	if err != nil {
+		return err
+	}
+	f, err := os.Create(filepath.Join(c.root, stateFilename))
+	if err != nil {
+		return err
+	}
+	defer f.Close()
+	os.Remove(filepath.Join(c.root, "checkpoint"))
+	return json.NewEncoder(f).Encode(state)
+}
+
+func (c *linuxContainer) currentStatus() (Status, error) {
+	if _, err := os.Stat(filepath.Join(c.root, "checkpoint")); err == nil {
+		return Checkpointed, nil
+	}
+	if c.initProcess == nil {
+		return Destroyed, nil
+	}
+	// return Running if the init process is alive
+	if err := syscall.Kill(c.initProcess.pid(), 0); err != nil {
+		if err == syscall.ESRCH {
+			return Destroyed, nil
+		}
+		return 0, newSystemError(err)
+	}
+	if c.config.Cgroups != nil && c.config.Cgroups.Freezer == configs.Frozen {
+		return Paused, nil
+	}
+	return Running, nil
+}
+
+func (c *linuxContainer) currentState() (*State, error) {
+	status, err := c.currentStatus()
+	if err != nil {
+		return nil, err
+	}
+	if status == Destroyed {
+		return nil, newGenericError(fmt.Errorf("container destroyed"), ContainerNotExists)
+	}
+	startTime, err := c.initProcess.startTime()
+	if err != nil {
+		return nil, newSystemError(err)
+	}
+	state := &State{
+		ID:                   c.ID(),
+		Config:               *c.config,
+		InitProcessPid:       c.initProcess.pid(),
+		InitProcessStartTime: startTime,
+		CgroupPaths:          c.cgroupManager.GetPaths(),
+		NamespacePaths:       make(map[configs.NamespaceType]string),
+		ExternalDescriptors:  c.initProcess.externalDescriptors(),
+	}
+	for _, ns := range c.config.Namespaces {
+		state.NamespacePaths[ns.Type] = ns.GetPath(c.initProcess.pid())
+	}
+	for _, nsType := range configs.NamespaceTypes() {
+		if _, ok := state.NamespacePaths[nsType]; !ok {
+			ns := configs.Namespace{Type: nsType}
+			state.NamespacePaths[ns.Type] = ns.GetPath(c.initProcess.pid())
+		}
+	}
+	return state, nil
+}

libcontainer/container_linux_test.go

@@ -0,0 +1,212 @@
+// +build linux
+
+package libcontainer
+
+import (
+	"fmt"
+	"os"
+	"testing"
+
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+)
+
+type mockCgroupManager struct {
+	pids  []int
+	stats *cgroups.Stats
+	paths map[string]string
+}
+
+func (m *mockCgroupManager) GetPids() ([]int, error) {
+	return m.pids, nil
+}
+
+func (m *mockCgroupManager) GetStats() (*cgroups.Stats, error) {
+	return m.stats, nil
+}
+
+func (m *mockCgroupManager) Apply(pid int) error {
+	return nil
+}
+
+func (m *mockCgroupManager) Set(container *configs.Config) error {
+	return nil
+}
+
+func (m *mockCgroupManager) Destroy() error {
+	return nil
+}
+
+func (m *mockCgroupManager) GetPaths() map[string]string {
+	return m.paths
+}
+
+func (m *mockCgroupManager) Freeze(state configs.FreezerState) error {
+	return nil
+}
+
+type mockProcess struct {
+	_pid    int
+	started string
+}
+
+func (m *mockProcess) terminate() error {
+	return nil
+}
+
+func (m *mockProcess) pid() int {
+	return m._pid
+}
+
+func (m *mockProcess) startTime() (string, error) {
+	return m.started, nil
+}
+
+func (m *mockProcess) start() error {
+	return nil
+}
+
+func (m *mockProcess) wait() (*os.ProcessState, error) {
+	return nil, nil
+}
+
+func (m *mockProcess) signal(_ os.Signal) error {
+	return nil
+}
+
+func (p *mockProcess) externalDescriptors() []string {
+	return []string{}
+}
+
+func (p *mockProcess) setExternalDescriptors(newFds []string) {
+}
+
+func TestGetContainerPids(t *testing.T) {
+	container := &linuxContainer{
+		id:            "myid",
+		config:        &configs.Config{},
+		cgroupManager: &mockCgroupManager{pids: []int{1, 2, 3}},
+	}
+	pids, err := container.Processes()
+	if err != nil {
+		t.Fatal(err)
+	}
+	for i, expected := range []int{1, 2, 3} {
+		if pids[i] != expected {
+			t.Fatalf("expected pid %d but received %d", expected, pids[i])
+		}
+	}
+}
+
+func TestGetContainerStats(t *testing.T) {
+	container := &linuxContainer{
+		id:     "myid",
+		config: &configs.Config{},
+		cgroupManager: &mockCgroupManager{
+			pids: []int{1, 2, 3},
+			stats: &cgroups.Stats{
+				MemoryStats: cgroups.MemoryStats{
+					Usage: cgroups.MemoryData{
+						Usage: 1024,
+					},
+				},
+			},
+		},
+	}
+	stats, err := container.Stats()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if stats.CgroupStats == nil {
+		t.Fatal("cgroup stats are nil")
+	}
+	if stats.CgroupStats.MemoryStats.Usage.Usage != 1024 {
+		t.Fatalf("expected memory usage 1024 but recevied %d", stats.CgroupStats.MemoryStats.Usage.Usage)
+	}
+}
+
+func TestGetContainerState(t *testing.T) {
+	var (
+		pid                 = os.Getpid()
+		expectedMemoryPath  = "/sys/fs/cgroup/memory/myid"
+		expectedNetworkPath = "/networks/fd"
+	)
+	container := &linuxContainer{
+		id: "myid",
+		config: &configs.Config{
+			Namespaces: []configs.Namespace{
+				{Type: configs.NEWPID},
+				{Type: configs.NEWNS},
+				{Type: configs.NEWNET, Path: expectedNetworkPath},
+				{Type: configs.NEWUTS},
+				// emulate host for IPC
+				//{Type: configs.NEWIPC},
+			},
+		},
+		initProcess: &mockProcess{
+			_pid:    pid,
+			started: "010",
+		},
+		cgroupManager: &mockCgroupManager{
+			pids: []int{1, 2, 3},
+			stats: &cgroups.Stats{
+				MemoryStats: cgroups.MemoryStats{
+					Usage: cgroups.MemoryData{
+						Usage: 1024,
+					},
+				},
+			},
+			paths: map[string]string{
+				"memory": expectedMemoryPath,
+			},
+		},
+	}
+	state, err := container.State()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if state.InitProcessPid != pid {
+		t.Fatalf("expected pid %d but received %d", pid, state.InitProcessPid)
+	}
+	if state.InitProcessStartTime != "010" {
+		t.Fatalf("expected process start time 010 but received %s", state.InitProcessStartTime)
+	}
+	paths := state.CgroupPaths
+	if paths == nil {
+		t.Fatal("cgroup paths should not be nil")
+	}
+	if memPath := paths["memory"]; memPath != expectedMemoryPath {
+		t.Fatalf("expected memory path %q but received %q", expectedMemoryPath, memPath)
+	}
+	for _, ns := range container.config.Namespaces {
+		path := state.NamespacePaths[ns.Type]
+		if path == "" {
+			t.Fatalf("expected non nil namespace path for %s", ns.Type)
+		}
+		if ns.Type == configs.NEWNET {
+			if path != expectedNetworkPath {
+				t.Fatalf("expected path %q but received %q", expectedNetworkPath, path)
+			}
+		} else {
+			file := ""
+			switch ns.Type {
+			case configs.NEWNET:
+				file = "net"
+			case configs.NEWNS:
+				file = "mnt"
+			case configs.NEWPID:
+				file = "pid"
+			case configs.NEWIPC:
+				file = "ipc"
+			case configs.NEWUSER:
+				file = "user"
+			case configs.NEWUTS:
+				file = "uts"
+			}
+			expected := fmt.Sprintf("/proc/%d/ns/%s", pid, file)
+			if expected != path {
+				t.Fatalf("expected path %q but received %q", expected, path)
+			}
+		}
+	}
+}

libcontainer/container_nouserns_linux.go

@@ -0,0 +1,13 @@
+// +build !go1.4
+
+package libcontainer
+
+import (
+	"fmt"
+	"syscall"
+)
+
+// not available before go 1.4
+func (c *linuxContainer) addUidGidMappings(sys *syscall.SysProcAttr) error {
+	return fmt.Errorf("User namespace is not supported in golang < 1.4")
+}

libcontainer/container_userns_linux.go

@@ -0,0 +1,26 @@
+// +build go1.4
+
+package libcontainer
+
+import "syscall"
+
+// Converts IDMap to SysProcIDMap array and adds it to SysProcAttr.
+func (c *linuxContainer) addUidGidMappings(sys *syscall.SysProcAttr) error {
+	if c.config.UidMappings != nil {
+		sys.UidMappings = make([]syscall.SysProcIDMap, len(c.config.UidMappings))
+		for i, um := range c.config.UidMappings {
+			sys.UidMappings[i].ContainerID = um.ContainerID
+			sys.UidMappings[i].HostID = um.HostID
+			sys.UidMappings[i].Size = um.Size
+		}
+	}
+	if c.config.GidMappings != nil {
+		sys.GidMappings = make([]syscall.SysProcIDMap, len(c.config.GidMappings))
+		for i, gm := range c.config.GidMappings {
+			sys.GidMappings[i].ContainerID = gm.ContainerID
+			sys.GidMappings[i].HostID = gm.HostID
+			sys.GidMappings[i].Size = gm.Size
+		}
+	}
+	return nil
+}

libcontainer/criu_opts.go

@@ -0,0 +1,16 @@
+package libcontainer
+
+type CriuPageServerInfo struct {
+	Address string // IP address of CRIU page server
+	Port    int32  // port number of CRIU page server
+}
+
+type CriuOpts struct {
+	ImagesDirectory         string             // directory for storing image files
+	WorkDirectory           string             // directory to cd and write logs/pidfiles/stats to
+	LeaveRunning            bool               // leave container in running state after checkpoint
+	TcpEstablished          bool               // checkpoint/restore established TCP connections
+	ExternalUnixConnections bool               // allow external unix connections
+	ShellJob                bool               // allow to dump and restore shell jobs
+	PageServer              CriuPageServerInfo // allow to dump to criu page server
+}

libcontainer/criurpc/Makefile

@@ -0,0 +1,2 @@
+gen: criurpc.proto
+	protoc --go_out=. criurpc.proto

libcontainer/criurpc/criurpc.pb.go

@@ -0,0 +1,602 @@
+// Code generated by protoc-gen-go.
+// source: criurpc.proto
+// DO NOT EDIT!
+
+package criurpc
+
+import proto "github.com/golang/protobuf/proto"
+import json "encoding/json"
+import math "math"
+
+// Reference proto, json, and math imports to suppress error if they are not otherwise used.
+var _ = proto.Marshal
+var _ = &json.SyntaxError{}
+var _ = math.Inf
+
+type CriuReqType int32
+
+const (
+	CriuReqType_EMPTY         CriuReqType = 0
+	CriuReqType_DUMP          CriuReqType = 1
+	CriuReqType_RESTORE       CriuReqType = 2
+	CriuReqType_CHECK         CriuReqType = 3
+	CriuReqType_PRE_DUMP      CriuReqType = 4
+	CriuReqType_PAGE_SERVER   CriuReqType = 5
+	CriuReqType_NOTIFY        CriuReqType = 6
+	CriuReqType_CPUINFO_DUMP  CriuReqType = 7
+	CriuReqType_CPUINFO_CHECK CriuReqType = 8
+)
+
+var CriuReqType_name = map[int32]string{
+	0: "EMPTY",
+	1: "DUMP",
+	2: "RESTORE",
+	3: "CHECK",
+	4: "PRE_DUMP",
+	5: "PAGE_SERVER",
+	6: "NOTIFY",
+	7: "CPUINFO_DUMP",
+	8: "CPUINFO_CHECK",
+}
+var CriuReqType_value = map[string]int32{
+	"EMPTY":         0,
+	"DUMP":          1,
+	"RESTORE":       2,
+	"CHECK":         3,
+	"PRE_DUMP":      4,
+	"PAGE_SERVER":   5,
+	"NOTIFY":        6,
+	"CPUINFO_DUMP":  7,
+	"CPUINFO_CHECK": 8,
+}
+
+func (x CriuReqType) Enum() *CriuReqType {
+	p := new(CriuReqType)
+	*p = x
+	return p
+}
+func (x CriuReqType) String() string {
+	return proto.EnumName(CriuReqType_name, int32(x))
+}
+func (x CriuReqType) MarshalJSON() ([]byte, error) {
+	return json.Marshal(x.String())
+}
+func (x *CriuReqType) UnmarshalJSON(data []byte) error {
+	value, err := proto.UnmarshalJSONEnum(CriuReqType_value, data, "CriuReqType")
+	if err != nil {
+		return err
+	}
+	*x = CriuReqType(value)
+	return nil
+}
+
+type CriuPageServerInfo struct {
+	Address          *string `protobuf:"bytes,1,opt,name=address" json:"address,omitempty"`
+	Port             *int32  `protobuf:"varint,2,opt,name=port" json:"port,omitempty"`
+	Pid              *int32  `protobuf:"varint,3,opt,name=pid" json:"pid,omitempty"`
+	Fd               *int32  `protobuf:"varint,4,opt,name=fd" json:"fd,omitempty"`
+	XXX_unrecognized []byte  `json:"-"`
+}
+
+func (m *CriuPageServerInfo) Reset()         { *m = CriuPageServerInfo{} }
+func (m *CriuPageServerInfo) String() string { return proto.CompactTextString(m) }
+func (*CriuPageServerInfo) ProtoMessage()    {}
+
+func (m *CriuPageServerInfo) GetAddress() string {
+	if m != nil && m.Address != nil {
+		return *m.Address
+	}
+	return ""
+}
+
+func (m *CriuPageServerInfo) GetPort() int32 {
+	if m != nil && m.Port != nil {
+		return *m.Port
+	}
+	return 0
+}
+
+func (m *CriuPageServerInfo) GetPid() int32 {
+	if m != nil && m.Pid != nil {
+		return *m.Pid
+	}
+	return 0
+}
+
+func (m *CriuPageServerInfo) GetFd() int32 {
+	if m != nil && m.Fd != nil {
+		return *m.Fd
+	}
+	return 0
+}
+
+type CriuVethPair struct {
+	IfIn             *string `protobuf:"bytes,1,req,name=if_in" json:"if_in,omitempty"`
+	IfOut            *string `protobuf:"bytes,2,req,name=if_out" json:"if_out,omitempty"`
+	XXX_unrecognized []byte  `json:"-"`
+}
+
+func (m *CriuVethPair) Reset()         { *m = CriuVethPair{} }
+func (m *CriuVethPair) String() string { return proto.CompactTextString(m) }
+func (*CriuVethPair) ProtoMessage()    {}
+
+func (m *CriuVethPair) GetIfIn() string {
+	if m != nil && m.IfIn != nil {
+		return *m.IfIn
+	}
+	return ""
+}
+
+func (m *CriuVethPair) GetIfOut() string {
+	if m != nil && m.IfOut != nil {
+		return *m.IfOut
+	}
+	return ""
+}
+
+type ExtMountMap struct {
+	Key              *string `protobuf:"bytes,1,req,name=key" json:"key,omitempty"`
+	Val              *string `protobuf:"bytes,2,req,name=val" json:"val,omitempty"`
+	XXX_unrecognized []byte  `json:"-"`
+}
+
+func (m *ExtMountMap) Reset()         { *m = ExtMountMap{} }
+func (m *ExtMountMap) String() string { return proto.CompactTextString(m) }
+func (*ExtMountMap) ProtoMessage()    {}
+
+func (m *ExtMountMap) GetKey() string {
+	if m != nil && m.Key != nil {
+		return *m.Key
+	}
+	return ""
+}
+
+func (m *ExtMountMap) GetVal() string {
+	if m != nil && m.Val != nil {
+		return *m.Val
+	}
+	return ""
+}
+
+type InheritFd struct {
+	Key              *string `protobuf:"bytes,1,req,name=key" json:"key,omitempty"`
+	Fd               *int32  `protobuf:"varint,2,req,name=fd" json:"fd,omitempty"`
+	XXX_unrecognized []byte  `json:"-"`
+}
+
+func (m *InheritFd) Reset()         { *m = InheritFd{} }
+func (m *InheritFd) String() string { return proto.CompactTextString(m) }
+func (*InheritFd) ProtoMessage()    {}
+
+func (m *InheritFd) GetKey() string {
+	if m != nil && m.Key != nil {
+		return *m.Key
+	}
+	return ""
+}
+
+func (m *InheritFd) GetFd() int32 {
+	if m != nil && m.Fd != nil {
+		return *m.Fd
+	}
+	return 0
+}
+
+type CgroupRoot struct {
+	Ctrl             *string `protobuf:"bytes,1,opt,name=ctrl" json:"ctrl,omitempty"`
+	Path             *string `protobuf:"bytes,2,req,name=path" json:"path,omitempty"`
+	XXX_unrecognized []byte  `json:"-"`
+}
+
+func (m *CgroupRoot) Reset()         { *m = CgroupRoot{} }
+func (m *CgroupRoot) String() string { return proto.CompactTextString(m) }
+func (*CgroupRoot) ProtoMessage()    {}
+
+func (m *CgroupRoot) GetCtrl() string {
+	if m != nil && m.Ctrl != nil {
+		return *m.Ctrl
+	}
+	return ""
+}
+
+func (m *CgroupRoot) GetPath() string {
+	if m != nil && m.Path != nil {
+		return *m.Path
+	}
+	return ""
+}
+
+type CriuOpts struct {
+	ImagesDirFd      *int32              `protobuf:"varint,1,req,name=images_dir_fd" json:"images_dir_fd,omitempty"`
+	Pid              *int32              `protobuf:"varint,2,opt,name=pid" json:"pid,omitempty"`
+	LeaveRunning     *bool               `protobuf:"varint,3,opt,name=leave_running" json:"leave_running,omitempty"`
+	ExtUnixSk        *bool               `protobuf:"varint,4,opt,name=ext_unix_sk" json:"ext_unix_sk,omitempty"`
+	TcpEstablished   *bool               `protobuf:"varint,5,opt,name=tcp_established" json:"tcp_established,omitempty"`
+	EvasiveDevices   *bool               `protobuf:"varint,6,opt,name=evasive_devices" json:"evasive_devices,omitempty"`
+	ShellJob         *bool               `protobuf:"varint,7,opt,name=shell_job" json:"shell_job,omitempty"`
+	FileLocks        *bool               `protobuf:"varint,8,opt,name=file_locks" json:"file_locks,omitempty"`
+	LogLevel         *int32              `protobuf:"varint,9,opt,name=log_level,def=2" json:"log_level,omitempty"`
+	LogFile          *string             `protobuf:"bytes,10,opt,name=log_file" json:"log_file,omitempty"`
+	Ps               *CriuPageServerInfo `protobuf:"bytes,11,opt,name=ps" json:"ps,omitempty"`
+	NotifyScripts    *bool               `protobuf:"varint,12,opt,name=notify_scripts" json:"notify_scripts,omitempty"`
+	Root             *string             `protobuf:"bytes,13,opt,name=root" json:"root,omitempty"`
+	ParentImg        *string             `protobuf:"bytes,14,opt,name=parent_img" json:"parent_img,omitempty"`
+	TrackMem         *bool               `protobuf:"varint,15,opt,name=track_mem" json:"track_mem,omitempty"`
+	AutoDedup        *bool               `protobuf:"varint,16,opt,name=auto_dedup" json:"auto_dedup,omitempty"`
+	WorkDirFd        *int32              `protobuf:"varint,17,opt,name=work_dir_fd" json:"work_dir_fd,omitempty"`
+	LinkRemap        *bool               `protobuf:"varint,18,opt,name=link_remap" json:"link_remap,omitempty"`
+	Veths            []*CriuVethPair     `protobuf:"bytes,19,rep,name=veths" json:"veths,omitempty"`
+	CpuCap           *uint32             `protobuf:"varint,20,opt,name=cpu_cap,def=4294967295" json:"cpu_cap,omitempty"`
+	ForceIrmap       *bool               `protobuf:"varint,21,opt,name=force_irmap" json:"force_irmap,omitempty"`
+	ExecCmd          []string            `protobuf:"bytes,22,rep,name=exec_cmd" json:"exec_cmd,omitempty"`
+	ExtMnt           []*ExtMountMap      `protobuf:"bytes,23,rep,name=ext_mnt" json:"ext_mnt,omitempty"`
+	ManageCgroups    *bool               `protobuf:"varint,24,opt,name=manage_cgroups" json:"manage_cgroups,omitempty"`
+	CgRoot           []*CgroupRoot       `protobuf:"bytes,25,rep,name=cg_root" json:"cg_root,omitempty"`
+	RstSibling       *bool               `protobuf:"varint,26,opt,name=rst_sibling" json:"rst_sibling,omitempty"`
+	InheritFd        []*InheritFd        `protobuf:"bytes,27,rep,name=inherit_fd" json:"inherit_fd,omitempty"`
+	XXX_unrecognized []byte              `json:"-"`
+}
+
+func (m *CriuOpts) Reset()         { *m = CriuOpts{} }
+func (m *CriuOpts) String() string { return proto.CompactTextString(m) }
+func (*CriuOpts) ProtoMessage()    {}
+
+const Default_CriuOpts_LogLevel int32 = 2
+const Default_CriuOpts_CpuCap uint32 = 4294967295
+
+func (m *CriuOpts) GetImagesDirFd() int32 {
+	if m != nil && m.ImagesDirFd != nil {
+		return *m.ImagesDirFd
+	}
+	return 0
+}
+
+func (m *CriuOpts) GetPid() int32 {
+	if m != nil && m.Pid != nil {
+		return *m.Pid
+	}
+	return 0
+}
+
+func (m *CriuOpts) GetLeaveRunning() bool {
+	if m != nil && m.LeaveRunning != nil {
+		return *m.LeaveRunning
+	}
+	return false
+}
+
+func (m *CriuOpts) GetExtUnixSk() bool {
+	if m != nil && m.ExtUnixSk != nil {
+		return *m.ExtUnixSk
+	}
+	return false
+}
+
+func (m *CriuOpts) GetTcpEstablished() bool {
+	if m != nil && m.TcpEstablished != nil {
+		return *m.TcpEstablished
+	}
+	return false
+}
+
+func (m *CriuOpts) GetEvasiveDevices() bool {
+	if m != nil && m.EvasiveDevices != nil {
+		return *m.EvasiveDevices
+	}
+	return false
+}
+
+func (m *CriuOpts) GetShellJob() bool {
+	if m != nil && m.ShellJob != nil {
+		return *m.ShellJob
+	}
+	return false
+}
+
+func (m *CriuOpts) GetFileLocks() bool {
+	if m != nil && m.FileLocks != nil {
+		return *m.FileLocks
+	}
+	return false
+}
+
+func (m *CriuOpts) GetLogLevel() int32 {
+	if m != nil && m.LogLevel != nil {
+		return *m.LogLevel
+	}
+	return Default_CriuOpts_LogLevel
+}
+
+func (m *CriuOpts) GetLogFile() string {
+	if m != nil && m.LogFile != nil {
+		return *m.LogFile
+	}
+	return ""
+}
+
+func (m *CriuOpts) GetPs() *CriuPageServerInfo {
+	if m != nil {
+		return m.Ps
+	}
+	return nil
+}
+
+func (m *CriuOpts) GetNotifyScripts() bool {
+	if m != nil && m.NotifyScripts != nil {
+		return *m.NotifyScripts
+	}
+	return false
+}
+
+func (m *CriuOpts) GetRoot() string {
+	if m != nil && m.Root != nil {
+		return *m.Root
+	}
+	return ""
+}
+
+func (m *CriuOpts) GetParentImg() string {
+	if m != nil && m.ParentImg != nil {
+		return *m.ParentImg
+	}
+	return ""
+}
+
+func (m *CriuOpts) GetTrackMem() bool {
+	if m != nil && m.TrackMem != nil {
+		return *m.TrackMem
+	}
+	return false
+}
+
+func (m *CriuOpts) GetAutoDedup() bool {
+	if m != nil && m.AutoDedup != nil {
+		return *m.AutoDedup
+	}
+	return false
+}
+
+func (m *CriuOpts) GetWorkDirFd() int32 {
+	if m != nil && m.WorkDirFd != nil {
+		return *m.WorkDirFd
+	}
+	return 0
+}
+
+func (m *CriuOpts) GetLinkRemap() bool {
+	if m != nil && m.LinkRemap != nil {
+		return *m.LinkRemap
+	}
+	return false
+}
+
+func (m *CriuOpts) GetVeths() []*CriuVethPair {
+	if m != nil {
+		return m.Veths
+	}
+	return nil
+}
+
+func (m *CriuOpts) GetCpuCap() uint32 {
+	if m != nil && m.CpuCap != nil {
+		return *m.CpuCap
+	}
+	return Default_CriuOpts_CpuCap
+}
+
+func (m *CriuOpts) GetForceIrmap() bool {
+	if m != nil && m.ForceIrmap != nil {
+		return *m.ForceIrmap
+	}
+	return false
+}
+
+func (m *CriuOpts) GetExecCmd() []string {
+	if m != nil {
+		return m.ExecCmd
+	}
+	return nil
+}
+
+func (m *CriuOpts) GetExtMnt() []*ExtMountMap {
+	if m != nil {
+		return m.ExtMnt
+	}
+	return nil
+}
+
+func (m *CriuOpts) GetManageCgroups() bool {
+	if m != nil && m.ManageCgroups != nil {
+		return *m.ManageCgroups
+	}
+	return false
+}
+
+func (m *CriuOpts) GetCgRoot() []*CgroupRoot {
+	if m != nil {
+		return m.CgRoot
+	}
+	return nil
+}
+
+func (m *CriuOpts) GetRstSibling() bool {
+	if m != nil && m.RstSibling != nil {
+		return *m.RstSibling
+	}
+	return false
+}
+
+func (m *CriuOpts) GetInheritFd() []*InheritFd {
+	if m != nil {
+		return m.InheritFd
+	}
+	return nil
+}
+
+type CriuDumpResp struct {
+	Restored         *bool  `protobuf:"varint,1,opt,name=restored" json:"restored,omitempty"`
+	XXX_unrecognized []byte `json:"-"`
+}
+
+func (m *CriuDumpResp) Reset()         { *m = CriuDumpResp{} }
+func (m *CriuDumpResp) String() string { return proto.CompactTextString(m) }
+func (*CriuDumpResp) ProtoMessage()    {}
+
+func (m *CriuDumpResp) GetRestored() bool {
+	if m != nil && m.Restored != nil {
+		return *m.Restored
+	}
+	return false
+}
+
+type CriuRestoreResp struct {
+	Pid              *int32 `protobuf:"varint,1,req,name=pid" json:"pid,omitempty"`
+	XXX_unrecognized []byte `json:"-"`
+}
+
+func (m *CriuRestoreResp) Reset()         { *m = CriuRestoreResp{} }
+func (m *CriuRestoreResp) String() string { return proto.CompactTextString(m) }
+func (*CriuRestoreResp) ProtoMessage()    {}
+
+func (m *CriuRestoreResp) GetPid() int32 {
+	if m != nil && m.Pid != nil {
+		return *m.Pid
+	}
+	return 0
+}
+
+type CriuNotify struct {
+	Script           *string `protobuf:"bytes,1,opt,name=script" json:"script,omitempty"`
+	Pid              *int32  `protobuf:"varint,2,opt,name=pid" json:"pid,omitempty"`
+	XXX_unrecognized []byte  `json:"-"`
+}
+
+func (m *CriuNotify) Reset()         { *m = CriuNotify{} }
+func (m *CriuNotify) String() string { return proto.CompactTextString(m) }
+func (*CriuNotify) ProtoMessage()    {}
+
+func (m *CriuNotify) GetScript() string {
+	if m != nil && m.Script != nil {
+		return *m.Script
+	}
+	return ""
+}
+
+func (m *CriuNotify) GetPid() int32 {
+	if m != nil && m.Pid != nil {
+		return *m.Pid
+	}
+	return 0
+}
+
+type CriuReq struct {
+	Type          *CriuReqType `protobuf:"varint,1,req,name=type,enum=CriuReqType" json:"type,omitempty"`
+	Opts          *CriuOpts    `protobuf:"bytes,2,opt,name=opts" json:"opts,omitempty"`
+	NotifySuccess *bool        `protobuf:"varint,3,opt,name=notify_success" json:"notify_success,omitempty"`
+	//
+	// When set service won't close the connection but
+	// will wait for more req-s to appear. Works not
+	// for all request types.
+	KeepOpen         *bool  `protobuf:"varint,4,opt,name=keep_open" json:"keep_open,omitempty"`
+	XXX_unrecognized []byte `json:"-"`
+}
+
+func (m *CriuReq) Reset()         { *m = CriuReq{} }
+func (m *CriuReq) String() string { return proto.CompactTextString(m) }
+func (*CriuReq) ProtoMessage()    {}
+
+func (m *CriuReq) GetType() CriuReqType {
+	if m != nil && m.Type != nil {
+		return *m.Type
+	}
+	return 0
+}
+
+func (m *CriuReq) GetOpts() *CriuOpts {
+	if m != nil {
+		return m.Opts
+	}
+	return nil
+}
+
+func (m *CriuReq) GetNotifySuccess() bool {
+	if m != nil && m.NotifySuccess != nil {
+		return *m.NotifySuccess
+	}
+	return false
+}
+
+func (m *CriuReq) GetKeepOpen() bool {
+	if m != nil && m.KeepOpen != nil {
+		return *m.KeepOpen
+	}
+	return false
+}
+
+type CriuResp struct {
+	Type             *CriuReqType        `protobuf:"varint,1,req,name=type,enum=CriuReqType" json:"type,omitempty"`
+	Success          *bool               `protobuf:"varint,2,req,name=success" json:"success,omitempty"`
+	Dump             *CriuDumpResp       `protobuf:"bytes,3,opt,name=dump" json:"dump,omitempty"`
+	Restore          *CriuRestoreResp    `protobuf:"bytes,4,opt,name=restore" json:"restore,omitempty"`
+	Notify           *CriuNotify         `protobuf:"bytes,5,opt,name=notify" json:"notify,omitempty"`
+	Ps               *CriuPageServerInfo `protobuf:"bytes,6,opt,name=ps" json:"ps,omitempty"`
+	CrErrno          *int32              `protobuf:"varint,7,opt,name=cr_errno" json:"cr_errno,omitempty"`
+	XXX_unrecognized []byte              `json:"-"`
+}
+
+func (m *CriuResp) Reset()         { *m = CriuResp{} }
+func (m *CriuResp) String() string { return proto.CompactTextString(m) }
+func (*CriuResp) ProtoMessage()    {}
+
+func (m *CriuResp) GetType() CriuReqType {
+	if m != nil && m.Type != nil {
+		return *m.Type
+	}
+	return 0
+}
+
+func (m *CriuResp) GetSuccess() bool {
+	if m != nil && m.Success != nil {
+		return *m.Success
+	}
+	return false
+}
+
+func (m *CriuResp) GetDump() *CriuDumpResp {
+	if m != nil {
+		return m.Dump
+	}
+	return nil
+}
+
+func (m *CriuResp) GetRestore() *CriuRestoreResp {
+	if m != nil {
+		return m.Restore
+	}
+	return nil
+}
+
+func (m *CriuResp) GetNotify() *CriuNotify {
+	if m != nil {
+		return m.Notify
+	}
+	return nil
+}
+
+func (m *CriuResp) GetPs() *CriuPageServerInfo {
+	if m != nil {
+		return m.Ps
+	}
+	return nil
+}
+
+func (m *CriuResp) GetCrErrno() int32 {
+	if m != nil && m.CrErrno != nil {
+		return *m.CrErrno
+	}
+	return 0
+}
+
+func init() {
+	proto.RegisterEnum("CriuReqType", CriuReqType_name, CriuReqType_value)
+}

libcontainer/criurpc/criurpc.proto

@@ -0,0 +1,127 @@
+message criu_page_server_info {
+	optional string		address	= 1;
+	optional int32		port	= 2;
+	optional int32		pid	= 3;
+	optional int32		fd	= 4;
+}
+
+message criu_veth_pair {
+	required string		if_in	= 1;
+	required string		if_out	= 2;
+};
+
+message ext_mount_map {
+	required string		key	= 1;
+	required string		val	= 2;
+};
+
+message inherit_fd {
+	required string		key	= 1;
+	required int32		fd	= 2;
+};
+
+message cgroup_root {
+	optional string		ctrl	= 1;
+	required string		path	= 2;
+};
+
+message criu_opts {
+	required int32			images_dir_fd	= 1;
+	optional int32			pid		= 2; /* if not set on dump, will dump requesting process */
+
+	optional bool			leave_running	= 3;
+	optional bool			ext_unix_sk	= 4;
+	optional bool			tcp_established	= 5;
+	optional bool			evasive_devices	= 6;
+	optional bool			shell_job	= 7;
+	optional bool			file_locks	= 8;
+	optional int32			log_level	= 9 [default = 2];
+	optional string			log_file	= 10; /* No subdirs are allowed. Consider using work-dir */
+
+	optional criu_page_server_info	ps		= 11;
+
+	optional bool			notify_scripts	= 12;
+
+	optional string			root		= 13;
+	optional string			parent_img	= 14;
+	optional bool			track_mem	= 15;
+	optional bool			auto_dedup	= 16;
+
+	optional int32			work_dir_fd	= 17;
+	optional bool			link_remap	= 18;
+	repeated criu_veth_pair		veths		= 19;
+
+	optional uint32			cpu_cap		= 20 [default = 0xffffffff];
+	optional bool			force_irmap	= 21;
+	repeated string			exec_cmd	= 22;
+
+	repeated ext_mount_map		ext_mnt		= 23;
+	optional bool			manage_cgroups	= 24;
+	repeated cgroup_root		cg_root		= 25;
+
+	optional bool			rst_sibling	= 26; /* swrk only */
+	repeated inherit_fd		inherit_fd	= 27;
+}
+
+message criu_dump_resp {
+	optional bool restored		= 1;
+}
+
+message criu_restore_resp {
+	required int32 pid		= 1;
+}
+
+message criu_notify {
+	optional string script		= 1;
+	optional int32	pid		= 2;
+}
+
+enum criu_req_type {
+	EMPTY		= 0;
+	DUMP		= 1;
+	RESTORE		= 2;
+	CHECK		= 3;
+	PRE_DUMP	= 4;
+	PAGE_SERVER	= 5;
+
+	NOTIFY		= 6;
+
+	CPUINFO_DUMP	= 7;
+	CPUINFO_CHECK	= 8;
+}
+
+/*
+ * Request -- each type corresponds to must-be-there
+ * request arguments of respective type
+ */
+
+message criu_req {
+	required criu_req_type		type		= 1;
+
+	optional criu_opts		opts		= 2;
+	optional bool			notify_success	= 3;
+
+	/*
+	 * When set service won't close the connection but
+	 * will wait for more req-s to appear. Works not
+	 * for all request types.
+	 */
+	optional bool			keep_open	= 4;
+}
+
+/*
+ * Responce -- it states whether the request was served
+ * and additional request-specific informarion
+ */
+
+message criu_resp {
+	required criu_req_type		type		= 1;
+	required bool			success		= 2;
+
+	optional criu_dump_resp		dump		= 3;
+	optional criu_restore_resp	restore		= 4;
+	optional criu_notify		notify		= 5;
+	optional criu_page_server_info	ps		= 6;
+
+	optional int32			cr_errno	= 7;
+}

libcontainer/devices/devices_test.go

@@ -0,0 +1,61 @@
+package devices
+
+import (
+	"errors"
+	"os"
+	"testing"
+)
+
+func TestDeviceFromPathLstatFailure(t *testing.T) {
+	testError := errors.New("test error")
+
+	// Override os.Lstat to inject error.
+	osLstat = func(path string) (os.FileInfo, error) {
+		return nil, testError
+	}
+
+	_, err := DeviceFromPath("", "")
+	if err != testError {
+		t.Fatalf("Unexpected error %v, expected %v", err, testError)
+	}
+}
+
+func TestHostDevicesIoutilReadDirFailure(t *testing.T) {
+	testError := errors.New("test error")
+
+	// Override ioutil.ReadDir to inject error.
+	ioutilReadDir = func(dirname string) ([]os.FileInfo, error) {
+		return nil, testError
+	}
+
+	_, err := HostDevices()
+	if err != testError {
+		t.Fatalf("Unexpected error %v, expected %v", err, testError)
+	}
+}
+
+func TestHostDevicesIoutilReadDirDeepFailure(t *testing.T) {
+	testError := errors.New("test error")
+	called := false
+
+	// Override ioutil.ReadDir to inject error after the first call.
+	ioutilReadDir = func(dirname string) ([]os.FileInfo, error) {
+		if called {
+			return nil, testError
+		}
+		called = true
+
+		// Provoke a second call.
+		fi, err := os.Lstat("/tmp")
+		if err != nil {
+			t.Fatalf("Unexpected error %v", err)
+		}
+
+		return []os.FileInfo{fi}, nil
+	}
+
+	_, err := HostDevices()
+	if err != testError {
+		t.Fatalf("Unexpected error %v, expected %v", err, testError)
+	}
+}

libcontainer/devices/devices_unix.go

@@ -0,0 +1,102 @@
+// +build linux freebsd
+
+package devices
+
+import (
+	"errors"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"syscall"
+
+	"github.com/docker/libcontainer/configs"
+)
+
+var (
+	ErrNotADevice = errors.New("not a device node")
+)
+
+// Testing dependencies
+var (
+	osLstat       = os.Lstat
+	ioutilReadDir = ioutil.ReadDir
+)
+
+// Given the path to a device and it's cgroup_permissions(which cannot be easily queried) look up the information about a linux device and return that information as a Device struct.
+func DeviceFromPath(path, permissions string) (*configs.Device, error) {
+	fileInfo, err := osLstat(path)
+	if err != nil {
+		return nil, err
+	}
+	var (
+		devType                rune
+		mode                   = fileInfo.Mode()
+		fileModePermissionBits = os.FileMode.Perm(mode)
+	)
+	switch {
+	case mode&os.ModeDevice == 0:
+		return nil, ErrNotADevice
+	case mode&os.ModeCharDevice != 0:
+		fileModePermissionBits |= syscall.S_IFCHR
+		devType = 'c'
+	default:
+		fileModePermissionBits |= syscall.S_IFBLK
+		devType = 'b'
+	}
+	stat_t, ok := fileInfo.Sys().(*syscall.Stat_t)
+	if !ok {
+		return nil, fmt.Errorf("cannot determine the device number for device %s", path)
+	}
+	devNumber := int(stat_t.Rdev)
+	return &configs.Device{
+		Type:        devType,
+		Path:        path,
+		Major:       Major(devNumber),
+		Minor:       Minor(devNumber),
+		Permissions: permissions,
+		FileMode:    fileModePermissionBits,
+		Uid:         stat_t.Uid,
+		Gid:         stat_t.Gid,
+	}, nil
+}
+
+func HostDevices() ([]*configs.Device, error) {
+	return getDevices("/dev")
+}
+
+func getDevices(path string) ([]*configs.Device, error) {
+	files, err := ioutilReadDir(path)
+	if err != nil {
+		return nil, err
+	}
+	out := []*configs.Device{}
+	for _, f := range files {
+		switch {
+		case f.IsDir():
+			switch f.Name() {
+			case "pts", "shm", "fd", "mqueue":
+				continue
+			default:
+				sub, err := getDevices(filepath.Join(path, f.Name()))
+				if err != nil {
+					return nil, err
+				}
+
+				out = append(out, sub...)
+				continue
+			}
+		case f.Name() == "console":
+			continue
+		}
+		device, err := DeviceFromPath(filepath.Join(path, f.Name()), "rwm")
+		if err != nil {
+			if err == ErrNotADevice {
+				continue
+			}
+			return nil, err
+		}
+		out = append(out, device)
+	}
+	return out, nil
+}

libcontainer/devices/devices_windows.go

@@ -0,0 +1,16 @@
+package devices
+
+import (
+	"github.com/docker/libcontainer/configs"
+)
+
+// TODO Windows. This can be factored out further - Devices are not supported
+// by Windows Containers.
+
+func DeviceFromPath(path, permissions string) (*configs.Device, error) {
+	return nil, nil
+}
+
+func HostDevices() ([]*configs.Device, error) {
+	return nil, nil
+}

libcontainer/devices/number.go

@@ -0,0 +1,24 @@
+// +build linux freebsd
+
+package devices
+
+/*
+
+This code provides support for manipulating linux device numbers.  It should be replaced by normal syscall functions once http://code.google.com/p/go/issues/detail?id=8106 is solved.
+
+You can read what they are here:
+
+ - http://www.makelinux.net/ldd3/chp-3-sect-2
+ - http://www.linux-tutorial.info/modules.php?name=MContent&pageid=94
+
+Note! These are NOT the same as the MAJOR(dev_t device);, MINOR(dev_t device); and MKDEV(int major, int minor); functions as defined in <linux/kdev_t.h> as the representation of device numbers used by go is different than the one used internally to the kernel! - https://github.com/torvalds/linux/blob/master/include/linux/kdev_t.h#L9
+
+*/
+
+func Major(devNumber int) int64 {
+	return int64((devNumber >> 8) & 0xfff)
+}
+
+func Minor(devNumber int) int64 {
+	return int64((devNumber & 0xff) | ((devNumber >> 12) & 0xfff00))
+}

libcontainer/docs/man/nsinit.1.md

@@ -0,0 +1,38 @@
+% nsinit User Manual
+% docker/libcontainer
+% JAN 2015
+
+NAME:
+   nsinit - A low-level utility for managing containers.
+	    It is used to spawn new containers or join existing containers.
+
+USAGE:
+   nsinit [global options] command [command options] [arguments...]
+
+VERSION:
+   0.1
+
+COMMANDS:
+	config	display the container configuration 
+	exec	execute a new command inside a container
+	init	runs the init process inside the namespace
+	oom	display oom notifications for a container
+	pause	pause the container's processes
+	stats	display statistics for the container
+	unpause	unpause the container's processes
+	help, h	shows a list of commands or help for one command
+
+EXAMPLES:
+
+Get the <container_id> of an already running docker container.
+`sudo docker ps` will return the list of all the running containers.
+
+take the <container_id> (e.g. 4addb0b2d307) and go to its config directory
+`/var/lib/docker/execdriver/native/4addb0b2d307` and here you can run the nsinit
+command line utility.
+
+e.g. `nsinit exec /bin/bash` will start a shell on the already running container.
+   
+# HISTORY
+Jan 2015, Originally compiled by Shishir Mahajan (shishir dot mahajan at redhat dot com)
+based on nsinit source material and internal work.	

libcontainer/error.go

@@ -0,0 +1,62 @@
+package libcontainer
+
+import "io"
+
+// API error code type.
+type ErrorCode int
+
+// API error codes.
+const (
+	// Factory errors
+	IdInUse ErrorCode = iota
+	InvalidIdFormat
+
+	// Container errors
+	ContainerNotExists
+	ContainerPaused
+	ContainerNotStopped
+	ContainerNotRunning
+
+	// Process errors
+	ProcessNotExecuted
+
+	// Common errors
+	ConfigInvalid
+	SystemError
+)
+
+func (c ErrorCode) String() string {
+	switch c {
+	case IdInUse:
+		return "Id already in use"
+	case InvalidIdFormat:
+		return "Invalid format"
+	case ContainerPaused:
+		return "Container paused"
+	case ConfigInvalid:
+		return "Invalid configuration"
+	case SystemError:
+		return "System error"
+	case ContainerNotExists:
+		return "Container does not exist"
+	case ContainerNotStopped:
+		return "Container is not stopped"
+	case ContainerNotRunning:
+		return "Container is not running"
+	default:
+		return "Unknown error"
+	}
+}
+
+// API Error type.
+type Error interface {
+	error
+
+	// Returns a verbose string including the error message
+	// and a representation of the stack trace suitable for
+	// printing.
+	Detail(w io.Writer) error
+
+	// Returns the error code for this error.
+	Code() ErrorCode
+}

libcontainer/error_test.go

@@ -0,0 +1,20 @@
+package libcontainer
+
+import "testing"
+
+func TestErrorCode(t *testing.T) {
+	codes := map[ErrorCode]string{
+		IdInUse:            "Id already in use",
+		InvalidIdFormat:    "Invalid format",
+		ContainerPaused:    "Container paused",
+		ConfigInvalid:      "Invalid configuration",
+		SystemError:        "System error",
+		ContainerNotExists: "Container does not exist",
+	}
+
+	for code, expected := range codes {
+		if actual := code.String(); actual != expected {
+			t.Fatalf("expected string %q but received %q", expected, actual)
+		}
+	}
+}

libcontainer/factory.go

@@ -0,0 +1,45 @@
+package libcontainer
+
+import (
+	"github.com/docker/libcontainer/configs"
+)
+
+type Factory interface {
+	// Creates a new container with the given id and starts the initial process inside it.
+	// id must be a string containing only letters, digits and underscores and must contain
+	// between 1 and 1024 characters, inclusive.
+	//
+	// The id must not already be in use by an existing container. Containers created using
+	// a factory with the same path (and file system) must have distinct ids.
+	//
+	// Returns the new container with a running process.
+	//
+	// errors:
+	// IdInUse - id is already in use by a container
+	// InvalidIdFormat - id has incorrect format
+	// ConfigInvalid - config is invalid
+	// Systemerror - System error
+	//
+	// On error, any partially created container parts are cleaned up (the operation is atomic).
+	Create(id string, config *configs.Config) (Container, error)
+
+	// Load takes an ID for an existing container and returns the container information
+	// from the state.  This presents a read only view of the container.
+	//
+	// errors:
+	// Path does not exist
+	// Container is stopped
+	// System error
+	Load(id string) (Container, error)
+
+	// StartInitialization is an internal API to libcontainer used during the reexec of the
+	// container.
+	//
+	// Errors:
+	// Pipe connection error
+	// System error
+	StartInitialization() error
+
+	// Type returns info string about factory type (e.g. lxc, libcontainer...)
+	Type() string
+}

libcontainer/factory_linux.go

@@ -0,0 +1,263 @@
+// +build linux
+
+package libcontainer
+
+import (
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"regexp"
+	"strconv"
+	"syscall"
+
+	"github.com/docker/docker/pkg/mount"
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/cgroups/fs"
+	"github.com/docker/libcontainer/cgroups/systemd"
+	"github.com/docker/libcontainer/configs"
+	"github.com/docker/libcontainer/configs/validate"
+)
+
+const (
+	stateFilename = "state.json"
+)
+
+var (
+	idRegex  = regexp.MustCompile(`^[\w_]+$`)
+	maxIdLen = 1024
+)
+
+// InitArgs returns an options func to configure a LinuxFactory with the
+// provided init arguments.
+func InitArgs(args ...string) func(*LinuxFactory) error {
+	return func(l *LinuxFactory) error {
+		name := args[0]
+		if filepath.Base(name) == name {
+			if lp, err := exec.LookPath(name); err == nil {
+				name = lp
+			}
+		}
+		l.InitPath = name
+		l.InitArgs = append([]string{name}, args[1:]...)
+		return nil
+	}
+}
+
+// InitPath returns an options func to configure a LinuxFactory with the
+// provided absolute path to the init binary and arguements.
+func InitPath(path string, args ...string) func(*LinuxFactory) error {
+	return func(l *LinuxFactory) error {
+		l.InitPath = path
+		l.InitArgs = args
+		return nil
+	}
+}
+
+// SystemdCgroups is an options func to configure a LinuxFactory to return
+// containers that use systemd to create and manage cgroups.
+func SystemdCgroups(l *LinuxFactory) error {
+	l.NewCgroupsManager = func(config *configs.Cgroup, paths map[string]string) cgroups.Manager {
+		return &systemd.Manager{
+			Cgroups: config,
+			Paths:   paths,
+		}
+	}
+	return nil
+}
+
+// Cgroupfs is an options func to configure a LinuxFactory to return
+// containers that use the native cgroups filesystem implementation to
+// create and manage cgroups.
+func Cgroupfs(l *LinuxFactory) error {
+	l.NewCgroupsManager = func(config *configs.Cgroup, paths map[string]string) cgroups.Manager {
+		return &fs.Manager{
+			Cgroups: config,
+			Paths:   paths,
+		}
+	}
+	return nil
+}
+
+// TmpfsRoot is an option func to mount LinuxFactory.Root to tmpfs.
+func TmpfsRoot(l *LinuxFactory) error {
+	mounted, err := mount.Mounted(l.Root)
+	if err != nil {
+		return err
+	}
+	if !mounted {
+		if err := syscall.Mount("tmpfs", l.Root, "tmpfs", 0, ""); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+// New returns a linux based container factory based in the root directory and
+// configures the factory with the provided option funcs.
+func New(root string, options ...func(*LinuxFactory) error) (Factory, error) {
+	if root != "" {
+		if err := os.MkdirAll(root, 0700); err != nil {
+			return nil, newGenericError(err, SystemError)
+		}
+	}
+	l := &LinuxFactory{
+		Root:      root,
+		Validator: validate.New(),
+		CriuPath:  "criu",
+	}
+	InitArgs(os.Args[0], "init")(l)
+	Cgroupfs(l)
+	for _, opt := range options {
+		if err := opt(l); err != nil {
+			return nil, err
+		}
+	}
+	return l, nil
+}
+
+// LinuxFactory implements the default factory interface for linux based systems.
+type LinuxFactory struct {
+	// Root directory for the factory to store state.
+	Root string
+
+	// InitPath is the absolute path to the init binary.
+	InitPath string
+
+	// InitArgs are arguments for calling the init responsibilities for spawning
+	// a container.
+	InitArgs []string
+
+	// CriuPath is the path to the criu binary used for checkpoint and restore of
+	// containers.
+	CriuPath string
+
+	// Validator provides validation to container configurations.
+	Validator validate.Validator
+
+	// NewCgroupsManager returns an initialized cgroups manager for a single container.
+	NewCgroupsManager func(config *configs.Cgroup, paths map[string]string) cgroups.Manager
+}
+
+func (l *LinuxFactory) Create(id string, config *configs.Config) (Container, error) {
+	if l.Root == "" {
+		return nil, newGenericError(fmt.Errorf("invalid root"), ConfigInvalid)
+	}
+	if err := l.validateID(id); err != nil {
+		return nil, err
+	}
+	if err := l.Validator.Validate(config); err != nil {
+		return nil, newGenericError(err, ConfigInvalid)
+	}
+	containerRoot := filepath.Join(l.Root, id)
+	if _, err := os.Stat(containerRoot); err == nil {
+		return nil, newGenericError(fmt.Errorf("Container with id exists: %v", id), IdInUse)
+	} else if !os.IsNotExist(err) {
+		return nil, newGenericError(err, SystemError)
+	}
+	if err := os.MkdirAll(containerRoot, 0700); err != nil {
+		return nil, newGenericError(err, SystemError)
+	}
+	return &linuxContainer{
+		id:            id,
+		root:          containerRoot,
+		config:        config,
+		initPath:      l.InitPath,
+		initArgs:      l.InitArgs,
+		criuPath:      l.CriuPath,
+		cgroupManager: l.NewCgroupsManager(config.Cgroups, nil),
+	}, nil
+}
+
+func (l *LinuxFactory) Load(id string) (Container, error) {
+	if l.Root == "" {
+		return nil, newGenericError(fmt.Errorf("invalid root"), ConfigInvalid)
+	}
+	containerRoot := filepath.Join(l.Root, id)
+	state, err := l.loadState(containerRoot)
+	if err != nil {
+		return nil, err
+	}
+	r := &nonChildProcess{
+		processPid:       state.InitProcessPid,
+		processStartTime: state.InitProcessStartTime,
+		fds:              state.ExternalDescriptors,
+	}
+	return &linuxContainer{
+		initProcess:   r,
+		id:            id,
+		config:        &state.Config,
+		initPath:      l.InitPath,
+		initArgs:      l.InitArgs,
+		criuPath:      l.CriuPath,
+		cgroupManager: l.NewCgroupsManager(state.Config.Cgroups, state.CgroupPaths),
+		root:          containerRoot,
+	}, nil
+}
+
+func (l *LinuxFactory) Type() string {
+	return "libcontainer"
+}
+
+// StartInitialization loads a container by opening the pipe fd from the parent to read the configuration and state
+// This is a low level implementation detail of the reexec and should not be consumed externally
+func (l *LinuxFactory) StartInitialization() (err error) {
+	pipefd, err := strconv.Atoi(os.Getenv("_LIBCONTAINER_INITPIPE"))
+	if err != nil {
+		return err
+	}
+	var (
+		pipe = os.NewFile(uintptr(pipefd), "pipe")
+		it   = initType(os.Getenv("_LIBCONTAINER_INITTYPE"))
+	)
+	// clear the current process's environment to clean any libcontainer
+	// specific env vars.
+	os.Clearenv()
+	defer func() {
+		// if we have an error during the initialization of the container's init then send it back to the
+		// parent process in the form of an initError.
+		if err != nil {
+			// ensure that any data sent from the parent is consumed so it doesn't
+			// receive ECONNRESET when the child writes to the pipe.
+			ioutil.ReadAll(pipe)
+			if err := json.NewEncoder(pipe).Encode(newSystemError(err)); err != nil {
+				panic(err)
+			}
+		}
+		// ensure that this pipe is always closed
+		pipe.Close()
+	}()
+	i, err := newContainerInit(it, pipe)
+	if err != nil {
+		return err
+	}
+	return i.Init()
+}
+
+func (l *LinuxFactory) loadState(root string) (*State, error) {
+	f, err := os.Open(filepath.Join(root, stateFilename))
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, newGenericError(err, ContainerNotExists)
+		}
+		return nil, newGenericError(err, SystemError)
+	}
+	defer f.Close()
+	var state *State
+	if err := json.NewDecoder(f).Decode(&state); err != nil {
+		return nil, newGenericError(err, SystemError)
+	}
+	return state, nil
+}
+
+func (l *LinuxFactory) validateID(id string) error {
+	if !idRegex.MatchString(id) {
+		return newGenericError(fmt.Errorf("Invalid id format: %v", id), InvalidIdFormat)
+	}
+	if len(id) > maxIdLen {
+		return newGenericError(fmt.Errorf("Invalid id format: %v", id), InvalidIdFormat)
+	}
+	return nil
+}

libcontainer/factory_linux_test.go

@@ -0,0 +1,179 @@
+// +build linux
+
+package libcontainer
+
+import (
+	"encoding/json"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/docker/docker/pkg/mount"
+	"github.com/docker/libcontainer/configs"
+)
+
+func newTestRoot() (string, error) {
+	dir, err := ioutil.TempDir("", "libcontainer")
+	if err != nil {
+		return "", err
+	}
+	return dir, nil
+}
+
+func TestFactoryNew(t *testing.T) {
+	root, rerr := newTestRoot()
+	if rerr != nil {
+		t.Fatal(rerr)
+	}
+	defer os.RemoveAll(root)
+	factory, err := New(root, Cgroupfs)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if factory == nil {
+		t.Fatal("factory should not be nil")
+	}
+	lfactory, ok := factory.(*LinuxFactory)
+	if !ok {
+		t.Fatal("expected linux factory returned on linux based systems")
+	}
+	if lfactory.Root != root {
+		t.Fatalf("expected factory root to be %q but received %q", root, lfactory.Root)
+	}
+
+	if factory.Type() != "libcontainer" {
+		t.Fatalf("unexpected factory type: %q, expected %q", factory.Type(), "libcontainer")
+	}
+}
+
+func TestFactoryNewTmpfs(t *testing.T) {
+	root, rerr := newTestRoot()
+	if rerr != nil {
+		t.Fatal(rerr)
+	}
+	defer os.RemoveAll(root)
+	factory, err := New(root, Cgroupfs, TmpfsRoot)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if factory == nil {
+		t.Fatal("factory should not be nil")
+	}
+	lfactory, ok := factory.(*LinuxFactory)
+	if !ok {
+		t.Fatal("expected linux factory returned on linux based systems")
+	}
+	if lfactory.Root != root {
+		t.Fatalf("expected factory root to be %q but received %q", root, lfactory.Root)
+	}
+
+	if factory.Type() != "libcontainer" {
+		t.Fatalf("unexpected factory type: %q, expected %q", factory.Type(), "libcontainer")
+	}
+	mounted, err := mount.Mounted(lfactory.Root)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !mounted {
+		t.Fatalf("Factory Root is not mounted")
+	}
+	mounts, err := mount.GetMounts()
+	if err != nil {
+		t.Fatal(err)
+	}
+	var found bool
+	for _, m := range mounts {
+		if m.Mountpoint == lfactory.Root {
+			if m.Fstype != "tmpfs" {
+				t.Fatalf("Fstype of root: %s, expected %s", m.Fstype, "tmpfs")
+			}
+			if m.Source != "tmpfs" {
+				t.Fatalf("Source of root: %s, expected %s", m.Source, "tmpfs")
+			}
+			found = true
+		}
+	}
+	if !found {
+		t.Fatalf("Factory Root is not listed in mounts list")
+	}
+}
+
+func TestFactoryLoadNotExists(t *testing.T) {
+	root, rerr := newTestRoot()
+	if rerr != nil {
+		t.Fatal(rerr)
+	}
+	defer os.RemoveAll(root)
+	factory, err := New(root, Cgroupfs)
+	if err != nil {
+		t.Fatal(err)
+	}
+	_, err = factory.Load("nocontainer")
+	if err == nil {
+		t.Fatal("expected nil error loading non-existing container")
+	}
+	lerr, ok := err.(Error)
+	if !ok {
+		t.Fatal("expected libcontainer error type")
+	}
+	if lerr.Code() != ContainerNotExists {
+		t.Fatalf("expected error code %s but received %s", ContainerNotExists, lerr.Code())
+	}
+}
+
+func TestFactoryLoadContainer(t *testing.T) {
+	root, err := newTestRoot()
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.RemoveAll(root)
+	// setup default container config and state for mocking
+	var (
+		id             = "1"
+		expectedConfig = &configs.Config{
+			Rootfs: "/mycontainer/root",
+		}
+		expectedState = &State{
+			InitProcessPid: 1024,
+			Config:         *expectedConfig,
+		}
+	)
+	if err := os.Mkdir(filepath.Join(root, id), 0700); err != nil {
+		t.Fatal(err)
+	}
+	if err := marshal(filepath.Join(root, id, stateFilename), expectedState); err != nil {
+		t.Fatal(err)
+	}
+	factory, err := New(root, Cgroupfs)
+	if err != nil {
+		t.Fatal(err)
+	}
+	container, err := factory.Load(id)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if container.ID() != id {
+		t.Fatalf("expected container id %q but received %q", id, container.ID())
+	}
+	config := container.Config()
+	if config.Rootfs != expectedConfig.Rootfs {
+		t.Fatalf("expected rootfs %q but received %q", expectedConfig.Rootfs, config.Rootfs)
+	}
+	lcontainer, ok := container.(*linuxContainer)
+	if !ok {
+		t.Fatal("expected linux container on linux based systems")
+	}
+	if lcontainer.initProcess.pid() != expectedState.InitProcessPid {
+		t.Fatalf("expected init pid %d but received %d", expectedState.InitProcessPid, lcontainer.initProcess.pid())
+	}
+}
+
+func marshal(path string, v interface{}) error {
+	f, err := os.Create(path)
+	if err != nil {
+		return err
+	}
+	defer f.Close()
+	return json.NewEncoder(f).Encode(v)
+}

libcontainer/generic_error.go

@@ -0,0 +1,74 @@
+package libcontainer
+
+import (
+	"fmt"
+	"io"
+	"text/template"
+	"time"
+
+	"github.com/docker/libcontainer/stacktrace"
+)
+
+var errorTemplate = template.Must(template.New("error").Parse(`Timestamp: {{.Timestamp}}
+Code: {{.ECode}}
+{{if .Message }}
+Message: {{.Message}}
+{{end}}
+Frames:{{range $i, $frame := .Stack.Frames}}
+---
+{{$i}}: {{$frame.Function}}
+Package: {{$frame.Package}}
+File: {{$frame.File}}@{{$frame.Line}}{{end}}
+`))
+
+func newGenericError(err error, c ErrorCode) Error {
+	if le, ok := err.(Error); ok {
+		return le
+	}
+	gerr := &genericError{
+		Timestamp: time.Now(),
+		Err:       err,
+		ECode:     c,
+		Stack:     stacktrace.Capture(1),
+	}
+	if err != nil {
+		gerr.Message = err.Error()
+	}
+	return gerr
+}
+
+func newSystemError(err error) Error {
+	if le, ok := err.(Error); ok {
+		return le
+	}
+	gerr := &genericError{
+		Timestamp: time.Now(),
+		Err:       err,
+		ECode:     SystemError,
+		Stack:     stacktrace.Capture(1),
+	}
+	if err != nil {
+		gerr.Message = err.Error()
+	}
+	return gerr
+}
+
+type genericError struct {
+	Timestamp time.Time
+	ECode     ErrorCode
+	Err       error `json:"-"`
+	Message   string
+	Stack     stacktrace.Stacktrace
+}
+
+func (e *genericError) Error() string {
+	return fmt.Sprintf("[%d] %s: %s", e.ECode, e.ECode, e.Message)
+}
+
+func (e *genericError) Code() ErrorCode {
+	return e.ECode
+}
+
+func (e *genericError) Detail(w io.Writer) error {
+	return errorTemplate.Execute(w, e)
+}

libcontainer/generic_error_test.go

@@ -0,0 +1,14 @@
+package libcontainer
+
+import (
+	"fmt"
+	"io/ioutil"
+	"testing"
+)
+
+func TestErrorDetail(t *testing.T) {
+	err := newGenericError(fmt.Errorf("test error"), SystemError)
+	if derr := err.Detail(ioutil.Discard); derr != nil {
+		t.Fatal(derr)
+	}
+}

libcontainer/hack/validate.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+set -e
+
+# This script runs all validations
+
+validate() {
+    export MAKEDIR=/go/src/github.com/docker/docker/hack/make
+    sed -i 's!docker/docker!docker/libcontainer!' /go/src/github.com/docker/docker/hack/make/.validate
+    bash /go/src/github.com/docker/docker/hack/make/validate-dco
+    bash /go/src/github.com/docker/docker/hack/make/validate-gofmt
+    go get golang.org/x/tools/cmd/vet
+    bash /go/src/github.com/docker/docker/hack/make/validate-vet
+}
+
+# run validations
+validate

libcontainer/init_linux.go

@@ -0,0 +1,330 @@
+// +build linux
+
+package libcontainer
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"strings"
+	"syscall"
+
+	"github.com/Sirupsen/logrus"
+	"github.com/docker/libcontainer/cgroups"
+	"github.com/docker/libcontainer/configs"
+	"github.com/docker/libcontainer/netlink"
+	"github.com/docker/libcontainer/seccomp"
+	"github.com/docker/libcontainer/system"
+	"github.com/docker/libcontainer/user"
+	"github.com/docker/libcontainer/utils"
+)
+
+type initType string
+
+const (
+	initSetns    initType = "setns"
+	initStandard initType = "standard"
+)
+
+type pid struct {
+	Pid int `json:"pid"`
+}
+
+// network is an internal struct used to setup container networks.
+type network struct {
+	configs.Network
+
+	// TempVethPeerName is a unique tempory veth peer name that was placed into
+	// the container's namespace.
+	TempVethPeerName string `json:"temp_veth_peer_name"`
+}
+
+// initConfig is used for transferring parameters from Exec() to Init()
+type initConfig struct {
+	Args             []string        `json:"args"`
+	Env              []string        `json:"env"`
+	Cwd              string          `json:"cwd"`
+	Capabilities     []string        `json:"capabilities"`
+	User             string          `json:"user"`
+	Config           *configs.Config `json:"config"`
+	Console          string          `json:"console"`
+	Networks         []*network      `json:"network"`
+	PassedFilesCount int             `json:"passed_files_count"`
+}
+
+type initer interface {
+	Init() error
+}
+
+func newContainerInit(t initType, pipe *os.File) (initer, error) {
+	var config *initConfig
+	if err := json.NewDecoder(pipe).Decode(&config); err != nil {
+		return nil, err
+	}
+	if err := populateProcessEnvironment(config.Env); err != nil {
+		return nil, err
+	}
+	switch t {
+	case initSetns:
+		return &linuxSetnsInit{
+			config: config,
+		}, nil
+	case initStandard:
+		return &linuxStandardInit{
+			parentPid: syscall.Getppid(),
+			config:    config,
+		}, nil
+	}
+	return nil, fmt.Errorf("unknown init type %q", t)
+}
+
+// populateProcessEnvironment loads the provided environment variables into the
+// current processes's environment.
+func populateProcessEnvironment(env []string) error {
+	for _, pair := range env {
+		p := strings.SplitN(pair, "=", 2)
+		if len(p) < 2 {
+			return fmt.Errorf("invalid environment '%v'", pair)
+		}
+		if err := os.Setenv(p[0], p[1]); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+// finalizeNamespace drops the caps, sets the correct user
+// and working dir, and closes any leaked file descriptors
+// before executing the command inside the namespace
+func finalizeNamespace(config *initConfig) error {
+	// Ensure that all unwanted fds we may have accidentally
+	// inherited are marked close-on-exec so they stay out of the
+	// container
+	if err := utils.CloseExecFrom(config.PassedFilesCount + 3); err != nil {
+		return err
+	}
+
+	capabilities := config.Config.Capabilities
+	if config.Capabilities != nil {
+		capabilities = config.Capabilities
+	}
+	w, err := newCapWhitelist(capabilities)
+	if err != nil {
+		return err
+	}
+	// drop capabilities in bounding set before changing user
+	if err := w.dropBoundingSet(); err != nil {
+		return err
+	}
+	// preserve existing capabilities while we change users
+	if err := system.SetKeepCaps(); err != nil {
+		return err
+	}
+	if err := setupUser(config); err != nil {
+		return err
+	}
+	if err := system.ClearKeepCaps(); err != nil {
+		return err
+	}
+	// drop all other capabilities
+	if err := w.drop(); err != nil {
+		return err
+	}
+	if config.Cwd != "" {
+		if err := syscall.Chdir(config.Cwd); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+// joinExistingNamespaces gets all the namespace paths specified for the container and
+// does a setns on the namespace fd so that the current process joins the namespace.
+func joinExistingNamespaces(namespaces []configs.Namespace) error {
+	for _, ns := range namespaces {
+		if ns.Path != "" {
+			f, err := os.OpenFile(ns.Path, os.O_RDONLY, 0)
+			if err != nil {
+				return err
+			}
+			err = system.Setns(f.Fd(), uintptr(ns.Syscall()))
+			f.Close()
+			if err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}
+
+// setupUser changes the groups, gid, and uid for the user inside the container
+func setupUser(config *initConfig) error {
+	// Set up defaults.
+	defaultExecUser := user.ExecUser{
+		Uid:  syscall.Getuid(),
+		Gid:  syscall.Getgid(),
+		Home: "/",
+	}
+	passwdPath, err := user.GetPasswdPath()
+	if err != nil {
+		return err
+	}
+	groupPath, err := user.GetGroupPath()
+	if err != nil {
+		return err
+	}
+	execUser, err := user.GetExecUserPath(config.User, &defaultExecUser, passwdPath, groupPath)
+	if err != nil {
+		return err
+	}
+
+	var addGroups []int
+	if len(config.Config.AdditionalGroups) > 0 {
+		addGroups, err = user.GetAdditionalGroupsPath(config.Config.AdditionalGroups, groupPath)
+		if err != nil {
+			return err
+		}
+	}
+
+	suppGroups := append(execUser.Sgids, addGroups...)
+	if err := syscall.Setgroups(suppGroups); err != nil {
+		return err
+	}
+
+	if err := system.Setgid(execUser.Gid); err != nil {
+		return err
+	}
+	if err := system.Setuid(execUser.Uid); err != nil {
+		return err
+	}
+	// if we didn't get HOME already, set it based on the user's HOME
+	if envHome := os.Getenv("HOME"); envHome == "" {
+		if err := os.Setenv("HOME", execUser.Home); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+// setupNetwork sets up and initializes any network interface inside the container.
+func setupNetwork(config *initConfig) error {
+	for _, config := range config.Networks {
+		strategy, err := getStrategy(config.Type)
+		if err != nil {
+			return err
+		}
+		if err := strategy.initialize(config); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func setupRoute(config *configs.Config) error {
+	for _, config := range config.Routes {
+		if err := netlink.AddRoute(config.Destination, config.Source, config.Gateway, config.InterfaceName); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func setupRlimits(config *configs.Config) error {
+	for _, rlimit := range config.Rlimits {
+		l := &syscall.Rlimit{Max: rlimit.Hard, Cur: rlimit.Soft}
+		if err := syscall.Setrlimit(rlimit.Type, l); err != nil {
+			return fmt.Errorf("error setting rlimit type %v: %v", rlimit.Type, err)
+		}
+	}
+	return nil
+}
+
+// killCgroupProcesses freezes then iterates over all the processes inside the
+// manager's cgroups sending a SIGKILL to each process then waiting for them to
+// exit.
+func killCgroupProcesses(m cgroups.Manager) error {
+	var procs []*os.Process
+	if err := m.Freeze(configs.Frozen); err != nil {
+		logrus.Warn(err)
+	}
+	pids, err := m.GetPids()
+	if err != nil {
+		m.Freeze(configs.Thawed)
+		return err
+	}
+	for _, pid := range pids {
+		if p, err := os.FindProcess(pid); err == nil {
+			procs = append(procs, p)
+			if err := p.Kill(); err != nil {
+				logrus.Warn(err)
+			}
+		}
+	}
+	if err := m.Freeze(configs.Thawed); err != nil {
+		logrus.Warn(err)
+	}
+	for _, p := range procs {
+		if _, err := p.Wait(); err != nil {
+			logrus.Warn(err)
+		}
+	}
+	return nil
+}
+
+func finalizeSeccomp(config *initConfig) error {
+	if config.Config.Seccomp == nil {
+		return nil
+	}
+	context := seccomp.New()
+	for _, s := range config.Config.Seccomp.Syscalls {
+		ss := &seccomp.Syscall{
+			Value:  uint32(s.Value),
+			Action: seccompAction(s.Action),
+		}
+		if len(s.Args) > 0 {
+			ss.Args = seccompArgs(s.Args)
+		}
+		context.Add(ss)
+	}
+	return context.Load()
+}
+
+func seccompAction(a configs.Action) seccomp.Action {
+	switch a {
+	case configs.Kill:
+		return seccomp.Kill
+	case configs.Trap:
+		return seccomp.Trap
+	case configs.Allow:
+		return seccomp.Allow
+	}
+	return seccomp.Error(syscall.Errno(int(a)))
+}
+
+func seccompArgs(args []*configs.Arg) seccomp.Args {
+	var sa []seccomp.Arg
+	for _, a := range args {
+		sa = append(sa, seccomp.Arg{
+			Index: uint32(a.Index),
+			Op:    seccompOperator(a.Op),
+			Value: uint(a.Value),
+		})
+	}
+	return seccomp.Args{sa}
+}
+
+func seccompOperator(o configs.Operator) seccomp.Operator {
+	switch o {
+	case configs.EqualTo:
+		return seccomp.EqualTo
+	case configs.NotEqualTo:
+		return seccomp.NotEqualTo
+	case configs.GreatherThan:
+		return seccomp.GreatherThan
+	case configs.LessThan:
+		return seccomp.LessThan
+	case configs.MaskEqualTo:
+		return seccomp.MaskEqualTo
+	}
+	return 0
+}

libcontainer/integration/checkpoint_test.go

@@ -0,0 +1,163 @@
+package integration
+
+import (
+	"bytes"
+	"io/ioutil"
+	"os"
+	"strings"
+	"testing"
+
+	"github.com/docker/libcontainer"
+)
+
+func TestCheckpoint(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	root, err := newTestRoot()
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.RemoveAll(root)
+
+	rootfs, err := newRootfs()
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+
+	factory, err := libcontainer.New(root, libcontainer.Cgroupfs)
+
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	container, err := factory.Create("test", config)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer container.Destroy()
+
+	stdinR, stdinW, err := os.Pipe()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	var stdout bytes.Buffer
+
+	pconfig := libcontainer.Process{
+		Args:   []string{"cat"},
+		Env:    standardEnvironment,
+		Stdin:  stdinR,
+		Stdout: &stdout,
+	}
+
+	err = container.Start(&pconfig)
+	stdinR.Close()
+	defer stdinW.Close()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	pid, err := pconfig.Pid()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	process, err := os.FindProcess(pid)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	imagesDir, err := ioutil.TempDir("", "criu")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.RemoveAll(imagesDir)
+
+	checkpointOpts := &libcontainer.CriuOpts{
+		ImagesDirectory: imagesDir,
+		WorkDirectory:   imagesDir,
+	}
+
+	if err := container.Checkpoint(checkpointOpts); err != nil {
+		t.Fatal(err)
+	}
+
+	state, err := container.Status()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if state != libcontainer.Checkpointed {
+		t.Fatal("Unexpected state: ", state)
+	}
+
+	stdinW.Close()
+	_, err = process.Wait()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// reload the container
+	container, err = factory.Load("test")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	restoreStdinR, restoreStdinW, err := os.Pipe()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	restoreProcessConfig := &libcontainer.Process{
+		Stdin:  restoreStdinR,
+		Stdout: &stdout,
+	}
+
+	err = container.Restore(restoreProcessConfig, &libcontainer.CriuOpts{
+		ImagesDirectory: imagesDir,
+	})
+	restoreStdinR.Close()
+	defer restoreStdinW.Close()
+
+	state, err = container.Status()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if state != libcontainer.Running {
+		t.Fatal("Unexpected state: ", state)
+	}
+
+	pid, err = restoreProcessConfig.Pid()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	process, err = os.FindProcess(pid)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	_, err = restoreStdinW.WriteString("Hello!")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	restoreStdinW.Close()
+	s, err := process.Wait()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if !s.Success() {
+		t.Fatal(s.String(), pid)
+	}
+
+	output := string(stdout.Bytes())
+	if !strings.Contains(output, "Hello!") {
+		t.Fatal("Did not restore the pipe correctly:", output)
+	}
+}

libcontainer/integration/doc.go

@@ -0,0 +1,2 @@
+// integration is used for integration testing of libcontainer
+package integration

libcontainer/integration/exec_test.go

@@ -0,0 +1,818 @@
+package integration
+
+import (
+	"bytes"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strconv"
+	"strings"
+	"syscall"
+	"testing"
+
+	"github.com/docker/libcontainer"
+	"github.com/docker/libcontainer/cgroups/systemd"
+	"github.com/docker/libcontainer/configs"
+)
+
+func TestExecPS(t *testing.T) {
+	testExecPS(t, false)
+}
+
+func TestUsernsExecPS(t *testing.T) {
+	if _, err := os.Stat("/proc/self/ns/user"); os.IsNotExist(err) {
+		t.Skip("userns is unsupported")
+	}
+	testExecPS(t, true)
+}
+
+func testExecPS(t *testing.T, userns bool) {
+	if testing.Short() {
+		return
+	}
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+	config := newTemplateConfig(rootfs)
+	if userns {
+		config.UidMappings = []configs.IDMap{{0, 0, 1000}}
+		config.GidMappings = []configs.IDMap{{0, 0, 1000}}
+		config.Namespaces = append(config.Namespaces, configs.Namespace{Type: configs.NEWUSER})
+	}
+
+	buffers, exitCode, err := runContainer(config, "", "ps")
+	if err != nil {
+		t.Fatalf("%s: %s", buffers, err)
+	}
+	if exitCode != 0 {
+		t.Fatalf("exit code not 0. code %d stderr %q", exitCode, buffers.Stderr)
+	}
+	lines := strings.Split(buffers.Stdout.String(), "\n")
+	if len(lines) < 2 {
+		t.Fatalf("more than one process running for output %q", buffers.Stdout.String())
+	}
+	expected := `1 root     ps`
+	actual := strings.Trim(lines[1], "\n ")
+	if actual != expected {
+		t.Fatalf("expected output %q but received %q", expected, actual)
+	}
+}
+
+func TestIPCPrivate(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	l, err := os.Readlink("/proc/1/ns/ipc")
+	ok(t, err)
+
+	config := newTemplateConfig(rootfs)
+	buffers, exitCode, err := runContainer(config, "", "readlink", "/proc/self/ns/ipc")
+	ok(t, err)
+
+	if exitCode != 0 {
+		t.Fatalf("exit code not 0. code %d stderr %q", exitCode, buffers.Stderr)
+	}
+
+	if actual := strings.Trim(buffers.Stdout.String(), "\n"); actual == l {
+		t.Fatalf("ipc link should be private to the container but equals host %q %q", actual, l)
+	}
+}
+
+func TestIPCHost(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	l, err := os.Readlink("/proc/1/ns/ipc")
+	ok(t, err)
+
+	config := newTemplateConfig(rootfs)
+	config.Namespaces.Remove(configs.NEWIPC)
+	buffers, exitCode, err := runContainer(config, "", "readlink", "/proc/self/ns/ipc")
+	ok(t, err)
+
+	if exitCode != 0 {
+		t.Fatalf("exit code not 0. code %d stderr %q", exitCode, buffers.Stderr)
+	}
+
+	if actual := strings.Trim(buffers.Stdout.String(), "\n"); actual != l {
+		t.Fatalf("ipc link not equal to host link %q %q", actual, l)
+	}
+}
+
+func TestIPCJoinPath(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	l, err := os.Readlink("/proc/1/ns/ipc")
+	ok(t, err)
+
+	config := newTemplateConfig(rootfs)
+	config.Namespaces.Add(configs.NEWIPC, "/proc/1/ns/ipc")
+
+	buffers, exitCode, err := runContainer(config, "", "readlink", "/proc/self/ns/ipc")
+	ok(t, err)
+
+	if exitCode != 0 {
+		t.Fatalf("exit code not 0. code %d stderr %q", exitCode, buffers.Stderr)
+	}
+
+	if actual := strings.Trim(buffers.Stdout.String(), "\n"); actual != l {
+		t.Fatalf("ipc link not equal to host link %q %q", actual, l)
+	}
+}
+
+func TestIPCBadPath(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+	config.Namespaces.Add(configs.NEWIPC, "/proc/1/ns/ipcc")
+
+	_, _, err = runContainer(config, "", "true")
+	if err == nil {
+		t.Fatal("container succeeded with bad ipc path")
+	}
+}
+
+func TestRlimit(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+	out, _, err := runContainer(config, "", "/bin/sh", "-c", "ulimit -n")
+	ok(t, err)
+	if limit := strings.TrimSpace(out.Stdout.String()); limit != "1025" {
+		t.Fatalf("expected rlimit to be 1025, got %s", limit)
+	}
+}
+
+func newTestRoot() (string, error) {
+	dir, err := ioutil.TempDir("", "libcontainer")
+	if err != nil {
+		return "", err
+	}
+	if err := os.MkdirAll(dir, 0700); err != nil {
+		return "", err
+	}
+	return dir, nil
+}
+
+func TestEnter(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	root, err := newTestRoot()
+	ok(t, err)
+	defer os.RemoveAll(root)
+
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+
+	container, err := factory.Create("test", config)
+	ok(t, err)
+	defer container.Destroy()
+
+	// Execute a first process in the container
+	stdinR, stdinW, err := os.Pipe()
+	ok(t, err)
+
+	var stdout, stdout2 bytes.Buffer
+
+	pconfig := libcontainer.Process{
+		Args:   []string{"sh", "-c", "cat && readlink /proc/self/ns/pid"},
+		Env:    standardEnvironment,
+		Stdin:  stdinR,
+		Stdout: &stdout,
+	}
+	err = container.Start(&pconfig)
+	stdinR.Close()
+	defer stdinW.Close()
+	ok(t, err)
+	pid, err := pconfig.Pid()
+	ok(t, err)
+
+	// Execute another process in the container
+	stdinR2, stdinW2, err := os.Pipe()
+	ok(t, err)
+	pconfig2 := libcontainer.Process{
+		Env: standardEnvironment,
+	}
+	pconfig2.Args = []string{"sh", "-c", "cat && readlink /proc/self/ns/pid"}
+	pconfig2.Stdin = stdinR2
+	pconfig2.Stdout = &stdout2
+
+	err = container.Start(&pconfig2)
+	stdinR2.Close()
+	defer stdinW2.Close()
+	ok(t, err)
+
+	pid2, err := pconfig2.Pid()
+	ok(t, err)
+
+	processes, err := container.Processes()
+	ok(t, err)
+
+	n := 0
+	for i := range processes {
+		if processes[i] == pid || processes[i] == pid2 {
+			n++
+		}
+	}
+	if n != 2 {
+		t.Fatal("unexpected number of processes", processes, pid, pid2)
+	}
+
+	// Wait processes
+	stdinW2.Close()
+	waitProcess(&pconfig2, t)
+
+	stdinW.Close()
+	waitProcess(&pconfig, t)
+
+	// Check that both processes live in the same pidns
+	pidns := string(stdout.Bytes())
+	ok(t, err)
+
+	pidns2 := string(stdout2.Bytes())
+	ok(t, err)
+
+	if pidns != pidns2 {
+		t.Fatal("The second process isn't in the required pid namespace", pidns, pidns2)
+	}
+}
+
+func TestProcessEnv(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	root, err := newTestRoot()
+	ok(t, err)
+	defer os.RemoveAll(root)
+
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+
+	container, err := factory.Create("test", config)
+	ok(t, err)
+	defer container.Destroy()
+
+	var stdout bytes.Buffer
+	pconfig := libcontainer.Process{
+		Args: []string{"sh", "-c", "env"},
+		Env: []string{
+			"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
+			"HOSTNAME=integration",
+			"TERM=xterm",
+			"FOO=BAR",
+		},
+		Stdin:  nil,
+		Stdout: &stdout,
+	}
+	err = container.Start(&pconfig)
+	ok(t, err)
+
+	// Wait for process
+	waitProcess(&pconfig, t)
+
+	outputEnv := string(stdout.Bytes())
+
+	// Check that the environment has the key/value pair we added
+	if !strings.Contains(outputEnv, "FOO=BAR") {
+		t.Fatal("Environment doesn't have the expected FOO=BAR key/value pair: ", outputEnv)
+	}
+
+	// Make sure that HOME is set
+	if !strings.Contains(outputEnv, "HOME=/root") {
+		t.Fatal("Environment doesn't have HOME set: ", outputEnv)
+	}
+}
+
+func TestProcessCaps(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	root, err := newTestRoot()
+	ok(t, err)
+	defer os.RemoveAll(root)
+
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+
+	container, err := factory.Create("test", config)
+	ok(t, err)
+	defer container.Destroy()
+
+	processCaps := append(config.Capabilities, "NET_ADMIN")
+
+	var stdout bytes.Buffer
+	pconfig := libcontainer.Process{
+		Args:         []string{"sh", "-c", "cat /proc/self/status"},
+		Env:          standardEnvironment,
+		Capabilities: processCaps,
+		Stdin:        nil,
+		Stdout:       &stdout,
+	}
+	err = container.Start(&pconfig)
+	ok(t, err)
+
+	// Wait for process
+	waitProcess(&pconfig, t)
+
+	outputStatus := string(stdout.Bytes())
+
+	lines := strings.Split(outputStatus, "\n")
+
+	effectiveCapsLine := ""
+	for _, l := range lines {
+		line := strings.TrimSpace(l)
+		if strings.Contains(line, "CapEff:") {
+			effectiveCapsLine = line
+			break
+		}
+	}
+
+	if effectiveCapsLine == "" {
+		t.Fatal("Couldn't find effective caps: ", outputStatus)
+	}
+
+	parts := strings.Split(effectiveCapsLine, ":")
+	effectiveCapsStr := strings.TrimSpace(parts[1])
+
+	effectiveCaps, err := strconv.ParseUint(effectiveCapsStr, 16, 64)
+	if err != nil {
+		t.Fatal("Could not parse effective caps", err)
+	}
+
+	var netAdminMask uint64
+	var netAdminBit uint
+	netAdminBit = 12 // from capability.h
+	netAdminMask = 1 << netAdminBit
+	if effectiveCaps&netAdminMask != netAdminMask {
+		t.Fatal("CAP_NET_ADMIN is not set as expected")
+	}
+}
+
+func TestAdditionalGroups(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	root, err := newTestRoot()
+	ok(t, err)
+	defer os.RemoveAll(root)
+
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+	config.AdditionalGroups = []string{"plugdev", "audio"}
+
+	factory, err := libcontainer.New(root, libcontainer.Cgroupfs)
+	ok(t, err)
+
+	container, err := factory.Create("test", config)
+	ok(t, err)
+	defer container.Destroy()
+
+	var stdout bytes.Buffer
+	pconfig := libcontainer.Process{
+		Args:   []string{"sh", "-c", "id", "-Gn"},
+		Env:    standardEnvironment,
+		Stdin:  nil,
+		Stdout: &stdout,
+	}
+	err = container.Start(&pconfig)
+	ok(t, err)
+
+	// Wait for process
+	waitProcess(&pconfig, t)
+
+	outputGroups := string(stdout.Bytes())
+
+	// Check that the groups output has the groups that we specified
+	if !strings.Contains(outputGroups, "audio") {
+		t.Fatalf("Listed groups do not contain the audio group as expected: %v", outputGroups)
+	}
+
+	if !strings.Contains(outputGroups, "plugdev") {
+		t.Fatalf("Listed groups do not contain the plugdev group as expected: %v", outputGroups)
+	}
+}
+
+func TestFreeze(t *testing.T) {
+	testFreeze(t, false)
+}
+
+func TestSystemdFreeze(t *testing.T) {
+	if !systemd.UseSystemd() {
+		t.Skip("Systemd is unsupported")
+	}
+	testFreeze(t, true)
+}
+
+func testFreeze(t *testing.T, systemd bool) {
+	if testing.Short() {
+		return
+	}
+	root, err := newTestRoot()
+	ok(t, err)
+	defer os.RemoveAll(root)
+
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+	f := factory
+	if systemd {
+		f = systemdFactory
+	}
+
+	container, err := f.Create("test", config)
+	ok(t, err)
+	defer container.Destroy()
+
+	stdinR, stdinW, err := os.Pipe()
+	ok(t, err)
+
+	pconfig := &libcontainer.Process{
+		Args:  []string{"cat"},
+		Env:   standardEnvironment,
+		Stdin: stdinR,
+	}
+	err = container.Start(pconfig)
+	stdinR.Close()
+	defer stdinW.Close()
+	ok(t, err)
+
+	err = container.Pause()
+	ok(t, err)
+	state, err := container.Status()
+	ok(t, err)
+	err = container.Resume()
+	ok(t, err)
+	if state != libcontainer.Paused {
+		t.Fatal("Unexpected state: ", state)
+	}
+
+	stdinW.Close()
+	waitProcess(pconfig, t)
+}
+
+func TestCpuShares(t *testing.T) {
+	testCpuShares(t, false)
+}
+
+func TestCpuSharesSystemd(t *testing.T) {
+	if !systemd.UseSystemd() {
+		t.Skip("Systemd is unsupported")
+	}
+	testCpuShares(t, true)
+}
+
+func testCpuShares(t *testing.T, systemd bool) {
+	if testing.Short() {
+		return
+	}
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+	if systemd {
+		config.Cgroups.Slice = "system.slice"
+	}
+	config.Cgroups.CpuShares = 1
+
+	_, _, err = runContainer(config, "", "ps")
+	if err == nil {
+		t.Fatalf("runContainer should failed with invalid CpuShares")
+	}
+}
+
+func TestRunWithKernelMemory(t *testing.T) {
+	testRunWithKernelMemory(t, false)
+}
+
+func TestRunWithKernelMemorySystemd(t *testing.T) {
+	if !systemd.UseSystemd() {
+		t.Skip("Systemd is unsupported")
+	}
+	testRunWithKernelMemory(t, true)
+}
+
+func testRunWithKernelMemory(t *testing.T, systemd bool) {
+	if testing.Short() {
+		return
+	}
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+	if systemd {
+		config.Cgroups.Slice = "system.slice"
+	}
+	config.Cgroups.KernelMemory = 52428800
+
+	_, _, err = runContainer(config, "", "ps")
+	if err != nil {
+		t.Fatalf("runContainer failed with kernel memory limit: %v", err)
+	}
+}
+
+func TestContainerState(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	root, err := newTestRoot()
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.RemoveAll(root)
+
+	rootfs, err := newRootfs()
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer remove(rootfs)
+
+	l, err := os.Readlink("/proc/1/ns/ipc")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	config := newTemplateConfig(rootfs)
+	config.Namespaces = configs.Namespaces([]configs.Namespace{
+		{Type: configs.NEWNS},
+		{Type: configs.NEWUTS},
+		// host for IPC
+		//{Type: configs.NEWIPC},
+		{Type: configs.NEWPID},
+		{Type: configs.NEWNET},
+	})
+
+	container, err := factory.Create("test", config)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer container.Destroy()
+
+	stdinR, stdinW, err := os.Pipe()
+	if err != nil {
+		t.Fatal(err)
+	}
+	p := &libcontainer.Process{
+		Args:  []string{"cat"},
+		Env:   standardEnvironment,
+		Stdin: stdinR,
+	}
+	err = container.Start(p)
+	if err != nil {
+		t.Fatal(err)
+	}
+	stdinR.Close()
+	defer stdinW.Close()
+
+	st, err := container.State()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	l1, err := os.Readlink(st.NamespacePaths[configs.NEWIPC])
+	if err != nil {
+		t.Fatal(err)
+	}
+	if l1 != l {
+		t.Fatal("Container using non-host ipc namespace")
+	}
+	stdinW.Close()
+	waitProcess(p, t)
+}
+
+func TestPassExtraFiles(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+
+	rootfs, err := newRootfs()
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+
+	container, err := factory.Create("test", config)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer container.Destroy()
+
+	var stdout bytes.Buffer
+	pipeout1, pipein1, err := os.Pipe()
+	pipeout2, pipein2, err := os.Pipe()
+	process := libcontainer.Process{
+		Args:       []string{"sh", "-c", "cd /proc/$$/fd; echo -n *; echo -n 1 >3; echo -n 2 >4"},
+		Env:        []string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"},
+		ExtraFiles: []*os.File{pipein1, pipein2},
+		Stdin:      nil,
+		Stdout:     &stdout,
+	}
+	err = container.Start(&process)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	waitProcess(&process, t)
+
+	out := string(stdout.Bytes())
+	// fd 5 is the directory handle for /proc/$$/fd
+	if out != "0 1 2 3 4 5" {
+		t.Fatalf("expected to have the file descriptors '0 1 2 3 4 5' passed to init, got '%s'", out)
+	}
+	var buf = []byte{0}
+	_, err = pipeout1.Read(buf)
+	if err != nil {
+		t.Fatal(err)
+	}
+	out1 := string(buf)
+	if out1 != "1" {
+		t.Fatalf("expected first pipe to receive '1', got '%s'", out1)
+	}
+
+	_, err = pipeout2.Read(buf)
+	if err != nil {
+		t.Fatal(err)
+	}
+	out2 := string(buf)
+	if out2 != "2" {
+		t.Fatalf("expected second pipe to receive '2', got '%s'", out2)
+	}
+}
+
+func TestMountCmds(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	root, err := newTestRoot()
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.RemoveAll(root)
+
+	rootfs, err := newRootfs()
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer remove(rootfs)
+
+	tmpDir, err := ioutil.TempDir("", "tmpdir")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.RemoveAll(tmpDir)
+
+	config := newTemplateConfig(rootfs)
+	config.Mounts = append(config.Mounts, &configs.Mount{
+		Source:      tmpDir,
+		Destination: "/tmp",
+		Device:      "bind",
+		Flags:       syscall.MS_BIND | syscall.MS_REC,
+		PremountCmds: []configs.Command{
+			{Path: "touch", Args: []string{filepath.Join(tmpDir, "hello")}},
+			{Path: "touch", Args: []string{filepath.Join(tmpDir, "world")}},
+		},
+		PostmountCmds: []configs.Command{
+			{Path: "cp", Args: []string{filepath.Join(rootfs, "tmp", "hello"), filepath.Join(rootfs, "tmp", "hello-backup")}},
+			{Path: "cp", Args: []string{filepath.Join(rootfs, "tmp", "world"), filepath.Join(rootfs, "tmp", "world-backup")}},
+		},
+	})
+
+	container, err := factory.Create("test", config)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer container.Destroy()
+
+	pconfig := libcontainer.Process{
+		Args: []string{"sh", "-c", "env"},
+		Env:  standardEnvironment,
+	}
+	err = container.Start(&pconfig)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// Wait for process
+	waitProcess(&pconfig, t)
+
+	entries, err := ioutil.ReadDir(tmpDir)
+	if err != nil {
+		t.Fatal(err)
+	}
+	expected := []string{"hello", "hello-backup", "world", "world-backup"}
+	for i, e := range entries {
+		if e.Name() != expected[i] {
+			t.Errorf("Got(%s), expect %s", e.Name(), expected[i])
+		}
+	}
+}
+
+func TestSystemProperties(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	root, err := newTestRoot()
+	ok(t, err)
+	defer os.RemoveAll(root)
+
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+
+	config := newTemplateConfig(rootfs)
+	config.SystemProperties = map[string]string{
+		"kernel.shmmni": "8192",
+	}
+
+	container, err := factory.Create("test", config)
+	ok(t, err)
+	defer container.Destroy()
+
+	var stdout bytes.Buffer
+	pconfig := libcontainer.Process{
+		Args:   []string{"sh", "-c", "cat /proc/sys/kernel/shmmni"},
+		Env:    standardEnvironment,
+		Stdin:  nil,
+		Stdout: &stdout,
+	}
+	err = container.Start(&pconfig)
+	ok(t, err)
+
+	// Wait for process
+	waitProcess(&pconfig, t)
+
+	shmmniOutput := strings.TrimSpace(string(stdout.Bytes()))
+	if shmmniOutput != "8192" {
+		t.Fatalf("kernel.shmmni property expected to be 8192, but is %s", shmmniOutput)
+	}
+}
+
+func TestSeccompNoChown(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	rootfs, err := newRootfs()
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer remove(rootfs)
+	config := newTemplateConfig(rootfs)
+	config.Seccomp = &configs.Seccomp{}
+	config.Seccomp.Syscalls = append(config.Seccomp.Syscalls, &configs.Syscall{
+		Value:  syscall.SYS_CHOWN,
+		Action: configs.Action(syscall.EPERM),
+	})
+	buffers, _, err := runContainer(config, "", "/bin/sh", "-c", "chown 1:1 /tmp")
+	if err == nil {
+		t.Fatal("running chown in a container should fail")
+	}
+	if s := buffers.String(); !strings.Contains(s, "not permitted") {
+		t.Fatalf("running chown should result in an EPERM but got %q", s)
+	}
+}

libcontainer/integration/execin_test.go

@@ -0,0 +1,334 @@
+package integration
+
+import (
+	"bytes"
+	"io"
+	"os"
+	"strings"
+	"testing"
+	"time"
+
+	"github.com/docker/libcontainer"
+)
+
+func TestExecIn(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+	config := newTemplateConfig(rootfs)
+	container, err := newContainer(config)
+	ok(t, err)
+	defer container.Destroy()
+
+	// Execute a first process in the container
+	stdinR, stdinW, err := os.Pipe()
+	ok(t, err)
+	process := &libcontainer.Process{
+		Args:  []string{"cat"},
+		Env:   standardEnvironment,
+		Stdin: stdinR,
+	}
+	err = container.Start(process)
+	stdinR.Close()
+	defer stdinW.Close()
+	ok(t, err)
+
+	buffers := newStdBuffers()
+	ps := &libcontainer.Process{
+		Args:   []string{"ps"},
+		Env:    standardEnvironment,
+		Stdin:  buffers.Stdin,
+		Stdout: buffers.Stdout,
+		Stderr: buffers.Stderr,
+	}
+
+	err = container.Start(ps)
+	ok(t, err)
+	waitProcess(ps, t)
+	stdinW.Close()
+	waitProcess(process, t)
+
+	out := buffers.Stdout.String()
+	if !strings.Contains(out, "cat") || !strings.Contains(out, "ps") {
+		t.Fatalf("unexpected running process, output %q", out)
+	}
+}
+
+func TestExecInRlimit(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+	config := newTemplateConfig(rootfs)
+	container, err := newContainer(config)
+	ok(t, err)
+	defer container.Destroy()
+
+	stdinR, stdinW, err := os.Pipe()
+	ok(t, err)
+	process := &libcontainer.Process{
+		Args:  []string{"cat"},
+		Env:   standardEnvironment,
+		Stdin: stdinR,
+	}
+	err = container.Start(process)
+	stdinR.Close()
+	defer stdinW.Close()
+	ok(t, err)
+
+	buffers := newStdBuffers()
+	ps := &libcontainer.Process{
+		Args:   []string{"/bin/sh", "-c", "ulimit -n"},
+		Env:    standardEnvironment,
+		Stdin:  buffers.Stdin,
+		Stdout: buffers.Stdout,
+		Stderr: buffers.Stderr,
+	}
+	err = container.Start(ps)
+	ok(t, err)
+	waitProcess(ps, t)
+
+	stdinW.Close()
+	waitProcess(process, t)
+
+	out := buffers.Stdout.String()
+	if limit := strings.TrimSpace(out); limit != "1025" {
+		t.Fatalf("expected rlimit to be 1025, got %s", limit)
+	}
+}
+
+func TestExecInError(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+	config := newTemplateConfig(rootfs)
+	container, err := newContainer(config)
+	ok(t, err)
+	defer container.Destroy()
+
+	// Execute a first process in the container
+	stdinR, stdinW, err := os.Pipe()
+	ok(t, err)
+	process := &libcontainer.Process{
+		Args:  []string{"cat"},
+		Env:   standardEnvironment,
+		Stdin: stdinR,
+	}
+	err = container.Start(process)
+	stdinR.Close()
+	defer func() {
+		stdinW.Close()
+		if _, err := process.Wait(); err != nil {
+			t.Log(err)
+		}
+	}()
+	ok(t, err)
+
+	unexistent := &libcontainer.Process{
+		Args: []string{"unexistent"},
+		Env:  standardEnvironment,
+	}
+	err = container.Start(unexistent)
+	if err == nil {
+		t.Fatal("Should be an error")
+	}
+	if !strings.Contains(err.Error(), "executable file not found") {
+		t.Fatalf("Should be error about not found executable, got %s", err)
+	}
+}
+
+func TestExecInTTY(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+	config := newTemplateConfig(rootfs)
+	container, err := newContainer(config)
+	ok(t, err)
+	defer container.Destroy()
+
+	// Execute a first process in the container
+	stdinR, stdinW, err := os.Pipe()
+	ok(t, err)
+	process := &libcontainer.Process{
+		Args:  []string{"cat"},
+		Env:   standardEnvironment,
+		Stdin: stdinR,
+	}
+	err = container.Start(process)
+	stdinR.Close()
+	defer stdinW.Close()
+	ok(t, err)
+
+	var stdout bytes.Buffer
+	ps := &libcontainer.Process{
+		Args: []string{"ps"},
+		Env:  standardEnvironment,
+	}
+	console, err := ps.NewConsole(0)
+	copy := make(chan struct{})
+	go func() {
+		io.Copy(&stdout, console)
+		close(copy)
+	}()
+	ok(t, err)
+	err = container.Start(ps)
+	ok(t, err)
+	select {
+	case <-time.After(5 * time.Second):
+		t.Fatal("Waiting for copy timed out")
+	case <-copy:
+	}
+	waitProcess(ps, t)
+
+	stdinW.Close()
+	waitProcess(process, t)
+
+	out := stdout.String()
+	if !strings.Contains(out, "cat") || !strings.Contains(string(out), "ps") {
+		t.Fatalf("unexpected running process, output %q", out)
+	}
+}
+
+func TestExecInEnvironment(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	rootfs, err := newRootfs()
+	ok(t, err)
+	defer remove(rootfs)
+	config := newTemplateConfig(rootfs)
+	container, err := newContainer(config)
+	ok(t, err)
+	defer container.Destroy()
+
+	// Execute a first process in the container
+	stdinR, stdinW, err := os.Pipe()
+	ok(t, err)
+	process := &libcontainer.Process{
+		Args:  []string{"cat"},
+		Env:   standardEnvironment,
+		Stdin: stdinR,
+	}
+	err = container.Start(process)
+	stdinR.Close()
+	defer stdinW.Close()
+	ok(t, err)
+
+	buffers := newStdBuffers()
+	process2 := &libcontainer.Process{
+		Args: []string{"env"},
+		Env: []string{
+			"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
+			"DEBUG=true",
+			"DEBUG=false",
+			"ENV=test",
+		},
+		Stdin:  buffers.Stdin,
+		Stdout: buffers.Stdout,
+		Stderr: buffers.Stderr,
+	}
+	err = container.Start(process2)
+	ok(t, err)
+	waitProcess(process2, t)
+
+	stdinW.Close()
+	waitProcess(process, t)
+
+	out := buffers.Stdout.String()
+	// check execin's process environment
+	if !strings.Contains(out, "DEBUG=false") ||
+		!strings.Contains(out, "ENV=test") ||
+		!strings.Contains(out, "HOME=/root") ||
+		!strings.Contains(out, "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin") ||
+		strings.Contains(out, "DEBUG=true") {
+		t.Fatalf("unexpected running process, output %q", out)
+	}
+}
+
+func TestExecinPassExtraFiles(t *testing.T) {
+	if testing.Short() {
+		return
+	}
+	rootfs, err := newRootfs()
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer remove(rootfs)
+	config := newTemplateConfig(rootfs)
+	container, err := newContainer(config)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer container.Destroy()
+
+	// Execute a first process in the container
+	stdinR, stdinW, err := os.Pipe()
+	if err != nil {
+		t.Fatal(err)
+	}
+	process := &libcontainer.Process{
+		Args:  []string{"cat"},
+		Env:   standardEnvironment,
+		Stdin: stdinR,
+	}
+	err = container.Start(process)
+	stdinR.Close()
+	defer stdinW.Close()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	var stdout bytes.Buffer
+	pipeout1, pipein1, err := os.Pipe()
+	pipeout2, pipein2, err := os.Pipe()
+	inprocess := &libcontainer.Process{
+		Args:       []string{"sh", "-c", "cd /proc/$$/fd; echo -n *; echo -n 1 >3; echo -n 2 >4"},
+		Env:        []string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"},
+		ExtraFiles: []*os.File{pipein1, pipein2},
+		Stdin:      nil,
+		Stdout:     &stdout,
+	}
+	err = container.Start(inprocess)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	waitProcess(inprocess, t)
+	stdinW.Close()
+	waitProcess(process, t)
+
+	out := string(stdout.Bytes())
+	// fd 5 is the directory handle for /proc/$$/fd
+	if out != "0 1 2 3 4 5" {
+		t.Fatalf("expected to have the file descriptors '0 1 2 3 4 5' passed to exec, got '%s'", out)
+	}
+	var buf = []byte{0}
+	_, err = pipeout1.Read(buf)
+	if err != nil {
+		t.Fatal(err)
+	}
+	out1 := string(buf)
+	if out1 != "1" {
+		t.Fatalf("expected first pipe to receive '1', got '%s'", out1)
+	}
+
+	_, err = pipeout2.Read(buf)
+	if err != nil {
+		t.Fatal(err)
+	}
+	out2 := string(buf)
+	if out2 != "2" {
+		t.Fatalf("expected second pipe to receive '2', got '%s'", out2)
+	}
+}

libcontainer/integration/init_test.go

@@ -0,0 +1,60 @@
+package integration
+
+import (
+	"os"
+	"runtime"
+	"testing"
+
+	"github.com/Sirupsen/logrus"
+	"github.com/docker/libcontainer"
+	"github.com/docker/libcontainer/cgroups/systemd"
+	_ "github.com/docker/libcontainer/nsenter"
+)
+
+// init runs the libcontainer initialization code because of the busybox style needs
+// to work around the go runtime and the issues with forking
+func init() {
+	if len(os.Args) < 2 || os.Args[1] != "init" {
+		return
+	}
+	runtime.GOMAXPROCS(1)
+	runtime.LockOSThread()
+	factory, err := libcontainer.New("")
+	if err != nil {
+		logrus.Fatalf("unable to initialize for container: %s", err)
+	}
+	if err := factory.StartInitialization(); err != nil {
+		logrus.Fatal(err)
+	}
+}
+
+var (
+	factory        libcontainer.Factory
+	systemdFactory libcontainer.Factory
+)
+
+func TestMain(m *testing.M) {
+	var (
+		err error
+		ret int = 0
+	)
+
+	logrus.SetOutput(os.Stderr)
+	logrus.SetLevel(logrus.InfoLevel)
+
+	factory, err = libcontainer.New(".", libcontainer.Cgroupfs)
+	if err != nil {
+		logrus.Error(err)
+		os.Exit(1)
+	}
+	if systemd.UseSystemd() {
+		systemdFactory, err = libcontainer.New(".", libcontainer.SystemdCgroups)
+		if err != nil {
+			logrus.Error(err)
+			os.Exit(1)
+		}
+	}
+
+	ret = m.Run()
+	os.Exit(ret)
+}

libcontainer/integration/template_test.go

@@ -0,0 +1,118 @@
+package integration
+
+import (
+	"syscall"
+
+	"github.com/docker/libcontainer/configs"
+)
+
+var standardEnvironment = []string{
+	"HOME=/root",
+	"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
+	"HOSTNAME=integration",
+	"TERM=xterm",
+}
+
+const defaultMountFlags = syscall.MS_NOEXEC | syscall.MS_NOSUID | syscall.MS_NODEV
+
+// newTemplateConfig returns a base template for running a container
+//
+// it uses a network strategy of just setting a loopback interface
+// and the default setup for devices
+func newTemplateConfig(rootfs string) *configs.Config {
+	return &configs.Config{
+		Rootfs: rootfs,
+		Capabilities: []string{
+			"CHOWN",
+			"DAC_OVERRIDE",
+			"FSETID",
+			"FOWNER",
+			"MKNOD",
+			"NET_RAW",
+			"SETGID",
+			"SETUID",
+			"SETFCAP",
+			"SETPCAP",
+			"NET_BIND_SERVICE",
+			"SYS_CHROOT",
+			"KILL",
+			"AUDIT_WRITE",
+		},
+		Namespaces: configs.Namespaces([]configs.Namespace{
+			{Type: configs.NEWNS},
+			{Type: configs.NEWUTS},
+			{Type: configs.NEWIPC},
+			{Type: configs.NEWPID},
+			{Type: configs.NEWNET},
+		}),
+		Cgroups: &configs.Cgroup{
+			Name:            "test",
+			Parent:          "integration",
+			AllowAllDevices: false,
+			AllowedDevices:  configs.DefaultAllowedDevices,
+		},
+		MaskPaths: []string{
+			"/proc/kcore",
+		},
+		ReadonlyPaths: []string{
+			"/proc/sys", "/proc/sysrq-trigger", "/proc/irq", "/proc/bus",
+		},
+		Devices:  configs.DefaultAutoCreatedDevices,
+		Hostname: "integration",
+		Mounts: []*configs.Mount{
+			{
+				Source:      "proc",
+				Destination: "/proc",
+				Device:      "proc",
+				Flags:       defaultMountFlags,
+			},
+			{
+				Source:      "tmpfs",
+				Destination: "/dev",
+				Device:      "tmpfs",
+				Flags:       syscall.MS_NOSUID | syscall.MS_STRICTATIME,
+				Data:        "mode=755",
+			},
+			{
+				Source:      "devpts",
+				Destination: "/dev/pts",
+				Device:      "devpts",
+				Flags:       syscall.MS_NOSUID | syscall.MS_NOEXEC,
+				Data:        "newinstance,ptmxmode=0666,mode=0620,gid=5",
+			},
+			{
+				Device:      "tmpfs",
+				Source:      "shm",
+				Destination: "/dev/shm",
+				Data:        "mode=1777,size=65536k",
+				Flags:       defaultMountFlags,
+			},
+			{
+				Source:      "mqueue",
+				Destination: "/dev/mqueue",
+				Device:      "mqueue",
+				Flags:       defaultMountFlags,
+			},
+			{
+				Source:      "sysfs",
+				Destination: "/sys",
+				Device:      "sysfs",
+				Flags:       defaultMountFlags | syscall.MS_RDONLY,
+			},
+		},
+		Networks: []*configs.Network{
+			{
+				Type:    "loopback",
+				Address: "127.0.0.1/0",
+				Gateway: "localhost",
+			},
+		},
+		Rlimits: []configs.Rlimit{
+			{
+				Type: syscall.RLIMIT_NOFILE,
+				Hard: uint64(1025),
+				Soft: uint64(1025),
+			},
+		},
+	}
+}

libcontainer/integration/utils_test.go

@@ -0,0 +1,140 @@
+package integration
+
+import (
+	"bytes"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"runtime"
+	"strings"
+	"syscall"
+	"testing"
+
+	"github.com/docker/libcontainer"
+	"github.com/docker/libcontainer/configs"
+)
+
+func newStdBuffers() *stdBuffers {
+	return &stdBuffers{
+		Stdin:  bytes.NewBuffer(nil),
+		Stdout: bytes.NewBuffer(nil),
+		Stderr: bytes.NewBuffer(nil),
+	}
+}
+
+type stdBuffers struct {
+	Stdin  *bytes.Buffer
+	Stdout *bytes.Buffer
+	Stderr *bytes.Buffer
+}
+
+func (b *stdBuffers) String() string {
+	s := []string{}
+	if b.Stderr != nil {
+		s = append(s, b.Stderr.String())
+	}
+	if b.Stdout != nil {
+		s = append(s, b.Stdout.String())
+	}
+	return strings.Join(s, "|")
+}
+
+// ok fails the test if an err is not nil.
+func ok(t testing.TB, err error) {
+	if err != nil {
+		_, file, line, _ := runtime.Caller(1)
+		t.Fatalf("%s:%d: unexpected error: %s\n\n", filepath.Base(file), line, err.Error())
+	}
+}
+
+func waitProcess(p *libcontainer.Process, t *testing.T) {
+	_, file, line, _ := runtime.Caller(1)
+	status, err := p.Wait()
+
+	if err != nil {
+		t.Fatalf("%s:%d: unexpected error: %s\n\n", filepath.Base(file), line, err.Error())
+	}
+
+	if !status.Success() {
+		t.Fatalf("%s:%d: unexpected status: %s\n\n", filepath.Base(file), line, status.String())
+	}
+}
+
+// newRootfs creates a new tmp directory and copies the busybox root filesystem
+func newRootfs() (string, error) {
+	dir, err := ioutil.TempDir("", "")
+	if err != nil {
+		return "", err
+	}
+	if err := os.MkdirAll(dir, 0700); err != nil {
+		return "", err
+	}
+	if err := copyBusybox(dir); err != nil {
+		return "", nil
+	}
+	return dir, nil
+}
+
+func remove(dir string) {
+	os.RemoveAll(dir)
+}
+
+// copyBusybox copies the rootfs for a busybox container created for the test image
+// into the new directory for the specific test
+func copyBusybox(dest string) error {
+	out, err := exec.Command("sh", "-c", fmt.Sprintf("cp -R /busybox/* %s/", dest)).CombinedOutput()
+	if err != nil {
+		return fmt.Errorf("copy error %q: %q", err, out)
+	}
+	return nil
+}
+
+func newContainer(config *configs.Config) (libcontainer.Container, error) {
+	f := factory
+
+	if config.Cgroups != nil && config.Cgroups.Slice == "system.slice" {
+		f = systemdFactory
+	}
+
+	return f.Create("testCT", config)
+}
+
+// runContainer runs the container with the specific config and arguments
+//
+// buffers are returned containing the STDOUT and STDERR output for the run
+// along with the exit code and any go error
+func runContainer(config *configs.Config, console string, args ...string) (buffers *stdBuffers, exitCode int, err error) {
+	container, err := newContainer(config)
+	if err != nil {
+		return nil, -1, err
+	}
+	defer container.Destroy()
+	buffers = newStdBuffers()
+	process := &libcontainer.Process{
+		Args:   args,
+		Env:    standardEnvironment,
+		Stdin:  buffers.Stdin,
+		Stdout: buffers.Stdout,
+		Stderr: buffers.Stderr,
+	}
+
+	err = container.Start(process)
+	if err != nil {
+		return buffers, -1, err
+	}
+	ps, err := process.Wait()
+	if err != nil {
+		return buffers, -1, err
+	}
+	status := ps.Sys().(syscall.WaitStatus)
+	if status.Exited() {
+		exitCode = status.ExitStatus()
+	} else if status.Signaled() {
+		exitCode = -int(status.Signal())
+	} else {
+		return buffers, -1, err
+	}
+	return
+}

libcontainer/label/label.go

@@ -0,0 +1,61 @@
+// +build !selinux !linux
+
+package label
+
+// InitLabels returns the process label and file labels to be used within
+// the container.  A list of options can be passed into this function to alter
+// the labels.
+func InitLabels(options []string) (string, string, error) {
+	return "", "", nil
+}
+
+func GenLabels(options string) (string, string, error) {
+	return "", "", nil
+}
+
+func FormatMountLabel(src string, mountLabel string) string {
+	return src
+}
+
+func SetProcessLabel(processLabel string) error {
+	return nil
+}
+
+func SetFileLabel(path string, fileLabel string) error {
+	return nil
+}
+
+func SetFileCreateLabel(fileLabel string) error {
+	return nil
+}
+
+func Relabel(path string, fileLabel string, relabel string) error {
+	return nil
+}
+
+func GetPidLabel(pid int) (string, error) {
+	return "", nil
+}
+
+func Init() {
+}
+
+func ReserveLabel(label string) error {
+	return nil
+}
+
+func UnreserveLabel(label string) error {
+	return nil
+}
+
+// DupSecOpt takes an process label and returns security options that
+// can be used to set duplicate labels on future container processes
+func DupSecOpt(src string) []string {
+	return nil
+}
+
+// DisableSecOpt returns a security opt that can disable labeling
+// support for future container processes
+func DisableSecOpt() []string {
+	return nil
+}