NUBOMEDIA: FP7/2007-2013 GA-610576

Docker: container virtualization and its impact in Nubomedia

  • Docker

Docker is an open-source project that automates the deployment of applications inside self-sufficient software containers. It is mainly targeted at developers and system administrators to build, ship, and run distributed applications. However, it can also be used in production systems requiring optimal utilization of hardware resources. Docker containers are independent of hardware, language, framework, and hosting provider. To achieve its goals, Docker is using kernel namespacing, cgroups, and LXC combined with a high level API. It provides a way to automate software development in a secure and repeatable environment. Hence, Docker is a way to manage LXC containers on a single machine.

Containers existed before Docker, but they were not standardized and not easy to use. The aim of Docker is to facilitate the use of containers and to make their use convenient on any platform. At this moment, Docker can run on any x86 Linux Kernel that supports cgroups and aufs, and it aims for full OpenStack compatibility.

In an architectural perspective, traditional virtual machines run on physical hardware via an intermediation layer, while containers run inside user space on top of an operating system kernel (currently Linux). This allows running multiple isolated user spaces on a single host. In addition, the elimination of hardware virtualization layers and the direct execution onto the metal provides higher performance.
Docker features include

  • Higher start up speeds (containers are started in milliseconds)
  • Developers focus on code and not on operations
  • Portable deployment across machines
  • Component re-use
  • Versioning of builds
  • Application centric
  • Improved performance

Docker architecture is based on the following components

  • Docker client and server. Docker is a client-server application where client is talking to the server which runs the containers.
  • Docker images. Containers are started from images, and they are the starting source code on top of which the container will be built. They use Union file system.
  • Registries. Docker stores the images that were built on registries. Registries can be public or private. The company behind Docker operates a service to store images called DockerHub. Furthermore, private registries can be stored freely for your organization.
  • Containers. Docker helps you build and deploy containers inside which you can package your applications and services. Containers are launched from images and can contain one or more processes. A container can execute any piece of software, ranging from a web server to a NoSQL database.

Around Docker, an open source software ecosystem is currently growing where many interesting tools are emerging. Among them we can find:

  • CoreOS, which is a Linux distribution that uses Linux containers (Docker) to manage services and clusters of servers. Cloud providers like Google and Digital Ocean have announced official support for CoreOS images.
  • Atomic Project, which is sponsored by RedHat is similar with CoreOS, but it is mostly aimed for enterprise users, and oriented around RedHat ecosystem (RedHat Enterprise Liunx, Fedora, CentOS). It provides an end-to-end solution around Docker and applications.
  • Kubernetes, in which Google has open sourced its tools for Docker cluster management. Companies like Microsoft, RedHat, IBM have already announced their support for this open container framework, and they will work closely to support common tools and avoid vendor lock-in.

Docker is currently being issued in NUBOMEDIA for Continuous Integration (CI) purposes. We use a Jenkins plugin named Docker plugin, that aims to provide Jenkins capability to use a Docker host to dynamically provision a slave, run a single build, then tear-down that slave. We configured a Jenkins slave node that hosts all Docker containers, and we created separate jobs to do nightly build images with Docker for each running environment needed in the CI system. When these jobs are done, fresh images are uploaded to Jenkins Docker machine, and new slave nodes with labels are added to the Jenkins master. The advantage of using this architecture is that Jenkins can run jobs on fresh and isolated Docker containers without installing any packages or changing configurations on a live Jenkins node.

In addition, Docker can be used in Nubomedia as a hypervisor driver for OpenStack, and instances can be deployed on Docker Nova Driver instead of Xen or KVM. Docker will fetch images from OpenStack Image service (Glance), and load them onto Docker file system. In the future, if OpenStack provides direct support for Docker containers,NUBOMEDIA could take advantage by eliminating the need of hardware virtualization layers so that all hardware resources would be available for media processing and transport.