Documentation for the Redock API

Main Redock API

The redock.api module defines two classes and two exception types:

class redock.api.Container(image, hostname=None, timeout=10)

Container is the main entry point to the Redock API. It aims to provide a simple to use representation of Docker containers (and in extension Docker images). You’ll probably never need most of the methods defined in this class; if you’re getting started with Redock you should focus on these methods:

After you create and start a container with Redock you can do with the container what you want by starting out with an SSH connection. When you’re done you either save your changes or discard them and kill the container. That’s probably all you need from Redock :-)

__init__(image, hostname=None, timeout=10)

Initialize a Container from the given arguments.

Parameters:
  • image – The repository and tag of the container’s image (in the format expected by Image.coerce()).
  • hostname – The host name to use inside the container. If none is given, the image’s tag is used.
  • timeout – The timeout in seconds while waiting for a container to become reachable over SSH (a couple of seconds should be plenty).
check_active()

Check if the Container is associated with a running Docker container. If no running Docker container is found, NoContainerRunning is raised.

commit(message=None, author=None)

Commit any changes to the running container to the associated image. Corresponds to the docker commit command.

Raises NoContainerRunning if an associated Docker container is not already running.

Parameters:
  • message – A short message describing the commit (a string).
  • author – The name of the author (a string).
delete()

Delete the image associated with the container (if any). The data in the image will be lost.

expand_id(short_id, candidate_ids)

docker.Client.create_container() and docker.Client.commit() report short ids (12 characters) while docker.Client.containers() and docker.Client.images() report long ids (65 characters). I’d rather use the full ids where possible. This method translates short ids into long ids at the expense of an additional API call (who cares).

Raises exceptions.Exception if no long id corresponding to the short id can be matched (this might well be a purely theoretical problem, it certainly shouldn’t happen during regular use).

Parameters:
  • short_id – A short id of 12 characters.
  • candidate_ids – A list of available long ids.
Returns:

The long id corresponding to the given short id.
find_container()

Check to see if the current Container has an associated Docker container that is currently running.

Returns:True when a running container exists, False otherwise.
find_image(image_to_find)

Find the most recent Docker image with the given repository and tag.

Parameters:image_to_find – The Image we’re looking for.
Returns:The most recent Image available, or None if no images were matched.
get_ssh_client_command(ip_address=None, port_number=None)

Generate an SSH client command line that connects to the container (assumed to be running).

Parameters:
  • ip_address – This optional argument overrides the default IP address (which is otherwise automatically discovered).
  • port_number – This optional argument overrides the default port number (which is otherwise automatically discovered).
Returns:

The SSH client command line as a list of strings containing the command and its arguments.
kill()

Kill and remove the container. All changes since the last time that Container.commit() was called will be lost.

revoke_ssh_access()

Remove the container’s SSH client configuration from ~/.ssh/config.

setup_ssh_access()

Update ~/.ssh/config to make it easy to connect to the container over SSH from the host system. This generates a host definition to include in the SSH client configuration file and uses update-dotdee to merge the generated host definition with the user’s existing SSH client configuration file.

ssh_alias

Get the SSH alias that should be used to connect to the container.

ssh_config_file

Get the pathname of the SSH client configuration for the container.

ssh_endpoint

Wait for the container to become reachable over SSH and get a tuple with the IP address and port number that can be used to connect to the container over SSH.

start()

Create and start the Docker container. On the first run of Redock this creates a base image using redock.base.create_base_image().

start_supervisor()

Starts the container and runs Supervisor inside the container.

class redock.api.Image(repository, tag, id=None)

Simple representation of Docker images.

__init__(repository, tag, id=None)

Initialize an Image instance from the given arguments.

Parameters:
  • repository – The name of the image’s repository.
  • tag – The image’s tag (name).
  • id – The unique hash of the image (optional).
static coerce(value)

Coerce strings to Image objects.

Raises exceptions.ValueError when a string with an incorrect format is given.

Parameters:value – The name of the image, expected to be a string of the form repository:tag. If an Image object is given it is returned unmodified.
Returns:An Image object.
key

Get a tuple with the image’s repository and tag.

name

Get the human readable name of an Image as a string of the form repository:tag.

unique_name

Get the machine readable unique name of an Image. If the image has a unique hash that will be used, otherwise a string of the form repository:tag is returned.

exception redock.api.NoContainerRunning

Raised by Container.check_active() when a Container doesn’t have an associated Docker container running.

exception redock.api.SecureShellTimeout

Raised by Container.ssh_endpoint when Redock fails to connect to the Docker container within a reasonable amount of time (10 seconds by default).

class redock.api.Session

Dumb object to hold session variables associated with a running Docker container.

__init__()
reset()

Reset all known session variables to None.

Base image handling API

The redock.base module implements the initialization of the base image used by Redock. You’ll probably never need to use this module directly because redock.api.Container.start() calls find_base_image() and create_base_image() as needed.

redock.base.create_base_image(client)

Create the base image that’s used by Redock to create new containers. This base image differs from the ubuntu:precise image (on which it is based) on a couple of points:

  • Automatic installation of recommended packages is disabled to conserve disk space.

  • The Ubuntu package mirror is set to a geographically close location to speed up downloading of system packages (see redock.utils.select_ubuntu_mirror()).

  • The package list is updated to make sure apt-get installs the most up to date packages.

  • The following system packages are installed:

    language-pack-en-base

    In a base Docker Ubuntu 12.04 image lots of commands complain loudly about the locale. This silences the warnings by fixing the problem (if you want to call it that).

    openssh-server

    After creating a new container Redock will connect to it over SSH, so having an SSH server installed is a good start :-)

    supervisor

    The base Docker Ubuntu 12.04 image has init (upstart) disabled. Indeed we don’t need all of the bagage that comes with init but it is nice to have a process runner for the SSH server (and eventually maybe more).

  • The initscripts and upstart system packages are marked ‘on hold’ so that apt-get will not upgrade them. This makes it possible to run apt-get dist-upgrade inside containers.

  • An SSH key pair is generated and the SSH public key is installed inside the base image so that Redock can connect to the container over SSH (you need ssh-keygen installed).

  • Supervisor is configured to automatically start the SSH server.

Parameters:client – Connection to Docker (instance of docker.Client)
Returns:The unique id of the base image.
redock.base.download_image(client, repository, tag)

Download the requested image. If the image is already available locally it won’t be downloaded again.

Parameters:
  • client – Connection to Docker (instance of docker.Client)
  • repository – The name of the image’s repository.
  • tag – The name of the image’s tag.
redock.base.find_base_image(client)

Find the id of the base image that’s used by Redock to create new containers. If the image doesn’t exist yet it will be created using create_base_image().

Parameters:client – Connection to Docker (instance of docker.Client)
Returns:The unique id of the base image.
redock.base.find_named_image(client, repository, tag)

Find the most recent Docker image with the given repository and tag.

Parameters:
  • repository – The name of the image’s repository.
  • tag – The name of the image’s tag.
Returns:

The unique id of the most recent image available, or None if no images were matched.

Bootstrap configuration management system

Bootstrap is a minimal configuration management system. Right now it’s just a toy module that I may or may not use to extend Redock beyond the existing redock start, redock commit and redock kill functionality and commands. Here is the design rationale behind Bootstrap (in its current form):

Specialized towards Debian
Bootstrap is specialized towards Debian Linux (and its derivatives) because I have several years of hands on experience with Debian and Ubuntu Linux and because Docker currently gravitates to Ubuntu Linux (although this will probably change over time).
Based on SSH connections
SSH is used to connect to remote hosts because it’s the lowest common denominator that works with Docker, VirtualBox, XenServer and physical servers while being secure and easy to use.
Remote code execution using Python
The execnet package is used to execute Python code on remote systems because I prefer the structure of Python code over shell scripts (Python avoids quoting hell). When Bootstrap connects to a remote system it automatically installs the system package python2.7 on the remote system because this is required to run execnet (on the other hand, execnet itself does not have to be installed on the remote system).
class redock.bootstrap.Bootstrap(ssh_alias)

The Bootstrap configuration management system is implemented as the class Bootstrap.

__init__(ssh_alias)

Initialize the configuration management system by creating an execnet gateway over an SSH connection. First we make sure the python2.7 package is installed; without it execnet won’t work.

Parameters:ssh_alias – Alias of remote host in SSH client configuration.
execute(*command, **kw)

Execute a remote command over SSH so that the output of the remote command (the standard output and standard error streams) is immediately visible on the local terminal. If no standard input is given, this allocates a pseudo-tty (using ssh -t) which means the operator can interact with the remote system should it prompt for input.

Raises ExternalCommandFailed if the remote command ends with a nonzero exit code.

Parameters:
  • command – A list with the remote command and its arguments.
  • input – The standard input for the command (expected to be a string). This is an optional keyword argument. If this argument is given, no pseudo-tty will be allocated.
install_packages(*packages)

Install the given system packages on the remote system.

Parameters:packages – The names of one or more packages to install (strings).
rsync(local_directory, remote_directory, cvs_exclude=True, delete=True)

Copy a directory on the host to the container using rsync over SSH.

Raises ExternalCommandFailed if the remote command ends with a nonzero exit code.

Parameters:
  • local_directory – The pathname of the source directory on the host.
  • remote_directory – The pathname of the target directory in the container.
  • cvs_exclude – Exclude version control files (enabled by default).
  • delete – Delete remote files that don’t exist locally (enabled by default).
update_system_packages()

Perform a full upgrade of all system packages on the remote system.

upload_file(pathname, contents)

Create a file on the remote file system.

Parameters:
  • pathname – The absolute pathname on the remote system.
  • contents – The contents of the file (a string).
exception redock.bootstrap.ExternalCommandFailed

Raised by Bootstrap.execute() and Bootstrap.rsync() when an external command fails (ends with a nonzero exit status).

Miscellaneous utility functions

class redock.utils.Config

Config encapsulates the bits of runtime configuration that Redock needs to persist to disk (to share state in between runs of Redock). UNIX file locking is used to guarantee that the datafile is not written to simultaneously by multiple processes (that could corrupt the state).

To use this class to update the configuration, use it like a context manager, like this:

>>> config = Config()
>>> with config as state:
...   state['containers'].clear()

When used like this, state is a dictionary which is saved to disk when the with block ends without raising an exception.

__init__()
load(exists=True)

Load the runtime configuration from disk. If the file doesn’t exist yet an empty configuration is returned. The configuration contains a version number which enables graceful upgrades to the format.

Returns:A dictionary with runtime configuration data.
class redock.utils.RemoteTerminal(container_id)

Attach to a running Docker container and show the output of the command(s) inside the container on the host’s terminal. Can be used as a context manager or by manually calling RemoteTerminal.attach() and RemoteTerminal.detach().

__init__(container_id)

Initialize the context manager for the docker attach process.

Parameters:container_id – The id of the container to attach to (a string).
attach()

Start the docker attach subprocess.

detach()

Kill the docker attach subprocess.

redock.utils.apt_get_install(*packages)

Generate a command to install the given packages with apt-get.

Parameters:packages – The names of the package(s) to be installed.
Returns:The ap-get command line as a single string.
redock.utils.create_configuration_directory()

Make sure Redock’s local configuration directory exists.

redock.utils.find_local_ip_addresses()

To connect to a running Docker container over TCP we need to connect to a specific port number on an IP address associated with a local network interface on the host system (specifically not a loop back interface).

Returns:A set of IP addresses associated with local network interfaces.
redock.utils.generate_ssh_key_pair()

Generate an SSH key pair for communication between the host system and containers created with Redock. Requires the ssh-keygen program.

redock.utils.get_ssh_public_key()

Get the contents of the SSH public key generated by Redock for use inside containers. If the SSH key pair hasn’t been generated yet, it will be generated using generate_ssh_key_pair().

Returns:The contents of the id_rsa.pub file.
redock.utils.quote_command_line(command)

Quote the tokens in a shell command line.

Parameters:command – A list with the command name and arguments.
Returns:The command line as a single string.
redock.utils.select_ubuntu_mirror(force=False)

Find an Ubuntu mirror that is geographically close to the current location for use inside Docker containers. We remember the choice in a file on the host system so that we always configure the same mirror in Docker containers (if you change the mirror, apt-get has to download all package metadata again, wasting a lot of time).

redock.utils.slug(text)

Convert text to a “slug”. Used by redock.api.Container.ssh_alias.

Parameters:text – The original text, e.g. “Some Random Text!”.
Returns:The slug text, e.g. “some-random-text”.
redock.utils.summarize_id(id)

Docker uses hexadecimal strings of 65 characters to uniquely identify containers, images and other objects. Docker’s API almost always reports full IDs of 65 characters, but the docker program abbreviates these IDs to 12 characters in the user interface. We do the same because it makes the output more user friendly.

Parameters:id – A hexadecimal ID of 65 characters.
Returns:A summarized ID of 12 characters.