After discovering how to use Docker Engine on OpenStack VMs in the previous entry, we are going to show how to create a Docker Swarm in this article. Docker Swarm is a native clustering system for Docker. Docker Swarm is responsible for running and deploying new containers. It orchestrates the hosts incorporated to the cluster into a single Docker entity.We will discover how to create and set up the cluster with master and slave nodes and how to run containers in the Swarm.

Swarm discovery token

To create the cluster, we need a Swarm discovery token. It is an unique token identifying the cluster. There is an image enabling us to generate such a token on Docker Hub. Running a container to generate the token is slightly different for Linux and OSX users because of the environment the container can run in.

Linux

Because Docker Engine runs on a Linux kernel, we can run Docker containers in local environment. We started the Docker daemon while following the installation steps with command sudo service docker start. That makes us eligible to run the command directly.

OSX

As mentioned in a paragraph above, Docker Engine runs on a Linux kernel so it is not possible to run Docker Engine directly on OSX machines. Therefore we initially have the default host running VirtualBox. We can perform the Docker Engine (docker ...) commands on the default VirtualBox host as well as Linux users use Docker Engine as one of the services.

We have to activate the desired machine. Generally, any host can be used for generating the token. We use the default host as it is initially there.

$ eval $(docker-machine env default)

Generate the token

Note: docker run ... command deploys a container on a given node and executes defined commands. Once the execution is complete, the container exits. Since then, we are unable to see it with docker ps command but we are able to see the exited nodes with docker ps -a command because it still remains inside the node. We can add a --rm flag to docker run command to remove the container once it exits.

Generate the discovery token by typing:

$ docker run swarm create
Unable to find image 'swarm:latest' locally  
...
22bd85055fe873e30b3c3fce0362b528  

Save the swarm token for a later use. The number above is an identifier of the swarm. It is unique for every single cluster.

Create a Swarm master

Every Swarm has its Swarm master and slave nodes. It is also possible (and for high-availability of the Swarm desirable) to have more Swarm masters in one Swarm. Nevertheless, there is always only one active. Swarm master can be actually considered being a container as well as other applications running on the node.

The Docker Machine uses TCP port 2376 for communication, in Docker Swarm, TCP port 3376 is used instead. We have to keep this port open for communication on every VM we want to use with Docker Swarm.

We can create the master node on OpenStack VM by executing this command:

$ docker-machine create -d generic \
--generic-ip-address <public_ip> \
--generic-ssh-key <path/to/private/key> \
--generic-ssh-user ubuntu \
--swarm \
--swarm-master \
--swarm-discovery=token://<discovery_token> \
swarmMaster  

To create the Swarm master, we add --swarm --swarm-master --swarm-discovery=token://<discovery_token> to regular host-creating script. This command expects a Ubuntu VM with public IP address already running. The rest of the command was already described in the Create a Docker host on OpenStack VM section of previous Docker post.

Now we can run docker-machine ls to see the new Swarm master.

NAME          ACTIVE   DRIVER       STATE     URL                         SWARM                  DOCKER    ERRORS  
default       *        virtualbox   Running   tcp://192.168.99.100:2376                          v1.10.2  
swarmMaster   -        generic      Running   tcp://1.2.3.4:2376          swarmMaster (master)   v1.10.2  

Create a Swarm node

Once we know how to create a Swarm master, the creation of a Swarm node is simple. The syntax is identical, we just leave out the --swarm-master flag. Just don’t forget to input the same swarm discovery token.

Let’s create a Swarm node called swarm01.

$ docker-machine create -d generic \
--generic-ip-address <public_ip> \
--generic-ssh-key <path/to/private/key> \
--generic-ssh-user ubuntu \
--swarm \
--swarm-discovery=token://<discovery_token> \
swarm01  

How to activate the Swarm master

Now when our two-node cluster is created, we are ready to deploy applications in it. We activate the whole Swarm by running the command eval $(docker-machine env --swarm swarmMaster). The --swarm flag defines the swarm port 3376 to be marked as the active one. See the difference that is caused by the flag:

$ docker-machine env --swarm swarmMaster | grep DOCKER_HOST
export DOCKER_HOST="tcp://1.2.3.4:3376"  
$ docker-machine env swarmMaster | grep DOCKER_HOST
export DOCKER_HOST="tcp://1.2.3.4:2376"  

We can choose whether we want to work with a single host or with the whole cluster. Run eval $(docker-machine env swarmMaster) to activate the swarmMaster node and see the output of docker-machine ls. The host swarmMaster is marked as active.

NAME          ACTIVE   DRIVER       STATE     URL                         SWARM                  DOCKER    ERRORS  
default       -        virtualbox   Running   tcp://192.168.99.100:2376                          v1.10.2  
swarm01       -        generic      Running   tcp://5.6.7.8:2376          swarmMaster            v1.10.2  
swarmMaster   *        generic      Running   tcp://1.2.3.4:2376          swarmMaster (master)   v1.10.2  

Now try the eval $(docker-machine env --swarm swarmMaster) command and see the difference in the docker-machine ls output:

NAME          ACTIVE      DRIVER       STATE     URL                         SWARM                  DOCKER    ERRORS  
default       -           virtualbox   Running   tcp://192.168.99.100:2376                          v1.10.2  
swarm01       -           generic      Running   tcp://5.6.7.8:2376          swarmMaster            v1.10.2  
swarmMaster   * (swarm)   generic      Running   tcp://1.2.3.4:2376          swarmMaster (master)   v1.10.2  

swarmMaster node is now marked with * (swarm) meaning that we are in hold of the swarm now.

Set up the Docker Swarm

When running a container in Swarm, the Swarm Master daemon decides on which host to run the container physically. Swarm Manager computes ranking for the nodes in the cluster to find out the best host for deploying the container. Initially, the container is deployed to the least busy node in order to balance the load on hosts across the cluster.

To influence this behaviour, we can choose some of the Docker Swarm strategies by using the --strategy <strategy> option. Read more about Docker Swarm Strategies here.

Another powerful tool for scheduling the cluster are Docker Swarm filters. There are two types of filters – node filters and container filters.

Node filters define which containers are allowed to run on specified nodes. It enables us for example to differentiate frontend and backend nodes or development and production nodes. Container filters define the relation of containers. This enables us for example to run two specific containers always on the same host.

Read more about filters here.

How to run containers in Docker Swarm

Now let’s run a few containers to see how the manager works:

$ for i in `seq 1 6`; \
do docker run -d --name container$i -p 808$i:80 tutum/hello-world; \  
done  

This command runs 6 containers in the Swarm. Each of the containers exposes its TCP port 80 to different port. The Hello-World website is available only when we allow communication on those ports in security groups.

The output of docker ps command contains the running containers. Notice the “NAMES” column where the particular node is mentioned.

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                        NAMES  
4847c347ab68        tutum/hello-world   "/bin/sh -c 'php-fpm "   4 minutes ago       Up 4 minutes        5.6.7.8:8086->80/tcp         swarm01/container6  
01b4306c6f6e        tutum/hello-world   "/bin/sh -c 'php-fpm "   4 minutes ago       Up 4 minutes        1.2.3.4:8084->80/tcp         swarmMaster/container4  
fe12bddf0867        tutum/hello-world   "/bin/sh -c 'php-fpm "   4 minutes ago       Up 4 minutes        5.6.7.8:8085->80/tcp         swarm01/container5  
ed8c4bd6dabd        tutum/hello-world   "/bin/sh -c 'php-fpm "   4 minutes ago       Up 4 minutes        1.2.3.4:8082->80/tcp         swarmMaster/container2  
7a4a7546ec25        tutum/hello-world   "/bin/sh -c 'php-fpm "   4 minutes ago       Up 4 minutes        5.6.7.8:8083->80/tcp         swarm01/container3  
d3c991bfbc44        tutum/hello-world   "/bin/sh -c 'php-fpm "   4 minutes ago       Up 4 minutes        5.6.7.8:8081->80/tcp         swarm01/container1  

Check the Swarm set-up

We have two hosts in the Swarm – the Swarm Master and the Swarm node. We can easily check running nodes with docker run swarm list token://<discovery_token>.

$ docker run swarm list token://<discovery_token>
5.6.7.8:2376  
1.2.3.4:2376  

To view more detailed info regarding the swarm, there is a docker info command. If the Swarm node is activated, the command prints an overview of the Swarm. We can find detailed information regarding the nodes and the set-up of the Swarm.

$ docker info
Containers: 9  
Running: 9
Paused: 0
Stopped: 0
Images: 5  
Server Version: swarm/1.1.2  
Role: primary  
Strategy: spread  
Filters: health, port, dependency, affinity, constraint  
Nodes: 2  
swarm01: 5.6.7.8:2376
└ Status: Healthy
└ Containers: 5
└ Reserved CPUs: 0 / 2
└ Reserved Memory: 0 B / 2.053 GiB
└ Labels: executiondriver=native-0.2, kernelversion=3.13.0-45-generic, operatingsystem=Ubuntu 14.04.1 LTS, provider=generic, storagedriver=aufs
└ Error: (none)
└ UpdatedAt: 2016-03-10T14:35:42Z
swarmMaster: 1.2.3.4:2376
└ Status: Healthy
└ Containers: 4
└ Reserved CPUs: 0 / 2
└ Reserved Memory: 0 B / 2.053 GiB
└ Labels: executiondriver=native-0.2, kernelversion=3.13.0-45-generic, operatingsystem=Ubuntu 14.04.1 LTS, provider=generic, storagedriver=aufs
└ Error: (none)
└ UpdatedAt: 2016-03-10T14:35:29Z
Plugins:  
Volume:
Network:
Kernel Version: 3.13.0-45-generic  
Operating System: linux  
Architecture: amd64  
CPUs: 4  
Total Memory: 4.105 GiB  
Name: swarmMaster  

The output shows that there are nine containers in the Swarm running – swarm-agent on each node, one swarm-master and the hello-world app six times. Furthermore, we are seeing the Labels of the nodes or the Strategy set to spread.

Purpose of creating a Swarm

Docker Swarm has proven to be an useful tool for orchestrating containers. The cluster liberates us from distributing the workload across the containers manually. It is capable of distributing it by itself even taking our preferences in concern. Moreover, it is able to give us information regarding the whole cluster and thus provide us with an overview of the environment we are building.

Although it is not covered in this blog entry, Docker Swarm is also a useful tool in case of outage on one of the nodes, because it can deploy a highly available cluster.