Series: How to create your own website based on Docker (Part 7 - Creating the mongodb Docker container)

Creating our mongodb database image

This is part 7 of the series: How to create your own website based on Docker.

It's about time to create our first image/container that is part of the real application stack. This container acts as persistence layer for the REST API (which we will create in the next part of this series). So the only component that talks to the database is the ioJS REST API container. In this part of the series, we'll have a look into how you can create your own mongodb container based on the official mongodb image.

Source code

All files mentioned in this series are available on Github, so you can play around with it! :)

Let's get started

Let's create a new directory called /opt/docker/mongodb/ and within this new directory we'll create two folders and one file:

# mkdir -p /opt/docker/mongodb/config/
# mkdir /opt/docker/mongodb/db/
# > /opt/docker/mongodb/Dockerfile

Since I don't want to re-invent the wheel, I'll have a look at the official mongodb Docker image and we're basically using the same mongodb 3.0 Dockerfile for our design. Since we want to run this mongodb database on our own Ubuntu Base Container, we need to make some changes to official mongodb Docker image.

The official mongodb Dockerfile uses Debian wheezy as base image, which is not what we want:

FROM debian:wheezy

We are going to use our own Ubuntu Base Image for the mongodb image and since we use Docker Compose, we must specify the correct base image name, which is a concatenation of "docker_" and the image name that we have specified in our docker-compose.yml - so in our case that would be "docker_ubuntubase". So we're changing the aforementioned line to use our base image:

# Pull base image.
FROM docker_ubuntubase 

Since the original Dockerfile only allows us to mount /data/db as volume, so we're extending it to also allow the mongodb log directory:

Replace the following line:

VOLUME /data/db

With this line:

VOLUME ["/data/db","/var/log/mongodb/"]

I'd like to have my configurations in a subfolder called "config", so we need to adjust another line:

Replace the following lines:

COPY docker-entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

With these lines:

COPY ./config/docker-entrypoint.sh /tmp/entrypoint.sh
RUN ["chmod", "+x", "/tmp/entrypoint.sh"]
ENTRYPOINT ["/tmp/entrypoint.sh"]

These lines copy a script called ./config/docker-entrypoint.sh to the /tmp/ folder in the container, make it executable and run it once, once the container has started. You can find the docker-entrypoint.sh file in the official mongodb docker repository on GitHub. Just copy that file into the config directory, which you have to create if you haven't done so already.

Let's create our own mongodb configuration file to set some parameters.

To do so, create a file called /opt/docker/mongodb/config/mongodb.conf and add the following lines (important: YAML does not accept tabs; use spaces instead!):

systemLog:
   destination: file
   path: "/var/log/mongodb/mongodb-projectwebdev.log"
   logAppend: true
storage:
   journal:
      enabled: true
net:
   port: 3333
   http:
       enabled: false
       JSONPEnabled: false
       RESTInterfaceEnabled: false

Now add the following lines to your Dockerfile to copy our new custom config file to our image:

RUN mkdir -p /var/log/mongodb && chown -R mongodb:mongodb /var/log/mongodb
COPY ./config/mongodb.conf /etc/mongod.conf

Since we want to load our custom config now, we'll need to changed the way we start mongodb, so we change the following line from

CMD ["mongod"]

to

CMD ["mongod", "-f", "/etc/mongod.conf"]

Our folder structure must look like this now:

+-- /opt/docker/mongodb
¦   +-- config
¦   ¦   +-- docker-entrypoint.sh
¦   ¦   +-- mongodb.conf
¦   +-- db
¦   +-- Dockerfile

Another thing we can remove is the EXPOSE instruction, since we already specified that in our docker-compose.yml.

So the complete Dockerfile will look like this now:

# Pull base image.
FROM docker_ubuntubase

# add our user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added
RUN groupadd -r mongodb && useradd -r -g mongodb mongodb

RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates curl \
numactl \
&& rm -rf /var/lib/apt/lists/*

# grab gosu for easy step-down from root
RUN gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4
RUN curl -o /usr/local/bin/gosu -SL "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture)" \
&& curl -o /usr/local/bin/gosu.asc -SL "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture).asc" \
&& gpg --verify /usr/local/bin/gosu.asc \
&& rm /usr/local/bin/gosu.asc \
&& chmod +x /usr/local/bin/gosu

# gpg: key 7F0CEB10: public key "Richard Kreuter <richard@10gen.com>" imported
RUN apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys 492EAFE8CD016A07919F1D2B9ECBEC467F0CEB10

ENV MONGO_MAJOR 3.0
ENV MONGO_VERSION 3.0.3

RUN echo "deb http://repo.mongodb.org/apt/debian wheezy/mongodb-org/$MONGO_MAJOR main" > /etc/apt/sources.list.d/mongodb-org.list

RUN set -x \
&& apt-get update \
&& apt-get install -y mongodb-org=$MONGO_VERSION \
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf /var/lib/mongodb \
&& mv /etc/mongod.conf /etc/mongod.conf.orig

RUN mkdir -p /data/db && chown -R mongodb:mongodb /data/db
RUN mkdir -p /var/log/mongodb && chown -R mongodb:mongodb /var/log/mongodb

VOLUME ["/data/db","/var/log/mongodb/"]

COPY ./config/docker-entrypoint.sh /tmp/entrypoint.sh
COPY ./config/mongodb.conf /etc/mongod.conf
RUN ["chmod", "+x", "/tmp/entrypoint.sh"]

ENTRYPOINT ["/tmp/entrypoint.sh"]

CMD ["mongod", "-f", "/etc/mongod.conf"]

Source: https://github.com/mastix/project-webdev-docker-demo/blob/master/mongodb/Dockerfile

This is pretty much it. That's all we need to create our mongodb database container, that will run on the default port 3333 - but will only be accessible from the REST API, since we linked that container to the ioJS REST API container only, see our docker-compose.yml file again:

projectwebdevapi:
    build: ./projectwebdev-api
    expose:
        - "3000"
    links:
        - mongodb:db
    volumes:
        - ./logs/:/var/log/supervisor/
        - ./projectwebdev-api/app:/var/www/html

In the next chapter it's getting more interesting: Let's create our REST API container, that talks to the mongodb container!

Series: How to create your own website based on Docker (Part 5 - Creating our Docker Compose file)

Let's implement our docker container architecture

This is part 5 of the series: How to create your own website based on Docker.

In the last part of the series, we have planned and created our Docker container architecture. So now it's about time to turn this architecture into a real scenario - and that's what we need Docker Compose for.

Source code

All files mentioned in this series are available on Github, so you can play around with it! :)

What is Docker Compose?

Compose is a tool for defining and running complex applications with Docker. With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running.

Compose is great for development environments, staging servers, and CI. We don't recommend that you use it in production yet. (Source: https://docs.docker.com/compose)

There are three steps involved when using docker compose:

1. We need image files for each container (we'll start with that in the next chapter)
2. Then we need to create a docker-compose.yml file that tell docker compose which containers must be started, including all options (like volumes, links, ports,...)
3. At last we need to run docker-compose up to start up our container architecture (the configuration from the YAML file)

Since we have just created our architecture, we're starting with step 2 now and will create the image files later. This will show you how we can create a docker compose yaml file based on our architecture.

Implementing our container design

Let's recap - this is what our architecture looks like:

We're going to create a web site called projectwebdev, so the following container names are based on that name of the site. In the diagram above we can see that we have the following containers and options:

1. nginx reverse proxy
• links:
• nginx website
• ioJS REST API
• volumes:
• log files (/opt/docker/logs)
2. nginx web site
• links:
• none
• volumes:
• web site files (/opt/docker/projectwebdev/html)
• log files (/opt/docker/logs)
3. ioJS REST API
• links:
• mongoDB database
• volumes:
• ioJS application files (/opt/docker/projectwebdev-api/app)
• log files (/opt/docker/logs)
4. mongoDB database
• links:
• none
• volumes:
• mongoDB files (/opt/docker/mongodb/db)
• log files (/opt/docker/logs)

The Docker directory structure on my VM

I will use the following folder structure on my Ubuntu VM to host all Docker images/containers:

/opt/docker/
│
├── logs
│
├── mongodb
│
├── nginx-reverse-proxy
│
├── projectwebdev
│
├── projectwebdev-api
│
├── ubuntu-base
│
└── docker-compose.yml

So the docker-compose.yml file will be in the root directory of all docker image directories (in which we will dive into later). With this setup, I can later just copy the /opt/docker/ folder on to another server and then just run docker-compose up to get everything up and running again.

You can also see that this directory structure already contains a logs/ directory, which will be the collection point for all container logs we've been talking about in the last part of this series.

Create the Docker Compose file

If you've never heard of YAML before, let me just tell you what it is. YAML is a recursive acronym for "YAML Ain't Markup Language". Early in its development, YAML was said to mean "Yet Another Markup Language", but it was then reinterpreted (backronyming the original acronym) to distinguish its purpose as data-oriented, rather than document markup. YAML’s purpose is to have a human friendly data serialization standard for all programming languages. (see: http://yaml.org)

In our YAML file we will tell Docker Compose how our containers must be started, which volumes should be mounted, which containers should be linked together and what ports should be exposed. So it's basically everything from that list above.

Let's get into details - This is what our docker-compose.yaml file looks like:

ubuntubase:
    build: ./ubuntu-base
projectwebdev:
    build: ./projectwebdev
    expose:
        - "8081"
    volumes:
        - ./logs/:/var/log/nginx/
        - ./projectwebdev/html:/var/www/html:ro
projectwebdevapi:
    build: ./projectwebdev-api
    expose:
        - "3000"
    links:
        - mongodb:db
    volumes:
        - ./logs/:/var/log/pm2/
        - ./projectwebdev-api/app:/var/www/html
mongodb:
    build: ./mongodb
    expose:
        - "3333"
    volumes:
        - ./logs/:/var/log/mongodb/
        - ./mongodb/db:/data/db
nginxreverseproxy:
    build: ./nginx-reverse-proxy
    expose:
        - "80"
        - "443"
    links:
        - projectwebdev:blog
        - projectwebdevapi:blogapi
    ports:
        - "80:80"
    volumes:
        - ./logs/:/var/log/nginx/

Source: https://github.com/mastix/project-webdev-docker-demo/blob/master/docker-compose.yml

Let's pick the nginx reverse proxy to explain our settings. Besides all other Docker Compose YAML possibilies, we'll only use build, expose, links, ports and volumes.

build: This is the path to the directory containing the Dockerfile for the image. We have supplied that value as a relative path, which means that it is interpreted relatively to the location of the YAML file itself. This directory is also the build context that is sent to the Docker daemon. All files belonging to the nginx reverse proxy reside in folder ./nginx-reverse-proxy, therefore we tell Docker Compose to build the image based on the following Dockerfile /opt/docker/nginx-reverse-proxy/Dockerfile, which we're going to create later.

expose: This section specifies the ports to be exposed without publishing them to the host machine - they'll only be accessible to linked services. Only the internal port can be specified - see the architecture diagram above, these exposed ports are the ports with the purple background color.

links: Here we specify all links to containers in other services. You can either specify both the service name and the link alias (SERVICE:ALIAS), or just the service name (which will also be used for the alias). In our design we'll use aliases, so we'll tell Docker that whenever we want to talk to our containers we want them to use blog (for the projectwebdev website) and blogapi (for our ioJS REST API).

ports: The ports we want to expose to the Docker host - see the yellow port in the architecture diagram above. You can either specify both ports (HOST:CONTAINER), or just the container port (a random host port will be chosen). Since we want to make sure that it's always the same port (in our case it's port 80) we specify the HOST and the CONTAINER port explicitly (which in both cases would be 80). If your nginx reverse proxy in your container uses port 8000 and you want that port to be accessible from outside via port 80, you'll specifiy it like this: "80:8000". Important: When mapping ports in the HOST:CONTAINER format, you may experience erroneous results when using a container port lower than 60, because YAML will parse numbers in the format xx:yy as sexagesimal (base 60). For this reason, Docker recommends always explicitly specifying your port mappings as String.

volumes: This section contains all mount paths as volumes, optionally specifying a path on the host machine (HOST:CONTAINER), or an access mode (HOST:CONTAINER:ro). The latter one (:ro = readonly) is used in our projectwebdev container, since we don't want the container to change the files for any reason. Only our host may provide the markup that is needed for the website.

We have now implemented our architecture with Docker Compose! Let's create each image and container so we can fire up docker compose. We'll start with our Ubuntu Base Image!

Series: How to create your own website based on Docker (Part 4 - Planning Docker container architecture)

Let's design our docker container architecture

This is part 4 of the series: How to create your own website based on Docker.

Docker and Docker Compose are now up and running. So it's about time to let them all play together.

Before we start planning our container architecture, we need to make sure that we understand what we're trying to achieve.

1. We want to be able to port our apps to any other platform as easy as possible
2. We want our applications to be as separated as possible (every container should have one purpose)
3. We want to create more instances of an application container if needed
4. We don't want crashed applications to crash other applications

So based on this list we need to figure out what components will be needed for the site.

Let's define all components to be "dockerized"

Component 1: nginx reverse proxy

I usually start with nginx as reverse proxy in front of all other services. This allows me to have a single entry point for all requests and to distribute them internally to all containers which should be accessible from the web. This reverse proxy will listen on port 80 and will redirect all requests based on the context root, the subdomain and/or the hostname.

Here's an example:

• www.project-webdev.com:80 => redirects to my blog which listens on port 8081 internally (the projectwebdev blog will be hosted on another nginx machine).
• api.project-webdev.com:80 => redirects to my REST API which listens on port 3001 internally (the API will be an ioJS/nodeJS application).
• Also possible: www.project-webdev.com:80/api => redirects to my REST API which listens on port 3001 internally (the API will be an ioJS/nodeJS application).

As you can see, all requests will go through the nginx reverse proxy and will be redirected internally to the appropriate service.

What's important to mention: The internal ports (e.g. 8081, 3001,...) should not be exposed to public, so it should not be possible to access the blog directly (like www.project-webdev.com:8081).

This would be our first component, right? Wait! Where should our new nginx reverse proxy run on?

We need an operating system... so basically our first component would actually be a container that contains the operating system. For that I'm going to use Ubuntu 14.04 LTS.

So right now our current architecture would look like this:

Component 2: nginx web server for our website

The next thing we'll need for our web site is the markup for the website. So all we need now is another nginx web service, but this time it will not act as reverse proxy, but as a real web server hosting our files. Since we don't want any port conflicts my basic rule is that 4-digit ports are never exposed to public. (Port 80 and 443 (SSL) are allowed to be accessed from outside).

Since Docker works with container links, we need to add a link to our reverse proxy, so that it can communicate internally with our blog application (nginx web site), which listens on port 8081.

As mentioned before, port 8081 can not be accessed from outside - therefore painted in purple.

You can also see that we've mounted a directory on the docker host into our blog's docker container. We're doing this, because we want to be able to change the website from outside later, without restarting the container.

Technically it would look like the following - I'll go into details later:
In the docker container (our provider's Ubuntu server), we'll create a directory called /opt/docker/projectwebdev/html/ which will be mounted as /var/www/html/ in the container (the directory which nginx uses to load the HTML, CSS and JS files from later). So whenever nginx (the one in our nginx web site container) receives a request from a visitor, it will load the files from our real server (from /opt/docker/projectwebdev/html/) and will provide it to him - I think you've got it, right? It's not that hard.

Component 3: ioJS REST API container

Our website should fetch information from a REST API. Therefore we will need an ioJS application that will provide all data for the website asynchronously. The website will use a call to http://api.project-webdev.com to fetch the contents or any other information needed.

Since this is a new URL, we need to link this container to our nginx as well, so that we can build our redirect to that container internally.

Component 4: mongoDB database container

A REST API does not make sense without any data persistence in the back. Therefore we need to add our mongoDB to that architecture as well. This instance listens on port 3333 and should only be accessible via REST API (and therefore implicitly via our nginx reverse proxy), which is why we need to add a link to our REST API so that it can access the data in the mongoDB.

Additional component: Logs

When running this application stack in the wild later, it's very important to be able to analyse the logs (e.g. using the ELK stack). Since we have several containers, it's does not make to get the logs from each instance separately. So we're creating another volume mount which acts as central storage for all log files of all used containers. This directory can later be used by log file analysis tools, so you can analyse the hell out of your logs. :)

Conclusion

We have now several containers that act as "platform" for a certain purpose. They are all completely encapsulated and are sharing their resources (thanks to Docker).

1. nginx reverse proxy
• links:
• nginx website
• ioJS REST API
• volumes:
• log files (/opt/docker/logs)
2. nginx web site
• links:
• none
• volumes:
• web site files (/opt/docker/projectwebdev/html)
• log files (/opt/docker/logs)
3. ioJS REST API
• links:
• mongoDB database
• volumes:
• ioJS application files (/opt/docker/projectwebdev-api/app)
• log files (/opt/docker/logs)
4. mongoDB database
• links:
• none
• volumes:
• mongoDB files (/opt/docker/mongodb/db)
• log files (/opt/docker/logs)

That's it... that's our "dockerized" architecture for our projectwebdev website based on Docker containers. Let's create our Docker Compose file now... in the next part of this series.

Series: How to create your own website based on Docker (Part 3 - Installing Docker)

It's time to get really started...

This is part 3 of the series: How to create your own website based on Docker.

If you still don't know what Docker is or what it does? Just read the official "What is docker" document!

In this part of the series, we're going to install Docker and Docker Compose - although Docker does not recommend to use Docker Compose for production use, we'll still give it a shot. If you've never heard of Docker Compose, let me tell you in a few words what it does.

Docker Compose is a tool for defining and running complex applications with Docker. With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running. In short: You'll define a YAML file in which you'll specify how the Docker containers must be started and how they are linked together. Docker Compose will then start them for you and will make sure that they are started in the right order. It will also take care of naming these containers baed on your settings. But we'll get into that in one of the next parts.

Let's install Docker & Docker Compose

As I've mentioned before, Docker requires a 64-bit installation regardless of your Ubuntu version. Additionally, your kernel must be 3.10 at minimum. The latest 3.10 minor version or a newer maintained version are also acceptable.

Run the following command, which will download a shell script and will trigger the installation of Docker. When running this command, it will ask you for your password. Just provide the password that you have set for johndoe in part 2 of this series.

# wget -qO- https://get.docker.com/ | sh

To verify that Docker has been installed correctly, just type the following to see the options that you have when running Docker:

# sudo docker --help

Now let's install Docker Compose by typing the following:

# curl -L https://github.com/docker/compose/releases/download/1.2.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
# chmod +x /usr/local/bin/docker-compose

Let's add johndoe to a new group called docker:

# sudo usermod -aG docker johndoe

Now you should be able to run docker containers without using sudo.

In order to start our service automatically when we reboot the system, we need to add it to the default runlevel:

sudo update-rc.d docker defaults

Finally we also need to setup our UFW again, so that it can work with docker, by exposing another port (remember we've done that already with the SSH port):

#sudo ufw allow 2375/tcp

We also need to change some UFW settings to make Docker work correctly:

# sudo vi /etc/default/ufw

Set the DEFAULT_FORWARD_POLICY policy to:

DEFAULT_FORWARD_POLICY="ACCEPT"

Reload UFW to use the new setting.

# sudo ufw reload

Now check the status of the firewall again:

# sudo ufw status

It should now look similar to this:

Status: active
To               Action      From
--               ------      ----
2233/tcp         ALLOW       Anywhere
2375/tcp         ALLOW       Anywhere

Now Docker as well Docker Compose are installed - let's think about a Docker architecture! :)

Series: How to create your own website based on Docker (Part 1 - Introduction)

What you are going to learn in this series

This is my first official blog post ever and it's not going to be the last. You probably came here, because you searched for a way to learn how to create a website using Docker and state-of-the-art web technologies. If yes - then you're right. This series will, however, not get into details of the actual implementation, but will basically explain how an Ubuntu machine can be set up to act as Docker host for a web site, based on different docker containers.

In other blog posts, besides writing about new technologies in the world of web development, I will also tell you how to write AngularJS 2.0 applications, what coding guidelines are important for web applications and what can be done to turbo-charge your website so that it performs well in slower networks.

In this series the basic story is that I'm going to build a simple web site using the following technologies:

• Ubuntu - hosting platform
• Docker - lightweight container virtualization
• Docker Compose - orchestration for my Docker containers
• ioJS- JavaScript backend for the my REST API
• nginx - reverse proxy and web server for the blog
• mongodb - database for blog entries and meta data
• Twitter Bootstrap - the basic grid layout for the page
• Gulp - the JavaScript build system
• Yeoman - A scaffolding tool for projects
• Hapi - A easy-to-use REST API for nodejs.
• HTML5 - markup language for the front end
• JavaScript - for the front- and backend code
• AngularJS 2.0- for the frontend logic (MVVM)
• CSS3 - we need to style the page somehow, right?

The goal is to create a simple website, that talks to a REST API (and a database) - and everything will be based on docker. You can use this setup later to build your own full blown website.

I will also use this technology to create my own blog later, which means that I'm currently using this blog (http://project-webdev.blogspot.de) only for documenting my way to my own blog, until I switch to the new one. The new blog will be available under the following URL: http://www.project-webdev.com - so stay tuned!

Who I am

Since this is Part 1 of the series, let me just start with an introduction of myself! :)

My name is Sascha Sambale and I'm a Software Engineer specialized in Java and JavaScript (as well as other frontend technologies). I work as a Software Architect (specialized in web development and web performance) for the Robert Bosch GmbH in Stuttgart, Germany. This blog is not affiliated with Robert Bosch GmbH in any kind of way - it's strictly personal and I'm only sharing my personal opinions and thoughts, that are in no way related to my job.

What will be covered

The blog that I'm going to create will be all about web technologies - new trends, tutorials, hints, tips and tricks. All you need to know to get started with new web technologies. It's not my goal to make money with this blog - it's all about knowledge sharing.

About this series

This series will show you how you can set up a web site using the aforementioned technologies (especially Docker). It will consist of several parts and will be updated as soon as I've managed to work on a new chapter. It will basically start with setting up Ubuntu as Docker host. I will include all necessary scripts and steps that will help you to get your system up and running.

Last words

Make sure that you add this blog to your RSS reader to get updated as soon as I publish a new chapter of this series. I am not going to promise that I'll update this blog every day, but you'll get updates as soon as I've implemented the next milestone.

Source code

All files mentioned in this series are available on Github, so you can play around with it! :)

The following parts are/will be available

• Part 1 - Introduction (you're just reading it!)
• Part 2 - Setting up Ubuntu for production use
• Part 3 - Installing Docker
• Part 4 - Planning Docker container architecture
• Part 5 - Creating our Docker compose file
• Part 6 - Creating the Ubuntu base Docker container
• Part 7 - Creating the mongodb Docker container
• Part 8 - Creating the ioJS REST API Docker container
• Part 9 - Creating the nginx web site Docker container
• Part 10 - Creating the nginx reverse proxy Docker container
• Part 11 - Run the whole application stack with Docker Compose

Have fun and I hope you can learn something from this blog,

Sascha