docker-compose advanced configuration

Everything I do with docker I use docker-compose. I wasn’t a fan of the tremendous command lines which just concatenated cli option after cli option starting with docker run. Running every container with a docker-compose file helps me document the cli options and keeps a version from the current run state of containers somewhere.

But the really cool things with docker-compose weren’t obvious to me, although the documentation is really great. I’ll tell you my best tips when working with docker-compose configuration files.

Chaining multiple compose files

When you execute docker-compose it will search for docker-compose.yml in the current working directory and use this file. But you can use multiple files and chain them together so that the subsequent files extend the base file. Calling:

docker-compose -f compose.base.yml -f compose.development.yml 

will extend the configuration in compose.base.yml with compose.development.yml. Pretty easy? Pretty handy! But there are pitfalls. You have to have a deep knowledge of the yaml format and how merging will work. Good thing if you know s.th. about Symfony configuration, it uses configuration merges as well. To debug the merges just use:

docker-compose -f file1.yml -f file2.yml config 

It will print the merged configuration file.
Ports cannot be overwritten. They will always be added (inline arrays in the extending file will not work either). Extending volumes is tricky and a little bit unintuitive. As a rule of thumb: make your base file pretty slim and do the fancy stuff in consecutive files.

Using an .env file

Writing -fmultiple times on every command is too much. But you can create a file .env in the directory next to your base compose file and reference the files like this:

COMPOSE_PATH_SEPARATOR=:
COMPOSE_FILE=compose.base.yml:etc/docker/compose.dev.yml

Defining SEPARATOR is optional, but the separator differs on Windows (;) and Unix (:). Setting it makes your .env file more portable. Compose will find the .env file and use the provided compose files. But be careful: the first used file defines the compose-working-directory, e.g. all host-binds are relative to that directory.
The env file can be used to define substitution-variables as well:

SYMFONY_ENV=dev
SYMFONY_DEBUG=1
SYMFONY_BASEURL=development.ich-will-ein-pony.de
BUILD_VERSION=dev
DOCKER_IMAGE_VERSION=development
DOCKER_HOSTIP=172.18.0.25
SUBNET=172.18.0.24/29
COMPOSE_PATH_SEPARATOR=:
COMPOSE_FILE=compose.base.yml:etc/docker/compose.dev.yml

But be careful here. Those variables will not propagate to your container. You have to pass them through:

services:
  php:
    image: ...
    environment:
      - SYMFONY_ENV
      - SYMFONY_DEBUG
    # or pass them all
    env_file:
      - .env

And: those variables will be overridden from actual environment variables in the shell calling docker-compose. Took me hours to find out…

The variables defined in .env can be used as substitution variables:

php:
 image: iwep/php:${DOCKER_IMAGE_VERSION}
   build:
     context: .
     dockerfile: ./etc/docker/Dockerfile
    extra_hosts:
      - docker-host:${DOCKER_HOSTIP}

And they can have defaults as well

web:
  image: nginx:1.13-alpine
  ports: 
    - "${NGINX_PORT:-10001}:80"

Defines a default value 10001. The - is mandatory.

• don't use quotes in the .env file, they will be part of the value
• you can define compose cli variables here. Set COMPOSE_PROJECT_NAME for nicer project names (because docker-compose will use the basename from the working directory as a default!)
• remember: shell environment variables will overwrite vars in .env
• you have to explicitly pass them to the environment of your containers (environment: or env_file: )

Unfortunately: the .env file is hidden on Unix by default. This leaves a little confusing situation for your co-workers: why is this compose file working? Where are these variables defined? well …

You can use the advanced docker-compose configuration in your continuous integration server and for different environments. My CI-Server writes the .env file dynamically in a build job. It helps to keep your configuration files DRY and fights the evil in the world.

Have fun with docker-compose!

Did this article help you?

Please clap it a few or more times, so others can find it and it’ll help them, too. Thanks!

Do you want to get even more awesome in docker?

https://www.ps-webforge.com/docker