Docker and environment variables
I'll preface this post by saying that I hope it will some day become obsolete, when Docker fixes what feels to me like oversights in its handling of environment variables.
One aspect of Docker that I struggled with when first integrating it in my development workflow was its treatment of environment variables that my services relied on for configuration. Considering that this is a recommended modern development practice, I felt (feel) that Docker fumbled a bit on this. With Compose, some of the issues were mitigated, but others remain and the documentation is either mistaken or misleading in some aspects.
Here are some suggestions to hopefully save yourself some time and head scratching.
Just use Compose
My very first advice if you rely on environment variables in your services is to just use Compose. Compose offers a much better interface to deal with environment variables than trying to manage them via the CLI that ships with Docker Engine. Without Compose, I've had to implement some pretty sad hacks, as seen on threads on StackOverflow and GitHub, just to have a faithful injection of my required variables in the container.
Get comfortable using the --env-file option
Compose is inexplicably committed to an environment file named .env by default. If you name it differently, you have to explicitly set the --env-file when executing a Compose command.
$ docker compose --env-file ./compose.dev.env up
There's no header in the Compose config file that can override this. In practice, however, people will want multiple environment files. It's also not inconceivable that you would want to avoid naming yours simply .env, because that could potentially clash with other tools or processes that also expect to find a file with the same name. Considering that the default configuration file for Compose is named compose.yml or docker-compose.yml, I think it's a shame that its default environment file would not be similarly namespaced (e.g. compose.env or .compose.env).
--env-file and env_file are not equivalent
The documentation insinuates that a service's env_file can somehow be an alternative to --env-file. This isn't true.
--env-file is a Compose-level flag and variables declared in files listed with it will only be available at Compose time, to be used within compose.yml and files pointed to by env_file. Let's for example declare an environment file for our services as docker compose --env-file ./compose.env up.
# compose.env
APP_ENV_FILE="./app.env"
DB_NAME=foo
We can then make use of its variables in compose.yml.
version: "3"
services:
app:
env_file: $APP_ENV_FILE
links:
- foodb:$DB_NAME
foodb:
image: postgres
These variables are not only available in compose.yml, they'll also be accessible in the various service-level environment files declared with env_file. They will not, however, be passed down to containers.
Consider the app.env file declared as the environment file for the app service.
# app.env
APP_DB_NAME=$DB_NAME
The variable $DB_NAME above is pulled from compose.env. Note, however, that APP_DB_NAME will not be available anywhere inside compose.yml, not even as part of the app service, despite being its declared environment file. The following would trigger an error.
version: "3"
services:
app:
env_file: ./app.env
links:
- foodb:$APP_DB_NAME # <-- error
foodb:
image: postgres
To recap on env_file, it's a service-level attribute that points to an environment file whose variables will only be available in the service container for which it's declared. The variables will not be available to the service's declaration in compose.yml. They still can access the Compose-level environment variables set in .env or with --env-file=./path/to/compose.env.