strange interpretation/parsing of .env file

[alekbarszczewski]

In native Max OS / Unix normally you can write:

MY_ENV_VAR1=abcdef
MY_ENV_VAR2="abcdef"

and both env vars will be equal (double quotes are omitted/ignored). Also libraries I know that use (parse) .env files work in a same way.

Docker compose however treats double quotes differently when parsing .env file passed as env_file: .env in docker-compose.yml. It does not ignore double quotes and treats them as a part of the string.

Is it intentional behaviour or it's a bug? Tested on docker-compose version 1.5.2, build 7240ff3.

[aanand]

It's the same behaviour as the Docker CLI, i.e. docker run --env-file. It would need to be fixed there before being fixed in Compose, because we try to keep behaviour exactly the same.

$ cat test.env
MY_ENV_VAR1=abcdef
MY_ENV_VAR2="abcdef"

$ docker run --env-file=test.env alpine sh -c 'echo $MY_ENV_VAR1'
abcdef

$ docker run --env-file=test.env alpine sh -c 'echo $MY_ENV_VAR2'
"abcdef"

You might want to check docker/docker to see if there's already an issue about it. I found moby/moby#12997, which is currently closed, but if there are other env file parsers out there with different behaviour, there's a strong case for us keeping parity with them.

[darkn3rd]

env_file are for containers, ./.env is for docker-compose. The compose variables, I know interpret quotes literally. This might be an advantage, as you don't have to worry about glob.

Alternatively, you can parse these out using a wrapper script to bootstrap your container to get full shell interpolation with your envfile, e.g. cat my_shell_envfile | xargs -n 1 export. If need extra shell features like subshells, e.g. START=$(date) then you'll need to re-interpolate it with eval before the export command.

[zoobab]

Just to convert the bash env.sh to env.docker:

$ sed -e "s/=\"/=/g" -e "s/\"$//g" env.sh > env.docker

It is annoying docker and docker-compose do not follow the same standard as shell.

[czert]

Here's an improved sed invocation, which will ignore "=" characters in a variable's value:

sed -e 's/="\(.*\)/=\1/g' -e 's/"$//g'

...or if you need to add quotes:

sed -e 's/=\(.*\)/="\1/g' -e 's/$/"/g'

This is useful if you want to keep the file in the broken, unquoted docker format, but use it somewhere else in a shell script. E. g., this will export all variables in the file:

export $(sed -e 's/=\(.*\)/="\1/g' -e 's/$/"/g' env.docker | xargs)

[komba]

sed -e 's/=\(.*\)/="\1/g' -e 's/$/"/g' env.docker won't work properly on file containing empty strings and comments.

export $(sed -e 's/=\(.*\)/="\1"/g' env.docker | grep -v "#" | xargs) would be a good alternative

[eduardolucioac]

This is quite strange behavior since .env files are regular BASH ("Shell") scripts.

However, BASH ("Shell") offers us powerful features, so let's use it to our advantage in a workaround solution.

My solution involves a Dockerfile, an env file, a BASH script file and the run subcommand (docker run) in a special way.

The strategy consists of injecting your environment variables using another environment variable set in the run subcommand and using the container itself to set these variables.

Workaround Solution

Create a Dockerfile

EXAMPLE

FROM python:3.10-slim-buster
WORKDIR /some-name
COPY . /some-name/
RUN apt-get -y update \
    && apt-get -y upgrade \
    [...]
ENTRYPOINT bash entrypoint.bash

Create an env file (BASH script file)

EXAMPLE

#!/bin/bash

# Some description a
SOME_ENV_VAR_A="some value a"

# Some description b
SOME_ENV_VAR_B="some value b"

# Some description c
SOME_ENV_VAR_C="some value c"
[...]

Create a BASH script for the ENTRYPOINT

EXAMPLE

#!/bin/bash

set -a;source <(echo -n "$ENV_VARS");set +a
python main.py

Injecting your environment variables using the run subcommand

EXAMPLE

docker run -it --name "some-ctn-name" --env ENV_VARS="$(cat ./.env)" "some-img-name:Dockerfile"

PLUS

The docker-compose does not have this problem as it uses YAML and YAML does not consider surrounding quotes and double quotes as part of the value of environment variables, which is something that is not done with docker run subcommand.

Done! 😎

[sourcehawk]

This is awfully inconvenient and leads to many very hard to debug problems.