WIP: Switch Docker Driver to use the API instead of CLI

[rickard-von-essen]

Supersedes #3921, and includes #3922.

These things must handled/fixed:

• run_command must be changed
• Add support for overriding DOCKER_HOST, DOCKER_CERT_PATH, and
  DOCKER_TLS_VERIFY
• Fixer to migrate from the old config with run_command to command etc.
• Login & Logout support
• Allow setting more options previously handled by run_command.
• Handle privileged mode
• Update docs

[rickard-von-essen]

Ping @vancluever

[vancluever]

Thanks @rickard-von-essen, yeah things have been pretty busy for me on this side and I haven't had time to do any work on the builder with other commitments. I do have some code that might help with some of the above items though, I'll take some time to dig it up this evening!

[rickard-von-essen]

Closes #2809

[vancluever]

Hey @rickard-von-essen, unfortunately I think I'm not going to have too much time to work on the builder as a whole right now, just given my current work and personal project load. I do have the code in a gist though, check out the URL below:

https://gist.github.com/vancluever/51b19bf900faab62c87a750b3bf79f06

This may need some fixing for build errors and what not but it should show what the config would look like.

[rickard-von-essen]

Ok, thanks for letting me know. I'll have some more work on this that I can share, hopefully soon.

…

On 5 December 2016 at 16:43, Chris Marchesi ***@***.***> wrote: Hey @rickard-von-essen </~https://github.com/rickard-von-essen>, unfortunately I think I'm not going to have too much time to work on the builder as a whole right now, just given my current work and personal project load. I do have the code in a gist though, check out the URL below: https://gist.github.com/vancluever/51b19bf900faab62c87a750b3bf79f06 This may need some fixing for build errors and what not but it should show what the config would look like. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4186 (comment)>, or mute the thread </~https://github.com/notifications/unsubscribe-auth/AAiCg7KDMjCU39Ns6sMyuGsf1fGV3NIEks5rFDE3gaJpZM4K3gqb> .

[mwhooker]

note to myself: closing a couple of docker issues that would be better implemented after this PR. look at referenced issues

[vtolstov]

Does this pr needs help from somebody to get it merged?

[dolobanko]

Have the same problem, when running Packer inside a Docker container.
Docker version 1.12.3
Packer version 0.12.2

[dolobanko]

Here is a shortcut from debug log:

[0;32m    docker: Run command: docker run -v /var/lib/jenkins/.packer.d/tmp/packer-docker826911088:/packer-files -d -i -t ubuntu:16.04 /bin/bash�[0m
�[0;32m    docker: Container ID: 4e9661cd0557d74cd662c9276aa96739f9a2aff75d5a972370a7c1011a91373e�[0m
�[1;32m==> docker: Provisioning with shell script: /tmp/packer-shell177956160�[0m
�[1;32m==> docker: Killing the container: 4e9661cd0557d74cd662c9276aa96739f9a2aff75d5a972370a7c1011a91373e�[0m
�[1;31mBuild 'docker' errored: Retryable error: Error uploading script: Upload failed with non-zero exit status: 1�[0m

==> Some builds didn't complete successfully and had errors:
--> docker: Retryable error: Error uploading script: Upload failed with non-zero exit status: 1

==> Builds finished but no artifacts were created.

[WeatherGod]

I am getting hit by this as well:
Docker version 1.13.1, build 092cba3
packer version 0.12.2

I am not doing anything fancy:

{
  "builders": [{
    "type": "docker",
    "image": "amazonlinux:latest",
    "export_path": "test.tar.gz"
}],
  "provisioners": [{
    "type": "shell",
    "inline": [
        "sudo useradd bizops"
    ]
  }]
}

It is a bit of an contrived example, but it fails just like my original. If I knew Go, I'd lend a hand in getting things completed. Is the problem limited to just inline shell provisioners? Is there any work-around? I would hate to have to go over to dockerfiles.

[WeatherGod]

I gave this branch a spin, and it did manage to attach onto the spun-up container image and tried to execute the script (that failed, but that is stemming from issues with the amazonlinux images not matching the AMIs).

[rickard-von-essen]

@WeatherGod sorry didn't really follow what you tested. This PR is only partially complete and require some more work before being complete.

[WeatherGod]

I followed the instructions in the contributions guide to go get the packer repository. I then fetched this branch, rebased it upon master, and fixed a couple tiny things (if len(foo) --> if len(foo) > 0). I then built the packer binary, and ran bin/packer build docker-test.json, which was the JSON file I posted above. It mostly worked. The failure I encountered was with the container not having a sudo command, surprisingly.

I'll be doing some more tests with it shortly, and I'll certainly provide more feedback if desired. Unfortunately, as I said, I have zero experience with Go (I am a Python dev, primarially), so I am afraid I can't really provide much help code-wise.

[WeatherGod]

So, I found an issue that is probably a result of the WIP nature of this PR. For my purposes to make the packer config the same between amazon-ebs and docker builders, I tweaked the run_command so that sudo -i -u ec2-user is used instead of /bin/bash. The docker container does indeed run that way (confirmed by manually attaching during -debug), but the provisioners are still executed as root.

[ktham]

I have the same issue as @dolobanko and ran into this pull request. @rickard-von-essen are you still working on this?

[rickard-von-essen]

@ktham Working on it might be a bit strong. I have very little time to spend on this right now, but I plan to finish it. The bigest problem is not just implementing the driver but that the config have to be migrated since it's tightly bound to the cli and that will require allot of work.

I'm glad for any assistance we can get on this.

[ktham]

I see, gotcha, when I get the time, I'll try to learn some Go and help chip in where/when I can. We'll just run Packer outside of Docker to workaround #4213 in the meantime.

[JMLX42]

Hi there!
Trying to use packer in docker-in-docker.
I was wondering if anyone was working on this?

[rickard-von-essen]

It's stalled. I plan to pick it up next year if no one steps up. There is quite some work needed to migrate a template.

[rickard-von-essen]

Closing this for now. I'll reopen when this is getting closer to done.

[ghostsquad]

Can this be re-opened? It doesn't make sense to close an issue that hasn't been resolved.

[rickard-von-essen]

This is a PR. I'll reopen it when I get back to work on it. My plan is to pick this up next year.

[bbrala]

@rickard-von-essen almost at the aniversary, any update? :)

[rickard-von-essen]

@bbrala 2018 hasn't ended yet 😏. Currently no ETA.

[bbrala]

Hehe, ok then 😄

[markmsmith]

Any plans to revisit this? I think I may be seeing a related issue with a file provisioner download hanging when running with Docker for Mac.

[rickard-von-essen]

@markmsmith Currently I don't have any time to work on such a big thing. And I think that it would be better to add a new builder (docker-api) to not have to mimic the behaviour and configuration of the current one.

[markmsmith]

ok, thanks. I'll open an issue to track the specific problem I'm seeing.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.