docker manifest commands don't work on 18.03.0-ce-rc3

[imdurgadas]

Description:
When using Docker 18.03.0-ce-rc3, docker manifest commands fail with "docker manifest is only supported when experimental cli features are enabled", regardless of the setting of experimental features in the /etc/docker/daemon.json

Steps to reproduce the issue:

Install Docker 18.03.0-ce-rc3 from the edge channel
Enable experimental features: (https://docs.docker.com/engine/reference/commandline/dockerd/#description)
On Linux, as root, edit /etc/docker/daemon.json to contain {"experimental":true}. Then, as root, run service docker restart to apply the setting.

{
        "experimental" : true,
        "storage-driver" : "devicemapper",
        "storage-opts" : [
                "dm.basesize=150G"
        ]
}

Describe the results you received:
$ docker manifest --help
docker manifest is only supported when experimental cli features are enabled

Describe the results you expected:
The help text for the docker manifest commands to be displayed

Output of docker version:

Client:
Client:
Version: 18.03.0-ce-rc3
API version: 1.37
Go version: go1.9.4
Git commit: e730959
Built: Thu Mar 8 20:18:59 2018
OS/Arch: linux/s390x
Experimental: false
Orchestrator: swarm

Server:
Engine:
Version: 18.03.0-ce-rc3
API version: 1.37 (minimum version 1.12)
Go version: go1.9.4
Git commit: e730959
Built: Thu Mar 8 20:17:58 2018
OS/Arch: linux/s390x
Experimental: true

Output of docker info:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 1
Server Version: 18.03.0-ce-rc3
Storage Driver: devicemapper
Pool Name: docker-0:40-94300172-pool
Pool Blocksize: 65.54kB
Base Device Size: 161.1GB
Backing Filesystem: xfs
Udev Sync Supported: true
Data file: /dev/loop0
Metadata file: /dev/loop1
Data loop file: /data/docker_14032018/devicemapper/devicemapper/data
Metadata loop file: /data/docker_14032018/devicemapper/devicemapper/metadata
Data Space Used: 222.4MB
Data Space Total: 107.4GB
Data Space Available: 107.2GB
Metadata Space Used: 815.1kB
Metadata Space Total: 2.147GB
Metadata Space Available: 2.147GB
Thin Pool Minimum Free Space: 10.74GB
Deferred Removal Enabled: true
Deferred Deletion Enabled: true
Deferred Deleted Device Count: 0
Library Version: 1.02.110 (2015-10-30)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfd04396dc68220d1cecbe686a6cc3aa5ce3667c
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
apparmor
Kernel Version: 4.4.0-116-generic
Operating System: Ubuntu 16.04.4 LTS
OSType: linux
Architecture: s390x
CPUs: 2
Total Memory: 7.666GiB
Name: ecos0055
ID: 7LDY:BRSI:4CJA:LQK7:6CXI:ER6Z:QSQ5:OLY2:UIEK:HMTT:N2MB:UK37
Docker Root Dir: /data/docker_14032018
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.
Use --storage-opt dm.thinpooldev to specify a custom block storage device.
WARNING: No swap limit support

[vdemeester]

@imdurgadas thanks for the report. As the message says "docker manifest is only supported when experimental cli features are enabled", it's on the cli side that the experimental configuration is required.

If you update your $HOME/.docker/config.json to sthg like the following, it should work :

{
        "experimental": "enabled"
}

[imdurgadas]

$HOME/.docker/config.json is another file that we need to maintain in addition to /etc/docker/daemon.json ?

I followed the approach to set experimental flag as mentioned in the documentation link
https://docs.docker.com/engine/reference/commandline/dockerd/#description

So do we have a documentation defect ?

[imdurgadas]

@vdemeester Thanks. Performing the steps you mentioned fixed my issue.

[vdemeester]

$HOME/.docker/config.json is another file that we need to maintain in addition to /etc/docker/daemon.json ?

This is the cli configuration file — and you can have 1 cli talking to "any number" of daemon so yeah those are 2 files to maintain. That said I do agree with "So do we have a documentation defect ?" 👼

• we may want to make the error message more clear
• or add a command to easily set the cli in experimental (and be clear on that in the error message)
• or activate cli experimental when the daemon we point to is set to experimental

There is definitely room for improvements 👼

/cc @dnephin @thaJeztah

[dnephin]

I propose we remove the config field and make "cli experimental" documentation only. Anyone can run the command, but --help and other docs should make it clear that the feature is experimental and the UI may change.

[ahh-docker]

@thaJeztah Given this is for a RC on 18.03, can you please let me know what's still valid with this ticket, or if should be closed. Thanks!

[olljanat]

@ahh-docker manifest is still experimental feature until PR #1355 is merged and that one waiting for that #866 and #1282 will be merged and those are waiting for that someone of maintainers will look about them.

[thaJeztah]

This should be fixed now that we enable experimental features by default in docker 20.10 (see #2774)

[koraniar]

I have this problem with docker 20.10
I am using

- setup_remote_docker:
          version: 20.10.7
- run:
          name: Test Image
          command: |
            if [ "${CIRCLE_BRANCH}" == "master" ]; then
              docker login -u $DOCKER_USER -p $DOCKER_PASSWORD

              docker manifest inspect repo/api:7.2.0
              if [ "$?" -ne "0" ]; then
                echo "The image does not exists"
              fi
            fi

But the build fails with the message

docker manifest inspect is only supported on a Docker cli with experimental cli features enabled

And following the docs to enable the experimental features fails with the error

/bin/bash: EOF: No such file or directory

[thaJeztah]

@koraniar check if the CLI that's running in your CI is also docker 20.10.x. I see you setup a remote docker 20.10, which likely only installs an engine 20.10, but the cli might still be on a different version.