Failure to Start on 3.0.2 with "user 101 not found"

[rehashedsalt]

Current Behavior

Attempting to start the container on my production instance fails with the following error:

[...]
▶️  Running the startup script /opt/netbox/startup_scripts/340_power_feeds.py
✅ Initialisation is done.
⏳ Waiting for control socket to be created... (1/10)
2021/09/18 21:26:58 [alert] 7#7 getpwnam("101") failed, user "101" not found

I'm also not the only one observing this problem: #585

Expected Behavior

The container starts.

Debug Information

The output of docker-compose version: N/A
The output of docker version:

Client: Docker Engine - Community
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:54:27 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:52:33 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

The command you used to start the project: It's complicated. Here's the Ansible task I use. The rest of the repository contains configuration for the rest of my environment, and I'd be happy to provide details if anything is unclear.

The output of docker inspect netboxcommunity/netbox:latest --format "{{json .Config.Labels}}":

{
   "BUILD_REASON":"netbox-docker",
   "NETBOX_GIT_BRANCH":"HEAD",
   "NETBOX_GIT_REF":"b55c85b2afdb0a6fdbbabdd38b1eed2eed333153",
   "NETBOX_GIT_URL":"/~https://github.com/netbox-community/netbox.git",
   "ORIGINAL_TAG":"docker.io/netboxcommunity/netbox:v3.0.2",
   "org.label-schema.build-date":"2021-09-17T09:36+00:00",
   "org.label-schema.description":"A container based distribution of NetBox, the free and open IPAM and DCIM solution.",
   "org.label-schema.name":"NetBox Docker",
   "org.label-schema.schema-version":"1.0",
   "org.label-schema.url":"/~https://github.com/netbox-community/netbox-docker",
   "org.label-schema.usage":"/~https://github.com/netbox-community/netbox-docker/wiki",
   "org.label-schema.vcs-ref":"6a52a48b71187eabb16db2149060cffc159f5f72",
   "org.label-schema.vcs-url":"/~https://github.com/netbox-community/netbox-docker.git",
   "org.label-schema.vendor":"The netbox-docker contributors.",
   "org.label-schema.version":"1.4.0",
   "org.opencontainers.image.authors":"The netbox-docker contributors.",
   "org.opencontainers.image.created":"2021-09-17T09:36+00:00",
   "org.opencontainers.image.description":"A container based distribution of NetBox, the free and open IPAM and DCIM solution.",
   "org.opencontainers.image.documentation":"/~https://github.com/netbox-community/netbox-docker/wiki",
   "org.opencontainers.image.licenses":"Apache-2.0",
   "org.opencontainers.image.revision":"6a52a48b71187eabb16db2149060cffc159f5f72",
   "org.opencontainers.image.source":"/~https://github.com/netbox-community/netbox-docker.git",
   "org.opencontainers.image.title":"NetBox Docker",
   "org.opencontainers.image.url":"/~https://github.com/netbox-community/netbox-docker",
   "org.opencontainers.image.vendor":"The netbox-docker contributors.",
   "org.opencontainers.image.version":"1.4.0"
}

The output of docker-compose logs netbox: N/A. Relevant log section is at the top of the issue.

The output of cat docker-compose.override.yml: N/A.

Additional Details

I was unable to replicate the issue using just the docker-compose provided in the repo, but observed the issue in my lab environment. This suggests that the issue manifests over the course of Netbox usage and/or when a particular feature is used.

[jcollie]

Does anyone have a workaround for this to get back running? Fortunately I had the 1.3.0 version of 3.0.2 cached so I was able to downgrade to that. netboxcommunity/netbox@sha256:cba93649cf093f756d938ef413ac36c9d03d85c02383471ef42e82ca99d2c23f is the digest for 1.3.0/3.0.2 that is still working for me.

[heitormotta]

Does anyone have a workaround for this to get back running? Fortunately I had the 1.3.0 version of 3.0.2 cached so I was able to downgrade to that. netboxcommunity/netbox@sha256:cba93649cf093f756d938ef413ac36c9d03d85c02383471ef42e82ca99d2c23f is the digest for 1.3.0/3.0.2 that is still working for me.

I changed the last 2 lines of launch-netbox.sh from "--user 101 --group 0" to "--user unit --group unit" and it worked for me.

[kkthxbye-code]

@rehashedsalt try to add the user option to your ansible task, equivilent to this line in the compose file:

/~https://github.com/netbox-community/netbox-docker/blob/release/docker-compose.yml#L11

I'm assuming that it fails because the user doesn't exist according to getpwnam(3).

Ansible task documentation for the option:

https://docs.ansible.com/ansible/latest/collections/community/docker/docker_container_module.html#parameter-user

[rehashedsalt]

@rehashedsalt try to add the user option to your ansible task, equivilent to this line in the compose file:

/~https://github.com/netbox-community/netbox-docker/blob/release/docker-compose.yml#L11

That change seems to have mitigated the issue.

[rehashedsalt]

Is this the proper solution, though? It looks to me that the purpose of #546 is to drop to a lesser user transparently when the container is spawned as root. That functionality, if I understand it correctly, is currently broken.

[tobiasge]

Could you test your deployment with the changes from PR #589 (You have to build an image with theses changes)?

Side note: It's not recommended to run the container as root. Also the user ID can be anything as long as the group ID is 0.

[Spiritus44]

Hi @tobiasge

I had the same issue as @rehashedsalt described.

This morning I built a docker image using your PR (please see gdelanoe/netbox:pr-589).
It's fix the issue for me.

Thanks a lot!