Nginx-unit unable to cache big client queries

[maximumG]

Current Behavior

When we try to send a big bulk update query to the Netbox API, Nginx-Unit respond with a HTTP 503 error code.

The big client queries are actually cached to a temporary directory /opt/unit/tmp/ before sending the query to WSGI. This directory is not writable by nginx-unit workers (username: unit / userid: 101)

bash-5.1# ls -lah
total 0      
drwxrwxr-x    4 root     root          30 Apr 28 06:04 .
drwxr-xr-x    1 root     root          32 Apr 28 06:04 ..
drwxrwxr-x    2 root     root           6 Apr 28 06:04 state
drwxrwxr-x    2 root     root           6 Apr 28 06:04 tmp

Expected Behavior

To process big client queries, nginx-unit workers should be able to write the query payload into the /opt/unit/tmp/ directory. The temp folder should have the following owner/group owner:

bash-5.1# ls -lah
total 0      
drwxrwxr-x    4 root     root          30 Apr 28 06:04 .
drwxr-xr-x    1 root     root          32 Apr 28 06:04 ..
drwxrwxr-x    2 root     root           6 Apr 28 06:04 state
drwxrwxr-x    2 root     unit           6 Apr 28 06:04 tmp

Suggested Solution

Change ownership of directory /opt/unit/tmp to be owned by root:unitin the Dockerfile

# Must set permissions for '/opt/netbox/netbox/media' directory
# to g+w so that pictures can be uploaded to netbox.
RUN mkdir -p static /opt/unit/state/ /opt/unit/tmp/ \
      && chmod -R g+w media /opt/unit/ \
      && chown -R root:unit /opt/unit/tmp/
      && SECRET_KEY="dummy" /opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py collectstatic --no-input

Debug Information

docker-compose version

docker-compose version 1.27.4, build 40524192
docker-py version: 4.3.1
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

docker version

Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:02:36 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:01:11 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

docker inspect netboxcommunity/netbox:v2.11

{
  "BUILD_REASON": "alpine netbox netbox-docker",
  "NETBOX_GIT_BRANCH": "HEAD",
  "NETBOX_GIT_REF": "1abaa5e60c2ecc15ae57db978d389023c136bdf7",
  "NETBOX_GIT_URL": "/~https://github.com/netbox-community/netbox.git",
  "ORIGINAL_TAG": "docker.io/netboxcommunity/netbox:v2.11.2",
  "org.label-schema.build-date": "2021-04-28T05:53+00:00",
  "org.label-schema.description": "A container based distribution of NetBox, the free and open IPAM and DCIM solution.",
  "org.label-schema.name": "NetBox Docker",
  "org.label-schema.schema-version": "1.0",
  "org.label-schema.url": "/~https://github.com/netbox-community/netbox-docker",
  "org.label-schema.usage": "/~https://github.com/netbox-community/netbox-docker/wiki",
  "org.label-schema.vcs-ref": "676ca798cd684ce30fca2692180e680d2becf76e",
  "org.label-schema.vcs-url": "/~https://github.com/netbox-community/netbox-docker.git",
  "org.label-schema.vendor": "The netbox-docker contributors.",
  "org.label-schema.version": "1.2.0",
  "org.opencontainers.image.authors": "The netbox-docker contributors.",
  "org.opencontainers.image.created": "2021-04-28T05:53+00:00",
  "org.opencontainers.image.description": "A container based distribution of NetBox, the free and open IPAM and DCIM solution.",
  "org.opencontainers.image.documentation": "/~https://github.com/netbox-community/netbox-docker/wiki",
  "org.opencontainers.image.licenses": "Apache-2.0",
  "org.opencontainers.image.revision": "676ca798cd684ce30fca2692180e680d2becf76e",
  "org.opencontainers.image.source": "/~https://github.com/netbox-community/netbox-docker.git",
  "org.opencontainers.image.title": "NetBox Docker",
  "org.opencontainers.image.url": "/~https://github.com/netbox-community/netbox-docker",
  "org.opencontainers.image.vendor": "The netbox-docker contributors.",
  "org.opencontainers.image.version": "1.2.0"
}

docker-compose logs netbox

"2021/04:28 14:34:03 [alert] 216#628 *4723 mkstemp (/opt/unit/tmp//req-XXXXXXX) failed (13: Permission denied\n)"

[cimnine]

With which user do you run the NetBox Docker container? Also uid 101, or 0 (i.e. root)?

[maximumG]

I have some constraints to run the container (pod) as root (UID 0) inside a K8S Cluster.

Are the permissions of dir /opt/unit/tmp changed at runtime If the pod or container is ran as UID 101 ?

[cimnine]

Unit changes it's permissions at runtime if it runs as root aparently, as described in #509 (comment). This does not happen if it does not run as root.