Skip to content
This repository has been archived by the owner on May 4, 2021. It is now read-only.

Forgive the case when a whiteout file trying to cover nonexistent path #358

Merged
merged 2 commits into from
Mar 15, 2021

Conversation

yiranwang52
Copy link
Contributor

@yiranwang52 yiranwang52 commented Mar 10, 2021

This error happens sometimes with python package's .dist-info or .pth files. For example, installing aiohttp package in a conda environment, then when applying cache of layer, this error message happens often:

[error] 2021-03-09 17:36:12.468002: failed to execute build plan: execute stage: build stage 0: build node: apply cache: untar reader: add hdr from tar to layer: update memfs with file /home/ray/anaconda3/lib/python3.7/site-packages/.wh.aiohttp-3.7.4.dist-info: whiteout nonexistent path /home/ray/anaconda3/lib/python3.7/site-packages/aiohttp-3.7.4.dist-info

Those files were probably created or removed asynchronously.
It doesn't hurt to ignore this case; Change to a warning message instead.

@yiranwang52 yiranwang52 requested review from evelynl94 and sema March 10, 2021 02:22
Copy link
Contributor

@sema sema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed this change with @yiranwang52

We both agree that this is likely related to async operations being done on these files, which Makisu does not currently handle well.

We don't fully understand how this may then manifest itself exactly like this when re-applying the cache layers. There may exist a race in how the cache layers are calculated when we have async operations.

Approving this to unblock the builds failing due to this, as the error in itself is not dangerous.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants