React App Compilation failing due to missing eslint cache file

[dylanrhysscott]

Can you reproduce the problem with latest npm?

Yes

Description

React app compilation is periodically / intermittently failing with the error due to a change in eslint-caching as per #1578. A cache file is missing in the node_modules directory

Expected behavior

App should compile with no errors

Actual behavior

The create-react-scripts throw the following error when compiling on file change:

Module build failed: Error: ENOENT: no such file or directory, open '/app/client/node_modules/.cache/eslint-loader/data.json'

On closer inspection this seems to happen when I rename / add a new file. However it happens so randomly it has been difficult to pin down.

Environment

Run these commands in the project folder and fill in their results:

1. npm ls react-scripts (if you haven’t ejected):

npm info it worked if it ends with ok
npm info using npm@4.1.2
npm info using node@v7.6.0
client@0.1.0 /app/client
`-- react-scripts@0.9.2

npm info ok

2. node -v:

v7.6.0

3. npm -v:

4.1.2

1. Operating system: OSX running Docker Image: https://hub.docker.com/r/dylanrhysscott/docker-react-dev/ which is based off of official node images

2. Browser and version: Chrome 56.0.2924.87 (64-bit)

Ping @Timer - Here's the issue. I can't provide my project as its on a private Git server internally but will do all I can to help troubleshoot. Hope this helps!

[dylanrhysscott]

One other thing I forgot to mention is - I'm using YARN rather than NPM directly. Could this also have an effect? My version of YARN is: 0.20.3

[viankakrisna]

hmm, is the process running in a container that reads host filesystem?

[dylanrhysscott]

@viankakrisna It's using a shared volume which mounts my source code yeah. I've been using this custom image for a while with a few other projects and not had issues. It downloads the latest version of the create-react-scripts at runtime. Is it worth doing a quick copy into the container avoiding the volume mount?

[viankakrisna]

Yup, can you try that? also, does setting the env CHOKIDAR_USEPOLLING=true helps? I think there are some delay between the filesystem and the watcher process. It helps in the vm, i don't know if it's also true in the docker

[dylanrhysscott]

@viankakrisna Hmmm just inspected my node_modules directory and the required json file is there. I did run an automated build which grabbed a fresh copy of the node base image just to rule that out. Looking at the last update to the node base image I was up to date anyway. The eslint-loader directory is there where it was missing before. Reinstalling the node_modules and it is still there. Possible my usage of YARN didn't actually install all the required folders? Or the watcher wasn't creating it due to volume delays? I will keep messing until I can get a definitive cause for you if there is one

[dylanrhysscott]

I appreciate the docker usage is probably outside your remit :)

[viankakrisna]

yup, can you also try the CHOKIDAR_USEPOLLING? need to rule out all option before fire up a docker image to dig on that. Here's the link to do that /~https://github.com/facebookincubator/create-react-app/blob/5155797983476260a6e6e8d93ca38b43233d82de/packages/react-scripts/template/README.md#npm-start-doesnt-detect-changes

[gaearon]

Looking at the eslint-loader implementation of caching, it seems to miss error handling in a few crucial places. For example, when writing the file, or when potentially creating a directory.

If babel-loader can't create a directory in node_modules, it will fall back to OS directory. However, eslint-loader could fail here, and the fallback will never kick in.

If babel-loader's cache gets deleted while the server is running, it will still attempt to create a directory before writing a file. eslint-loader will only do this once and not for subsequent writes even though the cache might have been deleted. And failing to write the file on subsequent writes will crash it instead of trying to fall back to OS directory like babel-loader does.

It seems like eslint-loader implementation of caching could use some error handling work. It would be awesome if someone could fix it up in a few days, but if not, we’ll likely revert the option because it’s just not as reliable as the Babel option.

[dylanrhysscott]

@gaearon Hi Dan, Thanks for the breakdown. I currently can't replicate this issue as the eslint-loader directory is being created with data.json seems to be an edge case. Will keep an eye on this issue and will update my images when a fix is available.

[RohovDmytro]

Bumped into this issue. Super frustrating 😞

[RohovDmytro]

From what I noticed: this issues consistently appears after yarn-installing new module.

[viankakrisna]

confirmed that this is consistently happening after yarn installing new module. I'm digging on it right now

[gaearon]

Going to revert this in the next version: #1665.
Unless you can figure out a patch to eslint-loader to fix this 😉

[viankakrisna]

here's the patch for the unwritable cache file
https://gist.github.com/viankakrisna/76ee7702e3592ab743cea63787e47295
@dylanrhysscott @rogovdm can you confirm that replacing index.js of eslint-loader with this file solve your problems?
@gaearon need your feedback on this too 😄

^^OUTDATED

Please use /~https://github.com/viankakrisna/eslint-loader/tree/port_babel_loader_fs_cache instead. You need to replace eslint-loader folder in react-scripts/node_modules/eslint-loaderwith this repo and do npm install there.

Note: you need to restart the dev server so webpack can read the patch

[dylanrhysscott]

@viankakrisna Hey I've tested this and it appears to work. I tested by running yarn add randomstring then compiling files. I also tried the inverse yarn remove randomstring and compiling files. Didn't get any compilation errors. Something that does concern me however is after I applied the fix and during the startup of my app (running YARN) my Macbook bombed out on a fatal exception (looked like a memory issue). Probably a specific of me using Docker on my setup. Never happened to me before. Just thought I'd mention it. Apart from that looks good! Nice work

[viankakrisna]

@dylanrhysscott Great! Glad that it helps. Not sure what causes the bomb though 😭

If anybody wants to try a reimplementation of eslint-loader cache with babel-loader's fs-cache, you can try it here webpack-contrib/eslint-loader#159 Feedbacks are gladly appreciated

[chrisziegler]

@viankakrisna Not sure if helpful, but I had the same issue after using yarn to add purecss. I tried your patch, and in my case it did not fix the issue. I don't have a great deal of dev experience, so please take that into consideration. I'm on Windows. I had to resort to deleting node_modules and reinstalling. Thanks to everyone for their contributions here.

[gaearon]

I tried your patch, and in my case it did not fix the issue.

Which of two patches in this thread did you try? Have you restated the server after applying it?

[chrisziegler]

@gaearon I used the patch located at: https://gist.github.com/viankakrisna/76ee7702e3592ab743cea63787e47295

I'm trying to remember, but I think it's possible that I overlooked restarting the server, as I recall spending some of my time trying to decipher the error message provided by create-react-app in the original browser window. In retrospect that should have been obvious. My apologies, you can most likely disregard my input.

[dylanrhysscott]

@viankakrisna Hi just ran into this error again after YARN adding a package. Still had the fix applied and it threw the same error. Looks like an issue with larger more complete YARN packages being added than the one I initially tested. After adding a package my .cache directory appears to be missing again :(. Any ideas?

[viankakrisna]

@dylanrhysscott I've updated the gist after the last time i publish it, can you try it?

Also, do you check that the patched file is intact inside the node_modules? I'm beginning to think that every time you run yarn it will overwrite the folders inside node_modules with its own cached version.

I've published another fix in here /~https://github.com/viankakrisna/eslint-loader/tree/port_babel_loader_fs_cache which uses loader-fs-cache ported from babel-loader's fs-cache and need some testing. Would you try it? You need to manually clone it to react-scripts node_modules and run yarn / npm install though because of the added dependencies.

The pr is here webpack-contrib/eslint-loader#159. Need to port the test from babel-loader to let the CI passes

@gaearon I'm beginning to think that it's wiser to do revert and wait until eslint-loader cache implementation tested and tried for a while before enabling it. It's too frustrating for the user to experience this...

And i'm sorry for the silly pull request that started it all...

[dylanrhysscott]

@viankakrisna Confirmed looks like YARN is overwriting the patch file....I've cloned your fixes into my create-react-scripts/node_modules and done an npm i. I had to also install eslint due to an unmet deps warning for your fix. For good measure I've also started with a clean copy of my node_modules for my project. With the watch task running I've added and removed the package that triggered the original eslint-loader error. The error has triggered when I modify one of my react components as part of my project. Looking in my node_modules the cache directory is missing. Not sure if I'm missing anything here? Cheers for the help on this!

[dylanrhysscott]

Hang on! Looks like my project is using a different directory from where I applied the fix... let me test :)

[dylanrhysscott]

@viankakrisna OK so I've successfully applied the fix now in the .cache directory I had a babel-loader related directory. Everything was working on initial install, compiling fine. I've removed a package with YARN and the .cache directory is now gone and the error is back with a vengeance. I think a revert might be in order?

[gaearon]

The only fix I want somebody to try is /~https://github.com/viankakrisna/eslint-loader/tree/port_babel_loader_fs_cache. Did anyone try it yet?

[viankakrisna]

@dylanrhysscott you need to clone it in your react-scripts/node_modules folder inside your application node_modules folder, overwriting eslint-loader folder there.

so the path is like this YOURAPP/node_modules/react-scripts/node_modules/eslint-loader and do npm i there.

I don't think it will persist while yarn is always overwriting the folders though.
Need to find another way to patch it.

And yes, revert should be in order, currently i'm stuck with tests to get the pr of porting fs-cache. merged.

[dylanrhysscott]

@viankakrisna Cheers I figured that after I did it :) Yeah YARN is overwriting it still @gaearon I think that was the one I tried once I got it in the correct directory. Let me know if there is any more tests you'd like me to run!

[viankakrisna]

I've setup a repo of react-scripts with eslint-loader-fs-cache (a published npm package for the current pr ). Just replace your devDependencies from

"devDependencies": {
    "react-scripts": "0.9.2"
  }

To

"devDependencies": {
    "react-scripts": "viankakrisna/react-scripts-eslint-loader-fs-cache"
  },

and do yarn or npm install.

[gaearon]

I’m going to move ahead with the revert for now, but happy to take another PR for 0.9.4 when this is sorted out.

[dylanrhysscott]

@viankakrisna Sweet! Swapping out those devDeps to your repo seems to have fixed it! I have added a few packages and can still compile. Same in reverse 👍

[viankakrisna]

@dylanrhysscott I will keep the repo until 0.9.3 released. I naively based this repo on the master branch. which uses webpack2...

[gaearon]

Yea, need to be careful with that 😛 Master branch is very unstable.

[dylanrhysscott]

@gaearon @viankakrisna Thanks for your help guys. Much appreciated!

[gaearon]

I landed a revert of this in 0.9.3.
Please update to it while we're fixing the underlying issue.

[viankakrisna]

deleted react-scripts patch repo and gists so we can focus on /~https://github.com/viankakrisna/eslint-loader/tree/port_babel_loader_fs_cache
@dylanrhysscott please update to 0.9.3!

[dylanrhysscott]

@viankakrisna Cheers will update as soon as I can. Looks like I'm having issues with a bug in YARN upgrades atm. It really hasn't been my week for bugs :P Thanks!

[dylanrhysscott]

@viankakrisna Just bumped the version on my project - all resolved!