Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wallaby fails to catch app bootstrapping errors that Jest properly identifies #2701

Closed
gCardinal opened this issue May 13, 2021 · 2 comments
Closed

Wallaby fails to catch app bootstrapping errors that Jest properly identifies #2701

gCardinal opened this issue May 13, 2021 · 2 comments

Comments

@gCardinal
Copy link

Issue description or question

I'm using WallabyJS to test a NestJS application. I have a test that simply bootstraps the application and validates that it properly booted. Jest will fail the test when I have a bootstrap issue, but Wallaby does not catch those errors.

I did a reproduction repository here: /~https://github.com/gCardinal/wallaby-jest-issue.

I also confirmed that the issue happens when Wallaby is set to use automatic configuration.

Wallaby diagnostics report

{
  editorVersion: 'PhpStorm 2021.1.1',
  pluginVersion: '1.0.216',
  editorType: 'IntelliJ',
  osVersion: 'linux 5.11.0-7612-generic',
  nodeVersion: 'v15.9.0',
  coreVersion: '1.0.1070',
  config: {
    diagnostics: {
      jest: {
        config: {
          configs: [
            {
              automock: false,
              cache: true,
              cacheDirectory: '/tmp/jest_rs',
              clearMocks: false,
              coveragePathIgnorePatterns: [ '/node_modules/' ],
              cwd: '<homeDir>/dev/station/wallaby-repro-repo',
              dependencyExtractor: undefined,
              detectLeaks: undefined,
              detectOpenHandles: undefined,
              displayName: undefined,
              errorOnDeprecated: false,
              extraGlobals: [],
              filter: undefined,
              forceCoverageMatch: [],
              globalSetup: undefined,
              globalTeardown: undefined,
              globals: {},
              haste: { computeSha1: false, throwOnModuleCollision: false },
              injectGlobals: true,
              moduleDirectories: [ 'node_modules' ],
              moduleFileExtensions: [ 'js', 'json', 'ts' ],
              moduleLoader: undefined,
              moduleNameMapper: [],
              modulePathIgnorePatterns: [],
              modulePaths: undefined,
              name: 'fd942a9e1e3d6819fa679e4699105319',
              prettierPath: 'prettier',
              resetMocks: false,
              resetModules: false,
              resolver: undefined,
              restoreMocks: false,
              rootDir: '<homeDir>/dev/station/wallaby-repro-repo/src',
              roots: [ '<homeDir>/dev/station/wallaby-repro-repo/src' ],
              runner: 'jest-runner',
              setupFiles: [],
              setupFilesAfterEnv: [],
              skipFilter: false,
              skipNodeResolution: undefined,
              slowTestThreshold: 5,
              snapshotResolver: undefined,
              snapshotSerializers: [],
              testEnvironment: '<homeDir>/dev/station/wallaby-repro-repo/node_modules/jest-environment-node/build/index.js',
              testEnvironmentOptions: {},
              testLocationInResults: false,
              testMatch: [],
              testPathIgnorePatterns: [ '/node_modules/' ],
              testRegex: [ '.*\\.spec\\.ts$' ],
              testRunner: '<homeDir>/dev/station/wallaby-repro-repo/node_modules/jest-jasmine2/build/index.js',
              testURL: 'http://localhost',
              timers: 'real',
              transform: [ [ '^.+\\.(t|j)s$', '<homeDir>/dev/station/wallaby-repro-repo/node_modules/ts-jest/dist/index.js', {} ] ],
              transformIgnorePatterns: [ '/node_modules/', '\\.pnp\\.[^\\/]+$' ],
              unmockedModulePathPatterns: undefined,
              watchPathIgnorePatterns: []
            }
          ],
          globalConfig: {
            bail: 0,
            changedFilesWithAncestor: false,
            changedSince: undefined,
            collectCoverage: false,
            collectCoverageFrom: [ '**/*.(t|j)s' ],
            collectCoverageOnlyFrom: undefined,
            coverageDirectory: '<homeDir>/dev/station/wallaby-repro-repo/coverage',
            coverageProvider: 'babel',
            coverageReporters: [ 'json', 'text', 'lcov', 'clover' ],
            coverageThreshold: undefined,
            detectLeaks: undefined,
            detectOpenHandles: undefined,
            enabledTestsMap: undefined,
            errorOnDeprecated: false,
            expand: false,
            filter: undefined,
            findRelatedTests: false,
            forceExit: false,
            globalSetup: undefined,
            globalTeardown: undefined,
            json: false,
            lastCommit: false,
            listTests: undefined,
            logHeapUsage: false,
            maxConcurrency: 5,
            maxWorkers: 7,
            noSCM: undefined,
            noStackTrace: false,
            nonFlagArgs: undefined,
            notify: false,
            notifyMode: 'failure-change',
            onlyChanged: false,
            onlyFailures: false,
            outputFile: undefined,
            passWithNoTests: undefined,
            projects: [],
            replname: undefined,
            reporters: undefined,
            rootDir: '<homeDir>/dev/station/wallaby-repro-repo/src',
            runTestsByPath: false,
            silent: undefined,
            skipFilter: false,
            testFailureExitCode: 1,
            testNamePattern: undefined,
            testPathPattern: '',
            testResultsProcessor: undefined,
            testSequencer: '<homeDir>/dev/station/wallaby-repro-repo/node_modules/@jest/test-sequencer/build/index.js',
            testTimeout: undefined,
            updateSnapshot: 'new',
            useStderr: false,
            verbose: undefined,
            watch: false,
            watchAll: false,
            watchPlugins: undefined,
            watchman: true
          },
          hasDeprecationWarnings: false,
          wallaby: {
            roots: [ 'src' ],
            watchPathIgnorePatterns: [ '/node_modules/', '\\./dist/|\\./build/|\\./coverage/|/\\..+/' ],
            testPathIgnorePatterns: [ '/node_modules/', '\\./dist/|\\./build/|\\./coverage/|/\\..+/' ],
            testMatch: [],
            testRegex: [ '.*\\.spec\\.ts$' ]
          }
        }
      }
    },
    testFramework: { version: 'jest@24.8.0', configurator: 'jest@24.8.0', reporter: 'jest@24.8.0', starter: 'jest@24.8.0', autoDetected: true },
    filesWithCoverageCalculated: [ '**/*.(t|j)s' ],
    filesWithNoCoverageCalculated: [],
    globalSetup: false,
    micromatch: true,
    files: [
      { pattern: '/node_modules/', regexp: /\/node_modules\//, ignore: true, trigger: true, load: true },
      { pattern: '\\./dist/|\\./build/|\\./coverage/|/\\..+/', regexp: /\.\/dist\/|\.\/build\/|\.\/coverage\/|\/\..+\//, ignore: true, trigger: true, load: true },
      { pattern: '**/**', ignore: false, trigger: true, load: true, order: 1 },
      { pattern: 'src/**', ignore: false, trigger: true, load: true, order: 2 },
      { pattern: '.*\\.spec\\.ts$', regexp: /.*\.spec\.ts$/, ignore: true, trigger: true, load: true }
    ],
    tests: [
      { pattern: '/node_modules/', regexp: /\/node_modules\//, ignore: true, trigger: true, load: true, test: true },
      { pattern: '\\./dist/|\\./build/|\\./coverage/|/\\..+/', regexp: /\.\/dist\/|\.\/build\/|\.\/coverage\/|\/\..+\//, ignore: true, trigger: true, load: true, test: true },
      { pattern: '.*\\.spec\\.ts$', regexp: /.*\.spec\.ts$/, ignore: false, trigger: true, load: true, test: true, order: 3 }
    ],
    runAllTestsInAffectedTestFile: false,
    updateNoMoreThanOneSnapshotPerTestFileRun: false,
    addModifiedTestFileToExclusiveTestRun: true,
    compilers: {},
    preprocessors: {},
    maxConsoleMessagesPerTest: 100,
    autoConsoleLog: true,
    delays: { run: 0, edit: 100, update: 0 },
    workers: { initial: 0, regular: 0, recycle: false },
    teardown: undefined,
    hints: {
      ignoreCoverage: '__REGEXP /ignore coverage|istanbul ignore/',
      ignoreCoverageForFile: '__REGEXP /ignore file coverage/',
      commentAutoLog: '?',
      testFileSelection: { include: '__REGEXP /file\\.only/', exclude: '__REGEXP /file\\.skip/' }
    },
    automaticTestFileSelection: true,
    runSelectedTestsOnly: false,
    mapConsoleMessagesStackTrace: false,
    extensions: {},
    env: {
      type: 'node',
      params: {},
      runner: '/home/linuxbrew/.linuxbrew/Cellar/node/15.9.0/bin/node',
      viewportSize: { width: 800, height: 600 },
      options: { width: 800, height: 600 },
      bundle: true
    },
    reportUnhandledPromises: true,
    slowTestThreshold: 75,
    lowCoverageThreshold: 80,
    loose: true,
    configCode: 'auto.detect#708642995'
  },
  packageJSON: {
    dependencies: {
      '@nestjs/common': '^7.6.15',
      '@nestjs/config': '^0.6.3',
      '@nestjs/core': '^7.6.15',
      '@nestjs/mapped-types': '*',
      '@nestjs/microservices': '^7.6.15',
      '@nestjs/platform-express': '^7.6.15',
      '@nestjs/terminus': '^7.1.2',
      'amqp-connection-manager': '^3.2.2',
      amqplib: '^0.7.1',
      'fastify-swagger': '^4.7.0',
      'get-port': '^5.1.1',
      mockdate: '^3.0.5',
      'reflect-metadata': '^0.1.13',
      rimraf: '^3.0.2',
      rxjs: '^6.6.6',
      'ts-mockito': '^2.6.1',
      'type-fest': '^1.1.0',
      'wait-for-expect': '^3.0.2',
      winston: '^3.3.3',
      ws: '^7.4.5'
    },
    devDependencies: {
      '@nestjs/cli': '^7.6.0',
      '@nestjs/schematics': '^7.3.0',
      '@nestjs/testing': '^7.6.15',
      '@types/express': '^4.17.11',
      '@types/faker': '^5.5.4',
      '@types/jest': '^26.0.22',
      '@types/lodash': '^4.14.168',
      '@types/node': '^14.14.36',
      '@types/supertest': '^2.0.10',
      '@types/ws': '^7.4.2',
      '@typescript-eslint/eslint-plugin': '^4.19.0',
      '@typescript-eslint/parser': '^4.19.0',
      eslint: '^7.22.0',
      'eslint-config-prettier': '^8.1.0',
      'eslint-plugin-prettier': '^3.3.1',
      faker: '^5.5.3',
      jest: '^26.6.3',
      lodash: '^4.17.21',
      prettier: '^2.2.1',
      supertest: '^6.1.3',
      'ts-jest': '^26.5.4',
      'ts-loader': '^8.0.18',
      'ts-node': '^9.1.1',
      'tsconfig-paths': '^3.9.0',
      typescript: '^4.2.3'
    }
  },
  fs: { numberOfFiles: 24 },
  debug: [
    '2021-05-13T13:11:08.258Z angular/cli config Angular CLI not found.\n',
    '2021-05-13T13:11:08.361Z jest/config Detected Jest.\n',
    '2021-05-13T13:11:08.362Z jest/config Configured Jest.\n',
    '2021-05-13T13:11:08.363Z project Wallaby Node version: v15.9.0\n',
    '2021-05-13T13:11:08.363Z project Wallaby config: <homeDir>/dev/station/wallaby-repro-repo/auto.detect\n',
    '2021-05-13T13:11:08.420Z project File cache: <homeDir>/.cache/JetBrains/PhpStorm2021.1/wallaby/projects/c45435bb66c10aec\n',
    '2021-05-13T13:11:08.482Z uiService Listening port 51235\n',
    '2021-05-13T13:11:08.484Z project package.json file change detected, invalidating local cache\n',
    '2021-05-13T13:11:08.494Z workers Parallelism for initial run: 6, for regular run: 3\n',
    '2021-05-13T13:11:08.494Z workers Starting run worker instance #0\n',
    '2021-05-13T13:11:08.494Z workers Starting run worker instance #1\n',
    '2021-05-13T13:11:08.494Z workers Starting run worker instance #2\n',
    '2021-05-13T13:11:08.494Z workers Starting run worker instance #3\n',
    '2021-05-13T13:11:08.494Z workers Starting run worker instance #4\n',
    '2021-05-13T13:11:08.495Z workers Starting run worker instance #5\n',
    '2021-05-13T13:11:08.495Z workers Web server is listening at 41591\n',
    '2021-05-13T13:11:08.495Z project File cache requires some updates, waiting required files from IDE\n',
    '2021-05-13T13:11:08.545Z project Stopping process pool\n',
    '2021-05-13T13:11:08.559Z project Test run started; run priority: 3\n',
    '2021-05-13T13:11:08.560Z project Running all tests\n',
    '2021-05-13T13:11:08.563Z workers Starting test run, priority: 3\n',
    '2021-05-13T13:11:08.563Z workers Distributing tests between 6 workers\n',
    '2021-05-13T13:11:08.564Z workers Running tests in parallel\n',
    '2021-05-13T13:11:08.564Z nodeRunner Starting sandbox [worker #0, session #loq5d]\n',
    '2021-05-13T13:11:08.564Z nodeRunner Starting sandbox [worker #1, session #800xw]\n',
    '2021-05-13T13:11:08.565Z nodeRunner Preparing sandbox [worker #0, session #loq5d]\n',
    '2021-05-13T13:11:08.565Z nodeRunner Preparing sandbox [worker #1, session #800xw]\n',
    '2021-05-13T13:11:08.688Z workers Started run worker instance (delayed) #0\n',
    '2021-05-13T13:11:08.689Z nodeRunner Prepared sandbox [worker #0, session #loq5d]\n',
    '2021-05-13T13:11:08.689Z workers [worker #0, session #loq5d] Running tests in sandbox\n',
    '2021-05-13T13:11:08.691Z workers Started run worker instance (delayed) #1\n',
    '2021-05-13T13:11:08.692Z nodeRunner Prepared sandbox [worker #1, session #800xw]\n',
    '2021-05-13T13:11:08.692Z workers [worker #1, session #800xw] Running tests in sandbox\n',
    '2021-05-13T13:11:08.695Z workers Started run worker instance (delayed) #2\n',
    '2021-05-13T13:11:08.708Z workers Started run worker instance (delayed) #3\n',
    '2021-05-13T13:11:08.723Z workers Started run worker instance (delayed) #5\n',
    '2021-05-13T13:11:08.737Z workers Started run worker instance (delayed) #4\n',
    '2021-05-13T13:11:13.035Z workers Scheduling Jest Test Run: 2021-05-13T13:11:08.906Z\n',
    '2021-05-13T13:11:13.094Z workers [800xw] Loaded unknown number of test(s)\n',
    '2021-05-13T13:11:13.095Z workers [800xw] Test executed: should be defined\n',
    '2021-05-13T13:11:13.095Z workers [800xw] Test executed: displays the date when the log was produced\n',
    '2021-05-13T13:11:13.096Z workers [800xw] Test executed: displays the time, with milliseconds, when the log was produced\n',
    '2021-05-13T13:11:13.096Z workers [800xw] Test executed: displays the timezone of the server when the log was produced\n',
    "2021-05-13T13:11:13.097Z workers [800xw] Test executed: logs the logger's global context\n",
    '2021-05-13T13:11:13.097Z workers [800xw] Test executed: logs the log message context\n',
    "2021-05-13T13:11:13.098Z workers [800xw] Test executed: prefers the log message's context over the global logger's context\n",
    '2021-05-13T13:11:13.098Z workers [800xw] Test executed: does not display logs below the log level\n',
    '2021-05-13T13:11:13.098Z workers [800xw] Test executed: displays the context\n',
    '2021-05-13T13:11:13.099Z workers [800xw] Test executed: displays the context\n',
    '2021-05-13T13:11:13.099Z workers [800xw] Test executed: displays the context\n',
    '2021-05-13T13:11:13.099Z workers [800xw] Test executed: displays the context\n',
    '2021-05-13T13:11:13.099Z workers [800xw] Test executed: does not try to display the context\n',
    '2021-05-13T13:11:13.106Z workers [800xw] Run 13 test(s), skipped 0 test(s)\n',
    '2021-05-13T13:11:13.107Z workers Jest Test Run Complete: 2021-05-13T13:11:13.033Z\n',
    '2021-05-13T13:11:13.118Z workers [800xw] Sandbox is responsive, closing it\n',
    '2021-05-13T13:11:13.890Z workers Scheduling Jest Test Run: 2021-05-13T13:11:08.893Z\n',
    '2021-05-13T13:11:13.917Z workers [loq5d] Loaded unknown number of test(s)\n',
    '2021-05-13T13:11:13.918Z workers [loq5d] Test executed: listens on the configured port\n',
    '2021-05-13T13:11:13.920Z workers [loq5d] Run 1 test(s), skipped 0 test(s)\n',
    '2021-05-13T13:11:13.920Z workers Jest Test Run Complete: 2021-05-13T13:11:13.903Z\n',
    '2021-05-13T13:11:13.922Z workers [loq5d] Sandbox is responsive, closing it\n',
    '2021-05-13T13:11:13.922Z workers Merging parallel test run results\n',
    '2021-05-13T13:11:13.923Z project Test run finished\n',
    '2021-05-13T13:11:13.923Z project Processed console.log entries\n',
    '2021-05-13T13:11:13.923Z project Processed loading sequences\n',
    '2021-05-13T13:11:13.923Z project Processed executed tests\n',
    '2021-05-13T13:11:13.925Z project Processed code coverage\n',
    '2021-05-13T13:11:13.954Z project Test run result processed and sent to IDE\n'
  ]
}
@ArtemGovorov
Copy link
Member

Thanks for reporting the issue. It is now fixed and the fix is published in the latest core v1.0.1071.

@gCardinal
Copy link
Author

Wow, crazy response time. Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ArtemGovorov @gCardinal