Running tests using CompilationTestHelper needs jvmArgs...

[asautins]

I am developing a custom error prone check for error prone 2.18 running on java 17. Using the net.ltgt.errorprone plugin works great for building. However, when I adda test class that uses CompilationTestHelper as follow:

public class LogMethodCallParameterTest
{
    private CompilationTestHelper compilationHelper;

    @BeforeEach
    public void setup() {
        compilationHelper = CompilationTestHelper.newInstance(CustomErrorProneCheck.class, getClass());
    }

    @Test
    public void simpleTestCase() {
        compilationHelper.addSourceFile("/testcase1.java").doTest();
    }
}

I get the following error when executing the test:

java.lang.IllegalAccessError: class com.google.errorprone.CompilationTestHelper (in unnamed module @0x614635c2) cannot access class com.sun.tools.javac.api.JavacTool (in module jdk.compiler) because module jdk.compiler does not export com.sun.tools.javac.api to unnamed module @0x614635c2

I am able to resolve the issue by adding the following to the test section of my build.gradle file:

test {
   useJUnitPlatform()

   jvmArgs = ['--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED',
              '--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED',
              '--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED',
              '--add-exports=jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED',
              '--add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED',
              '--add-exports=jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED',
              '--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED',
              '--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED',
              '--add-opens=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED',
              '--add-opens=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED']

}

These are the same args as in ErrorPronePlugin.kt JVM_ARGS_STRONG_ENCAPSULATION.

Am I missing something? Is there a flag I can add to the test section of my build.gradle file to add the strong encapsulation arguments when running tests using CompilationTestHelper?

[tbroyer]

Those are the flags from https://errorprone.info/docs/installation and yes you'd need to explicitly add them to the Test task's jvmArgs. There's no specific support for this in the plugin such that it would add those flags for you (writing a check is not the most common kind of projects, and technically you could very well write such a check without checking your code with ErrorProne and therefore without this plugin 😉 e.g. writing it in Kotlin, Groovy or Scala).

Regarding the need for the flags, I think it'd be a good reason to open an issue on the ErrorProne project for them to add some documentation about CompilationTestHelper and the required flags (which are, logically, just the same as when using ErrorProne as a compiler plugin, but used differently).

[asautins]

Ha! I can't imagine writing an error prone check without writing a test for it :).

I guess my perspective is iff I'm using Java 17 right now with gradle I pretty quickly come across this plugin. I then create a simple project to use error prone and think "sweet, it works" ( which is what I did :) ). Now I want to bring in my own errorprone checks and since the plugin helped me get this far I kindof hope the plugin could help with tests as well.

In my perfect world I would follow the plugin documentation and there'd be a section on "testing. your custom checks" and then be able to add something like:

test {
   useJUnitPlatform()
   jvmArgs = errorpone.JVM_ARGS_STRONG_ENCAPSULATION
}

Or something to that effect. Since you're going to all the effort to setup the flags for the builds, it would be lovely for people to not have to figure it out separately for testing their own custom checks.

[tbroyer]

There really aren't that many people writing their own checks. I'll think about providing some API to make it easier to run those tests, but currently the plugin still supports older ErrorProne versions (<= 2.10) that run with JDK 8, and test tasks can be configured to run with a specific Java toolchain so for example one could want to test with ErrorProne 2.10 with JDK 8, and with ErrorProne 2.16 with JDK 11 and/or JDK 17. Currently, I don't think the number of projects building their own ErrorProne checks is worth this effort.

IMO, having the ErrorProne documentation better explaining how you can use error_prone_test_helpers, with a Gradle snippet configuring the appropriate JVM arguments (for JDK 11+ as the latest version only supports those versions), would be enough and much more maintainable for everyone.

[tbroyer]

I opened google/error-prone#3554