Document code or automatically configure Jacoco with Junit 5 Gradle Plugin

[JLLeitschuh]

Feature (or Documentation) Request

Should the Junit 5 plugin automatically configure the junitPlatformTest task to use Jacoco if it detects the plugin?

Currently configuring jacoco with Junit 5 requires a bit of digging into the DSL and figuring out its intricacies.

Perhaps configure the Junit5 plugin should be expanded to have a enableJacoco flag that will add a jacocoJunit5TestReport task if set to true? What are the teams thoughts?

Current solutions

Anyone looking to configure jacoco with Junit 5 using the new Gradle Kotlin DSL you currently need to use a bit of a workaround because of a kotlin compiler bug. This code is known to work with Gradle 4.1

// Don't use this sample, see below
afterEvaluate {
    val junitPlatformTest by tasks
    jacoco {
        applyToHelper(junitPlatformTest)
    }
    task<JacocoReport>("jacocoJunit5TestReport") {
        executionData(junitPlatformTest)
        sourceSets(java.sourceSets["main"])
        sourceDirectories = files(java.sourceSets["main"].allSource.srcDirs)
        classDirectories = files(java.sourceSets["main"].output)
    }
}

/**
 * Workaround fix for calling [org.gradle.testing.jacoco.plugins.JacocoPluginExtension.applyTo]
 *
 * [Issue details here](/~https://github.com/gradle/kotlin-dsl/issues/458)
 */
fun org.gradle.testing.jacoco.plugins.JacocoPluginExtension.applyToHelper(task : Task) {
    val method = this::class.java.getMethod("applyTo", Task::class.java)
    method.invoke(this, task)
}

EDIT

Better solution with less code:

afterEvaluate {
    val junitPlatformTest : JavaExec by tasks
    jacoco {
        applyTo(junitPlatformTest)
    }
    task<JacocoReport>("jacocoJunit5TestReport") {
        executionData(junitPlatformTest)
        sourceSets(java.sourceSets["main"])
        sourceDirectories = files(java.sourceSets["main"].allSource.srcDirs)
        classDirectories = files(java.sourceSets["main"].output)
    }
}

Deliverables

• Documentation about how to configure Jacoco with Gradle or add code to the plugin to do it automatically

[marcphilipp]

IIRC Currently, the Jacoco plugin hooks into the test task. Could it be extended to support junitPlatformTest (at least until Gradle provides core support for the new JUnit platform)?

[alixwar]

@JLLeitschuh Do you know of a workaround for Gradle (Android) Java?

My attempt unfortunately only results in code coverage data for JUnit4 tests

[JLLeitschuh]

Does the android plugin do something special for testing? I'm not totally certain because I've never done anything with it.

I guess my first question is can you get the junit 5 plugin to correctly execute your tests? Once you have that working the above should work.
If you need the same logic but for groovy I can provide an example of that if it would help.

[alixwar]

@JLLeitschuh I have done the following for each submodule in the project:

project.afterEvaluate {
    apply plugin: "jacoco"
    jacoco {
        applyTo junitPlatformTestDebug
    }
}:

This is enough to generate code coverage for JUnit4 tests (inluding tests annotated @RunWith(MockitoJUnitRunner.class)) under the JUnit Vintage runner but not JUnit5 tests or Robolectric tests (JUnit4 tests annotated @RunWith(RobolectricTestRunner.class)). Would your groovy example look any different?

Does the android plugin do something special for testing?

Do you mean the Android Gradle plugin or the Android JUnit 5 plugin?

[mannodermaus]

Does the android plugin do something special for testing?

As opposed to the Java-based Gradle plugin for JUnit 5 provided by the junit5 team? It basically only extends its core functionality with the variant-awareness that Android projects have. Instead of adding just a single junitPlatformTest task, the android-junit5 plugin applies a task per variant, junitPlatformTestDebug & junitPlatformTestRelease, while also taking care of potential product flavors.

[alixwar]

@JLLeitschuh Any thoughts on the explanation from @mannodermaus ? Is there something that I or someone else can do to help clarify the issue?

[JLLeitschuh]

I don't work with any projects that use gradle for building android projects. I'm assuming that as long as you can get your android tests to run correctly using the junitPlatformTest task then the code snippet that I have provided should work.

If you have specific behaviour problems that you are experiencing then I may be able to help. At the moment I'm not totally certain I understand what the problem is.

[mannodermaus]

As discussed over at mannodermaus/android-junit5#4, the latest SNAPSHOT release of the Android-based plugin for JUnit 5 implements the first iteration of support for Jacoco out of the box, so there's nothing to be done from you guys in that regard.

As for the Java-based JUnit 5 Gradle plugin, it could be a nice improvement to apply the code snippet for Jacoco integration directly through the plugin in a future release. In the android-junit5 plugin, I'm conditionally extending each JUnit 5 task with a corresponding Jacoco reporting task only if the plugin is found to exist in a module.

[alixwar]

I'm happy to report that from my perspective the issue with JUnit5 is resolved on Android. It works fine with the latest snapshot release of the plugin.

[alixwar]

@JLLeitschuh I've created a sample project with the following tests (for a related issue):

• Vanilla (run by JUnit5 vintage runner) JUnit 4 test (OK)
• Vanilla (run by JUnit5 Juniper runner) JUnit 5 test (OK)
• JUnit 4 (run by JUnit5 vintage runner) test using MockitoTestRunner (OK)
• JUnit 4 (run by JUnit5 vintage runner) test using RobolectricTestRunner (No test coverage)

Sample project: mannodermaus/android-junit5#18

Steps to reproduce:

1. In the sample-robolectric folder, run: ..\gradlew clean jacocoTestReportDebug
2. Examine the resulting $buildDir\jacoco\junitPlatformTestDebug.exec

Expected:
Code coverage for Foo.junit4robolectric(), Foo.junit4(), Foo.junit4mockito() and Foo.junit5()

Actual:
No code coverage for Foo.junit4robolectric()

I realize that this is more of a problem for the Robolectric project (I've reported it here: robolectric/robolectric#3397) but if any of you guys, who know much more about this than I, could take a quick peek and pin point what seems to be wrong it would be easier for someone else to find a workaround or to make a pull request and fix the issue in the Robolectric project so that Android developers (Robolectric is quite common) can start with JUnit5.