Android instrumentTest does not show coverage for library

that's my first post on stackoverflow, so please advise me if something missing.

Here comes the long description - actual questions on bottom. --------------------------------------------------------------------------------

We're currently moving our Android App Project from Eclipse to Android Studio, as the Eclipse Plugin will be no longer supported.

We managed to migrate everything so it comes out of our build machine.

The problem is, that there is no more coverage for the library projects. So the project looks like:

• Module Application1
• Module Application2
• Module Application3
• Module BaseLibrary
• Module CoreLibrary

The CoreLibrary does the network stuff and holds the data.

The BaseLibrary (depends on CoreLibrary) does the actual app stuff like activities, fragments, menu, ...

Each Application (depends on BaseLibrary) basically only provides the actual App Name, various graphics, implements some interfaces to define which login method to use or restricts/enables different menu actions.... (Yeah I know, that should be managed as flavors...)

There are no actual unit tests on each Library. In old Eclipse project there was a separate (Test-)Application that also built on top of the BaseLibrary (like the other applications), provided the InstumentationRunner and defined the instrumented unit tests. It was also used for Calabash-Android Gui Tests. As the Test-Application was built with instrument, it contained the emma code coverage stuff and we could see the coverage for the whole source (including all the libraries).

With the new AndroidStudio project style, those tests where moved to

• Module Application
  • src
    • main
    • androidTest

Here you have the build gradles:

build.gradle

// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:1.5.0'
    }
}

allprojects {
    repositories {
        jcenter()
    }
}

ext {
    minSdkVersion = 15
    targetSdkVersion = 19
    compileSdkVersion = 23
    buildToolsVersion = "23.0.2"

    supportLibVersion = "23.1.1"
    junitVersion = "4.12"
    mockitoVersion = "1.10.19"
    jacocoVersion = "0.7.5.201505241946"
    hamcrestVersion = "1.3"
    runnerVersion = "0.4.1"
    rulesVersion = "0.4.1"
    espressoVersion = "2.2.1"
    uiautomatorVersion = "2.1.1"
}

There are some basic helper gradle files with common configurations:

../versionconfig.gradle

def computeVersionCode() {
    def code = 16120099;
    if (System.getenv("CURRVERSION") != null && System.getenv("BUILD_NR") != null) {
        def versioncode = System.getenv("CURRVERSION") + sprintf('%04d', System.getenv("BUILD_NR") as Integer)
        code = versioncode as Integer ?: 16120099
    }
    return code
}

android {
    compileSdkVersion rootProject.ext.compileSdkVersion
    buildToolsVersion rootProject.ext.buildToolsVersion

    defaultConfig {
        versionCode computeVersionCode()
        versionName "4.4.1"
        minSdkVersion rootProject.ext.minSdkVersion
        targetSdkVersion rootProject.ext.targetSdkVersion
    }
}

../coverageconfig.gradle

android {
    buildTypes {
        release {
            // Instrument all release builds for coverage
            testCoverageEnabled = project.has('globalCoverage')
        }
        debug {
            // Instrument all debug builds for coverage
            testCoverageEnabled = project.has('globalCoverage')
        }
    }
}

../signconfig.gradle

android {
    if (System.getenv("ANDROID_KEY_STORE") != null) {
        signingConfigs {
            release {
                storeFile file(System.getenv("ANDROID_KEY_STORE"))
                storePassword System.getenv("ANDROID_KEY_PASSWORD")
                keyAlias System.getenv("ANDROID_KEY_ALIAS")
                keyPassword System.getenv("ANDROID_KEY_PASSWORD")
            }
        }
    }
    buildTypes {
        release {
            if (System.getenv("ANDROID_KEY_STORE") != null) {
                signingConfig signingConfigs.release
            }
        }
    }
}

CoreLibrary

apply plugin: 'com.android.library'
apply plugin: 'jacoco'
apply from: '../versionconfig.gradle'
apply from: '../coverageconfig.gradle'
apply from: '../signconfig.gradle'

android {
    useLibrary 'org.apache.http.legacy'

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
        }
    }
}

dependencies {
    compile fileTree(dir: 'libs', include: '*.jar')
    compile 'com.android.support:support-v4:' + rootProject.ext.supportLibVersion
    compile 'com.google.guava:guava:18.0'
}

MainLibrary

apply plugin: 'com.android.library'
apply plugin: 'jacoco'
apply from: '../versionconfig.gradle'
apply from: '../coverageconfig.gradle'
apply from: '../signconfig.gradle'

android {
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
        }
    }
}

dependencies {
    compile project(':CoreLibrary')
}

Application

apply plugin: 'com.android.application'
apply plugin: 'jacoco'
apply from: '../versionconfig.gradle'
apply from: '../coverageconfig.gradle'
apply from: '../signconfig.gradle'

android {
    publishNonDefault true

    repositories {
        jcenter { url "http://ift.tt/1mQAvce" }
    }

    defaultConfig {
        applicationId "com.android.application"
        testApplicationId "com.android.application.test"
        testInstrumentationRunner "android.test.InstrumentationTestRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
        }
    }
}

dependencies {
    compile project(':MainLibrary')

    testCompile 'junit:junit:' + rootProject.ext.junitVersion

    androidTestCompile 'com.android.support:support-annotations:' + rootProject.ext.supportLibVersion
    androidTestCompile 'com.android.support.test:runner:' + rootProject.ext.runnerVersion
    androidTestCompile 'com.android.support.test:rules:' + rootProject.ext.rulesVersion
}

gradlew :Application:connectedCheck -pglobalCoverage=true produces 2 APKs:

• Application-debug.apk
• Application-debug-androidTest-unaligned.apk

and executes the androidTests and produces the coverage report.

But the coverage report only contains the files from the Application/src/main.

There is no coverage shown for Application/src/androidTest, MainLibrary/src/main or CoreLibrary/src/main

When I open the Application-debug.apk with BytecodeViewer it's showing

private static $jacocoInit() {

on every class - even those from the libraries ... so I assume, that the code is well instrumented.

Multiple possible solutions were tried, but I think all of them are for older versions of Android Studio/Build Tools/Gradle/Jacoco - so none of them are working with the current toolset.

This would be a list of some links I've been working through, but as this is my first post: "You need at least 10 reputation to post more than 2 links."

[Gradle Plugin User Guide - especially: Multi-projects reports]

[Chapter 61. The JaCoCo Plugin]

[Offline Instrumentation]

[JaCoCo coverage in Cucumber on Android]

[Everybody Tests - The adventures of a QA myrmidon in a still untested world.]

[The basics of Unit and Instrumentation Testing on Android]

[Android Gradle Jacoco: offline instrumentation for integration tests]

[OleksandrKucherenko/EspressoTestCoverageApp forked from Michenux/EspressoTestCoverageApp]

[Jacoco and Unit Tests Code Coverage with android-gradle-plugin >= 1.1]

As far as I could see in my researches, most people have problems to even get coverage from normal unit tests, where the androidTest already automatically produces the coverage report - but not satisfying for our case.

I've also stumbled over Issue 76373: Code Coverage does not include code from local library projects - where I'm not sure, if this should already be fixed or not, as it actually describes the problem I'm focused with.

The proposed workarounds like including the jacocoAgent with each library and remove it again before dexing or the "working solution" by Kucherenko or other solutions can not be applied as it's not allowed anymore to manipulate the gradle tasks for dexing since com.android.tools.build:gradle:1.4.0.

According to the documentation it should be enough, to just apply the plugin 'jacoco' to each library with testCoverageEnabled = true.... but apparently it's not :(

--------------------------------------------------------------------------------

So there are 2 questions:

1. Is it possible to have instrumented test at application level and get full source coverage including library modules?
2. Can the app be started in an instrumented way, so it produces the coverage.ec file while executed manually (or via calabash) and how to generate a coverage report based on that coverage.ec (in CI: prefered without source access, like emma provided the coverage.em at build time which could then be aggregated with coverage.ec, where source was only needed to show the proper lines in the report - but that's not necessary, actual coverage is required for reporting... but source could be available when not possbile otherwise)