EclEmma 3.1.9 Java Code Coverage for Eclipse Drag to your running Eclipse workspace to install EclEmma Java Code Coverage
Overview 
Installation 
User Guide 
Support 
Resources 
Developer Information 
Architecture 
EclEmma 2.0 
Release Checklist 
Research 
JaCoCo 
Change Log 
License 
Contact 
GitHub Home
Eclipse Community Award 2008
JaCoCo
Inspected with sonarqube
jobs at mtrail
The Java Specialists' Newsletter

EclEmma 2.0 Position Paper

The EclEmma code coverage plug-in for Eclipse originally was designed as a wrapper for the EMMA code coverage library to make EMMA available within the Eclipse IDE. Since its initial release in 2006 EclEmma became a popular addition to the Eclipse Java IDE. As EMMA maintenance has stopped some years ago and we wanted to see functional enhancements like branch coverage for EclEmma we started a completely new code coverage backend called JaCoCo under the umbrella of the EclEmma project. The purpose of this paper is to setup expectations for the EclEmma/JaCoCo user community and serve as a guide line for technical decisions. The first release fully backed by JaCoCo is "EclEmma 2.0".

Marc R. Hoffmann, Mountainminds GmbH & Co. KG, September 2011 (updated)

Starting Point and Maxims

Since its initial release in 2006 EclEmma became one of the most widely uses code coverage tool in the Java developer's tool box and is one of the most installed Eclipse plug-ins. As EMMA development has stopped shortly after the first EclEmma release in 2006 we started working on an alternative called JaCoCo in 2009. The aim of the JaCoCo Project is providing a lightweight, flexible and well documented library for integration with various build and development tools, see this mission statement for details. In the meantime JaCoCo is fully functional and has been adopted by several projects. For example the Eclipse platform integration builds are analyzed using the JaCoCo Ant tasks. Many open source as well as commercial projects use the JaCoCo plug-in for Sonar to control test quality.

As EclEmma was originally developed to provide code coverage analysis directly for the individual developer, the same now applies for JaCoCo. A consistent view to test coverage metrics in the automated builds as well as the local Eclipse IDE requires a JaCoCo plug-in for Eclipse. From the EclEmma project's point of view there are at least the following three paths towards a JaCoCo integration for Eclipse:

  • Option 1: Implement a new and separate Eclipse plug-in for JaCoCo.
  • Option 2: Create a plugable architecture that allows to integrate different code coverage technologies.
  • Option 3: Migrate EclEmma towards JaCoCo (preferred).

I propose to follow option 3, which means migrate the existing EclEmma project towards JaCoCo. Given the high exposure of EclEmma this options offers a set of benefits:

  • Smooth upgrade path to the new version of EclEmma.
  • Existing functionality and features of EclEmma like "one-click" launches and direct visual feedback within the IDE are kept.
  • Proven and robust code base.

The other two options appear less attractive due to two main aspects: The first point is the EMMA library itself. Given the issues described before and the fact that EMMA fails with Java 7 byte code the future use of this great but abandoned library is questionable. Therefore maintaining a EMMA-only plug-in does not look reasonable in the long run. Users explicitly bound to EMMA may continue using the EclEmma 1.x stream (see below).

Providing a plugable architecture for different coverage libraries looks like best practice at the first glance. This is how the overall Eclipse platform has been successfully crafted. When looking at different code coverage libraries features and integration strategies are very different. For example EMMA requires a huge overhead for instrumentation of local class files and hooks to tweak class paths. JaCoCo in contrast does not require this but gives additional coverage metrics and source highlighting for branch coverage. A code coverage framework would therefore either be

  • very small as the common overlapping is small, or
  • quite big containing several base services (like class path re-writing) for particular code coverage libraries.

On the other hand the Eclipse launch and debug framework already is a plugable architecture that allows different integrations for different code coverage technologies even today.

Functional Improvements

For EclEmma 2.0 the following functional improvements are planned:

Branch Coverage

This additional metric will show coverage of all decision points in the program flow due to if and switch statements or the ? operator. The result will be displayed directly in the Java editors similar to the JaCoCo reports using a new annotation icon in the left ruler. In addition the Coverage view can be switched to branch counters. Also the Coverage property page will show branch figures for all Java elements. In addition cyclomatic complexity is shown in the view as well as on the property page. The former "block" counters will not be supported any more.

Faster Launching

Due the way how the JaCoCo coverage library works there will be no additional delay any more when applications under test are launched. This is an significant performance improvement especially for large applications and test suites.

Less Invasive

Certain launch types and test scenarios require so called in-place instrumentation in EclEmma 1.x. With this option the original class files get modified on disk and need to be restored with a clean build when switching back to another launch mode. Also JAR files where excluded from coverage analysis when in-place mode is activated. Without in-place instrumentation the class path of the application under test was modified which caused trouble for some applications.

With EclElmma 2.0 class files on disk will never be modified and the class path of an applications stays untouched. This will remove several hassles especially for Eclipse application launches and JUnit plug-in tests.

Intermediate and Remote Coverage Analysis

JaCoCo 2.0 (or more likely one of its subsequent versions) will support intermediate coverage dumps on applications running locally or remote without stopping the applications under test. This will also allow to reset the collected coverage information for a running application.

Flexible Analysis Scope

The scope of a coverage analysis can be modified at any time afterwards. While in EclEmma 1.x the scope needs to be specified on the coverage launch dialog before the application is launched, with EclEmma 2.0 the scope can be altered at any time when the result of the coverage session is viewed.

The Plug-in Project Name

Back in 2006 EclEmma originally was an acronym formed from the project names Eclipse and EMMA. Today the name is widely used for code coverage in Eclipse without an focus on the technology used internally. Therefore the name should be kept, also for the plug-in Ids and Java package names. Which directly leads to the next point.

Update from EclEmma 1.x to EclEmma 2.0

Once EclEmma 2.0 has been stabilized and the first release is published EclEmma users should be able to get the new JaCoCo based implementation simply through the regular update site process.

System Requirements

EclEmma 2.0 will raise the bar a little bit and has the following system requirements:

  • Java Runtime: As JaCoCo requires Java 1.5 the same minimum JRE is required for the Eclipse instance running EclEmma and the application under test. Note that this does not apply to the Java class files under test: Any class file version ranging from Java 1.0 to 1.7 is possible.
  • Eclipse: The minimum Eclipse Version to install EclEmma 2.0 is not determined yet but the backward compatibility down to Eclipse 3.1 will be dropped to get rid of several switches and usages of APIs deprecated in the meantime. Anyhow a conservative approach is targeted, e.g. Eclipse 3.5 as the minimal supported version.
  • Operating System: As JaCoCo as well as EclEmma is pure Java all operating systems supported by Eclipse will also be supported including Windows, Linux, Solaris, HP-UX, AIX and Mac OSX on different architectures.

Future support for EMMA and EclEmma 1.x Maintenance

From EclEmma 2.0 on EMMA will not be supported any more as a coverage engine. The only supported backend will be JaCoCo. There are not plans for a "plugable" architecture to facilitate different coverage engines. The technologies are simply too different to efficiently integrate them based on a common framework.

Beside this we will try to maintain the EMMA based 1.x stream on a best effort base but with no functional enhancements planned. There will be a separate download for the 1.x versions.

Various Implementation Considerations

For a new EclEmma 2.x the following ideas should be considered:

  • API Compatibility: As JaCoCo was explicitly designed for integration, EclEmma 2.0 should be far less complex than the existing 1.x stream. To allow maximum reduction in code size and complexity several of the existing APIs can either be simplified or removed at all.
  • Code Coverage Model: Instead if defining a separate code coverage model for Eclipse IDE usage only the JaCoCo model will be used directly.
  • Agent Communication: Instead of writing Execution data to the file system, the TCP/IP base communication will be used. This will also allow intermediate dumps.
  • Increase Testability and Test Coverage: Cutting several static references to create more testable units will allow to increase unit test coverage of EclEmma itself.

References

Copyright © 2006, 2017 Mountainminds GmbH & Co. KG and Contributors Validate XHTML/CSS · Last Modified 2017-03-28 in Commit 1ce487ba