Nov 10, 2019 Gretel is a free code coverage tool for Java program by Carls Howells at the University of Oregon. In order to install Gretel JVM 1.3 or later, BCEL (Byte Code Engineering Library) is required. Key Features: It supports statement coverage for Java program. It assists residual test coverage too by re-instrumenting the program and removing the.
Java code coverage tools are of two types: first, tools that add statements to the Javasource code and require its recompilation. Second, tools that instrument the bytecode, either before or during execution. The goal is to find out which parts of the code are tested by registering the lines of code executed when running a test.
- 1JCov
- 2JaCoCo
- 8Notable historic tools
- 8.2EMMA
JCov[edit]
Developer(s) | Leonid Arbouzov, Alexander Petrov, Vladimir Generalov, Serguei Chukhontsev, Oleg Uliankin, Gregory Steuck, Pavel Ozhdikhin, Konstantin Bobrovsky, Robert Field, Alexander Kuzmin, Leonid Mesnik, Sergey Borodin, Andrey Titov, Dmitry Fazunenko, Alexey Fedorchenko, Leonid Kuskov |
---|---|
Stable release | 3.0 / September 1, 2014; 5 years ago |
Type | Code coverage |
License | GPL v2 (with the Classpath Exception) |
Website | wiki.openjdk.java.net/display/CodeTools/jcov |
JCov is the tool which has been developed and used with Sun JDK (and later Oracle JDK) from the very beginning of Java: from the version 1.1. JCov is capable of measuring and reporting Javacode coverage. JCov is distributed under the terms of the GNU Public License (version 2, with the Classpath Exception). JCov has become open-source as a part of OpenJDK code tools project in 2014.
![Tool Tool](http://www.cafeaulait.org/slides/albany/codecoverage/images/coverage.jpg)
Features[edit]
JCov is capable of reporting the following types of code coverage:
- Block coverage
- Line coverage
- Branch coverage
- Method coverage
JCov implements two different ways to save the collected data:
- Into a file on the filesystem
- Onto a server (a.k.a. 'network grabber')
JCov works by instrumenting Java bytecode using two different approaches:
- Static instrumentation which is done upfront, changing the tested code
- Dynamic instrumentation which is done on the fly by means of Java agent
JCov has a few more distinctive features which include, but are not limited to:
- Field coverage
- Abstract API coverage
- Direct/indirect coverage
- Per-test coverage information (a.k.a. 'test scales')
- Public API and SPI which makes it possible to implement custom filtering and/or mining the coverage data
Tools using JCov[edit]
- Oracle JDK (SE and ME)
- JCK (the Java Compatibility Kit)
- Various Java SE and Java ME TCKs
- Java FX SDK
- Java FX Scene Builder
JaCoCo[edit]
Developer(s) | Marc Hoffmann, Brock Janiczak, Evgeny Mandrikov, Mirko Friedenhagen |
---|---|
Stable release | 0.8.5 / October 11, 2019; 3 months ago |
Type | Code coverage |
License | EPL |
Website | www.jacoco.org/jacoco |
JaCoCo is an open-source toolkit for measuring and reporting Javacode coverage. JaCoCo is distributed under the terms of the Eclipse Public License. It was developed as a replacement for EMMA,[1] under the umbrella of the EclEmma plug-in for Eclipse.
Features[edit]
JaCoCo offers instructions, line and branch coverage.
In contrast to Atlassian Clover and OpenClover, which require instrumenting the source code, JaCoCo can instrument Java bytecode using two different approaches:
- like JCov on the fly while running the code with a Java agent[2]
- like Cobertura and JCov prior to execution (offline)
And can be configured to store the collected data in a file, or send it via TCP. Files from multiple runs or code parts can be merged easily.[3] Unlike Cobertura and EMMA it fully supports Java 7, Java 8,[4] Java 9, Java 10, Java 11, Java 12, Java 13 and Java 14.
Tools using or including JaCoCo[edit]
- SonarQube JaCoCo plugin — one of the defaults for coverage analyses within the code quality management platform SonarQube
- EclEmma Eclipse (software) Code Coverage Plugin, was formerly EMMA based[5]
- Jenkins JaCoCo Plugin[6]
- Netbeans JaCoCo support[7]
- IntelliJ IDEA since v11[4]
- Gradle JaCoCo Plugin[8]
- Visual Studio Team Services[9]
- TeamCity[10]
- STAMP (https://www.stamp-project.eu/)[11]
OpenClover[edit]
Developer(s) | Marek Parfianowicz, Grzegorz Lewandowski |
---|---|
Stable release | |
Type | Code coverage |
License | Apache License 2.0 |
Website | openclover.org |
OpenClover is a free and open-source successor of Atlassian Clover, created as a fork from the Clover code base published by Atlassian in 2017. It contains all features of the original Clover (the server edition). The OpenClover project is led by developers who maintained Clover in years 2012-2017.[12]
OpenClover uses source code instrumentation technique and handles Java, Groovy and AspectJ languages. Some of its features include: fine control over scope of coverage measurement, test optimisation and sophisticated reports.
OpenClover integrates with Ant, Maven, Gradle, Grails, Eclipse, IntelliJ IDEA, Bamboo, Jenkins, Hudson, Griffon, SonarQube and AspectJ.
Cobertura[edit]
Developer(s) | Steven Christou |
---|---|
Stable release | |
Repository | github.com/cobertura/cobertura |
Type | Code coverage |
License | GPL 2.0 |
Website | cobertura.github.io/cobertura/ |
Cobertura is an open-source tool for measuring code coverage. It does so by instrumenting the byte code.
Serenity[edit]
Developer(s) | Michael Couck |
---|---|
Stable release | |
Type | Code coverage |
License | Apache Software License version 2.0 |
Website | wiki.jenkins-ci.org/display/JENKINS/Serenity+Plugin |
Serenity is an open-source toolkit for measuring and reporting Javacode coverage. As well as coverage, major code metrics are measured:- cyclometric complexity, stability, abstractness and distance from main. The report data is persisted to an object database, and made available via Jenkins/Hudson. The interface replicates the Eclipse IDE interface visually.
Serenity dynamically enhances the byte code, making a post-compile step unnecessary. Ant and Maven projects are supported. Configuration is done in xml, an Ant example would be:
And a Maven configuration example would be:
For a full example of a configuration please refer to the Jenkins wiki at https://wiki.jenkins-ci.org/display/JENKINS/Serenity+Plugin.
Jenkins slaves as well as Maven multi module projects are supported.
Testwell CTC++ for Java[edit]
Developer(s) | Verifysoft Technology |
---|---|
Stable release | 8.0. / June 30, 2016; 3 years ago |
Type | Code coverage |
License | Proprietary |
Website | verifysoft.com/en_ctcpp.html |
Testwell CTC++ is a code coverage tool for C, C++, Java and C#. The development of this tool started in 1989 at Testwell in Finland. Since 2013 support and development has been continued by Verifysoft Technology, a company from Offenburg, Germany. Testwell CTC++ analyses for all code coverage levels up to Modified condition/decision coverage and Multicondition Coverage.[13] The tool works with all compilers.[14]
IntelliJ IDEA Code Coverage Agent[edit]
Developer(s) | JetBrains |
---|---|
Type | Code coverage |
License | Apache 2.0 |
Website | github.com/JetBrains/intellij-coverage |
IntelliJ IDEA Code Coverage Agent is a code coverage tool integrated in IntelliJ IDEA IDE and TeamCity CI server. It supports branch coverage and per-test coverage tracking.
Notable historic tools[edit]
Clover[edit]
Developer(s) | Atlassian |
---|---|
Stable release | 4.1.2 / October 11, 2016; 3 years ago |
Type | Code coverage |
License | Apache 2.0 |
Website | atlassian.com |
Clover is a Java code coverage analysis utility bought and further developed by Atlassian. In April 2017 Atlassian announced end-of-life of Clover and at the same time open-sourced it under Apache 2.0 license.
Clover uses a source code instrumentation technique (as opposed to Cobertura and JaCoCo, which use byte code instrumentation), which has its advantages (such as an ability to collect code metrics) and disadvantages (re-compilation of sources is necessary).[15] Some of its features include historical reporting, huge control over the coverage gathering process, command line toolset and API for legacy integration and more.
Clover also allows testing time to be reduced by only running the tests that cover the application code that was modified since the previous build. This is called Test Optimization[16] and can lead to huge drops in the amount of time spent waiting for automated tests to complete.
Clover comes with a number of integrations both developed by Atlassian (Ant, Maven, Grails, Eclipse, IDEA, Bamboo) and by open source community (Gradle, Griffon, Jenkins, Hudson, Sonar).
In April 2017, Atlassian announced that they would no longer release new versions of Clover after version 4.1.2, and its code was made available as open-source software hosted on Bitbucket.[17][18]
EMMA[edit]
Developer(s) | Vlad Roubtsov |
---|---|
Stable release | |
Repository | sourceforge.net/projects/emma/ |
Type | Code coverage |
License | Common Public License 1.0 |
Website | emma.sourceforge.net |
EMMA is an open-source toolkit for measuring and reporting Javacode coverage. EMMA is distributed under the terms of Common Public License v1.0.
EMMA is not currently under active development; the last stable release took place in mid-2005. As replacement, JaCoCo was developed.[19]EMMA works by wrapping each line of code and each condition with a flag, which is set when that line is executed.[20]
Features[edit]
- instrument classes for coverage either offline (before they are loaded) or on the fly (using an instrumenting application classloader).
- Supported coverage types: class, method, line, basic block. EMMA can detect when a single source code line is covered only partially.
- Coverage stats are aggregated at method, class, package, and 'all classes' levels.
- Output report types: plain text, HTML, XML. All report types support drill-down, to a user-controlled detail depth. The HTML report supports source code linking.
- Output reports can highlight items with coverage levels below user-provided thresholds.
- Coverage data obtained in different instrumentation or test runs can be merged.
- it is possible to dump or reset coverage data remotely and without a JVM exit.
- does not require access to the source code and degrades gracefully with decreasing amount of debug information available in the input classes.
- can instrument individual .class files or entire .jars (in place, if desired). Efficient coverage subset filtering is possible, too.
- Makefile and ANT build integration are supported on equal footing.
- The runtime overhead of added instrumentation is small (5–20%) and the bytecode instrumentor itself is very fast (mostly limited by file I/O speed). Memory overhead is a few hundred bytes per Java class.
- EMMA is 100% pure Java, has no external library dependencies, and works in any Java 2 JVM (even 1.2.x).
See also[edit]
References[edit]
- ^JaCoCo Mission
- ^Patroklos Papapetrou (19 December 2012). 'Code Coverage Tools (JaCoCo, Cobertura, Emma) Comparison in Sonar'. Only Software matters. Retrieved 3 March 2013.
- ^Measure Coverage by Integration Tests with Sonar – Updated
- ^ ab'Code Coverage'. IntelliJ IDEA 12.0 Web Help. JetBrains. Archived from the original on 26 April 2013. Retrieved 3 March 2013.
- ^EclEmma, Eclipse code coverage plugin
- ^Jenkins JaCoCo Plugin
- ^NetBeans JaCoCo support
- ^Gradle JaCoCo Plugin
- ^JaCoCo integration in Visual Studio Team Services
- ^JaCoCo integration in TeamCity
- ^Jacoco is used by STAMP DSpot tool as a test selector (seen as a fitness) to select tests that increase the coverage and have unique executed path
- ^OpenClover - About us
- ^Testwell CTC++ supports all coverage levels
- ^Testwell CTC++ supports all compilers
- ^'Why does Clover use source code instrumentation?'
- ^Test Optimization
- ^'Atlassian Clover is now open source'. atlassian.com. 11 April 2017.
- ^'atlassian/clover'. bitbucket.org.
- ^EMMA code coverage files on SourceForge.net
- ^Expert Spring MVC and Web Flow; By Seth Ladd, Darren Davison, Steven Devijver, Colin Yates, p. 289
External links[edit]
- Pick your code coverage tool in Sonar 2.2, a little outdated, as JaCoCo meanwhile supports branch coverage
Retrieved from 'https://en.wikipedia.org/w/index.php?title=Java_code_coverage_tools&oldid=935101815'
What is Test Coverage?
Test coverage is defined as a metric in Software Testing that measures the amount of testing performed by a set of test. It will include gathering information about which parts of a program are executed when running the test suite to determine which branches of conditional statements have been taken.
In simple terms, it is a technique to ensure that your tests are testing your code or how much of your code you exercised by running the test.
In this tutorial, you will learn
What Test Coverage does?
- Finding the area of a requirement not implemented by a set of test cases
- Helps to create additional test cases to increase coverage
- Identifying a quantitative measure of test coverage, which is an indirect method for quality check
- Identifying meaningless test cases that do not increase coverage
How Test Coverage can be accomplished?
- Test coverage can be done by exercising the static review techniques like peer reviews, inspections, and walkthrough
- By transforming the ad-hoc defects into executable test cases
- At code level or unit test level, test coverage can be achieved by availing the automated code coverage or unit test coverage tools
- Functional test coverage can be done with the help of proper test management tools
Benefits of Test Coverage
- It can assure the quality of the test
- It can help identify what portions of the code were actually touched for the release or fix
- It can help to determine the paths in your application that were not tested
- Prevent Defect leakage
- Time, scope and cost can be kept under control
- Defect prevention at an early stage of the project lifecycle
- It can determine all the decision points and paths used in the application, which allows you to increase test coverage
- Gaps in requirements, test cases and defects at the unit level and code level can be found in an easy way
What Are Main Differences Between Code Coverage And Test Coverage?
Code coverage and test coverage are measurement techniques which allow you to assess the quality of your application code.
Here, are some critical differences between booths of these coverage methods: Parameters | Code Coverage | Test Coverage |
Definition | Code coverage term used when application code is exercised when an application is running. | Test coverage means overall test-plan. |
Goal | Code coverage metrics can help the team monitor their automated tests. | Test coverage is given details about the level to which the written coding of an application has been tested. |
Subtypes | Code coverage divided with subtypes like statement coverage, condition coverage, Branch coverage, Toogle coverage, FSM coverage. | No subtype of Test coverage method. |
Formula to calculate Test Coverage
To calculate test coverage, you need to follow the below-given steps:
Step 1) The total lines of code in the piece of software quality you are testing
Step 2) The number of lines of code all test cases currently execute
Now, you need to find (X divided by Y) multiplied by 100. The result of this calculation is your test coverage %.
For example:
If the number of lines of code in a system component is 500 and the number of lines executed across all existing test cases is 50, then your test coverage is:
Examples of Test Coverage
Example 1:
For example, if 'knife' is an Item that you want to test. Then you need to focus on checking if it cuts the vegetables or fruits accurately or not. However, there are other aspects to look for like the user should able to handle it comfortably.
Example 2:
For example, if you want to check the notepad application. Then checking it's essential features is a must thing. However, you need to cover other aspects as notepad application responds expectedly while using other applications, the user understands the use of the application, not crash when the user tries to do something unusual, etc.
Drawbacks:
- Most of the tasks in the test coverage manual as there are no tools to automate. Therefore, it takes lots of effort to analyze the requirements and create test cases.
- Test coverage allows you to count features and then measure against several tests. However, there is always space for judgment errors.