Why You Need Code Coverage Metrics
Why You Need Code Coverage Metrics
Code Coverage metric shows the lines of code/statements/conditions executed and not executed in dynamic testing. Code Coverage metrics will assess you to improve the dynamic testing(unit testing/Integration testing/Functional testing) of your application in the runtime environment. Runtime environment might be in the build machine/embedded hardware/application server.
Why do you need Code Coverage Metrics?
When you test your applications in a runtime environment, i.e., unit testing/integration testing/functional testing, without code coverage reports, you will never know if all the lines of code/statements/conditions have been executed or not. For that matter, without having code coverage metrics data, you might be writing redundant test cases. Code Coverage metrics will let you know how thoroughly you have tested your application. It will also expose untested parts of code in your application. Code Coverage is not only about line coverage. In a single line, you can have multiple statements or conditions.
Your goal should be to improve the quality of testing with the help of code coverage metrics. You should not simply aim to increase the code coverage percentage.
100% code coverage != quality testing or quality of your product. However, code coverage reports help improve your testing process.
Code Coverage will not find bugs or expose bugs. Coverage data will show the line of code, statements, and multiple conditions covered during the testing phase in a runtime environment.
Code Coverage Types
Function coverage metrics show how many times function calls were made during dynamic testing. Function coverage gives you a quick overview of the number of times functions executed in unit testing/Integration testing/functional testing.RKTracer code coverage tool tracks and show the number of times the function/method was called in dynamic testing.
Here isprime function was executed twice in dynamic testing.
Line coverage is a metric that shows the number of executable lines (excluding comments ) covered in dynamic testing.
Line Coverage= Number of lines of code executed during testing/Total number of executable lines of code
In most programming languages, line coverage is not equal to statement coverage.
for loop has three statements:
for (dividend = 3; dividend <= value/dividend; dividend+=2)
Statement 1 : for (dividend = 3; dividend <= value/dividend; dividend+=2)
Statement 2 : dividend = 3;
Statement 3 : dividend <= value/dividend;
Line coverage is vital as it provides us a first-hand impression of total lines of code covered with color code. In a single line, we can have multiple statements or conditions. RKTracer tool will highlight the parts of code or condition covered in three different colors.
- Green color = Covered, i.e., part of code or condition will be highlighted in green color if the code or condition is executed for both true and false conditions.
- Yellow color = partially covered, i.e., the part of code or condition will be highlighted in yellow color if the code or condition is executed for either true or false conditions.
- Red color = Not covered, i.e., the part of code or condition will be highlighted in red color if the code or condition is executed, not executed at all.
With the RKTracer tool, you can check the summary of line coverage at the function level, source file level, and project level for better analysis. We can see all three colors in a single source code line based on conditional expressions executed for branches or conditions. Line coverage is dependent on the assessment of other formats of the code and hence is less reliable.
The statement coverage metric gives us the number of statements executed in dynamic testing.
For EX: variables, loops, conditions like “if,” “while,” “for,” or simple statements like a semicolon which is executable code, or anything ending with a semicolon is considered as a statement. Any empty statement with no machine code just ‘”;” a semicolon is not considered a statement.
Statement Coverage= Number of statements executed during testing/Total number of executable statements in an application
Statement coverage is not equal to line coverage in most programming languages. However, vice versa is impossible as the line contains multiple statements.
In programming languages like C, C++, Java, and C#. In a single line, we can have multiple statements or conditions, or branches. In these cases, some might be executed, some might be partially executed, and some may not have been executed at all in dynamic testing.
a++; b+=2; if (a) b=0; //Here we have four statements in a single line.
- statement 1 a++;
- statement b+=2;
- if-statement: if (a) b=0;
- statement b=0;
The branch coverage metric is also termed decision coverage, where each condition should be executed at least once to true or false conditions.
If (a>b || b>c )
If the above “if and else conditions” have been executed once to true and once to false consecutively, we have 100% branch coverage. Here sub-conditions(a>b and b>c ) are ignored. However, the Boolean expressions are ignored while accounting for the results of branches/decisions which are tested.
Condition coverage is also termed as expression coverage metric. Every sub-expression must be at least once to true and once to false.
If (a>b || b>c )
This coverage is similar to branch coverage with additional testing of sub-expressions (a>b and b>c) for true and false. It is used to check individual outcomes for each logical condition. However, the complete coverage in condition coverage does not imply a total branch/decision coverage rate.
MC/DC Code Coverage
Every sub-conditions should be executed at least once to true and once to false in a given decision block. As a result, the outcome of the decision block should be changed to either true or false.
Decsion block “if (value < 4 || value == 5 || value == 7)” complete if condition is decision block
sub-condition 1: value < 4
sub-condition 2: value ==5
sub-condition 3: value ==7
Let’s say we have following decision “if (value < 4 || value == 5 || value == 7)” block with three sub-conditions i.e “value < 4”, “value == 5” , “value == 7”. If we pass value==3 then sub condition “value<4” will be executed for true ,as value 3 is less then 4.However MC/DC states sub condition should be executed atleast once to true and once to false and as a result of it decision block should be changed to either true or false. Since “value<4” has been executed only to true ,so MC/DC condition has not met.
Let’s say we pass two values, i.e., value==3 and then value==4, so that the sub-condition “value<4” will be executed for once to true and one to false as a result decision block will be executed to true or false condition.
You can see that we have covered MC/DC coverage for sub condition “value<4,” and you need to take the same approach for the remaining two MC/DC conditions.
Multiple Condition Coverage
It is also known as a combination of condition coverage. Multiple condition coverage performs tests for every true/false value with the combination. Multiple condition coverage was conducted for every condition in the decision. The coverage is sought by dividing the total number of successful condition combos and line blocks by the total number of conditions and blocks in the project.
Multiple condition coverage is the most efficient and reliable coverage when the coverage rate is entirely in multi-condition coverage. You can be assured that the coverage is complete in all other covers.
There are four multiple condition coverage points covered in a given decision block if we pass value==3 to a given “if” block. Only one condition is covered.
Now let’s pass values 3,5,7 and 89. Now we can see that all the multiple conditions points are covered.