Code Coverage for Swift
Code coverage for Swift language is a measurement, that can collect executed and non-executed code when tests are performed, such as unit test cases or during testing. It is a measure of the white box that finds the areas of the software not exercised by a series of test cases.
As a baseline, we can use code coverage metrics to write new test cases to boost the coverage and thus determine calculated quantitative code coverage.
Table of Contents
One such software testing metric that can help the user determine the effectiveness of the testing and the quality aspects of any software is Code Coverage. It can provide a detailed report of how much of the application code has been covered during the test.
It’s a no-brainer that testing is a must when you’re developing. It is an essential step to determine the performance and predict user experience. While you may already know what tests you have to write, let’s also look at different tests performed on swift language programming so that you can have a keen eye on the quality and performance of the app. Testing in swift is simple; it’s almost like coding for the app itself. Xcode launches these tests and runs for them, and tracks how many lines have been executed. You have to write the test cases.
Let’s talk about the types of testing:
Unit Testing using Xcode
Unit testing involves creating small functionality bases test cases for a unit of code, which ensures other units also pass the test.
This is used to test the interactions with the UI. In UI testing, the developer has to identify UI objects through queries and by synthesizing events. Then the tester sends those events to the objects. The API helps test the object and state’s properties, which can then be compared with the expected state.
Performance test uses a block of code and runs it 10 times to collect the average execution times and the standard deviations. This average is compared to a benchmark value, which determines whether it is a success/failure.
Why measure Code Coverage?
Code coverage measurement helps you avoid test entropy. If your code has undergone multiple changes and versions, as new code is added, it might not meet the testing standards that you have put in the first place. Therefore by measuring code coverage, you can be sure that your project has been tested and tested well. That way, when you move on to the stage of production, you can be sure that there will be minimal problems.
Although, Code Coverage is not the remedy for everything. There are limitations to it. Increasing code coverage also becomes incrementally difficult as newer code is being added to the project. Therefore coverage measurement should not be used as a substitute for good code review and good programming practices.
Summing it together, it becomes of paramount importance that the code coverage standards that you set for the project should be sensible and aim for even coverage in all modules across the entire code. So, if you have a blanket coverage figure, it can hide large gaps in your code and later on become problematic for you.
What’s a good amount of code coverage?
There’s no one size fits all strategy to this. Aiming for 100% code coverage would be too ambitious and impractical. A fair aim is to have around 85% coverage. However, there isn’t much difference between 75% and 85% coverage in larger projects as well. Your aim should be so that your code is being tested enough for as many conditions as it can be and evenly covers all modules of your overall code.
RKTracer Code Coverage tool overview:
RKTracer is a code coverage tool for swift, python, golang, and kotlin. The problem with existing code coverage tools is that the existing open-source tools are difficult to use. They do not support different coverage levels such as line, statement, branch, condition, and multiple condition coverage. These are not really user friendly either, and they also have limited information. They have bugs and no backups for tools. As listed, there are many other problems.
On the other hand, the RKTracer tool is an easy to use tool which provides you with fast analysis and can generate reports without unit test cases. It also instruments hundreds of files within seconds. It has ready-to-use plugins for Jenkins, SonarQube, and easily integrates with the in-house CI. Not just that, for backup and support, you have a team of people with 20+ years of experience. The easy to understand reports help you increase the code coverage.
How to generate code coverage swift using RKTracer Tool:
The first step in this is instrumenting the project, and then the intermediate files will be generated. Later on, these files will be used to generate the reports. RKTracer generates reports in HTML and XML. The next step is to instrument the project and test it, whether it is manual testing or running the unit test cases. The third step would be to collect the data, and the last and final step is to generate the reports.
Let’s look at a technical demo with a sample project. The steps we are going to follow are below:
How to turn the tool and start instrumenting the project
How to generate code coverage without unit test cases
How to add test cases for uncovered code and increase the code coverage
How to generate and interpret the code coverage reports
The first step would be to instrument the project. You will get a readme.txt file along with all the installer files with the tool. It’s a short tutorial, and you can go through it.
From your files folder, open your test project in the terminal and write the code.
rktracer -on . swift -v
Once you enter that, you will see that all files will be instrumented, and you will get output like this.
You can see that the configuration files are ignored. We can also choose to ignore some files for selective instrumentation.
The next step is to clean the project and make sure there are no intermediate files by using the code.
swift package clean
Once the cleaning is done, let’s build the project.
Then, you run it by code.
This example is a prime example project. Therefore it prompts to enter the number for primality. If we input the number, it will determine whether it’s a prime or not.
You pass the number 5, and it will return input saying it’s a prime number. After that, you quit the test, showing how many tests have resulted in a prime number.
Now it is time to generate results with command.
And you will see that a new file is generated in the folder. That’s the code coverage data file. From your terminal, if you copy and paste the link, you should see the data.
It gives data for metrics like line coverage, statement coverage, multi-condition coverage, etc. It gives details on when the file was instrumented and what the project is about, how many files are there etc.
If you right-click on the files, you will be able to see the functions that are in the files. Also, if you hover over the pie charts, you should see how much percentage is fully covered, how much is partially, and how much has not been covered under different coverage metrics.
If you double click on the files on the left side menu, it will take you to the file’s source code. It shows pie charts and summaries on a higher level.
Now it says that 7 out of 12 lines have not been covered inline code coverage. 3 have been fully covered, and 2 are partially covered. If you count the lines, you will find it to be true.
Similarly, in statement coverage, it says 8 statements haven’t been covered. If you recheck the code, that confirms it again. You can click on the left side panel for multi-condition coverage, where the conditions are indicated to see whether they have been executed or not.
If you right-click on the report, it will also give you the source code’s function summary. Not just that, at any given time, you can get the file summary and project summary also.
Out of the conditions that have been partially executed, if you hover, it will show you whether it’s been executed or not (if yes, how many times is also shown). Let’s analyze that. We haven’t passed a value below 4, so it hasn’t been executed. But we have passed 5 in the next condition, so that’s why it says True 1 and False 0 in the second condition.
In the last condition, we haven’t passed 7 so, it says True 0 and False 0.
Let’s do one thing: let’s pass the right values to increase the code coverage. So, we will go back to the terminal and pass the numbers 3, 7, and maybe another large number.
Again now, you can look at the report’s summary and see which conditions have been executed and which haven’t in the partially executed code. By interpreting the reports and bypassing the right inputs covering all conditions, you can increase your code coverage.
The advantage of this is that we can achieve the code coverage percentage set by the standards bypassing the right values and ensuring all conditions are being executed within a short period of time.