What Are the Estimation Techniques in Software Testing?

Software testing is considered to be an integral part of the development process. Recently, different types of tests have emerged, and their complexity is increasing as well. What are modern estimation techniques in software testing and how are they used? Read this article to learn.

What Is Software Testing Estimation

Prior to beginning the testing, two issues that are significant for every client should be discussed: how long the process will take, and how much it will cost. To answer these questions, it’s necessary to estimate parameters such as “time,” “resources,” “cost ” and “human skills.”

Software testing estimation is a management activity to calculate and approximate time, resources and expenses needed to complete test execution in a specified environment.

It is a forecast that helps avoid exceeding time limits and overshooting budgets. Estimation is important to planning the project, allocating resources and optimizing activities, which results in high-grade software quality.

QA_SERVICES PUBLIC_SECTOR

Quality Assurance Services for Judicial Application

QA_SERVICES EMBEDDED FIRMWARE

Scaling QA-Process to Ensure Uncompromised Quality of Expanding Palette of Industrial Security Products

Let’s define why each parameter is essential for the team.

Time

The success of the testing team is determined by its ability to meet the deadline. We should estimate testing efforts, in man-days or man-hours, to find out how long it will take. Managers calculate the optimal duration for each part of the project and set up a clear schedule adhering to which helps execute tasks in a timely manner. This is why time estimation is one of the key factors that helps build a good reputation and create an audience of loyal customers.

Resources

To perform a test, the team needs a definite set of resources (infrastructure, technical capabilities, specialists, time and money). If there are not enough resources, a task can’t be completed in time. It’s necessary to estimate the available assets and those that have to be added.

Cost

Testing, like any other work, requires expenses (including expenses towards hardware, software and human efforts). The overall budget should be evaluated and agreed upon beforehand, in order to assure the customer that the process will be kept within the budget, and no additional payments will be needed.

Human skills

This evaluation considers the professional skills and experience of team members. In case they lack some knowledge, the process will slow down and the costs may increase.  

As we see, all the aspects are intertwined and may influence each other: the time schedule depends on the resources, human skills and the budget; the budget, in turn, is affected by human skills, resources, and time; resources include time and professional knowledge as well.

Traditional Software Testing Estimation Techniques

Now let’s figure it out how to do software testing estimation. In recent years, different techniques, tools and methods have been developed for this very purpose. Their implementation and usage greatly simplify the estimation process.

Here we are going to discuss the most popular and effective estimation techniques in software testing.

Work Breakdown Structure (WBS)

The essence of this technique is to divide a complex test project into small components to obtain the following hierarchy: the project is broken down into sub-modules; each sub-module, in turn, is divided into functionalities, which are split into sub-functionalities. As a result, we get a very detailed and transparent structure (the sample is given below).

Since small tasks are much easier to estimate, they are allocated among the team members. Every person assumes responsibility for a given task and provides the result in a certain period of time. The project can, therefore, be completely checked step by step, without missing any detail. This approach eliminates situations in which some testers may perform the same task simultaneously, while other tasks may remain unaccomplished.

The WBS is the most powerful and effective method among QA estimation techniques. It enables a project manager to create a clear schedule and to plan a reliable budget. It also cultivates team cooperation and personal commitment.

Three-Point Estimation

This is a statistical method, but it also breaks down the task into subtasks (in this it is similar to WBS). Then, three possible scenarios should be estimated for each sub-task.

1. The best case: assuming that you have a talented team and all of the necessary resources, and assuming that no problem occurs and everything goes right, you can complete the task, for example, in 100 man-hours (B). This is an optimistic scenario.
2. The most likely case: assuming that you have a good team, enough resources, and almost everything goes right, although some problems may occur, you can complete the task in 150 man-hours (M). This is a normal scenario.
3. The worst case: assuming that your team is not experienced, everything goes wrong and you have to solve numerous problems, you can complete the task in 200 man-hours (W). This is a pessimistic scenario.

Thus, you have three values:

B = 100

M = 150

W = 200

Now, you can calculate the average value for the test estimation (E) using the following formula:

E = (B + 4M + W) / 6

E = (100 + 4 x 150 + 200) / 6 = 150 man-hours

As the average value may fluctuate a little bit, to be more accurate, you need to calculate standard deviation (SD) — the limits within which E may change. The formula is as follows:

SD = (W – B) / 6

SD = (200 – 100) / 6 = 16.7 man-hours

You can present the final estimate as this: the team needs 150 +/- 16.7 person-hours to accomplish the sub-task.

Three-Point Estimation is one of the most effective methods for software testing when you have practice and data from the previous projects and an ability to apply them. The essence of this method is to find out the best and the worst working conditions for your team.

Function Point Analysis (FPA)

This technique indicates software functionality from the user’s perspective. It estimates the value of the total effort that can be considered as time, cost or size of the task. FPA is based on specification documents, such as SRS document or Design. Again, as with WBS, the project is split into modules. Each module — depending on its complexity — is assigned a functional point (FP). Simple tasks get lower points, difficult tasks get higher points. Total effort is calculated by the formula:

Total Effort = Total FP x Estimate Defined per FP

Estimate defined per FP is a given value that a test manager defines on the basis of the team experience. It is calculated with respect to time, cost or size (e.g. 10 hours/points, $100/points).

Example:

Let’s consider the total effort with respect to cost and take the estimate defined per FP as equal to $100/points.

The whole project is divided into three groups of modules:

• Complex modules (FP is 5) — 2 pieces
• Medium modules (FP is 3) — 10 pieces
• Simple modules (FP is 1) — 5 pieces

Total effort = 45 x 100 = $4,500

This means that to complete the project, you need $4,500.

FPA is not a really accurate method, as it involves risks and may give false results. Nevertheless, it is one of the more popular estimation techniques in software development and can be used in combination with other tools. Try to use this simple estimation template for your tasks, and you’ll see how it works.

Delphi Technique

This is one of the most widely used testing estimation techniques based on surveys of the experts involved in the testing process. The essence is that a panel of experts discuss the given task under the guidance of a manager and make anonymous personal forecasts (how many man-hours this task will take), providing the reasons for their opinions.

As a rule, after the first round, the range of answers is quite wide. Then, the experts are encouraged to revise their answers taking into account other members’ judgments. Several rounds may take place until the range of answers decreases and the average value can be calculated. The process is finished after a predefined criterion (i.e. after a limited number of rounds, or if the consensus is achieved, or when the results are stable).

Delphi technique is very simple and quite reliable due to the participation of experienced people and maintained anonymity. It gives qualitative and quantitative results and can be combined with other methods.

Agile Estimation

The above-described techniques are more suitable for Waterfall development and V-model development. They use a “bottom-up” approach when all the details and requirements should be defined and all the tasks should be estimated separately before you can plan the project schedule and budget. These development models, as well as the estimation approaches, have significant drawbacks due to the great speed of changes in the software industry. Thus, the number of tasks using these methodologies is currently decreasing.

More and more customers and providers prefer using agile development methodology, as it emphasizes continuous delivery, makes the process user-focused and gives a competitive edge. Agile estimation fundamentally differs from traditional techniques and uses a “top-down” approach.

It means that, initially, you process currently available data and provide a higher level estimation that is enough to start the project. Later on, as the project progresses, new information is revealed and more details are uncovered at lower levels. New knowledge is incorporated into the estimation, the values are refined and improved without stopping the development process. This approach makes the team capable of estimating new features effectively.

Principles of Agile Estimation:

• Collaboration — all team members are included in the process; no one person can be blamed for an incorrect estimate
• Quickness — it is even more important than accuracy
• Relative comparison — e.g. points are used instead of days or dollars and compared to each other

Kanban and Scrum are two popular methodologies for agile development.

Kanban is a method of visualization. The whole workflow is organized on a Kanban board, enabling team members to track progress and see new features or difficulties as soon as they occur. Thus, the team can decide on the appropriate workload.

In Kanban, the estimation of the item duration is optional. After completing one item, team members just proceed with another one from the backlog. Some teams still prefer carrying out the estimation using a simple method: they ensure that all items are of equal size, therefore they can be completed in an equal period of time.

Scrum is a framework for structuring the process. Using Scrum, developers can break their work into sprints. At the beginning of each sprint, they make plans, estimate the size of stories, manage priorities and forecast releases. At the end of each sprint, working software is delivered.

Here is a  list of some popular agile estimation techniques:

• Planning Poker
• Affinity Mapping
• Bucket System
• Big/Uncertain/Small
• T-Shirt Sizes
• Dot Voting
• Ordering Protocol

There are also methods for estimating the size and effort of test automation, for instance, using  Katalon Studio.

Estimation Rules

For the estimation of any testing project, we highly recommend following these rules:

• Base the estimation of the software requirements
• Make use of the data, experience and knowledge from previous projects (it helps get results cheaper and faster, up to 25%)
• Add reasonable buffer time
• Consider possible internal and external risks (downtime, skill improvements, lack of resources, etc.) and the complexity of the application
• Consider resources availability and team knowledge and skills
• Update the estimation if there are changes in the project
• Verify the estimation and ask an outside expert to review it

Ensure the Quality of Your Project

Estimation is an essential part of the test management that lays the foundation for the testing schedule and the budget. Different estimation techniques in software engineering are used through the whole software testing lifecycle and help efficiently run the project and accomplish it in due time. Certainly, there can’t be one right and appropriate technique for estimation, as every task has its own requirements and conditions.

The available methods are not fully accurate and can provide only approximate values. But even the most experienced professional isn’t able to give a completely spot-on forecast, as there are always unpredictable difficulties once the project is underway. Therefore, test case estimation techniques are extremely helpful in obtaining good results and reducing the time and financial expenses.

SaM Solutions is a company with vast experience in software development and testing processes. Our skilled team is able to find an agile approach to every customer. We help our clients to reach the highest standards of quality involving  QA specialists in all stages of a project. We provide end-to-end services for setting up testing processes that include planning, estimation, the creation of test scenarios for various environments, execution of manual and automated tests, and providing detailed reports and statistical data. Contact us for more information.