Testing 101: The role of test data in software testing and the related effort for test cycles
Test data and their important role in SAP testing
In our section Testing 101 we would like to go into the role of test data in software tests and in particular into testing in SAP systems.
Apart from the sheer iteration of test cases, test data has a significant role, especially in the context of test automation, which is often quite underestimated.
In the following we will highlight, after a brief example, different aspects to be considered related to test data.
Considerations regarding test data and an example
Test data in a software test is defined as those entries which are entered in a process or a test case iteration in order to ensure the process as well as the correctness of the iteration.
This may sound quite simple for now but it has to be taken under detailed consideration, which we would like to respond to in the following example.
E.g.: You create a customer order for a material. The test requirement (which defines what should be tested) in our example is that an order can only be executed successfully if enough material is in stock and available and the minimum quantity will be ordered.
Now, before we start our test case we have to search for a material which is currently in stock and available in the minimum quantity that we need for our test case.
To speak in software testing language: the requirements regarding test data for our test case are:
- Valid material available in our system
- In stock
- And in amount which we have planned for the test iteration and which at least matches the minimum quantity required.
All these considerations have to be applied to respective requirements of our test data in all our test cases which we want to execute in our test plan.
Tip: These requirements for the test data should be established as precisely and explicitly as possible. This is as important for the documentation and rerun of the test cases as for the documentation of the process iteration which has to be tested. The documentation of the test data preconditions helps you to test objectively correct and above all enables you to run them conclusively. Your colleagues who will be running the test cases in the future can then test the testing requirements as contemplated in the conception phase for the test case.
Back to our example:
We create our customer order in the system. We choose copy paper as our material which is deposited with 30 cartons in the system.
Our test case can then be executed successfully because 50 cartons of the copy paper are available.
Test data grow old, get used up and interact
Accompany us on a short journey through time which takes us to the next three months in the future. You are in a test cycle which enables you to test the correct functionality of your SAP system after the installation of an update.
Since our last test your test system has not changed, which means that you run the test case example with the same material once again and receive an error message, because there are not enough copy paper in stock.
You then have to choose a new material which meets all your test data preconditions and with which you can execute your test case.
Our test data in our current stock is worn away and leads to an error message while executing the test case, due to the test data (quality) and not because of lacking quality of the test object (create order). This leads to additional effort since we have to look for new test data and repeat the test case.
- Test data wear away over time
Think about other examples: Let us assume that employees who have been working longer than 20 years in our company receive an employee bonus. Experience and good testing practice recommends to select test data at the edge of the interval which is required for the precondition of your test data.
We now want to verify the negative as well as the positive case of the test case. Thus you look for an employee who has almost reached his 21st year of employment, and an employee who has just begun his 21st year of employment.
Our test case will run correctly, provided that our test object (rule verification of bonus payout for 20+ years of employment) works properly, which means the bonus will be paid on one occasion and not on the other. Again another journey through time, 3 months in the future, with the same test data could give a completely different picture: the employee who is supposedly not long enough in our company would receive the bonus as well. This is because this employee started his 21st year of employment just 3 months later and thus receives correctly his bonus.
- Test data grow old
As if that was not enough, test data of different test cases can of course interact.
If one of the test cases, for which you have prepared your test data with lots of effort, change or use up the data of a following test case during its transaction, then it might at the worst require some searching work once more for the follow-up test cases.
– Test data interact
Test data – some important thoughts in short
As you have seen in our examples, a lot of aspects can flow in the correctness and validity of test data.
The 5 most important considerations related to supplying of the test data which you should reflect about:
- Are the requirements for the test data fulfilled?
- Are they worn away since the last test run?
- Have they grown old since the last test run and thus do not meet the requirements?
- Are they in the margins of my interval acceptance in order to achieve the highest validity as possible?
- Do they interact in my different test cases?
Please take all these aspects into consideration and undertake some restructuring deliberations, such as writing down the test data criteria for each test case in general. In such a manner you could economize much effort and cost in your current and as well in all your following test cycles, which could otherwise make your testing and your quality management difficult and costly.