Advanced
PDF verification for cross-technology End2End testing of SAP processes
The digital transformation of many organisations means that they are handling their business processes electronically from start to finish and digitally connecting their business partners – whether suppliers, authorities, partners or customers – to these processes. These can be both incoming and outgoing interfaces, machine-readable files or even popular output formats such as PDFs. Media or system breaks always pose a certain risk of errors. In addition to interface tests between individual systems, particular attention must therefore be paid to output formats such as PDF, which are also complex (both technologically and in terms of automatic verifiability), when it comes to quality assurance. Errors at interfaces between ERP systems and file outputs can have far-reaching consequences for the company, including legal consequences. If companies use PDFs as output formats, it is essential to check them regularly and precisely for content, format and correct presentation. Scenarios such as outgoing invoices, order confirmations and contracts – to name just a few – are popular use cases in which unchangeable PDF documents are output from SAP systems. In this context, it is not enough to rely on checking the input sources for these PDFs before they are generated. The format break between SAP and the generated output document harbours a source of risk that cannot be ignored in state-of-the-art quality assurance. Fortunately, the market is aware of this source of error and many companies are paying a certain amount of attention to checking input and output formats as part of their quality assurance. The flip side of the coin, however, is that this quality assurance is usually carried out manually. For many companies, the complexity and possibilities of automated verification represent too great an entry barrier to test these parts of the processes automatically. A recent survey of companies with regard to the focus of quality assurance of interfaces and incoming and outgoing business documents shows that the tests are focussed on the following aspects: Is the correct and relevant content displayed on the PDF? Nothing is more fatal in terms of data protection, for example, than transmitting sensitive data to the wrong recipients. Of course, the mere presence of content on a document is usually not enough. It is also crucial Is all legally relevant content displayed appropriately for the respective scenario? For example, are legal formulations required as invoice addenda? This concerns, for example, the presentation of the different tax treatment of domestic, EU or non-EU deliveries, discount rates and much more. Companies cannot afford to send unclear or, even worse, incorrect documents to customers, authorities, partners or suppliers. As can be seen from the above points, the verification requirements are heterogeneous and together increase the complexity of the quality assurance requirements. The associated effort and demands on manual testing increase enormously and represent a risk for companies. Automated testing must aim to maximise the verification options and, ideally, reduce the testing effort to almost zero. Standard market automation approaches fail on one of two key requirements: The challenge for test automation is therefore to offer a wide range of functions for recognising individual elements in PDF documents – combined with a high level of usability and a comprehensive toolbox of verification options. Usability is usually very easy to judge. Judge for yourself whether the interface is intuitive and easy to understand and whether you would be able to find your way around after weeks of not using the tool. However, the following question is essential! Are test cases easy to map without any programming steps or manageable without any other technical skills? The technical stability of the generated test cases, the range of verification options offered and the simple mapping of similar variants that differ in just a few points – coupled with usability – help to meet your company’s verification requirements for the critical test aspect ‘PDF output format’. Most PDF test tools on the market are complex to use and are mainly image or screen position based. This means that content is recognised based on its appearance and/or position on the screen. If this position changes, recognition is no longer clearly possible. Automated verification of image-based documents, such as PDF formats, has not been able to gain a foothold in the past due to the limited range of functions, complex use and unreliable test results, which has led many companies to rely on manual testing of these documents. If automation tools such as the suxxesso Tool Suite also support the content structure-based recognition of elements, this opens up numerous additional possibilities for carrying out extensive verifications of your incoming and outgoing documents. For example, according to the study cited above, the essential verification of relating elements to each other based on their position in the document. Suddenly it is very easy to verify item line information individually and reliably. Modern test automation tools enable a massive increase in your test coverage, thus ensuring your system quality and maximising the efficiency of your test cycles. Convince yourself of the scope and user-friendliness of state-of-the-art test automation approaches such as the suxxesso Tool Suite!
Fundamentals
SAP Cloud ALM in quality assurance
SAP announced some time ago that it would be phasing out SAP Solution Manager from maintenance as of 2027, thus indirectly announcing SAP Cloud ALM as its successor. Is this just an embellished Solution Manager or is there more to it? Where are the differences in terms of ALM functionalities, what does this mean for quality assurance and thus test automation which is indispensable in S/4HANA projects and agile cloud system landscapes? Let’s take a look at these questions together in the following blog post! SAP Cloud ALM is an application lifecycle management tool that was launched by SAP in mid-2020 as SaaS – the licenses for SAP Cloud ALM are included in Enterprise Support, Cloud Edition. The basic idea is and was to use this product to expand SAP’s ALM track for cloud applications, which was previously occupied by SAP Solution Manager. Application Lifecycle Management tools help companies to efficiently manage and organize the complete lifecycle of an application from development to operation. This includes tasks such as requirement management, project management, software development, quality assurance as well as the roll-out of changes (deployment) and maintenance during operation. Modern, agile project approaches and continuous delivery require efficient processes, including close-meshed quality assurance, in order to keep up with the pace of development. ALM is designed to help accelerate deployments and massively shorten release cycles. Due to its Saas architecture, SAP is able to provide continuous updates and new functionalities for SAP Cloud ALM, which – by the way – is urgently needed especially at the current state (07/2023), as important features are missing in order to be able to manage and control a complete lifecycle of applications. The basic idea was to design SAP Solution Manager for on-premise products and SAP Cloud ALM as the ALM tool for cloud applications. Quickly, SAP customers were caught up with the future outlook when, as mentioned earlier, SAP announced the end of maintenance for SAP Solution Manager as 2027. Of course, Solution Manager can also be used for hybrid system landscapes and especially Cloud ALM, as from 2027 companies will be driven towards replacing SAP Solution Manager. The most exciting aspect for test automation tool vendors like suxxesso is, of course, quality assurance. As one of only two currently valid SAP-certified test automation tools integrated into SAP Solution Manager, suxxesso gained extensive know-how in the interaction of Solution Manager with our suxxesso Tool Suite. This knowledge includes, among other things, the effort required to perform efficient quality assurance in Solution Manager, as well as knowledge of the additional preparatory work required – in consideration of the (lack of) usability of Solution Manager. At the same time, suxxesso drives forward and will continue to push ahead the integration in Cloud ALM in order to offer those customers who are among the early adopters the usual comprehensive and stable integrations in SAP tools. The first conclusion, viewed purely in terms of the integration and the test process, is entirely positive. Important core elements of test automation, such as managing “systems under test”, the ability to orchestrate test chains, run role-based tests and also integrate third-party applications into quality assurance, have been placed in the hands of test automation tools. Integrated tools such as our suxxesso Tool Suite are not limited to pure automation and test case playback, but can bring their full strength to bear – as stated in the points above. SAP has given suxxesso gmbh massive support, especially in supporting all suxxesso Data Manager functionalities, role steps and cross-system test cases, and has also opened up the interface to our requirements. This means that Cloud ALM customers can also use the full range of THE most highly specialized SAP test automation solution on the market, in contrast to the limited integration option in SAP Solution Manager. This again brings massive advantages over traditional, generalist test automation tools. Convince yourself of the possibilities and the integration of suxxesso Tool Suite in SAP Cloud ALM during the DSAG Annual Congress in Bremen, where we will offer a live insight into our integration and highly specialized SAP test automation in our partner presentation “Live Test Automation for Cloud ALM: Gamechanger in SAP-Projects”
Advanced
The Role of Test Automation in Change and Transport Management during SAP S/4HANA Implementation
SAP S/4HANA implementations are more than conventional projects, regardless whether it is a greenfield or a brownfield approach. In fact, this implies rather a profound change in processes as well as in functionalities regarding the digital transformation of system landscapes. The impact on the staff here should not be forgotten: they experience the changes often in daily activities and tasks containing adjusted processes, unfamiliar user interfaces and complete new systems. In order to be able to insure a structured and organized transition during the implementation of S/4HANA, a proper change and transport management is essential. Well-structured documentation and thorough supervision of the complete process, beginning with the ideas of a change or a new functionality through to the change transports in the productive systems, is crucial for such big changes in a company. Change management processes including transport management are set up for these tasks which are run and maintained software-assisted according to the latest standards. These should absolutely start before the actual implementation project. Sustainable Change and Transport Management supports companies not only in the implementation of S/4 HANA systems but also during the whole lifetime of the systems in daily business. The range, the number and also the extensive scope of these changes in such transformation projects make a precise quality assurance of all changes essential. Not only selective, affected changes of a certain object have to be accordingly examined and carefully quality assured, but also its possible side effect on other objects and functionalities. Especially in transformation projects, it is vital for a company to gain the trust of its employees involved and their acceptance of the (partially) new systems. Instable systems including constantly recurring errors jeopardize even in the implementation phase the acceptance of the project and its support by employees. That is why from the beginning the goal must be to uphold the quality of the project. The number of changes for companies which will come up soon, makes quality assurance assisted by test automation indispensable. Companies are thus faced with great challenges and a good deal of project costs due to the number of changes as well as transports and the associated test workload. Once test automation has been intelligently implemented and in change and transport management integrated, massive effort and cost savings within the SAP S/4HANA systems´ implementation project can be achieved and can further lead to quality guarantee during operation. Automated test runs that have been started previous to the transport and are dynamically compiled, including the test cases which test the modified and altered functionalities, make sure that no unwanted side effects and unforeseen errors occur in the systems. Fully automated responses of these test runs act as quality gate prior to the actual transport, which will only be executed once the quality criteria of the quality gates are met. Companies often face challenges while testing all changes and transports and their possible side effects merely manually, which then run into a partly highly-restricted quality assurance and therefore at the expense of the core systems of a company. A test concept based on sheer manual testing – in addition to the mere aspect of problems caused by insufficient quality assurance – have a negative effect on daily business, thus resources are tied down to the quality assurance of the systems. Within the digital transformation, test automation plays a central role in implementation and operation of a modern change and transport management, since this cannot be executed with justifiable efforts without an objective quality assurance and an extensive test coverage for SAP S/4HANA projects.
Advanced
Using Decision Trees in SAP Test Automation
Test automation as part of a company´s quality assurance measures has to comply with principles, methods and measures of acknowledged test methodology and testing practice, in order to deliver an objective and reliable evaluation of SAP systems´ quality, just like all other components of this highly sensitive professional discipline. Our blog article today covers the topic application of decision trees in SAP test automation which is often subject of controversial discussion between the experts. Decision trees, in programming context If-ElseIf-Else clauses or If-Then conditions, are used when processes behave differently from a specific point on depending on the specification of a criteria (e.g. type of material). Thus, applying decision trees in a test case leads to the fact that not only the test case process itself but also the result can be different depending on the execution. Your first reaction may be: That must not be! A test case has to be conclusive and distinct in its behavior regarding process and result. Before we amplify our approach to decision trees and their application in SAP test automation, let us look briefly at a practical example and some approved testing guidelines which we think are particularly important. We test the variant configuration of our SAP systems and want to equip them with the highest test coverage possible. Now, our variants differ in certain parts based on the required data and input fields. Therefore, particular process flows come about at certain points which quite differ from each other. In terms of quality assurance, this means: a test case must be created for each explicit process flow. Two guidelines of renowned international testing methods provide us with a fundament for our problem: Based on the two presumptions above, we now want to outline why in our opinion decision trees definitely have the right to exist in test automation and this conforms to approved testing practice. However, it is important that both stated guidelines above are fulfilled. As already stated above, without the application of decision trees a separate test case has to be created for each version of our practical example. These separate and explicit test cases meet all criteria of approved testing practice: When applied to test automation, this is linked to a certain effort, since all different variants of the process cycle have to be represented in single test cases. Decision trees provide us with a possibility to represent processes which differ in some parts from each other in one single test case. So far, so good! But is this with regards to a neat test method valid as well? Following aspects have to be clearly and precisely determined for each iteration of our decision tree test case BEFORE the execution: For each it must be clear which will be run through and which so that we can qualify the test case iteration as successful. Are all these 3 aspects considered, then decision tree test cases fulfil all quoted guidelines of testing practice: In conclusion, we think that decision trees in compliance with the determined rules fulfil all criteria of testing methodology in the application of test automation. Decision trees, if used correctly, help to optimize the effort of creating but as well the maintenance of test cases and thus manage complex and extensive test scenarios more easily, such as those for variant configuration.
Fundamentals
The most important Success Factors for sustainable and efficient SAP Test Automation
Once companies start thinking about making their SAP quality assurance better and more efficient, they can´t get around the topic of test automation anymore. In order to make use of the benefits of automation supported testing to its full extent, some important success factors have to be considered. The following blog post is based on several customer conversations, their retrospectives and our decades of test automation experience. The underlying cause for every change is an unsatisfying initial situation. With respect to quality assurance of your SAP systems, these can be various aspects which lead you to a desire for change and efficiency improvement. These or similar concerns are the reason for companies contemplating about adjustments in their testing strategy towards test automation or rather a radical change of their test automation solutions which are neither implemented efficiently nor sustainably. The decision for automation-supported testing has great impacts on the quality assurance and the testing process. The benefit achieved through test automation solutions depend on important strategic considerations which you should face BEFORE your decision. Let us attend to the essential question and issues which determine success or flop in test automation. In addition to quality and function range of the solution, a key aspect regarding your automation strategy choice has to be whether the applied solutions can be implemented by your own team or if you need massive external support. Parameters involved are the increased implementation costs (project costs), great costs for knowledge transfer between experts and test automators as well as a planning dependency on external resources. Keys to success of a test automation are the high usability and the intuitive setup of the automation solution. Thereby the internal experts are capable of setting up and maintaining the SAP test automation entirely on their own and in fact without any additional external intervention. Often test sets are automated and work, virtually brand new, according to plan. However over the time, the number of functioning test cases recede and thus greater maintenance efforts are needed in order to correct the instable selections and to keep the automation machine running. Parameters involved are the increasing maintenance costs for test automation solution which eat up the benefit of automation compared to manual testing and the engagement of internal and external resources which have to take care of the unnecessary high maintenance of the solution. Key to success is testing business-critical systems such as your SAP landscape with highly specialized automation tools that are, regarding to the technologies in use, far more stable by definition as the generalized solutions which offer a broad technology support. The automation of your SAP processes represents a great ease for your testing team. However, this is only the midway! Your testing team has still to make sure that the test data is correct before each test run, is up-to-date and not already consumed for each test case. Parameters involved are unused potential savings in the testing process, error impacts which ascribe to incorrect test data, lost resources for daily business due to unnecessary high absorption of resources in the testing process. Key to success is the full automation! Rely on solutions which in addition to process automation provide a fully automated test delivery! Unpopular and disliked test data searching belongs thus to the past and your experts can attend to the important daily business instead of repetitive, monotonous test data searching. Mere process automation solutions are used cautiously because each run causes the already mentioned test data provisioning. As a result, the number of the test cycles arise hardly in comparison to manual testing and thus the possible improvement in quality will not be utilized. Parameters affected are a delayed return on investment, automation solutions which are maintained poorly (since scarcely used) and none-usage of the full potential of test automation. Key to success is, as already mentioned above, the full automation! You can achieve a great increase of test cycles since no extra manual effort is needed for additional test runs! Use your automation solution as often as you like and maximize thereby the benefit of your strategic investment! You invest in test automation, create a test automation set, and it is too complicated to use it in different systems? Do you use it only in your Q-environment? Parameters affected are unused potentials by usage in merely one stage (often Q) and unnecessary efforts for the adaption of your test sets for other mandates, company codes, … Key to success is once again the full automation as well as the parameterability of the test cases. Due to full automation, your experts have automated your processes by their own as well as helped configurate the fully automated data provision. And how does that do for you? All of a sudden your developers are able to run fully automated, functional tests on the DEV-Stage without manual efforts whatsoever. The system searches test data, regardless on which stage, and runs the automated test cases. Thus, you benefit from the great quality gain on the DEV-Stage before the developments go to the Q-environment. Provided the test cases are parameterizable, you can thus run the test set on different mandates, company codes, … without unnecessary configuration efforts, not to mention duplication of test cases. All these and many other considerations help you to maximize the benefit of your SAP test automation and not to fall into traps which minimize your benefit or even potentially nullify your investments.