Implementing Test Automation Framework

Implementing Test Automation Framework

Key facts:

- One solution that both automatically configures environments and executes tests

- Reduced time for testing each release from 4-8 weeks to 1-2 weeks

- 682 releases per year done by the customer’s team of five people

Customer

Based in Austria, the customer is a leading manufacturer of sophisticated test and measurement equipment for the automotive industry.

Challenge

The customer creates and markets unique software-hardware solutions for the world’s largest vehicle manufacturers. Test Center, its major testing facility, carries out quality assurance and controls all hotfixes, patches and releases of the company’s products. Approximately 1 500 physical machines united into 500 test systems are used to ensure Test Center’s operation. On average, 3 to 4 environments of different products and versions are configured on each physical machine. The number of environments on virtual machines reaches 5 000.

Besides such an impressive volume, software testing itself is very specific for the following reasons:

- each product version should be tested both in the standard and client’s specific environment;
- each product has a separate test system that was specially developed to meet the product’s needs.

Performing these challenging tasks manually could require dozens of software testing specialists and several months to test each release. There was a clear need to implement a test automation framework. The customer wished to invest in a solution that could automatically configure environments and execute tests. Having analyzed the test automation tools available on the market, the specialists on the customer side decided on Octopus, an automated testing control system by Applied Systems.

Solution

In close cooperation with the customer, Applied Systems’ specialists analyzed several different test systems. The analysis showed the generic parts of the systems shared a set of common problems and it would be more efficient to develop a common testing solution with a unified environment and reports.

To meet these requirements, Applied Systems customized and implemented its Octopus, automated the testing control system, specifically for the customer’s products. The team did it best to make a solution whose logic, algorithm, and architecture are easy to understand. Applied Systems provided the necessary support and documentation throughout the whole project.

Results

Implementing the test automation framework based on Octopus has increased the efficiency of the customer’s software testing process. In particular, it helps reuse tests disregarding their physical location and synchronize them both on the customer and developer side. As a result, the quality and reliability of the customer’s products have improved since it became possible to execute more tests and configure more environments.

Other achievements gained from test automation include the following:

- deploying the environment automatically on a pool of machines;
- executing a set of standard tests on each released product version;
- grouping installation steps from different products, preparing the system and performing this as a single task;
- backing up with the customer’s existing testing tools;
- easy parameterization through a centralized web-based client.

“Automated software testing allowed us to get rid of time-consuming manual operations like environment configuration, test launch, preparing test sequences, reporting, analysis, and processing of bugs. For example, now I can prepare the whole system for tests just by changing a couple of words in the command lines. Thus I am maximizing the use of time and assign my team more creative tasks”

Emil Stefan, Head of QA

Screenshots

Octopus in Test Center screen #1
Octopus in Test Center screen #2