Leveraging Benefits of AI and ML in Software Quality Assurance

In the last few years, there has been lot of development in the field of artificial intelligence and machine learning. A significant level of product development around self-driving cars, automation, bots and smart cities clearly indicates that the world is moving towards a new digital revolution. This evolving software and digital landscape is putting strain on product quality lifecycle. Today, the consumers have no tolerance for product defects and zero amount of patience resulting in products to be launched as soon as its conceptualization.

 

 

These transformations naturally have an impact on the quality assurance operations of a company. The QA process is increasingly becoming complex and vast. To add into this, the urgency for fast delivery is also increasing day by day. Today, there is a need for end-to-end product testing, but the time frame is very small that makes it challenging to create test cases that cover all the crucial test scenarios.

In this scenario, we can leverage the potential of Artificial Intelligence (AI) and Machine Learning (ML) to achieve high test automation.  These technologies can help in covering high risk test scenarios, achieving complete test coverage, and creating self-healing, independent test automation architecture.

 

What is Cognitive QA?

Cognitive QA is the process of using artificial cognitive abilities to achieve optimized high quality automation at fast rate. To define cognitive QA, we will look separately at the terms ‘cognitive’ and ‘QA’:

Cognitive means conscious thinking, reasoning, intellectual efforts, knowledge acquisition and decision-making.

Quality Assurance (QA) is the systematic approach of analyzing whether a product or software meets the quality standards and pre-defined requirements.

Therefore, cognitive QA can be defined as “Dashboarding and Monitoring solution to enable clients achieve an accelerated and optimized quality by using intelligent approach to QA leveraging artificial intelligence, machine learning and advanced analytical technologies (Digital marketplace UK government, 2019).” The World Quality Report 2017-18 has also cited that cognitive abilities, automation, machine learning (ML) and predictive analysis can augment test automation outcomes.

Cognitive QA leverage machine learning and AI to mimic human brain and extract patterns from existing data. Also, AI has the ability to deal with large amount of data to offer crucial insights in a very short period of time. AI enabled bots would have the ability to conduct all the testing and monitor the burgeoning amount of data, test cases and changing test environment.

The main components of cognitive QA system are:

• Ingestion: Interpreting information from raw data; transforming data so that it can be easily used by other systems
• Natural Language Processing: Ability to grab concepts and knowledge from simple plain text data;  ability of a machine to understand data just like human beings
• Machine Learning: Ability of machines to interpret data and use it for decision-making & making meaningful interpretations
• Rest API: It facilitate communication between different components of machine and system interface
• Deep Learning: It refers to the voice recognition and face recognition ability of a system

Cognitive QA helps the organizations to automate with intelligence. The predictive analytics and AI can be used to develop self-healing test architectures that have several features like pipeline failure analysis (reasons why a build failed) or hot-spot analysis (determining which software code might have defects). We can develop a self-aware and self-adapting test environment, which advances with time. This system can interact with software testers, IoT devices; cloud services and turns suitable data to label the testing sequence.

Quality Intelligence: Modes of Information

In making the necessary shift to cognitive QA, gathering actionable and contextual intelligence is vital. For this, we need to identify information channels that can deliver valuable insights to the social, technical, and business requirements. There are several sources of information such as build system, test management, operational analytics and social insights. Although, all these information are crucial, business can achieve powerful insights by correlating the information gathered from different sources.

Different channels offer different information such as social media analytics offer insight about new feature requirements and consumer expectations, whereas, operational data can shed light on the workflow and business operations. There is also plethora of information in the software development lifecycle which can reduce the release cycle and boost the team’s performance.  The business organizations need to create a unified approach of collecting these insights and using it to improve QA process. For this, the collected information should be audited for quality and consumer usage patterns as data quality and maturity is essential for accurate QA estimation.

Challenges in Cognitive QA

As discussed, collecting quality information and feeding it to the system is of paramount importance for the success of cognitive QA. However, the major challenge is developing element of learning in these systems. The system needs to learn from the past data and till then it will be ineffective and inaccurate.

Being a high potential technology, it comes with its own set of challenges:

• Extensive System Training: Cognitive QA requires extensive system training in application architecture, industry trends and using data for making accurate predictions. It requires huge initial investment in terms of cost and time.
• Dependency of Data: The cognitive systems derive its intelligence from the data or information entered into the system. If incorrect, misleading or misaligned data is fed into the system, it can jeopardize the testing process.
• There are several technical challenges such as building flexible test automation architecture, reliable test environment and finding the right skills and experience. Cognitive QA solutions should be able to offer accurate analysis of risk scenarios and quick remedial solutions.

 

Infusting Cognitive into QA

The Cognitive QA system should be able to work as an independent entity and should be able to design test by studying business operation flow. The first step in developing a cognitive QA approach is to create a transparent and reliable database. Cognitive QA is dependent upon the transparency of data as it will predict the automation requirements based on the past data. Keeping real-time information about key application features like number of application incidents and schedule for upcoming releases will also be beneficial in getting meaningful insight into application quality requirements. It can be achieved through a customizable system that can pull relevant information from different streams in real-time.

The next step is building algorithms that can take right decisions at different phases of software testing lifecycle. It needs a robust algorithm that can objectively identify the test cases where defects have not been identified, which test cases do not need automation and which test cases have a recurring feature that makes it more suitable for automation. These algorithms must be able to remove human bias and emotions from the data analysis process. It is important to align testing with actual user behavior smartly and achieving comprehensive data analysis.

This gives us better insight regarding the overall test automation process. The third and final step in developing a cognitive QA approach is creating complete test automation architecture. It should have features that can automatically select proper test scenarios, generate test cases, access synthetic test data, and create virtualized test environment. It should be able to facilitate automatic generation of test scripts, high productivity, and optimized test environment.

Figure: Steps of Developing Cognitive QA Approach Cognitive QA has high potential as they can penetrate through complex requirements and large set of conditions with minimal redundancies. Ideally, such systems should always be learning from the past test data which will be constantly fed into the system. Therefore, it requires intensive documentation of current software components and past software testing case studies. If this information is missing, then the machines will not accurately predict high risk-scenarios.