The wider adoption of Agile and DevOps practices in recent years have led to a significant shift in the way software is developed and tested. 2 emerging concepts are shift left testing (moving testing to the left on the project timeline) and shift right testing (performing testing in production). Both carry their own advantages, and in this article we’ll explore primarily the concept of shift right testing and explore the differences between shift left vs. shift right.

What Is Shift Right Testing?

Shift right testing, also known as testing in production, focuses on conducting testing activities after the development process. It involves gathering insights from real user feedback and interactions after the software has been deployed. Developers then use those insights to improve software quality and come up with new feature ideas.

Shift Right Testing Activity

Shift right testing aims to bring testing closer to the end users, hence the term “shift right”. There are 3 major criteria for any shift right testing activity:

Stability
Performance
Usability

shift right testing

Shift right testing is also where experiments happen, with QA teams trying out new features or improvements (usually with a carefully selected set of users). The only catch is that the software, application, or website needs to receive a considerable number of user visits in the first place for the experiment results to be statistically significant.

The table below lists out several notable activities you can try during your shift right testing sessions:

Activity	What To Do
Production Monitoring	Continuously monitor the performance of the software in the live production environment. Important items to measure include: Performance metrics Response times Error rates User interactions (user activities, clicks, navigation paths, usage patterns) Server health
Log Analysis	Analyzing logs generated by the production system to detect any abnormal patterns and troubleshoot issues early on
User Feedback Collection	Actively engaging with real users to gather feedback on their experience. The team can build an in-app survey form to collect their opinions when a certain event is triggered (such as installing the app, or during registration on the website)
A/B Testing	Create 2 versions of your application/webpage and split the incoming user visits between those 2 versions. Run them for a sufficient duration to collect data on any metrics you are measuring.
Feature Toggling	Use feature toggles to selectively enable or disable specific features or configurations in the live environment, allowing for more controlled rollouts and quick adjustments based on user feedback.
Continuous Improvement	Using insights gained from monitoring, feedback, testing, and analysis to drive continuous enhancements, prioritize feature refinements, optimize performance, and address usability issues.

Benefits Of Shift Right Testing

Real-time Feedback: the sole purpose of shift right testing is to test with real data for real results instead of using simulated data. This brings back authentic feedback and even helps to uncover valuable insights on user preferences.
Improved User Experience: the insights brought back from shift right testing sessions is translated into technical improvements for the software and boosts its business value. Going beyond just fixing “bugs”, now the dev team can also optimize certain features to bring users a better experience.
Continuous Improvement: the shift right approach encourages continuous testing and learning about the software, knowing that there is always room for optimization for the software as more and more features are added. Over time, these insights can even be “consolidated” into principles about the software and the business itself, serving as reference for strategy development.

Tools For Shift Right Testing

Application performance monitoring (APM) tools: to monitor the performance and behavior of live applications in real-time to identify performance issues. Examples: New Relic, Dynatrace, AppDynamics
Log analysis tools: to extract meaningful data from logs to pinpoint the root cause of any system error. Examples: ELK Stack (Elasticsearch, Logstash, Kibana), Splunk, Graylog
User analytics tools: to help you collect user behavior data and gain insights into their preferences, guiding UX optimization activities. Examples: Google Analytics, Mixpanel, Amplitude
Feedback collection tools: to gather feedback from end-users to understand their experiences and make necessary improvements. Examples: UserVoice, Zendesk, Intercom

What Is Shift Left Testing?

Steps To Shift Left Testing: requirement analysis, design, development, test, then production and maintenance

Shift Left testing is an approach that focuses on conducting testing activities earlier in the development process. The idea is to shift testing to earlier stages rather than waiting until the final stages. This proactive approach aims to identify and resolve defects early, preventing them from spreading throughout the entire application, reducing the cost and effort required for fixing issues.

Many QA teams in the past often delayed testing until the final stage, believing that testing should be separate from development and performed only when the product is fully built. However, this also means that there is not enough time for testing, causing delays before the app can be released.

Shift left testing is an innovative approach where testing and development happens almost at the same time, with the testing giving continuous feedback to the development team as they build the software.

Read More: What is Shift Left Testing?

Benefits Of Shift Left Testing

Early Bug Detection: at earlier stages of development, bugs are less expensive and easier to fix since there is less code and fewer dependencies between components. By minimizing the number of interactions and connections between software features, bugs are less entrenched in the software design. Testers can troubleshoot without making drastic changes to the existing code.
More Flexibility For Both Teams: with shift left testing, the development team can build the app and quickly get feedback from testing. Similarly, the QA team can test smaller chunks of code one by one instead of a full build with a complex network of bugs. Both teams benefit from this approach.
Improved collaboration: shift left testing leverages the expertise of both dev and testing teams to increase project efficiency. It fosters innovation through the diversity of skills and perspectives involved. However, they must have a shared understanding of expectations, so using a testing tool with collaboration capabilities is a recommended starting point.

Tools For Shift Left Testing

Static code analysis tools: to help scan code and identify coding errors issues early in the development process, allowing developers to quickly review their code. Popular tools: SonarQube, Checkstyle, ESLint
Unit testing frameworks: to automate unit testing, so that developers can run tests themselves before committing code. Example frameworks: JUnit, NUnit, pytest
Test automation tools: to automate test creation and test execution, helping QA teams quickly catch bugs and find root cause. Examples: Katalon, Selenium
Continuous integration tools: to automate the build and testing processes, ensuring that code changes integrate smoothly and identify issues promptly. Examples: Jenkins, Travis CI, CircleCI

Shift Left vs. Shift Right Testing

So what is the difference between Shift Left and Shift Right Testing? The answer is that they are only 2 sides of the same coin. They don’t cancel each other, but rather complement each other, and using both approaches can actually bring a more comprehensive testing experience to the team.

In this table, we compare shift left vs shift right testing based on 7 criteria:

	Shift Left Testing	Shift Right Testing
When It Happens	Early stages of software development life cycle	Post-production, in the live environment
Objective	Early bug detection and prevention	Real-time monitoring and user feedback for improvements
Activities	Static code analysis, unit testing, code reviews	User acceptance testing, monitoring, feedback analysis
Emphasis	Prevention-oriented	Improvement-oriented
Test Environments	Development and testing environments	Production environment
Collaboration	Developers, testers, QA teams	Developers, testers, operations, customer support
Iterative Approach	Yes	Yes

Level Up Your Testing With Katalon

Katalon logo

Katalon is a modern, AI-augmented test automation solution and quality management platform for web, mobile, API, and desktop applications. It provides a unified platform for teams to plan, design, execute, and manage automated testing efforts. Most importantly, Katalon has everything you need to shift left all testing activities of your organization.

Early automated testing with Katalon Studio: Katalon has an extensive library of Built-in Keywords, which are essentially code snippets to perform any interactions you want with the system, such as Open Web Browser, Click, SetText, and many more. Just drag-and-drop them in order and you’ll have a full test script ready to be executed across a wide variety of environments, OS, browsers, and devices.
CI/CD integrations: Katalon seamlessly integrates with popular CI/CD tools as well as collaboration tools such as Slack, Microsoft Teams, Git, improving transparency and enhanced communication between team members.
Test Management Capabilities: Testers can store application elements in the Object Repository, a centralized location for test artifacts. This allows for sharing and reusing test cases and data among team members and projects, enhancing test reusability.