Choose an Automation Tool that Supports Cross-Platform Testing?

I would like to share with you an exciting project our automation tool team had worked on. It so happened that there was an existing Web UI Test automation framework built by another team which had failed in production. That framework was designed to test the business functions of the client’s newly launched web UI-based eCommerce shopping website.

Our team was assigned the responsibility to fix it. We were informed that the Automation tests had missed catching several defects, eventually reaching the customer – which obviously turned out expensive to the client.

As we started analyzing that framework, we were actually pleasantly impressed with its overall architecture and design. We also observed how well it covered the business rules to be tested. They had a traceability matrix planned and executed with regards to the business requirements coverage. When we initially ran the tests on our Windows 10 desktop, using the Chrome browser, all went fine. There were no errors. So, what had gone wrong then, any guesses?

Let me share with you the defects that were reported, and this should help you get a gist of what had gone wrong. Note that we never saw the below-mentioned issues when we tested it on the Windows 10 Desktop, using the Chrome browser–

A) Several iPhone users reported that the Shopping Cart was not functioning. The counter in the cart failed to reflect the correct count as items got added and deleted.
B) Some Linux OS users who used the Mozilla Firefox browser reported that the Webui layout was looking misplaced.

Yes, as you may have noticed, these defects had been caught by the user because the tests (before the application was deployed for users) were run only on Windows/Chrome combination, and like I mentioned there were no such issues in this platform combination. The automation tests were designed with no cross-platform/OS/browser support.

With this perspective in mind, we as an automation team set off to improve the automation framework to consider the real-world use-cases. We kept in mind that the web application ought to be compatible with all types of mobile phones, tablets, iPads (which come in several screen sizes and platforms) as well. Hence, we had to build the framework such that it should be possible to test the real-world scenarios on all the possible combinations of the platforms. We then treaded towards enhancing the test automation to provide Cross-Platform/OS/Browser/Mobile technology support. This endeavor was an insightful journey indeed. Glad to share the tips that helped us during this journey!

Table of Contents

Tip 1 – Research the market for automation tools that support cross-platform testing. Perform lots of brainstorming as a team! Involve the stakeholders.

We together researched several automation tools that support cross-platform/operating system/browser/device testing. In the process, we discovered many tools like TestProject, MicroFocus, TestingBot, etc. We even found some tools that offered to let us try the tool for free and some were available with trial options. This was a boon for us as it allowed us to experiment to check the several features.

Several projects forget to consider mobile platform support. These days a lot of the web UI applications are being accessed using mobiles more than ever before. The tools that we found that matched our requirement had the capability that we could plug the mobile/tablets to our desktops and automate tests on our desktops.

We needed to be sure that the tool supports the maximum number of OS/Platform/Browsers/Mobile technologies. We checked to see which latest versions of platforms it supported. Most necessarily, we checked that in the future, if there are new versions of the platform/browser, would it pick up supporting the latest version. As a shortlisting strategy, we checked if the automation tool helps to quickly incorporate the latest version of the platform/OS/browser as soon as it gets launched. Hence, do not forget to check for immediate version support.

Tip 2 – What about optimum coverage?

Encapsulating all the combinations of platforms, browsers, etc. would result in lots of tests that cannot be run in a limited time. Hence we ventured out to search for tools that support combinatorial test design / pairwise testing. We planned to make this as an input to our Automation tool. This resulted in an optimized input data set of platform choice to the Automation tool. Having such CTD tools as an add-on/extension will help the project in the long run.

Tip 3 – Check if the automation tool has a self-healing AI feature. Self-maintaining tests to the rescue!

Heard about self-healing AI? Automation tools that have this magical feature result in tests that do not require manual maintenance. The self-healing AI capability helps the tool fix errors on its and proceeds testing. This is particularly useful in scenarios of cross-platform support. Some tools that support self-healing AI are- TestProject, Parasoft, etc.

Tip 4- Beautiful Test Analytics support!

There will be several combinations of tests being considered. Eventually, it would be ideal if the tool helps you explore in-depth insights into your tests across multiple platforms and environments. This will help the team analyze failures. It will also help the team identify testing trends by showing your product’s trends across various platforms, browsers, and devices. We found TestProject to have reports that helped us in this regard.

Image credits: screenshot taken from the website

Tip 5 – Keeping the communication running between the web application development team, business analysts, and the web automation team.

Keep the communication running between the team members. Also, you could involve the Business analysts in the whole process to understand more about the prioritization of platforms while you design the tests. Make use of the analytics report feature to share info with the other teams.

In Conclusion

With these quick, handy tips in hand, I am sure your web UI automation project will be a success. Even if the client doesn’t specifically mention requiring cross-platform support, it is imperative that the automation team starts up the discussion and provides information on why it is vital for the client’s success, because of the diverse platforms that the Web applications can be accessed from. Strategize well in this regard, and then start your automation journey!