Automated and manual testing: Confrontation or Collaboration?

“… you just can’t differentiate between a robot and the very best of humans.”

Isaac Asimov

Today’s confrontation of manual and automated testing bears a resemblance to the reflections of Isaac Azimov about the relationships between humans and robots. In his “I, robot” novel he questions, if the robot could substitute human. Robots perform better, faster and avoid mistakes. In turn, humans can work on issues inaccessible to any script. The debate about which testing type is better has lasted for years because each testing type is good in its own way and for its purposes.

A word in defence of manual testing

There are cases in which manual testing is unconditionally better:

• Exploratory testing – Automation is almost useless here, because skilled manual QA can create test cases in the course of testing.
• Conversance with the app – You just receive an iOS or Android build and don’t know what are you dealing with. You have to perform preliminary manual testing, detect the issues, and understand what part should be automated.  
• Testing visual artifacts – With regards to human perception, a tester could see incorrect text, distracting elements, low contrast etc. Automated script can’t.
• Localization testing – Only a human can evaluate the conformance of the product to the language and cultural standards.
• Testing in unstable or dynamic environments – Any changes in the source code prompts changes in the testing script. Automated testing requires some stability. If you have to adjust your script too often – manual testing is the only solution.
• If you are running short term – Test automation requires primary investments that will be compensated in time. On quick little projects, you may use less expensive manual testing, which will be wiser.

What about startups?

Unthinkable opportunities for entrepreneurial people have given rise to countless startups in the IT industry. Technology makes it possible to create a revolutionary product with a tiny budget, but as with any other custom software, these products need testing. And testing processes for startups have their own specific restrictions:

• Being a major investment, automation could be unaffordable for some entrepreneurs who run a startup.
• Startups have foremost dynamic products.  Changing behaviors require changing the automation and therefore double the expenses which can be critical for the budget.
• Some startups just have tight deadlines and a list of certain tests to perform, so there is no need for automation.

The compromise

Drawing again an analogy with the difference between the robot and the human – robots were invented to assist people, save time and automate routine processes; Automated testing has also been invented to perform routine processes. Luckily, the majority of testers and software development companies come to an understanding that each type should complement each other, and automated testing should follow manual.

Software testing archetypes

To demonstrate the difference between the manual and automated testing more vividly, let us fantasize a little. Imagine your brand new product as an unknown territory your army of testers should explore and conquer. Manual testers in this case will be the trailblazers or scouts. They are the first to explore the software, its functionality and how it impacts the user. Not a single script, even the most sophisticated, can’t simulate human flow of thought and previous experience.  After they have made a site sketch, detailed where the critical points are and gotten into the places that could be reached only by human beings, the heavy artillery arrives – automation testers.  They set up their massive weaponry, say “ready or not, here we come. You bugs, can’t hide” and begin to work. Regression, load and performance testing should be repeated consistently with each new update of the product – that’s where automated testing is at its best.  As in the case of the artilleryman, an automation tester requires more specific skills and knowledge than the manual QA engineer. Little projects will require several people, but to deal with huge ones you will need artillery.

“And” instead of “versus”

“You can prove anything you want by coldly logical reason—if you pick the proper postulates.”

Isaac Azimov

Solomon’s decision for deciding which testing type to use would be very simple: both. In this way you will benefit from each and minimize the shortcomings of rejecting one of them.

• Manual testing is for unveiling problems of any kind, not only from the user experience, but also those from serious bugs. It is also the only way to perform exploratory testing.
• Automated testing will significantly boost up iterative testing cycles, routine and time-consuming testing, saving you time for any unusual issues.

Eventually the conflict is mostly far-fetched. The only reasonable approach is to combine both methods. And in spite of the fact that automated testing is becoming more and more solicited, manual testing is definitely here to stay for a long time to come.

Ubertesters is happy to provide hardcore manual testers at your service.