At our company, QA (Quality Assurance) is part of the scrum team which also has developers, product owners, designers collaborating to arrive at a working product at the end of each sprint.
Having a core product in an e-commerce space with forever evolving feature set, we, the QA/testing team, have utilized open source test automation libraries towards product testing. With time not being a luxury, we chose to rest the NIH (Not Invented Here) factor for a while until our framework matures and demands enhancements.
In the world of open source tools, Selenium WebDriver is the most spoken about and arguably the most used test automation tool. Also taking into account selenium’s apt documentation and support, we started to develop our custom Functional/UI automation framework.
We wanted to have a simple yet scalable test framework with an idea to have most of the testing team members contributing towards test case development. We looked at different test frameworks like data driven, keyword driven, hybrid, etc and opted to go with keyword driven approach. Keyword driven provides that scope as it doesn’t require you to code or be a seasoned programmer.
And for mobile apps, we just added Appium which comes with a built in web driver API specifying a client-server protocol called JSON wire protocol.
We setup the libraries on eclipse in windows operating system and later extended to Mac OS X. Please refer to the respective website for setup instructions.
The Mac OS X setup, especially for Android and iOS apps, has additional steps but there is help available to debug configuration issues.
Note: Xcode and Android SDK are also required
We also made sure to document the setup instructions including our custom hacks in a wiki. I would recommend the same to everyone.
After the setup, we successfully deployed our Functional/UI test framework and tested for feasibility. Everything worked like a charm and hence we named our tool “Dexter” and this was the birth of “Dexter v1.0”.
As a start, we considered to write automated test cases for smoke test suite for Firefox and Chrome browsers by utilizing just the two engineers outside the scrum teams. With a lot of customization, the end framework is mostly hybrid.
The only effort was in identifying the elements and their identifiers like xpath and populating the object properties file. Of course we also had to write test cases in an excel sheet which has references to keywords and values.
If there are new features or changes to the design which change the element or object identifiers (xpath), we just need to look at correcting/adding to the properties file accordingly and correcting/adding to the Excel sheet. So it becomes very simple to maintain.
Once developed, this framework needs very little interruptions to the code.
We managed to develop smoke test case suite for all the platforms in about a month’s time and now it is hosted on Jenkins and on separate windows and Mac Mini test machines. You can choose which combination to run and submit a build and our framework takes care of running the test case suite and also sends you an email with results depicted as a pie chart and a web link to detailed results.
One of the metric that we looked at was “Time”. Now parallel test suites run within minutes to complete smoke test. For mobile apps, test case development is still in progress but our login suite takes ~5 minutes to complete on iPhone simulator/device and roughly 3 minutes on an android device emulator/device (we use Genymotion for android emulators).
We already have savings to report!
So we did achieve what we had set out to achieve: * single framework to trigger UI/Functional automation tests for both web and mobile apps * flexibility of any engineer contributing towards test case development
Simple, effective and scalable.
Tons of changes to follow as Dexter will be enhanced to 2.0, 3.0, etc, and tested for efficiency and robustness.