Mobile apps market is growing at a phenomenal pace, with leading research firm Gartner reporting that Mobile apps will generate revenues of $77 Billion in revenue by 2017. Millions of mobile applications are not just entertaining people but are becoming an integral part of our lives with people relying upon them for almost every daily need.
With many options present for users, a mobile app developer cannot risk publishing a product which is not thoroughly tested. And with the upsurge of mobile devices mobile testing activity including functionality, performance and security of mobile applications and devices, is a fast rising discipline in its own.
As of now iOS & Android and Windows 8 are the most flourishing mobile platforms; however Nokia’s Symbian, Maemo and MeeGo, Palm/HP’s WebOS, Samsung’s Bada & RIM’s BlackBerry OS are also active mobile platforms in today’s world. Thus, it becomes a tremendous challenge to verify the apps across the plethora of devices of different form factors and OS flavors present in the market.
With these daunting challenges in mind, we have come up with some strategies and best practices in mobile testing.
Strategies to be considered in Mobile Application Testing– In order to test a mobile application systematically, strategies need to be finalized in the initial phase. One can categorize the strategies as:
- Short listing devices: One should finalize a set of devices based on a market study & also by targeting a few which are higher in the population so that duration & cost of testing can be optimized. This will also result into the most realistic view of a test outcome.
- Utilizing Simulators: Applications can be easily deployed & tested without investing in any device. The simulators are generally freeware & various types of testing can be performed on them. Ready to use emulators are easily available in the market such as – iPhone Tester, Mpbile Phone Emulator, MobiReady, etc.
- Mobile Testing in Cloud: Cloud-based testing provides a host of benefits like testing on ‘n’ number of devices without hardware investment, ability to scale up by adding more devices for testing and testing several devices in parallel. Thus, optimizing testing time and the most significant upside is, one can use them on a pay-as-you-go model.
- Selecting Testing methodologies and types: Another important aspect of testing would be choosing testing methodologies which can be one or multiple or all of the following- Installation/Uninstallation Testing,Functional Testing,Usability Testing, Stress Testing, Load Testing,Security Testing,Performance Testing,Compatibility Testing
Best Practices in Mobile Testing: Below are some of the important best practices without which testing activity cannot be treated as completed.
- Use of Manual Testing: Manual Testing helps to verify application on device for real. It includes Device Testing (in Network), Device Testing (on Wi-Fi Network + Network Simulation tool). Manual Testing can never be completely replaced by automation.
- Test Efficiently & Effectively: App developers know how difficult it is to be among the top results in an app store for a particular category. With millions of competitors, you don’t want a bug to push your great idea to the bottom ranks. One should cover comprehensive factors like:
- Incoming call handling – e.g. Proper visible and audible notification should be displayed on screen. User should be able to take/ cancel the call without crashing or hanging application.
- SMS Handling – SMS alert should be received when application is running
- Charger Effect and Low battery – Notification should be displayed when charger is inserted or removed or device is running out of battery
- Verification of mobile application as per the specification
- Network Connection Testing i.e. 2G, 3G, Wi-Fi, CDMA, GSM connection
- Offline profile access correctly when making network connection. Application Start/ Restart are quick.
- Exiting the application should be easily possible through all exit modes
- Security: Security is paramount when it comes to mobile apps. The security testing for mobile should cover OS related, application related and/or network related vulnerabilities.
- Data being sent and received must be encrypted
- Test for unauthorized access and improve authentication systems
- Use of Automation: Automated testing has become a significant part of any testing activity. Once test cases are built they can be reused and are also scalable, thus saving precious time and money. The results of test are mapped to the expected behavior and crucial reports of the success or failure is also generated. Some of the automation tools available in market for functionality and usability test are:
Robotium for Android, Testquest, try, and digia for Symbian, FoneMonkey for IPhones, Memory sweep for IPhone, eggplant, VNC Robot, Hopper and TestQuest - Test for user friendliness of the app and ensure design is simple: An app with no clear navigability and abrupt information flows can ruin user experience. Following is a handy checklist related to testing the design part of your app
- Check the look & feel of the application
- Check for the logo, layout, design, format and content
- Check for the error messages and network error
- Continual keypad entry should not cause any error
- Battery & Memory Consumption: A resource hungry application may not find wider acceptance. Test your application for resource consumption and its impact on mobiles’ energy levels, and redesign till you get the optimum resource consumption.
- Test for Speed: Continuing with the practice mentioned above, a slow loading app would not find many takers. One should test the app’s performance metrics on all the device and OS versions they are targeting.
Mobile application testing is an evolving discipline, I have tried to cover some of the strategies and best practices, and this by far is not an exhaustive list. I look forward to hear your views and experiences in mobile application testing.
