Before being published to app stores, each mobile app is tested. However, the mobile app testing process is not so simple and clear-cut. We have outlined the key steps of the mobile application testing process that will help you obtain high-quality results for the entire project and establish a solid testing strategy.
When conducting a technical audit , we at Orangesoft often face the fact that QA specialists join the project later than they should. If you start the software testing process only after you get the first build of the application, there is a risk that mistakes were made at the stage of determining technical requirements and specifications. Reworking the final product is always more challenging. Below we describe the important first steps of your testing strategy that will save you time and money.
Once you have your project idea, your business logic, and your target users , you should determine the mobile devices and the OS versions your mobile app will be compatible with. These will be the devices you will run your tests on as well.
To figure out the best mobile devices for your app, you need to answer the following questions:
As a rule, we use the following resources: Apple and Google distribution dashboard, Device Atlas , and Statcounter (a very convenient resource with information about platforms, devices, and data organized by country).
Your choice of the devices and the operating system will affect the functional requirements, application design, and the devices and OS used for software testing.
Usually, we recommend choosing at least 3-5 reference devices for each platform with a different screen aspect ratio, different DPI, and different versions of OS.
Once the technical specification is completed, a QA specialist should be involved. If the design is ready as well, having a QA specialist at this stage is even more important. The specification and design prototype testing allow you to identify defects before they become an integral part of the mobile app .
A QA specialist will identify the incompleteness of descriptions, ambiguous wording, contradictions, and imprecisions. Design prototype testing eliminates the discrepancy in the technical specification and can help to identify problems with end user experience.
This approach will save you from challenges later down the line, and it will save time for the entire team and keep you on budget.
At this point, the project has the final approved specification and design. The project management methodology, which usually determines the appropriate software testing method, has also been chosen. Everything is ready for the next stages of development . However, before you start testing, you need to take a few more key steps.
Well planned is half done.
The fundamental document for the QA process is the test plan. This is a regulation document that describes everything related to testing a project. It includes:
This includes the introductory information that will allow any specialist to easily understand the idea and purpose of the project, its target audience, description of all parts of the project, and the parts that are subject to software testing. Most of the time, the project has a back-end, an administration system, as well as at least one mobile app. Sometimes the project may contain several administration systems or a website in addition to mobile applications.
This includes everything that may be useful for QA specialists, including the links to project space, technical specifications , project stack information, design source files, and a list of test logins and passwords.
Links to test checklists, test scenarios and test cases should be added to this block as soon as they are prepared. At the start of the description in this section, links to document templates can be placed.
In the first step, we identified the device models and OS versions that the application should support. Now, we need to determine which device models and OS versions will be tested. It’s extremely rare to test 100% of OS versions and device combinations.
Using emulators is another issue to consider. When you perform mobile app testing using an emulator, some difficulties may arise. For example, issues may come up with Face / Touch ID, fingerprint sensor, camera, voice control, device’s memory testing, voicing interface elements, shaking the device, multi-tap, and other control gestures. Plus, there is a significant difference between the application's behavior in the emulator and on a real device. Thus, some bugs are reproduced only on real devices and vice versa.
If the mobile app involves integrations with other apps or different devices, the QA should know the details. It is important to determine how integration testing will be performed and how to access the devices. For example, this is relevant for testing IoT mobile applications, where you need to have specific knowledge about the device function or the object which the app is integrated with.
The test plan should specify a team of testers and the role of each member. However, it’s important to not only fix the team members on paper but also to hold an intro meeting with the team to discuss the roles, plans, tasks, and expectations of all project participants. All QA specialists must keep all testing documentation up-to-date throughout the entire development process.
The whole team should know the exact list of systems and tools used on the project. For example, here are a few of our favorites: ADB , Crashlytics , Postman , Instabug .
We recommend devoting time to learning new testing tools regularly.
Formulate the criteria and the deadlines for testing if the release date is fixed. If you plan to arrange private demonstrations during the development process, these dates are also important to fix in the QA team work plan.
At this stage, it is necessary to analyze the project’s requirements, assess the risks , and consider options for reducing them with minimal costs. Usually, during this analysis, we try to answer the following questions:
Let's take a closer look at all the possible forms and types of mobile app testing. Usually, you would decide on the type of testing based on the project requirements, the budget for testing, the planned release date, and the team’s expertise.
Testing forms are the following:
As a rule, the most winning scenario is a combination of testing forms and types. For example, automated tests give the most accurate results and are available for reuse, but writing them can take an unreasonably long time. You can also perform manual testing of any app, sometimes even without any preparation (so-called “hallway testing”). The farm allows you to run testing simultaneously and even remotely on multiple devices.
The correct combination of manual and automated testing forms and types allows you to optimize the testing budget and get a high-quality mobile app .
Let's move on to the testing types.
This is the type of testing that ensures that the application performs its functions correctly and meets the expectations described in the technical specification of the project. Depending on the features of mobile devices, this type of testing can include:
Automation testing does not replace the manual one since the creation of autotests and their maintenance are time-consuming, especially when the requirements change quickly enough. However, autotests are a good option for performance testing and regression testing of important application features. Regression tests must be carried out every time after a bug has been fixed or new functionality has been added to make sure that those actions haven’t affected the rest of the functions. Due to the frequent repetition of the same actions, the human factor comes into play, and QA engineers could miss critical errors. Autotests avoid the human factor and free up time for more creative manual checks and new features testing.
This concerns the assessment of the interface elements' impact on performance. The app's user interface is checked to be compliant with the approved design prototype and the current guidelines from Apple and Google . Also, this type of testing includes checking the syntax (texts and formulations) for compliance with the style of the application, its requirements, and logic.
This is an evaluation of the application’s clarity and ease of use. It is difficult for team members to see the app with the eyes of the unprepared and unfamiliar with the specification and design users. For usability testing, analytics is the first thing to set up. Analytics allows testers to see how much time users stay on the screen, how much time they take to fill in the forms, how often they go back to the app, and how many times they activate a certain button. Once this has been accomplished, the users are divided into two or three groups that are given access to the application builds that contain small differences in one element of the interface or any particular part of the functions. Based on the collected statistics, we can decide which changes are perceived by users better. This type of testing should be taken into account whenever you estimate the time and budget for testing.
If the mobile app is designed for users from different regions, text translation and its display should be accurate. Cultural specificity and differences in the format of data output, as well as different units of measure should be equally taken into account. For example, the decimal separator can be a period or a comma, and the date display can differ depending on the region. Our team loves to use the Localise tool to organize localization in mobile apps.
This is the assessment of how convenient the work with the app is for people with various health conditions. For example, it tests whether the interface elements are correctly voiced. We recommend reading about the basic rules and tips for checking the inclusiveness of a mobile application on the Global Infrastructure Hub website or in a series of articles from Usable Net on Medium .
Security testing is targeted at finding vulnerabilities in the security mechanisms of handling end user data. For example, you cannot proceed with the registration in the application with a password that is too simple , like “qwerty” or “1234567.” The app should not request unnecessary permissions and should work properly, even if the user hasn’t given permission to use the requested data; sensitive data like logins and passwords should not be transmitted in an unencrypted form in requests to the server.
With the help of special tools, this type of testing checks the back-end performance with normal predicted load (the number of simultaneously active end users and the amount of transmitted data), with higher than planned load, and during prolonged work with a normal load.
The test plan is the first document on the list. Based on the test plan, forms, and types of testing, the following documents are drawn up:
It is important to describe the acceptance criteria, as these criteria will help evaluate whether the application performance is acceptable.
The test case includes a sufficient number of tests that cover all the functionality. As we previously mentioned, it is unfeasible to test all the possible combinations of input parameters. Choosing the right data is key to identifying potential bugs.
Once test cases are written, they get divided into sets that will run in different situations, for example:
As a result of this stage, you will get a list of test cases containing a description of the idea validation, the necessary steps, the expected result, and test-suites. Test suites are sets of sequentially executed test cases, for example, for end-to-end testing.
When working with checklists and test cases, QA specialists look for possible inconsistencies between the expected and the actual results in the app. However, simply finding them is not enough. It is important to describe all the bugs' details in a way that will allow the development team to easily reproduce and fix them.
For this reason, we use our bug report template for each project. No doubt, bug reports may look different for different types of testing and at different project stages. The type of project space and the project management methodology can influence the reports’ contents as well. However, there is a standard set of data/fields in every bug report:
The testing process lasts until all the planned functionality is implemented and the required level of quality is achieved. For this reason, keep in mind the following:
Based on the results of a significant intermediate or final testing, we draw up an additional report - a test report. Essentially, this is an extended version of the bug report, which gives the Project Manager and the Product Owner an overview of the product’s quality. This kind of test report may contain a link to the executed test suites indicating the devices and the OS covered, functional quality assessment, information about the severity status of the remaining bugs , and compliance with the acceptance criteria, as well as recommendations for improvement.
Acceptance test includes precise and complete testing, which may still be required to stabilize the app performance and identify minor bugs. At this stage of testing, it's significant to compare the planned functionality and the one accepted to be sent to the stores for publication. You may need to eliminate some redundant functions or add new ones.
The whole process of mobile application testing should be well-documented and be as up-to-date as possible. Double-check that the necessary entries have been made, then make your test summary report.
Test summary report should describe:
The decision to publish the app in mobile app stores should be made after the test summary report.
The work of the QA team does not end after the app release. If the team continues to work on the second version of the project, the whole process may start from the very beginning. However, even when further work is not required and the project is functioning successfully, it is best to set time aside for testing. In this case, the testing process will be a periodic check, which may include one or all of the following events:
Mobile application testing is a complex process, and setting it up requires effort from the entire team. You can adjust the steps of the testing process described above for each project. However, do not skip the above 10 steps during the testing process. This way, you are guaranteed to get a reliable result in the form of a high-quality mobile application.
Ready to test your app? Our dedicated testing professionals are here to help. Book a consultation to know more about our quality assurance services.
