Troubleshooting Native Mobile Tests

This document lists common issues and errors that can occur when running mobile tests in native apps and how to troubleshoot them. Native apps may be run on VMs or real devices.

Introduction

You can test mobile apps using our mobile VMs. To start, upload each app. Do this in the Setting > Sites > Apps area of the Rainforest application.
See this help article for instructions.

The apps should be correctly built and configured to run on the VMs. For detailed information, please see these help articles:

VM configuration considerations

iOS VMs

  • Should be a .zip file with the .app file included.
  • Should be compiled for x86.
  • The following line in the App Logs indicates that an app was not build to run in the simulators: Program specified by service does not contain one of the requested architectures
  • When creating the .app file the user must inform the -sdk iphonesimulator param in the xcodebuild command.
  • Test that your .app works on a local iOS simulator by dragging and dropping the .app file onto your local xcode simulator. If it does not work on your local iOS simulator, it won't work on ours.
  • If you need more help with xcodebuild check out Apple's documentation here and here.

Android VMs

  • Should be a .apk file.
  • Should be compiled to run on x86.

Real Device configuration considerations

iOS real devices

  • It should be a .ipa file.
  • Build with a valid "Minimum OS Version".
  • Apps build with iOS 10.0 as the "Minimum OS Version" can't run on iOS 9 devices!

Android real devices

  • Should be a .apk file.
  • It should be compiled to run on x86 and ARM.
  • Samsung Galaxy Tab 3 10.1" runs on x86!

Common Error Messages

Runs failing with "Couldn't download file…"

We recommend uploading the app via the CLI or directly in-app if possible.
See this help article for details. This error occurs when the application fails to download from the provided app direct download link.
To resolve this:

  1. Check to make sure the test is pointing to an App Source and not a Site URL on all environments.
  2. Check if you can download the app using only the specified App URL without a redirect, providing any credentials, or having to click on any link/button to start the download.
  3. Apps hosted on Dropbox should specify dl=1 as a query parameter on the App URL to force the app to download.

Runs failing with "Please check that your app is configured properly."

The error usually occurs when it was not possible to install or launch the app on the VM.

Runs failing with "There are not available __ devices at the moment…"

The error means there are no (real device) available phones of the requested type due to maintenance issues on AWS Device Farm. It can also show up when the app was not built to support the device's OS version.

  1. Verify that it is possible to reproduce the same error on the preview. If this can be reproduced, this means that this device is on maintenance on Device Farm.
  2. When this occurs let us know at support@rainforestqa.com or through the blue chat button so that we can monitor this. Usually, AWS will make the device available again after a few hours.
  3. If the preview works, please check that the app was built to run on the device's OS version. It is particularly important to check the app's "Minimum OS Version".

The browser loaded instead of the App at the beginning of the test

If the testers report that the browser was showing instead of the app this usually means that the app crashed right after being launched. If this occurs, please check the App Logs for the error event and watch the tester video.

App not installed on Android real devices

On Android tests using real devices, the app will not be installed if it does not support the Android version installed on the devices.

  • Usually, the following error message will appear when the app requires a newer Android version: "Requires newer sdk version XX (current version is YY)"

If you encounter an error that is not detailed above, please let us know at support@rainforestqa.com.

Did this answer your question?