Xamarin Test Cloud: An Introduction

The best way to validate the behavior of an application is by testing it continuously. If the behavior matches the expected results, with no crashing or returning incorrect results, the application is said to be in good form and can be implemented without issues. This process of testing the UI of an application by using and interacting with it is known as UI Acceptance Testing.

One of the best solutions to perform UI Acceptance Testing is through the Xamarin Test Cloud which is a cloud-based service. It enables developers to perform automation testing on a mobile application UI on the cloud which offers numerous mobile device simulations. Being cloud-based, the maintenance efforts are distant for the test consumer. This service ensures that the application performs suitably and efficiently across a variety of devices with minimum efforts.

Frameworks for Xamarin Test Cloud:

Automated UI tests can be written using either of two frameworks, namely Xamarin.UITest and Calabash.

Xamarin.UITest – It allows tests to be written in C# by using the NUnit testing library.

Calabash – It allows tests to be written in Ruby using Cucumber. Calabash enables the user to write and execute tests which validate the functionality of iOS and Android Apps. Calabash tests are focused on creating executable specifications popularly known as Behaviour Driven Development.
banner-img-1

These automated tests run on the application’s UI and validate the application’s Behaviour irrespective of the framework used to write the tests. Once the tests have been created locally, the mobile app and tests are uploaded to Test Cloud, where the app is installed and tests are run on hundreds of physical devices. Test Cloud sends a notification with the test results after all the tests have been conducted. Both frameworks for the Xamarin Test Cloud use a process which is known as the Test run.

Xamarin Test Cloud Agent:

Frameworks used in Xamarin Test Cloud services cannot interact all along with the mobile applications. An automation library is required to run the tests on the device. This automation library is provided by Xamarin Test Cloud Agent. The Xamarin Test Cloud Agent stands as a bridge that enables to run the tests on the IOS and Android mobile devices. It provides APIs that interacts with native iOS and Android apps running on mobile devices.
The Xamarin Test Cloud Agent contains libraries which enable test-code to interact with native and hybrid apps emulating end-user actions. The action can be:

  • Gestures: Touches (e.g., tap, swipe, and rotate the screen)
  • Assertions: For e.g., “Login” button on the Login screen
  • Screenshots: Screenshots of the current screen on the current device model

banner-img-2
The Xamarin Test Cloud Agent further splits into two parts because the tests running on the device are actually initiated and queried externally, either from the command line when run locally, or from Xamarin Test Cloud when running in the Cloud.
Xamarin.UITest and Calabash uses a client-server model in which the server is a web server that runs on the device and allows a client to connect and initiate the tests. It then communicates through JSON over HTTP protocol.
Therefore, the actual tests run in a host, and the actions that are specified in the tests are executed on the device.

Xamarin Test Cloud Agent in iOS:

Test Cloud uses the Client-Server model in which the server is built in the app under test. The communication between the Test framework and the Application is set up through JSON over HTTP protocol.
banner-img-3

Xamarin Test Cloud Agent in Android:

Test Cloud in Android also uses the same Client-Server model. However, the server runs as a separate application which can instrument the app under test. This becomes possible because of the Test Cloud signs both application packages (Server and App) with the same key, so they can bypass the security sandbox on Android.

Calabash and Xamarin.UITest are very potent UI testing tools for any mobile app. Both Ruby and NUnit approaches allow testing of all types of UI. Combining these possibilities with Xamarin Test Cloud Technology you can provide your users with the maximum level of quality across many Android and iOS devices with different OS versions. The limitation of the Xamarin Test Cloud is the lack of support for other platforms (For e.g., Windows Phone, Windows RT) and the current license price of Xamarin Test Cloud.The best way to validate the behavior of an application is by testing it continuously. If the behavior matches the expected results, with no crashing or returning incorrect results, the application is said to be in good form and can be implemented without issues. This process of testing the UI of an application by using and interacting with it is known as UI Acceptance Testing.
One of the best solutions to perform UI Acceptance Testing is through the Xamarin Test Cloud which is a cloud-based service. It enables developers to perform automation testing on a mobile application UI on the cloud which offers numerous mobile device simulations. Being cloud-based, the maintenance efforts are distant for the test consumer. This service ensures that the application performs suitably and efficiently across a variety of devices with minimum efforts.
Frameworks for Xamarin Test Cloud:
Automated UI tests can be written using either of two frameworks, namely Xamarin.UITest and Calabash.
Xamarin.UITest – It allows tests to be written in C# by using the NUnit testing library.
Calabash – It allows tests to be written in Ruby using Cucumber. Calabash enables the user to write and execute tests which validate the functionality of iOS and Android Apps. Calabash tests are focused on creating executable specifications popularly known as Behaviour Driven Development.
These automated tests run on the application’s UI and validate the application’s Behaviour irrespective of the framework used to write the tests. Once the tests have been created locally, the mobile app and tests are uploaded to Test Cloud, where the app is installed and tests are run on hundreds of physical devices. Test Cloud sends notification with the test results after all the tests have been conducted. Both frameworks for the Xamarin Test Cloud use a process which is known as the Test run.
Xamarin Test Cloud Agent:
Frameworks used in Xamarin Test Cloud cannot interact all along with the mobile applications. An automation library is required to run the tests on the device. This automation library is provided by Xamarin Test Cloud Agent. The Xamarin Test Cloud Agent stands as a bridge that enables to run the tests on the IOS and Android mobile devices. It provides APIs that interacts with native iOS and Android apps running on mobile devices.
The Xamarin Test Cloud Agent contains libraries which enable test-code to interact with native and hybrid apps emulating end-user actions. The action can be:

  • Gestures: Touches (e.g., tap, swipe, and rotate the screen)
  • Assertions: For e.g., “Login” button on the Login screen
  • Screenshots: Screenshots of the current screen on the current device model

The Xamarin Test Cloud Agent further splits into two parts because the tests running on the device are actually initiated and queried externally, either from the command line when run locally, or from Xamarin Test Cloud when running in the Cloud.
Xamarin.UITest and Calabash uses a client-server model in which the server is a web server that runs on the device and allows a client to connect and initiate the tests. It then communicates through JSON over HTTP protocol.
Therefore, the actual tests run in a host, and the actions that are specified in the tests are executed on the device.
Xamarin Test Cloud Agent in iOS:
Test Cloud uses the Client-Server model in which the server is built in the app under test. The communication between the Test framework and the Application is set up through JSON over HTTP protocol.
Xamarin Test Cloud Agent in Android:
Test Cloud in Android also uses the same Client-Server model. However, the server runs as a separate application which can instrument the app under test. This becomes possible because of the Test Cloud signs both application packages (Server and App) with the same key, so they can bypass the security sandbox on Android.
Summary
Calabash and Xamarin.UITest are very potent UI testing tools for any mobile app. Both Ruby and NUnit approaches allow testing of all types of UI. Combining these possibilities with Xamarin Test Cloud you can provide your users with the maximum level of quality across many Android and iOS devices with different OS versions. The limitation of the Xamarin Test Cloud is the lack of support for other platforms (For e.g., Windows Phone, Windows RT) and the current license price of Xamarin Test Cloud.[:]