From in-house testing to the cloud - how to scale your device farm

Instead of buying test devices and maintaining them on-premise, there is an ongoing trend to facilitate cloud providers for UI test automation. This talk will explain the basics and provide an overview of the advantages and limitations of this approach. It will also tell the story of how we moved from running a handful of tests against four of our own test devices to executing 100+ scenarios (and counting) on AWS Device Farm, Browserstack and a local simulator farm.


Outline/Structure of the Case Study

  • What are the options for end-to-end mobile testing?
  • Simulators vs. real devices
  • In-house vs. cloud farm
  • Code examples and demos (Java + Appium)
  • Pricing models
  • Tool integration and API support
  • Criteria for vendor selection

Learning Outcome

Attendees will learn about the pros and cons of using a commercial cloud testing platform vs. building their own in-house solutions. They will get an overview of the current market situation and the main cloud testing solutions which support Appium. They will also receive a simple set of criteria for selecting a suitable vendor for their own use-cases.

Target Audience

Everyone looking into scaling up their device farm



schedule Submitted 2 years ago

  • Srinivasan Sekar

    Srinivasan Sekar / Sai Krishna - Native mobile commands in Appium

    45 Mins

    Apple and Google’s test automation framework does not natively support W3C standards for few web driver spec implementations directly for e.g TouchActions interface in XCTest, etc. Although test automation frameworks support a rich set of those functions specific to platforms, Appium does provide ways to directly invoke these functions e.g gestures, biometric handling, etc.

    Many special behaviors and workarounds are made available and achieved only through executing platform-specific native commands of Appium. For instance, there are 100+ issues been reported on date picker or handling picker wheel in the appium organization but it can be achieved quite easily by executing native mobile commands.

    There are so many that testers might not get chance to go through each one of these and get acquainted with all of those. Native mobile commands help to handle much complex use cases like biometric handling, talking to Siri, performance profiling, etc quite easily.

  • Jonah Stiennon

    Jonah Stiennon - All Desired Capabilities

    Jonah Stiennon
    Jonah Stiennon
    Cloud Grey
    schedule 2 years ago
    Sold Out!
    90 Mins

    Appium has so many desired capabilities! How many exactly? Let's count!

    Join Appium contributor Jonah Stiennon as he iterates through every desired capability supported by Appium.

    Many special behaviors and specific workarounds are made available only through desired capabilities. Often the key to selecting an element reliably, launching an app, or avoiding a timeout, is picking the right set of desired capabilities. There's so many that testers don't get the chance to sit down and become acquainted with them all.

    There's too many to easily memorize and documentation can be sparse. Jonah will introduce novel ways to classify, visualize, and organize all of the desired capabilities, making it easier to find ones which can be useful.

  • 45 Mins
    Case Study

    Experimentation and quick feedback is the key to success of any product, while of course ensuring a good quality product with new and better features is being shipped out at a decent / regular frequency to the users.

    In this session, we will discuss how to enable experimentation, get quick feedback and reduce risk for the product by using a case study of a media / entertainment domain product, used by millions of users across 10+ countries - i.e. - we will discuss Testing Strategy and the Release process an Android & iOS Native app - that will help enable CI & CD.

    To understand these techniques, we will quickly recap the challenges and quirks of testing Native Apps and how that is different than Web / Mobile Web Apps.

    The majority of the discussion will focus on different techniques / practices related to Testing & Releases that can be established to achieve our goals, some of which are listed below:

    • Functional Automation approach - identify and automate user scenarios, across supported regions
    • Testing approach - what to test, when to test, how to test!
    • Manual Sanity before release - and why it was important!
    • Staged roll-outs via Google’s Play Store and Apple’s App Store
    • Extensive monitoring of the release as users come on board, and comparing the key metrics (ex: consumer engagement) with prior releases
    • Understanding Consumer Sentiments (Google’s Play Store / Apple’s App Store review comments, Social Media scans, Issues reported to / by Support, etc.)

  • Sravan Kumar

    Sravan Kumar - Bootcamp to understand Appium android internals

    Sravan Kumar
    Sravan Kumar
    Sr. Software Engineer
    schedule 2 years ago
    Sold Out!
    45 Mins

    Appium is a world's most popular open source mobile test automation framework developed using WebDriver protocol and I am fortunate enough to get a chance to contribute to appium-uiautomator2-server and appium-uiautomator2-driver modules.

    The goal of the session is to help the Appium community to have a better understanding of Appium and how it works.

    In this session, I will be discussing Appium architecture specifically towards its Android modules(UiAutomator2 and Espresso), how the communication happens between Appium modules and how to add new handler/API in appium

  • Jonathan Lipps

    Jonathan Lipps / Daniel Graham / Kazuaki Matsuo - Fix a bug become a committer

    480 Mins

    Have you ever wondered how Appium works under the covers? Do you get frustrated with locators not locating, app screens not loading, or test behaving inconsistently from one run to the next? Appium is an attempt to unify thousands of disparate elements across a wide spectrum of challenges into a single, common interface that works seamlessly across all the major mobile and desktop OSs - and yet only a handful of volunteers work to maintain this gigantic effort. If you would like to enhance your own Appium experience while contributing back to the software that has defined so many of our careers, come to this workshop. We'll dissect the different elements of Appium, dive into its internals, learn how it was built and how to make changes to it, and even write a unit test you can contribute on the same day!

  • Sai Krishna

    Sai Krishna / Srinivasan Sekar - Life Cycle of an Appium command

    45 Mins

    Every command you write using Appium Client Library will be converted to JSON Wire Protocol over HTTP which in turn will be passed to the drivers(AndroidDriver, IOSDriver). The respective driver doesn’t just send or execute the command directly on the device. Appium has a Lifecycle of modules which sends the command to the devices.


    The goal of the session is to help the Appium community to have a better understanding of Appium and how it works internally.

    Will discuss on following aspects:

    • What happens under the hood when the session is created, mainly how appium connect to the corresponding device, installing the app, launching the app and the different steps involved in the process.

    • Command life cycle - How client request converted into an HTTP request, how received request passes through appium drivers(ios-driver & AndroidDriver) and the respective driver don’t just send or execute the command directly on the device. Appium has a Lifecycle of modules which sends the command to the devices.

    • How the final response passed to the client through different layers.
  • Aisyah Dzulqaidah

    Aisyah Dzulqaidah - Cloud Based Mobile Application Testing with Appium

    Aisyah Dzulqaidah
    Aisyah Dzulqaidah
    schedule 2 years ago
    Sold Out!
    20 Mins

    Testing our mobile application using real device is not scalable and need high maintenance. As the test is bigger , we need to make our test is parallel to make it faster and efficient. We also need to test it agains many variety of device and OS version. Running testing using device cloud is the solution, so in this session I will show you how to setup and run parallel automation testing using Android device emulator and iOS simulator. In this case, I wil show the demonstration using Genymotion Cloud, Appium, and Robot Framework.

  • Anand Bagmar

    Anand Bagmar - Build your own MAD-LAB - for Mobile Test Automation for CD

    45 Mins
    Case Study

    In this age of a variety of cloud-based-services for virtual Mobile Test Labs, building a real-(mobile)-device lab for Test Automation is NOT a common thing – it is difficult, high maintenance, expensive! Yet, I had to do it!

    Attend this talk to understand the Why, What and How I built my own MAD-LAB (Mobile Automation Devices LAB). The discussion will include –

    • Understanding the context,
    • Why no cloud-based solution worked for me,
    • The automation strategy for Android / iOS / Web platform,
    • The tech-stack (cucumber-jvm / Appium), and,
    • The core implementation to build your own MAD-LAB! (already open-sourced)
    • How this results in Continuous Delivery (CD)