Appium or No Appium? Tales of Test Automation Failure

Automation is everywhere and mobile app automation using Appium is a boon for many. This talk is about the multiple times' people have burned their fingers trying to automate without proper planning beforehand. Instead of diving deep into writing automated checks, it is a good idea to crosscheck against few guidelines. A realistic plan would help everyone and save cost, increase value and reduce the risk. A simple mistake multiple testers do is to pick a tool first and then figure out what to automate. The reason to automate could be driven by management, development teams or peer pressure. When we think of automation, it actually amplifies, enables, extends and maximizes our intended tasks.

If we choose the right tasks to automate, we would save time, money and effort. At the same time, give the tool garbage and we get more garbage quicker at a higher cost. This is simple to understand, yet a majority of teams fail to understand the basic philosophy behind what and when to automate.

Let us look at common tales of test automation failure and how teams can recover from them. We will discuss a few tough questions related to adding automation to test plan and how to derive value from automation efforts.


Outline/Structure of the Talk

  • Benefits of test automation
  • Common failures in mobile app automation
  • Checklist for good foundations in test automation
  • Key skills complementary to automation skills

Learning Outcome

  • Know when, what and how to automate
  • Use the power of automation across the life cycle
  • Understand the foundations of good automation

Target Audience

Testers getting started in automation, Team leaders who manage automation teams.

Prerequisites for Attendees

There is no prerequisite for the attendees. This is a session for the beginner in test automation.

schedule Submitted 2 years ago

  • Justin Ison
    Justin Ison
    Sr. Software Engineer
    schedule 2 years ago
    Sold Out!
    45 Mins

    In today’s agile world the time to market is becoming increasingly shorter. There is a constant desire to release ASAP to keep ahead of the competition and to please users with updated/new features. Because of this, we have less time to fully do manual and exploratory testing of our apps. Especially, when you consider all the combinations of OS's, Locales, Accessibility, Orientations & Resolutions apps support. Running anywhere from 1 to 100's of Appium crawler bots (covering all of those combinations) at once we can discover more issues quickly and efficiently without having to write a line of code.

    UI Automation also has its limitations as it only tests for expected results. Crawler bots test the unexpected, by collecting metadata such as logs, app strings, screenshots, memory and reporting back it’s finding for review so we can test all these combinations quickly and more efficiently. In this talk, I will go over the current challenges we face in today's development world, why we need more tools to help us keep pace, and cover how you can build your own Appium crawler.

    I've open sourced this tool and is available here for everyone to use:

  • 45 Mins
    Case Study

    Experimentation and quick feedback is the key to success of any product, while of course ensuring a good quality product with new and better features is being shipped out at a decent / regular frequency to the users.

    In this session, we will discuss how to enable experimentation, get quick feedback and reduce risk for the product by using a case study of a media / entertainment domain product, used by millions of users across 10+ countries - i.e. - we will discuss Testing Strategy and the Release process an Android & iOS Native app - that will help enable CI & CD.

    To understand these techniques, we will quickly recap the challenges and quirks of testing Native Apps and how that is different than Web / Mobile Web Apps.

    The majority of the discussion will focus on different techniques / practices related to Testing & Releases that can be established to achieve our goals, some of which are listed below:

    • Functional Automation approach - identify and automate user scenarios, across supported regions
    • Testing approach - what to test, when to test, how to test!
    • Manual Sanity before release - and why it was important!
    • Staged roll-outs via Google’s Play Store and Apple’s App Store
    • Extensive monitoring of the release as users come on board, and comparing the key metrics (ex: consumer engagement) with prior releases
    • Understanding Consumer Sentiments (Google’s Play Store / Apple’s App Store review comments, Social Media scans, Issues reported to / by Support, etc.)

  • Diego Molina

    Diego Molina - Building a Mobile Testing Infrastructure: What I Learned

    Diego Molina
    Diego Molina
    Sr. Software Engineer
    Sauce Labs
    schedule 2 years ago
    Sold Out!
    45 Mins
    Case Study

    Mobile is everywhere, and everyone wants a piece of it. All companies want to have mobile apps for their products, lots developers want to code mobile apps, and lots testers want to build awesome tests and frameworks for mobile apps…

    But, what about the infrastructure? Does someone want to spend endless days building a test infrastructure for Android and iOS apps? Not so many people have the time and patience to put all the pieces together and make them work like a Swiss watch. It just seems that many people forget that tests also need a solid infrastructure to be reliable.

    In this talk, I will narrate the journey I went through while doing the job that not so many people want, building a mobile testing infrastructure. The attendants will see all the steps taken in the process of planning the system architecture, how all the components were selected, and how they were put together.

    More importantly, I will share the lessons learned and the open source components that are being used in the final solution, and also guidelines on how to scope the needed resources when building your own mobile testing infrastructure.

  • Manjunath Halakoppa

    Manjunath Halakoppa - A rollercoaster ride with Appium

    45 Mins

    We love Appium and we realized it a bit late. We did not start with Appium and the ride was a bit bumpy to our dislike. Once we picked Appium, it was not a straightforward journey either. New challenges popped up and we started loving how we solved them with Appium. This talk highlights our journey from MonkeyTalk to using Appium on Windows for Android apps and slowly moving on to Appium on Mac. Few interesting stories along the way like the migration challenge or the lack of community support or the last minute surprises with the autogenerated scripts will be shared with the audience. With our realistic and relatable stories, join us in this talk to know how we conquered our fear of Appium and finally use it to its fullest potential bringing smiles on our clients’ faces.

    The rollercoaster ride also includes our discovery of Appium’s power and how we moved away from emulators once for all. If you have just started your journey with Appium, this talk is a good preview of some of the challenges you might face and get inspired by how we solved them for everyone’s satisfaction.