Skip to main content

Crushing Fragmentation using the Factory Design Pattern with UiAutomator

Android Fragmentation, UI Testing and You

Last year, Eddie Vassallo over at Gigaom.com joined the rest of the well-informed tech bloggers in agreeing that at least as far as developers are concerned, "[it's] 2014, and Android fragmentation is no longer a problem." While the developers have tools and patterns that will allow them to successfully build appealing, rich experiences for any screen size and support devices many years past their prime, just the term "Android fragmentation" can intimidate many people trying to decide what phone or tablet to buy. The developer has the ability to access powerful adaptive interface layout APIs and rely on Google Play Services to update crucial dependencies such as Maps and Push Notifications. That behind-the-scenes flexibility is meaningless to an end user staring at a dizzying array of devices from OEMs hellbent on stamping their unique mark on the user experience. Looking at phones such as the Samsung Galaxy Note Edge alongside the HTC One M8 or the Motorola Moto X can make it hard for the uninitiated observer to believe the same OS is powering each of those to say nothing of car infotainment units, wristwatches, televisions, and magical little HDMI dongles.

You. Yeah you; the one who laughed when I said "dongles". We're all adults here, so please keep it together or leave the blog while the grown-ups talk.

Where was I? Oh yes, fragmentation. Fragmentation therefore doesn't just affect developers. It clearly arose as the result of OEMs desiring to offer the benefit of choice through distinguishing themselves visually and to a lesser extent functionally from one another. But there is another less talked about group impacted by fragmentation; a group not unlike the end user but better informed of the underlying technology yet crazy enough to feign ignorance in order to ensure a quality product.

You might be thinking I'm referring to the mighty yet poorly appreciated Black Box Tester. While their role is often to empathize with the user, as intellectual and observant technicians they are able to apply reason and intuition to rise above variations in the system interfaces to accomplish things like setting the system time or date and turning Airplane Mode on or off. No, it is not the Black Box Tester who has to truly wrestle the digital OctoBear. I'm referring instead to the pitiable but diligent Black Box Automator. There are many tools out there for driving the system UI programmatically but to be honest, I hate every one of them that is not UiAutomator. I've written about UiAutomator before but the key reason for it's centrality in any discussion of Android fragmentation and Black Box Automation comes from two realities:

  1. It uses the Accessibility API to know what is on the screen, what to click on, what to scroll, etc. Think of it like a system meant to enable users WHO CANNOT SEE your carefully designed, beautiful, buttery smooth GUI to still interact with your app through the touch screen. That means your tests are about as well informed what they're doing to your app as a blind person. Sooo good luck.
  2. Because UiAutomator is slow to execute and somewhat poorly informed of what it is interacting with, the best justification to run your UI tests with it versus the new hotness (Espresso 2.0, so hot right now, Espresso 2.0) is that unlike Espresso or any other Junit-based framework compiled within your application's namespace, UiAutomator can touch ANYTHING. A. USER. CAN. Which while amazingly powerful from an automation perspective, carries the heavy price of having to deal with every tiny difference the OEMs choose to make in their flavor of Android.

Remember, simple, readable, maintainable test cases are the goal of any good automation writer. Compared to writing test instructions for an experienced and well-informed Black Box Tester, you must assume you are dealing with a pedantic, hyper-literal child who doesn't care if they do it right when you write test automation. Remember writing exercises where you had to learn how to communicate without making assumptions about whether your audience understands your typical idiomatic style? This is a lot like that except your job is at stake. No pressure.

So let me pause briefly to illustrate what kinds of things constitute system UI fragmentation, particularly in the context of how it might affect a test case. Once we're done looking at pretty pictures, I'll lay out one approach to handling system UI fragmentation in code: using the Factory Design Pattern. For some of you, this could be really useful. For the rest, this could just be another relentless ordeal that makes you feel like you picked a bad week to quit smoking.

Minor Fragmented UI Example: Date and Time Settings Views

Exhibit A: Stock Android Lollipop date and time settings view
Holy whitespace, Batman!

Exhibit B: Same view, this time on a Samsung Galaxy S5
Notice minor text differences, nothing too scary right?
Okay so what's the big deal right? You could handle those minor text differences using a stack of conditionals in a single helper class. And so what if the Nexus 7 running Lollipop doesn't actually support NPT? You can just automate picking a time zone as needed. Why go through the trouble of building factories, interfaces, and all this extra cruft over stuff that would take only a few extra lines of code to handle? Well read on, my friend. Like Mountain Dew Voltage, it's about to get EXTREME!!11ONEONE. Which is to say, the visual differences are going to become very apparent and unpleasant to stomach the implications for your once simple stack of conditionals in a single helper class. Be a dear and uncheck that "Automatic date and/& time" box then tap on "Set date" would you please?

Moderate Fragmented UI Example: DatePicker Dialogs

Exhibit C: DatePicker Dialog in TouchWiz on Samsung Galaxy S5
Okay, looks simple. I like simple. I don't know what up does though.

Exhibit D: DatePicker Dialog on Nexus 7 2012 running Lollipop
Well crap. That looks... ...complicated. Okay, I can probably swipe but how do I change years?
Well Google DID warn you that Lollipop was going to come with a complete design overhaul called "Material Design". The Nexus 7's calendar-looking dialog is much more complex-looking which means there is likely to be very little shared code between any helper methods for it and the previous view where you could probably guess you're really just tapping up and down arrows to cycle through your options. Okay, at least now you start to imagine that there will be some justification for a UI helper factory. Oh and incidentally that first dialog with the 3 columns and up and down buttons? Yeah, that STILL takes hundreds of lines of code to properly handle by itself (about 208). Now the time picker can't be that bad, right? It's just a digital clock. No months or years to hassle with right?

Major Fragmented UI Example: TimePicker Dialogs

Exhibit E: Samsung's TouchWiz TimePicker
Looks pretty safe but why does the dialog title match the content exactly?

Exhibit F: Oh come on, Lollipop!
If you think this is bad, try setting minutes.
Okay, what the CRAP, Google? Sure it is pretty but just looking at it gives me no ideas how it should work. Maybe that spinner thingy can be moved by tapping the numbers but I definitely can't reuse any code here. Are you telling me that the Samsung TouchWiz TimePicker is more automatable?

Exhibit G: Nope
Mommy, why does the text in the screenshot not match the text in the Node Detail?
Ah, Samsung. Always outsmarting yourselves. That's right, where everyone else just shows "50" for the text in that field, you've secretly appended ". Double tap to edit." It turns out they did that in system UI EditText views all over the place. Spoiler alert: those arrow buttons aren't standard issue for AOSP Android 4.4.x and putting that redundant title in the dialog means that you can't programmatically verify that indeed you raised the DatePicker dialog when you tapped on the "Set Date" text. Ha ha ha, Samsung. Here comes Google to show you how a proper engineering company does accessibility right. Right?

Exhibit H: FUUUUuuuuu...
Accessibility affordance: 0. Nil. Zilch. Nada.
Okay so maybe I can just set it to 24hr time and do a little math on the radius to click and send clicks to coordinates blindly and pray to sweet, baby Jesus it works...

Exhibit I: Nope nope nope nope
Nope nope nope nope nope nope nope nope nope nope nope
By now you shouldn't need any more convincing. This is just 2 devices and two variations of the OS in a single area of the system UI. I haven't gotten into WiFi settings, Airplane Mode, the notification drawer, or anything else. This is just to show you the rapid growth of complexity in the interface and accessibility differences that will affect how you write code in your test cases

Writing Clean, Concise Test Cases That Can Run Anywhere

Test automation should function like documentation for how the application should work. That means that within the test methods themselves, you should minimize any code that doesn't serve to enrich or clarify those expectations. It might be tempting to write custom test cases for each device variation and then carefully select which ones to run based on the test target. That gets to be a lot of code to rewrite and maintain for each new device you add to your matrix. There has to be a better way. In order to achieve the goals of simple, readable, and maintainable test cases we need to write a single case for each behavior we want to test not for each device. And that's where the Factory Design Pattern comes into play.

Rather than explain exhaustively how the Factory Design Pattern works in paragraph after paragraph, I'll just supply you with this handy link and summarize in the following way:
  1. Knowing that your test case will only call a single set of methods which need to map to a variety of helpers, you add an interface class defining those shared methods.
  2. Then you add a factory class that has a create() method of the interface type from above
  3. Now you write your helper classes that implement that interface INCLUDING ALL methods implemented by that interface
Your test classes that have tests that need those helper methods will need to instantiate that interface in a member variable then assign it via the factory's create method in your setUp() call. In a traditional Factory Design Pattern example, the method call to create() passes the parameter which selects the desired helper class. In my case, since I'm talking about inherent system UI variations, I keep things simpler by getting the Build.FINGERPRINT string and comparing it to a whitelist of known, tested devices for whom I've already written the appropriate helper classes. The end result is that the test case knows exactly what device it is running on and the framework automatically selects the appropriate code for exercising the same path in the UI. Take THAT Android system UI fragmentation!

Enough Words, Monkey Boy, Show Me Some Sample Code

I'll do even better. I've posted a sample project to Github for a workshop I'm hosting in Seattle. I wanted to demonstrate a test case that needed to exercise the system UI such as setting the time. Because this is a Black Box Automation project, I don't need to own the code for the app under test, I can just download one. I chose Google Keep. It is a fantastic app for jotting down quick notes, making shopping lists, etc. It uses Google Play Services to sync notes across your account, has a fantastic widget, and I use it all the time already. You should check it out.

It took some digging to figure out certain shortcuts like launching the date and time settings activities directly or launching Google Keep directly via shell commands rather than tapping around the UI even more than needed. I'll write up another post sometime that lays out those tricks. For now, let's just focus on the guts of the test case.

The test scenario is simple:
  1. Set the system time to something convenient
  2. Launch the app
  3. Create a note
  4. Set a reminder for that note to some convenient time in the future
  5. Set the system time to that time
  6. Verify that a notification is generated containing the content of the note
To whet your appetite, here are Github Gists for the factory and interface classes. Check out the sample project on Github and try it on your own device by making sure you have Google Keep installed first (consider disabling sync for Google Keep in your Google Account settings first before running any automation against it) importing it into Eclipse, updating the helper and factory classes for your device(s) and extending the project with additional test scenarios. Keep in mind the handy included build/deploy/test shell script assumes you're able to run BASH and have added adb to your PATH environment variable.

Exhibit J: Factory Class:

Exhibit K: Interface Class:

What Was That Bit About 0 Accessibility in The Lollipop TimePicker

Ah, you're very astute. I almost forgot to clarify. Yes, you *COULD* have just looked at the helper classes in the supplied example project but that wouldn't tell you anything about how I figured it out. UIAutomator DOES require the Accessibility API to send touch events through the UI. That is true. Without any accessibility affordance in those dialogs, I could have been really screwed, forced to use click-by-coordinates methods like some kind of animal. However because I didn't have a device running Lollipop yet when I first looked into it, I had to try and figure out what was going on in that interface via an emulator. And that's when I found the magic.

That view has listeners for other kinds of events, not just touch events. Because I am lazy and prefer to use my keyboard to enter text on an emulator instead of tapping around the on-screen keyboard with a mouse, I checked the box for "Hardware keyboard present" when I created that emulator. That view listens for key events too. I only discovered that critical fact when I was tapping around idly on the emulator and out of impotent rage and frustration, started typing the number keys on my keyboard. Suddenly things started happening. Magical things. Things that probably should have been more visibly documented somewhere. Hooray for undocumented features.

Exhibit L: Check this box always
Magic.

Comments

  1. What is your real point here? I don't understand. Are you testing others system level applications? Of course not. If you are trying to set the time for your own tests, you can use the following before your tests.

    adb shell date -s `date +%G%m%d.%H%M%S`

    ReplyDelete
    Replies
    1. Hi Said,
      Sorry I missed your comment before. The point here is merely to demonstrate using UiAutomator to interact with the system UI. There are two ways I do this in this example:
      1) editing the system time through the UI
      2) verifying the appropriate notification is generated on time

      I am not interested in necessarily setting the time BEFORE the test as what I'm really doing is adjusting the time midway through the test in order to trigger the notification. There are plenty of scenarios when testing notifications, alerts, alarms, and other time-based triggers where you'd want to manipulate the system time DURING the test rather than before/after, particularly if you're building hermetic tests rather than chaining them together as dependencies.

      The reason to use Google Keep is that you could easily install it on your device and check out my sample code to update it and learn from my approach. I am far too lazy to write up my own sample app for the sake of a blog post :D I hope you can appreciate the convenience. Plus Google Keep is a great app worth checking out anyway.

      Finally, I'd check whether adb shell date is applicable to non-rooted devices. I tend not to root test devices for my purposes and with a device lab full of them, this approach is designed to allow me to run the same test cases across devices without concern for variances not only in the system UI, but also in the undocumented permissions differences in the filesystem and other tools impacting the usefulness of adb shell commands (we're looking at YOU, Samsung, for breaking run-as).

      Delete
  2. Awesome Post. How did you find out what the MAIN launch activity was for Google Keep?

    ReplyDelete
    Replies
    1. Hi Tim,
      Thanks for the compliment. The approach I used to discover the MAIN launch activity for Google Keep is a bit tricky. You'd need to be comfortable using

      adb shell dumpsys

      In my case I used:

      adb shell dumpsys activity

      You can find out more about these commands from this helpful StackOverflow thread here:
      http://stackoverflow.com/questions/11201659/whats-the-android-adb-shell-dumpsys-tool-and-what-are-its-benefits

      My process was to cold launch Google Keep then run that dumpsys command and read through the resulting output for the main activity for Keep. Sometimes it works really well, other times there is some wrangling to do. Samsung uses a lot of injection so the package names and launch strings are all over the place for their system apps. It takes some guessing to figure out what their version of the OS will honor.

      Delete

Post a Comment

Popular posts from this blog

Jenkins + Devices + AndroidJUnitRunner

New Android build system and test runner, same goose chase using undocumented features and hacks

As I've posted before, I am a big fan of Jenkins. It is extremely flexible, open source, and supported by a staggering array of plugins actively developed by engineers running over 100,000 instances of the server worldwide. With it's distributed node model, you can even build your own device cloud for hosting enterprise-scale automation, economizing hardware investments by sharing resources across multiple projects as well as speeding up automation by parallelizing test runs. I had been using a Jenkins-based system in the past to support instrumentation automation with Robotium quite happily. For the last couple years however, my work hasn't required that as much and I've found myself doing a lot more manual testing and using UiAutomator which didn't require a tight integration between the product codebase and the test code. As a result, I've been slow to adopt the…

UiAutomator and Watchers: Adding Async Robustness to UI Automation

"I'm looking over your shoulder... only because I've got your back." ~ Stephen Colbert
After my recent UiAutomator review a user brought up an important question about the use of UiWatcher. The watchers serve as async guardians of the test flow, making sure the odd dialog window doesn't completely frustrate your tests. Having a tool that automatically watches your back when you're focused on the functional flow of your tests is awesome. 100% pure awesomesauce. Since the API documentation on watchers is scant and the UI Testing tutorial on the Android dev guide doesn't cover their use in depth, I figured I should add a post here that goes over a simple scenario demonstrating how to use this fundamentally important UI automation tool.

In my example code below, I'm using uiautomator to launch the API Demo app (meaning run this against an Emulator built in API level 17 - I used the Galaxy Nexus image included in the latest ADT and platform tools). The te…

Run-As Like the Wind: Getting private app data off non-rooted devices using adb run-as and a debuggable app

"You're some kind of big, fat, smart-bug aren't you?"
~Johnny Rico, Starship Troopers (1997) One of the most important things about writing bugs is making them descriptive but concise. Screenshots, video, debug logging, and hardware snapshots are all fairly standard or available to Android testers these days and can save you enormously on text when communicating what constitutes the bug. Sometimes though, the app gets into a weird state due to some transient data issue where you may not own the API or the complexity with forcing the app data into a certain state is prohibitively high. In those cases it is very handy to directly inspect the data the app has in its own directories.

Getting at this data is trivial on emulators and rooted devices but due to file system permissions, this data is otherwise completely private to the app itself. If you're like me, you tend to test using devices rather than emulators and you probably prefer not to root your devices since t…