Skip to main content

OpenSTF + Jenkins = Scalable In-House Device Sharing

I first learned about OpenSTF from a tweet by Square's Jake Wharton. It's frickin' awesome. I believe I've posted about this before but seriously go check it out if you're an Android developer or tester who has a pool of shared test devices to work with. It's FOSS and boss. Go check it out.

At the time of Jake's tweet, there was no integration between the awesome little project and my cluster of devices configured to host test automation in Jenkins. Because of the open source natures of both OpenSTF and Jenkins, and the robust plugin system on Jenkins, it seemed inevitable to me that a plugin would be written eventually to marry the two. I even speculated that I might write one if I got the chance.

Luckily for everyone that wasn't going to be necessary.

It's actually really straightforward to set up and easy to use but here are a couple of issues with work-arounds you might want to consider:

  • Issue -The API key from OpenSTF for the plugin's global config is associated with the username on the OpenSTF service that created it.
    • Solution - Create your API key on OpenSTF for the Jenkins Plugin using a service account with the name "Jenkins" so it is obvious who is using the device when a job configured with the plugin checks one out on the OpenSTF service.
  • Issue - The OpenSTF Plugin uses the Android Emulator Plugin to manage device IO which means you can't run all your tests across all devices in a given pool in parallel with a normal job config.
    • Solution - The suggested solution by the plugin maintainer is to use a matrix job but I find the Jenkins implementation of a matrix job to be a suboptimal experience. Instead, I prefer to configure my jobs in the following way:
      • restrict it to the node hosting your OpenSTF service
      • use the OpenSTF plugin to configure the job to use a single, specific device and name it in a semantically unique way that identifies the device as it's desired target (i.e. "testMyApp_Nexus5X_SDK24")
      • use the "$ANDROID_SERIAL" environment variable in all shell script calls to adb (see my previous posts on how to set up Jenkins and devices to be scalable for examples of this).
      • Granted, if you're running a job across dozens of devices, a Matrix Job might be a better config but in our case, less than 10 devices means single jobs work better visually and for discoverability
I may present on this at a future Seattle Android Developers meetup depending on timing and opportunity but if there's enough call for it here, I can update this post with a more elaborate description including screenshots and step-by-step instructions to fully illustrate my approach.

Comments

Popular posts from this blog

Jenkins + Devices + AndroidJUnitRunner

New Android build system and test runner, same goose chase using undocumented features and hacks

As I've posted before, I am a big fan of Jenkins. It is extremely flexible, open source, and supported by a staggering array of plugins actively developed by engineers running over 100,000 instances of the server worldwide. With it's distributed node model, you can even build your own device cloud for hosting enterprise-scale automation, economizing hardware investments by sharing resources across multiple projects as well as speeding up automation by parallelizing test runs. I had been using a Jenkins-based system in the past to support instrumentation automation with Robotium quite happily. For the last couple years however, my work hasn't required that as much and I've found myself doing a lot more manual testing and using UiAutomator which didn't require a tight integration between the product codebase and the test code. As a result, I've been slow to adopt the…

UiAutomator and Watchers: Adding Async Robustness to UI Automation

"I'm looking over your shoulder... only because I've got your back." ~ Stephen Colbert
After my recent UiAutomator review a user brought up an important question about the use of UiWatcher. The watchers serve as async guardians of the test flow, making sure the odd dialog window doesn't completely frustrate your tests. Having a tool that automatically watches your back when you're focused on the functional flow of your tests is awesome. 100% pure awesomesauce. Since the API documentation on watchers is scant and the UI Testing tutorial on the Android dev guide doesn't cover their use in depth, I figured I should add a post here that goes over a simple scenario demonstrating how to use this fundamentally important UI automation tool.

In my example code below, I'm using uiautomator to launch the API Demo app (meaning run this against an Emulator built in API level 17 - I used the Galaxy Nexus image included in the latest ADT and platform tools). The te…

Run-As Like the Wind: Getting private app data off non-rooted devices using adb run-as and a debuggable app

"You're some kind of big, fat, smart-bug aren't you?"
~Johnny Rico, Starship Troopers (1997) One of the most important things about writing bugs is making them descriptive but concise. Screenshots, video, debug logging, and hardware snapshots are all fairly standard or available to Android testers these days and can save you enormously on text when communicating what constitutes the bug. Sometimes though, the app gets into a weird state due to some transient data issue where you may not own the API or the complexity with forcing the app data into a certain state is prohibitively high. In those cases it is very handy to directly inspect the data the app has in its own directories.

Getting at this data is trivial on emulators and rooted devices but due to file system permissions, this data is otherwise completely private to the app itself. If you're like me, you tend to test using devices rather than emulators and you probably prefer not to root your devices since t…