Skip to main content

Why is the Android Monkey so Naughty?

I love monkeys; the way they look like us, the way they act like naughty children, even the way they're misused as analogs for random-event-generators in statistical metaphors. When it comes to the Android platform, we have a very naughty monkey of our own: the UI Exerciser Monkey.

On the one hand it is extremely useful to have an off-the-shelf option for random UI stress testing. On the other hand the monkey is kind of limited in some of its capabilities (e.g. it gets stuck if it hits a login/logout page and the app doesn't use the "isUserAMonkey() method wisely on click listeners) and scary in others (e.g. potentially dialing phone numbers, sending emails, cranking on music, taking unnecessary screenshots, etc). What the monkey DOES provide is a dumb event generator that flails with swipes, touches, and keypresses. What the monkey DOES NOT do is spider your app, carefully looking for dead-ends, traps, and unhandled touch states.

In my typical dealings with the UI Exerciser Monkey, I configure a job on Jenkins to pull a build from an upstream job, uninstall the previous build from a target device, install the new build, and launch the monkey using the $BUILD_NUMBER Jenkins environment variable as the seed. The string for this command looks like:

adb -s $DEVICE_SERIAL shell monkey -p $PACKAGE_NAME -s $BUILD_NUMBER --throttle $MONKEY_THROTTLE -v -v -v $MONKEY_EVENT_COUNT > monkeyresults.txt

As you can see I like to parameterize most of my commands as much as possible so that I can exercise granular control when running jobs manually and enjoy the fact that this makes it easy to boilerplate new jobs based on existing jobs with little updating required. Please check the documentation on the developer site for what these various flags mean if you're curious.

Up until recently I rarely looked too far into the output of the monkey except for logging bugs for any ANRs and crashes. However I noticed something happening that caught my attention. In addition to sometimes starting music playback, the monkey would also occasionally capture screenshots, potentially filling disk space with useless garbage. I had noticed the music playback before but the few times I cared to see what happened I saw the monkey hitting a player widget on the homescreen of a test device so I wrote it off. The screenshots were a different story so I chose to look into it this week and figure out what was happening.

My first step was to look at the output in my monkeyresult.txt for a recent run where I noticed screenshots happening. The log was full of things like:
:Sending Trackball (ACTION_MOVE): 0:(2.0,2.0)
:Sending Trackball (ACTION_MOVE): 0:(2.0,-4.0)
:Sending Key (ACTION_DOWN): 23    // KEYCODE_DPAD_CENTER
:Sending Key (ACTION_UP): 23    // KEYCODE_DPAD_CENTER

I didn't see anything referencing screenshots via the normal hardware button combinations so it got me wondering which keycode was responsible. I looked up the documentation for Android's KeyEvent and how to send it via adb and then dutifully started progressing through one at a time. After getting about 100 events through I realized two things:

  1. Although this was satisfying my curiosity in a trance-inducing, mechanically repetitive way, it was taking forever
  2. This was something that could easily be scripted
So after a little trial and error I came up with the following:

#!/bin/bash

# KeyEvents IDs are ordered in continuous sequence for roughly 0-256
i=0
while [ $i -le 255 ]
do
    adb shell input keyevent $i
    echo "running shell input keyevent $i"
    sleep 1
    (( i++ ))

done

# Beginning with 256, KeyEvent IDs are ordered as 2^X where X ranges from 8 to 22
j=1
event_id=256
while [ $j -le 14 ]
do    
    adb shell input keyevent $event_id
    echo "running shell input keyevent $event_id"
    sleep 1
    event_id=$(($event_id * 2))
    (( j++ ))
done

The "sleep 1" was there so I could ctrl+c the script if I noticed the event of a screenshot. So after going through that I found the 2 KeyEvents that made the monkey so naughty: KeyEvent 120 was the offending trigger on those errant screenshots and KeyEvent 209 was the offending trigger for the music playback. What was interesting though was that the monkey appears to have been scolded by Google for trying to launch music playback in advance according to the logs. For example you might see the following:

:Sending Key (ACTION_DOWN): 209    // KEYCODE_MUSIC
    // Rejecting start of Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.google.android.music/com.android.music.activitymanagement.TopLevelActivity } in package com.google.android.music
:Sending Key (ACTION_UP): 209    // KEYCODE_MUSIC

However despite rejecting that intent, the music player launched and occasionally resumed playback of a previous track. It made me wonder why the same rejection was not applied to the screenshotting. Knowing that I'm probably not the only one annoyed by the random music playback, I chose to look at that first and see if anyone had a solution. I found this partial solution which didn't address the fundamental problem of actually preventing specific keycodes from being used by the monkey. The closest I have as an option is the following mysterious text from the developer site:
--pct-anyevent <percent>Adjust percentage of other types of events. This is a catch-all for all other types of events such as keypresses, other less-used buttons on the device, and so forth.
I don't know about you but "other less-used buttons on the device, and so forth" is not very helpful to me. For now I am stuck with just going through and clearing the screenshots folder in the gallery every so often. The monkey continues being naughty and somewhere, someone at probably thinks it is funny that I have to clean up after their monkey.

UPDATE

After consulting with my friend Joe Rogers, who is a rather formidable android dev, on the topic the investigation deepened. We considered the mysterious --pct-anyevent flag found on line 809 in the source code for the Monkey.

                } else if (opt.equals("--pct-anyevent")) {
                    int i = MonkeySourceRandom.FACTOR_ANYTHING;
                    mFactors[i] = -nextOptionLong("any events percentage");

Looking into MonkeySourceRandom for further direction, we found that really all keycodes were handled into a giant lump. This means you can't filter for individual keypresses. It is an all-or-nothing affair. So here was my minor dilemma:

  • Leave in keypresses
    • At the current observed average rate of 3 screenshots per 5000 monkey events means I'd fill the average 12GB free on most phones in 200 million events at 100KB per screenshot. Right now that would cost me maybe an hour of time per device every 9 years or so given the pace of monkey stress runs where I work.
  • Take out keypresses
    • Given that the tiny and only areas of the app which are paying any attention to keyboard inputs are behind a login flow which as I mentioned at the beginning are a roadblock to the monkey entirely, I can set --pct-anyevent to 0 and not really cost my testing much coverage from the random UI stress tool 
The mystery of why the music intent was being rejected remains however having taken out keypresses I am not getting the music player launched at all anymore anyway.

Comments

Popular posts from this blog

Jenkins + Devices + AndroidJUnitRunner

New Android build system and test runner, same goose chase using undocumented features and hacks

As I've posted before, I am a big fan of Jenkins. It is extremely flexible, open source, and supported by a staggering array of plugins actively developed by engineers running over 100,000 instances of the server worldwide. With it's distributed node model, you can even build your own device cloud for hosting enterprise-scale automation, economizing hardware investments by sharing resources across multiple projects as well as speeding up automation by parallelizing test runs. I had been using a Jenkins-based system in the past to support instrumentation automation with Robotium quite happily. For the last couple years however, my work hasn't required that as much and I've found myself doing a lot more manual testing and using UiAutomator which didn't require a tight integration between the product codebase and the test code. As a result, I've been slow to adopt the…

UiAutomator and Watchers: Adding Async Robustness to UI Automation

"I'm looking over your shoulder... only because I've got your back." ~ Stephen Colbert
After my recent UiAutomator review a user brought up an important question about the use of UiWatcher. The watchers serve as async guardians of the test flow, making sure the odd dialog window doesn't completely frustrate your tests. Having a tool that automatically watches your back when you're focused on the functional flow of your tests is awesome. 100% pure awesomesauce. Since the API documentation on watchers is scant and the UI Testing tutorial on the Android dev guide doesn't cover their use in depth, I figured I should add a post here that goes over a simple scenario demonstrating how to use this fundamentally important UI automation tool.

In my example code below, I'm using uiautomator to launch the API Demo app (meaning run this against an Emulator built in API level 17 - I used the Galaxy Nexus image included in the latest ADT and platform tools). The te…

Run-As Like the Wind: Getting private app data off non-rooted devices using adb run-as and a debuggable app

"You're some kind of big, fat, smart-bug aren't you?"
~Johnny Rico, Starship Troopers (1997) One of the most important things about writing bugs is making them descriptive but concise. Screenshots, video, debug logging, and hardware snapshots are all fairly standard or available to Android testers these days and can save you enormously on text when communicating what constitutes the bug. Sometimes though, the app gets into a weird state due to some transient data issue where you may not own the API or the complexity with forcing the app data into a certain state is prohibitively high. In those cases it is very handy to directly inspect the data the app has in its own directories.

Getting at this data is trivial on emulators and rooted devices but due to file system permissions, this data is otherwise completely private to the app itself. If you're like me, you tend to test using devices rather than emulators and you probably prefer not to root your devices since t…