Skip to main content

My Evil Plan to Make Everyone Eat My Dog Food

"There are professional taste testers at the dog food companies that eat dog food all day. If you can afford dry dog food, see if u can afford a box of oatmeal and half a dozen eggs for protein. You are much better off eating human food."
~in response to "Is dog food safe to eat?" via Yahoo! Answers

Imagine you're at a board meeting for a major dog food brand. Now imagine that the CEO begin the meeting by picking up a fork and cracking open a can of grey, slime-doused kibbles and taking a large, oddly enthusiastic bite. According to Wikipedia, this is the origin story for the phrase "eating your own dog food" from which the practice of dogfooding derived its name. While the image above may be literally quite unpalatable, the ideals behind it are sound in this author's humble opinion. Why ask your clients to accept something you yourself haven't vetted outside the team of those who built it?

So how do you empower a large crowd of people to test for you effectively without the bias and expertise of being on your project for a long time? How do you help them to integrate within your bug reporting process without needing to train them on the bug tracking system? In the best case scenario where everyone using your app sees and wants to report bugs, how can you manage that without needing to intercept and triage an endless flood of duplicate reports on low-hanging fruit and common bugs? How can you do this on mobile devices without users needing to jump between their device and a web interface? How can you efficiently message to users what the quality state is for any given view in the app being tested?

I am not the only one thinking of these opportunities. uTest recently purchased the entire Apphance division from Polidea in order to centralize their device-based reporting system. Jira has Jira Mobile Connect. Google have been doing this sort of thing with a Chrome browser plugin for awhile. Microsoft does similar things with their office and OS deployments. Anytime you can enlarge the net of users looking at your app prior to release (or even during post-launch iterations on internal build channels), you stand to make huge gains in coverage with minimal ongoing costs assuming of course that you have a lightweight, easily understood, and usable system connecting users and developers alike to the bug tracking system.

Given that I work at a large consultancy, let's assume my first target is simply everyone within the studio. We're talking about technically-minded, passionate individuals who only want to associate their names with the highest quality of software. As a QAE, this is my crowd, my kind of people. They get me and what I want. They're just lazy and they hired me to test stuff for them. Because I like getting a paycheck, I certainly can't begrudge them that. So how do I put the test builds into their hands and get them to look at stuff and report/review bugs?

I guess these questions can use some guidance from the titular evil plan...

Step 1. Identify and engage the potential audience for daily user-level experience with the app.
Step 2. Identify the distribution requirements and method for the app
Step 3. Include something like JMC in the build
Step 4. ???
Step 5. Test all the things

Step 1. Here we scope and prepare our potential tester pool. Selection criteria include: applicable devices, general interest, organizational mandate, susceptibility to begging, you know, the usual. Most importantly I want to let people know what to expect when it comes to dog food testing in the context of what I'm capable of supporting. This means setting the tone for how to identify issues, report them, check progress, get new builds, and for how long the dog food period will last.

Step 2. Not all mobile platforms are created equal. Android has one set of options for distribution, let's call this set "anything I want to do works because it is Android and Android is awesome." iOS has another set of options we're going to call "basically just TestFlight, oh and don't get your hopes up for a super-broad audience due to the limited number of devices you can have on a developer portal". It is a long name for a set but it helps me remember why I love testing on Android.

Step 3. Here's where the engineering excellence of my organization, the creative power of the open source community, and several individuals who are deserving of all the high fives...   ...haven't quite closed the gap. I'm very interested in Atlassian's JMC as a solution but this is a problem that has yet to have been solved to my liking.

Step 4. This is the magical "build, deploy, engage, train, and track" phase. Not much to say here because really this depends on how the other things play. Basically this is where the dog food gets et.

Step 5. In which all the things are tested, everyone is testing, and life is glorious and grand. Every bug no matter how obscure is chased out of the app with righteous anger and smote upon the smoking ruin of your competition's dreams.

So yeah, that's my evil plan. Sure, there are some kinks to work out but I'm confident the technological issues can be resolved. I'm slightly less confident that my Jedi QA powers are sufficiently advanced to get the whole company on board with this plan just like that. Stay tuned...

Comments

Popular posts from this blog

Jenkins + Devices + AndroidJUnitRunner

New Android build system and test runner, same goose chase using undocumented features and hacks

As I've posted before, I am a big fan of Jenkins. It is extremely flexible, open source, and supported by a staggering array of plugins actively developed by engineers running over 100,000 instances of the server worldwide. With it's distributed node model, you can even build your own device cloud for hosting enterprise-scale automation, economizing hardware investments by sharing resources across multiple projects as well as speeding up automation by parallelizing test runs. I had been using a Jenkins-based system in the past to support instrumentation automation with Robotium quite happily. For the last couple years however, my work hasn't required that as much and I've found myself doing a lot more manual testing and using UiAutomator which didn't require a tight integration between the product codebase and the test code. As a result, I've been slow to adopt the…

UiAutomator and Watchers: Adding Async Robustness to UI Automation

"I'm looking over your shoulder... only because I've got your back." ~ Stephen Colbert
After my recent UiAutomator review a user brought up an important question about the use of UiWatcher. The watchers serve as async guardians of the test flow, making sure the odd dialog window doesn't completely frustrate your tests. Having a tool that automatically watches your back when you're focused on the functional flow of your tests is awesome. 100% pure awesomesauce. Since the API documentation on watchers is scant and the UI Testing tutorial on the Android dev guide doesn't cover their use in depth, I figured I should add a post here that goes over a simple scenario demonstrating how to use this fundamentally important UI automation tool.

In my example code below, I'm using uiautomator to launch the API Demo app (meaning run this against an Emulator built in API level 17 - I used the Galaxy Nexus image included in the latest ADT and platform tools). The te…

Run-As Like the Wind: Getting private app data off non-rooted devices using adb run-as and a debuggable app

"You're some kind of big, fat, smart-bug aren't you?"
~Johnny Rico, Starship Troopers (1997) One of the most important things about writing bugs is making them descriptive but concise. Screenshots, video, debug logging, and hardware snapshots are all fairly standard or available to Android testers these days and can save you enormously on text when communicating what constitutes the bug. Sometimes though, the app gets into a weird state due to some transient data issue where you may not own the API or the complexity with forcing the app data into a certain state is prohibitively high. In those cases it is very handy to directly inspect the data the app has in its own directories.

Getting at this data is trivial on emulators and rooted devices but due to file system permissions, this data is otherwise completely private to the app itself. If you're like me, you tend to test using devices rather than emulators and you probably prefer not to root your devices since t…