Hopper for ISV’s – part I
I have been reflecting on the things Javier and I learned from our Hopper talk at MEDC and I wanted to thank everyone that attended our talk. I also wanted to give special kudos to those who took the time to fill out the evaluations – this post is your voice in action. The one theme that rang louder than all others is we did not properly address or set expectations for our ISV customers and how they can benefit from Hopper.
Most readers already know all Windows Mobile OEM devices must pass Hopper testing to receive their logo certification – this includes all inrom-ISV applications. However, applications installed after Logo are not covered by this requirement and can introduce stability risks to the end user.
The answer to this dilemma is to evangelize Hopper testing to all applications destined for the Windows Mobile platform – not just for those shipping on the device.
Why should ISV’s care about Hopper?
Hopper is good - Hopper will find bugs in your application that can affect the stability and usability of your device. Hopper is an input stress tool that randomly sends keystrokes to the device and will put your application through scenarios you likely haven’t though of. Who wouldn’t want to find bugs in the application they are trying sell?
How can ISV’s get Hopper?
If you are an ISV, this tool is provided automatically when you install the Windows Mobile 6 SDK.
How to run ISV-Hopper
Hopper for ISV’s is just like Hopper for OEM’s, the only different being the addition of a “focus app” customized to keep the intended application in the foreground. When your application is in the foreground, Hopper will test only it and not the rest of the system – the “focus app” is needed to bring your application back to foreground if Hopper tries to navigate away. The “focus app” blog and source code is described in The Cat Parade.
The specifics of running Hopper.exe on your device are documented in the readme.exe from the Power Toys directory and will depend on if you are running connected to a debugger (recommended) or not. The easiest way to run Hopper.exe is to simply copy the binary to your device and execute directly.
How can I debug application bugs found with Hopper?
This is really a loaded question and will depend on many factors, including whether or not you are connected to a debugger (recommended). If you have a debugger connected, exceptions and hangs should be easy to diagnose if you have correct symbols. If you are not connected (NOT recommended), then having current symbols and some way of capturing the debug output is a good start.
Luckily this question is the exact intent of the blog you are reading so each entry located on the HoppeRx site will help with this.
How many hours should my application survive?
It is entirely up to you and it depends on the complexity of your application, but 20 hours with a “focused Hopper” is a good goal. Anything longer than that will likely give you decreasing returns as the same code paths are likely being re-executed. But don’t feel limited by just “focused Hopper” – can you add additional stress elements to create interesting user scenarios?
Comments
Anonymous
June 07, 2006
The comment has been removedAnonymous
June 12, 2006
Klein, you bring up a very good point – if I am writing a phone application and I point the focus app on it – won’t the likelihood of calling 911 be increased? The answer is: it depends.
Hopper has several safeguards built into its key selection logic that should reduce the number of times this occurs (and prevent this on SP platforms), but after reviewing the code, they appear to look for specific application names and window names to detect the dialer. This obviously won’t work for ISV’s writing their own.
Instead, our resident MTTF Dude created a very thorough writeup (including samples) to prevent emergency dialing: http://blogs.msdn.com/hopperx/archive/2006/02/01/522434.aspxAnonymous
August 28, 2006
The survive hours is interesting, why 25 hours is required for Smartphone product? Is there any guidelines or reference on calculating the hours to using random input testing method?Anonymous
August 30, 2006
Thanks for this comment, its importance has prompted me to write a full blog entry on the subject. Please check back to the main page in a day or two for the full post. Thanks!Anonymous
May 15, 2007
I have updated the post to include Hopper is available in the WM6 SDK. It is no longer required that you download this tool from Jetstream.Anonymous
July 27, 2007
Is hopper a good tool for fullscreen app? My app is a full screen app, it has its own menus which is basically buttons. And has a button to quit app. Running Hopper, I can see it cannot go out of my app and for about 30min, it will pop up a a window saying something about "MTTF", I softreset the device and get the log file, at the bottom it says "Start menu is dead"Anonymous
July 27, 2007
Hi Echo, This blog can address your concenr. http://blogs.msdn.com/hopperx/archive/2006/02/01/522434.aspxAnonymous
July 27, 2007
Continue with the Hopper test problem I have: When I have a full screen app hopper test, the hopper can only run for about 30 min, and then it stops. I got a dialog saying "MTTF test existing, hit OK to get runtime! Note: You can always restart the test to get runtime" Click OK, I get another dialog saying "Build = 18120 (OS 1616) (Hop Feb 26 2006) Random Seed = 196220. Previous runtime = 24 mins(0 hrs 24 mins). Ended by: Start menu dead, not responding! Boot count (prev): 2 (0) SCORE ERROR: 0 Runtime" I call SHFullScreen to make my app fullscreen, there is no way the start menu will respond. How can I make hopper running longer enough to test my app?Anonymous
November 15, 2007
I am a software tester,where can I download Hopper.exe? Thank youAnonymous
March 04, 2008
Please help with where I can find the hopper.exe fileAnonymous
June 22, 2009
The Hopper.exe is part of the WM6 SDK. Download and install it, and you will find it in the tools folderAnonymous
September 05, 2011
Hello shinde I am searching about testing tool, i got the Hooper so can u tell me how i install on my device and or if make a video so it will very beneficial for me strating from 1steps.. plz Thanks Sourabh