Inside Deep Thought (Why the UI, Part 6)

  • Artykuł

This is the sixth part in my eight-part series of entries in which I outline some of the reasons we decided to pursue a new user interface for Office 2007.

Microsoft is tracking your every move!

Soon after you install Office 2003 on your computer, a balloon pops up asking if you would like to "Help Make Office Better." If you click on it, you are given the opportunity to enroll in something called the Microsoft Office Customer Experience Improvement Program. If you opt-in, anonymous data about how you use Office are uploaded to Microsoft occasionally in the background.

If you're the curious type, you might have wondered where your data goes. Well, today I'm here to answer the question: it goes into an Excel spreadsheet I have sitting on my desktop.

OK, back up. Back in the olden days of designing software at Microsoft (say, pre-2003), design decisions were mostly supported by guesswork. There's a classic Microsoft interview question (that I've never heard of anyone actually using) "How many gas stations are there in the United States?" Many have criticized that type of question as being feckless; personally, I agree and it's not representative of how I choose to spend my interview time with a candidate. But the rough "estimate an answer and defend it" style required to answer the gas station question was at the heart of how many design decisions used to be made at Microsoft.

Suppose you were designing the adaptive menus in Office 2000 and you wanted to know what features people use the most. Well, you start by asking a "guru" who has worked in the product for a long time. "Everyone uses AutoText a lot," the guru says. The louder the "experts" are, the more their opinions count. Then you move on to the anecdotal evidence: "I was home over Christmas, and I saw my mom using Normal View... that's probably what most beginners use." And mix in advice from the helpful expert: "most people run multi-monitor, I heard that from the guy at Best Buy."

So much of what we did was based on feel, estimation, and guesswork. How much that was true only became clear with the introduction of a technology called SQM (pronounced "skwim") .

SQM, which stands for "Service Quality Monitoring" is our internal name for what because known externally as the Customer Experience Improvement Program. It works like this: Office 2003 users have the opportunity to opt-in to the program. From these people, we collect anonymous, non-traceable data points detailing how the software is used and and on what kind of hardware. (Of course, no personally identifiable data is collected whatsoever.)

As designers, we define data points we're interested in learning about and the software is instrumented to collect that data. All of the incoming data is then aggregated together on a huge server where people like me use it to help drive decisions.


Hard at work in the SQM data center.

What kind of data do we collect? We know everything from the frequency of which commands are used to the number of Outlook mail folders you have. We know which keyboard shortcuts you use. We know how much time you spend in the Calendar, and we know if you customize your toolbars. In short, we collect anything we think might be interesting and useful as long as it doesn't compromise a user's privacy.

How much data have we collected?

  • About 1.3 billion sessions since we shipped Office 2003 (each session contains all the data points over a certain fixed time period.)
  • Over 352 million command bar clicks in Word over the last 90 days.

Reflected in these numbers is that we don't even retain all the data points we receive... particularly, we get so much Word and Outlook data that 70% of it is thrown away.

So, one of the biggest reasons that we decided to do the new user interface for Office 12 is simply that, for the first time, we have the data we need to make intelligent decisions. Anything we would have done in the past would have been based more on guesswork and bias than on reality. Data is just one input to the design process, of course, but there's something extraordinarily empowering about knowing which commands people use often and which they don't. And knowing which commands are used in sequence with which other commands. And which commands are used 7x more with the keyboard than with the mouse. And how big people's screens are... and how much of the time they use Excel maximized... and how many documents they use at once... and which commands literally are never used... and which are used much more frequently by East Asian users... and on and on...

Knowledge is power. And having that knowledge makes this the right time to reinvent the user interface of Office.

Want to guess what is the most-used command in Microsoft Word? The top 5 commands used? Post your guesses in a comment and I'll answer in the next post.

(OK, this is a repost so a lot of you know the answer already. If you want to view the guesses from the first time I posed this question, view the comments with the original post here.)

Comments

  • Anonymous
    April 05, 2006
    I guess the following commands are still at the top...

    Paste
    Save
    Copy
    Undo
    Bold

  • Anonymous
    April 05, 2006
    file.save
    edit.cut
    edit.copy
    edit.paste
    edit.undo

  • Anonymous
    April 05, 2006
    Save
    Paste
    Copy
    Cut
    Bold

  • Anonymous
    April 05, 2006
    Today SQM stands for Software Quality Metrics. The term "Service Quality Monitoring" dates back to the humble beginnings of SQM when it was used to measure the client performance and usability of the MSN Client software.

    As more applications adopted SQM, the name was changed to more accurately describe what SQM did.

  • Anonymous
    April 05, 2006
    Paste
    Save
    Copy
    Undo
    Bold

    I know this because having discovered this blog last week Tuesday, I've spent the last few days faithfully reading every entry only to discovered that if I'd discovered it on the Wednesday I could have limited myself to the most important bits.

    Thanks Jensen.

  • Anonymous
    April 05, 2006
    The comment has been removed

  • Anonymous
    April 05, 2006
    Come on Jensen, you can do better than this.  You really would be better off posting nothing.  I'm quite sure you are very busy........

  • Anonymous
    April 05, 2006
    >>Come on Jensen,
    >>you can do better than this.
    > You really would be better off
    >>posting nothing

    may be he ( or MS ) thought some "evangelization" was necessary at this point ... to help users swallow the pill of all this Office fundamental UI Changes.

  • Anonymous
    April 05, 2006
    PingBack from http://toddmundt.com/blog/2006/04/05/links-for-2006-04-06/

  • Anonymous
    April 05, 2006
    PingBack from http://www.davidtouvet.com/blog/archives/2006/04/06/quand-microsoft-ameliore-lergonomie-on-parle-de-millions/

  • Anonymous
    April 05, 2006
    "What is the average IQ of the population of the United States?"

  • Anonymous
    April 06, 2006
    Jensen,

    Is it possible to tell us what % of Office 2003 users opted into the CEIP?

    I'd also be interested in knowing how representative the sample population is. The new UI is so utterly different from how I use Office, that it really makes me wonder.

    Whereas the most used command in the sampled population seems to be paste, the paste "command" I use most often is a PasteUnformatted macro. Would the CEIP have been able to detect that, or would it have reflected only that I pressed Ctrl+Shift+V, or had executed a macro of some kind?

    I'm also wondering if there is any systematic bias in the data. I.e.: are power users more likely to opt out? I know that people [like me] who are cynical and suspicious are more likely to opt out. But, what about the population from whom the CEIP has been collecting data. Is MS able to draw any inferences based on hardware or other data points collected?

  • Anonymous
    April 06, 2006
    Herb, thanks.  Would love to know what percentage of users use a Paste-Unformatted command.

    It would be super if MS could allow users to change CTRL-V so it pasted unformatted by default.

  • Anonymous
    April 06, 2006
    The comment has been removed

  • Anonymous
    April 07, 2006
    This is the seventh part in my eight-part series of entries in which I outline some of the reasons we...

  • Anonymous
    April 08, 2006
    Add me to the list who want to be able to set  paste unformatted as the default.

  • Anonymous
    April 09, 2006
    Should be "where the data go", not "where the data goes". :-. And the dinosaur advertisements for Office ("this data is multiplying like rabbits") irk me as well; I have to resist the urge to draw green wavy underlines on them.

  • Anonymous
    April 10, 2006
    Keep on resisting, John.  Data is singular, not plural.

  • Anonymous
    April 11, 2006
    This is the eighth part in my eight-part series of entries in which I outline some of the reasons we...

  • Anonymous
    April 12, 2006
    I mentioned in mid-March that we had been putting a lot of thought and effort into improving the Office...

  • Anonymous
    April 16, 2006
    Regular readers may remember how I have complained in the past about the way Word interferes with key...

  • Anonymous
    April 17, 2006
    One of the most discussed aspects of the new Office 2007 UI has been: "Does it take up too much room?"...

  • Anonymous
    May 08, 2006
    Good design!
    [url=http://zzmkiizz.com/aghh/pgri.html]My homepage[/url] | [url=http://evmaitdn.com/pqev/gnnt.html]Cool site[/url]

  • Anonymous
    May 08, 2006
    Well done!
    <a href="http://zzmkiizz.com/aghh/pgri.html">My homepage</a> | <a href="http://dwltmcbn.com/ieel/mqzd.html">Please visit</a>

  • Anonymous
    May 08, 2006
    Good design!
    http://zzmkiizz.com/aghh/pgri.html | http://dtouganr.com/ivft/rqkk.html

  • Anonymous
    May 08, 2006
    Good design!
    http://zzmkiizz.com/aghh/pgri.html | http://dtouganr.com/ivft/rqkk.html

  • Anonymous
    May 08, 2006
    Good design!
    http://zzmkiizz.com/aghh/pgri.html | http://dtouganr.com/ivft/rqkk.html

  • Anonymous
    May 28, 2006
    Thanks for taking the time to blog, makes us remeber that real people work at Micro$oft -- not just BillyG ;)

    ...and I've always wanted to know where my data went.

  • Anonymous
    June 02, 2006
    We are wellocme to it's configuration.

  • Anonymous
    August 22, 2006
    One of the most well-understood and salient principles underlying the ergonomics of graphical user interface...

  • Anonymous
    September 03, 2006
    While walking through Jensen's museum of Office Past I came across several fun posts: Combating the perception

  • Anonymous
    October 09, 2006
    PingBack from http://pschmid.net/blog/2006/10/09/58

  • Anonymous
    October 14, 2006
    PingBack from http://pschmid.net/blog/2006/10/14/66

  • Anonymous
    November 25, 2006
    The Windows Vista SDK is live and we want to get to know you better. For the first time, the Microsoft

  • Anonymous
    March 06, 2007
    Catching up on my reading, today I finally read this Microsoft Watch article on the Office 2007 redesign:

  • Anonymous
    April 02, 2008
    Copy Paste Unformatted Text in Word

  • Anonymous
    August 20, 2008
    PingBack from http://p1uton.ru/2008/08/21/inside-deep-hought/

  • Anonymous
    September 19, 2008
    PingBack from http://alsedi.com/blog/zachem-pridumyvali-novyj-interfejs-microsoft-office/

  • Anonymous
    October 09, 2008
    PingBack from http://fresh.gui.ru/2008/09/19/ms-office-ui-history/

  • Anonymous
    October 27, 2008
    PingBack from http://mstechnews.info/2008/10/the-office-2007-ui-bible/

  • Anonymous
    June 02, 2009
    PingBack from http://woodtvstand.info/story.php?id=48912

  • Anonymous
    June 16, 2009
    PingBack from http://workfromhomecareer.info/story.php?id=13476

  • Anonymous
    June 18, 2009
    PingBack from http://barstoolsite.info/story.php?id=78