What do the rating values mean?

  • Article

For every bug or suggestion submitted in the PFC, you are required to vote. How do you decide if an issue is a 1 or a 5? Does a 3 rating for a suggestion mean the same thing as a 3 for a bug? Here's the concensus internally on what the ratings should indicate:

Rating

Bugs

 

Rating

Suggestions

1

This is a trivial or cosmetic bug and it doesn’t keep me from using the product or my application.

 

1

Implementing this suggestion would negatively impact the product and my experience. Please do not implement it.

2

This is a bug is a minor annoyance that doesn’t cause serious problems.

 

2

I’m neutral, implementing this feature would not improve the product nor benefit me in anyway but it wouldn’t affect me negatively either.

3

This bug affects functionality but there’s an acceptable workaround.

 

3

This is a good idea, this feature would moderately improve my experience.

4

This bug has a significant impact on using the product or my application.

 

4

Adding this feature would provide a significant positive impact on the product, my productivity, or my application.

5

This bug is blocking me from using the product or preventing my application from working as I need it to. It should absolutely be fixed.

 

5

This is a must-have feature and Microsoft should absolutely implement it soon. Without this feature I am blocked in key use scenarios.

Many customers have suggested that we adjust the rating system to be -1 to 4 as opposed to 1 to 5. I think this is not necessary if we define the each rating value as we've done above (in fact we could use A, B, C, D, E as the rating scale if we all agree on what each value means).

So as a PFC user, is this how do you decide how to vote on an issue. Is it close to what you see above?

Marie Hagman

Program Manager

Visual Studio

Comments

  • Anonymous
    November 17, 2004
    Marie, I've suggested this numerous times - change values on Product Feedback from numbers to text.
    This will solve problem with all users voting 5 for their suggestions.

    I do not understand why did not do this up to date.
    Posting comments to each rating value to blog is nice - but changing them on Product Feedback site will be much better - as not everybody read this blog.

    Additionally - I've posted a suggestion in March 2004 to use time in rating calculations. But up to date - you does not use it. Even for breaking ties. You order suggestions with same rating / number of votes alphabetically (instead of submission time).
    ( http://blogs.msdn.com/productfeedback/archive/2004/08/27/221727.aspx#FeedBack
    http://lab.msdn.microsoft.com/productfeedback/viewfeedback.aspx?feedbackid=FDBK10304
    )

    I'm very unhappy.
  • Anonymous
    November 17, 2004
    The way that I vote coincides very well with your descriptions. I am glad to read this clarification. For several months now I have been unsure how Microsoft would interpret some of my votes, especially for suggestions that I really disagree with -- I've been giving them 1's which seemed a little special-cased.

    However, I agree with AT (Tag?)--I would rather vote by description than a number. Or leave the numbers but put a description immediately after each number just like in this posting. I think this will ensure that people vote with the same criteria and intension in mind.

    Regards.
  • Anonymous
    November 17, 2004
    I agree that we absolutely need the descriptions of what each ranking number means diplayed or linked on the submission form but I still like using the 1 to 5 ranking system becuase it's consistent with the rating system many community sites use. This helps users infer the meaning of the numbers with out having to read the text every time.

    The Ladybug team was planning to add the rating descriptions to the site during the latest update last week but this was postponed.

    Also, AT mentions a problem with users marking their suggestions as 5. I have not seen this to be true. Users seem to be fairly judicious in their ratings and many suggestions are listed as vote value of 1, 2, or 3 even though they only have one vote. Since we don't put issues on the top list unless it has at least 3 votes, this should provide some validation for issues that are not marked appropriately from the beginning.