Fixed Ranking

  • Article

Ranking bugs/suggestions by average score*number of votes is wrong and we finally fixed it! I apologize for not following up on this sooner. This fix has been in the works for sometime and unfortunately the bug was not updated for those of you that added it to your tracking list to receive notifications of progress. The solution implemented works as follows, we welcome feedback on this as always…

The top bugs determined by:

  • Rank by avgerage vote value
  • Tie sin average broken by number of votes
  • Exclude fixed issues from the list
  • Include bugs with at least 3 votes

Moving forward we certainly plan to put more focus on top issues such as this and maintain an active feedback loop surrounding hot issues.

As a customer, what would you like to see us do better in terms of discussion of issues via the blog and PFC?

Marie Hagman

Program Manager

Visual Studio

Comments

  • Anonymous
    November 10, 2004
    wohoooo!! Good news at last! :)

    Sam
  • Anonymous
    November 10, 2004
    What in case if reports has the same number of votes and same score ? How do you resolve this tie ?

    I've told this once - and can repeat. I would like you get rid of numbers in votes. Change them to words and allow to express negative vote.
    Additionaly - there is were a standart chart at all MSDN websites that show how people voted.
    http://lab.msdn.microsoft.com/ProductFeedback/viewfeedback.aspx?feedbackid=FDBK17743
  • Anonymous
    November 15, 2004
    In my opinion, this is much worse.

    Now, an item with 58 votes and an average of 4.72 is less amportant than an item with 2 votes for 5.0.
  • Anonymous
    November 15, 2004
    There is a three vote minimum so you are right that it's possible to have a situation where an item with a rating of 5.0 and 3 votes makes the list but an item with 58 votes and 4.72 does not.

    By giving the 5.0 item visibility on the list, if it actually deserves a lower rating then the community will vote on it and keep it where it is or bump it off the list. We're hoping the system will be self regulating. The ideal situation is that the most important/severe bugs show up on the list. It's the validity of the rating that I think is most important not necessary how many people voted on it. That said, the more people vote on an issue, the higher the likelyhood the rating is accurate.

    Can you suggest an alternative method that keeps 5.0 with many votes on the list, gets 4.72 with even more votes on the list, and keeps low rated items with many votes off the list?
  • Anonymous
    November 17, 2004
    A slight modification to the original algorithm would make a big difference:

    Votes * (Average - 1.5)

    The Average - 1.5 part lowers the influence of the average on the rank, while still keeping it as a major factor. The "1.5" can be changed up or down to decrease or increase (respectively) the effect of the average until you're happy with the balance.

    With this algorithm, you'd get a sort like this (assuming only 3 bugs existed in the DB):

    58@4.72
    91@3.3
    3@5

    If you wanted to keep really low-ranked stuff off the list, you can add a cutoff to the listing (like AlgorithmResult must be greater than 12.0 to be listed)
  • Anonymous
    November 17, 2004
    The ranking algorithm we use highly depends on what we want reflected. Internally we use the average vote value to help us determine the priority of an issue. So a 5 with 100 votes is higher priority then a 5 with 3 votes becuase we know that first one has more visibility/impact for customers. Does that make the 5 with 3 votes less important then the 3 with 100 votes? Does a bug that is literaly blocking at minimum 3 customers have a lower priority than a bug with an acceptable workaround that affects at least 100 people?