Visual Studio 10 -- Your Performance Feedback plus Beta 1
At Microsoft you can't say you're excited about anything you have to say that you're "super excited". I don't know why that's just the way it is. So, I'm happy to say that I'm super excited about the release of Beta 1 and I encourage you all to visit Jason's blog (again) to see screen-shots and get the download info. And more importantly to actually try it out and give us feedback. Your feedback so far has made a big difference and it will continue to do so, if only because it gives me ammunition with which to beat up -- err, I mean, educate -- executives here in DevDiv :)
Right now we're at a stage where it's crucial to lock in the gains we have and make sure everything we do is only making things better. That means detecting any performance regressions builds, day to day, is paramount. Now I have some general advice about that, I've been trying to teach for year that consumption metrics are very important and often a better way to track performance even though they aren't directly what your customers care about -- this old article discussed that a bit with regard to memory use. For regression detection I think it's even more important to focus on consumption rather than elapsed time. I was trying to explain this yesterday and I came up with this pithy quote:
"Trying to control your performance by measuring only elapsed time is kind of like trying to control your weight by measuring how much time you spend eating."
It's -- dare I say it -- super-important to use a variety of metrics and watch your overall consumption because too many extraneous, impossible to control, variables affect elapsed time run-to-run.
Now moving on to some feedback, I first wanted to thank everyone that wrote in response to my last posting and encourage you to keep giving feedback. There were a few common themes that I wanted to discuss in detail.
Add/Remove References Dialog
I've personally made a pitch to make this better and I expect to see it getting attention. At the very least I want the default tab changed to be a faster one. This dialog is universally despised and got more negative comments than any other feature.
Unit Tests, especially batch runs
That team is looking at larger run cases and trying to optimize the amount of data gathered versus the run speed. Hope to see some nice gains there.
General Typing speed (C#, VB, C++)
I've been looking at that scenario personally and with a lot of help from our languages team. There are many factors that affect speed, not the least of which is the language service that provides intellisense but there are other important factors like the raw speed of the new editor, the amount of outlining, etc.. We've made big improvements here since the PDC but we're still not happy with it so we're pushing for even more gains before we ship.
"Cat" commented that "I'd hazard that partial class support was added to C# just because of this problem." I can assure you that that was not the case. Partial classes were added for several reasons but I would say the main one was to allow better separation of designer generated code vs. human generated code.
Silverlight and WPF tooling
Some things are already better because WPF is loaded first but other things have suffered because there were many changes in that tool suite and it's still in need of performance work. I think you'll find Beta1 is better than the PDC but I expect you'll still find some areas wanting. Typing speed in XAML files was something that we were especially wanting to look at.
Help
I really wish we could be showing you more of this in the beta it was so close to being ready but the internal version just wasn't quite there. We've totally overhauled how it works, the core storage is a much simpler standards-based solution, the reader is much simplified, the first load should be vastly faster, installing new help and getting updates will be vastly faster. There's a bit of information on April's blog and in particular you can try out the lo-band version of online help (which I think is a lot closer to what our final look will be). I've been very involved in this effort and I'm especially looking forward to getting the new help technology into more people's hands.
Copy Local
Patrick brought up an opportunity to improve Copy Local based on his experience. I'm hoping we can move that solution to use NTFS hard links when they are available to provide that functionality without moving so many bits. I noticed the same activity in some of our test cases -- the problem is of course not unique to NUnit.
WPF Optimizations
Personally I'm focused on WPF changes that will help Visual Studio -- like for instance improvements in how Toolbars are arranged, and general synchronization between the UI thread and the rendering thread. Some of this work translates to gains that are helpful universally, some not so much. But I am encouraging the WPF team to read this posting and that should help too.
Solution/Project management
Lots of work is happening in this space. Customers sometimes have huge solutions and they can't afford to wait for us to load everything before they are allowed to start working. I'm actively driving improvements in that area. I wish I could promise more but, I'm not especially merciful, I can promise that :)
Specific Bugs
There were several bug reports already in the feedback, I will do my best to get the right people to look at this stuff. A lot of times all I can do is shrug -- it's a big product, I know a lot of it but, even after 20 years, I don't know it all.
I'm sorry if I didn't respond to your particular comment, I did read every word but this posting is already too long.
Comments
Anonymous
May 20, 2009
PingBack from http://asp-net-hosting.simplynetdev.com/visual-studio-10-your-performance-feedback-plus-beta-1/Anonymous
May 20, 2009
Speaking of the help system -- do you know anything about the long-awaited successor to HTML Help 1.x for standalone applications? Will we get individually deployable MSHelp 2 packages, or something else?Anonymous
May 20, 2009
Never mind, I just saw the announcement for MS Help 3. The team says they are not replacing HTML Help just yet.Anonymous
May 20, 2009
That copy local issue is a killer, which I'm sure you're aware of. Compilation time is tied almost solely to disk time, and copy local really drags the build speed down. This is especially true in large solutions that are using more projects than NUnit (with more justification hopefully). In the past I've found putting a RAID on a machine has a massive impact on build time. I'm sure I'm not the only one who's found that. It would be great if there was some way the copy local issue could be addressed or improved. I know you can completely reconfigure your build to have assembly references instead of project references but that brings headaches of its own. NiallAnonymous
May 20, 2009
The comment has been removedAnonymous
May 20, 2009
I see I haven't written anything here since February. I've been very busy on two main fronts - lots ofAnonymous
May 20, 2009
Your analogy makes little sense. Often when people from Microsoft start trying to redefine the meaning of popular phrases or expectations, it means they're in the process of dropping the ball on something, which makes me worried. Elapsed time is exactly how most people measure, and perceive, UI performance. Good luck on all the performance work!Anonymous
May 20, 2009
Are you sure hardlinks are the best way to go here? They are indistinguishable in shell from regular files. Could be very confusing to modify one file only to have that propagate to an indeterminate number of others possibly all over the place. I suppose you can't use symlinks because of their lack of XP support. (vs 2010 runs on XP right?) The shell understands those at least. Ideally you could use some sort of "fast-copy" Copy-On-Write hardlink thing - doesn't exist in windows though :(. Can't you just fix the build process to be able to look directly at the artifacts of other projects? :)Anonymous
May 20, 2009
Oh and ILMerge should be integrated into the IDE and build system :DAnonymous
May 21, 2009
Rico In fact the problem that we have with big sulutions is that all the time is getting cunsumed by the disk. We created an interesed solution with a ram disk, we have a bat that move all our working code to the ram disk and we work from there and do commits from it to avoid lost the changes. Before we go we copy the code back. We do all this because we can configure to generate the intermediate /obj stuff in the ram disk only. There is a way to configure that ? Best Regards marcos [at] filehelpers.comAnonymous
May 21, 2009
Rico I forgot to said that running Visual Studio with the code from the RAM disk Visual Studio is blazing fast for any size of solutions. So if u want I can share my experiencies with that scenario and maybe I can get a bit of help on how to make Visual Studio to use the RAM to generate the intermediate obj and the Release Would be so cool to get an option in the project that said "Use volatil intermediate and release" and that project never gets writed to disk In solutions with 10 projects we only want the output of the last one in the chain, so why write all the intermediate and make the buils pain slow with the Copy Local If u want more info, contact me to marcos [at] filehelpers.com Best Regards MarcosAnonymous
May 21, 2009
The comment has been removedAnonymous
May 21, 2009
Hey, what's that there? When you hit F1 or go to the Help menu and select Visual Studio Product Documentation,Anonymous
May 21, 2009
Rico, So far in my day or so of usage with Visual Studio 2010, there a couple of performance problems. The add new item dialog can often take ages (to the extent that I thought the product had died) to appear, also the performance of typing in the new editor is substantially slower than it was Visual Studio 2008. ...StefanAnonymous
May 22, 2009
On my laptop (2ghz, 2gb ram) scrolling a c# text window is just ridiculously slow. i haven't seen anyone else complain about that so i guess the reason for that is my machine not vs10. vs08 however runs smoothly. one issue that is independent of the pc vs is running on is text rendering. well.. how can i say politely that ... all text is blurry? i immediately noticed that the solution treeview is still unmanaged because it was sharp. what does that say about wpf text rendering? you cannot ship like this. the wpf team has to make some changes in this regard. you cannot ship with blurry text. you cannot ship with blurry text.Anonymous
May 22, 2009
@tobi - the WPF text rendering is apparently going to be fixed for Beta2, not sure when that's due though. The current problem, which is extremely severe in some cases, is disingenuously acknowledged as "Text may appear slightly blurry" in the release notes. 'Slightly' is somewhat of an understatement on this machine, and I'm not one of the anti-anti-aliasing crowd by any means...Anonymous
May 22, 2009
@Will: thanks for the clarification. this makes me sleep easy at night again. but i'm curious: how will it be fixed? if it works for vs then probably all wpf apps can leverage that improvement, right?Anonymous
May 22, 2009
@tobi - WPF4 is getting a completely new text renderer, which can be told to optimise the placement of characters onto native pixels rather than position them at the theoretically perfect sub-pixel position. (Which is what it does at the moment.) So yes, this is a benefit which all WPF apps will be able to take advantage of (they may need to enable it specifically though, I'm not sure about that). Of course, a cynic might think that a major chunk of new functionality like this ought to be in place for something called a 'beta'... But let's be gracious about that and talk instead about the fantastic and long awaited new help viewer. Ah. Well. Never mind - at least the perf has been inproved hugely. Oh, oops! Well, what about MVC then - that's great. Darn, where is it? The new WPF ribbon is good though. Or so I hear. Let's face it, this is the CTP which was expected to stop the PDC demo timing-out at the end of last year. It's the {Heros happening}[Here#%] crowd who thought this was a beta. With any luck, Sinofsky will be along shortly, though...Anonymous
May 22, 2009
I'm a long VS fan, going back to VS6 I use VS2005/VS2008 every day at work. Thats 8-10hrs every working day Today i installed VS2010 and .. man this thing is sloooow. I can't believe Microsoft dares to release this beta? Nobody who workes daily with VS is going to accept this. VS2008 was already slow (when compared to VS2005) but VS2010 is simply UN-USABLE Besides that,
- the text is blurry
- it failed to compile one of my solutions So please fix it and make it as fast as VS2005 Dissapointed...
Anonymous
May 22, 2009
>>If I were you, I'd just go and change the default tab in the source code. It seems unlikely that anyone else in MS will either notice or care. Sssssshhhh Not so loud!Anonymous
May 22, 2009
>>Can't you just fix the build process to be able to look directly at the artifacts of other projects? It's not the build process that's the problem, some solutions really like to have the dependent assemblies be in exactly the right place so that they are discovered/loaded by say an .exe in that directory that doesn't know the assemblies even came from different places. The build process itself doesn't care :)Anonymous
May 22, 2009
Things every devdiv exec now knows:
- anti-aliased text can't be the only option
- the perf of the beta isn't good enough, generally
- the add references dialog is the bane of humanity Feel free to write about those things more but at this point you should do it only for recreational purposes :) Do keep the comments coming, they really help!
- Anonymous
May 22, 2009
Anti-Aliased text is fine... just don't make up your own. Cleartype == Awesome.
Of course at the end of the proccess everything needs to be where it is supposed to be, but there are many things on disk that are touched/created/copied/etc that aren't used ever again.
Anonymous
May 22, 2009
It crashes every 10 minutes. Experience in a nutshellAnonymous
May 22, 2009
people complain about vs betas/releases being unstable - well i've had no such problems until this baby.how about Visual Studio Beta Asteriod Edition for a name?Anonymous
May 22, 2009
who cares if the beta is buggy or crashes? those will obviously be fixed in the release and hopefully already in beta 2.Anonymous
May 23, 2009
"Things every devdiv exec now knows:" Rico, You're one of the good guys, please don't turn to the dark side now! None of these things is a new problem, they've been around for years, and widely complained about, and some of them even have trivial fixes. There is absolutely no evidence of a serious 'exec-level' effort to improve what's already there, despite all the assertions to the contrary. At the point Soma stops writing about 'pillars' and 'value' and sets his sights on making a folk record, I'll believe there's a chance for a change. VS2010 is going to be Vista - let's just hope that VS2012 is Win7. And it's not your fault, I do appreciate that.Anonymous
May 23, 2009
tobi - a crash then again i'm fine with, but i was being kind when i said ten minutes.Its almost unevaluatable.F# stuff's a bit of a let down as well in this release... although the docs have got better i must admit.Anonymous
May 24, 2009
- anti-aliased text can't be the only option Exciting to hear this, I'll take native text over CheatType anyday.
Anonymous
May 24, 2009
The comment has been removedAnonymous
May 24, 2009
Um... native text uses ClearType. People object to the reversion to Apple style "perfection" at the expense of readability that is WPF text rendering v native ClearType. (Anyway. I'm sure the WPF people know that different people like different styles of text rendering, as long as they provide at least OS native parity they can't be blamed ;)Anonymous
May 24, 2009
People who are, like me, monumentally disappointed with B1 should have a look at this video: http://channel9.msdn.com/posts/Charles/Jason-Zander-Perspectives-on-Visual-Studio-2008-and-the-Future-of-VS/ I'm not sure if Jason Zander counts as an 'exec', but he's pretty darn senior wrt VS, and he basically admits that there's a performance crisis which they're now throwing people at (under the guidance of Rico). It's a great shame that there isn't (yet) the corporate courage to put this sort of stuff into the release notes, but it does feel to me that perf is now being addressed by people who may have enough clout to put some resource on it. So personally, I shall suspend my churlish whining until B2... Good luck with your hit-squad, Rico.Anonymous
May 25, 2009
Tried opening a vb file with 37000 Lines of code (I know, don't say anything, it's generated code), the memory consumption skyrockets until the VPC is at it limits (1.3 GB), then VS thinks for a minute or 5 and concludes that an error has happened and that it should close. Opening the same file with VS 2008 is not noticeable in memory consumption.Anonymous
May 25, 2009
The comment has been removedAnonymous
May 25, 2009
Rakesh, seems to me you'd be much better filing this sort of thing as a proper bug through the feedback option.Anonymous
May 26, 2009
The comment has been removedAnonymous
May 26, 2009
I've noticed that changing something in Tools|Options... results in a >10second hang on my computer (font size changes don't seem to do anything either). Not sure what's up with that.Anonymous
May 30, 2009
Everything is slow on my machine (3.66Ghz Core2 + 8GB ram on x64 Win7): Here is the main points. The more the stars, the slower it is:Starting VS2010 (even on subsequent runs) *** Opening a Solution
Opening a file within a solution ** Editing the file
Compiling C# ** Compiling C++ ** Launching the debugger (F5) **** Debugging (Steps and Steps In are painfully slow) C# ***** Historical debugging And I get a lot of random crashes on various actions. I crash at least 10x a day. I hope that the data MS is collecting from me participating in the "Customer Feedback Program" includes performance and crashes with a stack mini dump. This is by far the least stable Beta1 I haver ever tested of VS, and I have been beta testing since VS97. Good luck Rico, you have your work cut out for you.
Anonymous
May 30, 2009
Keep the comments coming, they really help!Anonymous
May 31, 2009
The new code editor is really, really slow with large files. For example, VS2008 works acceptably with this file, wheras VS2010 Beta 1 is simply unusable: http://opentk.svn.sourceforge.net/viewvc/opentk/trunk/Source/OpenTK/Graphics/GL/GLDelegates.cs?revision=1803 Interestingly, the VS2010 editor works much better with a 5.5MB file, compared to the VS2008 editor: http://opentk.svn.sourceforge.net/viewvc/opentk/trunk/Source/OpenTK/Graphics/GL/GL.cs?revision=1806 Both files contain parts of auto-generated OpenGL bindings. While this is not a commont use case, it is good to see improvements in this area. It's a little mystifying why the first file performs worse - bug?Anonymous
June 01, 2009
Hi n, Your concerns about stepping are my concerns as well. If you turn off historical debugging (in Tools->Options) is your stepping performance still poor? If you have specific scenarios that seem too slow, it's a good idea to file a bug here: http://connect.microsoft.com/VisualStudio/content/content.aspx?ContentID=12362 This will allow us to engage more directly and reproduce the issues that you're seeing. Thanks, BradAnonymous
June 01, 2009
The comment has been removedAnonymous
June 03, 2009
Please do file your performance bugs, it really helps us to focus on what customers need the most. Even if we already know consider it like a vote.Anonymous
June 03, 2009
The comment has been removedAnonymous
June 04, 2009
I can reproduce these findings both in VS2008 and VS2010: opening as a text file works fine (and VS2010 looks faster than VS2008 in this case). Performance also picks up after you leave the whole solution open for some time. The language process presumably settles down and you can browse either of these files without issue.Anonymous
June 05, 2009
That test case has surfaced several things I didn't like. It's getting attention :)Anonymous
June 08, 2009
@Rakesh Patel I've gotten other reports through Watson (uploaded crash dumps) of this OutOfMemoryException but to make progress I need a full heap dump which I don't have yet. If you can make one at the point of failure, please let me know at msbuild@microsoft.com. Otherwise, I'll wait for one to get uploaded by another customer through the watson system. Thanks Dan (msbuild team)Anonymous
June 14, 2009
There is a horrible performance bug in IntelliSense in the VS2010 Beta. It can slow to a crawl. The following reproduction works invariably on my machine.
- Open a C# project.
- Wait for the IntelliSense documentation cache to construct.
- In a method body, type "System.Linq.Expressions.Expression.Call(". (popping up IntelliSense for the Call overloads)
- Press and hold the up key
- Watch IntelliSense grow progressively slower while it browses through the overloads. Interestingly, browsing with the down key instead of the up key only does not cause any slowdowns. Unfortunately, once you do the repro, method overload browsing is slowed down for all methods, not just the method that caused the slowdown. Closing the solution seems to reset the problem. Fortunately, not all methods can cause this slowdown. I haven't been able to pin down why some methods would have problems and others not. But at least it doesn't seem to matter that the number of overloads for the affected methods (14 for Expresson.Call) is large: On the one hand, Console.WriteLine has more overloads (19) than Expression.Call, but it will not cause the slowdown. On the other hand, Enumerable.SelectMany has a lot less overloads (4), but will cause the slowdown. I hope you're able to reproduce the bug as easily as I am.