Debugging Dynamically Generated Code (Reflection.Emit)

Article
02/03/2005

The CLR supports the ability to generate code at runtime (Reflection.Emit). This is great for the many dynamic languages targeting the runtime (such as Iron Python).

We also support that ability to provide debugging information for that code so that you can debug it when it’s executed. This means that even the dynamic languages written for .Net get free debugging support.

Sample Code
Joel Pobar had a great sample here of using Reflection.Emit to print out “Hello World!”. I’ve extended his example to demonstrate emitting debugging information.

In my example, the code we emit is from the pseudo code in the file ‘Source.txt’ (download from here) :

1| // Test
2| xyz = "hello";
3| Write(xyz);
4| return;

Here’s the C# code to emit that. I’ve highlighted the additions related to debugging information:

 // Sample of emitting debugging information for dynamic modules
// Reflection emit example adopted from https://blogs.msdn.com/joelpob/archive/2004/01/21/61411.aspx
using System;
using System.Reflection;
using System.Reflection.Emit;
using System.Threading;
using System.Diagnostics.SymbolStore;
using System.Diagnostics;

public class EmitHelloWorld{
    static void Main(string[] args)
    {
        // create a dynamic assembly and module 
        AssemblyName assemblyName = new AssemblyName();
        assemblyName.Name = "HelloWorld";
        AssemblyBuilder assemblyBuilder = Thread.GetDomain().DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.RunAndSave);

        // Mark generated code as debuggable. 
        // See https://blogs.msdn.com/rmbyers/archive/2005/06/26/432922.aspx for explanation.        
        Type daType = typeof(DebuggableAttribute);
        ConstructorInfo daCtor = daType.GetConstructor(new Type[] { typeof(DebuggableAttribute.DebuggingModes) });
        CustomAttributeBuilder daBuilder = new CustomAttributeBuilder(daCtor, new object[] { 
            DebuggableAttribute.DebuggingModes.DisableOptimizations | 
            DebuggableAttribute.DebuggingModes.Default });
        assemblyBuilder.SetCustomAttribute(daBuilder);

        ModuleBuilder module = assemblyBuilder.DefineDynamicModule("HelloWorld.exe", true); // <-- pass 'true' to track debug info.

        // Tell Emit about the source file that we want to associate this with. 
        ISymbolDocumentWriter doc = module.DefineDocument(@"Source.txt", Guid.Empty, Guid.Empty, Guid.Empty);

        // create a new type to hold our Main method 
        TypeBuilder typeBuilder = module.DefineType("HelloWorldType", TypeAttributes.Public | TypeAttributes.Class);

        // create the Main(string[] args) method 
        MethodBuilder methodbuilder = typeBuilder.DefineMethod("Main", MethodAttributes.HideBySig | MethodAttributes.Static | MethodAttributes.Public, typeof(void), new Type[] { typeof(string[]) });

        // generate the IL for the Main method 
        ILGenerator ilGenerator = methodbuilder.GetILGenerator();

        // Create a local variable of type 'string', and call it 'xyz'
        LocalBuilder localXYZ = ilGenerator.DeclareLocal(typeof(string));
        localXYZ.SetLocalSymInfo("xyz"); // Provide name for the debugger. 

        // Emit sequence point before the IL instructions. This is start line, start col, end line, end column, 

        // Line 2: xyz = "hello"; 
        ilGenerator.MarkSequencePoint(doc, 2, 1, 2, 100);
        ilGenerator.Emit(OpCodes.Ldstr, "Hello world!");
        ilGenerator.Emit(OpCodes.Stloc, localXYZ);

        // Line 3: Write(xyz); 
        MethodInfo infoWriteLine = typeof(System.Console).GetMethod("WriteLine", new Type[] { typeof(string) });
        ilGenerator.MarkSequencePoint(doc, 3, 1, 3, 100);
        ilGenerator.Emit(OpCodes.Ldloc, localXYZ);
        ilGenerator.EmitCall(OpCodes.Call, infoWriteLine, null);

        // Line 4: return; 
        ilGenerator.MarkSequencePoint(doc, 4, 1, 4, 100);
        ilGenerator.Emit(OpCodes.Ret);

        // bake it 
        Type helloWorldType = typeBuilder.CreateType();

        // This now calls the newly generated method. We can step into this and debug our emitted code!! 
        helloWorldType.GetMethod("Main").Invoke(null, new string[] { null }); // <-- step into this call 
    }
}

The key take away here is that judging by the small amount of highlighting, it’s very easy to make your reflection-emit code debuggable. Reflection-Emit provides nice integration and wrappers for emitting the interesting debugging information. This makes naming local variables, parameters, etc trivial.

The other interesting thing is the call to ILGenerator.MarkSequencePoint. Sequence points associate IL instructions with source files. Let’s dissect the call:

ilGenerator.MarkSequencePoint(doc, 2, 1, 2, 100);

In this case, the ilGenerator object keeps track of the IL offset for us. The ‘doc’ parameter to MarkSequencePoint refers back to the DefineDocument call earlier which associates this method with the file “Source.txt”. The next 4 numbers are the starting line, starting column, ending line, and ending column (all 1-based). You can verify that these match back up to Source.txt. I pick the column range (1,100) to specify the whole line.

The one other thing to note is that we emit the sequence points before the opcodes they correspond to.

See the MSDN reference here for more information about making reflection.emit debuggable.

Proof that it’s debuggable

As proof that it’s debuggable, run the sample in Visual Studio. Place a breakpoint at the last call to Invoke, on this line:

helloWorldType.GetMethod("Main").Invoke(null, new string[] { null }); // <-- step into this call

Make sure Source.txt is loaded in VS. And then step in (via F11). You’ll see the VS debugger stepped into our dynamically generated code. (I've verified this works on VS RC0).

You’ll notice:
- we’re doing source-level debugging of the emitted code.
- our declared local (‘xyz’) shows up in the locals window
- our code shows up on the callstack.

You can also debug it in Mdbg (or any 3^rd-party debugger). In fact, Mdbg’s ildasm extension can even show you the disassembly:

[t#:0] mdbg> print
xyz ="Hello world!"
unnamed_param_0=<null>

[t#:0] mdbg> w
Thread [#:0]
*0. HelloWorldType.Main (Source.txt:3)
1. System.RuntimeMethodHandle.InvokeMethodFast (source line information unavailable)
2. System.Reflection.RuntimeMethodInfo.Invoke (source line information unavailable)
3. System.Reflection.MethodBase.Invoke (source line information unavailable)
4. EmitHelloWorld.Main (program.cs:56)

[t#:0] mdbg> ild
code size: 13
current IL-IP: 6
mapping : MAPPING_EXACT
URL: HelloWorldType.Main
IL_0 : ldstr "Hello world!"
IL_5 : stloc.0
* IL _6 : ldloc.0
IL_7 : call System.Console.WriteLine
IL_C : ret

Update: The Debuggable Attribute
The System.Diagnostics.DebuggableAttribute specifies some jit properties such as if the code is jitted optimized or not. If we want to be able to debug the code, we may want to explicitly use this attribute to disable optimizations. Rick Byers talks more about that here and I've updated the sample with his comments.

One caveat:
V2.0 offers light-weight-codegen (LCG) which is a very fast reflection-emit that lets you just create methods without having to create modules, assemblies, types, or metadata. The methods also get garbage-collected when they are no longer used (which is much nicer than waiting handling an appdomain unload). Unfortunately, we don’t yet support debugging info for LCG. This was primarily a scheduling problem (it was effectively cut). The problem is that the whole managed debugging API is effectively an extension to the metadata API, and so debugging LCG without metadata breaks our model. We’ll find a way to fix this in V3.0. Update (10/25/05) : Here are some tips for debugging LCG. Here is a visualizer to display the IL for a dynamic method.

In the meantime, if you want LCG methods to be debuggable, you could consider having a debug mode that obtains the ILGenerator from traditional emit (as demonstrated above), instead of DynamicMethod.GetILGenerator.

Comments

Anonymous
February 03, 2005
Mike this is great stuff, I look forward to using it!! Hope to see you next week at the Compiler Lab.

- Ernie
Anonymous
February 03, 2005
The comment has been removed
Anonymous
February 03, 2005
Wow, you keep putting out some great posts. :)

I tried this with the latest release of Visual C# Express and found that it hit the breakpoint just as you describe. I now want to go a step further and get Edit & Continue working on the emited code. I just want to get it working with C#, I have no plans to implement my own language!

I tried changing Source.txt to Source.cs and replacing it with valid C# code. I then fixed the MarkSequencePoint calls to make them correlate (roughly) with the new code. I also changed DefineDocument to use SymLanguageType.ILAssembly, SymLanguageVendor.Microsoft, SymDocumentType.Text.

I was hoping this would allow me to use Edit & Continue on my generated code. No such luck though, the sequence points just dissapear when I edit the code. Is there a trick to let Visual Studio know that generated code is C# and a valid target for E&C?

Thanks! Jamie.
Anonymous
February 03, 2005
Ernie / Jamie - Thanks! Glad you find it useful.

Russ (Smidgeonsoft) - It's cool that PEBrowse can handle this. Nice! Reflection emit is not creating a new executable, and so won't show up as a new process in taskman. This is all in a single process.

Jamie - EnC doesn't work w/ Ref-emit. I think that deserves its own post. Stay tuned.
Anonymous
February 04, 2005
Jamie - I just posted about why you can't do EnC w/ emit in more detail: http://blogs.msdn.com/jmstall/archive/2005/02/04/367159.aspx
Anonymous
February 05, 2005
This is SERIOUSLY cool. I can't wait!!
Anonymous
February 06, 2005
Wembly - You don't have to wait. This is all in v1.0!
Anonymous
June 26, 2005
Mike Stall has a great little sample showing how to make your dynamically generated code debuggable.&nbsp;...
Anonymous
July 26, 2005
I mentioned here&nbsp; that if your language compiles to IL, then you get free debugging support with...
Anonymous
July 27, 2005
I mentioned here&nbsp; that if your language compiles to IL, then you get free debugging support with...
Anonymous
July 28, 2005
I mentioned here&nbsp; that if your language compiles to IL, then you get free debugging support with...
Anonymous
September 19, 2005
Question from the mail bag:

I am trying to get some information on what I can do usefully with the...
Anonymous
September 05, 2006
You probably already read this 100 times, but Iron Python 1.0 is released. Check out Jim Hugunin's reflections...
Anonymous
September 11, 2006
ICorDebug notifies a debugger when a managed class is loaded via LoadClass debug event. For dynamic-modules,...
Anonymous
November 14, 2006
I mentioned earlier that you can debug Reflection.Emit code . Unfortunately, Ref.Emit code can't be unloaded
Anonymous
December 08, 2006
I'm looking for feedback about Ref.Emit usage patterns. When using Ref.Emit , how many types do you generally
Anonymous
December 14, 2006
PingBack from http://blogs.owasp.org/diniscruz/2006/12/14/firefox-dump/
Anonymous
January 17, 2007
When you attach to a managed debuggee (via ICorDebug::DebugActiveProcess), ICorDebug generates a set
Anonymous
January 17, 2007
I forgot about UpdateModuleSysmbols when I described the fake debug events sent on attach . This event
Anonymous
March 15, 2007
There are several different APIs for handling Types in .NET. Criteria : For each category I want to call
Anonymous
May 24, 2007
The CLR team recently had a compiler dev Lab on campus in building 20, with a strong focus on dynamic
Anonymous
November 11, 2007
Earlier this week, I spent some quality time porting a moderately complicated build over to MSBUILD 3.5,
Anonymous
February 27, 2008
We need some customer feedback to determine if we fix a regression that was added in VS2008. Any language
Anonymous
May 07, 2008
PingBack from http://codingly.com/2008/05/06/continuer-loptimisation-avec-la-lightweight-code-generation-lcg/
Anonymous
January 21, 2009
PingBack from http://www.keyongtech.com/432492-dynamic-debugging-problem

Share via

Debugging Dynamically Generated Code (Reflection.Emit)

Comments

Additional resources