Saturday, January 21, 2012

Unusual coverage in VB.NET

Recently a user posted on StackOverflow on why he was seeing unusual coverage results in VB.NET with MSTEST and Visual Studio. The the question already had answers that helped the questioner but I decided to delve a little deeper and find out why the solution proposed worked.

The issue was that in his code sample the End Try was not being shown as covered even though he had exercised the Try and the Catch parts of his code.

First I broke his sample down into something simpler and I have highlighted the offending line.

 07  Function Method() As String  
 08    Try  
 09      Return ""  
 10    Catch ex As Exception  
 11      Return ""  
 12    End Try
 13  End Function  

In debug we can extract the following sequence points (I am, obviously, using OpenCover for this.)

<SequencePoints>  
  <SequencePoint offset="0" ordinal="0" uspid="261" vc="0" ec="32" el="7" sc="5" sl="7"/>  
  <SequencePoint offset="1" ordinal="1" uspid="262" vc="0" ec="12" el="8" sc="9" sl="8"/>  
  <SequencePoint offset="2" ordinal="2" uspid="263" vc="0" ec="22" el="9" sc="13" sl="9"/>  
  <SequencePoint offset="19" ordinal="3" uspid="264" vc="0" ec="30" el="10" sc="9" sl="10"/>  
  <SequencePoint offset="20" ordinal="4" uspid="265" vc="0" ec="22" el="11" sc="13" sl="11"/>  
  <SequencePoint offset="40" ordinal="5" uspid="266" vc="0" ec="16" el="12" sc="9" sl="12"/>  
  <SequencePoint offset="41" ordinal="6" uspid="267" vc="0" ec="17" el="13" sc="5" sl="13"/>  
</SequencePoints>  
(where sl = start line, el = end line, sc = start column, ec = end column and offset = IL offset in decimal)

However these only make sense when you look at the IL...

.method public static
    string Method () cil managed
{
    // Method begins at RVA 0x272c
    // Code size 43 (0x2b)
    .maxstack 2
    .locals init (
        [0] string Method,
        [1] class [mscorlib]System.Exception ex
    )

    IL_0000: nop
    IL_0001: nop
    .try
    {
        IL_0002: ldstr ""
        IL_0007: stloc.0
        IL_0008: leave.s IL_0029

        IL_000a: leave.s IL_0028
    } // end .try
    catch [mscorlib]System.Exception
    {
        IL_000c: dup
        IL_000d: call void [Microsoft.VisualBasic]Microsoft.VisualBasic.CompilerServices.ProjectData::SetProjectError(class [mscorlib]System.Exception)
        IL_0012: stloc.1
        IL_0013: nop
        IL_0014: ldstr ""
        IL_0019: stloc.0
        IL_001a: call void [Microsoft.VisualBasic]Microsoft.VisualBasic.CompilerServices.ProjectData::ClearProjectError()
        IL_001f: leave.s IL_0029

        IL_0021: call void [Microsoft.VisualBasic]Microsoft.VisualBasic.CompilerServices.ProjectData::ClearProjectError()
        IL_0026: leave.s IL_0028
    } // end handler

    IL_0028: nop

    IL_0029: ldloc.0
    IL_002a: ret
} // end of method Module1::Method

Now as you can see the End Try line that is causing concern would only be marked as hit (assuming they are using similar instrumentation to OpenCover) if the code reached IL instruction at offset 40 (IL_0028) however when one looks at the IL produced it is not possible to see how you would ever reach that instruction due to the odd IL produced (leave.s is a small jump like instruction that is used to exit try/catch/finally blocks) and if you follow the code you see that you will always reach a leave.s that jumps to IL_0029 first.

In release the IL changes to something more like what I was expecting beforehand and it has no unusual extra IL...

.method public static
    string Method () cil managed
{
    // Method begins at RVA 0x2274
    // Code size 30 (0x1e)
    .maxstack 2
    .locals init (
        [0] string Method,
        [1] class [mscorlib]System.Exception ex
    )

    .try
    {
        IL_0000: ldstr ""
        IL_0005: stloc.0
        IL_0006: leave.s IL_001c
    } // end .try
    catch [mscorlib]System.Exception
    {
        IL_0008: dup
        IL_0009: call void [Microsoft.VisualBasic]Microsoft.VisualBasic.CompilerServices.ProjectData::SetProjectError(class [mscorlib]System.Exception)
        IL_000e: stloc.1
        IL_000f: ldstr ""
        IL_0014: stloc.0
        IL_0015: call void [Microsoft.VisualBasic]Microsoft.VisualBasic.CompilerServices.ProjectData::ClearProjectError()
        IL_001a: leave.s IL_001c
    } // end handler

    IL_001c: ldloc.0
    IL_001d: ret
} // end of method Module1::Method

but so do the sequence points...

<SequencePoints>
  <SequencePoint offset="0" ordinal="0" uspid="33" vc="0" ec="22" el="9" sc="13" sl="9"/>
  <SequencePoint offset="15" ordinal="1" uspid="34" vc="0" ec="22" el="11" sc="13" sl="11"/>
  <SequencePoint offset="28" ordinal="2" uspid="35" vc="0" ec="17" el="13" sc="5" sl="13"/>
</SequencePoints>

So now one will never see your try/catch lines marked covered, so this is not helpful.

So lets try changing your code as suggested and go back to debug (because that is where you will be running coverage from usually.)

15   Function Method2() As String
16        Dim x As String
17        Try
18            x = ""
19        Catch ex As Exception
20            x = ""
21        End Try
22        Return x
23    End Function

Again we look at the sequence points...

<SequencePoints>
  <SequencePoint offset="0" ordinal="0" uspid="268" vc="0" ec="33" el="15" sc="5" sl="15"/>
  <SequencePoint offset="1" ordinal="1" uspid="269" vc="0" ec="12" el="17" sc="9" sl="17"/>
  <SequencePoint offset="2" ordinal="2" uspid="270" vc="0" ec="19" el="18" sc="13" sl="18"/>
  <SequencePoint offset="17" ordinal="3" uspid="271" vc="0" ec="30" el="19" sc="9" sl="19"/>
  <SequencePoint offset="18" ordinal="4" uspid="272" vc="0" ec="19" el="20" sc="13" sl="20"/>
  <SequencePoint offset="31" ordinal="5" uspid="273" vc="0" ec="16" el="21" sc="9" sl="21"/>
  <SequencePoint offset="32" ordinal="6" uspid="274" vc="0" ec="17" el="22" sc="9" sl="22"/>
  <SequencePoint offset="36" ordinal="7" uspid="275" vc="0" ec="17" el="23" sc="5" sl="23"/>
</SequencePoints>

and the IL...

.method public static
    string Method2 () cil managed
{
    // Method begins at RVA 0x282c
    // Code size 38 (0x26)
    .maxstack 2
    .locals init (
        [0] string Method2,
        [1] string x,
        [2] class [mscorlib]System.Exception ex
    )

    IL_0000: nop
    IL_0001: nop
    .try
    {
        IL_0002: ldstr ""
        IL_0007: stloc.1
        IL_0008: leave.s IL_001f
    } // end .try
    catch [mscorlib]System.Exception
    {
        IL_000a: dup
        IL_000b: call void [Microsoft.VisualBasic]Microsoft.VisualBasic.CompilerServices.ProjectData::SetProjectError(class [mscorlib]System.Exception)
        IL_0010: stloc.2
        IL_0011: nop
        IL_0012: ldstr ""
        IL_0017: stloc.1
        IL_0018: call void [Microsoft.VisualBasic]Microsoft.VisualBasic.CompilerServices.ProjectData::ClearProjectError()
        IL_001d: leave.s IL_001f
    } // end handler

    IL_001f: nop
    IL_0020: ldloc.1
    IL_0021: stloc.0
    IL_0022: br.s IL_0024

    IL_0024: ldloc.0
    IL_0025: ret
} // end of method Module1::Method2

So for the End Try to be covered we need line 21 to be hit and that is offset 31 (IL_001F) and as it can be seen both leave.s instructions jump to that point so now that line will be marked as covered.