Debugging hybrid VB6/C/C# app crashes with WinDbg

I was troubleshooting one of two (2) different app crashes this past week in which the app simply dies without any prior warning.  At first I thought: crappy VB6! But then, after looking in the Event Log, I found out this was not Visual Basic 6.0 Runtime’s fault, but rather it’s the .NET Runtime version 2.0 Fatal Execution Engine that erred fatally. Below is how I went about debugging the crash.

First of all, two sets of tools are needed:

1. Debugging Tools for Windows (WinDbg)
2. Debugging symbol (.pdb) files. These are useful but not essential.   PDB files are generated automatically in Visual Studio .NET, and can be turned on in VB6 IDE via the project properties under the Compile tab.

In using WinDbg to troubleshoot app crashes, I found this blog post Using WinDbg - Hunting Exceptions  by Johan Straarup to be tremendously helpful.

As anyone who’s ever involved in troubleshooting any software bug would know, the hardest part is to reproduce it in a consistent manner, and an app crash is (or should be) the rarest (and most feared by programmers) of bugs. Yet, an app crash is nothing more than an exception that has gone uncaught by the application.

In WinDbg, I used the Attach to a Process menu option to connect to the running app. Then, I issued the gn command (Go [until] Unhandled Exception) command, and let the app run its course.  Eventually, it came a stop with this message:

(4a4.1e4): CLR exception - code e0434f4d (!!! second chance !!!)
eax=0129ecfc ebx=0017d328 ecx=00000000 edx=00000025 esi=0129ed88 edi=e0434f4d
eip=7c812afb esp=0129ecf8 ebp=0129ed4c iopl=0         nv up ei pl nz na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000202
…

I then used “!analyze –v” and subsequently “!pe”, which gave me this:

0:002> !PrintException
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll -
PDB symbol for mscorwks.dll not loaded
Exception object: 019138a0
Exception type: System.IO.FileLoadException
Message: Could not load file or assembly 'Interop.SHDocVw, Version=1.1.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
InnerException: <none>
StackTrace (generated):
    SP       IP       Function
    00000000 00000001 MyAssembly1.MyClass1.Finalize()

So, this crash was the result of a referenced assembly version mismatch in one of the plugin components. That's one easy problem solved.

The other crash is a bit more elusive, which, after two weeks, I'm still in the process of trying to reproduce.

Unable to cast COM object of type ‘X’ to interface type ‘_X’

I added a new property to my COM class and consequently bumped up its minor version by 1, did some testing with it, and then decided that I wanted to run a clean build.  Now my existing NUnit test cases that use this class are failing with the following message:

Unable to cast COM object of type MyLib.MyClass' to interface type MyLib._MyClass'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{…}' failed due to the following error: No such interface supported (Exception from HRESULT: 0x80004002 (E_NOINTERFACE)).,

The interesting thing is that everything compiled without a hitch.  Is this  .NET’s version of BadImplementsRefInCompatLib?

Looking that the COM library’s IDL info via oleview.exe , it turned out that the interop assembly was bound to version 1.3 of the COM library, whereas 1.2 was registered on my machine. I regenerated the COM interop assembly for 1.2, and the problem went away.

VB6 BadImplementsRefInCompatLib and .NET-COM Interop

You've created a VB6 COM library, MyLib1 (1.0).
You then have two usage scenarios for MyLib1:

1. MyLib1 is, in turn, used by another VB6 library, MyLib2


2. MyLib1 is used to generate a COM Interop assembly which is consumed by a .NET assembly called MyDotNetLib, which itself is also a COM interop library.

It so happened that you needed to add some new public methods to MyLib1, which would bring its versioning to 1.1. You were careful to craft the change such that it's backward compatible. Yet, something happened--you don't yet know exactly what it was--and when to try to recompile the above two libraries, you get the following problem scenarios:

1. Recompile MyLib2, you get this error: BadImplementsRefInCompatLib


2. Recompile MyDotNetLib, you get:
   Error 63 The assembly "..MyDotNetLib.dll" could not be converted to a type library. Type library exporter encountered an error while processing 'MyDotNetLib.MyComponent, MyDotNetLib'. Error: Referenced type is defined in managed component, which is imported from a type library that could not be loaded (type: 'MyLib1._IComponent'; component: '...\Interop.MyLib1.dll')
   

What did you do wrong?
As it turns out, the "something" in "something happened" above was an IDE debug session that had previously crashed, and MyLib1.vbp (1.1) remained registered in the system.

Why did that cause the above problems to occur?
The IDE looks for the latest version of the library (same GUID) to bind to.

How to avoid it?
Well, you can't avoid it, any more than you can avoid making VB6 crash.

What to do when it happens?

• Temporarily remove reference to MyLib1 from your MyLib2 project, then attempt to re-add the reference, looking through the list of registered libraries to see how many instances of MyLib1 are registered--there should be only one. If you see a MyLib1.vbp showing in your list, you'll need to go through the registry and manually remove all CLSIDs with InprocServer32="MyLib1.vbp". The automatic registry clean tools, that I've tried, couldn't remove these entries for me.


• Retrace your changes to make sure that you haven't made any modification to public interfaces that could break binary compatibility.


• Use the OLE/COM Object Viewer tool to view Type Library info of MyLib1.dll. Then compare the output with a previous version of the binary to see if any of the interface GUIDs got inadvertently changed.
  

---
Pre-emptive comment:
Yes, I know VB6 (the IDE) has reached its end-of-life, which means MS support for it will be scarce--all the more reason for writing this post.

Hosting .NET controls in VB6

I recently ran into a need to interop a C# .NET user control with a VB6 form.

We all know that VB6 only allows you to reference a user control if it resides in an OCX.

The Interop Forms Toolkit, mentioned here, seems to be only good for VB.NET user controls.

Prior to this, I've been able to get by with dynamically adding the control via Controls.Add {COMClassname}, {ControlName}

But now, I need to be able to manipulate the control in design time.

So, based on the above experience, I created a very basic VB6 user control called DotNetControlWrapper. The tricks with this control are the following:

• There's a public property called DotNetClassName, which is a COM class name of the .NET control.


• It delegates resizing logic to the .NET control.


• It exposes the .NET control via a DotNetObject property, thus allowing the user to sync the control's events if needed.

The bulk of DotNetControlWrapper.ctl looks like this:

Option Explicit

Private object As Object
Private ctlExtender As VBControlExtender
Private  mstrDotNetClass As String 

Public Property Get  DotNetClassName() As String
   DotNetClassName = mstrDotNetClass
End Property 
Public Property Let DotNetClassName(ByVal clsname As String)
   mstrDotNetClass = clsname
   If mstrDotNetClass = "" Then Exit Property
   ' initialize the .NET control
   If Not ctlExtender Is Nothing Then
     Controls.Remove UserControl.Extender.Name + "_dotnet"
   End If
   Set ctlExtender = Controls.Add(mstrDotNetClass, UserControl.Extender.Name + "_dotnet")
   ctlExtender.ZOrder 0
   Set object = ctlExtender.object
End Property

Private Sub UserControl_Resize()
   If object Is Nothing Then Exit Sub
   ctlExtender.Move 0, 0, UserControl.Width, UserControl.Height
   object.BringToFront ' Bring the .NET control to foreground
End Sub

Private Sub UserControl_Show()
   If ctlExtender Is Nothing Then Exit Sub
   ctlExtender.Visible = True
End Sub

Public Property Get DotNetObject() As Object
   Set DotNetObject = object
End Property

Private Sub UserControl_Terminate()
   If ctlExtender Is Nothing Then Exit Sub
   Set ctlExtender = Nothing
   object.Dispose
   Set object = Nothing
End Sub

Private Sub UserControl_WriteProperties(PropBag As PropertyBag)
   PropBag.WriteProperty "DotNetClassName", mstrDotNetClass
   PropBag.WriteProperty "Width", UserControl.Width
   PropBag.WriteProperty "Height", UserControl.Height
End Sub

Private Sub UserControl_ReadProperties(PropBag As PropertyBag)
   On Error Resume Next
   DotNetClassName = PropBag.ReadProperty("DotNetClassName")
   Width = PropBag.ReadProperty("Width")
   Height = PropBag.ReadProperty("Height")
End Sub

VB Interview Question

This is one of the easy-level questions that I give my interview candidates on the written test part of the interview:

Assume that a and b are Integer variables.

i. If a And b Then Call DoSomething()
ii. If (a And b) = b Then Call DoSomething()

Statements (i) and (ii) are equivalent. True or False?

I thought this was a no-brainer, but apparently not. This is often the source of many bugs in VB code.

So how many of you think the answer is True? ;-)

Using the .NET Framework Class Library from Visual Basic 6

I stumbled onto this link while reading another blog. Using .NET framework classes from VB6? Now that's just plain crazy talk!
Or is it?

HOW NOT TO compute the inverse of a colour

I once wrote a simple VB utility that acted like a window decorator, allowing the user to redact any window on the desktop by highlighting on a rectangular region within it. The highlighting code tracked the user mouse movements and drew the selected region as a hollow rectangle with black border.

My highlighting code had a problem: what if the window you're highlighting had a black background? That would render the highlighting rectangle virtually invisible, wouldn't it?

The code really should have used, for its border colour, the inverse colour of whatever the window's background colour is. OK...How the heck do we compute the inverse colour in VB?

Well, some quick googling yielded this link, which seemed to be somewhat useful:

Take any point in the [RGB] cube, then draw a line from this point to the centroid. If you then extend this line through the centroid the same length as between the original point and the centroid, you will have found the inverse colour to that defined by the original point. Inverting an RGB image involves a kind of turning inside out of the colour values.

uhh, yeah...

So, anyway, I did some more googling, and it turned out that there was a simpler way to do this in VB, thanks to this link.

The trick is in using a PictureBox and the vbInvert constant. The PictureBox will serve as the drawing canvas, onto which a snapshot of the actual window is placed for selection. The following simplified pseudocode illustrates the rest:

' Initializes PictureBox drawing settings
Private Sub Form_Load()
   ' Set drawmode to draw using the invert of the background colour
   Picture1.DrawMode = vbInvert
   ' Set pen width
   Picture1.DrawWidth = 2
   ... 
End Sub
...
...
Private Sub Picture1_MouseDown(Button As Integer, _ 
                               Shift As Integer, _
                               X As Single, _
                               Y As Single)
   If Button <> LEFT Then Exit Sub
   LeftDowned = True
   ' Store [X,Y] as [Left,Top]
   ...
   ...
End Sub
...
...
Private Sub Picture1_MouseMove(Button As Integer, _
                               Shift As Integer, _
                               X As Single, _
                               Y As Single)
   If Not LeftDowned Then Exit Sub

   ' Clear previous drawn rectangle
   ...   
   ' Get current drag region [Top, Left,Bottom,Right]
   ...
   ' Draw new rectangle
   Picture1.Line (Left, Top)-(Right, Bottom), vbBlack, B
End Sub
...
...
Private Sub Picture1_MouseUp(Button As Integer, _ 
                             Shift As Integer, _
                             X As Single, _
                             Y As Single)
   ' Re-set [Left,Top,Right,Bottom] = [0,0,0,0]
   ...
   LeftDowned = False
   ...
End Sub

The moral of the story is this: There is a simple solution out there somewhere, waiting to be discovered. So keep googling, ya lazy bastard!