Symptoms
Many Visual Basic application developers are moving to Windows NT as theirdevelopment platform because of its robustness and excellent multitaskingcapabilities. This article shows by example how Visual Basic developers canuse the Windbg debugger to debug a dynamic link library (.DLL file) orcustom control (.VBX file) called from a Visual Basic application runningunder Windows NT.
CodeView for Windows doesn’t run under Windows NT because CodeView uses aVxD (Virtual Device Driver) not supported by Windows NT.
For information on how to use CodeView for Windows to debug Visual BasicCustom controls (.VBX files), please see the following article in theMicrosoft Knowledge Base:
75612?(http://support.microsoft.com/kb/75612/EN-US/): How to Use CodeView for Windows (CVW.EXE) with Visual Basic
WARNING: Using Windbg as described in this article is not supported byMicrosoft, so you should save all source files before using this procedure.Windbg should exit without hanging the system. However, you should notmodify your Visual Basic source while running under Windbg unless youalways make sure you have saved any changes before running the code. Thisprocedure has been tested on Windows NT version 3.5. Preliminary testing onWindows NT version 3.51 shows problems when switching to Visual Basic whilerunning under Windbg, so you should not use this procedure with Windows NTversion 3.51.
Resolution
This article follows the steps and contains much of the material in 75612?(http://support.microsoft.com/kb/75612/EN-US/),but it has been modified to describe how to use the Windbg debugger todebug a dynamic link library (.DLL file) or custom control (.VBX file)called from Visual Basic running under Windows NT.
Windbg is one of the debuggers supplied with the Win32 SDK. It is a Windowsdebugger but retains many of the commands of the NTSD debugger that is alsosupplied with the Win32 SDK. More information may be found for both Windbgand NTSD in the Win32 Tools Users Guide.
NOTE: You can build custom controls using the Control Development Kit (CDK)for Visual Basic. The Visual Basic CDK, formerly shipped separately as anadd-on product from Microsoft, is now shipped as part of the ProfessionalEdition of Microsoft Visual Basic for Windows.
Information on installing Windbg can be found in the Help files suppliedwith the Win32 SDK. There are several supporting .DLL files that arerequired to use Windbg, so it is not sufficient to copy just the WINDBG.EXEfile from the SDK.
How to Invoke WindbgMake sure your system is backed up and can be recovered if it should hang.This is highly unlikely because Windows NT has been designed to preventthis kind of problem.
You can invoke Windbg from the Windows Program Manager in any of thefollowing ways:
From the Windows Program Manager File menu, choose New, andspecify WINDBG.EXE as a Program Item with proper arguments. You canthen double-click the WINDBG icon to run WINDBG.EXE.From the Windows Program Manager File menu, choose Run, and enterWINDBG.EXE and its command line arguments.Invoke WINDBG with no arguments from a command prompt, and follow thesteps below to load the various files and modules required for thedebug process.
Step-by-Step ExampleThis example demonstrates how to invoke CIRC3.VBX, which comes with theMicrosoft Visual Basic Control Development Kit (CDK) or the Visual Basicprofessional edition:
Run WINDBG.EXE from the Program Manager or command prompt:
[path]WINDBG.EXE [path]VB.EXE
NOTE: You can just specify an .EXE program that was written in theVisual Basic environment instead of specifying the VB.EXE environmentitself. If you do this, skip steps 7, 8, and 9 below.
The CIRC3.VBX is not loaded from the command line along with Windbg asit is with CodeView. It will automatically be loaded by Windbg when itis added to the Visual Basic development environment. Note that the .VBXfile must be built so that it includes symbol information.If you invoked WINDBG.EXE with no command line arguments, load VB.EXEor the Visual Basic application by choosing Open from the Program menu.Choose New to bring up the Open dialog box.From the Open dialog box, locate VB.EXE or the application you want torun, and open it. Click OK when you return to the Open dialog box.
NOTE: If you are debugging a Visual Basic application rather thanloading VB.EXE, you must load the CIRC3.VBX explicitly to be able toset breakpoints. From the Options menu, choose User DLLs. Click theBrowse button and locate the CIRC3.VBX. Click OK to add it to the listof modules. Click OK again to exit the dialog box.From the File menu, choose Open to load the CIRC3.VBX source code.Locate the directory with the CIRC3.C file. Select and open this file.An MDI window will open in Windbg; it contains the source code forthe .VBX file.From the Options menu, choose Debug, and select the Separate WOW/VDMcheck box in the Debugger Options dialog box. Click OK.Press the F5 key or choose Go from the Run menu. Visual Basic starts up.You should see various system-level .DLL files loaded. This informationappears in the Command window in Windbg.From the File menu in Visual Basic, choose Add File. In the Files box,select the CIRC3.VBX file. The CIRC3 tool appears in the toolbox. Again,you should see various .DLL files loaded. These are system files thatsupport the loading of the .VBX file. You will also see the CIRC3.VBXfile loaded.Select the custom control from the toolbox, and add it to your form.You may now set a breakpoint in the CIRC3.C source code. Locate theWM_LBUTTONDBLCLK message, and set a breakpoint on the first IFstatement.
If the line turns red, this indicates that the breakpoint has been setsuccessfully. If it is purple, the breakpoint is not correctly set, soit is never triggered. This can occur if the .VBX file does not includesymbol information.Press the F5 key from within Visual Basic to run your program.Double-click the circle. When your breakpoint is encountered, focusis set to the debugger, and execution stops at your breakpoint. You cannow step through your program, set watchpoints, and so on.Press the F5 key to return to the Visual Basic program and stop thedebugging.NOTE: When you exit Visual Basic, you receive an “Assertion Failed” messagefrom within Windbg. Click OK and Windbg exits. You may get another dialogbox depending on how you have your system configured for exceptionhandling. Do not click Cancel to debug unless you have the kernel debuggeron your system, otherwise your system will appear to be hung.