LeakMon :- Part 3 Opening a file in visual studio through automation

This is the 3rd post related to the LeakMon tool. In this post we will see how the DumpViewer( DumpViewer is an application used to analyze the leak dump created by LeakMon ) is able to open a perticular file in Visual Studio and highlight the specified line.

Step 1:- Finding out the version of Visual studio to use
It is 2011 now and we have Visual studio 2010. But there are peoples or companies who still use Visual Studio 6. Things have changed a lot from Visual Studio 6 to Visual Studion 10, even the EXE name has changed. So the first task is to find out which verion of Visual Studio to use.

DumpViewer uses the same Visual Studio version that is reigstered as the default program for documents such as .c, .cpp, .h, .hpp in your machine. For instance, say we have the following line shown in DumpViewer..

func2 t:\naveen\pgms\cpp\2008\mulitthread\mulitthreaddlg.cpp(178)

DumpViewer will extract the file path and line number, from the above line and pass it to the FindExecutable function. This function will return the path of executable that is registered for above file types, which will be usually Visual Studio. If it is not Visual Studio, sorry it will fail!

In the file name of the executable returned by FindExecutable function, we will check there is a string "msdev.exe" or "devenv.exe" in it. If there is "msdev.exe" in the string, code that does the automation of Visual Studio 6 is called or otherwise the code to handle higher version of Visual Studio.

const LPCTSTR VC_6_APP_NAME = _T("msdev.exe");
const LPCTSTR VC_8_APP_NAME = _T("devenv.exe");
if( 0 == csExe.CompareNoCase( VC_6_APP_NAME ))
{
 // launch vc6
 OpenUsingVS<IApplication>( IID_IDispatch, L"MSDEV.Application",csFileName_i, nLineNo );
}
else if( 0 == csExe.CompareNoCase( VC_8_APP_NAME ) )
{
 // launch vs7 or any other higher versions of VS
 OpenUsingVS<_DTE>( EnvDTE::IID__DTE, 
   L"VisualStudio.DTE",csFileName_i, nLineNo );
}

Step 2:- Getting a pointer to already running instance of VisualStudio

Since we are going to open a file through auotmation, we need a pointer of either IApplication or _DTE (IApplication for Visual Studio 6 and _DTE for Visual Studio 7 and higher ) . We can create and instance of above type using the CoCreateInstance by passing the IID and CLSID. But this will create a new instance of VisualStudio each time we try to open a new file. So what we want is to get a pointer to IApplication / _DTE of an already running instance of Visual Studio if any. This can be done by enumerating the Moniker in the ROT (Running Object Table).

template<class T>
bool OpenUsingVS( IID riid, CString csProgID, CString csFileName_i, int nLineNo)
{
    HRESULT hRes; 
    CLSID clsid; 
    CComPtr<IUnknown> punk; 
    CComPtr<T> dte; 
    RETURN_ON_FAIL( ::CLSIDFromProgID( csProgID.operator LPCTSTR(), &clsid) ); 

    // Search through the Running Object Table for an instance of Visual Studio 
    // to use that either has the correct solution already open or does not have  
    // any solution open. 
    CComPtr<IRunningObjectTable> ROT; 
    RETURN_ON_FAIL( GetRunningObjectTable( 0, &ROT ) ); 

    CComPtr<IBindCtx> bindCtx; 
    RETURN_ON_FAIL( CreateBindCtx( 0, &bindCtx ) ); 

    CComPtr<IEnumMoniker> enumMoniker; 
    RETURN_ON_FAIL( ROT->EnumRunning( &enumMoniker ) ); 

    CComPtr<IMoniker> dteMoniker; 
    RETURN_ON_FAIL( CreateClassMoniker( clsid, &dteMoniker ) ); 

    CComPtr<IMoniker> moniker; 
    CComPtr<IMoniker> moniker2; 
    ULONG monikersFetched = 0; 
    while ( enumMoniker->Next( 1, &moniker, &monikersFetched ) == S_OK) 
    { 
        moniker2 =NULL;
        moniker->Reduce( bindCtx, MKRREDUCE_ALL, NULL, &moniker2 );
        IMoniker* pMon = (0 != moniker2)?moniker2:moniker;
        if ( moniker2->IsEqual( dteMoniker ) ) 
        { 
            hRes = ROT->GetObject( moniker, &punk ); 
            if ( hRes == S_OK ) 
            { 
                dte = punk; 
 
                if ( dte ) 
                { 
                    // We have got an instance pointer. Lets try to open
                    // the file using that pointer
                    if( OpenFile( dte,csFileName_i, nLineNo ))
                        return true;
                    dte = 0;
                } 
            } 
            punk = NULL; 
        } 
        moniker = NULL; 
    } 

    if ( !dte ) 
    {                
        // we didnt get any pointer from ROT. So lets go an create and new
        // instance.
        RETURN_ON_FAIL( ::CoCreateInstance( clsid, NULL, CLSCTX_LOCAL_SERVER, riid , (LPVOID*)&punk ) ); 
        dte = punk; 
        if ( !dte ) 
            return false; 
        OnLaunch( dte );
        OpenFile( dte,csFileName_i, nLineNo );
    }
    return true;
}

As you can in the above code, if we failed to get a vaild instance pointer of Visual Studio, we create a new one using CoCreateInstance. In Visual Studio 7 and higher, as soon as we release the com pointer that we got from CoCreateInstance, Visual Studio will terminate. To prevent Visual Studio from exiting like this, we need to call put_UserControl of DTE interface.

Step 3 Opening a File and selecting a line of it
From this part on, the processing is different for Visual Studio 6 and other versions of Visual Studio.

In Visual Studio 6

#define dsWindowStateMaximized 1
bool OpenFile(IApplication* m_spApplication,CString csFileName_i, int nLineNo )
 {
    CComPtr<IDispatch> pDispatch;
    // Make it visible just in case if it is not
    m_spApplication->put_Visible( VARIANT_TRUE );
    // Maximize the window
    m_spApplication->put_WindowState( dsWindowStateMaximized );
    // Get a pointer to the IDocument interface
    m_spApplication->get_Documents( &pDispatch );
    CComQIPtr<IDocuments> pDocs = pDispatch;
    CComVariant type="Auto";
    CComVariant read="False";
    pDispatch = 0;
    // The Open function of IDocument opens the file.
    pDocs->Open( CComBSTR(csFileName_i),type, read, &pDispatch );
    // Now for selecting a line, we need to get pointer to ITextSelection
    // Interface
    CComQIPtr<ITextDocument> pDoc = pDispatch;
    pDispatch = 0;
    pDoc->get_Selection( &pDispatch );
    CComQIPtr <ITextSelection> pTextSelection = pDispatch;
    CComVariant varReserved( FALSE );
    // now just set the cursor to the correct line and call SelectLine of ITextSelection
    pTextSelection->StartOfDocument( varReserved );
    pTextSelection->LineDown( varReserved, CComVariant(nLineNo-1));
    Sleep( 1000 );
    pTextSelection->SelectLine();
    return true;
 }

In Visual Studio 7 and higher

bool OpenFile( _DTE* pDTE,CString csFileName_i, int nLineNo )
{
    HRESULT hRes = 0; 
    CComPtr<ItemOperations> Operations;
    // Get the ItemOperations pointer
    RETURN_ON_FAIL(pDTE->get_ItemOperations( &Operations ));
    CComPtr<Window> Wnd;
    // Open the file using ItemOperations pointer
    RETURN_ON_FAIL(Operations->OpenFile( CComBSTR( csFileName_i),
                   CComBSTR(vsViewKindTextView), &Wnd ))
    
    // It might take some time to open the file.
    // so we will wait in a loop
    CComPtr<Document> pDoc;
    for( int nIdx =0; nIdx < 5; nIdx ++ )
    {
        pDTE->get_ActiveDocument( &pDoc );
        //Sleep( 100 );
        if( pDoc )
        {
            break;
        }
        if( nIdx == 4 )
        {
            return false;
        }
        Sleep( 100 );
    }
    // Active the main window. 
    CComPtr<Window> pMainWnd;
    RETURN_ON_FAIL(pDTE->get_MainWindow( &pMainWnd ))
    pMainWnd->Activate();
    
    // Get the TextSelection pointer
    CComPtr<IDispatch> pDisp = 0;
    RETURN_ON_FAIL(pDoc->get_Selection( &pDisp ));
    CComQIPtr<TextSelection> pSelection = pDisp;
     //GotoLine with VARIANT_TRUE will make the line selected
    pSelection->GotoLine( nLineNo, VARIANT_TRUE);
    return true;
}