Inline assembly in 64bit mode of Visual C++
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 9 of 9

Thread: Inline assembly in 64bit mode of Visual C++

  1. #1
    Join Date
    Dec 2010
    Posts
    97

    Inline assembly in 64bit mode of Visual C++

    If inline assembly code is totally invalid in Visual C++, should I just retype the whole thing in another .asm file or something
    I didn't know how the original author can compile this code, maybe he is using another compiler or set?

    Code:
    DWORD OverHead = 0;
    __declspec(naked) void ProxyProlog(void){
        __asm
    	{
            pushfd
            pushad
    
            rdtsc
            shrd    eax, edx, 8
            push    eax
            sub        eax, OverHead
            push    eax
            lea        eax, [esp + 8]
            push    eax
            call    ProxyEntrySafe
            pop        ecx
            rdtsc
            shrd    eax, edx, 8
            sub        eax, ecx
            add        OverHead, eax
    
            popad
            popfd
            retn
        }
    }
    
    
    
    void __stdcall ProxyEntrySafe( SPROXYENTRYSTRUCT *pInfo, DWORD dwEnterTacts ){    
        DWORD id = pInfo->funcid;
    
        KHOOK* pHook = g_pHookMgr->GetHookById( id ); 
        PFN_PREHANDLER PreHandler = (PFN_PREHANDLER)pHook->GetMainHandler();
    
        LPVOID FakeFuncAddr = PreHandler( pInfo, pHook );
    
        //Return address
        pInfo->funcid = (DWORD)FakeFuncAddr;//Change function address from plug in value
    }
     
    
    void __declspec(naked) stub_0(){
        __asm{
            retn    
        }
    }
    
    void __declspec(naked) stub_1(){
        __asm{
            retn    4
        }
    }
    void __declspec(naked) stub_2(){
        __asm{
            retn    8
        }
    }
    void __declspec(naked) stub_3(){
        __asm{
            retn    0Ch
        }
    }
    void __declspec(naked) stub_4(){
        __asm{
            retn    10h
        }
    }
    void __declspec(naked) stub_5(){
        __asm{
            retn    14h
        }
    }
    void __declspec(naked) stub_6(){
        __asm{
            retn    18h
        }
    }
    void __declspec(naked) stub_7(){
        __asm{
            retn    1Ch
        }
    }
    void __declspec(naked) stub_8(){
        __asm{
            retn    20h
        }
    }
    void __declspec(naked) stub_9(){
        __asm{
            retn    24h
        }
    }
    void __declspec(naked) stub_10(){
        __asm{
            retn    28h
        }
    }
    
    void __declspec(naked) stub_11(){
        __asm{
            retn    2Ch
        }
    }
    
    void __declspec(naked) stub_12(){
        __asm{
            retn    30h
        }
    }
    
    void __declspec(naked) stub_13(){
        __asm{
            retn    34h
        }
    }
    
    void __declspec(naked) stub_14(){
        __asm{
            retn    38h
        }
    }

    But how do I use 32bit mode in win64
    Code:
    .data
    
    .code
    
     
    ProxyProlog PROC 
         
        pushfd
        pushad
    
        rdtsc
        shrd    eax, edx, 8
        push    eax
        sub        eax, OverHead
        push    eax
        lea        eax, [esp + 8]
        push    eax
        call    ProxyEntrySafe
        pop        ecx
        rdtsc
        shrd    eax, edx, 8
        sub        eax, ecx
        add        OverHead, eax
    
        popad
        popfd
        retn
    ProxyProlog endp
    
    
    stub_0 PROC 
        retn
    stub_0 endp    
         
    stub_1 PROC 
        retn    4
    stub_1 endp
         
    stub_2 PROC 
        retn    8
    stub_2 endp
         
    stub_3 PROC 
        retn    0Ch
    stub_3 endp
         
    stub_4 PROC 
         retn    10h
    stub_4 endp
        
    stub_5 PROC 
        retn    14h
    stub_5 endp
         
    stub_6 PROC 
        retn    18h
    stub_6 endp
         
    stub_7 PROC 
        retn    1Ch
    stub_7 endp
         
    stub_8 PROC  
        retn    20h
    stub_8 endp
         
    stub_9 PROC 
        retn    24h
    stub_9 endp
         
    stub_10 PROC 
        retn    28h
    stub_10 endp
         
    stub_11 PROC 
        retn    2Ch
    stub_11 endp
         
    stub_12 PROC 
        retn    30h
    stub_12 endp
         
    stub_13 PROC 
        retn    34h
    stub_13 endp
          
    stub_14 PROC 
        retn    38h
    stub_14 endp
         
    end
    Code:
    1>KHookMgrASM.asm(10): error A2008: syntax error : pushfd
    1>KHookMgrASM.asm(11): error A2008: syntax error : pushad
    1>KHookMgrASM.asm(15): error A2070: invalid instruction operands
    1>KHookMgrASM.asm(17): error A2070: invalid instruction operands
    1>KHookMgrASM.asm(19): error A2070: invalid instruction operands
    1>KHookMgrASM.asm(21): error A2070: invalid instruction operands
    ....
    Last edited by luckiejacky; August 25th, 2017 at 09:57 PM.

  2. #2
    Join Date
    Dec 2010
    Posts
    97

    Re: Inline assembly in 64bit mode of Visual C++

    Code:
    extrn ProxyEntrySafe : PROC
    
    void __stdcall ProxyEntrySafe( SPROXYENTRYSTRUCT *pInfo, DWORD dwEnterTacts );
    
    1>KHookMgrASM.obj : error LNK2019: unresolved external symbol ProxyEntrySafe referenced in function _ProxyProlog
    1>Release\intruder.dll : fatal error LNK1120: 1 unresolved externals
    I can't even reach out for the stdcall function anyways, it's cpp, and adding extern "C" to no avail.

  3. #3
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    5,837

    Re: Inline assembly in 64bit mode of Visual C++

    In-line assembly code for the VS 64-bit compiler is not supported. See https://docs.microsoft.com/en-us/cpp...nline-assembly. It is only supported for the 32-bit compiler.
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++17 Compiler: Microsoft VS2017 (15.4.4)

  4. #4
    Join Date
    Dec 2010
    Posts
    97

    Re: Inline assembly in 64bit mode of Visual C++

    Hi,
    Could I ask a related question?
    Because I had to rewrite some parts of the program

    Code:
    HRESULT _IDirect3DDevice9_CreateVertexBuffer(
    		PFN_IDirect3DDevice9_CreateVertexBuffer e, 
    		IDirect3DDevice9 *pDev, 
    		UINT Length, 
    		DWORD Usage, 
    		DWORD FVF, 
    		D3DPOOL Pool,
    		IDirect3DVertexBuffer9 **ppVertexBuffer, 
    		HANDLE *pSharedHandle
    		);
    Do you know why such method can't be cast to a LPVOID?
    I am passing it to a third party injector...., it just requires a LPVOID....
    Is LPVOID 32bit, of course I am on 64bit platform now.

  5. #5
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    5,837

    Re: Inline assembly in 64bit mode of Visual C++

    LPVOID is a pointer to void, so the cast is only for pointers. HRESULT isn't a pointer so can't be cast to LPVOID. You can do

    Code:
    HRESULT cvb = _IDirect3DDevice9_CreateVertexBuffer()
    ...
    third_party((LPVOID)&cvb);
    because &cvb is a pointer (the memory address of cvb).
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++17 Compiler: Microsoft VS2017 (15.4.4)

  6. #6
    Join Date
    Dec 2010
    Posts
    97

    Re: Inline assembly in 64bit mode of Visual C++

    Sorry, 2kaud... I said it wrong... I actually meant to pass the pointer to the method itself, not the HRESULT.
    I want to pass a handler to a hooker, so that it will call my function first then forward to the original method/function....

    Code:
    HRESULT _IDirect3DSwapChain9_Present(
    		PFN_IDirect3DSwapChain9_Present e,
    		IDirect3DSwapChain9 *pSwapChain, 
    		CONST RECT * pSourceRect,
    		CONST RECT * pDestRect,
    		HWND hDestWindowOverride,
    		CONST RGNDATA * pDirtyRegion,
    		DWORD dwFlags
    		);
    
    MH_CreateHookApi(
    			L"d3dx9_36.dll", "SwapChain9_Present", 
    			&_IDirect3DSwapChain9_Present, reinterpret_cast<LPVOID*>(
    			tramps["SwapChain9_Present"]));
    I am talking about the &_IDirect3DSwapChain9_Present

  7. #7
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    5,837

    Re: Inline assembly in 64bit mode of Visual C++

    What is the function definition for MH_CreateHookApi()?

    Note that LPVOID* is a pointer to LPVOID which is a pointer to void so LPVOID* is actually void**. Is this what is required?
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++17 Compiler: Microsoft VS2017 (15.4.4)

  8. #8
    Join Date
    Dec 2010
    Posts
    97

    Re: Inline assembly in 64bit mode of Visual C++

    I post the prototype here:
    Code:
     MH_STATUS WINAPI MH_CreateHookApi(
            LPCWSTR pszModule, LPCSTR pszProcName, LPVOID pDetour, LPVOID *ppOriginal);

  9. #9
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    5,837

    Re: Inline assembly in 64bit mode of Visual C++

    To pass a function as LPVOID, you just cast it as LPVOID. Consider

    Code:
    LPVOID lpfunc = (LPVOID)_IDirect3DSwapChain9_Present;
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++17 Compiler: Microsoft VS2017 (15.4.4)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This a Codeguru.com survey!


On-Demand Webinars (sponsored)