Get x64 masm build procedure arguments

Question

I have a function with a signature:

extern "C" int foo(int a, int b, int c, int d, int e); 

which is actually recorded in the assembly.

With ml (32 bits), using the standard calling convention, you can pretty much write

 .code foo PROC a: DWORD, b: DWORD ,c: DWORD, d: DWORD, e: DWORD mov eax, d mov ebx, e 

and start using these tags to access your arguments

With ml64 (64 bits) fastcall is the only convention available. I have no problem accessing the first arguments stored in registers, but to access them on the stack ( e in this example): I tried

 .code foo PROC a: DWORD, b: DWORD ,c: DWORD, d: DWORD, e: DWORD 

and

 .code foo PROC e: DWORD 

but the value in e is garbage.

I found that if I use the stack address directly, I find its value.

 .code foo PROC e: DWORD mov eax, r9 ; d mov ebx, DWORD PTR[rbp + 48] ; e 

Is there another way?

Answer 1

The documentation explains everything ... In Windows, the first four integer parameters are passed to the registers RCX , RDX , R8 , R9 and floating point to XMM0 , XMM1 , XMM2 , XMM3 , more than four parameters are transferred to the stack over the shadow space. For a Unix-like OS, this is slightly different.

So, your example is correct - mov ebx, DWORD PTR[rbp + 48] ; e mov ebx, DWORD PTR[rbp + 48] ; e

Shadow space = 32 + saved rbp = 40 + 5th parameter = 48

Answer 2

given

 extern "C" int foo(int a, int b, int c, int d, int e); 

I found out that visual studio 2010 does not save the RBP base pointer if

 .code foo PROC 

but keep the base pointer if

 .code foo PROC e: DWORD 

Later versions (vs2015) do not allow the use of the second code.

There is an optional optimization on x64 systems where RBP not used (hard path detected). It says:

The traditional use of% rbp as the frame pointer for the stack frame can be avoided by using% rsp (stack pointer) to index onto the frame stack. This method stores two instructions in the prolog and epilogue and makes one additional general-purpose register (% rbp) available.

Thus, it is possible that either foo PROC e: DWORD does not compile (vs2015) or foo PROC fails because RBP is null.

The correct way to retrieve stack arguments is to use the RSP stack pointer, provided that

 RBP = RSP + 8 * num_saved_reg 

Where num_saved_reg is the number of registers specified in the PROC directive. Therefore, when rbp is not saved (otherwise add 8)

 PROC -> DWORD PTR[rsp + 40] PROC use RDI -> DWORD PTR[rsp + 40 + 8] PROC use RDI RSI RBX -> DWORD PTR[rsp + 40 + 24]