Passing gcdasm() Arguments
Two int arguments are passed through registers, not onto the stack. The first and second arguments are passed in the lower half of rdi and rsi (i.e. edi and esi ) respectively. So, by the sign extending edi and esi to rax and rbx respectively, you load the passed arguments into these registers:
movsx rax, edi ;load rax with x movsx rbx, esi ;load rbx with y
However, note that rbx not a zero register, so the called user needs to save it before modifying it, and then restore it before leaving the gcdasm function.
You can simply replace rbx with rcx (which is not a user-registered register) anywhere in your code. You don't need rbp at all, so you can delete all instructions where rbp appears.
Other problems
There is also a problem with the program logic:
mov rax, rdx ;reminder rdx as next dividend
Instead, the divisor ( rcx ) should become a dividend ( rax ), and the remainder ( rdx ) should become a divisor ( rcx ), that is:
mov rax, rcx mov rcx, rdx
When dividing signed values, you should use the idiv , not the div .
Improvement
There are also some reasons regarding performance and code size for using test rdx, rdx instead of cmp rdx, 0 for comparing rdx with zero .
Given all of the above:
;gcdasm.nasm bits 64 section .text global gcdasm gcdasm: movsx rax, edi ;load rax with x movsx rcx, esi ;load rcx with y top: cmp rax, rcx ;x(rax) has to be larger than y(rcx) je exit ;if x=y then exit and return value y jb xchange ;if x<y then swap x and y modulo: cqo ;sign extend RDX:RAX idiv rcx ;rdx:rax/rcx (signed values) test rdx, rdx ;check whether remainder is zero je exit ;if reminder is 0 then exit return y mov rax, rcx ;divisor becomes dividend mov rcx, rdx ;remainder becomes divisor jmp modulo ;loop xchange: xchg rax, rcx ;swap x and y jmp modulo exit: mov rax, rcx ;Return c program with the divisor y ret
source share