In short, this is how a system call works:
- First, the user application sets the arguments for the system call.
- After all parameters are configured, the program executes the "system call" command.
This command throws an exception : an event that forces the processor to jump to a new address and start executing code there.
The instructions at the new address save your state of the user program, determine which system call you want, call the function in the kernel that implements this system call, restore the state of your user program, and return control back to the user program.
Visual explanation of the user application calling the open() system call:

It should be noted that the system call interface (it serves as a reference to the system calls provided by the operating system) calls the intended system call in the kernel of the OS and returns the status of the system call and any return values. The subscriber does not need to know anything about how the system call is made or what it does at run time .
Another example: a C program that calls the library printf() call, a write() system call

For a more detailed explanation, see section 1.5.1 in CH-1 and section 2.3 in CH-2 from the Operating System Concept .
source share