What are system calls? Lets understand with an example

Overview

When a C program wants to read a file using read(), it needs help from the operating system (OS). This is because reading a file involves interacting with hardware (disk), and user-space programs don't have direct access to hardware.

The OS provides system calls as a way for user programs to request such privileged operations.

---

Why Do We Need System Calls?

A user program cannot read a file directly because:

1. Security & Isolation – Allowing user programs direct access to disk or memory could lead to data corruption, security breaches, or even system crashes. The kernel ensures only authorized programs can access certain files.

2. Hardware Abstraction – The OS provides a uniform API (via system calls) to interact with different storage devices without worrying about hardware differences.

3. Memory & Process Protection – User programs run in user mode, meaning they cannot access system resources directly. The kernel enforces memory safety to prevent malicious or accidental access.

4. Resource Management – The OS manages file access, preventing multiple programs from interfering with each other (e.g., locking a file to avoid conflicts).

Interview Question: Name three of your favorite system calls and why?

---

What is a TRAP?

A TRAP is a mechanism used to switch from user mode to kernel mode. It is not a signal or a system call itself, but rather a special CPU instruction that:

• Interrupts normal program execution.

• Transfers control to the operating system.

• Changes CPU mode from user mode to kernel mode.

Different architectures implement TRAP differently:

• x86 (32-bit): Uses int 0x80.

• x86-64 (64-bit): Uses syscall instruction.

• ARM: Uses svc (Supervisor Call).

TRAPs are often called software interrupts because they function similarly to hardware interrupts but are triggered intentionally by the program (eg. glibc).

---

Step-by-Step Breakdown

Execution flow:

1. User Program Prepares the System Call

When your C program executes read(fd, buffer, size), the following happens:

• Stores arguments

  • The program places the file descriptor fd, buffer address, and size in registers (or stack, depending on the OS and architecture).

• Loads the syscall number

  • Each system call has a unique ID. For example, read might have syscall number 0x00.

  • This number is placed in a register (like eax in x86).

• Issues a TRAP instruction

  • This special instruction (like syscall in x86-64) switches execution from user mode to kernel mode.

Where to Find TRAP in glibc?

The system call wrapper functions in glibc use inline assembly to issue TRAP instructions. You can find them in:

• Source Code Location: glibc/sysdeps/unix/sysv/linux/x86_64/syscall.S

• The wrapper function sets up the registers and executes the syscall instruction in x86-64.

Example (Linux x86-64 inline assembly for syscall):

ENTRY (syscall)
	movq %rdi, %rax		/* Syscall number -> rax.  */
	movq %rsi, %rdi		/* shift arg1 - arg5.  */
	movq %rdx, %rsi
	movq %rcx, %rdx
	movq %r8, %r10
	movq %r9, %r8
	movq 8(%rsp),%r9	/* arg6 is on the stack.  */
	syscall			/* Do the system call.  */
	cmpq $-4095, %rax	/* Check %rax for error.  */
	jae SYSCALL_ERROR_LABEL	/* Jump to error handler if error.  */
	ret			/* Return to caller.  */

PSEUDO_END (syscall)

---

2. CPU Switches from User Space to Kernel Space

• The CPU switches to kernel mode by:

  • Setting the mode bit (also called user bit) to 0 (indicating kernel mode).

  • Changing the program counter (PC) to a predefined memory location where the OS's syscall handler is located.

  • Switching to a new stack (kernel stack) because user programs cannot access kernel memory.

  • Updating the Memory Management Unit (MMU) to restrict access to user-space pages.

---

3. Kernel Handles the System Call

Once in kernel mode:

1. Find the syscall handler

   • The CPU looks up the syscall table using the syscall number.

   • If the number corresponds to read(), it jumps to the sys_read() function inside the kernel.

2. Check Arguments & Memory Access

   • The kernel verifies the file descriptor (fd) is valid.

   • It checks if the buffer is in user-space memory (to prevent malicious access to kernel memory).

   • If the buffer is in an invalid memory page, the OS terminates the process.

3. Perform the File Read

   • The kernel communicates with the filesystem to locate the file and read data from disk into the kernel buffer.

   • The kernel copies the data from the kernel buffer to the user buffer (using functions like copy_to_user() in Linux).

---

4. Return to User Space

Once the syscall is complete:

• The return value (number of bytes read or an error code) is placed in a register (eax in x86).

• The CPU restores the previous user-mode state:

  • Sets the mode bit back to 1 (indicating user mode).

  • Restores the program counter to resume execution in the user program.

  • Switches back to the user stack.

• Execution continues after the syscall instruction in the user program.

---

Summary

1. The user program stores syscall arguments and issues a TRAP instruction.

2. The CPU switches to kernel mode, saving user state.

3. The kernel executes the syscall (checking permissions, performing the requested action).

4. The CPU switches back to user mode and returns the result to the program.

---

This is the core mechanism behind system calls in Linux. Let me know if you want more details! 😊