How Does Interrupt Handling Work in XV6?

Today, I want to thoroughly study the interrupt handling flow in XV6. Previously, I worked on related labs in MIT 6.S081, but after some time, I’ve forgotten most of it. During an interview yesterday, the interviewer pressed me hard on interrupts, and I couldn’t recall how the “interrupt vector table” is implemented in XV6. I only remembered that it’s different from the system call table, so I’m organizing my notes and reviewing it today. All code references are from the official repository.

Review—What Is an Interrupt?

Let’s clarify the concept of interrupts. Here, I’ll provide a brief version; you can search for more details as needed. An interrupt is a mechanism by which a computer, upon encountering an urgent situation or specific event during execution, suspends the current program and switches to execute a handler for that event. After handling, control returns to the original program.

Common interrupt sources include:

• Hardware interrupts: External devices (keyboard, mouse, printer), timers, hardware faults (power failure, memory errors)
• Software interrupts: Program-specific instructions or execution exceptions, such as system calls, division by zero, illegal instructions, etc.

The Interrupt Flow in XV6

uservec

XV6 is an operating system based on the RISC-V architecture, so it’s essential to understand how RISC-V handles interrupts. RISC-V provides the stvec and scause registers. When an interrupt occurs, the processor stores the interrupt number in scause and jumps to the address in stvec to execute the corresponding handler.

Interrupt and Exception Codes

Exceptions are typically triggered by errors or special instructions during program execution; the highest bit of scause is 0. Interrupts are triggered by external devices or timers; the highest bit of scause is 1.

• Common Exception Codes

• Common Interrupt Codes

How the Jump to uservec Occurs

RISC-V does not have a physical “interrupt vector table”; instead, on an interrupt, it jumps directly to the address in stvec and stores the cause in scause. The code to set stvec is as follows:

  // kernel/trap.c:99-101 usertrapret()
  // send syscalls, interrupts, and exceptions to uservec in trampoline.S
  uint64 trampoline_uservec = TRAMPOLINE + (uservec - trampoline);
  w_stvec(trampoline_uservec);

Thus, when an interrupt occurs, XV6 jumps directly to the uservec assembly code in kernel/trampoline.S:

...
# Declare uservec as a global symbol for reference in other files
.globl uservec
# uservec label, the actual entry point for trap handling
uservec:
    #
    # trap.c sets the stvec register to point here,
    # so when a trap occurs in user space, execution starts here.
    # At this point, we're in supervisor mode but using the user page table.
    #

    # Save the value of the user-space a0 register to sscratch,
    # freeing up a0 to access the trap frame (TRAPFRAME).
    csrw sscratch, a0

    # Each process has its own p->trapframe memory region,
    # but in each process's user page table, it's mapped to the same virtual address (TRAPFRAME).
    # Load the address of TRAPFRAME into a0
    li a0, TRAPFRAME

    # Save user-space registers to the trap frame (TRAPFRAME)
    sd ra, 40(a0)
    sd sp, 48(a0)
    sd gp, 56(a0)
    sd tp, 64(a0)
    sd t0, 72(a0)
    sd t1, 80(a0)
    sd t2, 88(a0)
    sd s0, 96(a0)
    sd s1, 104(a0)
    sd a1, 120(a0)
    sd a2, 128(a0)
    sd a3, 136(a0)
    sd a4, 144(a0)
    sd a5, 152(a0)
    sd a6, 160(a0)
    sd a7, 168(a0)
    sd s2, 176(a0)
    sd s3, 184(a0)
    sd s4, 192(a0)
    sd s5, 200(a0)
    sd s6, 208(a0)
    sd s7, 216(a0)
    sd s8, 224(a0)
    sd s9, 232(a0)
    sd s10, 240(a0)
    sd s11, 248(a0)
    sd t3, 256(a0)
    sd t4, 264(a0)
    sd t5, 272(a0)
    sd t6, 280(a0)

    # Save the value of user-space a0 to p->trapframe->a0
    # Read the previously saved user a0 from sscratch into t0
    csrr t0, sscratch
    # Store t0 (user a0) at offset 112 in the trap frame
    sd t0, 112(a0)

    # Load the kernel stack pointer from p->trapframe->kernel_sp into sp
    ld sp, 8(a0)

    # Save the current hardware thread ID (hartid) in tp, from p->trapframe->kernel_hartid
    ld tp, 32(a0)

    # Load the address of usertrap() from p->trapframe->kernel_trap into t0
    ld t0, 16(a0)

    # Load the kernel page table address from p->trapframe->kernel_satp into t1
    ld t1, 0(a0)

    # Ensure previous memory operations complete, using the user page table
    sfence.vma zero, zero

    # Install the kernel page table by writing its address to satp
    csrw satp, t1

    # Flush outdated user page table entries from the TLB
    sfence.vma zero, zero

    # Jump to usertrap() to begin trap handling; this function does not return
    jr t0

In summary, this assembly code saves the user context when a trap occurs in user space, switches to the kernel page table, and jumps to the usertrap() function for trap handling. Key points:

1. The sscratch register temporarily saves the value of a0 for later restoration.
2. The system call number is stored at proc->trapframe->a7.
3. TLB must be flushed with sfence.vma zero, zero when switching page tables.
4. The satp register holds the page table base address.

usertrap

The code is quite clear; let’s look at it directly:

void
usertrap(void)
{
  int which_dev = 0;

  if((r_sstatus() & SSTATUS_SPP) != 0)
    panic("usertrap: not from user mode");

  // send interrupts and exceptions to kerneltrap(),
  // since we're now in the kernel.
  w_stvec((uint64)kernelvec);

  struct proc *p = myproc();

  // save user program counter.
  p->trapframe->epc = r_sepc();

  if(r_scause() == 8){
    // system call

    if(killed(p))
      exit(-1);

    // sepc points to the ecall instruction,
    // but we want to return to the next instruction.
    p->trapframe->epc += 4;

    // an interrupt will change sepc, scause, and sstatus,
    // so enable only now that we're done with those registers.
    intr_on();

    syscall();
  } else if((which_dev = devintr()) != 0){
    // ok
  } else {
    printf("usertrap(): unexpected scause 0x%lx pid=%d\n", r_scause(), p->pid);
    printf("            sepc=0x%lx stval=0x%lx\n", r_sepc(), r_stval());
    setkilled(p);
  }

  if(killed(p))
    exit(-1);

  // give up the CPU if this is a timer interrupt.
  if(which_dev == 2)
    yield();

  usertrapret();
}

This function performs the following tasks:

1. Interrupts can also occur in kernel mode, so it sets the stvec register to kerneltrap.
2. Saves the user’s program counter (pc).
3. If scause is 8, it handles a system call (enabling interrupts).
4. If scause is another value, it calls devintr() (which checks scause for PLIC, timer, or other interrupts).
5. If it’s a timer interrupt, it calls yield.
6. Calls usertrapret to return.

In short, usertrap checks the value of scause: if it’s 8, it handles a system call; otherwise, it calls the appropriate handler. Notably, to add page fault handling, you only need to check if r_scause() == 13.

A system call sets scause to 8 (user-initiated interrupt), entering the syscall() function. This function uses a system call table, with the function selected by the value in the a7 register and arguments in a0–a5. This part is straightforward; after all interrupt handlers finish, the program calls usertrapret to return from kernel to user mode.

usertrapret

//
// return to user space
//
void
usertrapret(void)
{
  struct proc *p = myproc();

  // we're about to switch the destination of traps from
  // kerneltrap() to usertrap(), so turn off interrupts until
  // we're back in user space, where usertrap() is correct.
  intr_off();

  // send syscalls, interrupts, and exceptions to uservec in trampoline.S
  uint64 trampoline_uservec = TRAMPOLINE + (uservec - trampoline);
  w_stvec(trampoline_uservec);

  // set up trapframe values that uservec will need when
  // the process next traps into the kernel.
  p->trapframe->kernel_satp = r_satp();         // kernel page table
  p->trapframe->kernel_sp = p->kstack + PGSIZE; // process's kernel stack
  p->trapframe->kernel_trap = (uint64)usertrap;
  p->trapframe->kernel_hartid = r_tp();         // hartid for cpuid()

  // set up the registers that trampoline.S's sret will use
  // to get to user space.
  
  // set S Previous Privilege mode to User.
  unsigned long x = r_sstatus();
  x &= ~SSTATUS_SPP; // clear SPP to 0 for user mode
  x |= SSTATUS_SPIE; // enable interrupts in user mode
  w_sstatus(x);

  // set S Exception Program Counter to the saved user pc.
  w_sepc(p->trapframe->epc);

  // tell trampoline.S the user page table to switch to.
  uint64 satp = MAKE_SATP(p->pagetable);

  // jump to userret in trampoline.S at the top of memory, which 
  // switches to the user page table, restores user registers,
  // and switches to user mode with sret.
  uint64 trampoline_userret = TRAMPOLINE + (userret - trampoline);
  ((void (*)(uint64))trampoline_userret)(satp);
}

This function mainly does the following:

1. Disables interrupts: Prevents interference during the transition, ensuring atomicity.
2. Sets the trap vector: Points the trap handler to uservec in trampoline.S for subsequent user-mode trap handling.
3. Configures the trap frame:
   • Kernel page table: Stores the kernel page table address (kernel_satp).
   • Kernel stack: Points to the top of the process’s kernel stack (kernel_sp).
   • Trap handler: Specifies usertrap as the next kernel-mode trap entry.
   • Hart ID: Records the current CPU core ID (kernel_hartid).
4. Sets privilege registers:
   • sstatus:
     • Clears the SPP flag (sets privilege mode to user).
     • Enables user-mode interrupts (sets SPIE).
   • sepc: Restores the user program counter (epc), specifying where to resume execution in user mode.
5. Switches page tables: Computes the user page table’s satp value and passes it to userret in trampoline.S.
6. Jumps to user mode:
   • Calls userret, which:
     • Switches from kernel to user page table.
     • Restores user registers.
     • Uses the sret instruction to enter user mode and enable interrupts.

The userret function restores all register values from the TRAPFRAME and resumes execution of the original user thread.

What If an Interrupt Occurs During Another Interrupt?

If, during a system call, another interrupt (e.g., a timer interrupt) occurs while already in user mode, what happens? XV6 handles this scenario. At the start of usertrap, it sets stvec to kernelvec, so if another interrupt occurs, execution jumps to the assembly code at kernelvec. This assembly is relatively simple; to summarize:

1. Decrease sp by 256 to reserve space for registers.
2. Save current register values to the stack.
3. Call kerneltrap.
4. Restore register values saved in step 2.
5. Increase sp by 256 to release the reserved space.

The kerneltrap function only checks for hardware and timer interrupts.

How Is the First Interrupt Set Up?

If you study the code carefully, you’ll notice that stvec is only set during calls to usertrapret. So, when is usertrapret first called? After a quick search, I found it’s invoked in the forkret function, which is set up in allocproc. Thus, each process’s stvec is configured immediately upon creation, ensuring correct interrupt handling during execution.

Where is the first process executed? In userinit(). Let’s look at the OS main function:

void
main()
{
  if(cpuid() == 0){
    consoleinit();
    printfinit();
    printf("\n");
    printf("xv6 kernel is booting\n");
    printf("\n");
    kinit();         // physical page allocator
    ...
    userinit();      // first user process
    __sync_synchronize();
    started = 1;
  } else {
    ...
  }

  scheduler();
}

The first user process is created in main via userinit. This is the ancestor process; all other processes are forked from it. The first process is quite important; let’s briefly see what it does:

int
main(void)
{
  int pid, wpid;

  if(open("console", O_RDWR) < 0){
    mknod("console", CONSOLE, 0);
    open("console", O_RDWR);
  }
  dup(0);  // stdout
  dup(0);  // stderr

  for(;;){
    printf("init: starting sh\n");
    pid = fork();
    if(pid < 0){
      printf("init: fork failed\n");
      exit(1);
    }
    if(pid == 0){
      exec("sh", argv);
      printf("init: exec sh failed\n");
      exit(1);
    }

    for(;;){
      // this call to wait() returns if the shell exits,
      // or if a parentless process exits.
      wpid = wait((int *) 0);
      if(wpid == pid){
        // the shell exited; restart it.
        break;
      } else if(wpid < 0){
        printf("init: wait returned an error\n");
        exit(1);
      } else {
        // it was a parentless process; do nothing.
      }
    }
  }
}

Essentially, it sets up standard input/output, forks a process to run the shell, and then loops indefinitely.

Timer Interrupts and Process Scheduling

After usertrap is called in the OS main function, the system enters the scheduler function to schedule processes. This is a very simple round-robin scheduler. All interrupts are handled in processes created by userinit; the scheduler function itself is unaffected. To understand this, let’s look at the scheduler code:

void
scheduler(void)
{
  struct proc *p;
  struct cpu *c = mycpu();

  c->proc = 0;
  for(;;){
    // The most recent process to run may have had interrupts
    // turned off; enable them to avoid a deadlock if all
    // processes are waiting.
    intr_on();

    int found = 0;
    for(p = proc; p < &proc[NPROC]; p++) {
      acquire(&p->lock);
      if(p->state == RUNNABLE) {
        ...
        swtch(&c->context, &p->context);

        ...
      }
      release(&p->lock);
    }
    ...
  }
}

The swtch function switches to the corresponding process. If a timer interrupt occurs, the yield function in usertrap is called, which returns directly to the next line after swtch in scheduler.

Summary

XV6 implements user and kernel mode interrupt handling through the closed loop of uservec → usertrap → usertrapret, leveraging RISC-V privileged instructions (sret, csrw) and dynamic page table switching to ensure efficient and secure interrupt response. The core design centers on register context management and dynamic page table switching, providing foundational support for multitasking concurrency.

For timer interrupts, pay particular attention to the interplay between the swtch and yield functions in the scheduler.