It’s been a while since I last posted and in fact I haven’t been working on the project much recently. My job is keeping me extremely busy again. Still, I made some progress on faults and interrupts mechanism for the CPU which I never managed to document. I simulated and tested the design, so it is time to post the summary of changes here and move forward to memory module which is going to be a final step before declaring the entire design complete.
As a quick reminder, the CPU will support both faults and interrupts, with faults being mostly internal breaks in normal processing flow due to exceptional condition (e.g. user mode tries to run instruction that can only be executed in kernel mode), whereas interrupts are a way for an external device (keyboard, serial port, real time clock) to break the program flow and be served. On hardware level, faults and interrupts are based on two priority encoders, generating addresses of special opcodes (not directly accessible by the programmer, at addresses $100 and above). The difference between them is that faults are processed immediately (there is no need or it is undesired to wait until all instruction steps complete), and interrupt processing starts only on machine instruction boundary (on fetch).
Typically, when executing user mode code, the following should happen during a context switch resulting from the interrupt (for faults it is pretty much the same):
- disable interrupts
- store machine status word (MSW)
- switch to supervisor mode
- store remaining machine state onto stack (all registers)
- get interrupt vector and jump to ISR (this is actually fetch with PC set to ISR address)
All the above happens during a special instruction invoked by the microcode sequencing logic in case there is a pending interrupt. When the ISR is complete it explicitly executes an IRET instruction which must be a final instruction of every fault/interrupt handler and does the reverse:
- restore registers from stack
- restore MSW (and original CPU mode)
- enable interrupts
- fetch and continue normal execution (original PC is restored before this fetch)
The above two schemed immediately raised a lot of questions and generated new hardware requirements. Below is a summary of new functionalities added to the design to correctly handle faults and interrupts.
Setting CPU mode and enabling/disabling interrupts from microcode. There are two special machine status bits (low active) that play a role here. The S flag, when set denotes that the CPU is in supervisor (kernel) mode and the I flag is used to enable and disable (mask) interrupt handling. In order to individually set them I needed additional microcode. With only few microcode bits left I took advantage of the fact that 2 bits of BISIFCMODE field are only relevant in case ALU-data bus interface is enabled (BUSIFCEN=0). In other case, it is OK to use them. This means that I can never latch mode bits and use the bus interface at the same time but a quick look at my current microcode proved that it will never be necessary. So, I remodeled the microcode word, added some extra decoding logic and ended up with something like this (snapshot from microcode word definition):
| Field | Pos | Length | Values | Description |
| BUSIFCMODE | 6 | 2 | MEM2ALULO, MEM2ALUHI, ALULO2MEM, ALUHI2MEM | Bus interface direction and mode (if BUSIFCEN=0) |
| MISC_LATCH | 6 | 2 | LATCH_I, LATCH_S, nc, LATCH_NONE | Latch individual MSW flags (BUSIFCEN=1) |
| BUSIFCEN | 8 | 1 | enable, disable | Bus interface enable signal |
Now in order to set and reset the flags from microcode, I first have to drive the ALU bus with either all zeros or all ones (both are easily available in ‘181) and assert the LATCH_I or LATCH_S signal (only one MSW bit is latched in both cases). Here is a sample microcode snippet:
0, *, MDR <- -1; LATCH_I // disable further interrupts
// …
3, *, MDR <- -1 + 1; LATCH_S // enable supervisor mode
Determining fault or interrupt number and locating interrupt vector table in memory. Another problem we are encountering is determining the number (code) of fault or interrupt in the microcode, and reading the correct interrupt vector from memory. This is relevant in order to revert processing to the correct ISR. The solution was quick and easy. I tied the output of the priority encoder to the right ALU bus, putting a three-state driver (74’244) in between. Such pseudo-register lets me read the fault/interrupt code, load it into MAR, read interrupt vector memory contents, load it into PC and proceed with interrupt handling. The only problem is that my right ALU bus drive enable microcode field was just two bits long and adding a new pseudo register to it (code 0x03, called it IPTR) used up the last empty slot which soon required me to increase the field size (also see below). However, the solution works fine. The IPTR produces values $00-$0f for faults and $10-$1f for interrupts – priority encoder output is shifted left one bit to produce direct pointers to 2-byte interrupt vectors. As a result I reserve the memory region $0000-$001f in supervisor data space for the interrupt vector table.
Keeping track of the instruction’s initial PC to be able to re-run it after fault. The typical source of faults is an instruction attempting to perform an operation which is not possible given the current machine state, e.g. user mode program invokes privileged instruction, or the code is trying to access memory region which is not paged in (typical page fault). A correct fault handling routine will fix the situation (e.g. perform disk/memory page swap) and revert back to original code to repeat the instruction. And here is the catch. Where is the instruction’s initial address the PC should refer to before returning? Instructions are variable in length – there is one byte of opcode in code memory followed by one or two bytes of operand (this is the case in my current ISA but I can imagine lengthier instruction encodings for some special ops). On faulting situation we may be anywhere in the middle of processing, e.g. having read just one byte of immediate operand with the other byte still remaining unread (and PC not advanced). The solution here is to keep track of the instruction’s initial PC in a special purpose register (PPC – previous program counter) which is only latched on instruction boundaries. When switching context on fault it is the PPC (not PC) that is pushed to the stack to be later retrieved during IRET operation. It worked fine but I ran into the problem of free microcode word slots with this new register. Since my IPTR register described in the previous paragraph consumed the last free right ALU bus drive enable signal slot, I was forced to increase the field’s length to 3 bits. This leaves me with only 2 unused microcode word bits – dangerously close to hard limit. Fortunately, the design is nearly complete so I hope to be able to fit into 4-byte microcode word and never have to add additional chip. Here is the current snapshot of LBUS and RBUS field in microcode word specs:
| Field | Pos | Length | Values | Description |
| LBUS | 0 | 2 | MDR, A, X, Y | Left bus drive enable register select |
| RBUS | 2 | 3 | MDR, MSW, ABUS, IPTR, PPC, nc, nc, nc | Right bus drive enable register select |
Right ALU bus drive enable signal has grown to 3 bits with three slots currently unused.
Keeping track of SP correctly in user and supervisor modes. While thinking about user-supervisor context switch sequence outlined a few paragraphs above, at some point I ran into yet another problem, which initially seemed like a catch 22 situation. Consider again a context switch from user mode program to supervisor mode in case of an interrupt. Storing the machine state to stack raised a few questions. Switching from user to supervisor will change the machine’s address space (the memory subsystem will assure that and this will happen automatically). Should the state be saved to user program’s stack (before changing state) or to the kernel’s stack (after the switch)? Since interrupts should be fully transparent to the user program and I don’t want the programmer to have to make any kind of assumptions regarding the stack location (in order to leave enough space to possible context switch frames) I will store the context in kernel space. Ok, but where is the kernel stack? On user to supervisor transition the SP register points to a random location considering kernel space (as it could have been set to any location by user program) and in order to store the context, it needs to be set to a location of a stack in the kernel (or some other spill space that the OS reserves for such data). Fine, but if we change the SP, we destroy the SP’s value from user mode. At first glance I thought that maybe I should first store the original value of SP to one of the scratch registers (MAR or MDR) and then set the SP to kernel’s spill space, having first obtained it somewhere from kernel memory. This would require storing the SP to this arbitrary memory location before passing control from kernel to user, and restoring the value during user to kernel switch. This requires additional memory accesses on context switching (slow), and requires generating new immediates in the microcode (limited possibilities here). With this in mind I decided to go for a hardware solution and have two stack pointers, the KSP in kernel mode and USP in user mode. On microcode and machine language level nothing changes – we use SP to refer to both and KSP/USP are transparent. Hardware assures that KSP is accessed whenever the S flag in the MSW is low (CPU is in supervisor mode) and USP is access in the other case. This way during user to kernel context transition SP immediately points to the kernel spill space (the same value that was set by the OS before passing control to the user mode program). Similarly, after a switch from kernel to user program, SP points to USP and the programmer may set it to any location in the program’s 64k data segment (the KSP remains intact).
With the above problems solved, here is the microcode for a fault/interrupt context switch:
// fault0 op($100) 0, *, MDR <- -1; LATCH_I // disable further interrupts 1, *, MAR <- SP // back up SP 2, *, MDR <- MSW // back up MSW before (to store original CPU mode) 3, *, MDR <- -1 + 1; LATCH_S // enable supervisor mode (now SP is KSP), MDR not latched here 4, *, SP-- // this is KSP 5, *, MEM(SP) <- LO(MDR); SP-- // store MSW 6, *, MEM(SP) <- HI(MDR); SP-- 7, *, MDR <- MAR // store SP 8, *, MEM(SP) <- LO(MDR); SP-- 9, *, MEM(SP) <- HI(MDR); SP-- 10, *, MEM(SP) <- LO(A); SP-- // store A 11, *, MEM(SP) <- HI(A); SP-- 12, *, MEM(SP) <- LO(X); SP-- // store X 13, *, MEM(SP) <- HI(X); SP-- 14, *, MEM(SP) <- LO(Y); SP-- // store Y 15, *, MEM(SP) <- HI(Y); SP-- 16, *, MDR <- DP // store DP 17, *, MEM(SP) <- LO(MDR); SP-- 18, *, MEM(SP) <- HI(MDR); SP-- 19, *, MEM(SP) <- LO(PPC); SP-- // store PPC (the instruction's starting address) 20, *, MEM(SP) <- HI(PPC); 21, *, MAR <- IPTR // jump to ISR (IPTR contains the fault/IRQ number) 22, *, HI(MDR) <- MEM(MAR); MAR++ 23, *, LO(MDR) <- MEM(MAR) 24, *, PC <- MDR 25, *, fetch endop
Also, here is my current IRET (instruction to return to program execution after fault/interrupt handling is complete):
// IRET op($1F) 0, *, HI(MDR) <- MEM(SP); SP++ // restore PC 1, *, LO(MDR) <- MEM(SP); SP++ 2, *, PC <- MDR 3, *, HI(MDR) <- MEM(SP); SP++ // restore DP 4, *, LO(MDR) <- MEM(SP); SP++ 5, *, DP <- MDR 6, *, HI(MDR) <- MEM(SP); SP++ // restore Y 7, *, LO(MDR) <- MEM(SP); SP++ 8, *, Y <- MDR 9, *, HI(MDR) <- MEM(SP); SP++ // restore X 10, *, LO(MDR) <- MEM(SP); SP++ 11, *, X <- MDR 12, *, HI(A) <- MEM(SP); SP++ // restore A 13, *, LO(A) <- MEM(SP); SP++ 14, *, HI(MAR) <- MEM(SP); SP++ // restore SP value (but not yet set SP) 15, *, LO(MAR) <- MEM(SP); SP++ 16, *, HI(MDR) <- MEM(SP); SP++ // restore MSW value (but not yet set MSW) 17, *, LO(MDR) <- MEM(SP); SP++ 18, *, MSW <- MDR // restore machine status word (CPU mode, etc.) 19, *, SP <- MAR // restore stack pointer (KSP or USP, depending on restored CPU mode) 20, *, MDR <- -1 + 1; LATCH_I; fetch // enable interrupts and fetch endop
Finally, as a summary of today’s post and for the sake of good documentation, here is a complete block diagram of the CPU containing new elements – IPTR and PPC registers.
Faults and interrupts development also required changes in the entire toolchain (mostly the microcode assembler and the simulator). Also, I wrote and ran a couple of simple tests, which were incorporated into the test suite. All files were updated and are available in the downloads section with today’s date.