Refactor tutorial 12
parent
1496e003d8
commit
85b88788d0
@ -1,992 +0,0 @@
|
||||
# Tutorial 12 - CPU Exceptions: Part 1
|
||||
|
||||
## tl;dr
|
||||
|
||||
We lay the groundwork for all the architectural `CPU exceptions`. For now, only print an elaborate
|
||||
system state through a `panic!` call, and halt execution; This will help finding bugs during
|
||||
development and runtime.
|
||||
|
||||
For demo purposes, MMU `page faults` are used to demonstrate (i) returning from an exception and
|
||||
(ii) the default `panic!` behavior.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Exception Types](#exception-types)
|
||||
- [Exception entry](#exception-entry)
|
||||
* [Exception Vectors](#exception-vectors)
|
||||
- [Handler Code and Offsets](#handler-code-and-offsets)
|
||||
- [Rust and Assembly Implementation](#rust-and-assembly-implementation)
|
||||
* [Context Save and Restore](#context-save-and-restore)
|
||||
* [Exception Vector Table](#exception-vector-table)
|
||||
* [Implementing handlers](#implementing-handlers)
|
||||
- [Causing an Exception - Testing the Code](#causing-an-exception---testing-the-code)
|
||||
- [Test it](#test-it)
|
||||
- [Diff to previous](#diff-to-previous)
|
||||
|
||||
## Introduction
|
||||
|
||||
Now that we are executing in `EL1`, and have activated the `MMU`, time is due for implementing `CPU
|
||||
exceptions`. For now, we only set up a scaffold with very basic functionality that will help us to
|
||||
find bugs along the way. A follow-up `Interrupt` tutorial in the future will continue the work we
|
||||
start here.
|
||||
|
||||
Please note that this tutorial is specific to the `AArch64` architecture. It does not contain any
|
||||
generic exception handling code yet.
|
||||
|
||||
## Exception Types
|
||||
|
||||
In `AArch64`, it is differentiated between four types of exceptions. These are:
|
||||
- Synchronous
|
||||
- For example, a `data abort` (e.g. `page fault`) or a `system call`. They happen in direct
|
||||
consequence of executing a certain instruction, hence _synchronously_.
|
||||
- Interrupt Request (`IRQ`)
|
||||
- For example, an external device, like a timer, is asserting a physical interrupt line. IRQs
|
||||
happen _asynchronously_.
|
||||
- Fast Interrupt Request (`FIQ`)
|
||||
- These are basically interrupts that take priority over normal IRQs and have some more traits
|
||||
that make them suitable to implement super-fast processing. However, this is out of scope for
|
||||
this tutorial. For the sake of keeping these tutorials compact and concise, we will more or less
|
||||
ignore FIQs and only implement a dummy handler that would halt the CPU core.
|
||||
- System Error (`SError`)
|
||||
- Like IRQs, SErrors happen asynchronously and are technically more or less the same. They are
|
||||
intended to signal rather fatal errors in the system, e.g. if a transaction times out on the
|
||||
`SoC` interconnect. They are very implementation specific and it is up to the SoC vendor to
|
||||
decide which events are delivered as SErrors instead of normal IRQs.
|
||||
|
||||
## Exception entry
|
||||
|
||||
I recommend to read pages 1874-1876 of the [ARMv8 Architecture Reference Manual][ARMv8_Manual] to
|
||||
understand the mechanisms of taking an exception.
|
||||
|
||||
Here's an excerpt of important features for this tutorial:
|
||||
- Exception entry moves the processor to the same or a higher `Exception Level`, but never to a
|
||||
lower `EL`.
|
||||
- The program status is saved in the `SPSR_ELx` register at the target `EL`.
|
||||
- The preferred return address is saved in the `ELR_ELx` register.
|
||||
- "Preferred" here means that `ELR_ELx` may hold the instruction address of the instructions that
|
||||
caused the exception (`synchronous case`) or the first instruction that did not complete due to
|
||||
an `asynchronous` exception. Details in Chapter D1.10.1 of the [ARMv8 Architecture Reference
|
||||
Manual][ARMv8_Manual].
|
||||
- All kinds of exceptions are turned off upon taking an exception, so that by default, exception
|
||||
handlers can not get interrupted themselves.
|
||||
- Taking an exception will select the dedicated stack pointer of the target `EL`.
|
||||
- For example, if an exception in `EL0` is taken, the Stack Pointer Select register `SPSel` will
|
||||
switch from `0` to `1`, meaning that `SP_EL1` will be used by the exception vector code unless
|
||||
you explicitly change it back to `SP_EL0`.
|
||||
|
||||
|
||||
### Exception Vectors
|
||||
|
||||
`AArch64` has a total of `16` exception vectors. There is one for each of the four kinds that were
|
||||
introduced already, and additionally, it is taken into account _where_ the exception was taken from
|
||||
and what the circumstances were.
|
||||
|
||||
Here is a copy of the decision table as shown in Chapter D1.10.2 of the [ARMv8 Architecture
|
||||
Reference Manual][ARMv8_Manual]:
|
||||
|
||||
[ARMv8_Manual]: https://developer.arm.com/docs/ddi0487/latest/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th rowspan=2>Exception taken from </th>
|
||||
<th colspan=4>Offset for exception type</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Synchronous</th>
|
||||
<th>IRQ or vIRQ</th>
|
||||
<th>FIQ or vFIQ</th>
|
||||
<th>SError or vSError</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td width="40%">Current Exception level with SP_EL0.</td>
|
||||
<td align="center">0x000</td>
|
||||
<td align="center">0x080</td>
|
||||
<td align="center">0x100</td>
|
||||
<td align="center">0x180</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Current Exception level with SP_ELx, x>0.</td>
|
||||
<td align="center">0x200</td>
|
||||
<td align="center">0x280</td>
|
||||
<td align="center">0x300</td>
|
||||
<td align="center">0x380</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Lower Exception level, where the implemented level immediately lower than the target level is using AArch64.</td>
|
||||
<td align="center">0x400</td>
|
||||
<td align="center">0x480</td>
|
||||
<td align="center">0x500</td>
|
||||
<td align="center">0x580</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Lower Exception level, where the implemented level immediately lower than the target level is using AArch32.</td>
|
||||
<td align="center">0x600</td>
|
||||
<td align="center">0x680</td>
|
||||
<td align="center">0x700</td>
|
||||
<td align="center">0x780</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
Since our kernel runs in `EL1`, using `SP_EL1`, if we'd cause a synchronous exception, the exception
|
||||
vector at offset `0x200` would be executed. But what does that even mean?
|
||||
|
||||
## Handler Code and Offsets
|
||||
|
||||
In many architectures, Operating Systems register their exception handlers (aka vectors) by
|
||||
compiling an architecturally defined data structure that stores function pointers to the different
|
||||
handlers. This can be as simple as an ordinary array of function pointers. The `base address` of
|
||||
this data structure is then stored into a special purpose register so that the CPU can branch to the
|
||||
respective handler function upon taking an exception. The classic `x86_64` architecture follows this
|
||||
principle, for example.
|
||||
|
||||
In `AArch64`, it is a bit different. Here, we have the special purpose register as well, called
|
||||
`VBAR_EL1`: Vector Base Address Register.
|
||||
|
||||
However, it does not store the base address of an array of function pointers, but the base address
|
||||
of a **memory location that contains code** for the 16 handlers, one handler back-to-back after the
|
||||
other. Each handler can take a maximum space of `0x80` bytes, aka `128` bytes. That's why the table
|
||||
above shows `offsets`: To indicate at which offset a certain handler starts.
|
||||
|
||||
Of course, you are not obliged to cram all your handler code into only 128 bytes. You are free to
|
||||
branch off to any other functions at any time. Actually, that is needed in most cases anyways,
|
||||
because the context-saving code alone would take up most of the available space (you'll learn what
|
||||
context saving is shortly).
|
||||
|
||||
Additionally, there is a requirement that the `Vector Base Address` is aligned to `0x800` aka `2048`
|
||||
bytes.
|
||||
|
||||
## Rust and Assembly Implementation
|
||||
|
||||
The implementation uses a mix of `Rust` and `Assembly` code.
|
||||
|
||||
### Context Save and Restore
|
||||
|
||||
Exception vectors, just like any other code, use a bunch of commonly shared processor resources.
|
||||
Most of all, the set of `General Purpose Registers` (GPRs) that each core in `AArch64` provides
|
||||
(`x0`-`x30`).
|
||||
|
||||
In order to not taint these registers when executing exception vector code, it is general practice
|
||||
to save these shared resources in memory (the stack, to be precise) as the very first action. This
|
||||
is commonly described as *saving the context*. Exception vector code can then use the shared
|
||||
resources in its own code without bothering, and as a last action before returning from exception
|
||||
handling code, restore the context, so that the processor can continue where it left off before
|
||||
taking the exception.
|
||||
|
||||
Context save and restore is one of the few places in system software where it is strongly advised to
|
||||
to use some hand-crafted assembly. Introducing `exception.S`:
|
||||
|
||||
```asm
|
||||
/// Call the function provided by parameter `\handler` after saving exception context. Provide the
|
||||
/// context as the first parameter to '\handler'.
|
||||
.macro CALL_WITH_CONTEXT handler
|
||||
// Make room on the stack for the exception context.
|
||||
sub sp, sp, #16 * 17
|
||||
|
||||
// Store all general purpose registers on the stack.
|
||||
stp x0, x1, [sp, #16 * 0]
|
||||
stp x2, x3, [sp, #16 * 1]
|
||||
stp x4, x5, [sp, #16 * 2]
|
||||
stp x6, x7, [sp, #16 * 3]
|
||||
stp x8, x9, [sp, #16 * 4]
|
||||
stp x10, x11, [sp, #16 * 5]
|
||||
stp x12, x13, [sp, #16 * 6]
|
||||
stp x14, x15, [sp, #16 * 7]
|
||||
stp x16, x17, [sp, #16 * 8]
|
||||
stp x18, x19, [sp, #16 * 9]
|
||||
stp x20, x21, [sp, #16 * 10]
|
||||
stp x22, x23, [sp, #16 * 11]
|
||||
stp x24, x25, [sp, #16 * 12]
|
||||
stp x26, x27, [sp, #16 * 13]
|
||||
stp x28, x29, [sp, #16 * 14]
|
||||
|
||||
// Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1).
|
||||
mrs x1, ELR_EL1
|
||||
mrs x2, SPSR_EL1
|
||||
|
||||
stp lr, x1, [sp, #16 * 15]
|
||||
str w2, [sp, #16 * 16]
|
||||
|
||||
// x0 is the first argument for the function called through `\handler`.
|
||||
mov x0, sp
|
||||
|
||||
// Call `\handler`.
|
||||
bl \handler
|
||||
|
||||
// After returning from exception handling code, replay the saved context and return via `eret`.
|
||||
b __exception_restore_context
|
||||
.endm
|
||||
```
|
||||
|
||||
First, a macro for saving the context is defined. It eventually jumps to follow-up handler code, and
|
||||
finally restores the context. In advance, we reserve space on the stack for the context. That is,
|
||||
the 30 `GPRs`, the `link register`, the `saved program status` and the `exception link register`
|
||||
(holding the preferred return address). Afterwards, we store those registers, save the current stack
|
||||
address in `x0` and branch off to follow-up handler-code, whose function name is supplied as an
|
||||
argument to the macro (`\handler`).
|
||||
|
||||
The handler code will be written in Rust, but use the platform's `C` ABI. This way, we can define a
|
||||
function signature that has a pointer to the context-data on the stack as its first argument, and
|
||||
know that this argument is expected to be in the `x0` register. We need to use the `C` ABI here
|
||||
because `Rust` has no stable convention ([yet](https://github.com/rust-lang/rfcs/issues/600)).
|
||||
|
||||
### Exception Vector Table
|
||||
|
||||
Next, we craft the exception vector table:
|
||||
|
||||
```asm
|
||||
.section .exception_vectors, "ax", @progbits
|
||||
|
||||
// Align by 2^11 bytes, as demanded by the AArch64 spec. Same as ALIGN(2048) in an ld script.
|
||||
.align 11
|
||||
|
||||
// Export a symbol for the Rust code to use.
|
||||
__exception_vector_start:
|
||||
|
||||
// Current exception level with SP_EL0.
|
||||
// .org sets the offset relative to section start.
|
||||
//
|
||||
// It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes.
|
||||
.org 0x000
|
||||
CALL_WITH_CONTEXT current_el0_synchronous
|
||||
.org 0x080
|
||||
CALL_WITH_CONTEXT current_el0_irq
|
||||
.org 0x100
|
||||
FIQ_SUSPEND
|
||||
.org 0x180
|
||||
CALL_WITH_CONTEXT current_el0_serror
|
||||
|
||||
// Current exception level with SP_ELx, x > 0.
|
||||
.org 0x200
|
||||
CALL_WITH_CONTEXT current_elx_synchronous
|
||||
.org 0x280
|
||||
CALL_WITH_CONTEXT current_elx_irq
|
||||
.org 0x300
|
||||
FIQ_SUSPEND
|
||||
.org 0x380
|
||||
CALL_WITH_CONTEXT current_elx_serror
|
||||
|
||||
[...]
|
||||
```
|
||||
|
||||
Note how each vector starts at the required offset from the section start using the `.org`
|
||||
directive. Each macro call introduces an explicit handler function name, which is implemented in
|
||||
`Rust` in `exception.rs`.
|
||||
|
||||
### Implementing handlers
|
||||
|
||||
The file `exception.rs` first defines a `struct` of the exception context that is stored on the
|
||||
stack by the assembly code:
|
||||
|
||||
```rust
|
||||
/// The exception context as it is stored on the stack on exception entry.
|
||||
#[repr(C)]
|
||||
struct ExceptionContext {
|
||||
// General Purpose Registers.
|
||||
gpr: [u64; 30],
|
||||
// The link register, aka x30.
|
||||
lr: u64,
|
||||
// Exception link register. The program counter at the time the exception happened.
|
||||
elr_el1: u64,
|
||||
// Saved program status.
|
||||
spsr_el1: SpsrEL1,
|
||||
}
|
||||
```
|
||||
|
||||
The handlers take a `struct ExceptionContext` argument. Since we do not plan to implement handlers
|
||||
for each exception yet, a default handler is provided:
|
||||
|
||||
```rust
|
||||
/// Print verbose information about the exception and the panic.
|
||||
fn default_exception_handler(e: &ExceptionContext) {
|
||||
panic!(
|
||||
"\n\nCPU Exception!\n\
|
||||
FAR_EL1: {:#018x}\n\
|
||||
{}\n\
|
||||
{}",
|
||||
FAR_EL1.get(),
|
||||
EsrEL1 {},
|
||||
e
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
The actual handlers referenced from the assembly can now branch to it for the time being, e.g.:
|
||||
|
||||
```rust
|
||||
#[no_mangle]
|
||||
unsafe extern "C" fn current_el0_synchronous(e: &mut ExceptionContext) {
|
||||
default_exception_handler(e);
|
||||
}
|
||||
```
|
||||
|
||||
## Causing an Exception - Testing the Code
|
||||
|
||||
We want to see two cases in action:
|
||||
1. How taking, handling and returning from an exception works.
|
||||
2. How the `panic!` print for unhandled exceptions looks like.
|
||||
|
||||
|
||||
So after setting up exceptions in `main.rs` by calling
|
||||
|
||||
```rust
|
||||
arch::enable_exception_handling();
|
||||
```
|
||||
|
||||
we cause a data abort exception by reading from memory address `8 GiB`:
|
||||
|
||||
```rust
|
||||
// Cause an exception by accessing a virtual address for which no translation was set up. This
|
||||
// code accesses the address 8 GiB, which is outside the mapped address space.
|
||||
//
|
||||
// For demo purposes, the exception handler will catch the faulting 8 GiB address and allow
|
||||
// execution to continue.
|
||||
info!("");
|
||||
info!("Trying to write to address 8 GiB...");
|
||||
let mut big_addr: u64 = 8 * 1024 * 1024 * 1024;
|
||||
unsafe { core::ptr::read_volatile(big_addr as *mut u64) };
|
||||
```
|
||||
|
||||
This triggers our exception code, because we try to read from a virtual address for which no mapping
|
||||
has been installed. Remember, we only installed identity-mapped page tables for the first `1 GiB`
|
||||
(RPi3) or `4 GiB` (RPi4) of address space in the previous tutorial.
|
||||
|
||||
To survive this exception, the respective handler has a special demo case:
|
||||
|
||||
```rust
|
||||
/// Asynchronous exception taken from the current EL, using SP of the current EL.
|
||||
#[no_mangle]
|
||||
unsafe extern "C" fn current_elx_synchronous(e: &mut ExceptionContext) {
|
||||
let far_el1 = FAR_EL1.extract().get();
|
||||
|
||||
// This catches the demo case for this tutorial. If the fault address happens to be 8 GiB,
|
||||
// advance the exception link register for one instruction, so that execution can continue.
|
||||
if far_el1 == 8 * 1024 * 1024 * 1024 {
|
||||
e.elr_el1 += 4;
|
||||
|
||||
asm::eret()
|
||||
}
|
||||
|
||||
default_exception_handler(e);
|
||||
}
|
||||
```
|
||||
|
||||
It checks if the faulting address equals `8 GiB`, and if so, advances the copy of the `ELR` by 4,
|
||||
which makes it point to the next instruction after the instruction that caused the exception. When
|
||||
this handler returns, execution continues in the assembly macro we introduced before. The macro has
|
||||
only one more line left: `b __exception_restore_context`, which jumps to an assembly function that
|
||||
plays back our saved context before finally executing `eret` to return from the exception.
|
||||
|
||||
This will kick us back into `main.rs`. But we also want to see the `panic!` print.
|
||||
|
||||
Therefore, a second read is done, this time from address `9 GiB`. A case which the handler will not
|
||||
catch, eventually triggering the `panic!` call from the default handler.
|
||||
|
||||
## Test it
|
||||
|
||||
Emphasis on the events at timestamps > `6.xxxxxx`.
|
||||
|
||||
```console
|
||||
» make chainboot
|
||||
[...]
|
||||
Minipush 1.0
|
||||
|
||||
[MP] ⏳ Waiting for /dev/ttyUSB0
|
||||
[MP] ✅ Connected
|
||||
__ __ _ _ _ _
|
||||
| \/ (_)_ _ (_) | ___ __ _ __| |
|
||||
| |\/| | | ' \| | |__/ _ \/ _` / _` |
|
||||
|_| |_|_|_||_|_|____\___/\__,_\__,_|
|
||||
|
||||
Raspberry Pi 3
|
||||
|
||||
[ML] Requesting binary
|
||||
[MP] ⏩ Pushing 64 KiB ========================================🦀 100% 32 KiB/s Time: 00:00:02
|
||||
[ML] Loaded! Executing the payload now
|
||||
|
||||
[ 2.913260] Booting on: Raspberry Pi 3
|
||||
[ 2.914344] MMU online. Special regions:
|
||||
[ 2.916256] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data
|
||||
[ 2.920338] 0x3f000000 - 0x3fffffff | 16 MiB | Dev RW PXN | Device MMIO
|
||||
[ 2.923901] Current privilege level: EL1
|
||||
[ 2.925812] Exception handling state:
|
||||
[ 2.927593] Debug: Masked
|
||||
[ 2.929156] SError: Masked
|
||||
[ 2.930720] IRQ: Masked
|
||||
[ 2.932284] FIQ: Masked
|
||||
[ 2.933848] Architectural timer resolution: 52 ns
|
||||
[ 2.936150] Drivers loaded:
|
||||
[ 2.937496] 1. GPIO
|
||||
[ 2.938756] 2. PL011Uart
|
||||
[ 2.940233] Timer test, spinning for 1 second
|
||||
[ 3.942362]
|
||||
[ 3.942366] Trying to write to address 8 GiB...
|
||||
[ 3.944531] ************************************************
|
||||
[ 3.947310] Whoa! We recovered from a synchronous exception!
|
||||
[ 3.950091] ************************************************
|
||||
[ 3.952870]
|
||||
[ 3.953566] Let's try again
|
||||
[ 3.954912] Trying to write to address 9 GiB...
|
||||
|
||||
Kernel panic:
|
||||
|
||||
CPU Exception!
|
||||
FAR_EL1: 0x0000000240000000
|
||||
ESR_EL1: 0x96000004
|
||||
Exception Class (EC) : 0x25 - Data Abort, current EL
|
||||
Instr Specific Syndrome (ISS): 0x4
|
||||
ELR_EL1: 0x0000000000080e50
|
||||
SPSR_EL1: 0x600003c5
|
||||
Flags:
|
||||
Negative (N): Not set
|
||||
Zero (Z): Set
|
||||
Carry (C): Set
|
||||
Overflow (V): Not set
|
||||
Exception handling state:
|
||||
Debug (D): Masked
|
||||
SError (A): Masked
|
||||
IRQ (I): Masked
|
||||
FIQ (F): Masked
|
||||
Illegal Execution State (IL): Not set
|
||||
|
||||
General purpose register:
|
||||
x0 : 0x0000000000000000 x1 : 0x000000000008594e
|
||||
x2 : 0x0000000000000026 x3 : 0x0000000000082b38
|
||||
x4 : 0x000000000007fc5c x5 : 0x0000000000000003
|
||||
x6 : 0x0000000000000000 x7 : 0xd3d1c80822850243
|
||||
x8 : 0x0000000240000000 x9 : 0x000000000008594e
|
||||
x10: 0x0000000000000414 x11: 0x000000003f201000
|
||||
x12: 0x0000000000000019 x13: 0x000000000007fc5d
|
||||
x14: 0x000000000007fda8 x15: 0x0000000000000040
|
||||
x16: 0x0000000000000000 x17: 0x0000000000000040
|
||||
x18: 0x9cc47880812f1200 x19: 0x0000000000090008
|
||||
x20: 0x000000003b9aca00 x21: 0x00000000000003e8
|
||||
x22: 0x0000000000083070 x23: 0x00000000000831e4
|
||||
x24: 0x00000000000f4240 x25: 0x00000000000852a8
|
||||
x26: 0x0000000000085738 x27: 0x0000000000085818
|
||||
x28: 0x00000000000831e4 x29: 0x0000000000085588
|
||||
lr : 0x0000000000080e44
|
||||
```
|
||||
|
||||
## Diff to previous
|
||||
```diff
|
||||
|
||||
diff -uNr 11_virtual_memory/src/arch/aarch64/exception.rs 12_cpu_exceptions_part1/src/arch/aarch64/exception.rs
|
||||
--- 11_virtual_memory/src/arch/aarch64/exception.rs
|
||||
+++ 12_cpu_exceptions_part1/src/arch/aarch64/exception.rs
|
||||
@@ -4,12 +4,248 @@
|
||||
|
||||
//! Exception handling.
|
||||
|
||||
-use cortex_a::regs::*;
|
||||
+use core::fmt;
|
||||
+use cortex_a::{asm, barrier, regs::*};
|
||||
+use register::InMemoryRegister;
|
||||
+
|
||||
+// Assembly counterpart to this file.
|
||||
+global_asm!(include_str!("exception.S"));
|
||||
+
|
||||
+/// Wrapper struct for memory copy of SPSR_EL1.
|
||||
+#[repr(transparent)]
|
||||
+struct SpsrEL1(InMemoryRegister<u32, SPSR_EL1::Register>);
|
||||
+
|
||||
+/// The exception context as it is stored on the stack on exception entry.
|
||||
+#[repr(C)]
|
||||
+struct ExceptionContext {
|
||||
+ // General Purpose Registers.
|
||||
+ gpr: [u64; 30],
|
||||
+ // The link register, aka x30.
|
||||
+ lr: u64,
|
||||
+ // Exception link register. The program counter at the time the exception happened.
|
||||
+ elr_el1: u64,
|
||||
+ // Saved program status.
|
||||
+ spsr_el1: SpsrEL1,
|
||||
+}
|
||||
+
|
||||
+/// Wrapper struct for pretty printing ESR_EL1.
|
||||
+struct EsrEL1;
|
||||
+
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+// Exception vector implementation
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+
|
||||
+/// Print verbose information about the exception and the panic.
|
||||
+fn default_exception_handler(e: &ExceptionContext) {
|
||||
+ panic!(
|
||||
+ "\n\nCPU Exception!\n\
|
||||
+ FAR_EL1: {:#018x}\n\
|
||||
+ {}\n\
|
||||
+ {}",
|
||||
+ FAR_EL1.get(),
|
||||
+ EsrEL1 {},
|
||||
+ e
|
||||
+ );
|
||||
+}
|
||||
+
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+// Current, EL0
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn current_el0_synchronous(e: &mut ExceptionContext) {
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn current_el0_irq(e: &mut ExceptionContext) {
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn current_el0_serror(e: &mut ExceptionContext) {
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+// Current, ELx
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+
|
||||
+/// Asynchronous exception taken from the current EL, using SP of the current EL.
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn current_elx_synchronous(e: &mut ExceptionContext) {
|
||||
+ let far_el1 = FAR_EL1.get();
|
||||
+
|
||||
+ // This catches the demo case for this tutorial. If the fault address happens to be 8 GiB,
|
||||
+ // advance the exception link register for one instruction, so that execution can continue.
|
||||
+ if far_el1 == 8 * 1024 * 1024 * 1024 {
|
||||
+ e.elr_el1 += 4;
|
||||
+
|
||||
+ asm::eret()
|
||||
+ }
|
||||
+
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn current_elx_irq(e: &mut ExceptionContext) {
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn current_elx_serror(e: &mut ExceptionContext) {
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+// Lower, AArch64
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn lower_aarch64_synchronous(e: &mut ExceptionContext) {
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn lower_aarch64_irq(e: &mut ExceptionContext) {
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn lower_aarch64_serror(e: &mut ExceptionContext) {
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+// Lower, AArch32
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn lower_aarch32_synchronous(e: &mut ExceptionContext) {
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn lower_aarch32_irq(e: &mut ExceptionContext) {
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+#[no_mangle]
|
||||
+unsafe extern "C" fn lower_aarch32_serror(e: &mut ExceptionContext) {
|
||||
+ default_exception_handler(e);
|
||||
+}
|
||||
+
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+// Pretty printing
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+
|
||||
+/// Human readable ESR_EL1.
|
||||
+#[rustfmt::skip]
|
||||
+impl fmt::Display for EsrEL1 {
|
||||
+ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
|
||||
+ let esr_el1 = ESR_EL1.extract();
|
||||
+
|
||||
+ // Raw print of whole register.
|
||||
+ writeln!(f, "ESR_EL1: {:#010x}", esr_el1.get())?;
|
||||
+
|
||||
+ // Raw print of exception class.
|
||||
+ write!(f, " Exception Class (EC) : {:#x}", esr_el1.read(ESR_EL1::EC))?;
|
||||
+
|
||||
+ // Exception class, translation.
|
||||
+ let ec_translation = match esr_el1.read_as_enum(ESR_EL1::EC) {
|
||||
+ Some(ESR_EL1::EC::Value::DataAbortCurrentEL) => "Data Abort, current EL",
|
||||
+ _ => "N/A",
|
||||
+ };
|
||||
+ writeln!(f, " - {}", ec_translation)?;
|
||||
+
|
||||
+ // Raw print of instruction specific syndrome.
|
||||
+ write!(f, " Instr Specific Syndrome (ISS): {:#x}", esr_el1.read(ESR_EL1::ISS))?;
|
||||
+
|
||||
+ Ok(())
|
||||
+ }
|
||||
+}
|
||||
+
|
||||
+/// Human readable SPSR_EL1.
|
||||
+#[rustfmt::skip]
|
||||
+impl fmt::Display for SpsrEL1 {
|
||||
+ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
|
||||
+ // Raw value.
|
||||
+ writeln!(f, "SPSR_EL1: {:#010x}", self.0.get())?;
|
||||
+
|
||||
+ let to_flag_str = |x| -> _ {
|
||||
+ if x { "Set" } else { "Not set" }
|
||||
+ };
|
||||
+
|
||||
+ writeln!(f, " Flags:")?;
|
||||
+ writeln!(f, " Negative (N): {}", to_flag_str(self.0.is_set(SPSR_EL1::N)))?;
|
||||
+ writeln!(f, " Zero (Z): {}", to_flag_str(self.0.is_set(SPSR_EL1::Z)))?;
|
||||
+ writeln!(f, " Carry (C): {}", to_flag_str(self.0.is_set(SPSR_EL1::C)))?;
|
||||
+ writeln!(f, " Overflow (V): {}", to_flag_str(self.0.is_set(SPSR_EL1::V)))?;
|
||||
+
|
||||
+ let to_mask_str = |x| -> _ {
|
||||
+ if x { "Masked" } else { "Unmasked" }
|
||||
+ };
|
||||
+
|
||||
+ writeln!(f, " Exception handling state:")?;
|
||||
+ writeln!(f, " Debug (D): {}", to_mask_str(self.0.is_set(SPSR_EL1::D)))?;
|
||||
+ writeln!(f, " SError (A): {}", to_mask_str(self.0.is_set(SPSR_EL1::A)))?;
|
||||
+ writeln!(f, " IRQ (I): {}", to_mask_str(self.0.is_set(SPSR_EL1::I)))?;
|
||||
+ writeln!(f, " FIQ (F): {}", to_mask_str(self.0.is_set(SPSR_EL1::F)))?;
|
||||
+
|
||||
+ write!(f, " Illegal Execution State (IL): {}",
|
||||
+ to_flag_str(self.0.is_set(SPSR_EL1::IL))
|
||||
+ )?;
|
||||
+
|
||||
+ Ok(())
|
||||
+ }
|
||||
+}
|
||||
+
|
||||
+/// Human readable print of the exception context.
|
||||
+impl fmt::Display for ExceptionContext {
|
||||
+ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
|
||||
+ writeln!(f, "ELR_EL1: {:#018x}", self.elr_el1)?;
|
||||
+ writeln!(f, "{}", self.spsr_el1)?;
|
||||
+ writeln!(f)?;
|
||||
+ writeln!(f, "General purpose register:")?;
|
||||
+
|
||||
+ #[rustfmt::skip]
|
||||
+ let alternating = |x| -> _ {
|
||||
+ if x modulo 2 == 0 { " " } else { "\n" }
|
||||
+ };
|
||||
+
|
||||
+ // Print two registers per line.
|
||||
+ for (i, reg) in self.gpr.iter().enumerate() {
|
||||
+ write!(f, " x{: <2}: {: >#018x}{}", i, reg, alternating(i))?;
|
||||
+ }
|
||||
+ write!(f, " lr : {:#018x}", self.lr)?;
|
||||
+
|
||||
+ Ok(())
|
||||
+ }
|
||||
+}
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Arch-public
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
+/// Set the exception vector base address register.
|
||||
+///
|
||||
+/// # Safety
|
||||
+///
|
||||
+/// - The vector table and the symbol `__exception_vector_table_start` from the linker script must
|
||||
+/// adhere to the alignment and size constraints demanded by the AArch64 spec.
|
||||
+pub unsafe fn set_vbar_el1() {
|
||||
+ // Provided by exception.S.
|
||||
+ extern "C" {
|
||||
+ static mut __exception_vector_start: u64;
|
||||
+ }
|
||||
+ let addr: u64 = &__exception_vector_start as *const _ as u64;
|
||||
+
|
||||
+ VBAR_EL1.set(addr);
|
||||
+
|
||||
+ // Force VBAR update to complete before next instruction.
|
||||
+ barrier::isb(barrier::SY);
|
||||
+}
|
||||
+
|
||||
pub trait DaifField {
|
||||
fn daif_field() -> register::Field<u32, DAIF::Register>;
|
||||
}
|
||||
|
||||
diff -uNr 11_virtual_memory/src/arch/aarch64/exception.S 12_cpu_exceptions_part1/src/arch/aarch64/exception.S
|
||||
--- 11_virtual_memory/src/arch/aarch64/exception.S
|
||||
+++ 12_cpu_exceptions_part1/src/arch/aarch64/exception.S
|
||||
@@ -0,0 +1,133 @@
|
||||
+// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
+//
|
||||
+// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
+
|
||||
+/// Call the function provided by parameter `\handler` after saving exception context. Provide the
|
||||
+/// context as the first parameter to '\handler'.
|
||||
+.macro CALL_WITH_CONTEXT handler
|
||||
+ // Make room on the stack for the exception context.
|
||||
+ sub sp, sp, #16 * 17
|
||||
+
|
||||
+ // Store all general purpose registers on the stack.
|
||||
+ stp x0, x1, [sp, #16 * 0]
|
||||
+ stp x2, x3, [sp, #16 * 1]
|
||||
+ stp x4, x5, [sp, #16 * 2]
|
||||
+ stp x6, x7, [sp, #16 * 3]
|
||||
+ stp x8, x9, [sp, #16 * 4]
|
||||
+ stp x10, x11, [sp, #16 * 5]
|
||||
+ stp x12, x13, [sp, #16 * 6]
|
||||
+ stp x14, x15, [sp, #16 * 7]
|
||||
+ stp x16, x17, [sp, #16 * 8]
|
||||
+ stp x18, x19, [sp, #16 * 9]
|
||||
+ stp x20, x21, [sp, #16 * 10]
|
||||
+ stp x22, x23, [sp, #16 * 11]
|
||||
+ stp x24, x25, [sp, #16 * 12]
|
||||
+ stp x26, x27, [sp, #16 * 13]
|
||||
+ stp x28, x29, [sp, #16 * 14]
|
||||
+
|
||||
+ // Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1).
|
||||
+ mrs x1, ELR_EL1
|
||||
+ mrs x2, SPSR_EL1
|
||||
+
|
||||
+ stp lr, x1, [sp, #16 * 15]
|
||||
+ str w2, [sp, #16 * 16]
|
||||
+
|
||||
+ // x0 is the first argument for the function called through `\handler`.
|
||||
+ mov x0, sp
|
||||
+
|
||||
+ // Call `\handler`.
|
||||
+ bl \handler
|
||||
+
|
||||
+ // After returning from exception handling code, replay the saved context and return via `eret`.
|
||||
+ b __exception_restore_context
|
||||
+.endm
|
||||
+
|
||||
+.macro FIQ_SUSPEND
|
||||
+1: wfe
|
||||
+ b 1b
|
||||
+.endm
|
||||
+
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+// The exception vector table.
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+.section .exception_vectors, "ax", @progbits
|
||||
+
|
||||
+// Align by 2^11 bytes, as demanded by the AArch64 spec. Same as ALIGN(2048) in an ld script.
|
||||
+.align 11
|
||||
+
|
||||
+// Export a symbol for the Rust code to use.
|
||||
+__exception_vector_start:
|
||||
+
|
||||
+// Current exception level with SP_EL0.
|
||||
+// .org sets the offset relative to section start.
|
||||
+//
|
||||
+// It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes.
|
||||
+.org 0x000
|
||||
+ CALL_WITH_CONTEXT current_el0_synchronous
|
||||
+.org 0x080
|
||||
+ CALL_WITH_CONTEXT current_el0_irq
|
||||
+.org 0x100
|
||||
+ FIQ_SUSPEND
|
||||
+.org 0x180
|
||||
+ CALL_WITH_CONTEXT current_el0_serror
|
||||
+
|
||||
+// Current exception level with SP_ELx, x > 0.
|
||||
+.org 0x200
|
||||
+ CALL_WITH_CONTEXT current_elx_synchronous
|
||||
+.org 0x280
|
||||
+ CALL_WITH_CONTEXT current_elx_irq
|
||||
+.org 0x300
|
||||
+ FIQ_SUSPEND
|
||||
+.org 0x380
|
||||
+ CALL_WITH_CONTEXT current_elx_serror
|
||||
+
|
||||
+// Lower exception level, aarch64
|
||||
+.org 0x400
|
||||
+ CALL_WITH_CONTEXT lower_aarch64_synchronous
|
||||
+.org 0x480
|
||||
+ CALL_WITH_CONTEXT lower_aarch64_irq
|
||||
+.org 0x500
|
||||
+ FIQ_SUSPEND
|
||||
+.org 0x580
|
||||
+ CALL_WITH_CONTEXT lower_aarch64_serror
|
||||
+
|
||||
+// Lower exception level, aarch32
|
||||
+.org 0x600
|
||||
+ CALL_WITH_CONTEXT lower_aarch32_synchronous
|
||||
+.org 0x680
|
||||
+ CALL_WITH_CONTEXT lower_aarch32_irq
|
||||
+.org 0x700
|
||||
+ FIQ_SUSPEND
|
||||
+.org 0x780
|
||||
+ CALL_WITH_CONTEXT lower_aarch32_serror
|
||||
+.org 0x800
|
||||
+
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+// Helper functions
|
||||
+//--------------------------------------------------------------------------------------------------
|
||||
+__exception_restore_context:
|
||||
+ ldr w19, [sp, #16 * 16]
|
||||
+ ldp lr, x20, [sp, #16 * 15]
|
||||
+
|
||||
+ msr SPSR_EL1, x19
|
||||
+ msr ELR_EL1, x20
|
||||
+
|
||||
+ ldp x0, x1, [sp, #16 * 0]
|
||||
+ ldp x2, x3, [sp, #16 * 1]
|
||||
+ ldp x4, x5, [sp, #16 * 2]
|
||||
+ ldp x6, x7, [sp, #16 * 3]
|
||||
+ ldp x8, x9, [sp, #16 * 4]
|
||||
+ ldp x10, x11, [sp, #16 * 5]
|
||||
+ ldp x12, x13, [sp, #16 * 6]
|
||||
+ ldp x14, x15, [sp, #16 * 7]
|
||||
+ ldp x16, x17, [sp, #16 * 8]
|
||||
+ ldp x18, x19, [sp, #16 * 9]
|
||||
+ ldp x20, x21, [sp, #16 * 10]
|
||||
+ ldp x22, x23, [sp, #16 * 11]
|
||||
+ ldp x24, x25, [sp, #16 * 12]
|
||||
+ ldp x26, x27, [sp, #16 * 13]
|
||||
+ ldp x28, x29, [sp, #16 * 14]
|
||||
+
|
||||
+ add sp, sp, #16 * 17
|
||||
+
|
||||
+ eret
|
||||
|
||||
diff -uNr 11_virtual_memory/src/arch/aarch64.rs 12_cpu_exceptions_part1/src/arch/aarch64.rs
|
||||
--- 11_virtual_memory/src/arch/aarch64.rs
|
||||
+++ 12_cpu_exceptions_part1/src/arch/aarch64.rs
|
||||
@@ -106,6 +106,15 @@
|
||||
}
|
||||
}
|
||||
|
||||
+/// Enable exception handling.
|
||||
+///
|
||||
+/// # Safety
|
||||
+///
|
||||
+/// - Changes the HW state of the processing element.
|
||||
+pub unsafe fn enable_exception_handling() {
|
||||
+ exception::set_vbar_el1();
|
||||
+}
|
||||
+
|
||||
/// Return a reference to an `interface::mm::MMU` implementation.
|
||||
pub fn mmu() -> &'static impl interface::mm::MMU {
|
||||
&MMU
|
||||
|
||||
diff -uNr 11_virtual_memory/src/bsp/rpi/virt_mem_layout.rs 12_cpu_exceptions_part1/src/bsp/rpi/virt_mem_layout.rs
|
||||
--- 11_virtual_memory/src/bsp/rpi/virt_mem_layout.rs
|
||||
+++ 12_cpu_exceptions_part1/src/bsp/rpi/virt_mem_layout.rs
|
||||
@@ -15,7 +15,7 @@
|
||||
// BSP-public
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
-pub const NUM_MEM_RANGES: usize = 3;
|
||||
+pub const NUM_MEM_RANGES: usize = 2;
|
||||
|
||||
pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout::new(
|
||||
memory_map::END_INCLUSIVE,
|
||||
@@ -54,19 +54,6 @@
|
||||
},
|
||||
},
|
||||
RangeDescriptor {
|
||||
- name: "Remapped Device MMIO",
|
||||
- virtual_range: || {
|
||||
- // The last 64 KiB slot in the first 512 MiB
|
||||
- RangeInclusive::new(0x1FFF_0000, 0x1FFF_FFFF)
|
||||
- },
|
||||
- translation: Translation::Offset(memory_map::mmio::BASE + 0x20_0000),
|
||||
- attribute_fields: AttributeFields {
|
||||
- mem_attributes: MemAttributes::Device,
|
||||
- acc_perms: AccessPermissions::ReadWrite,
|
||||
- execute_never: true,
|
||||
- },
|
||||
- },
|
||||
- RangeDescriptor {
|
||||
name: "Device MMIO",
|
||||
virtual_range: || {
|
||||
RangeInclusive::new(memory_map::mmio::BASE, memory_map::mmio::END_INCLUSIVE)
|
||||
|
||||
diff -uNr 11_virtual_memory/src/bsp.rs 12_cpu_exceptions_part1/src/bsp.rs
|
||||
--- 11_virtual_memory/src/bsp.rs
|
||||
+++ 12_cpu_exceptions_part1/src/bsp.rs
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
//! Conditional exporting of Board Support Packages.
|
||||
|
||||
-pub mod driver;
|
||||
+mod driver;
|
||||
|
||||
#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))]
|
||||
mod rpi;
|
||||
|
||||
diff -uNr 11_virtual_memory/src/main.rs 12_cpu_exceptions_part1/src/main.rs
|
||||
--- 11_virtual_memory/src/main.rs
|
||||
+++ 12_cpu_exceptions_part1/src/main.rs
|
||||
@@ -22,6 +22,7 @@
|
||||
#![allow(incomplete_features)]
|
||||
#![feature(const_generics)]
|
||||
#![feature(format_args_nl)]
|
||||
+#![feature(global_asm)]
|
||||
#![feature(panic_info_message)]
|
||||
#![feature(trait_alias)]
|
||||
#![no_main]
|
||||
@@ -57,6 +58,8 @@
|
||||
unsafe fn kernel_init() -> ! {
|
||||
use interface::mm::MMU;
|
||||
|
||||
+ arch::enable_exception_handling();
|
||||
+
|
||||
if let Err(string) = arch::mmu().init() {
|
||||
panic!("MMU: {}", string);
|
||||
}
|
||||
@@ -102,13 +105,28 @@
|
||||
info!("Timer test, spinning for 1 second");
|
||||
arch::timer().spin_for(Duration::from_secs(1));
|
||||
|
||||
- let remapped_uart = unsafe { bsp::driver::PL011Uart::new(0x1FFF_1000) };
|
||||
- writeln!(
|
||||
- remapped_uart,
|
||||
- "[ !!! ] Writing through the remapped UART at 0x1FFF_1000"
|
||||
- )
|
||||
- .unwrap();
|
||||
+ // Cause an exception by accessing a virtual address for which no translation was set up. This
|
||||
+ // code accesses the address 8 GiB, which is outside the mapped address space.
|
||||
+ //
|
||||
+ // For demo purposes, the exception handler will catch the faulting 8 GiB address and allow
|
||||
+ // execution to continue.
|
||||
+ info!("");
|
||||
+ info!("Trying to write to address 8 GiB...");
|
||||
+ let mut big_addr: u64 = 8 * 1024 * 1024 * 1024;
|
||||
+ unsafe { core::ptr::read_volatile(big_addr as *mut u64) };
|
||||
+
|
||||
+ info!("************************************************");
|
||||
+ info!("Whoa! We recovered from a synchronous exception!");
|
||||
+ info!("************************************************");
|
||||
+ info!("");
|
||||
+ info!("Let's try again");
|
||||
+
|
||||
+ // Now use address 9 GiB. The exception handler won't forgive us this time.
|
||||
+ info!("Trying to write to address 9 GiB...");
|
||||
+ big_addr = 9 * 1024 * 1024 * 1024;
|
||||
+ unsafe { core::ptr::read_volatile(big_addr as *mut u64) };
|
||||
|
||||
+ // Will never reach here in this tutorial.
|
||||
info!("Echoing input now");
|
||||
loop {
|
||||
let c = bsp::console().read_char();
|
||||
|
||||
```
|
@ -1,21 +0,0 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Conditional exporting of processor architecture code.
|
||||
|
||||
#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))]
|
||||
mod aarch64;
|
||||
|
||||
#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))]
|
||||
pub use aarch64::*;
|
||||
|
||||
/// Architectural privilege level.
|
||||
#[allow(missing_docs)]
|
||||
#[derive(PartialEq)]
|
||||
pub enum PrivilegeLevel {
|
||||
User,
|
||||
Kernel,
|
||||
Hypervisor,
|
||||
Unknown,
|
||||
}
|
@ -1,53 +0,0 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Synchronization primitives.
|
||||
|
||||
use crate::interface;
|
||||
use core::cell::UnsafeCell;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Arch-public
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// A pseudo-lock for teaching purposes.
|
||||
///
|
||||
/// Used to introduce [interior mutability].
|
||||
///
|
||||
/// In contrast to a real Mutex implementation, does not protect against concurrent access to the
|
||||
/// contained data. This part is preserved for later lessons.
|
||||
///
|
||||
/// The lock will only be used as long as it is safe to do so, i.e. as long as the kernel is
|
||||
/// executing single-threaded, aka only running on a single core with interrupts disabled.
|
||||
///
|
||||
/// [interior mutability]: https://doc.rust-lang.org/std/cell/index.html
|
||||
pub struct NullLock<T: ?Sized> {
|
||||
data: UnsafeCell<T>,
|
||||
}
|
||||
|
||||
unsafe impl<T: ?Sized + Send> Send for NullLock<T> {}
|
||||
unsafe impl<T: ?Sized + Send> Sync for NullLock<T> {}
|
||||
|
||||
impl<T> NullLock<T> {
|
||||
/// Wraps `data` into a new `NullLock`.
|
||||
pub const fn new(data: T) -> NullLock<T> {
|
||||
NullLock {
|
||||
data: UnsafeCell::new(data),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// OS interface implementations
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
impl<T> interface::sync::Mutex for &NullLock<T> {
|
||||
type Data = T;
|
||||
|
||||
fn lock<R>(&mut self, f: impl FnOnce(&mut Self::Data) -> R) -> R {
|
||||
// In a real lock, there would be code encapsulating this line that ensures that this
|
||||
// mutable reference will ever only be given out once at a time.
|
||||
f(unsafe { &mut *self.data.get() })
|
||||
}
|
||||
}
|
@ -1,85 +0,0 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Board Support Package for the Raspberry Pi.
|
||||
|
||||
mod memory_map;
|
||||
mod virt_mem_layout;
|
||||
|
||||
use super::driver;
|
||||
use crate::{interface, memory::KernelVirtualLayout};
|
||||
use core::fmt;
|
||||
|
||||
/// Used by `arch` code to find the early boot core.
|
||||
pub const BOOT_CORE_ID: u64 = 0;
|
||||
|
||||
/// The early boot core's stack address.
|
||||
pub const BOOT_CORE_STACK_START: u64 = 0x80_000;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Global BSP driver instances
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
static GPIO: driver::GPIO = unsafe { driver::GPIO::new(memory_map::mmio::GPIO_BASE) };
|
||||
static PL011_UART: driver::PL011Uart =
|
||||
unsafe { driver::PL011Uart::new(memory_map::mmio::PL011_UART_BASE) };
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Implementation of the kernel's BSP calls
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Board identification.
|
||||
pub fn board_name() -> &'static str {
|
||||
#[cfg(feature = "bsp_rpi3")]
|
||||
{
|
||||
"Raspberry Pi 3"
|
||||
}
|
||||
|
||||
#[cfg(feature = "bsp_rpi4")]
|
||||
{
|
||||
"Raspberry Pi 4"
|
||||
}
|
||||
}
|
||||
|
||||
/// Return a reference to a `console::All` implementation.
|
||||
pub fn console() -> &'static impl interface::console::All {
|
||||
&PL011_UART
|
||||
}
|
||||
|
||||
/// In case of a panic, the panic handler uses this function to take a last shot at printing
|
||||
/// something before the system is halted.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// - Use only for printing during a panic.
|
||||
pub unsafe fn panic_console_out() -> impl fmt::Write {
|
||||
let uart = driver::PanicUart::new(memory_map::mmio::PL011_UART_BASE);
|
||||
uart.init();
|
||||
uart
|
||||
}
|
||||
|
||||
/// Return an array of references to all `DeviceDriver` compatible `BSP` drivers.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// The order of devices is the order in which `DeviceDriver::init()` is called.
|
||||
pub fn device_drivers() -> [&'static dyn interface::driver::DeviceDriver; 2] {
|
||||
[&GPIO, &PL011_UART]
|
||||
}
|
||||
|
||||
/// BSP initialization code that runs after driver init.
|
||||
pub fn post_driver_init() {
|
||||
// Configure PL011Uart's output pins.
|
||||
GPIO.map_pl011_uart();
|
||||
}
|
||||
|
||||
/// Return the address space size in bytes.
|
||||
pub const fn addr_space_size() -> usize {
|
||||
memory_map::END_INCLUSIVE + 1
|
||||
}
|
||||
|
||||
/// Return a reference to the virtual memory layout.
|
||||
pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ virt_mem_layout::NUM_MEM_RANGES }> {
|
||||
&virt_mem_layout::LAYOUT
|
||||
}
|
@ -1,27 +0,0 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! The board's memory map.
|
||||
|
||||
#[cfg(feature = "bsp_rpi3")]
|
||||
#[rustfmt::skip]
|
||||
pub const END_INCLUSIVE: usize = 0x3FFF_FFFF;
|
||||
|
||||
#[cfg(feature = "bsp_rpi4")]
|
||||
#[rustfmt::skip]
|
||||
pub const END_INCLUSIVE: usize = 0xFFFF_FFFF;
|
||||
|
||||
/// Physical devices.
|
||||
#[rustfmt::skip]
|
||||
pub mod mmio {
|
||||
#[cfg(feature = "bsp_rpi3")]
|
||||
pub const BASE: usize = 0x3F00_0000;
|
||||
|
||||
#[cfg(feature = "bsp_rpi4")]
|
||||
pub const BASE: usize = 0xFE00_0000;
|
||||
|
||||
pub const GPIO_BASE: usize = BASE + 0x0020_0000;
|
||||
pub const PL011_UART_BASE: usize = BASE + 0x0020_1000;
|
||||
pub const END_INCLUSIVE: usize = super::END_INCLUSIVE;
|
||||
}
|
@ -1,147 +0,0 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Trait definitions for coupling `kernel` and `BSP` code.
|
||||
//!
|
||||
//! ```
|
||||
//! +-------------------+
|
||||
//! | Interface (Trait) |
|
||||
//! | |
|
||||
//! +--+-------------+--+
|
||||
//! ^ ^
|
||||
//! | |
|
||||
//! | |
|
||||
//! +----------+--+ +--+----------+
|
||||
//! | Kernel code | | BSP Code |
|
||||
//! | | | |
|
||||
//! +-------------+ +-------------+
|
||||
//! ```
|
||||
|
||||
/// System console operations.
|
||||
pub mod console {
|
||||
use core::fmt;
|
||||
|
||||
/// Console write functions.
|
||||
pub trait Write {
|
||||
/// Write a single character.
|
||||
fn write_char(&self, c: char);
|
||||
|
||||
/// Write a Rust format string.
|
||||
fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result;
|
||||
|
||||
/// Block execution until the last character has been physically put on the TX wire
|
||||
/// (draining TX buffers/FIFOs, if any).
|
||||
fn flush(&self);
|
||||
}
|
||||
|
||||
/// Console read functions.
|
||||
pub trait Read {
|
||||
/// Read a single character.
|
||||
fn read_char(&self) -> char {
|
||||
' '
|
||||
}
|
||||
|
||||
/// Clear RX buffers, if any.
|
||||
fn clear(&self);
|
||||
}
|
||||
|
||||
/// Console statistics.
|
||||
pub trait Statistics {
|
||||
/// Return the number of characters written.
|
||||
fn chars_written(&self) -> usize {
|
||||
0
|
||||
}
|
||||
|
||||
/// Return the number of characters read.
|
||||
fn chars_read(&self) -> usize {
|
||||
0
|
||||
}
|
||||
}
|
||||
|
||||
/// Trait alias for a full-fledged console.
|
||||
pub trait All = Write + Read + Statistics;
|
||||
}
|
||||
|
||||
/// Synchronization primitives.
|
||||
pub mod sync {
|
||||
/// Any object implementing this trait guarantees exclusive access to the data contained within
|
||||
/// the mutex for the duration of the lock.
|
||||
///
|
||||
/// The trait follows the [Rust embedded WG's
|
||||
/// proposal](https://github.com/korken89/wg/blob/master/rfcs/0377-mutex-trait.md) and therefore
|
||||
/// provides some goodness such as [deadlock
|
||||
/// prevention](https://github.com/korken89/wg/blob/master/rfcs/0377-mutex-trait.md#design-decisions-and-compatibility).
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// Since the lock function takes an `&mut self` to enable deadlock-prevention, the trait is
|
||||
/// best implemented **for a reference to a container struct**, and has a usage pattern that
|
||||
/// might feel strange at first:
|
||||
///
|
||||
/// ```
|
||||
/// static MUT: Mutex<RefCell<i32>> = Mutex::new(RefCell::new(0));
|
||||
///
|
||||
/// fn foo() {
|
||||
/// let mut r = &MUT; // Note that r is mutable
|
||||
/// r.lock(|data| *data += 1);
|
||||
/// }
|
||||
/// ```
|
||||
pub trait Mutex {
|
||||
/// Type of data encapsulated by the mutex.
|
||||
type Data;
|
||||
|
||||
/// Creates a critical section and grants temporary mutable access to the encapsulated data.
|
||||
fn lock<R>(&mut self, f: impl FnOnce(&mut Self::Data) -> R) -> R;
|
||||
}
|
||||
}
|
||||
|
||||
/// Driver interfaces.
|
||||
pub mod driver {
|
||||
/// Driver result type, e.g. for indicating successful driver init.
|
||||
pub type Result = core::result::Result<(), ()>;
|
||||
|
||||
/// Device Driver functions.
|
||||
pub trait DeviceDriver {
|
||||
/// Return a compatibility string for identifying the driver.
|
||||
fn compatible(&self) -> &str;
|
||||
|
||||
/// Called by the kernel to bring up the device.
|
||||
fn init(&self) -> Result {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Timekeeping interfaces.
|
||||
pub mod time {
|
||||
use core::time::Duration;
|
||||
|
||||
/// Timer functions.
|
||||
pub trait Timer {
|
||||
/// The timer's resolution.
|
||||
fn resolution(&self) -> Duration;
|
||||
|
||||
/// The uptime since power-on of the device.
|
||||
///
|
||||
/// This includes time consumed by firmware and bootloaders.
|
||||
fn uptime(&self) -> Duration;
|
||||
|
||||
/// Spin for a given duration.
|
||||
fn spin_for(&self, duration: Duration);
|
||||
}
|
||||
}
|
||||
|
||||
/// Memory Management interfaces.
|
||||
pub mod mm {
|
||||
/// MMU functions.
|
||||
pub trait MMU {
|
||||
/// Called by the kernel during early init. Supposed to take the page tables from the
|
||||
/// `BSP`-supplied `virt_mem_layout()` and install/activate them for the respective MMU.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// - Changes the HW's global state.
|
||||
unsafe fn init(&self) -> Result<(), &'static str>;
|
||||
}
|
||||
}
|
@ -1,135 +0,0 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
// Rust embedded logo for `make doc`.
|
||||
#![doc(html_logo_url = "https://git.io/JeGIp")]
|
||||
|
||||
//! The `kernel`
|
||||
//!
|
||||
//! The `kernel` is composed by glueing together code from
|
||||
//!
|
||||
//! - [Hardware-specific Board Support Packages] (`BSPs`).
|
||||
//! - [Architecture-specific code].
|
||||
//! - HW- and architecture-agnostic `kernel` code.
|
||||
//!
|
||||
//! using the [`kernel::interface`] traits.
|
||||
//!
|
||||
//! [Hardware-specific Board Support Packages]: bsp/index.html
|
||||
//! [Architecture-specific code]: arch/index.html
|
||||
//! [`kernel::interface`]: interface/index.html
|
||||
|
||||
#![allow(incomplete_features)]
|
||||
#![feature(const_generics)]
|
||||
#![feature(format_args_nl)]
|
||||
#![feature(global_asm)]
|
||||
#![feature(panic_info_message)]
|
||||
#![feature(trait_alias)]
|
||||
#![no_main]
|
||||
#![no_std]
|
||||
|
||||
// Conditionally includes the selected `architecture` code, which provides the `_start()` function,
|
||||
// the first function to run.
|
||||
mod arch;
|
||||
|
||||
// `_start()` then calls `runtime_init()`, which on completion, jumps to `kernel_init()`.
|
||||
mod runtime_init;
|
||||
|
||||
// Conditionally includes the selected `BSP` code.
|
||||
mod bsp;
|
||||
|
||||
mod interface;
|
||||
mod memory;
|
||||
mod panic_wait;
|
||||
mod print;
|
||||
|
||||
/// Early init code.
|
||||
///
|
||||
/// Concerned with with initializing `BSP` and `arch` parts.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// - Only a single core must be active and running this function.
|
||||
/// - The init calls in this function must appear in the correct order:
|
||||
/// - Virtual memory must be activated before the device drivers.
|
||||
/// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device
|
||||
/// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on
|
||||
/// the RPi SoCs.
|
||||
unsafe fn kernel_init() -> ! {
|
||||
use interface::mm::MMU;
|
||||
|
||||
arch::enable_exception_handling();
|
||||
|
||||
if let Err(string) = arch::mmu().init() {
|
||||
panic!("MMU: {}", string);
|
||||
}
|
||||
|
||||
for i in bsp::device_drivers().iter() {
|
||||
if let Err(()) = i.init() {
|
||||
panic!("Error loading driver: {}", i.compatible())
|
||||
}
|
||||
}
|
||||
bsp::post_driver_init();
|
||||
// println! is usable from here on.
|
||||
|
||||
// Transition from unsafe to safe.
|
||||
kernel_main()
|
||||
}
|
||||
|
||||
/// The main function running after the early init.
|
||||
fn kernel_main() -> ! {
|
||||
use core::time::Duration;
|
||||
use interface::{console::All, time::Timer};
|
||||
|
||||
info!("Booting on: {}", bsp::board_name());
|
||||
|
||||
info!("MMU online. Special regions:");
|
||||
bsp::virt_mem_layout().print_layout();
|
||||
|
||||
let (_, privilege_level) = arch::state::current_privilege_level();
|
||||
info!("Current privilege level: {}", privilege_level);
|
||||
|
||||
info!("Exception handling state:");
|
||||
arch::state::print_exception_state();
|
||||
|
||||
info!(
|
||||
"Architectural timer resolution: {} ns",
|
||||
arch::timer().resolution().as_nanos()
|
||||
);
|
||||
|
||||
info!("Drivers loaded:");
|
||||
for (i, driver) in bsp::device_drivers().iter().enumerate() {
|
||||
info!(" {}. {}", i + 1, driver.compatible());
|
||||
}
|
||||
|
||||
info!("Timer test, spinning for 1 second");
|
||||
arch::timer().spin_for(Duration::from_secs(1));
|
||||
|
||||
// Cause an exception by accessing a virtual address for which no translation was set up. This
|
||||
// code accesses the address 8 GiB, which is outside the mapped address space.
|
||||
//
|
||||
// For demo purposes, the exception handler will catch the faulting 8 GiB address and allow
|
||||
// execution to continue.
|
||||
info!("");
|
||||
info!("Trying to write to address 8 GiB...");
|
||||
let mut big_addr: u64 = 8 * 1024 * 1024 * 1024;
|
||||
unsafe { core::ptr::read_volatile(big_addr as *mut u64) };
|
||||
|
||||
info!("************************************************");
|
||||
info!("Whoa! We recovered from a synchronous exception!");
|
||||
info!("************************************************");
|
||||
info!("");
|
||||
info!("Let's try again");
|
||||
|
||||
// Now use address 9 GiB. The exception handler won't forgive us this time.
|
||||
info!("Trying to write to address 9 GiB...");
|
||||
big_addr = 9 * 1024 * 1024 * 1024;
|
||||
unsafe { core::ptr::read_volatile(big_addr as *mut u64) };
|
||||
|
||||
// Will never reach here in this tutorial.
|
||||
info!("Echoing input now");
|
||||
loop {
|
||||
let c = bsp::console().read_char();
|
||||
bsp::console().write_char(c);
|
||||
}
|
||||
}
|
@ -0,0 +1,479 @@
|
||||
# Tutorial 12 - Exceptions Part 1: Groundwork
|
||||
|
||||
## tl;dr
|
||||
|
||||
We lay the groundwork for all the architectural `CPU exceptions`. For now, only print an elaborate
|
||||
system state through a `panic!` call, and halt execution; This will help finding bugs during
|
||||
development and runtime.
|
||||
|
||||
For demo purposes, MMU `page faults` are used to demonstrate (i) returning from an exception and
|
||||
(ii) the default `panic!` behavior.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Exception Types](#exception-types)
|
||||
- [Exception entry](#exception-entry)
|
||||
* [Exception Vectors](#exception-vectors)
|
||||
- [Handler Code and Offsets](#handler-code-and-offsets)
|
||||
- [Rust and Assembly Implementation](#rust-and-assembly-implementation)
|
||||
* [Context Save and Restore](#context-save-and-restore)
|
||||
* [Exception Vector Table](#exception-vector-table)
|
||||
* [Implementing handlers](#implementing-handlers)
|
||||
- [Causing an Exception - Testing the Code](#causing-an-exception---testing-the-code)
|
||||
- [Test it](#test-it)
|
||||
- [Diff to previous](#diff-to-previous)
|
||||
|
||||
## Introduction
|
||||
|
||||
Now that we are executing in `EL1`, and have activated the `MMU`, time is due for implementing `CPU
|
||||
exceptions`. For now, we only set up a scaffold with very basic functionality that will help us to
|
||||
find bugs along the way. A follow-up `Interrupt` tutorial in the future will continue the work we
|
||||
start here.
|
||||
|
||||
Please note that this tutorial is specific to the `AArch64` architecture. It does not contain any
|
||||
generic exception handling code yet.
|
||||
|
||||
## Exception Types
|
||||
|
||||
In `AArch64`, it is differentiated between four types of exceptions. These are:
|
||||
- Synchronous
|
||||
- For example, a `data abort` (e.g. `page fault`) or a `system call`. They happen in direct
|
||||
consequence of executing a certain instruction, hence _synchronously_.
|
||||
- Interrupt Request (`IRQ`)
|
||||
- For example, an external device, like a timer, is asserting a physical interrupt line. IRQs
|
||||
happen _asynchronously_.
|
||||
- Fast Interrupt Request (`FIQ`)
|
||||
- These are basically interrupts that take priority over normal IRQs and have some more traits
|
||||
that make them suitable to implement super-fast processing. However, this is out of scope for
|
||||
this tutorial. For the sake of keeping these tutorials compact and concise, we will more or less
|
||||
ignore FIQs and only implement a dummy handler that would halt the CPU core.
|
||||
- System Error (`SError`)
|
||||
- Like IRQs, SErrors happen asynchronously and are technically more or less the same. They are
|
||||
intended to signal rather fatal errors in the system, e.g. if a transaction times out on the
|
||||
`SoC` interconnect. They are very implementation specific and it is up to the SoC vendor to
|
||||
decide which events are delivered as SErrors instead of normal IRQs.
|
||||
|
||||
## Exception entry
|
||||
|
||||
I recommend to read pages 1874-1876 of the [ARMv8 Architecture Reference Manual][ARMv8_Manual] to
|
||||
understand the mechanisms of taking an exception.
|
||||
|
||||
Here's an excerpt of important features for this tutorial:
|
||||
- Exception entry moves the processor to the same or a higher `Exception Level`, but never to a
|
||||
lower `EL`.
|
||||
- The program status is saved in the `SPSR_ELx` register at the target `EL`.
|
||||
- The preferred return address is saved in the `ELR_ELx` register.
|
||||
- "Preferred" here means that `ELR_ELx` may hold the instruction address of the instructions that
|
||||
caused the exception (`synchronous case`) or the first instruction that did not complete due to
|
||||
an `asynchronous` exception. Details in Chapter D1.10.1 of the [ARMv8 Architecture Reference
|
||||
Manual][ARMv8_Manual].
|
||||
- All kinds of exceptions are turned off upon taking an exception, so that by default, exception
|
||||
handlers can not get interrupted themselves.
|
||||
- Taking an exception will select the dedicated stack pointer of the target `EL`.
|
||||
- For example, if an exception in `EL0` is taken, the Stack Pointer Select register `SPSel` will
|
||||
switch from `0` to `1`, meaning that `SP_EL1` will be used by the exception vector code unless
|
||||
you explicitly change it back to `SP_EL0`.
|
||||
|
||||
|
||||
### Exception Vectors
|
||||
|
||||
`AArch64` has a total of `16` exception vectors. There is one for each of the four kinds that were
|
||||
introduced already, and additionally, it is taken into account _where_ the exception was taken from
|
||||
and what the circumstances were.
|
||||
|
||||
Here is a copy of the decision table as shown in Chapter D1.10.2 of the [ARMv8 Architecture
|
||||
Reference Manual][ARMv8_Manual]:
|
||||
|
||||
[ARMv8_Manual]: https://developer.arm.com/docs/ddi0487/latest/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th rowspan=2>Exception taken from </th>
|
||||
<th colspan=4>Offset for exception type</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Synchronous</th>
|
||||
<th>IRQ or vIRQ</th>
|
||||
<th>FIQ or vFIQ</th>
|
||||
<th>SError or vSError</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td width="40%">Current Exception level with SP_EL0.</td>
|
||||
<td align="center">0x000</td>
|
||||
<td align="center">0x080</td>
|
||||
<td align="center">0x100</td>
|
||||
<td align="center">0x180</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Current Exception level with SP_ELx, x>0.</td>
|
||||
<td align="center">0x200</td>
|
||||
<td align="center">0x280</td>
|
||||
<td align="center">0x300</td>
|
||||
<td align="center">0x380</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Lower Exception level, where the implemented level immediately lower than the target level is using AArch64.</td>
|
||||
<td align="center">0x400</td>
|
||||
<td align="center">0x480</td>
|
||||
<td align="center">0x500</td>
|
||||
<td align="center">0x580</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Lower Exception level, where the implemented level immediately lower than the target level is using AArch32.</td>
|
||||
<td align="center">0x600</td>
|
||||
<td align="center">0x680</td>
|
||||
<td align="center">0x700</td>
|
||||
<td align="center">0x780</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
Since our kernel runs in `EL1`, using `SP_EL1`, if we'd cause a synchronous exception, the exception
|
||||
vector at offset `0x200` would be executed. But what does that even mean?
|
||||
|
||||
## Handler Code and Offsets
|
||||
|
||||
In many architectures, Operating Systems register their exception handlers (aka vectors) by
|
||||
compiling an architecturally defined data structure that stores function pointers to the different
|
||||
handlers. This can be as simple as an ordinary array of function pointers. The `base address` of
|
||||
this data structure is then stored into a special purpose register so that the CPU can branch to the
|
||||
respective handler function upon taking an exception. The classic `x86_64` architecture follows this
|
||||
principle, for example.
|
||||
|
||||
In `AArch64`, it is a bit different. Here, we have the special purpose register as well, called
|
||||
`VBAR_EL1`: Vector Base Address Register.
|
||||
|
||||
However, it does not store the base address of an array of function pointers, but the base address
|
||||
of a **memory location that contains code** for the 16 handlers, one handler back-to-back after the
|
||||
other. Each handler can take a maximum space of `0x80` bytes, aka `128` bytes. That's why the table
|
||||
above shows `offsets`: To indicate at which offset a certain handler starts.
|
||||
|
||||
Of course, you are not obliged to cram all your handler code into only 128 bytes. You are free to
|
||||
branch off to any other functions at any time. Actually, that is needed in most cases anyways,
|
||||
because the context-saving code alone would take up most of the available space (you'll learn what
|
||||
context saving is shortly).
|
||||
|
||||
Additionally, there is a requirement that the `Vector Base Address` is aligned to `0x800` aka `2048`
|
||||
bytes.
|
||||
|
||||
## Rust and Assembly Implementation
|
||||
|
||||
The implementation uses a mix of `Rust` and `Assembly` code.
|
||||
|
||||
### Context Save and Restore
|
||||
|
||||
Exception vectors, just like any other code, use a bunch of commonly shared processor resources.
|
||||
Most of all, the set of `General Purpose Registers` (GPRs) that each core in `AArch64` provides
|
||||
(`x0`-`x30`).
|
||||
|
||||
In order to not taint these registers when executing exception vector code, it is general practice
|
||||
to save these shared resources in memory (the stack, to be precise) as the very first action. This
|
||||
is commonly described as *saving the context*. Exception vector code can then use the shared
|
||||
resources in its own code without bothering, and as a last action before returning from exception
|
||||
handling code, restore the context, so that the processor can continue where it left off before
|
||||
taking the exception.
|
||||
|
||||
Context save and restore is one of the few places in system software where it is strongly advised to
|
||||
to use some hand-crafted assembly. Introducing `exception.S`:
|
||||
|
||||
```asm
|
||||
/// Call the function provided by parameter `\handler` after saving the exception context. Provide
|
||||
/// the context as the first parameter to '\handler'.
|
||||
.macro CALL_WITH_CONTEXT handler
|
||||
// Make room on the stack for the exception context.
|
||||
sub sp, sp, #16 * 17
|
||||
|
||||
// Store all general purpose registers on the stack.
|
||||
stp x0, x1, [sp, #16 * 0]
|
||||
stp x2, x3, [sp, #16 * 1]
|
||||
stp x4, x5, [sp, #16 * 2]
|
||||
stp x6, x7, [sp, #16 * 3]
|
||||
stp x8, x9, [sp, #16 * 4]
|
||||
stp x10, x11, [sp, #16 * 5]
|
||||
stp x12, x13, [sp, #16 * 6]
|
||||
stp x14, x15, [sp, #16 * 7]
|
||||
stp x16, x17, [sp, #16 * 8]
|
||||
stp x18, x19, [sp, #16 * 9]
|
||||
stp x20, x21, [sp, #16 * 10]
|
||||
stp x22, x23, [sp, #16 * 11]
|
||||
stp x24, x25, [sp, #16 * 12]
|
||||
stp x26, x27, [sp, #16 * 13]
|
||||
stp x28, x29, [sp, #16 * 14]
|
||||
|
||||
// Add the exception link register (ELR_EL1) and the saved program status (SPSR_EL1).
|
||||
mrs x1, ELR_EL1
|
||||
mrs x2, SPSR_EL1
|
||||
|
||||
stp lr, x1, [sp, #16 * 15]
|
||||
str w2, [sp, #16 * 16]
|
||||
|
||||
// x0 is the first argument for the function called through `\handler`.
|
||||
mov x0, sp
|
||||
|
||||
// Call `\handler`.
|
||||
bl \handler
|
||||
|
||||
// After returning from exception handling code, replay the saved context and return via `eret`.
|
||||
b __exception_restore_context
|
||||
.endm
|
||||
```
|
||||
|
||||
First, a macro for saving the context is defined. It eventually jumps to follow-up handler code, and
|
||||
finally restores the context. In advance, we reserve space on the stack for the context. That is,
|
||||
the 30 `GPRs`, the `link register`, the `saved program status` and the `exception link register`
|
||||
(holding the preferred return address). Afterwards, we store those registers, save the current stack
|
||||
address in `x0` and branch off to follow-up handler-code, whose function name is supplied as an
|
||||
argument to the macro (`\handler`).
|
||||
|
||||
The handler code will be written in Rust, but use the platform's `C` ABI. This way, we can define a
|
||||
function signature that has a pointer to the context-data on the stack as its first argument, and
|
||||
know that this argument is expected to be in the `x0` register. We need to use the `C` ABI here
|
||||
because `Rust` has no stable convention ([yet](https://github.com/rust-lang/rfcs/issues/600)).
|
||||
|
||||
### Exception Vector Table
|
||||
|
||||
Next, we craft the exception vector table:
|
||||
|
||||
```asm
|
||||
.section .exception_vectors, "ax", @progbits
|
||||
|
||||
// Align by 2^11 bytes, as demanded by ARMv8-A. Same as ALIGN(2048) in an ld script.
|
||||
.align 11
|
||||
|
||||
// Export a symbol for the Rust code to use.
|
||||
__exception_vector_start:
|
||||
|
||||
// Current exception level with SP_EL0.
|
||||
//
|
||||
// .org sets the offset relative to section start.
|
||||
//
|
||||
// # Safety
|
||||
//
|
||||
// - It must be ensured that `CALL_WITH_CONTEXT` <= 0x80 bytes.
|
||||
.org 0x000
|
||||
CALL_WITH_CONTEXT current_el0_synchronous
|
||||
.org 0x080
|
||||
CALL_WITH_CONTEXT current_el0_irq
|
||||
.org 0x100
|
||||
FIQ_SUSPEND
|
||||
.org 0x180
|
||||
CALL_WITH_CONTEXT current_el0_serror
|
||||
|
||||
// Current exception level with SP_ELx, x > 0.
|
||||
.org 0x200
|
||||
CALL_WITH_CONTEXT current_elx_synchronous
|
||||
.org 0x280
|
||||
CALL_WITH_CONTEXT current_elx_irq
|
||||
.org 0x300
|
||||
FIQ_SUSPEND
|
||||
.org 0x380
|
||||
CALL_WITH_CONTEXT current_elx_serror
|
||||
|
||||
[...]
|
||||
```
|
||||
|
||||
Note how each vector starts at the required offset from the section start using the `.org`
|
||||
directive. Each macro call introduces an explicit handler function name, which is implemented in
|
||||
`Rust` in `exception.rs`.
|
||||
|
||||
### Implementing handlers
|
||||
|
||||
The file `exception.rs` first defines a `struct` of the exception context that is stored on the
|
||||
stack by the assembly code:
|
||||
|
||||
```rust
|
||||
/// The exception context as it is stored on the stack on exception entry.
|
||||
#[repr(C)]
|
||||
struct ExceptionContext {
|
||||
/// General Purpose Registers.
|
||||
gpr: [u64; 30],
|
||||
|
||||
/// The link register, aka x30.
|
||||
lr: u64,
|
||||
|
||||
/// Exception link register. The program counter at the time the exception happened.
|
||||
elr_el1: u64,
|
||||
|
||||
/// Saved program status.
|
||||
spsr_el1: SpsrEL1,
|
||||
}
|
||||
```
|
||||
|
||||
The handlers take a `struct ExceptionContext` argument. Since we do not plan to implement handlers
|
||||
for each exception yet, a default handler is provided:
|
||||
|
||||
```rust
|
||||
/// Print verbose information about the exception and the panic.
|
||||
fn default_exception_handler(e: &ExceptionContext) {
|
||||
panic!(
|
||||
"\n\nCPU Exception!\n\
|
||||
FAR_EL1: {:#018x}\n\
|
||||
{}\n\
|
||||
{}",
|
||||
FAR_EL1.get(),
|
||||
EsrEL1 {},
|
||||
e
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
The actual handlers referenced from the assembly can now branch to it for the time being, e.g.:
|
||||
|
||||
```rust
|
||||
#[no_mangle]
|
||||
unsafe extern "C" fn current_el0_synchronous(e: &mut ExceptionContext) {
|
||||
default_exception_handler(e);
|
||||
}
|
||||
```
|
||||
|
||||
## Causing an Exception - Testing the Code
|
||||
|
||||
We want to see two cases in action:
|
||||
1. How taking, handling and returning from an exception works.
|
||||
2. How the `panic!` print for unhandled exceptions looks like.
|
||||
|
||||
So after setting up exceptions in `main.rs` by calling
|
||||
|
||||
```rust
|
||||
exception::handling_init();
|
||||
```
|
||||
|
||||
we cause a data abort exception by reading from memory address `8 GiB`:
|
||||
|
||||
```rust
|
||||
// Cause an exception by accessing a virtual address for which no translation was set up. This
|
||||
// code accesses the address 8 GiB, which is outside the mapped address space.
|
||||
//
|
||||
// For demo purposes, the exception handler will catch the faulting 8 GiB address and allow
|
||||
// execution to continue.
|
||||
info!("");
|
||||
info!("Trying to write to address 8 GiB...");
|
||||
let mut big_addr: u64 = 8 * 1024 * 1024 * 1024;
|
||||
unsafe { core::ptr::read_volatile(big_addr as *mut u64) };
|
||||
```
|
||||
|
||||
This triggers our exception code, because we try to read from a virtual address for which no mapping
|
||||
has been installed. Remember, we only installed identity-mapped page tables for the first `1 GiB`
|
||||
(RPi3) or `4 GiB` (RPi4) of address space in the previous tutorial.
|
||||
|
||||
To survive this exception, the respective handler has a special demo case:
|
||||
|
||||
```rust
|
||||
#[no_mangle]
|
||||
unsafe extern "C" fn current_elx_synchronous(e: &mut ExceptionContext) {
|
||||
let far_el1 = FAR_EL1.get();
|
||||
|
||||
// This catches the demo case for this tutorial. If the fault address happens to be 8 GiB,
|
||||
// advance the exception link register for one instruction, so that execution can continue.
|
||||
if far_el1 == 8 * 1024 * 1024 * 1024 {
|
||||
e.elr_el1 += 4;
|
||||
|
||||
asm::eret()
|
||||
}
|
||||
|
||||
default_exception_handler(e);
|
||||
}
|
||||
```
|
||||
|
||||
It checks if the faulting address equals `8 GiB`, and if so, advances the copy of the `ELR` by 4,
|
||||
which makes it point to the next instruction after the instruction that caused the exception. When
|
||||
this handler returns, execution continues in the assembly macro we introduced before. The macro has
|
||||
only one more line left: `b __exception_restore_context`, which jumps to an assembly function that
|
||||
plays back our saved context before finally executing `eret` to return from the exception.
|
||||
|
||||
This will kick us back into `main.rs`. But we also want to see the `panic!` print.
|
||||
|
||||
Therefore, a second read is done, this time from address `9 GiB`. A case which the handler will not
|
||||
catch, eventually triggering the `panic!` call from the default handler.
|
||||
|
||||
## Test it
|
||||
|
||||
Emphasis on the events at timestamps > `4.xxxxxx`.
|
||||
|
||||
```console
|
||||
$ make chainboot
|
||||
[...]
|
||||
Minipush 1.0
|
||||
|
||||
[MP] ⏳ Waiting for /dev/ttyUSB0
|
||||
[MP] ✅ Connected
|
||||
__ __ _ _ _ _
|
||||
| \/ (_)_ _ (_) | ___ __ _ __| |
|
||||
| |\/| | | ' \| | |__/ _ \/ _` / _` |
|
||||
|_| |_|_|_||_|_|____\___/\__,_\__,_|
|
||||
|
||||
Raspberry Pi 3
|
||||
|
||||
[ML] Requesting binary
|
||||
[MP] ⏩ Pushing 64 KiB ========================================🦀 100% 32 KiB/s Time: 00:00:02
|
||||
[ML] Loaded! Executing the payload now
|
||||
|
||||
[ 3.006343] Booting on: Raspberry Pi 3
|
||||
[ 3.007428] MMU online. Special regions:
|
||||
[ 3.009339] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data
|
||||
[ 3.013422] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO
|
||||
[ 3.016985] Current privilege level: EL1
|
||||
[ 3.018895] Exception handling state:
|
||||
[ 3.020676] Debug: Masked
|
||||
[ 3.022240] SError: Masked
|
||||
[ 3.023804] IRQ: Masked
|
||||
[ 3.025368] FIQ: Masked
|
||||
[ 3.026931] Architectural timer resolution: 52 ns
|
||||
[ 3.029234] Drivers loaded:
|
||||
[ 3.030580] 1. BCM GPIO
|
||||
[ 3.032014] 2. BCM PL011 UART
|
||||
[ 3.033708] Timer test, spinning for 1 second
|
||||
[ 4.035837]
|
||||
[ 4.035841] Trying to write to address 8 GiB...
|
||||
[ 4.038006] ************************************************
|
||||
[ 4.040785] Whoa! We recovered from a synchronous exception!
|
||||
[ 4.043565] ************************************************
|
||||
[ 4.046345]
|
||||
[ 4.047040] Let's try again
|
||||
[ 4.048387] Trying to write to address 9 GiB...
|
||||
|
||||
Kernel panic:
|
||||
|
||||
CPU Exception!
|
||||
FAR_EL1: 0x0000000240000000
|
||||
ESR_EL1: 0x96000004
|
||||
Exception Class (EC) : 0x25 - Data Abort, current EL
|
||||
Instr Specific Syndrome (ISS): 0x4
|
||||
ELR_EL1: 0x0000000000080db4
|
||||
SPSR_EL1: 0x600003c5
|
||||
Flags:
|
||||
Negative (N): Not set
|
||||
Zero (Z): Set
|
||||
Carry (C): Set
|
||||
Overflow (V): Not set
|
||||
Exception handling state:
|
||||
Debug (D): Masked
|
||||
SError (A): Masked
|
||||
IRQ (I): Masked
|
||||
FIQ (F): Masked
|
||||
Illegal Execution State (IL): Not set
|
||||
|
||||
General purpose register:
|
||||
x0 : 0x0000000000000000 x1 : 0x00000000000858f6
|
||||
x2 : 0x0000000000000026 x3 : 0x0000000000082a0c
|
||||
x4 : 0x000000000007fc7c x5 : 0x0000000000000003
|
||||
x6 : 0x0000000000000000 x7 : 0x7f91bc052b2b0208
|
||||
x8 : 0x0000000240000000 x9 : 0x00000000000858f6
|
||||
x10: 0x000000000000041d x11: 0x000000003f201000
|
||||
x12: 0x0000000000000019 x13: 0x000000000007fc7d
|
||||
x14: 0x000000000007fdc8 x15: 0x0000000000000040
|
||||
x16: 0x0000000000000000 x17: 0x0000000000000040
|
||||
x18: 0x9e06782800000028 x19: 0x000000003b9aca00
|
||||
x20: 0x00000000000003e8 x21: 0x0000000000082f58
|
||||
x22: 0x00000000000830cc x23: 0x0000000000090008
|
||||
x24: 0x00000000000f4240 x25: 0x0000000000085248
|
||||
x26: 0x00000000000856e0 x27: 0x00000000000857c0
|
||||
x28: 0x00000000000830cc x29: 0x0000000000085530
|
||||
lr : 0x0000000000080da8
|
||||
```
|
||||
|
||||
## Diff to previous
|
Binary file not shown.
Binary file not shown.
@ -0,0 +1,97 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Architectural processor code.
|
||||
|
||||
use crate::{bsp, cpu};
|
||||
use cortex_a::{asm, regs::*};
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Boot Code
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// The entry of the `kernel` binary.
|
||||
///
|
||||
/// The function must be named `_start`, because the linker is looking for this exact name.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// - Linker script must ensure to place this function at `0x80_000`.
|
||||
#[naked]
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn _start() -> ! {
|
||||
// Expect the boot core to start in EL2.
|
||||
if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id())
|
||||
&& (CurrentEL.get() == CurrentEL::EL::EL2.value)
|
||||
{
|
||||
el2_to_el1_transition()
|
||||
} else {
|
||||
// If not core0, infinitely wait for events.
|
||||
wait_forever()
|
||||
}
|
||||
}
|
||||
|
||||
/// Transition from EL2 to EL1.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// - The HW state of EL1 must be prepared in a sound way.
|
||||
/// - Exception return from EL2 must must continue execution in EL1 with
|
||||
/// `runtime_init::runtime_init()`.
|
||||
#[inline(always)]
|
||||
unsafe fn el2_to_el1_transition() -> ! {
|
||||
use crate::runtime_init;
|
||||
|
||||
// Enable timer counter registers for EL1.
|
||||
CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET);
|
||||
|
||||
// No offset for reading the counters.
|
||||
CNTVOFF_EL2.set(0);
|
||||
|
||||
// Set EL1 execution state to AArch64.
|
||||
HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64);
|
||||
|
||||
// Set up a simulated exception return.
|
||||
//
|
||||
// First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a
|
||||
// stack pointer.
|
||||
SPSR_EL2.write(
|
||||
SPSR_EL2::D::Masked
|
||||
+ SPSR_EL2::A::Masked
|
||||
+ SPSR_EL2::I::Masked
|
||||
+ SPSR_EL2::F::Masked
|
||||
+ SPSR_EL2::M::EL1h,
|
||||
);
|
||||
|
||||
// Second, let the link register point to runtime_init().
|
||||
ELR_EL2.set(runtime_init::runtime_init as *const () as u64);
|
||||
|
||||
// Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it.
|
||||
SP_EL1.set(bsp::cpu::BOOT_CORE_STACK_START);
|
||||
|
||||
// Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1.
|
||||
asm::eret()
|
||||
}
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Code
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
pub use asm::nop;
|
||||
|
||||
/// Spin for `n` cycles.
|
||||
#[inline(always)]
|
||||
pub fn spin_for_cycles(n: usize) {
|
||||
for _ in 0..n {
|
||||
asm::nop();
|
||||
}
|
||||
}
|
||||
|
||||
/// Pause execution on the core.
|
||||
#[inline(always)]
|
||||
pub fn wait_forever() -> ! {
|
||||
loop {
|
||||
asm::wfe()
|
||||
}
|
||||
}
|
@ -0,0 +1,22 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Architectural symmetric multiprocessing.
|
||||
|
||||
use cortex_a::regs::*;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Code
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Return the executing core's id.
|
||||
#[inline(always)]
|
||||
pub fn core_id<T>() -> T
|
||||
where
|
||||
T: From<u8>,
|
||||
{
|
||||
const CORE_MASK: u64 = 0b11;
|
||||
|
||||
T::from((MPIDR_EL1.get() & CORE_MASK) as u8)
|
||||
}
|
@ -0,0 +1,71 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Architectural asynchronous exception handling.
|
||||
|
||||
use cortex_a::regs::*;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Private Definitions
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
trait DaifField {
|
||||
fn daif_field() -> register::Field<u32, DAIF::Register>;
|
||||
}
|
||||
|
||||
struct Debug;
|
||||
struct SError;
|
||||
struct IRQ;
|
||||
struct FIQ;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Private Code
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
impl DaifField for Debug {
|
||||
fn daif_field() -> register::Field<u32, DAIF::Register> {
|
||||
DAIF::D
|
||||
}
|
||||
}
|
||||
|
||||
impl DaifField for SError {
|
||||
fn daif_field() -> register::Field<u32, DAIF::Register> {
|
||||
DAIF::A
|
||||
}
|
||||
}
|
||||
|
||||
impl DaifField for IRQ {
|
||||
fn daif_field() -> register::Field<u32, DAIF::Register> {
|
||||
DAIF::I
|
||||
}
|
||||
}
|
||||
|
||||
impl DaifField for FIQ {
|
||||
fn daif_field() -> register::Field<u32, DAIF::Register> {
|
||||
DAIF::F
|
||||
}
|
||||
}
|
||||
|
||||
fn is_masked<T: DaifField>() -> bool {
|
||||
DAIF.is_set(T::daif_field())
|
||||
}
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Code
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Print the AArch64 exceptions status.
|
||||
#[rustfmt::skip]
|
||||
pub fn print_state() {
|
||||
use crate::info;
|
||||
|
||||
let to_mask_str = |x| -> _ {
|
||||
if x { "Masked" } else { "Unmasked" }
|
||||
};
|
||||
|
||||
info!(" Debug: {}", to_mask_str(is_masked::<Debug>()));
|
||||
info!(" SError: {}", to_mask_str(is_masked::<SError>()));
|
||||
info!(" IRQ: {}", to_mask_str(is_masked::<IRQ>()));
|
||||
info!(" FIQ: {}", to_mask_str(is_masked::<FIQ>()));
|
||||
}
|
@ -0,0 +1,38 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Top-level BSP file for the Raspberry Pi 3 and 4.
|
||||
|
||||
pub mod console;
|
||||
pub mod cpu;
|
||||
pub mod driver;
|
||||
pub mod memory;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Global instances
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
use super::device_driver;
|
||||
|
||||
static GPIO: device_driver::GPIO =
|
||||
unsafe { device_driver::GPIO::new(memory::map::mmio::GPIO_BASE) };
|
||||
|
||||
static PL011_UART: device_driver::PL011Uart =
|
||||
unsafe { device_driver::PL011Uart::new(memory::map::mmio::PL011_UART_BASE) };
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Code
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Board identification.
|
||||
pub fn board_name() -> &'static str {
|
||||
#[cfg(feature = "bsp_rpi3")]
|
||||
{
|
||||
"Raspberry Pi 3"
|
||||
}
|
||||
|
||||
#[cfg(feature = "bsp_rpi4")]
|
||||
{
|
||||
"Raspberry Pi 4"
|
||||
}
|
||||
}
|
@ -0,0 +1,30 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! BSP console facilities.
|
||||
|
||||
use super::{super::device_driver, memory::map};
|
||||
use crate::console;
|
||||
use core::fmt;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Code
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// In case of a panic, the panic handler uses this function to take a last shot at printing
|
||||
/// something before the system is halted.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// - Use only for printing during a panic.
|
||||
pub unsafe fn panic_console_out() -> impl fmt::Write {
|
||||
let mut uart = device_driver::PanicUart::new(map::mmio::PL011_UART_BASE);
|
||||
uart.init();
|
||||
uart
|
||||
}
|
||||
|
||||
/// Return a reference to the console.
|
||||
pub fn console() -> &'static impl console::interface::All {
|
||||
&super::PL011_UART
|
||||
}
|
@ -0,0 +1,15 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! BSP Processor code.
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Definitions
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Used by `arch` code to find the early boot core.
|
||||
pub const BOOT_CORE_ID: usize = 0;
|
||||
|
||||
/// The early boot core's stack address.
|
||||
pub const BOOT_CORE_STACK_START: u64 = 0x80_000;
|
@ -0,0 +1,49 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! BSP driver support.
|
||||
|
||||
use crate::driver;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Definitions
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Device Driver Manager type.
|
||||
pub struct BSPDriverManager {
|
||||
device_drivers: [&'static (dyn DeviceDriver + Sync); 2],
|
||||
}
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Global instances
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager {
|
||||
device_drivers: [&super::GPIO, &super::PL011_UART],
|
||||
};
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Code
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Return a reference to the driver manager.
|
||||
pub fn driver_manager() -> &'static impl driver::interface::DriverManager {
|
||||
&BSP_DRIVER_MANAGER
|
||||
}
|
||||
|
||||
//------------------------------------------------------------------------------
|
||||
// OS Interface Code
|
||||
//------------------------------------------------------------------------------
|
||||
use driver::interface::DeviceDriver;
|
||||
|
||||
impl driver::interface::DriverManager for BSPDriverManager {
|
||||
fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] {
|
||||
&self.device_drivers[..]
|
||||
}
|
||||
|
||||
fn post_device_driver_init(&self) {
|
||||
// Configure PL011Uart's output pins.
|
||||
super::GPIO.map_pl011_uart();
|
||||
}
|
||||
}
|
@ -0,0 +1,42 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! BSP Memory Management.
|
||||
|
||||
pub mod mmu;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Definitions
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// The board's memory map.
|
||||
#[rustfmt::skip]
|
||||
pub(super) mod map {
|
||||
pub const END_INCLUSIVE: usize = 0xFFFF_FFFF;
|
||||
|
||||
pub const GPIO_OFFSET: usize = 0x0020_0000;
|
||||
pub const UART_OFFSET: usize = 0x0020_1000;
|
||||
|
||||
/// Physical devices.
|
||||
#[cfg(feature = "bsp_rpi3")]
|
||||
pub mod mmio {
|
||||
use super::*;
|
||||
|
||||
pub const BASE: usize = 0x3F00_0000;
|
||||
pub const GPIO_BASE: usize = BASE + GPIO_OFFSET;
|
||||
pub const PL011_UART_BASE: usize = BASE + UART_OFFSET;
|
||||
pub const END_INCLUSIVE: usize = 0x4000_FFFF;
|
||||
}
|
||||
|
||||
/// Physical devices.
|
||||
#[cfg(feature = "bsp_rpi4")]
|
||||
pub mod mmio {
|
||||
use super::*;
|
||||
|
||||
pub const BASE: usize = 0xFE00_0000;
|
||||
pub const GPIO_BASE: usize = BASE + GPIO_OFFSET;
|
||||
pub const PL011_UART_BASE: usize = BASE + UART_OFFSET;
|
||||
pub const END_INCLUSIVE: usize = 0xFF84_FFFF;
|
||||
}
|
||||
}
|
@ -0,0 +1,54 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! System console.
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Definitions
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Console interfaces.
|
||||
pub mod interface {
|
||||
use core::fmt;
|
||||
|
||||
/// Console write functions.
|
||||
pub trait Write {
|
||||
/// Write a single character.
|
||||
fn write_char(&self, c: char);
|
||||
|
||||
/// Write a Rust format string.
|
||||
fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result;
|
||||
|
||||
/// Block execution until the last character has been physically put on the TX wire
|
||||
/// (draining TX buffers/FIFOs, if any).
|
||||
fn flush(&self);
|
||||
}
|
||||
|
||||
/// Console read functions.
|
||||
pub trait Read {
|
||||
/// Read a single character.
|
||||
fn read_char(&self) -> char {
|
||||
' '
|
||||
}
|
||||
|
||||
/// Clear RX buffers, if any.
|
||||
fn clear(&self);
|
||||
}
|
||||
|
||||
/// Console statistics.
|
||||
pub trait Statistics {
|
||||
/// Return the number of characters written.
|
||||
fn chars_written(&self) -> usize {
|
||||
0
|
||||
}
|
||||
|
||||
/// Return the number of characters read.
|
||||
fn chars_read(&self) -> usize {
|
||||
0
|
||||
}
|
||||
}
|
||||
|
||||
/// Trait alias for a full-fledged console.
|
||||
pub trait All = Write + Read + Statistics;
|
||||
}
|
@ -0,0 +1,12 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Processor code.
|
||||
|
||||
#[cfg(target_arch = "aarch64")]
|
||||
#[path = "_arch/aarch64/cpu.rs"]
|
||||
mod arch_cpu;
|
||||
pub use arch_cpu::*;
|
||||
|
||||
pub mod smp;
|
@ -0,0 +1,10 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Symmetric multiprocessing.
|
||||
|
||||
#[cfg(target_arch = "aarch64")]
|
||||
#[path = "../_arch/aarch64/cpu/smp.rs"]
|
||||
mod arch_cpu_smp;
|
||||
pub use arch_cpu_smp::*;
|
@ -0,0 +1,41 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Driver support.
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Definitions
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Driver interfaces.
|
||||
pub mod interface {
|
||||
|
||||
/// Device Driver functions.
|
||||
pub trait DeviceDriver {
|
||||
/// Return a compatibility string for identifying the driver.
|
||||
fn compatible(&self) -> &str;
|
||||
|
||||
/// Called by the kernel to bring up the device.
|
||||
fn init(&self) -> Result<(), ()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Device driver management functions.
|
||||
///
|
||||
/// The `BSP` is supposed to supply one global instance.
|
||||
pub trait DriverManager {
|
||||
/// Return a slice of references to all `BSP`-instantiated drivers.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// - The order of devices is the order in which `DeviceDriver::init()` is called.
|
||||
fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)];
|
||||
|
||||
/// Initialization code that runs after driver init.
|
||||
///
|
||||
/// For example, device driver code that depends on other drivers already being online.
|
||||
fn post_device_driver_init(&self);
|
||||
}
|
||||
}
|
@ -0,0 +1,26 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Synchronous and asynchronous exception handling.
|
||||
|
||||
#[cfg(target_arch = "aarch64")]
|
||||
#[path = "_arch/aarch64/exception.rs"]
|
||||
mod arch_exception;
|
||||
pub use arch_exception::*;
|
||||
|
||||
pub mod asynchronous;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Definitions
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Kernel privilege levels.
|
||||
#[allow(missing_docs)]
|
||||
#[derive(PartialEq)]
|
||||
pub enum PrivilegeLevel {
|
||||
User,
|
||||
Kernel,
|
||||
Hypervisor,
|
||||
Unknown,
|
||||
}
|
@ -0,0 +1,10 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Asynchronous exception handling.
|
||||
|
||||
#[cfg(target_arch = "aarch64")]
|
||||
#[path = "../_arch/aarch64/exception/asynchronous.rs"]
|
||||
mod arch_exception_async;
|
||||
pub use arch_exception_async::*;
|
@ -0,0 +1,226 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
// Rust embedded logo for `make doc`.
|
||||
#![doc(html_logo_url = "https://git.io/JeGIp")]
|
||||
|
||||
//! The `kernel` binary.
|
||||
//!
|
||||
//! # TL;DR - Overview of important Kernel entities
|
||||
//!
|
||||
//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface].
|
||||
//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface].
|
||||
//! - [`memory::mmu::mmu()`] - Returns a reference to the kernel's [MMU interface].
|
||||
//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface].
|
||||
//!
|
||||
//! [console interface]: ../libkernel/console/interface/index.html
|
||||
//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html
|
||||
//! [MMU interface]: ../libkernel/memory/mmu/interface/trait.MMU.html
|
||||
//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html
|
||||
//!
|
||||
//! # Code organization and architecture
|
||||
//!
|
||||
//! The code is divided into different *modules*, each representing a typical **subsystem** of the
|
||||
//! `kernel`. Top-level module files of subsystems reside directly in the `src` folder. For example,
|
||||
//! `src/memory.rs` contains code that is concerned with all things memory management.
|
||||
//!
|
||||
//! ## Visibility of processor architecture code
|
||||
//!
|
||||
//! Some of the `kernel`'s subsystems depend on low-level code that is specific to the target
|
||||
//! processor architecture. For each supported processor architecture, there exists a subfolder in
|
||||
//! `src/_arch`, for example, `src/_arch/aarch64`.
|
||||
//!
|
||||
//! The architecture folders mirror the subsystem modules laid out in `src`. For example,
|
||||
//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go
|
||||
//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in
|
||||
//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's
|
||||
//! module organization. That means a public function `foo()` defined in
|
||||
//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only.
|
||||
//!
|
||||
//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy.
|
||||
//! Rather, it's contents are conditionally pulled into respective files using the `#[path =
|
||||
//! "_arch/xxx/yyy.rs"]` attribute.
|
||||
//!
|
||||
//! ## BSP code
|
||||
//!
|
||||
//! `BSP` stands for Board Support Package. `BSP` code is organized under `src/bsp.rs` and contains
|
||||
//! target board specific definitions and functions. These are things such as the board's memory map
|
||||
//! or instances of drivers for devices that are featured on the respective board.
|
||||
//!
|
||||
//! Just like processor architecture code, the `BSP` code's module structure tries to mirror the
|
||||
//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means
|
||||
//! whatever is provided must be called starting from the `bsp` namespace, e.g.
|
||||
//! `bsp::driver::driver_manager()`.
|
||||
//!
|
||||
//! ## Kernel interfaces
|
||||
//!
|
||||
//! Both `arch` and `bsp` contain code that is conditionally compiled depending on the actual target
|
||||
//! and board for which the kernel is compiled. For example, the `interrupt controller` hardware of
|
||||
//! the `Raspberry Pi 3` and the `Raspberry Pi 4` is different, but we want the rest of the `kernel`
|
||||
//! code to play nicely with any of the two without much hassle.
|
||||
//!
|
||||
//! In order to provide a clean abstraction between `arch`, `bsp` and `generic kernel code`,
|
||||
//! `interface` traits are provided *whenever possible* and *where it makes sense*. They are defined
|
||||
//! in the respective subsystem module and help to enforce the idiom of *program to an interface,
|
||||
//! not an implementation*. For example, there will be a common IRQ handling interface which the two
|
||||
//! different interrupt controller `drivers` of both Raspberrys will implement, and only export the
|
||||
//! interface to the rest of the `kernel`.
|
||||
//!
|
||||
//! ```
|
||||
//! +-------------------+
|
||||
//! | Interface (Trait) |
|
||||
//! | |
|
||||
//! +--+-------------+--+
|
||||
//! ^ ^
|
||||
//! | |
|
||||
//! | |
|
||||
//! +----------+--+ +--+----------+
|
||||
//! | kernel code | | bsp code |
|
||||
//! | | | arch code |
|
||||
//! +-------------+ +-------------+
|
||||
//! ```
|
||||
//!
|
||||
//! # Summary
|
||||
//!
|
||||
//! For a logical `kernel` subsystem, corresponding code can be distributed over several physical
|
||||
//! locations. Here is an example for the **memory** subsystem:
|
||||
//!
|
||||
//! - `src/memory.rs` and `src/memory/**/*`
|
||||
//! - Common code that is agnostic of target processor architecture and `BSP` characteristics.
|
||||
//! - Example: A function to zero a chunk of memory.
|
||||
//! - Interfaces for the memory subsystem that are implemented by `arch` or `BSP` code.
|
||||
//! - Example: An `MMU` interface that defines `MMU` function prototypes.
|
||||
//! - `src/bsp/__board_name__/memory.rs` and `src/bsp/__board_name__/memory/**/*`
|
||||
//! - `BSP` specific code.
|
||||
//! - Example: The board's memory map (physical addresses of DRAM and MMIO devices).
|
||||
//! - `src/_arch/__arch_name__/memory.rs` and `src/_arch/__arch_name__/memory/**/*`
|
||||
//! - Processor architecture specific code.
|
||||
//! - Example: Implementation of the `MMU` interface for the `__arch_name__` processor
|
||||
//! architecture.
|
||||
//!
|
||||
//! From a namespace perspective, **memory** subsystem code lives in:
|
||||
//!
|
||||
//! - `crate::memory::*`
|
||||
//! - `crate::bsp::memory::*`
|
||||
|
||||
#![allow(incomplete_features)]
|
||||
#![feature(const_generics)]
|
||||
#![feature(format_args_nl)]
|
||||
#![feature(global_asm)]
|
||||
#![feature(naked_functions)]
|
||||
#![feature(panic_info_message)]
|
||||
#![feature(trait_alias)]
|
||||
#![no_main]
|
||||
#![no_std]
|
||||
|
||||
// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls
|
||||
// `runtime_init()`, which jumps to `kernel_init()`.
|
||||
|
||||
mod bsp;
|
||||
mod console;
|
||||
mod cpu;
|
||||
mod driver;
|
||||
mod exception;
|
||||
mod memory;
|
||||
mod panic_wait;
|
||||
mod print;
|
||||
mod runtime_init;
|
||||
mod synchronization;
|
||||
mod time;
|
||||
|
||||
/// Early init code.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// - Only a single core must be active and running this function.
|
||||
/// - The init calls in this function must appear in the correct order:
|
||||
/// - Virtual memory must be activated before the device drivers.
|
||||
/// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device
|
||||
/// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on
|
||||
/// the RPi SoCs.
|
||||
unsafe fn kernel_init() -> ! {
|
||||
use driver::interface::DriverManager;
|
||||
use memory::mmu::interface::MMU;
|
||||
|
||||
exception::handling_init();
|
||||
|
||||
if let Err(string) = memory::mmu::mmu().init() {
|
||||
panic!("MMU: {}", string);
|
||||
}
|
||||
|
||||
for i in bsp::driver::driver_manager().all_device_drivers().iter() {
|
||||
if i.init().is_err() {
|
||||
panic!("Error loading driver: {}", i.compatible())
|
||||
}
|
||||
}
|
||||
bsp::driver::driver_manager().post_device_driver_init();
|
||||
// println! is usable from here on.
|
||||
|
||||
// Transition from unsafe to safe.
|
||||
kernel_main()
|
||||
}
|
||||
|
||||
/// The main function running after the early init.
|
||||
fn kernel_main() -> ! {
|
||||
use console::interface::All;
|
||||
use core::time::Duration;
|
||||
use driver::interface::DriverManager;
|
||||
use time::interface::TimeManager;
|
||||
|
||||
info!("Booting on: {}", bsp::board_name());
|
||||
|
||||
info!("MMU online. Special regions:");
|
||||
bsp::memory::mmu::virt_mem_layout().print_layout();
|
||||
|
||||
let (_, privilege_level) = exception::current_privilege_level();
|
||||
info!("Current privilege level: {}", privilege_level);
|
||||
|
||||
info!("Exception handling state:");
|
||||
exception::asynchronous::print_state();
|
||||
|
||||
info!(
|
||||
"Architectural timer resolution: {} ns",
|
||||
time::time_manager().resolution().as_nanos()
|
||||
);
|
||||
|
||||
info!("Drivers loaded:");
|
||||
for (i, driver) in bsp::driver::driver_manager()
|
||||
.all_device_drivers()
|
||||
.iter()
|
||||
.enumerate()
|
||||
{
|
||||
info!(" {}. {}", i + 1, driver.compatible());
|
||||
}
|
||||
|
||||
info!("Timer test, spinning for 1 second");
|
||||
time::time_manager().spin_for(Duration::from_secs(1));
|
||||
|
||||
// Cause an exception by accessing a virtual address for which no translation was set up. This
|
||||
// code accesses the address 8 GiB, which is outside the mapped address space.
|
||||
//
|
||||
// For demo purposes, the exception handler will catch the faulting 8 GiB address and allow
|
||||
// execution to continue.
|
||||
info!("");
|
||||
info!("Trying to write to address 8 GiB...");
|
||||
let mut big_addr: u64 = 8 * 1024 * 1024 * 1024;
|
||||
unsafe { core::ptr::read_volatile(big_addr as *mut u64) };
|
||||
|
||||
info!("************************************************");
|
||||
info!("Whoa! We recovered from a synchronous exception!");
|
||||
info!("************************************************");
|
||||
info!("");
|
||||
info!("Let's try again");
|
||||
|
||||
// Now use address 9 GiB. The exception handler won't forgive us this time.
|
||||
info!("Trying to write to address 9 GiB...");
|
||||
big_addr = 9 * 1024 * 1024 * 1024;
|
||||
unsafe { core::ptr::read_volatile(big_addr as *mut u64) };
|
||||
|
||||
// Will never reach here in this tutorial.
|
||||
info!("Echoing input now");
|
||||
loop {
|
||||
let c = bsp::console::console().read_char();
|
||||
bsp::console::console().write_char(c);
|
||||
}
|
||||
}
|
@ -0,0 +1,31 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2018-2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Memory Management.
|
||||
|
||||
pub mod mmu;
|
||||
|
||||
use core::ops::Range;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Code
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Zero out a memory region.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// - `range.start` and `range.end` must be valid.
|
||||
/// - `range.start` and `range.end` must be `T` aligned.
|
||||
pub unsafe fn zero_volatile<T>(range: Range<*mut T>)
|
||||
where
|
||||
T: From<u8>,
|
||||
{
|
||||
let mut ptr = range.start;
|
||||
|
||||
while ptr < range.end {
|
||||
core::ptr::write_volatile(ptr, T::from(0));
|
||||
ptr = ptr.offset(1);
|
||||
}
|
||||
}
|
@ -0,0 +1,91 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Synchronization primitives.
|
||||
|
||||
use core::cell::UnsafeCell;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Definitions
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Synchronization interfaces.
|
||||
pub mod interface {
|
||||
|
||||
/// Any object implementing this trait guarantees exclusive access to the data contained within
|
||||
/// the Mutex for the duration of the provided closure.
|
||||
///
|
||||
/// The trait follows the [Rust embedded WG's
|
||||
/// proposal](https://github.com/korken89/wg/blob/master/rfcs/0377-mutex-trait.md) and therefore
|
||||
/// provides some goodness such as [deadlock
|
||||
/// prevention](https://github.com/korken89/wg/blob/master/rfcs/0377-mutex-trait.md#design-decisions-and-compatibility).
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// Since the lock function takes an `&mut self` to enable deadlock-prevention, the trait is
|
||||
/// best implemented **for a reference to a container struct**, and has a usage pattern that
|
||||
/// might feel strange at first:
|
||||
///
|
||||
/// ```
|
||||
/// static MUT: Mutex<RefCell<i32>> = Mutex::new(RefCell::new(0));
|
||||
///
|
||||
/// fn foo() {
|
||||
/// let mut r = &MUT; // Note that r is mutable
|
||||
/// r.lock(|data| *data += 1);
|
||||
/// }
|
||||
/// ```
|
||||
pub trait Mutex {
|
||||
/// The type of encapsulated data.
|
||||
type Data;
|
||||
|
||||
/// Creates a critical section and grants temporary mutable access to the encapsulated data.
|
||||
fn lock<R>(&mut self, f: impl FnOnce(&mut Self::Data) -> R) -> R;
|
||||
}
|
||||
}
|
||||
|
||||
/// A pseudo-lock for teaching purposes.
|
||||
///
|
||||
/// Used to introduce [interior mutability].
|
||||
///
|
||||
/// In contrast to a real Mutex implementation, does not protect against concurrent access from
|
||||
/// other cores to the contained data. This part is preserved for later lessons.
|
||||
///
|
||||
/// The lock will only be used as long as it is safe to do so, i.e. as long as the kernel is
|
||||
/// executing single-threaded, aka only running on a single core with interrupts disabled.
|
||||
///
|
||||
/// [interior mutability]: https://doc.rust-lang.org/std/cell/index.html
|
||||
pub struct NullLock<T: ?Sized> {
|
||||
data: UnsafeCell<T>,
|
||||
}
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Code
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
unsafe impl<T: ?Sized> Sync for NullLock<T> {}
|
||||
|
||||
impl<T> NullLock<T> {
|
||||
/// Wraps `data` into a new `NullLock`.
|
||||
pub const fn new(data: T) -> Self {
|
||||
Self {
|
||||
data: UnsafeCell::new(data),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
//------------------------------------------------------------------------------
|
||||
// OS Interface Code
|
||||
//------------------------------------------------------------------------------
|
||||
|
||||
impl<T> interface::Mutex for &NullLock<T> {
|
||||
type Data = T;
|
||||
|
||||
fn lock<R>(&mut self, f: impl FnOnce(&mut Self::Data) -> R) -> R {
|
||||
// In a real lock, there would be code encapsulating this line that ensures that this
|
||||
// mutable reference will ever only be given out once at a time.
|
||||
let data = unsafe { &mut *self.data.get() };
|
||||
|
||||
f(data)
|
||||
}
|
||||
}
|
@ -0,0 +1,35 @@
|
||||
// SPDX-License-Identifier: MIT OR Apache-2.0
|
||||
//
|
||||
// Copyright (c) 2020 Andre Richter <andre.o.richter@gmail.com>
|
||||
|
||||
//! Timer primitives.
|
||||
|
||||
#[cfg(target_arch = "aarch64")]
|
||||
#[path = "_arch/aarch64/time.rs"]
|
||||
mod arch_time;
|
||||
pub use arch_time::*;
|
||||
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
// Public Definitions
|
||||
//--------------------------------------------------------------------------------------------------
|
||||
|
||||
/// Timekeeping interfaces.
|
||||
pub mod interface {
|
||||
use core::time::Duration;
|
||||
|
||||
/// Time management functions.
|
||||
///
|
||||
/// The `BSP` is supposed to supply one global instance.
|
||||
pub trait TimeManager {
|
||||
/// The timer's resolution.
|
||||
fn resolution(&self) -> Duration;
|
||||
|
||||
/// The uptime since power-on of the device.
|
||||
///
|
||||
/// This includes time consumed by firmware and bootloaders.
|
||||
fn uptime(&self) -> Duration;
|
||||
|
||||
/// Spin for a given duration.
|
||||
fn spin_for(&self, duration: Duration);
|
||||
}
|
||||
}
|
Loading…
Reference in New Issue