diff --git a/05_drivers_gpio_uart/README.md b/05_drivers_gpio_uart/README.md index cbd43e71..3fe5e7b2 100644 --- a/05_drivers_gpio_uart/README.md +++ b/05_drivers_gpio_uart/README.md @@ -2,35 +2,81 @@ ## tl;dr -- Now that we enabled safe globals in the previous tutorial, the infrastructure is laid for adding - the first real device drivers. -- We throw out the magic QEMU console and use a real `UART` now. Like serious embedded hackers do! - -## Notable additions - -- For the first time, we will be able to run the code on the real hardware. - - Therefore, building is now differentiated between the **RPi 3** and the **RPi4**. - - By default, all `Makefile` targets will build for the **RPi 3**. - - In order to build for the the **RPi4**, prepend `BSP=rpi4` to each target. For example: - - `BSP=rpi4 make` - - `BSP=rpi4 make doc` - - Unfortunately, QEMU does not yet support the **RPi4**, so `BSP=rpi4 make qemu` won't work. -- A `driver::interface::DeviceDriver` trait is added for abstracting `BSP` driver implementations - from kernel code. -- Drivers are stored in `src/bsp/device_driver`, and can be reused between `BSP`s. - - We introduce the `GPIO` driver, which pinmuxes (that is, routing signals from inside the `SoC` - to actual HW pins) the RPi's PL011 UART. - - Note how this driver differentiates between **RPi 3** and **RPi4**. Their HW is different, - so we have to account for it in SW. - - Most importantly, the `PL011Uart` driver: It implements the `console::interface::*` traits and - is from now on used as the main system console output. -- `BSP`s now contain a memory map in `src/bsp/raspberrypi/memory.rs`. In the specific case, they - contain the Raspberry's `MMIO` addresses which are used to instantiate the respective device - drivers. +- Drivers for the real `UART` and the `GPIO` controller are added. +- **For the first time, we will be able to run the code on the real hardware** (scroll down for + instructions). + +## Introduction + +Now that we enabled safe globals in the previous tutorial, the infrastructure is laid for adding the +first real device drivers. We throw out the magic QEMU console and introduce a `driver manager`, +which allows the `BSP` to register device drivers with the `kernel`. + +## Driver Manager + +The first step consists of adding a `driver subsystem` to the kernel. The corresponding code will +live in `src/driver.rs`. The subsystem introduces `interface::DeviceDriver`, a common trait that +every device driver will need to implement and that is known to the kernel. A global +`DRIVER_MANAGER` instance (of type `DriverManager`) that is instantiated in the same file serves as +the central entity that can be called to manage all things device drivers in the kernel. For +example, by using the globally accessible `crate::driver::driver_manager().register_driver(...)`, +any code can can register an object with static lifetime that implements the +`interface::DeviceDriver` trait. + +During kernel init, a call to `crate::driver::driver_manager().init_drivers(...)` will let the +driver manager loop over all registered drivers and kick off their initialization, and also execute +an optional `post-init callback` that can be registered alongside the driver. For example, this +mechanism is used to switch over to the `UART` driver as the main system console after the `UART` +driver has been initialized. + +## BSP Driver Implementation + +In `src/bsp/raspberrypi/driver.rs`, the function `init()` takes care of registering the `UART` and +`GPIO` drivers. It is therefore important that during kernel init, the correct order of (i) first +initializing the BSP driver subsystem, and only then (ii) calling the `driver_manager()` is +followed, like the following excerpt from `main.rs` shows: + +```rust +unsafe fn kernel_init() -> ! { + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); + } + + // Initialize all device drivers. + driver::driver_manager().init_drivers(); + // println! is usable from here on. +``` + + + +The drivers themselves are stored in `src/bsp/device_driver`, and can be reused between `BSP`s. The +first driver added in these tutorials is the `PL011Uart` driver: It implements the +`console::interface::*` traits and is from now on used as the main system console. The second driver +is the `GPIO` driver, which pinmuxes (that is, routing signals from inside the `SoC` to actual HW +pins) the RPi's PL011 UART accordingly. Note how the `GPIO` driver differentiates between **RPi3** +and **RPi4**. Their HW is different, so we have to account for it in SW. + +The `BSP`s now also contain a memory map in `src/bsp/raspberrypi/memory.rs`. It provides the +Raspberry's `MMIO` addresses which are used by the `BSP` to instantiate the respective device +drivers, so that the driver code knows where to find the device's registers in memory. ## Boot it from SD card -Some steps for preparing the SD card differ between RPi3 and RPi4, so be careful. +Since we have real `UART` output now, we can run the code on the real hardware. Building is +differentiated between the **RPi 3** and the **RPi4** due to before mentioned differences in the +`GPIO` driver. By default, all `Makefile` targets will build for the **RPi 3**. In order to build +for the the **RPi4**, prepend `BSP=rpi4` to each target. For example: + +```console +$ BSP=rpi4 make +$ BSP=rpi4 make doc +``` + +Unfortunately, QEMU does not yet support the **RPi4**, so `BSP=rpi4 make qemu` won't work. + +**Some steps for preparing the SD card differ between RPi3 and RPi4, so be careful in the +following.** ### Common for both @@ -1072,7 +1118,7 @@ diff -uNr 04_safe_globals/src/bsp/raspberrypi/console.rs 05_drivers_gpio_uart/sr diff -uNr 04_safe_globals/src/bsp/raspberrypi/driver.rs 05_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs --- 04_safe_globals/src/bsp/raspberrypi/driver.rs +++ 05_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs -@@ -0,0 +1,55 @@ +@@ -0,0 +1,71 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2022 Andre Richter @@ -1080,53 +1126,69 @@ diff -uNr 04_safe_globals/src/bsp/raspberrypi/driver.rs 05_drivers_gpio_uart/src +//! BSP driver support. + +use super::memory::map::mmio; -+use crate::{bsp::device_driver, driver}; ++use crate::{bsp::device_driver, console, driver as generic_driver}; ++use core::sync::atomic::{AtomicBool, Ordering}; + +//-------------------------------------------------------------------------------------------------- -+// Private Definitions ++// Global instances +//-------------------------------------------------------------------------------------------------- + -+/// Device Driver Manager type. -+struct BSPDriverManager { -+ device_drivers: [&'static (dyn DeviceDriver + Sync); 2], -+} ++static PL011_UART: device_driver::PL011Uart = ++ unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; ++static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + +//-------------------------------------------------------------------------------------------------- -+// Global instances ++// Private Code +//-------------------------------------------------------------------------------------------------- + -+pub(super) static PL011_UART: device_driver::PL011Uart = -+ unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; ++/// This must be called only after successful init of the UART driver. ++fn post_init_uart() -> Result<(), &'static str> { ++ console::register_console(&PL011_UART); + -+static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; ++ Ok(()) ++} + -+static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { -+ device_drivers: [&PL011_UART, &GPIO], -+}; ++/// This must be called only after successful init of the GPIO driver. ++fn post_init_gpio() -> Result<(), &'static str> { ++ GPIO.map_pl011_uart(); ++ Ok(()) ++} + -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- ++fn driver_uart() -> Result<(), &'static str> { ++ let uart_descriptor = ++ generic_driver::DeviceDriverDescriptor::new(&PL011_UART, Some(post_init_uart)); ++ generic_driver::driver_manager().register_driver(uart_descriptor); + -+/// Return a reference to the driver manager. -+pub fn driver_manager() -> &'static impl driver::interface::DriverManager { -+ &BSP_DRIVER_MANAGER ++ Ok(()) +} + -+//------------------------------------------------------------------------------ -+// OS Interface Code -+//------------------------------------------------------------------------------ -+use driver::interface::DeviceDriver; ++fn driver_gpio() -> Result<(), &'static str> { ++ let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio)); ++ generic_driver::driver_manager().register_driver(gpio_descriptor); + -+impl driver::interface::DriverManager for BSPDriverManager { -+ fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { -+ &self.device_drivers[..] -+ } ++ Ok(()) ++} + -+ fn post_device_driver_init(&self) { -+ // Configure PL011Uart's output pins. -+ GPIO.map_pl011_uart(); ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++/// Initialize the driver subsystem. ++/// ++/// # Safety ++/// ++/// See child function calls. ++pub unsafe fn init() -> Result<(), &'static str> { ++ static INIT_DONE: AtomicBool = AtomicBool::new(false); ++ if INIT_DONE.load(Ordering::Relaxed) { ++ return Err("Init already done"); + } ++ ++ driver_uart()?; ++ driver_gpio()?; ++ ++ INIT_DONE.store(true, Ordering::Relaxed); ++ Ok(()) +} diff -uNr 04_safe_globals/src/bsp/raspberrypi/memory.rs 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs @@ -1174,9 +1236,11 @@ diff -uNr 04_safe_globals/src/bsp/raspberrypi/memory.rs 05_drivers_gpio_uart/src diff -uNr 04_safe_globals/src/bsp/raspberrypi.rs 05_drivers_gpio_uart/src/bsp/raspberrypi.rs --- 04_safe_globals/src/bsp/raspberrypi.rs +++ 05_drivers_gpio_uart/src/bsp/raspberrypi.rs -@@ -6,3 +6,22 @@ +@@ -4,5 +4,23 @@ + + //! Top-level BSP file for the Raspberry Pi 3 and 4. - pub mod console; +-pub mod console; pub mod cpu; +pub mod driver; +pub mod memory; @@ -1211,10 +1275,67 @@ diff -uNr 04_safe_globals/src/bsp.rs 05_drivers_gpio_uart/src/bsp.rs mod raspberrypi; +diff -uNr 04_safe_globals/src/console/null_console.rs 05_drivers_gpio_uart/src/console/null_console.rs +--- 04_safe_globals/src/console/null_console.rs ++++ 05_drivers_gpio_uart/src/console/null_console.rs +@@ -0,0 +1,41 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2022 Andre Richter ++ ++//! Null console. ++ ++use super::interface; ++use core::fmt; ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++pub struct NullConsole; ++ ++//-------------------------------------------------------------------------------------------------- ++// Global instances ++//-------------------------------------------------------------------------------------------------- ++ ++pub static NULL_CONSOLE: NullConsole = NullConsole {}; ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++impl interface::Write for NullConsole { ++ fn write_char(&self, _c: char) {} ++ ++ fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { ++ fmt::Result::Ok(()) ++ } ++ ++ fn flush(&self) {} ++} ++ ++impl interface::Read for NullConsole { ++ fn clear_rx(&self) {} ++} ++ ++impl interface::Statistics for NullConsole {} ++impl interface::All for NullConsole {} + diff -uNr 04_safe_globals/src/console.rs 05_drivers_gpio_uart/src/console.rs --- 04_safe_globals/src/console.rs +++ 05_drivers_gpio_uart/src/console.rs -@@ -16,8 +16,25 @@ +@@ -4,7 +4,9 @@ + + //! System console. + +-use crate::bsp; ++mod null_console; ++ ++use crate::synchronization::{self, NullLock}; + + //-------------------------------------------------------------------------------------------------- + // Public Definitions +@@ -16,8 +18,25 @@ /// Console write functions. pub trait Write { @@ -1240,7 +1361,7 @@ diff -uNr 04_safe_globals/src/console.rs 05_drivers_gpio_uart/src/console.rs } /// Console statistics. -@@ -26,10 +43,15 @@ +@@ -26,19 +45,37 @@ fn chars_written(&self) -> usize { 0 } @@ -1257,6 +1378,30 @@ diff -uNr 04_safe_globals/src/console.rs 05_drivers_gpio_uart/src/console.rs } //-------------------------------------------------------------------------------------------------- ++// Global instances ++//-------------------------------------------------------------------------------------------------- ++ ++static CUR_CONSOLE: NullLock<&'static (dyn interface::All + Sync)> = ++ NullLock::new(&null_console::NULL_CONSOLE); ++ ++//-------------------------------------------------------------------------------------------------- + // Public Code + //-------------------------------------------------------------------------------------------------- ++use synchronization::interface::Mutex; ++ ++/// Register a new console. ++pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { ++ CUR_CONSOLE.lock(|con| *con = new_console); ++} + +-/// Return a reference to the console. ++/// Return a reference to the currently registered console. + /// + /// This is the global console used by all printing macros. + pub fn console() -> &'static dyn interface::All { +- bsp::console::console() ++ CUR_CONSOLE.lock(|con| *con) + } diff -uNr 04_safe_globals/src/cpu.rs 05_drivers_gpio_uart/src/cpu.rs --- 04_safe_globals/src/cpu.rs @@ -1274,13 +1419,29 @@ diff -uNr 04_safe_globals/src/cpu.rs 05_drivers_gpio_uart/src/cpu.rs diff -uNr 04_safe_globals/src/driver.rs 05_drivers_gpio_uart/src/driver.rs --- 04_safe_globals/src/driver.rs +++ 05_drivers_gpio_uart/src/driver.rs -@@ -0,0 +1,44 @@ +@@ -0,0 +1,167 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2022 Andre Richter + +//! Driver support. + ++use crate::{ ++ println, ++ synchronization::{interface::Mutex, NullLock}, ++}; ++ ++//-------------------------------------------------------------------------------------------------- ++// Private Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++const NUM_DRIVERS: usize = 5; ++ ++struct DriverManagerInner { ++ next_index: usize, ++ descriptors: [Option; NUM_DRIVERS], ++} ++ +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- @@ -1301,22 +1462,129 @@ diff -uNr 04_safe_globals/src/driver.rs 05_drivers_gpio_uart/src/driver.rs + Ok(()) + } + } ++} ++ ++/// Tpye to be used as an optional callback after a driver's init() has run. ++pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; ++ ++/// A descriptor for device drivers. ++#[derive(Copy, Clone)] ++pub struct DeviceDriverDescriptor { ++ device_driver: &'static (dyn interface::DeviceDriver + Sync), ++ post_init_callback: Option, ++} ++ ++/// Provides device driver management functions. ++pub struct DriverManager { ++ inner: NullLock, ++} ++ ++//-------------------------------------------------------------------------------------------------- ++// Global instances ++//-------------------------------------------------------------------------------------------------- ++ ++static DRIVER_MANAGER: DriverManager = DriverManager::new(); ++ ++//-------------------------------------------------------------------------------------------------- ++// Private Code ++//-------------------------------------------------------------------------------------------------- ++ ++impl DriverManagerInner { ++ /// Create an instance. ++ pub const fn new() -> Self { ++ Self { ++ next_index: 0, ++ descriptors: [None; NUM_DRIVERS], ++ } ++ } ++} ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++impl DeviceDriverDescriptor { ++ /// Create an instance. ++ pub fn new( ++ device_driver: &'static (dyn interface::DeviceDriver + Sync), ++ post_init_callback: Option, ++ ) -> Self { ++ Self { ++ device_driver, ++ post_init_callback, ++ } ++ } ++} ++ ++/// Return a reference to the global DriverManager. ++pub fn driver_manager() -> &'static DriverManager { ++ &DRIVER_MANAGER ++} ++ ++impl DriverManager { ++ /// Create an instance. ++ pub const fn new() -> Self { ++ Self { ++ inner: NullLock::new(DriverManagerInner::new()), ++ } ++ } ++ ++ /// Register a device driver with the kernel. ++ pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { ++ self.inner.lock(|inner| { ++ inner.descriptors[inner.next_index] = Some(descriptor); ++ inner.next_index += 1; ++ }) ++ } ++ ++ /// Helper for iterating over registered drivers. ++ fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { ++ self.inner.lock(|inner| { ++ inner ++ .descriptors ++ .iter() ++ .filter_map(|x| x.as_ref()) ++ .for_each(f) ++ }) ++ } + -+ /// Device driver management functions. ++ /// Fully initialize all drivers. + /// -+ /// The `BSP` is supposed to supply one global instance. -+ pub trait DriverManager { -+ /// Return a slice of references to all `BSP`-instantiated drivers. -+ /// -+ /// # Safety -+ /// -+ /// - The order of devices is the order in which `DeviceDriver::init()` is called. -+ fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; ++ /// # Safety ++ /// ++ /// - During init, drivers might do stuff with system-wide impact. ++ pub unsafe fn init_drivers(&self) { ++ self.for_each_descriptor(|descriptor| { ++ // 1. Initialize driver. ++ if let Err(x) = descriptor.device_driver.init() { ++ panic!( ++ "Error initializing driver: {}: {}", ++ descriptor.device_driver.compatible(), ++ x ++ ); ++ } + -+ /// Initialization code that runs after driver init. -+ /// -+ /// For example, device driver code that depends on other drivers already being online. -+ fn post_device_driver_init(&self); ++ // 2. Call corresponding post init callback. ++ if let Some(callback) = &descriptor.post_init_callback { ++ if let Err(x) = callback() { ++ panic!( ++ "Error during driver post-init callback: {}: {}", ++ descriptor.device_driver.compatible(), ++ x ++ ); ++ } ++ } ++ }); ++ } ++ ++ /// Enumerate all registered device drivers. ++ pub fn enumerate(&self) { ++ let mut i: usize = 1; ++ self.for_each_descriptor(|descriptor| { ++ println!(" {}. {}", i, descriptor.device_driver.compatible()); ++ ++ i += 1; ++ }); + } +} @@ -1339,21 +1607,20 @@ diff -uNr 04_safe_globals/src/main.rs 05_drivers_gpio_uart/src/main.rs mod panic_wait; mod print; mod synchronization; -@@ -125,13 +127,50 @@ +@@ -125,13 +127,42 @@ /// # Safety /// /// - Only a single core must be active and running this function. +/// - The init calls in this function must appear in the correct order. unsafe fn kernel_init() -> ! { - use console::console; -+ use driver::interface::DriverManager; -+ -+ for i in bsp::driver::driver_manager().all_device_drivers().iter() { -+ if let Err(x) = i.init() { -+ panic!("Error loading driver: {}: {}", i.compatible(), x); -+ } ++ // Initialize the BSP driver subsystem. ++ if let Err(x) = bsp::driver::init() { ++ panic!("Error initializing BSP driver subsystem: {}", x); + } -+ bsp::driver::driver_manager().post_device_driver_init(); ++ ++ // Initialize all device drivers. ++ driver::driver_manager().init_drivers(); + // println! is usable from here on. - println!("[0] Hello from Rust!"); @@ -1365,7 +1632,6 @@ diff -uNr 04_safe_globals/src/main.rs 05_drivers_gpio_uart/src/main.rs +/// The main function running after the early init. +fn kernel_main() -> ! { + use console::console; -+ use driver::interface::DriverManager; - println!("[2] Stopping here."); - cpu::wait_forever() @@ -1377,13 +1643,7 @@ diff -uNr 04_safe_globals/src/main.rs 05_drivers_gpio_uart/src/main.rs + println!("[1] Booting on: {}", bsp::board_name()); + + println!("[2] Drivers loaded:"); -+ for (i, driver) in bsp::driver::driver_manager() -+ .all_device_drivers() -+ .iter() -+ .enumerate() -+ { -+ println!(" {}. {}", i + 1, driver.compatible()); -+ } ++ driver::driver_manager().enumerate(); + + println!("[3] Chars written: {}", console().chars_written()); + println!("[4] Echoing input now"); diff --git a/05_drivers_gpio_uart/src/bsp/raspberrypi.rs b/05_drivers_gpio_uart/src/bsp/raspberrypi.rs index a9a7261a..fe940677 100644 --- a/05_drivers_gpio_uart/src/bsp/raspberrypi.rs +++ b/05_drivers_gpio_uart/src/bsp/raspberrypi.rs @@ -4,7 +4,6 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -pub mod console; pub mod cpu; pub mod driver; pub mod memory; diff --git a/05_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs b/05_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs index 8b683ed8..ea843066 100644 --- a/05_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs +++ b/05_drivers_gpio_uart/src/bsp/raspberrypi/driver.rs @@ -5,51 +5,67 @@ //! BSP driver support. use super::memory::map::mmio; -use crate::{bsp::device_driver, driver}; +use crate::{bsp::device_driver, console, driver as generic_driver}; +use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- -// Private Definitions +// Global instances //-------------------------------------------------------------------------------------------------- -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: [&'static (dyn DeviceDriver + Sync); 2], -} +static PL011_UART: device_driver::PL011Uart = + unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; //-------------------------------------------------------------------------------------------------- -// Global instances +// Private Code //-------------------------------------------------------------------------------------------------- -pub(super) static PL011_UART: device_driver::PL011Uart = - unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +/// This must be called only after successful init of the UART driver. +fn post_init_uart() -> Result<(), &'static str> { + console::register_console(&PL011_UART); -static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + Ok(()) +} -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: [&PL011_UART, &GPIO], -}; +/// This must be called only after successful init of the GPIO driver. +fn post_init_gpio() -> Result<(), &'static str> { + GPIO.map_pl011_uart(); + Ok(()) +} -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +fn driver_uart() -> Result<(), &'static str> { + let uart_descriptor = + generic_driver::DeviceDriverDescriptor::new(&PL011_UART, Some(post_init_uart)); + generic_driver::driver_manager().register_driver(uart_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +fn driver_gpio() -> Result<(), &'static str> { + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio)); + generic_driver::driver_manager().register_driver(gpio_descriptor); -impl driver::interface::DriverManager for BSPDriverManager { - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { - &self.device_drivers[..] - } + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - fn post_device_driver_init(&self) { - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } + + driver_uart()?; + driver_gpio()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) } diff --git a/05_drivers_gpio_uart/src/console.rs b/05_drivers_gpio_uart/src/console.rs index c1fb0e53..02b43df9 100644 --- a/05_drivers_gpio_uart/src/console.rs +++ b/05_drivers_gpio_uart/src/console.rs @@ -4,7 +4,9 @@ //! System console. -use crate::bsp; +mod null_console; + +use crate::synchronization::{self, NullLock}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -54,13 +56,26 @@ pub mod interface { pub trait All: Write + Read + Statistics {} } +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static CUR_CONSOLE: NullLock<&'static (dyn interface::All + Sync)> = + NullLock::new(&null_console::NULL_CONSOLE); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +use synchronization::interface::Mutex; + +/// Register a new console. +pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { + CUR_CONSOLE.lock(|con| *con = new_console); +} -/// Return a reference to the console. +/// Return a reference to the currently registered console. /// /// This is the global console used by all printing macros. pub fn console() -> &'static dyn interface::All { - bsp::console::console() + CUR_CONSOLE.lock(|con| *con) } diff --git a/05_drivers_gpio_uart/src/console/null_console.rs b/05_drivers_gpio_uart/src/console/null_console.rs new file mode 100644 index 00000000..2c64d499 --- /dev/null +++ b/05_drivers_gpio_uart/src/console/null_console.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter + +//! Null console. + +use super::interface; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct NullConsole; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +pub static NULL_CONSOLE: NullConsole = NullConsole {}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl interface::Write for NullConsole { + fn write_char(&self, _c: char) {} + + fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { + fmt::Result::Ok(()) + } + + fn flush(&self) {} +} + +impl interface::Read for NullConsole { + fn clear_rx(&self) {} +} + +impl interface::Statistics for NullConsole {} +impl interface::All for NullConsole {} diff --git a/05_drivers_gpio_uart/src/driver.rs b/05_drivers_gpio_uart/src/driver.rs index 2fcc7562..e324ecf8 100644 --- a/05_drivers_gpio_uart/src/driver.rs +++ b/05_drivers_gpio_uart/src/driver.rs @@ -4,6 +4,22 @@ //! Driver support. +use crate::{ + println, + synchronization::{interface::Mutex, NullLock}, +}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner { + next_index: usize, + descriptors: [Option; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -24,21 +40,128 @@ pub mod interface { Ok(()) } } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor { + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager { + inner: NullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner { + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], + } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager { + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: NullLock::new(DriverManagerInner::new()), + } + } - /// Device driver management functions. + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.lock(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.lock(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Return a slice of references to all `BSP`-instantiated drivers. - /// - /// # Safety - /// - /// - The order of devices is the order in which `DeviceDriver::init()` is called. - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } - /// Initialization code that runs after driver init. - /// - /// For example, device driver code that depends on other drivers already being online. - fn post_device_driver_init(&self); + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } + + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + println!(" {}. {}", i, descriptor.device_driver.compatible()); + + i += 1; + }); } } diff --git a/05_drivers_gpio_uart/src/main.rs b/05_drivers_gpio_uart/src/main.rs index c9b73382..9d158238 100644 --- a/05_drivers_gpio_uart/src/main.rs +++ b/05_drivers_gpio_uart/src/main.rs @@ -129,14 +129,13 @@ mod synchronization; /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order. unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - bsp::driver::driver_manager().post_device_driver_init(); + + // Initialize all device drivers. + driver::driver_manager().init_drivers(); // println! is usable from here on. // Transition from unsafe to safe. @@ -146,7 +145,6 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { use console::console; - use driver::interface::DriverManager; println!( "[0] {} version {}", @@ -156,13 +154,7 @@ fn kernel_main() -> ! { println!("[1] Booting on: {}", bsp::board_name()); println!("[2] Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - println!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); println!("[3] Chars written: {}", console().chars_written()); println!("[4] Echoing input now"); diff --git a/06_uart_chainloader/README.md b/06_uart_chainloader/README.md index 6e4e4530..e321e603 100644 --- a/06_uart_chainloader/README.md +++ b/06_uart_chainloader/README.md @@ -369,6 +369,27 @@ diff -uNr 05_drivers_gpio_uart/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 0 {} } +diff -uNr 05_drivers_gpio_uart/src/bsp/raspberrypi/console.rs 06_uart_chainloader/src/bsp/raspberrypi/console.rs +--- 05_drivers_gpio_uart/src/bsp/raspberrypi/console.rs ++++ 06_uart_chainloader/src/bsp/raspberrypi/console.rs +@@ -1,16 +0,0 @@ +-// SPDX-License-Identifier: MIT OR Apache-2.0 +-// +-// Copyright (c) 2018-2022 Andre Richter +- +-//! BSP console facilities. +- +-use crate::console; +- +-//-------------------------------------------------------------------------------------------------- +-// Public Code +-//-------------------------------------------------------------------------------------------------- +- +-/// Return a reference to the console. +-pub fn console() -> &'static dyn console::interface::All { +- &super::driver::PL011_UART +-} + diff -uNr 05_drivers_gpio_uart/src/bsp/raspberrypi/kernel.ld 06_uart_chainloader/src/bsp/raspberrypi/kernel.ld --- 05_drivers_gpio_uart/src/bsp/raspberrypi/kernel.ld +++ 06_uart_chainloader/src/bsp/raspberrypi/kernel.ld @@ -437,10 +458,41 @@ diff -uNr 05_drivers_gpio_uart/src/bsp/raspberrypi/memory.rs 06_uart_chainloader + map::BOARD_DEFAULT_LOAD_ADDRESS as _ +} +diff -uNr 05_drivers_gpio_uart/src/driver.rs 06_uart_chainloader/src/driver.rs +--- 05_drivers_gpio_uart/src/driver.rs ++++ 06_uart_chainloader/src/driver.rs +@@ -4,10 +4,7 @@ + + //! Driver support. + +-use crate::{ +- println, +- synchronization::{interface::Mutex, NullLock}, +-}; ++use crate::synchronization::{interface::Mutex, NullLock}; + + //-------------------------------------------------------------------------------------------------- + // Private Definitions +@@ -154,14 +151,4 @@ + } + }); + } +- +- /// Enumerate all registered device drivers. +- pub fn enumerate(&self) { +- let mut i: usize = 1; +- self.for_each_descriptor(|descriptor| { +- println!(" {}. {}", i, descriptor.device_driver.compatible()); +- +- i += 1; +- }); +- } + } + diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs --- 05_drivers_gpio_uart/src/main.rs +++ 06_uart_chainloader/src/main.rs -@@ -143,34 +143,55 @@ +@@ -142,27 +142,55 @@ kernel_main() } @@ -454,7 +506,6 @@ diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs /// The main function running after the early init. fn kernel_main() -> ! { use console::console; -- use driver::interface::DriverManager; - println!( - "[0] {} version {}", @@ -462,41 +513,35 @@ diff -uNr 05_drivers_gpio_uart/src/main.rs 06_uart_chainloader/src/main.rs - env!("CARGO_PKG_VERSION") - ); - println!("[1] Booting on: {}", bsp::board_name()); -- -- println!("[2] Drivers loaded:"); -- for (i, driver) in bsp::driver::driver_manager() -- .all_device_drivers() -- .iter() -- .enumerate() -- { -- println!(" {}. {}", i + 1, driver.compatible()); + println!("{}", MINILOAD_LOGO); + println!("{:^37}", bsp::board_name()); + println!(); + println!("[ML] Requesting binary"); + console().flush(); -+ + +- println!("[2] Drivers loaded:"); +- driver::driver_manager().enumerate(); + // Discard any spurious received characters before starting with the loader protocol. + console().clear_rx(); -+ -+ // Notify `Minipush` to send the binary. -+ for _ in 0..3 { -+ console().write_char(3 as char); - } - println!("[3] Chars written: {}", console().chars_written()); - println!("[4] Echoing input now"); -+ // Read the binary's size. -+ let mut size: u32 = u32::from(console().read_char() as u8); -+ size |= u32::from(console().read_char() as u8) << 8; -+ size |= u32::from(console().read_char() as u8) << 16; -+ size |= u32::from(console().read_char() as u8) << 24; ++ // Notify `Minipush` to send the binary. ++ for _ in 0..3 { ++ console().write_char(3 as char); ++ } - // Discard any spurious received characters before going into echo mode. - console().clear_rx(); - loop { - let c = console().read_char(); - console().write_char(c); ++ // Read the binary's size. ++ let mut size: u32 = u32::from(console().read_char() as u8); ++ size |= u32::from(console().read_char() as u8) << 8; ++ size |= u32::from(console().read_char() as u8) << 16; ++ size |= u32::from(console().read_char() as u8) << 24; ++ + // Trust it's not too big. + console().write_char('O'); + console().write_char('K'); diff --git a/06_uart_chainloader/src/bsp/raspberrypi.rs b/06_uart_chainloader/src/bsp/raspberrypi.rs index a9a7261a..fe940677 100644 --- a/06_uart_chainloader/src/bsp/raspberrypi.rs +++ b/06_uart_chainloader/src/bsp/raspberrypi.rs @@ -4,7 +4,6 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -pub mod console; pub mod cpu; pub mod driver; pub mod memory; diff --git a/06_uart_chainloader/src/bsp/raspberrypi/console.rs b/06_uart_chainloader/src/bsp/raspberrypi/console.rs deleted file mode 100644 index 0a630eef..00000000 --- a/06_uart_chainloader/src/bsp/raspberrypi/console.rs +++ /dev/null @@ -1,16 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2022 Andre Richter - -//! BSP console facilities. - -use crate::console; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return a reference to the console. -pub fn console() -> &'static dyn console::interface::All { - &super::driver::PL011_UART -} diff --git a/06_uart_chainloader/src/bsp/raspberrypi/driver.rs b/06_uart_chainloader/src/bsp/raspberrypi/driver.rs index 8b683ed8..ea843066 100644 --- a/06_uart_chainloader/src/bsp/raspberrypi/driver.rs +++ b/06_uart_chainloader/src/bsp/raspberrypi/driver.rs @@ -5,51 +5,67 @@ //! BSP driver support. use super::memory::map::mmio; -use crate::{bsp::device_driver, driver}; +use crate::{bsp::device_driver, console, driver as generic_driver}; +use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- -// Private Definitions +// Global instances //-------------------------------------------------------------------------------------------------- -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: [&'static (dyn DeviceDriver + Sync); 2], -} +static PL011_UART: device_driver::PL011Uart = + unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; //-------------------------------------------------------------------------------------------------- -// Global instances +// Private Code //-------------------------------------------------------------------------------------------------- -pub(super) static PL011_UART: device_driver::PL011Uart = - unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +/// This must be called only after successful init of the UART driver. +fn post_init_uart() -> Result<(), &'static str> { + console::register_console(&PL011_UART); -static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + Ok(()) +} -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: [&PL011_UART, &GPIO], -}; +/// This must be called only after successful init of the GPIO driver. +fn post_init_gpio() -> Result<(), &'static str> { + GPIO.map_pl011_uart(); + Ok(()) +} -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +fn driver_uart() -> Result<(), &'static str> { + let uart_descriptor = + generic_driver::DeviceDriverDescriptor::new(&PL011_UART, Some(post_init_uart)); + generic_driver::driver_manager().register_driver(uart_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +fn driver_gpio() -> Result<(), &'static str> { + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio)); + generic_driver::driver_manager().register_driver(gpio_descriptor); -impl driver::interface::DriverManager for BSPDriverManager { - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { - &self.device_drivers[..] - } + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - fn post_device_driver_init(&self) { - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } + + driver_uart()?; + driver_gpio()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) } diff --git a/06_uart_chainloader/src/console.rs b/06_uart_chainloader/src/console.rs index c1fb0e53..02b43df9 100644 --- a/06_uart_chainloader/src/console.rs +++ b/06_uart_chainloader/src/console.rs @@ -4,7 +4,9 @@ //! System console. -use crate::bsp; +mod null_console; + +use crate::synchronization::{self, NullLock}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -54,13 +56,26 @@ pub mod interface { pub trait All: Write + Read + Statistics {} } +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static CUR_CONSOLE: NullLock<&'static (dyn interface::All + Sync)> = + NullLock::new(&null_console::NULL_CONSOLE); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +use synchronization::interface::Mutex; + +/// Register a new console. +pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { + CUR_CONSOLE.lock(|con| *con = new_console); +} -/// Return a reference to the console. +/// Return a reference to the currently registered console. /// /// This is the global console used by all printing macros. pub fn console() -> &'static dyn interface::All { - bsp::console::console() + CUR_CONSOLE.lock(|con| *con) } diff --git a/06_uart_chainloader/src/console/null_console.rs b/06_uart_chainloader/src/console/null_console.rs new file mode 100644 index 00000000..2c64d499 --- /dev/null +++ b/06_uart_chainloader/src/console/null_console.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter + +//! Null console. + +use super::interface; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct NullConsole; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +pub static NULL_CONSOLE: NullConsole = NullConsole {}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl interface::Write for NullConsole { + fn write_char(&self, _c: char) {} + + fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { + fmt::Result::Ok(()) + } + + fn flush(&self) {} +} + +impl interface::Read for NullConsole { + fn clear_rx(&self) {} +} + +impl interface::Statistics for NullConsole {} +impl interface::All for NullConsole {} diff --git a/06_uart_chainloader/src/driver.rs b/06_uart_chainloader/src/driver.rs index 2fcc7562..fb44bbd9 100644 --- a/06_uart_chainloader/src/driver.rs +++ b/06_uart_chainloader/src/driver.rs @@ -4,6 +4,19 @@ //! Driver support. +use crate::synchronization::{interface::Mutex, NullLock}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner { + next_index: usize, + descriptors: [Option; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -24,21 +37,118 @@ pub mod interface { Ok(()) } } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; - /// Device driver management functions. +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor { + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager { + inner: NullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner { + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], + } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager { + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: NullLock::new(DriverManagerInner::new()), + } + } + + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.lock(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.lock(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Return a slice of references to all `BSP`-instantiated drivers. - /// - /// # Safety - /// - /// - The order of devices is the order in which `DeviceDriver::init()` is called. - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } - /// Initialization code that runs after driver init. - /// - /// For example, device driver code that depends on other drivers already being online. - fn post_device_driver_init(&self); + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); } } diff --git a/06_uart_chainloader/src/main.rs b/06_uart_chainloader/src/main.rs index 08bd9ad9..bef62822 100644 --- a/06_uart_chainloader/src/main.rs +++ b/06_uart_chainloader/src/main.rs @@ -129,14 +129,13 @@ mod synchronization; /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order. unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - bsp::driver::driver_manager().post_device_driver_init(); + + // Initialize all device drivers. + driver::driver_manager().init_drivers(); // println! is usable from here on. // Transition from unsafe to safe. diff --git a/07_timestamps/README.md b/07_timestamps/README.md index 2e8df3c2..4a5905b4 100644 --- a/07_timestamps/README.md +++ b/07_timestamps/README.md @@ -501,6 +501,28 @@ diff -uNr 06_uart_chainloader/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 07 {} } +diff -uNr 06_uart_chainloader/src/bsp/raspberrypi/driver.rs 07_timestamps/src/bsp/raspberrypi/driver.rs +--- 06_uart_chainloader/src/bsp/raspberrypi/driver.rs ++++ 07_timestamps/src/bsp/raspberrypi/driver.rs +@@ -57,6 +57,17 @@ + /// # Safety + /// + /// See child function calls. ++/// ++/// # Note ++/// ++/// Using atomics here relieves us from needing to use `unsafe` for the static variable. ++/// ++/// On `AArch64`, which is the only implemented architecture at the time of writing this, ++/// [`AtomicBool::load`] and [`AtomicBool::store`] are lowered to ordinary load and store ++/// instructions. They are therefore safe to use even with MMU + caching deactivated. ++/// ++/// [`AtomicBool::load`]: core::sync::atomic::AtomicBool::load ++/// [`AtomicBool::store`]: core::sync::atomic::AtomicBool::store + pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + diff -uNr 06_uart_chainloader/src/bsp/raspberrypi/kernel.ld 07_timestamps/src/bsp/raspberrypi/kernel.ld --- 06_uart_chainloader/src/bsp/raspberrypi/kernel.ld +++ 07_timestamps/src/bsp/raspberrypi/kernel.ld @@ -580,6 +602,37 @@ diff -uNr 06_uart_chainloader/src/cpu.rs 07_timestamps/src/cpu.rs -#[cfg(feature = "bsp_rpi3")] -pub use arch_cpu::spin_for_cycles; +diff -uNr 06_uart_chainloader/src/driver.rs 07_timestamps/src/driver.rs +--- 06_uart_chainloader/src/driver.rs ++++ 07_timestamps/src/driver.rs +@@ -4,7 +4,10 @@ + + //! Driver support. + +-use crate::synchronization::{interface::Mutex, NullLock}; ++use crate::{ ++ info, ++ synchronization::{interface::Mutex, NullLock}, ++}; + + //-------------------------------------------------------------------------------------------------- + // Private Definitions +@@ -151,4 +154,14 @@ + } + }); + } ++ ++ /// Enumerate all registered device drivers. ++ pub fn enumerate(&self) { ++ let mut i: usize = 1; ++ self.for_each_descriptor(|descriptor| { ++ info!(" {}. {}", i, descriptor.device_driver.compatible()); ++ ++ i += 1; ++ }); ++ } + } + diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs --- 06_uart_chainloader/src/main.rs +++ 07_timestamps/src/main.rs @@ -604,7 +657,7 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs /// Early init code. /// -@@ -143,55 +147,37 @@ +@@ -142,55 +146,30 @@ kernel_main() } @@ -618,9 +671,7 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs /// The main function running after the early init. fn kernel_main() -> ! { - use console::console; -+ use core::time::Duration; -+ use driver::interface::DriverManager; - +- - println!("{}", MINILOAD_LOGO); - println!("{:^37}", bsp::board_name()); - println!(); @@ -633,26 +684,8 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs - // Notify `Minipush` to send the binary. - for _ in 0..3 { - console().write_char(3 as char); -+ info!( -+ "{} version {}", -+ env!("CARGO_PKG_NAME"), -+ env!("CARGO_PKG_VERSION") -+ ); -+ info!("Booting on: {}", bsp::board_name()); -+ -+ info!( -+ "Architectural timer resolution: {} ns", -+ time::time_manager().resolution().as_nanos() -+ ); -+ -+ info!("Drivers loaded:"); -+ for (i, driver) in bsp::driver::driver_manager() -+ .all_device_drivers() -+ .iter() -+ .enumerate() -+ { -+ info!(" {}. {}", i + 1, driver.compatible()); - } +- } ++ use core::time::Duration; - // Read the binary's size. - let mut size: u32 = u32::from(console().read_char() as u8); @@ -670,10 +703,29 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs - for i in 0..size { - core::ptr::write_volatile(kernel_addr.offset(i as isize), console().read_char() as u8) - } -- } ++ info!( ++ "{} version {}", ++ env!("CARGO_PKG_NAME"), ++ env!("CARGO_PKG_VERSION") ++ ); ++ info!("Booting on: {}", bsp::board_name()); ++ ++ info!( ++ "Architectural timer resolution: {} ns", ++ time::time_manager().resolution().as_nanos() ++ ); ++ ++ info!("Drivers loaded:"); ++ driver::driver_manager().enumerate(); ++ + // Test a failing timer case. + time::time_manager().spin_for(Duration::from_nanos(1)); - ++ ++ loop { ++ info!("Spinning for 1 second"); ++ time::time_manager().spin_for(Duration::from_secs(1)); + } +- - println!("[ML] Loaded! Executing the payload now\n"); - console().flush(); - @@ -682,10 +734,6 @@ diff -uNr 06_uart_chainloader/src/main.rs 07_timestamps/src/main.rs - - // Jump to loaded kernel! - kernel() -+ loop { -+ info!("Spinning for 1 second"); -+ time::time_manager().spin_for(Duration::from_secs(1)); -+ } } diff -uNr 06_uart_chainloader/src/panic_wait.rs 07_timestamps/src/panic_wait.rs diff --git a/07_timestamps/src/bsp/raspberrypi.rs b/07_timestamps/src/bsp/raspberrypi.rs index a9a7261a..fe940677 100644 --- a/07_timestamps/src/bsp/raspberrypi.rs +++ b/07_timestamps/src/bsp/raspberrypi.rs @@ -4,7 +4,6 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -pub mod console; pub mod cpu; pub mod driver; pub mod memory; diff --git a/07_timestamps/src/bsp/raspberrypi/console.rs b/07_timestamps/src/bsp/raspberrypi/console.rs deleted file mode 100644 index 0a630eef..00000000 --- a/07_timestamps/src/bsp/raspberrypi/console.rs +++ /dev/null @@ -1,16 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2022 Andre Richter - -//! BSP console facilities. - -use crate::console; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return a reference to the console. -pub fn console() -> &'static dyn console::interface::All { - &super::driver::PL011_UART -} diff --git a/07_timestamps/src/bsp/raspberrypi/driver.rs b/07_timestamps/src/bsp/raspberrypi/driver.rs index 8b683ed8..4a42b84f 100644 --- a/07_timestamps/src/bsp/raspberrypi/driver.rs +++ b/07_timestamps/src/bsp/raspberrypi/driver.rs @@ -5,51 +5,78 @@ //! BSP driver support. use super::memory::map::mmio; -use crate::{bsp::device_driver, driver}; +use crate::{bsp::device_driver, console, driver as generic_driver}; +use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- -// Private Definitions +// Global instances //-------------------------------------------------------------------------------------------------- -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: [&'static (dyn DeviceDriver + Sync); 2], -} +static PL011_UART: device_driver::PL011Uart = + unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; //-------------------------------------------------------------------------------------------------- -// Global instances +// Private Code //-------------------------------------------------------------------------------------------------- -pub(super) static PL011_UART: device_driver::PL011Uart = - unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +/// This must be called only after successful init of the UART driver. +fn post_init_uart() -> Result<(), &'static str> { + console::register_console(&PL011_UART); -static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + Ok(()) +} -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: [&PL011_UART, &GPIO], -}; +/// This must be called only after successful init of the GPIO driver. +fn post_init_gpio() -> Result<(), &'static str> { + GPIO.map_pl011_uart(); + Ok(()) +} -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +fn driver_uart() -> Result<(), &'static str> { + let uart_descriptor = + generic_driver::DeviceDriverDescriptor::new(&PL011_UART, Some(post_init_uart)); + generic_driver::driver_manager().register_driver(uart_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +fn driver_gpio() -> Result<(), &'static str> { + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio)); + generic_driver::driver_manager().register_driver(gpio_descriptor); -impl driver::interface::DriverManager for BSPDriverManager { - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { - &self.device_drivers[..] - } + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - fn post_device_driver_init(&self) { - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +/// +/// # Note +/// +/// Using atomics here relieves us from needing to use `unsafe` for the static variable. +/// +/// On `AArch64`, which is the only implemented architecture at the time of writing this, +/// [`AtomicBool::load`] and [`AtomicBool::store`] are lowered to ordinary load and store +/// instructions. They are therefore safe to use even with MMU + caching deactivated. +/// +/// [`AtomicBool::load`]: core::sync::atomic::AtomicBool::load +/// [`AtomicBool::store`]: core::sync::atomic::AtomicBool::store +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } + + driver_uart()?; + driver_gpio()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) } diff --git a/07_timestamps/src/console.rs b/07_timestamps/src/console.rs index c1fb0e53..02b43df9 100644 --- a/07_timestamps/src/console.rs +++ b/07_timestamps/src/console.rs @@ -4,7 +4,9 @@ //! System console. -use crate::bsp; +mod null_console; + +use crate::synchronization::{self, NullLock}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -54,13 +56,26 @@ pub mod interface { pub trait All: Write + Read + Statistics {} } +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static CUR_CONSOLE: NullLock<&'static (dyn interface::All + Sync)> = + NullLock::new(&null_console::NULL_CONSOLE); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +use synchronization::interface::Mutex; + +/// Register a new console. +pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { + CUR_CONSOLE.lock(|con| *con = new_console); +} -/// Return a reference to the console. +/// Return a reference to the currently registered console. /// /// This is the global console used by all printing macros. pub fn console() -> &'static dyn interface::All { - bsp::console::console() + CUR_CONSOLE.lock(|con| *con) } diff --git a/07_timestamps/src/console/null_console.rs b/07_timestamps/src/console/null_console.rs new file mode 100644 index 00000000..2c64d499 --- /dev/null +++ b/07_timestamps/src/console/null_console.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter + +//! Null console. + +use super::interface; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct NullConsole; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +pub static NULL_CONSOLE: NullConsole = NullConsole {}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl interface::Write for NullConsole { + fn write_char(&self, _c: char) {} + + fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { + fmt::Result::Ok(()) + } + + fn flush(&self) {} +} + +impl interface::Read for NullConsole { + fn clear_rx(&self) {} +} + +impl interface::Statistics for NullConsole {} +impl interface::All for NullConsole {} diff --git a/07_timestamps/src/driver.rs b/07_timestamps/src/driver.rs index 2fcc7562..a798c86d 100644 --- a/07_timestamps/src/driver.rs +++ b/07_timestamps/src/driver.rs @@ -4,6 +4,22 @@ //! Driver support. +use crate::{ + info, + synchronization::{interface::Mutex, NullLock}, +}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner { + next_index: usize, + descriptors: [Option; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -24,21 +40,128 @@ pub mod interface { Ok(()) } } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor { + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager { + inner: NullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner { + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], + } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager { + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: NullLock::new(DriverManagerInner::new()), + } + } - /// Device driver management functions. + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.lock(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.lock(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Return a slice of references to all `BSP`-instantiated drivers. - /// - /// # Safety - /// - /// - The order of devices is the order in which `DeviceDriver::init()` is called. - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } - /// Initialization code that runs after driver init. - /// - /// For example, device driver code that depends on other drivers already being online. - fn post_device_driver_init(&self); + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } + + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); + + i += 1; + }); } } diff --git a/07_timestamps/src/main.rs b/07_timestamps/src/main.rs index 90c39a37..df863f32 100644 --- a/07_timestamps/src/main.rs +++ b/07_timestamps/src/main.rs @@ -133,14 +133,13 @@ mod time; /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order. unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - bsp::driver::driver_manager().post_device_driver_init(); + + // Initialize all device drivers. + driver::driver_manager().init_drivers(); // println! is usable from here on. // Transition from unsafe to safe. @@ -150,7 +149,6 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { use core::time::Duration; - use driver::interface::DriverManager; info!( "{} version {}", @@ -165,13 +163,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); // Test a failing timer case. time::time_manager().spin_for(Duration::from_nanos(1)); diff --git a/08_hw_debug_JTAG/README.md b/08_hw_debug_JTAG/README.md index 53f86807..69b6b82e 100644 --- a/08_hw_debug_JTAG/README.md +++ b/08_hw_debug_JTAG/README.md @@ -401,4 +401,26 @@ diff -uNr 07_timestamps/Makefile 08_hw_debug_JTAG/Makefile ## Testing targets ##-------------------------------------------------------------------------------------------------- +diff -uNr 07_timestamps/src/bsp/raspberrypi/driver.rs 08_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs +--- 07_timestamps/src/bsp/raspberrypi/driver.rs ++++ 08_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs +@@ -57,17 +57,6 @@ + /// # Safety + /// + /// See child function calls. +-/// +-/// # Note +-/// +-/// Using atomics here relieves us from needing to use `unsafe` for the static variable. +-/// +-/// On `AArch64`, which is the only implemented architecture at the time of writing this, +-/// [`AtomicBool::load`] and [`AtomicBool::store`] are lowered to ordinary load and store +-/// instructions. They are therefore safe to use even with MMU + caching deactivated. +-/// +-/// [`AtomicBool::load`]: core::sync::atomic::AtomicBool::load +-/// [`AtomicBool::store`]: core::sync::atomic::AtomicBool::store + pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + ``` diff --git a/08_hw_debug_JTAG/src/bsp/raspberrypi.rs b/08_hw_debug_JTAG/src/bsp/raspberrypi.rs index a9a7261a..fe940677 100644 --- a/08_hw_debug_JTAG/src/bsp/raspberrypi.rs +++ b/08_hw_debug_JTAG/src/bsp/raspberrypi.rs @@ -4,7 +4,6 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -pub mod console; pub mod cpu; pub mod driver; pub mod memory; diff --git a/08_hw_debug_JTAG/src/bsp/raspberrypi/console.rs b/08_hw_debug_JTAG/src/bsp/raspberrypi/console.rs deleted file mode 100644 index 0a630eef..00000000 --- a/08_hw_debug_JTAG/src/bsp/raspberrypi/console.rs +++ /dev/null @@ -1,16 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2022 Andre Richter - -//! BSP console facilities. - -use crate::console; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return a reference to the console. -pub fn console() -> &'static dyn console::interface::All { - &super::driver::PL011_UART -} diff --git a/08_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs b/08_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs index 8b683ed8..ea843066 100644 --- a/08_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs +++ b/08_hw_debug_JTAG/src/bsp/raspberrypi/driver.rs @@ -5,51 +5,67 @@ //! BSP driver support. use super::memory::map::mmio; -use crate::{bsp::device_driver, driver}; +use crate::{bsp::device_driver, console, driver as generic_driver}; +use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- -// Private Definitions +// Global instances //-------------------------------------------------------------------------------------------------- -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: [&'static (dyn DeviceDriver + Sync); 2], -} +static PL011_UART: device_driver::PL011Uart = + unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; //-------------------------------------------------------------------------------------------------- -// Global instances +// Private Code //-------------------------------------------------------------------------------------------------- -pub(super) static PL011_UART: device_driver::PL011Uart = - unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +/// This must be called only after successful init of the UART driver. +fn post_init_uart() -> Result<(), &'static str> { + console::register_console(&PL011_UART); -static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + Ok(()) +} -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: [&PL011_UART, &GPIO], -}; +/// This must be called only after successful init of the GPIO driver. +fn post_init_gpio() -> Result<(), &'static str> { + GPIO.map_pl011_uart(); + Ok(()) +} -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +fn driver_uart() -> Result<(), &'static str> { + let uart_descriptor = + generic_driver::DeviceDriverDescriptor::new(&PL011_UART, Some(post_init_uart)); + generic_driver::driver_manager().register_driver(uart_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +fn driver_gpio() -> Result<(), &'static str> { + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio)); + generic_driver::driver_manager().register_driver(gpio_descriptor); -impl driver::interface::DriverManager for BSPDriverManager { - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { - &self.device_drivers[..] - } + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - fn post_device_driver_init(&self) { - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } + + driver_uart()?; + driver_gpio()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) } diff --git a/08_hw_debug_JTAG/src/console.rs b/08_hw_debug_JTAG/src/console.rs index c1fb0e53..02b43df9 100644 --- a/08_hw_debug_JTAG/src/console.rs +++ b/08_hw_debug_JTAG/src/console.rs @@ -4,7 +4,9 @@ //! System console. -use crate::bsp; +mod null_console; + +use crate::synchronization::{self, NullLock}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -54,13 +56,26 @@ pub mod interface { pub trait All: Write + Read + Statistics {} } +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static CUR_CONSOLE: NullLock<&'static (dyn interface::All + Sync)> = + NullLock::new(&null_console::NULL_CONSOLE); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +use synchronization::interface::Mutex; + +/// Register a new console. +pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { + CUR_CONSOLE.lock(|con| *con = new_console); +} -/// Return a reference to the console. +/// Return a reference to the currently registered console. /// /// This is the global console used by all printing macros. pub fn console() -> &'static dyn interface::All { - bsp::console::console() + CUR_CONSOLE.lock(|con| *con) } diff --git a/08_hw_debug_JTAG/src/console/null_console.rs b/08_hw_debug_JTAG/src/console/null_console.rs new file mode 100644 index 00000000..2c64d499 --- /dev/null +++ b/08_hw_debug_JTAG/src/console/null_console.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter + +//! Null console. + +use super::interface; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct NullConsole; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +pub static NULL_CONSOLE: NullConsole = NullConsole {}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl interface::Write for NullConsole { + fn write_char(&self, _c: char) {} + + fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { + fmt::Result::Ok(()) + } + + fn flush(&self) {} +} + +impl interface::Read for NullConsole { + fn clear_rx(&self) {} +} + +impl interface::Statistics for NullConsole {} +impl interface::All for NullConsole {} diff --git a/08_hw_debug_JTAG/src/driver.rs b/08_hw_debug_JTAG/src/driver.rs index 2fcc7562..a798c86d 100644 --- a/08_hw_debug_JTAG/src/driver.rs +++ b/08_hw_debug_JTAG/src/driver.rs @@ -4,6 +4,22 @@ //! Driver support. +use crate::{ + info, + synchronization::{interface::Mutex, NullLock}, +}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner { + next_index: usize, + descriptors: [Option; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -24,21 +40,128 @@ pub mod interface { Ok(()) } } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor { + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager { + inner: NullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner { + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], + } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager { + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: NullLock::new(DriverManagerInner::new()), + } + } - /// Device driver management functions. + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.lock(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.lock(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Return a slice of references to all `BSP`-instantiated drivers. - /// - /// # Safety - /// - /// - The order of devices is the order in which `DeviceDriver::init()` is called. - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } - /// Initialization code that runs after driver init. - /// - /// For example, device driver code that depends on other drivers already being online. - fn post_device_driver_init(&self); + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } + + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); + + i += 1; + }); } } diff --git a/08_hw_debug_JTAG/src/main.rs b/08_hw_debug_JTAG/src/main.rs index 90c39a37..df863f32 100644 --- a/08_hw_debug_JTAG/src/main.rs +++ b/08_hw_debug_JTAG/src/main.rs @@ -133,14 +133,13 @@ mod time; /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order. unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - bsp::driver::driver_manager().post_device_driver_init(); + + // Initialize all device drivers. + driver::driver_manager().init_drivers(); // println! is usable from here on. // Transition from unsafe to safe. @@ -150,7 +149,6 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { use core::time::Duration; - use driver::interface::DriverManager; info!( "{} version {}", @@ -165,13 +163,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); // Test a failing timer case. time::time_manager().spin_for(Duration::from_nanos(1)); diff --git a/09_privilege_level/README.md b/09_privilege_level/README.md index 7f7c58d1..5710fdb8 100644 --- a/09_privilege_level/README.md +++ b/09_privilege_level/README.md @@ -519,15 +519,15 @@ diff -uNr 08_hw_debug_JTAG/src/main.rs 09_privilege_level/src/main.rs mod panic_wait; mod print; mod synchronization; -@@ -149,6 +150,7 @@ +@@ -148,6 +149,7 @@ /// The main function running after the early init. fn kernel_main() -> ! { + use console::console; use core::time::Duration; - use driver::interface::DriverManager; -@@ -159,6 +161,12 @@ + info!( +@@ -157,6 +159,12 @@ ); info!("Booting on: {}", bsp::board_name()); @@ -540,9 +540,9 @@ diff -uNr 08_hw_debug_JTAG/src/main.rs 09_privilege_level/src/main.rs info!( "Architectural timer resolution: {} ns", time::time_manager().resolution().as_nanos() -@@ -173,11 +181,15 @@ - info!(" {}. {}", i + 1, driver.compatible()); - } +@@ -165,11 +173,15 @@ + info!("Drivers loaded:"); + driver::driver_manager().enumerate(); - // Test a failing timer case. - time::time_manager().spin_for(Duration::from_nanos(1)); diff --git a/09_privilege_level/src/bsp/raspberrypi.rs b/09_privilege_level/src/bsp/raspberrypi.rs index a9a7261a..fe940677 100644 --- a/09_privilege_level/src/bsp/raspberrypi.rs +++ b/09_privilege_level/src/bsp/raspberrypi.rs @@ -4,7 +4,6 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -pub mod console; pub mod cpu; pub mod driver; pub mod memory; diff --git a/09_privilege_level/src/bsp/raspberrypi/console.rs b/09_privilege_level/src/bsp/raspberrypi/console.rs deleted file mode 100644 index 0a630eef..00000000 --- a/09_privilege_level/src/bsp/raspberrypi/console.rs +++ /dev/null @@ -1,16 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2022 Andre Richter - -//! BSP console facilities. - -use crate::console; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return a reference to the console. -pub fn console() -> &'static dyn console::interface::All { - &super::driver::PL011_UART -} diff --git a/09_privilege_level/src/bsp/raspberrypi/driver.rs b/09_privilege_level/src/bsp/raspberrypi/driver.rs index 8b683ed8..ea843066 100644 --- a/09_privilege_level/src/bsp/raspberrypi/driver.rs +++ b/09_privilege_level/src/bsp/raspberrypi/driver.rs @@ -5,51 +5,67 @@ //! BSP driver support. use super::memory::map::mmio; -use crate::{bsp::device_driver, driver}; +use crate::{bsp::device_driver, console, driver as generic_driver}; +use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- -// Private Definitions +// Global instances //-------------------------------------------------------------------------------------------------- -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: [&'static (dyn DeviceDriver + Sync); 2], -} +static PL011_UART: device_driver::PL011Uart = + unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; //-------------------------------------------------------------------------------------------------- -// Global instances +// Private Code //-------------------------------------------------------------------------------------------------- -pub(super) static PL011_UART: device_driver::PL011Uart = - unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +/// This must be called only after successful init of the UART driver. +fn post_init_uart() -> Result<(), &'static str> { + console::register_console(&PL011_UART); -static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + Ok(()) +} -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: [&PL011_UART, &GPIO], -}; +/// This must be called only after successful init of the GPIO driver. +fn post_init_gpio() -> Result<(), &'static str> { + GPIO.map_pl011_uart(); + Ok(()) +} -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +fn driver_uart() -> Result<(), &'static str> { + let uart_descriptor = + generic_driver::DeviceDriverDescriptor::new(&PL011_UART, Some(post_init_uart)); + generic_driver::driver_manager().register_driver(uart_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +fn driver_gpio() -> Result<(), &'static str> { + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio)); + generic_driver::driver_manager().register_driver(gpio_descriptor); -impl driver::interface::DriverManager for BSPDriverManager { - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { - &self.device_drivers[..] - } + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - fn post_device_driver_init(&self) { - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } + + driver_uart()?; + driver_gpio()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) } diff --git a/09_privilege_level/src/console.rs b/09_privilege_level/src/console.rs index c1fb0e53..02b43df9 100644 --- a/09_privilege_level/src/console.rs +++ b/09_privilege_level/src/console.rs @@ -4,7 +4,9 @@ //! System console. -use crate::bsp; +mod null_console; + +use crate::synchronization::{self, NullLock}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -54,13 +56,26 @@ pub mod interface { pub trait All: Write + Read + Statistics {} } +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static CUR_CONSOLE: NullLock<&'static (dyn interface::All + Sync)> = + NullLock::new(&null_console::NULL_CONSOLE); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +use synchronization::interface::Mutex; + +/// Register a new console. +pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { + CUR_CONSOLE.lock(|con| *con = new_console); +} -/// Return a reference to the console. +/// Return a reference to the currently registered console. /// /// This is the global console used by all printing macros. pub fn console() -> &'static dyn interface::All { - bsp::console::console() + CUR_CONSOLE.lock(|con| *con) } diff --git a/09_privilege_level/src/console/null_console.rs b/09_privilege_level/src/console/null_console.rs new file mode 100644 index 00000000..2c64d499 --- /dev/null +++ b/09_privilege_level/src/console/null_console.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter + +//! Null console. + +use super::interface; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct NullConsole; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +pub static NULL_CONSOLE: NullConsole = NullConsole {}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl interface::Write for NullConsole { + fn write_char(&self, _c: char) {} + + fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { + fmt::Result::Ok(()) + } + + fn flush(&self) {} +} + +impl interface::Read for NullConsole { + fn clear_rx(&self) {} +} + +impl interface::Statistics for NullConsole {} +impl interface::All for NullConsole {} diff --git a/09_privilege_level/src/driver.rs b/09_privilege_level/src/driver.rs index 2fcc7562..a798c86d 100644 --- a/09_privilege_level/src/driver.rs +++ b/09_privilege_level/src/driver.rs @@ -4,6 +4,22 @@ //! Driver support. +use crate::{ + info, + synchronization::{interface::Mutex, NullLock}, +}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner { + next_index: usize, + descriptors: [Option; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -24,21 +40,128 @@ pub mod interface { Ok(()) } } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor { + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager { + inner: NullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner { + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], + } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager { + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: NullLock::new(DriverManagerInner::new()), + } + } - /// Device driver management functions. + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.lock(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.lock(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Return a slice of references to all `BSP`-instantiated drivers. - /// - /// # Safety - /// - /// - The order of devices is the order in which `DeviceDriver::init()` is called. - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } - /// Initialization code that runs after driver init. - /// - /// For example, device driver code that depends on other drivers already being online. - fn post_device_driver_init(&self); + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } + + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); + + i += 1; + }); } } diff --git a/09_privilege_level/src/main.rs b/09_privilege_level/src/main.rs index 79a6716e..c60aee8e 100644 --- a/09_privilege_level/src/main.rs +++ b/09_privilege_level/src/main.rs @@ -134,14 +134,13 @@ mod time; /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order. unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - bsp::driver::driver_manager().post_device_driver_init(); + + // Initialize all device drivers. + driver::driver_manager().init_drivers(); // println! is usable from here on. // Transition from unsafe to safe. @@ -152,7 +151,6 @@ unsafe fn kernel_init() -> ! { fn kernel_main() -> ! { use console::console; use core::time::Duration; - use driver::interface::DriverManager; info!( "{} version {}", @@ -173,13 +171,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/10_virtual_mem_part1_identity_mapping/README.md b/10_virtual_mem_part1_identity_mapping/README.md index f1d22d98..946500dc 100644 --- a/10_virtual_mem_part1_identity_mapping/README.md +++ b/10_virtual_mem_part1_identity_mapping/README.md @@ -296,7 +296,6 @@ Turning on virtual memory is now the first thing we do during kernel init: ```rust unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; use memory::mmu::interface::MMU; if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { @@ -1135,7 +1134,7 @@ diff -uNr 09_privilege_level/src/main.rs 10_virtual_mem_part1_identity_mapping/s mod panic_wait; mod print; mod synchronization; -@@ -132,9 +137,17 @@ +@@ -132,8 +137,17 @@ /// # Safety /// /// - Only a single core must be active and running this function. @@ -1145,25 +1144,25 @@ diff -uNr 09_privilege_level/src/main.rs 10_virtual_mem_part1_identity_mapping/s +/// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ +/// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; + use memory::mmu::interface::MMU; + + if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { + panic!("MMU: {}", string); + } - - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { -@@ -150,7 +163,7 @@ ++ + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); +@@ -149,7 +163,7 @@ /// The main function running after the early init. fn kernel_main() -> ! { - use console::console; + use console::{console, interface::Write}; use core::time::Duration; - use driver::interface::DriverManager; -@@ -161,6 +174,9 @@ + info!( +@@ -159,6 +173,9 @@ ); info!("Booting on: {}", bsp::board_name()); @@ -1173,7 +1172,7 @@ diff -uNr 09_privilege_level/src/main.rs 10_virtual_mem_part1_identity_mapping/s let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -@@ -184,6 +200,13 @@ +@@ -176,6 +193,13 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs index a9a7261a..fe940677 100644 --- a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs +++ b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi.rs @@ -4,7 +4,6 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -pub mod console; pub mod cpu; pub mod driver; pub mod memory; diff --git a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/console.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/console.rs deleted file mode 100644 index 0a630eef..00000000 --- a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/console.rs +++ /dev/null @@ -1,16 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2022 Andre Richter - -//! BSP console facilities. - -use crate::console; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return a reference to the console. -pub fn console() -> &'static dyn console::interface::All { - &super::driver::PL011_UART -} diff --git a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs index 8b683ed8..ea843066 100644 --- a/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs +++ b/10_virtual_mem_part1_identity_mapping/src/bsp/raspberrypi/driver.rs @@ -5,51 +5,67 @@ //! BSP driver support. use super::memory::map::mmio; -use crate::{bsp::device_driver, driver}; +use crate::{bsp::device_driver, console, driver as generic_driver}; +use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- -// Private Definitions +// Global instances //-------------------------------------------------------------------------------------------------- -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: [&'static (dyn DeviceDriver + Sync); 2], -} +static PL011_UART: device_driver::PL011Uart = + unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; //-------------------------------------------------------------------------------------------------- -// Global instances +// Private Code //-------------------------------------------------------------------------------------------------- -pub(super) static PL011_UART: device_driver::PL011Uart = - unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +/// This must be called only after successful init of the UART driver. +fn post_init_uart() -> Result<(), &'static str> { + console::register_console(&PL011_UART); -static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + Ok(()) +} -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: [&PL011_UART, &GPIO], -}; +/// This must be called only after successful init of the GPIO driver. +fn post_init_gpio() -> Result<(), &'static str> { + GPIO.map_pl011_uart(); + Ok(()) +} -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +fn driver_uart() -> Result<(), &'static str> { + let uart_descriptor = + generic_driver::DeviceDriverDescriptor::new(&PL011_UART, Some(post_init_uart)); + generic_driver::driver_manager().register_driver(uart_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +fn driver_gpio() -> Result<(), &'static str> { + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio)); + generic_driver::driver_manager().register_driver(gpio_descriptor); -impl driver::interface::DriverManager for BSPDriverManager { - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { - &self.device_drivers[..] - } + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - fn post_device_driver_init(&self) { - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } + + driver_uart()?; + driver_gpio()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) } diff --git a/10_virtual_mem_part1_identity_mapping/src/console.rs b/10_virtual_mem_part1_identity_mapping/src/console.rs index c1fb0e53..02b43df9 100644 --- a/10_virtual_mem_part1_identity_mapping/src/console.rs +++ b/10_virtual_mem_part1_identity_mapping/src/console.rs @@ -4,7 +4,9 @@ //! System console. -use crate::bsp; +mod null_console; + +use crate::synchronization::{self, NullLock}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -54,13 +56,26 @@ pub mod interface { pub trait All: Write + Read + Statistics {} } +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static CUR_CONSOLE: NullLock<&'static (dyn interface::All + Sync)> = + NullLock::new(&null_console::NULL_CONSOLE); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +use synchronization::interface::Mutex; + +/// Register a new console. +pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { + CUR_CONSOLE.lock(|con| *con = new_console); +} -/// Return a reference to the console. +/// Return a reference to the currently registered console. /// /// This is the global console used by all printing macros. pub fn console() -> &'static dyn interface::All { - bsp::console::console() + CUR_CONSOLE.lock(|con| *con) } diff --git a/10_virtual_mem_part1_identity_mapping/src/console/null_console.rs b/10_virtual_mem_part1_identity_mapping/src/console/null_console.rs new file mode 100644 index 00000000..2c64d499 --- /dev/null +++ b/10_virtual_mem_part1_identity_mapping/src/console/null_console.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter + +//! Null console. + +use super::interface; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct NullConsole; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +pub static NULL_CONSOLE: NullConsole = NullConsole {}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl interface::Write for NullConsole { + fn write_char(&self, _c: char) {} + + fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { + fmt::Result::Ok(()) + } + + fn flush(&self) {} +} + +impl interface::Read for NullConsole { + fn clear_rx(&self) {} +} + +impl interface::Statistics for NullConsole {} +impl interface::All for NullConsole {} diff --git a/10_virtual_mem_part1_identity_mapping/src/driver.rs b/10_virtual_mem_part1_identity_mapping/src/driver.rs index 2fcc7562..a798c86d 100644 --- a/10_virtual_mem_part1_identity_mapping/src/driver.rs +++ b/10_virtual_mem_part1_identity_mapping/src/driver.rs @@ -4,6 +4,22 @@ //! Driver support. +use crate::{ + info, + synchronization::{interface::Mutex, NullLock}, +}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner { + next_index: usize, + descriptors: [Option; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -24,21 +40,128 @@ pub mod interface { Ok(()) } } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor { + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager { + inner: NullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner { + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], + } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager { + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: NullLock::new(DriverManagerInner::new()), + } + } - /// Device driver management functions. + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.lock(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.lock(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Return a slice of references to all `BSP`-instantiated drivers. - /// - /// # Safety - /// - /// - The order of devices is the order in which `DeviceDriver::init()` is called. - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } - /// Initialization code that runs after driver init. - /// - /// For example, device driver code that depends on other drivers already being online. - fn post_device_driver_init(&self); + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } + + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); + + i += 1; + }); } } diff --git a/10_virtual_mem_part1_identity_mapping/src/main.rs b/10_virtual_mem_part1_identity_mapping/src/main.rs index e038e093..b40a37bf 100644 --- a/10_virtual_mem_part1_identity_mapping/src/main.rs +++ b/10_virtual_mem_part1_identity_mapping/src/main.rs @@ -142,19 +142,19 @@ mod time; /// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ /// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; use memory::mmu::interface::MMU; if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { panic!("MMU: {}", string); } - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - bsp::driver::driver_manager().post_device_driver_init(); + + // Initialize all device drivers. + driver::driver_manager().init_drivers(); // println! is usable from here on. // Transition from unsafe to safe. @@ -165,7 +165,6 @@ unsafe fn kernel_init() -> ! { fn kernel_main() -> ! { use console::{console, interface::Write}; use core::time::Duration; - use driver::interface::DriverManager; info!( "{} version {}", @@ -189,13 +188,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/11_exceptions_part1_groundwork/README.md b/11_exceptions_part1_groundwork/README.md index 3e7ca05e..12812acb 100644 --- a/11_exceptions_part1_groundwork/README.md +++ b/11_exceptions_part1_groundwork/README.md @@ -1024,8 +1024,8 @@ diff -uNr 10_virtual_mem_part1_identity_mapping/src/exception.rs 11_exceptions_p diff -uNr 10_virtual_mem_part1_identity_mapping/src/main.rs 11_exceptions_part1_groundwork/src/main.rs --- 10_virtual_mem_part1_identity_mapping/src/main.rs +++ 11_exceptions_part1_groundwork/src/main.rs -@@ -145,6 +145,8 @@ - use driver::interface::DriverManager; +@@ -144,6 +144,8 @@ + unsafe fn kernel_init() -> ! { use memory::mmu::interface::MMU; + exception::handling_init(); @@ -1040,9 +1040,9 @@ diff -uNr 10_virtual_mem_part1_identity_mapping/src/main.rs 11_exceptions_part1_ - use console::{console, interface::Write}; + use console::console; use core::time::Duration; - use driver::interface::DriverManager; -@@ -200,13 +202,28 @@ + info!( +@@ -193,13 +195,28 @@ info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/11_exceptions_part1_groundwork/src/bsp/raspberrypi.rs b/11_exceptions_part1_groundwork/src/bsp/raspberrypi.rs index a9a7261a..fe940677 100644 --- a/11_exceptions_part1_groundwork/src/bsp/raspberrypi.rs +++ b/11_exceptions_part1_groundwork/src/bsp/raspberrypi.rs @@ -4,7 +4,6 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -pub mod console; pub mod cpu; pub mod driver; pub mod memory; diff --git a/11_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs deleted file mode 100644 index 0a630eef..00000000 --- a/11_exceptions_part1_groundwork/src/bsp/raspberrypi/console.rs +++ /dev/null @@ -1,16 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2022 Andre Richter - -//! BSP console facilities. - -use crate::console; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return a reference to the console. -pub fn console() -> &'static dyn console::interface::All { - &super::driver::PL011_UART -} diff --git a/11_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs index 8b683ed8..ea843066 100644 --- a/11_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs +++ b/11_exceptions_part1_groundwork/src/bsp/raspberrypi/driver.rs @@ -5,51 +5,67 @@ //! BSP driver support. use super::memory::map::mmio; -use crate::{bsp::device_driver, driver}; +use crate::{bsp::device_driver, console, driver as generic_driver}; +use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- -// Private Definitions +// Global instances //-------------------------------------------------------------------------------------------------- -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: [&'static (dyn DeviceDriver + Sync); 2], -} +static PL011_UART: device_driver::PL011Uart = + unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; //-------------------------------------------------------------------------------------------------- -// Global instances +// Private Code //-------------------------------------------------------------------------------------------------- -pub(super) static PL011_UART: device_driver::PL011Uart = - unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +/// This must be called only after successful init of the UART driver. +fn post_init_uart() -> Result<(), &'static str> { + console::register_console(&PL011_UART); -static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + Ok(()) +} -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: [&PL011_UART, &GPIO], -}; +/// This must be called only after successful init of the GPIO driver. +fn post_init_gpio() -> Result<(), &'static str> { + GPIO.map_pl011_uart(); + Ok(()) +} -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +fn driver_uart() -> Result<(), &'static str> { + let uart_descriptor = + generic_driver::DeviceDriverDescriptor::new(&PL011_UART, Some(post_init_uart)); + generic_driver::driver_manager().register_driver(uart_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +fn driver_gpio() -> Result<(), &'static str> { + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio)); + generic_driver::driver_manager().register_driver(gpio_descriptor); -impl driver::interface::DriverManager for BSPDriverManager { - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { - &self.device_drivers[..] - } + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - fn post_device_driver_init(&self) { - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } + + driver_uart()?; + driver_gpio()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) } diff --git a/11_exceptions_part1_groundwork/src/console.rs b/11_exceptions_part1_groundwork/src/console.rs index c1fb0e53..02b43df9 100644 --- a/11_exceptions_part1_groundwork/src/console.rs +++ b/11_exceptions_part1_groundwork/src/console.rs @@ -4,7 +4,9 @@ //! System console. -use crate::bsp; +mod null_console; + +use crate::synchronization::{self, NullLock}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -54,13 +56,26 @@ pub mod interface { pub trait All: Write + Read + Statistics {} } +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static CUR_CONSOLE: NullLock<&'static (dyn interface::All + Sync)> = + NullLock::new(&null_console::NULL_CONSOLE); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +use synchronization::interface::Mutex; + +/// Register a new console. +pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { + CUR_CONSOLE.lock(|con| *con = new_console); +} -/// Return a reference to the console. +/// Return a reference to the currently registered console. /// /// This is the global console used by all printing macros. pub fn console() -> &'static dyn interface::All { - bsp::console::console() + CUR_CONSOLE.lock(|con| *con) } diff --git a/11_exceptions_part1_groundwork/src/console/null_console.rs b/11_exceptions_part1_groundwork/src/console/null_console.rs new file mode 100644 index 00000000..2c64d499 --- /dev/null +++ b/11_exceptions_part1_groundwork/src/console/null_console.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter + +//! Null console. + +use super::interface; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct NullConsole; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +pub static NULL_CONSOLE: NullConsole = NullConsole {}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl interface::Write for NullConsole { + fn write_char(&self, _c: char) {} + + fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { + fmt::Result::Ok(()) + } + + fn flush(&self) {} +} + +impl interface::Read for NullConsole { + fn clear_rx(&self) {} +} + +impl interface::Statistics for NullConsole {} +impl interface::All for NullConsole {} diff --git a/11_exceptions_part1_groundwork/src/driver.rs b/11_exceptions_part1_groundwork/src/driver.rs index 2fcc7562..a798c86d 100644 --- a/11_exceptions_part1_groundwork/src/driver.rs +++ b/11_exceptions_part1_groundwork/src/driver.rs @@ -4,6 +4,22 @@ //! Driver support. +use crate::{ + info, + synchronization::{interface::Mutex, NullLock}, +}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner { + next_index: usize, + descriptors: [Option; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -24,21 +40,128 @@ pub mod interface { Ok(()) } } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor { + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager { + inner: NullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner { + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], + } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager { + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: NullLock::new(DriverManagerInner::new()), + } + } - /// Device driver management functions. + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.lock(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.lock(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Return a slice of references to all `BSP`-instantiated drivers. - /// - /// # Safety - /// - /// - The order of devices is the order in which `DeviceDriver::init()` is called. - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } - /// Initialization code that runs after driver init. - /// - /// For example, device driver code that depends on other drivers already being online. - fn post_device_driver_init(&self); + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } + + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); + + i += 1; + }); } } diff --git a/11_exceptions_part1_groundwork/src/main.rs b/11_exceptions_part1_groundwork/src/main.rs index 8e632fa5..9d300bf0 100644 --- a/11_exceptions_part1_groundwork/src/main.rs +++ b/11_exceptions_part1_groundwork/src/main.rs @@ -142,7 +142,6 @@ mod time; /// e.g. the yet-to-be-introduced spinlocks in the device drivers (which currently employ /// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; use memory::mmu::interface::MMU; exception::handling_init(); @@ -151,12 +150,13 @@ unsafe fn kernel_init() -> ! { panic!("MMU: {}", string); } - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - bsp::driver::driver_manager().post_device_driver_init(); + + // Initialize all device drivers. + driver::driver_manager().init_drivers(); // println! is usable from here on. // Transition from unsafe to safe. @@ -167,7 +167,6 @@ unsafe fn kernel_init() -> ! { fn kernel_main() -> ! { use console::console; use core::time::Duration; - use driver::interface::DriverManager; info!( "{} version {}", @@ -191,13 +190,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); info!("Timer test, spinning for 1 second"); time::time_manager().spin_for(Duration::from_secs(1)); diff --git a/12_integrated_testing/Makefile b/12_integrated_testing/Makefile index 670c30c4..2d480119 100644 --- a/12_integrated_testing/Makefile +++ b/12_integrated_testing/Makefile @@ -291,7 +291,7 @@ test_boot: $(KERNEL_BIN) ## Helpers for unit and integration test targets ##------------------------------------------------------------------------------ define KERNEL_TEST_RUNNER - #!/usr/bin/env bash +#!/usr/bin/env bash # The cargo test runner seems to change into the crate under test's directory. Therefore, ensure # this script executes from the root. diff --git a/12_integrated_testing/README.md b/12_integrated_testing/README.md index c5d032e8..ae68d269 100644 --- a/12_integrated_testing/README.md +++ b/12_integrated_testing/README.md @@ -266,10 +266,8 @@ implementation in `lib.rs`: #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); test_main(); @@ -277,12 +275,12 @@ unsafe fn kernel_init() -> ! { } ``` -Note the call to `bsp::driver::driver_manager().qemu_bring_up_console()`. Since we are running all -our tests inside `QEMU`, we need to ensure that whatever peripheral implements the kernel's -`console` interface is initialized, so that we can print from our tests. If you recall [tutorial -03], bringing up peripherals in `QEMU` might not need the full initialization as is needed on real -hardware (setting clocks, config registers, etc...) due to the abstractions in `QEMU`'s emulation -code. So this is an opportunity to cut down on setup code. +Note the call to `bsp::driver::qemu_bring_up_console()`. Since we are running all our tests inside +`QEMU`, we need to ensure that whatever peripheral implements the kernel's `console` interface is +initialized, so that we can print from our tests. If you recall [tutorial 03], bringing up +peripherals in `QEMU` might not need the full initialization as is needed on real hardware (setting +clocks, config registers, etc...) due to the abstractions in `QEMU`'s emulation code. So this is an +opportunity to cut down on setup code. [tutorial 03]: ../03_hacky_hello_world @@ -622,15 +620,13 @@ your test code into individual chunks. For example, take a look at `tests/01_tim #![test_runner(libkernel::test_runner)] use core::time::Duration; -use libkernel::{bsp, cpu, driver, exception, time}; +use libkernel::{bsp, cpu, exception, time}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. @@ -643,12 +639,12 @@ unsafe fn kernel_init() -> ! { #[kernel_test] fn timer_is_counting() { assert!(time::time_manager().uptime().as_nanos() > 0) - assert!(time::time_manager().resolution().as_nanos() < 100) } /// Timer resolution must be sufficient. #[kernel_test] fn timer_resolution_is_sufficient() { + assert!(time::time_manager().resolution().as_nanos() > 0); assert!(time::time_manager().resolution().as_nanos() < 100) } ``` @@ -719,15 +715,14 @@ so the wanted outcome is a `panic!`. Here is the whole test (minus some inline c mod panic_exit_success; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; use memory::mmu::interface::MMU; exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); @@ -794,15 +789,14 @@ The subtest first sends `"ABC"` over the console to the kernel, and then expects /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{bsp, console, cpu, driver, exception, print}; +use libkernel::{bsp, console, cpu, exception, print}; #[no_mangle] unsafe fn kernel_init() -> ! { use console::console; - use driver::interface::DriverManager; exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/12_integrated_testing/kernel/src/bsp/raspberrypi.rs b/12_integrated_testing/kernel/src/bsp/raspberrypi.rs index a9a7261a..fe940677 100644 --- a/12_integrated_testing/kernel/src/bsp/raspberrypi.rs +++ b/12_integrated_testing/kernel/src/bsp/raspberrypi.rs @@ -4,7 +4,6 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -pub mod console; pub mod cpu; pub mod driver; pub mod memory; diff --git a/12_integrated_testing/kernel/src/bsp/raspberrypi/console.rs b/12_integrated_testing/kernel/src/bsp/raspberrypi/console.rs deleted file mode 100644 index 0a630eef..00000000 --- a/12_integrated_testing/kernel/src/bsp/raspberrypi/console.rs +++ /dev/null @@ -1,16 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2022 Andre Richter - -//! BSP console facilities. - -use crate::console; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return a reference to the console. -pub fn console() -> &'static dyn console::interface::All { - &super::driver::PL011_UART -} diff --git a/12_integrated_testing/kernel/src/bsp/raspberrypi/driver.rs b/12_integrated_testing/kernel/src/bsp/raspberrypi/driver.rs index f8acd335..beaee16b 100644 --- a/12_integrated_testing/kernel/src/bsp/raspberrypi/driver.rs +++ b/12_integrated_testing/kernel/src/bsp/raspberrypi/driver.rs @@ -5,54 +5,74 @@ //! BSP driver support. use super::memory::map::mmio; -use crate::{bsp::device_driver, driver}; +use crate::{bsp::device_driver, console, driver as generic_driver}; +use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- -// Private Definitions +// Global instances //-------------------------------------------------------------------------------------------------- -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: [&'static (dyn DeviceDriver + Sync); 2], -} +static PL011_UART: device_driver::PL011Uart = + unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; //-------------------------------------------------------------------------------------------------- -// Global instances +// Private Code //-------------------------------------------------------------------------------------------------- -pub(super) static PL011_UART: device_driver::PL011Uart = - unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +/// This must be called only after successful init of the UART driver. +fn post_init_uart() -> Result<(), &'static str> { + console::register_console(&PL011_UART); -static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + Ok(()) +} -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: [&PL011_UART, &GPIO], -}; +/// This must be called only after successful init of the GPIO driver. +fn post_init_gpio() -> Result<(), &'static str> { + GPIO.map_pl011_uart(); + Ok(()) +} -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +fn driver_uart() -> Result<(), &'static str> { + let uart_descriptor = + generic_driver::DeviceDriverDescriptor::new(&PL011_UART, Some(post_init_uart)); + generic_driver::driver_manager().register_driver(uart_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +fn driver_gpio() -> Result<(), &'static str> { + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio)); + generic_driver::driver_manager().register_driver(gpio_descriptor); -impl driver::interface::DriverManager for BSPDriverManager { - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { - &self.device_drivers[..] - } + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - fn post_device_driver_init(&self) { - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self) {} + driver_uart()?; + driver_gpio()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) +} + +/// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps +/// than on real hardware due to QEMU's abstractions. +#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() { + console::register_console(&PL011_UART); } diff --git a/12_integrated_testing/kernel/src/console.rs b/12_integrated_testing/kernel/src/console.rs index c1fb0e53..02b43df9 100644 --- a/12_integrated_testing/kernel/src/console.rs +++ b/12_integrated_testing/kernel/src/console.rs @@ -4,7 +4,9 @@ //! System console. -use crate::bsp; +mod null_console; + +use crate::synchronization::{self, NullLock}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -54,13 +56,26 @@ pub mod interface { pub trait All: Write + Read + Statistics {} } +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static CUR_CONSOLE: NullLock<&'static (dyn interface::All + Sync)> = + NullLock::new(&null_console::NULL_CONSOLE); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +use synchronization::interface::Mutex; + +/// Register a new console. +pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { + CUR_CONSOLE.lock(|con| *con = new_console); +} -/// Return a reference to the console. +/// Return a reference to the currently registered console. /// /// This is the global console used by all printing macros. pub fn console() -> &'static dyn interface::All { - bsp::console::console() + CUR_CONSOLE.lock(|con| *con) } diff --git a/12_integrated_testing/kernel/src/console/null_console.rs b/12_integrated_testing/kernel/src/console/null_console.rs new file mode 100644 index 00000000..2c64d499 --- /dev/null +++ b/12_integrated_testing/kernel/src/console/null_console.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter + +//! Null console. + +use super::interface; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct NullConsole; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +pub static NULL_CONSOLE: NullConsole = NullConsole {}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl interface::Write for NullConsole { + fn write_char(&self, _c: char) {} + + fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { + fmt::Result::Ok(()) + } + + fn flush(&self) {} +} + +impl interface::Read for NullConsole { + fn clear_rx(&self) {} +} + +impl interface::Statistics for NullConsole {} +impl interface::All for NullConsole {} diff --git a/12_integrated_testing/kernel/src/driver.rs b/12_integrated_testing/kernel/src/driver.rs index 12d60ad4..a798c86d 100644 --- a/12_integrated_testing/kernel/src/driver.rs +++ b/12_integrated_testing/kernel/src/driver.rs @@ -4,6 +4,22 @@ //! Driver support. +use crate::{ + info, + synchronization::{interface::Mutex, NullLock}, +}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner { + next_index: usize, + descriptors: [Option; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -24,26 +40,128 @@ pub mod interface { Ok(()) } } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor { + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager { + inner: NullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner { + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], + } + } +} - /// Device driver management functions. +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager { + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: NullLock::new(DriverManagerInner::new()), + } + } + + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.lock(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.lock(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Return a slice of references to all `BSP`-instantiated drivers. - /// - /// # Safety - /// - /// - The order of devices is the order in which `DeviceDriver::init()` is called. - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } - /// Initialization code that runs after driver init. - /// - /// For example, device driver code that depends on other drivers already being online. - fn post_device_driver_init(&self); + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } + + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); - /// Minimal code needed to bring up the console in QEMU (for testing only). This is often - /// less steps than on real hardware due to QEMU's abstractions. - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self); + i += 1; + }); } } diff --git a/12_integrated_testing/kernel/src/lib.rs b/12_integrated_testing/kernel/src/lib.rs index 35698c74..3f5a846f 100644 --- a/12_integrated_testing/kernel/src/lib.rs +++ b/12_integrated_testing/kernel/src/lib.rs @@ -182,10 +182,8 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); test_main(); diff --git a/12_integrated_testing/kernel/src/main.rs b/12_integrated_testing/kernel/src/main.rs index bf4d7fe3..9cfcf315 100644 --- a/12_integrated_testing/kernel/src/main.rs +++ b/12_integrated_testing/kernel/src/main.rs @@ -26,7 +26,6 @@ use libkernel::{bsp, console, driver, exception, info, memory, time}; /// NullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; use memory::mmu::interface::MMU; exception::handling_init(); @@ -35,12 +34,13 @@ unsafe fn kernel_init() -> ! { panic!("MMU: {}", string); } - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - bsp::driver::driver_manager().post_device_driver_init(); + + // Initialize all device drivers. + driver::driver_manager().init_drivers(); // println! is usable from here on. // Transition from unsafe to safe. @@ -50,7 +50,6 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { use console::console; - use driver::interface::DriverManager; info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); @@ -70,13 +69,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); info!("Echoing input now"); diff --git a/12_integrated_testing/kernel/tests/00_console_sanity.rs b/12_integrated_testing/kernel/tests/00_console_sanity.rs index e12a711f..69313428 100644 --- a/12_integrated_testing/kernel/tests/00_console_sanity.rs +++ b/12_integrated_testing/kernel/tests/00_console_sanity.rs @@ -11,15 +11,14 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{bsp, console, cpu, driver, exception, print}; +use libkernel::{bsp, console, cpu, exception, print}; #[no_mangle] unsafe fn kernel_init() -> ! { use console::console; - use driver::interface::DriverManager; exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/12_integrated_testing/kernel/tests/01_timer_sanity.rs b/12_integrated_testing/kernel/tests/01_timer_sanity.rs index a0eb732b..b86016b6 100644 --- a/12_integrated_testing/kernel/tests/01_timer_sanity.rs +++ b/12_integrated_testing/kernel/tests/01_timer_sanity.rs @@ -11,15 +11,13 @@ #![test_runner(libkernel::test_runner)] use core::time::Duration; -use libkernel::{bsp, cpu, driver, exception, time}; +use libkernel::{bsp, cpu, exception, time}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff --git a/12_integrated_testing/kernel/tests/02_exception_sync_page_fault.rs b/12_integrated_testing/kernel/tests/02_exception_sync_page_fault.rs index 62908377..46501960 100644 --- a/12_integrated_testing/kernel/tests/02_exception_sync_page_fault.rs +++ b/12_integrated_testing/kernel/tests/02_exception_sync_page_fault.rs @@ -17,15 +17,14 @@ /// or indirectly. mod panic_exit_success; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; use memory::mmu::interface::MMU; exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); diff --git a/12_integrated_testing/kernel/tests/03_exception_restore_sanity.rs b/12_integrated_testing/kernel/tests/03_exception_restore_sanity.rs index 3f4eae79..cba9285f 100644 --- a/12_integrated_testing/kernel/tests/03_exception_restore_sanity.rs +++ b/12_integrated_testing/kernel/tests/03_exception_restore_sanity.rs @@ -12,7 +12,7 @@ mod panic_wait_forever; use core::arch::asm; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[inline(never)] fn nested_system_call() { @@ -30,11 +30,10 @@ fn nested_system_call() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; use memory::mmu::interface::MMU; exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing exception restore"); diff --git a/13_exceptions_part2_peripheral_IRQs/Makefile b/13_exceptions_part2_peripheral_IRQs/Makefile index 670c30c4..2d480119 100644 --- a/13_exceptions_part2_peripheral_IRQs/Makefile +++ b/13_exceptions_part2_peripheral_IRQs/Makefile @@ -291,7 +291,7 @@ test_boot: $(KERNEL_BIN) ## Helpers for unit and integration test targets ##------------------------------------------------------------------------------ define KERNEL_TEST_RUNNER - #!/usr/bin/env bash +#!/usr/bin/env bash # The cargo test runner seems to change into the crate under test's directory. Therefore, ensure # this script executes from the root. diff --git a/13_exceptions_part2_peripheral_IRQs/README.md b/13_exceptions_part2_peripheral_IRQs/README.md index 3d6d5f16..a8c503d5 100644 --- a/13_exceptions_part2_peripheral_IRQs/README.md +++ b/13_exceptions_part2_peripheral_IRQs/README.md @@ -110,17 +110,16 @@ The trait is defined as `exception::asynchronous::interface::IRQManager`: ```rust pub trait IRQManager { /// The IRQ number type depends on the implementation. - type IRQNumberType; + type IRQNumberType: Copy; /// Register a handler. fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: super::IRQDescriptor, + irq_handler_descriptor: super::IRQHandlerDescriptor, ) -> Result<(), &'static str>; /// Enable an interrupt in the controller. - fn enable(&self, irq_number: Self::IRQNumberType); + fn enable(&self, irq_number: &Self::IRQNumberType); /// Handle pending interrupts. /// @@ -136,7 +135,7 @@ pub trait IRQManager { ); /// Print list of registered handlers. - fn print_handler(&self); + fn print_handler(&self) {} } ``` @@ -175,22 +174,21 @@ sufficient to uniquely encode the IRQs, because their ranges overlap. In the dri controller, we therefore define the associated type as follows: ```rust -pub type LocalIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; -pub type PeripheralIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; +pub type LocalIRQ = BoundedUsize<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; +pub type PeripheralIRQ = BoundedUsize<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; -/// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. +/// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. #[derive(Copy, Clone)] +#[allow(missing_docs)] pub enum IRQNumber { Local(LocalIRQ), Peripheral(PeripheralIRQ), } ``` -The type `exception::asynchronous::IRQNumber` is a newtype around an `usize` that uses a [const -generic] to ensure that the value of the encapsulated IRQ number is in the allowed range (e.g. -`0..MAX_LOCAL_IRQ_NUMBER` for `LocalIRQ`, with `MAX_LOCAL_IRQ_NUMBER == 11`). +The type `BoundedUsize` is a newtype around an `usize` that uses a [const generic] to ensure that +the value of the encapsulated IRQ number is in the allowed range (e.g. `0..MAX_LOCAL_IRQ_NUMBER` for +`LocalIRQ`, with `MAX_LOCAL_IRQ_NUMBER == 11`). [const generic]: https://github.com/rust-lang/rfcs/blob/master/text/2000-const-generics.md @@ -206,7 +204,7 @@ identifier for the IRQs. We define the type as follows: ```rust /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. -pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER }>; +pub type IRQNumber = BoundedUsize<{ GICv2::MAX_IRQ_NUMBER }>; ``` #### Registering IRQ Handlers @@ -264,34 +262,41 @@ respective drivers themselves. Therefore, we added a new function to the standar trait in `driver::interface::DeviceDriver` that must be implemented if IRQ handling is supported: ```rust -/// Called by the kernel to register and enable the device's IRQ handlers, if any. +/// Called by the kernel to register and enable the device's IRQ handler. /// /// Rust's type system will prevent a call to this function unless the calling instance /// itself has static lifetime. -fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - Ok(()) +fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, +) -> Result<(), &'static str> { + panic!( + "Attempt to enable IRQ {} for device {}, but driver does not support this", + irq_number, + self.compatible() + ) } ``` Here is the implementation for the `PL011Uart`: ```rust -fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - use exception::asynchronous::{irq_manager, IRQDescriptor}; +fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, +) -> Result<(), &'static str> { + use exception::asynchronous::{irq_manager, IRQHandlerDescriptor}; - let descriptor = IRQDescriptor { - name: Self::COMPATIBLE, - handler: self, - }; + let descriptor = IRQHandlerDescriptor::new(*irq_number, Self::COMPATIBLE, self); - irq_manager().register_handler(self.irq_number, descriptor)?; - irq_manager().enable(self.irq_number); + irq_manager().register_handler(descriptor)?; + irq_manager().enable(irq_number); Ok(()) } ``` -The `bsp::exception::asynchronous::irq_manager()` function used here returns a reference to an +The `exception::asynchronous::irq_manager()` function used here returns a reference to an implementor of the `IRQManager` trait. Since the implementation is supposed to be done by the platform's interrupt controller, this call will redirect to the `kernel`'s instance of either the driver for the `BCM` controller (`Raspberry Pi 3`) or the driver for the `GICv2` (`Pi 4`). We will @@ -300,33 +305,57 @@ later. The gist here is that the calls on `irq_manager()` will make the platform controller aware that the `UART` driver (i) wants to handle its interrupt and (ii) which function it provides to do so. -Also note how `irq_number` is a member of the `PL011Uart` struct and not hardcoded. The reason is -that the `UART` driver code is agnostic about the **IRQ numbers** that are associated to it. This is +Also note how `irq_number` is supplied as a function argument and not hardcoded. The reason is that +the `UART` driver code is agnostic about the **IRQ numbers** that are associated to it. This is vendor-supplied information and as such typically part of the Board Support Package (`BSP`). It can vary from `BSP` to `BSP`, same like the board's memory map, which provides the `UART`'s MMIO -register addresses. Therefore, we extend the instantiation of the `UART` driver accordingly, so that -the `BSP` now additionally provides the IRQ number as an argument: +register addresses. + +With all this in place, we can finally let drivers register and enable their IRQ handlers with the +interrupt controller, and unmask IRQ reception on the boot CPU core during the kernel init phase. +The global `driver_manager` takes care of this in the function `init_drivers_and_irqs()` (before +this tutorial, the function's name was `init_drivers()`), where this happens as the third and last +step of initializing all registered device drivers: ```rust -static PL011_UART: device_driver::PL011Uart = unsafe { - device_driver::PL011Uart::new( - memory::map::mmio::PL011_UART_BASE, - exception::asynchronous::irq_map::PL011_UART, - ) -}; +pub unsafe fn init_drivers_and_irqs(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + // omitted for brevity + } + + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + // omitted for brevity + } + }); + + // 3. After all post-init callbacks were done, the interrupt controller should be + // registered and functional. So let drivers register with it now. + self.for_each_descriptor(|descriptor| { + if let Some(irq_number) = &descriptor.irq_number { + if let Err(x) = descriptor + .device_driver + .register_and_enable_irq_handler(irq_number) + { + panic!( + "Error during driver interrupt handler registration: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); +} ``` -With all this in place, we can finally let drivers register and enable their IRQ handlers with the -interrupt controller, and unmask IRQ reception on the boot CPU core during the kernel init phase in -`main.rs`. After unmasking, IRQ handling is live: + +In `main.rs`, IRQs are unmasked right afterwards, after which point IRQ handling is live: ```rust -// Let device drivers register and enable their handlers with the interrupt controller. -for i in bsp::driver::driver_manager().all_device_drivers() { - if let Err(msg) = i.register_and_enable_irq_handler() { - warn!("Error registering IRQ handler: {}", msg); - } -} +// Initialize all device drivers. +driver::driver_manager().init_drivers_and_irqs(); // Unmask interrupts on the boot CPU core. exception::asynchronous::local_irq_unmask(); @@ -489,8 +518,8 @@ Calls to `register_handler()` result in the driver inserting the provided handle specific table (the handler reference is a member of `IRQDescriptor`): ```rust -type HandlerTable = - [Option; InterruptController::NUM_PERIPHERAL_IRQS]; +type HandlerTable = [Option>; + PeripheralIRQ::MAX_INCLUSIVE + 1]; ``` One of the requirements for safe operation of the `kernel` is that those handlers are not @@ -547,6 +576,10 @@ only call is happening before the transition from `kernel_init()` to `kernel_mai state::state_manager().transition_to_single_core_main(); ``` +P.S.: Since the use case for the `InitStateLock` also applies to a few other places in the kernel +(for example, registering the system-wide console during early boot), `InitStateLock`s have been +incorporated in those other places as well. + #### The GICv2 Driver (Pi 4) As we learned earlier, the ARM `GICv2` in the `Raspberry Pi 4` features a continuous interrupt @@ -1218,7 +1251,7 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs 1 + } + + /// Enable an interrupt. -+ pub fn enable(&self, irq_num: super::IRQNumber) { ++ pub fn enable(&self, irq_num: &super::IRQNumber) { + let irq_num = irq_num.get(); + + // Each bit in the u32 enable register corresponds to one IRQ number. Shift right by 5 @@ -1249,7 +1282,7 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs 1 diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/arm/gicv2.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2.rs --- 12_integrated_testing/kernel/src/bsp/device_driver/arm/gicv2.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2.rs -@@ -0,0 +1,220 @@ +@@ -0,0 +1,226 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2022 Andre Richter @@ -1331,20 +1364,25 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/arm/gicv2.rs 13_exc +mod gicc; +mod gicd; + -+use crate::{bsp, cpu, driver, exception, synchronization, synchronization::InitStateLock}; ++use crate::{ ++ bsp::{self, device_driver::common::BoundedUsize}, ++ cpu, driver, exception, synchronization, ++ synchronization::InitStateLock, ++}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + -+type HandlerTable = [Option; IRQNumber::NUM_TOTAL]; ++type HandlerTable = [Option>; ++ IRQNumber::MAX_INCLUSIVE + 1]; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. -+pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER }>; ++pub type IRQNumber = BoundedUsize<{ GICv2::MAX_IRQ_NUMBER }>; + +/// Representation of the GIC. +pub struct GICv2 { @@ -1376,7 +1414,7 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/arm/gicv2.rs 13_exc + Self { + gicd: gicd::GICD::new(gicd_mmio_start_addr), + gicc: gicc::GICC::new(gicc_mmio_start_addr), -+ handler_table: InitStateLock::new([None; IRQNumber::NUM_TOTAL]), ++ handler_table: InitStateLock::new([None; IRQNumber::MAX_INCLUSIVE + 1]), + } + } +} @@ -1387,6 +1425,8 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/arm/gicv2.rs 13_exc +use synchronization::interface::ReadWriteEx; + +impl driver::interface::DeviceDriver for GICv2 { ++ type IRQNumberType = IRQNumber; ++ + fn compatible(&self) -> &'static str { + Self::COMPATIBLE + } @@ -1408,23 +1448,22 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/arm/gicv2.rs 13_exc + + fn register_handler( + &self, -+ irq_number: Self::IRQNumberType, -+ descriptor: exception::asynchronous::IRQDescriptor, ++ irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, + ) -> Result<(), &'static str> { + self.handler_table.write(|table| { -+ let irq_number = irq_number.get(); ++ let irq_number = irq_handler_descriptor.number().get(); + + if table[irq_number].is_some() { + return Err("IRQ handler already registered"); + } + -+ table[irq_number] = Some(descriptor); ++ table[irq_number] = Some(irq_handler_descriptor); + + Ok(()) + }) + } + -+ fn enable(&self, irq_number: Self::IRQNumberType) { ++ fn enable(&self, irq_number: &Self::IRQNumberType) { + self.gicd.enable(irq_number); + } + @@ -1447,7 +1486,7 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/arm/gicv2.rs 13_exc + None => panic!("No handler registered for IRQ {}", irq_number), + Some(descriptor) => { + // Call the IRQ handler. Panics on failure. -+ descriptor.handler.handle().expect("Error handling IRQ"); ++ descriptor.handler().handle().expect("Error handling IRQ"); + } + } + }); @@ -1464,7 +1503,7 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/arm/gicv2.rs 13_exc + self.handler_table.read(|table| { + for (i, opt) in table.iter().skip(32).enumerate() { + if let Some(handler) = opt { -+ info!(" {: >3}. {}", i + 32, handler.name); ++ info!(" {: >3}. {}", i + 32, handler.name()); + } + } + }); @@ -1488,12 +1527,14 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/arm.rs 13_exception diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs --- 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs -@@ -6,7 +6,7 @@ +@@ -5,8 +5,8 @@ + //! GPIO Driver. use crate::{ - bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, +- bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, - synchronization::NullLock, -+ synchronization::IRQSafeNullLock, ++ bsp::device_driver::common::MMIODerefWrapper, driver, exception::asynchronous::IRQNumber, ++ synchronization, synchronization::IRQSafeNullLock, }; use tock_registers::{ interfaces::{ReadWriteable, Writeable}, @@ -1515,6 +1556,15 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs } } +@@ -216,6 +216,8 @@ + use synchronization::interface::Mutex; + + impl driver::interface::DeviceDriver for GPIO { ++ type IRQNumberType = IRQNumber; ++ + fn compatible(&self) -> &'static str { + Self::COMPATIBLE + } diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs --- 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -1572,7 +1622,8 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru +/// Abstraction for the ReadOnly parts of the associated MMIO registers. +type ReadOnlyRegisters = MMIODerefWrapper; + -+type HandlerTable = [Option; PeripheralIRQ::NUM_TOTAL]; ++type HandlerTable = [Option>; ++ PeripheralIRQ::MAX_INCLUSIVE + 1]; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions @@ -1604,7 +1655,7 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru + Self { + wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(mmio_start_addr)), + ro_registers: ReadOnlyRegisters::new(mmio_start_addr), -+ handler_table: InitStateLock::new([None; PeripheralIRQ::NUM_TOTAL]), ++ handler_table: InitStateLock::new([None; PeripheralIRQ::MAX_INCLUSIVE + 1]), + } + } + @@ -1627,23 +1678,22 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru + + fn register_handler( + &self, -+ irq: Self::IRQNumberType, -+ descriptor: exception::asynchronous::IRQDescriptor, ++ irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, + ) -> Result<(), &'static str> { + self.handler_table.write(|table| { -+ let irq_number = irq.get(); ++ let irq_number = irq_handler_descriptor.number().get(); + + if table[irq_number].is_some() { + return Err("IRQ handler already registered"); + } + -+ table[irq_number] = Some(descriptor); ++ table[irq_number] = Some(irq_handler_descriptor); + + Ok(()) + }) + } + -+ fn enable(&self, irq: Self::IRQNumberType) { ++ fn enable(&self, irq: &Self::IRQNumberType) { + self.wo_registers.lock(|regs| { + let enable_reg = if irq.get() <= 31 { + ®s.ENABLE_1 @@ -1669,7 +1719,7 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru + None => panic!("No handler registered for IRQ {}", irq_number), + Some(descriptor) => { + // Call the IRQ handler. Panics on failure. -+ descriptor.handler.handle().expect("Error handling IRQ"); ++ descriptor.handler().handle().expect("Error handling IRQ"); + } + } + } @@ -1684,7 +1734,7 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru + self.handler_table.read(|table| { + for (i, opt) in table.iter().enumerate() { + if let Some(handler) = opt { -+ info!(" {: >3}. {}", i, handler.name); ++ info!(" {: >3}. {}", i, handler.name()); + } + } + }); @@ -1694,7 +1744,7 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs --- 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs -@@ -0,0 +1,131 @@ +@@ -0,0 +1,152 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2022 Andre Richter @@ -1703,7 +1753,12 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru + +mod peripheral_ic; + -+use crate::{driver, exception}; ++use crate::{ ++ bsp::device_driver::common::BoundedUsize, ++ driver, ++ exception::{self, asynchronous::IRQHandlerDescriptor}, ++}; ++use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions @@ -1718,10 +1773,8 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru +// Public Definitions +//-------------------------------------------------------------------------------------------------- + -+pub type LocalIRQ = -+ exception::asynchronous::IRQNumber<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; -+pub type PeripheralIRQ = -+ exception::asynchronous::IRQNumber<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; ++pub type LocalIRQ = BoundedUsize<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; ++pub type PeripheralIRQ = BoundedUsize<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; + +/// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. +#[derive(Copy, Clone)] @@ -1764,6 +1817,15 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru +// Public Code +//-------------------------------------------------------------------------------------------------- + ++impl fmt::Display for IRQNumber { ++ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { ++ match self { ++ Self::Local(number) => write!(f, "Local({})", number), ++ Self::Peripheral(number) => write!(f, "Peripheral({})", number), ++ } ++ } ++} ++ +impl InterruptController { + // Restrict to 3 for now. This makes future code for local_ic.rs more straight forward. + const MAX_LOCAL_IRQ_NUMBER: usize = 3; @@ -1788,6 +1850,8 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru +//------------------------------------------------------------------------------ + +impl driver::interface::DeviceDriver for InterruptController { ++ type IRQNumberType = IRQNumber; ++ + fn compatible(&self) -> &'static str { + Self::COMPATIBLE + } @@ -1798,16 +1862,23 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru + + fn register_handler( + &self, -+ irq: Self::IRQNumberType, -+ descriptor: exception::asynchronous::IRQDescriptor, ++ irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, + ) -> Result<(), &'static str> { -+ match irq { ++ match irq_handler_descriptor.number() { + IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), -+ IRQNumber::Peripheral(pirq) => self.periph.register_handler(pirq, descriptor), ++ IRQNumber::Peripheral(pirq) => { ++ let periph_descriptor = IRQHandlerDescriptor::new( ++ pirq, ++ irq_handler_descriptor.name(), ++ irq_handler_descriptor.handler(), ++ ); ++ ++ self.periph.register_handler(periph_descriptor) ++ } + } + } + -+ fn enable(&self, irq: Self::IRQNumberType) { ++ fn enable(&self, irq: &Self::IRQNumberType) { + match irq { + IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), + IRQNumber::Peripheral(pirq) => self.periph.enable(pirq), @@ -1830,18 +1901,21 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_interru diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -10,8 +10,8 @@ +@@ -10,8 +10,11 @@ //! - use crate::{ - bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, synchronization, - synchronization::NullLock, -+ bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, -+ synchronization, synchronization::IRQSafeNullLock, ++ bsp::device_driver::common::MMIODerefWrapper, ++ console, cpu, driver, ++ exception::{self, asynchronous::IRQNumber}, ++ synchronization, ++ synchronization::IRQSafeNullLock, }; use core::fmt; use tock_registers::{ -@@ -134,6 +134,52 @@ +@@ -134,6 +137,52 @@ ] ], @@ -1894,7 +1968,7 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_u /// Interrupt Clear Register. ICR [ /// Meta field for all pending interrupts. -@@ -152,7 +198,10 @@ +@@ -152,7 +201,10 @@ (0x28 => FBRD: WriteOnly), (0x2c => LCR_H: WriteOnly), (0x30 => CR: WriteOnly), @@ -1906,17 +1980,16 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_u (0x44 => ICR: WriteOnly), (0x48 => @END), } -@@ -179,7 +228,8 @@ +@@ -179,7 +231,7 @@ /// Representation of the UART. pub struct PL011Uart { - inner: NullLock, + inner: IRQSafeNullLock, -+ irq_number: bsp::device_driver::IRQNumber, } //-------------------------------------------------------------------------------------------------- -@@ -247,6 +297,14 @@ +@@ -247,6 +299,14 @@ .LCR_H .write(LCR_H::WLEN::EightBit + LCR_H::FEN::FifosEnabled); @@ -1931,37 +2004,39 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_u // Turn the UART on. self.registers .CR -@@ -335,9 +393,13 @@ - /// # Safety - /// +@@ -337,7 +397,7 @@ /// - The user must ensure to provide a correct MMIO start address. -- pub const unsafe fn new(mmio_start_addr: usize) -> Self { -+ pub const unsafe fn new( -+ mmio_start_addr: usize, -+ irq_number: bsp::device_driver::IRQNumber, -+ ) -> Self { + pub const unsafe fn new(mmio_start_addr: usize) -> Self { Self { - inner: NullLock::new(PL011UartInner::new(mmio_start_addr)), + inner: IRQSafeNullLock::new(PL011UartInner::new(mmio_start_addr)), -+ irq_number, } } } +@@ -348,6 +408,8 @@ + use synchronization::interface::Mutex; + + impl driver::interface::DeviceDriver for PL011Uart { ++ type IRQNumberType = IRQNumber; ++ + fn compatible(&self) -> &'static str { + Self::COMPATIBLE + } @@ -357,6 +419,20 @@ Ok(()) } + -+ fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { -+ use exception::asynchronous::{irq_manager, IRQDescriptor}; ++ fn register_and_enable_irq_handler( ++ &'static self, ++ irq_number: &Self::IRQNumberType, ++ ) -> Result<(), &'static str> { ++ use exception::asynchronous::{irq_manager, IRQHandlerDescriptor}; + -+ let descriptor = IRQDescriptor { -+ name: Self::COMPATIBLE, -+ handler: self, -+ }; ++ let descriptor = IRQHandlerDescriptor::new(*irq_number, Self::COMPATIBLE, self); + -+ irq_manager().register_handler(self.irq_number, descriptor)?; -+ irq_manager().enable(self.irq_number); ++ irq_manager().register_handler(descriptor)?; ++ irq_manager().enable(irq_number); + + Ok(()) + } @@ -2010,6 +2085,56 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/bcm.rs 13_exception +pub use bcm2xxx_interrupt_controller::*; pub use bcm2xxx_pl011_uart::*; +diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver/common.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/common.rs +--- 12_integrated_testing/kernel/src/bsp/device_driver/common.rs ++++ 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/common.rs +@@ -4,7 +4,7 @@ + + //! Common device driver code. + +-use core::{marker::PhantomData, ops}; ++use core::{fmt, marker::PhantomData, ops}; + + //-------------------------------------------------------------------------------------------------- + // Public Definitions +@@ -15,6 +15,10 @@ + phantom: PhantomData T>, + } + ++/// A wrapper type for usize with integrated range bound check. ++#[derive(Copy, Clone)] ++pub struct BoundedUsize(usize); ++ + //-------------------------------------------------------------------------------------------------- + // Public Code + //-------------------------------------------------------------------------------------------------- +@@ -36,3 +40,25 @@ + unsafe { &*(self.start_addr as *const _) } + } + } ++ ++impl BoundedUsize<{ MAX_INCLUSIVE }> { ++ pub const MAX_INCLUSIVE: usize = MAX_INCLUSIVE; ++ ++ /// Creates a new instance if number <= MAX_INCLUSIVE. ++ pub const fn new(number: usize) -> Self { ++ assert!(number <= MAX_INCLUSIVE); ++ ++ Self(number) ++ } ++ ++ /// Return the wrapped number. ++ pub const fn get(self) -> usize { ++ self.0 ++ } ++} ++ ++impl fmt::Display for BoundedUsize<{ MAX_INCLUSIVE }> { ++ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { ++ write!(f, "{}", self.0) ++ } ++} + diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver.rs --- 12_integrated_testing/kernel/src/bsp/device_driver.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver.rs @@ -2031,72 +2156,111 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/device_driver.rs 13_exceptions_pa diff -uNr 12_integrated_testing/kernel/src/bsp/raspberrypi/driver.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/driver.rs --- 12_integrated_testing/kernel/src/bsp/raspberrypi/driver.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/driver.rs -@@ -4,29 +4,43 @@ +@@ -4,8 +4,12 @@ //! BSP driver support. -use super::memory::map::mmio; +-use crate::{bsp::device_driver, console, driver as generic_driver}; +use super::{exception, memory::map::mmio}; - use crate::{bsp::device_driver, driver}; ++use crate::{ ++ bsp::device_driver, ++ console, driver as generic_driver, ++ exception::{self as generic_exception}, ++}; + use core::sync::atomic::{AtomicBool, Ordering}; -+pub use device_driver::IRQNumber; + //-------------------------------------------------------------------------------------------------- +@@ -16,6 +20,14 @@ + unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; + static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + ++#[cfg(feature = "bsp_rpi3")] ++static INTERRUPT_CONTROLLER: device_driver::InterruptController = ++ unsafe { device_driver::InterruptController::new(mmio::PERIPHERAL_IC_START) }; ++ ++#[cfg(feature = "bsp_rpi4")] ++static INTERRUPT_CONTROLLER: device_driver::GICv2 = ++ unsafe { device_driver::GICv2::new(mmio::GICD_START, mmio::GICC_START) }; + //-------------------------------------------------------------------------------------------------- - // Private Definitions + // Private Code //-------------------------------------------------------------------------------------------------- - - /// Device Driver Manager type. - struct BSPDriverManager { -- device_drivers: [&'static (dyn DeviceDriver + Sync); 2], -+ device_drivers: [&'static (dyn DeviceDriver + Sync); 3], +@@ -33,21 +45,43 @@ + Ok(()) } - //-------------------------------------------------------------------------------------------------- - // Global instances - //-------------------------------------------------------------------------------------------------- ++/// This must be called only after successful init of the interrupt controller driver. ++fn post_init_interrupt_controller() -> Result<(), &'static str> { ++ generic_exception::asynchronous::register_irq_manager(&INTERRUPT_CONTROLLER); ++ ++ Ok(()) ++} ++ + fn driver_uart() -> Result<(), &'static str> { +- let uart_descriptor = +- generic_driver::DeviceDriverDescriptor::new(&PL011_UART, Some(post_init_uart)); ++ let uart_descriptor = generic_driver::DeviceDriverDescriptor::new( ++ &PL011_UART, ++ Some(post_init_uart), ++ Some(exception::asynchronous::irq_map::PL011_UART), ++ ); + generic_driver::driver_manager().register_driver(uart_descriptor); + + Ok(()) + } --pub(super) static PL011_UART: device_driver::PL011Uart = -- unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; -+pub(super) static PL011_UART: device_driver::PL011Uart = unsafe { -+ device_driver::PL011Uart::new( -+ mmio::PL011_UART_START, -+ exception::asynchronous::irq_map::PL011_UART, -+ ) -+}; + fn driver_gpio() -> Result<(), &'static str> { +- let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio)); ++ let gpio_descriptor = ++ generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio), None); + generic_driver::driver_manager().register_driver(gpio_descriptor); - static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + Ok(()) + } -+#[cfg(feature = "bsp_rpi3")] -+pub(super) static INTERRUPT_CONTROLLER: device_driver::InterruptController = -+ unsafe { device_driver::InterruptController::new(mmio::PERIPHERAL_INTERRUPT_CONTROLLER_START) }; ++fn driver_interrupt_controller() -> Result<(), &'static str> { ++ let interrupt_controller_descriptor = generic_driver::DeviceDriverDescriptor::new( ++ &INTERRUPT_CONTROLLER, ++ Some(post_init_interrupt_controller), ++ None, ++ ); ++ generic_driver::driver_manager().register_driver(interrupt_controller_descriptor); + -+#[cfg(feature = "bsp_rpi4")] -+pub(super) static INTERRUPT_CONTROLLER: device_driver::GICv2 = -+ unsafe { device_driver::GICv2::new(mmio::GICD_START, mmio::GICC_START) }; ++ Ok(()) ++} + - static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { -- device_drivers: [&PL011_UART, &GPIO], -+ device_drivers: [&PL011_UART, &GPIO, &INTERRUPT_CONTROLLER], - }; - //-------------------------------------------------------------------------------------------------- + // Public Code + //-------------------------------------------------------------------------------------------------- +@@ -65,6 +99,7 @@ + + driver_uart()?; + driver_gpio()?; ++ driver_interrupt_controller()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) diff -uNr 12_integrated_testing/kernel/src/bsp/raspberrypi/exception/asynchronous.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/exception/asynchronous.rs --- 12_integrated_testing/kernel/src/bsp/raspberrypi/exception/asynchronous.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/exception/asynchronous.rs -@@ -0,0 +1,36 @@ +@@ -0,0 +1,28 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2022 Andre Richter + +//! BSP asynchronous exception handling. + -+use crate::{bsp, bsp::driver, exception}; ++use crate::bsp; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + ++/// Export for reuse in generic asynchronous.rs. ++pub use bsp::device_driver::IRQNumber; ++ +#[cfg(feature = "bsp_rpi3")] +pub(in crate::bsp) mod irq_map { + use super::bsp::device_driver::{IRQNumber, PeripheralIRQ}; @@ -2110,17 +2274,6 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/raspberrypi/exception/asynchronou + + pub const PL011_UART: IRQNumber = IRQNumber::new(153); +} -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- -+ -+/// Return a reference to the IRQ manager. -+pub fn irq_manager() -> &'static impl exception::asynchronous::interface::IRQManager< -+ IRQNumberType = bsp::device_driver::IRQNumber, -+> { -+ &driver::INTERRUPT_CONTROLLER -+} diff -uNr 12_integrated_testing/kernel/src/bsp/raspberrypi/exception.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/exception.rs --- 12_integrated_testing/kernel/src/bsp/raspberrypi/exception.rs @@ -2145,11 +2298,11 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/raspberrypi/memory.rs 13_exceptio - pub const GPIO_START: usize = START + GPIO_OFFSET; - pub const PL011_UART_START: usize = START + UART_OFFSET; - pub const END_INCLUSIVE: usize = 0x4000_FFFF; -+ pub const START: usize = 0x3F00_0000; -+ pub const PERIPHERAL_INTERRUPT_CONTROLLER_START: usize = START + 0x0000_B200; -+ pub const GPIO_START: usize = START + GPIO_OFFSET; -+ pub const PL011_UART_START: usize = START + UART_OFFSET; -+ pub const END_INCLUSIVE: usize = 0x4000_FFFF; ++ pub const START: usize = 0x3F00_0000; ++ pub const PERIPHERAL_IC_START: usize = START + 0x0000_B200; ++ pub const GPIO_START: usize = START + GPIO_OFFSET; ++ pub const PL011_UART_START: usize = START + UART_OFFSET; ++ pub const END_INCLUSIVE: usize = 0x4000_FFFF; } /// Physical devices. @@ -2166,8 +2319,8 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/raspberrypi/memory.rs 13_exceptio diff -uNr 12_integrated_testing/kernel/src/bsp/raspberrypi.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi.rs --- 12_integrated_testing/kernel/src/bsp/raspberrypi.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi.rs -@@ -7,6 +7,7 @@ - pub mod console; +@@ -6,6 +6,7 @@ + pub mod cpu; pub mod driver; +pub mod exception; @@ -2175,6 +2328,47 @@ diff -uNr 12_integrated_testing/kernel/src/bsp/raspberrypi.rs 13_exceptions_part //-------------------------------------------------------------------------------------------------- +diff -uNr 12_integrated_testing/kernel/src/console.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/console.rs +--- 12_integrated_testing/kernel/src/console.rs ++++ 13_exceptions_part2_peripheral_IRQs/kernel/src/console.rs +@@ -6,7 +6,7 @@ + + mod null_console; + +-use crate::synchronization::{self, NullLock}; ++use crate::synchronization; + + //-------------------------------------------------------------------------------------------------- + // Public Definitions +@@ -60,22 +60,22 @@ + // Global instances + //-------------------------------------------------------------------------------------------------- + +-static CUR_CONSOLE: NullLock<&'static (dyn interface::All + Sync)> = +- NullLock::new(&null_console::NULL_CONSOLE); ++static CUR_CONSOLE: InitStateLock<&'static (dyn interface::All + Sync)> = ++ InitStateLock::new(&null_console::NULL_CONSOLE); + + //-------------------------------------------------------------------------------------------------- + // Public Code + //-------------------------------------------------------------------------------------------------- +-use synchronization::interface::Mutex; ++use synchronization::{interface::ReadWriteEx, InitStateLock}; + + /// Register a new console. + pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { +- CUR_CONSOLE.lock(|con| *con = new_console); ++ CUR_CONSOLE.write(|con| *con = new_console); + } + + /// Return a reference to the currently registered console. + /// + /// This is the global console used by all printing macros. + pub fn console() -> &'static dyn interface::All { +- CUR_CONSOLE.lock(|con| *con) ++ CUR_CONSOLE.read(|con| *con) + } + diff -uNr 12_integrated_testing/kernel/src/cpu/smp.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/cpu/smp.rs --- 12_integrated_testing/kernel/src/cpu/smp.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/src/cpu/smp.rs @@ -2210,32 +2404,267 @@ diff -uNr 12_integrated_testing/kernel/src/cpu.rs 13_exceptions_part2_peripheral diff -uNr 12_integrated_testing/kernel/src/driver.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/driver.rs --- 12_integrated_testing/kernel/src/driver.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/src/driver.rs -@@ -23,6 +23,14 @@ +@@ -5,9 +5,10 @@ + //! Driver support. + + use crate::{ +- info, +- synchronization::{interface::Mutex, NullLock}, ++ exception, info, ++ synchronization::{interface::ReadWriteEx, InitStateLock}, + }; ++use core::fmt; + + //-------------------------------------------------------------------------------------------------- + // Private Definitions +@@ -15,9 +16,12 @@ + + const NUM_DRIVERS: usize = 5; + +-struct DriverManagerInner { ++struct DriverManagerInner ++where ++ T: 'static, ++{ + next_index: usize, +- descriptors: [Option; NUM_DRIVERS], ++ descriptors: [Option>; NUM_DRIVERS], + } + + //-------------------------------------------------------------------------------------------------- +@@ -28,6 +32,9 @@ + pub mod interface { + /// Device Driver functions. + pub trait DeviceDriver { ++ /// Different interrupt controllers might use different types for IRQ number. ++ type IRQNumberType: super::fmt::Display; ++ + /// Return a compatibility string for identifying the driver. + fn compatible(&self) -> &'static str; + +@@ -39,6 +46,21 @@ unsafe fn init(&self) -> Result<(), &'static str> { Ok(()) } + -+ /// Called by the kernel to register and enable the device's IRQ handlers, if any. ++ /// Called by the kernel to register and enable the device's IRQ handler. + /// + /// Rust's type system will prevent a call to this function unless the calling instance + /// itself has static lifetime. -+ fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { -+ Ok(()) ++ fn register_and_enable_irq_handler( ++ &'static self, ++ irq_number: &Self::IRQNumberType, ++ ) -> Result<(), &'static str> { ++ panic!( ++ "Attempt to enable IRQ {} for device {}, but driver does not support this", ++ irq_number, ++ self.compatible() ++ ) + } } + } + +@@ -47,27 +69,37 @@ + + /// A descriptor for device drivers. + #[derive(Copy, Clone)] +-pub struct DeviceDriverDescriptor { +- device_driver: &'static (dyn interface::DeviceDriver + Sync), ++pub struct DeviceDriverDescriptor ++where ++ T: 'static, ++{ ++ device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, ++ irq_number: Option, + } + + /// Provides device driver management functions. +-pub struct DriverManager { +- inner: NullLock, ++pub struct DriverManager ++where ++ T: 'static, ++{ ++ inner: InitStateLock>, + } + + //-------------------------------------------------------------------------------------------------- + // Global instances + //-------------------------------------------------------------------------------------------------- - /// Device driver management functions. +-static DRIVER_MANAGER: DriverManager = DriverManager::new(); ++static DRIVER_MANAGER: DriverManager = DriverManager::new(); + + //-------------------------------------------------------------------------------------------------- + // Private Code + //-------------------------------------------------------------------------------------------------- + +-impl DriverManagerInner { ++impl DriverManagerInner ++where ++ T: 'static + Copy, ++{ + /// Create an instance. + pub const fn new() -> Self { + Self { +@@ -81,43 +113,48 @@ + // Public Code + //-------------------------------------------------------------------------------------------------- + +-impl DeviceDriverDescriptor { ++impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( +- device_driver: &'static (dyn interface::DeviceDriver + Sync), ++ device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, ++ irq_number: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, ++ irq_number, + } + } + } + + /// Return a reference to the global DriverManager. +-pub fn driver_manager() -> &'static DriverManager { ++pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER + } + +-impl DriverManager { ++impl DriverManager ++where ++ T: fmt::Display + Copy, ++{ + /// Create an instance. + pub const fn new() -> Self { + Self { +- inner: NullLock::new(DriverManagerInner::new()), ++ inner: InitStateLock::new(DriverManagerInner::new()), + } + } + + /// Register a device driver with the kernel. +- pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { +- self.inner.lock(|inner| { ++ pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { ++ self.inner.write(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. +- fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { +- self.inner.lock(|inner| { ++ fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { ++ self.inner.read(|inner| { + inner + .descriptors + .iter() +@@ -126,12 +163,12 @@ + }) + } + +- /// Fully initialize all drivers. ++ /// Fully initialize all drivers and their interrupts handlers. + /// + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. +- pub unsafe fn init_drivers(&self) { ++ pub unsafe fn init_drivers_and_irqs(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { +@@ -150,6 +187,23 @@ + descriptor.device_driver.compatible(), + x + ); ++ } ++ } ++ }); ++ ++ // 3. After all post-init callbacks were done, the interrupt controller should be ++ // registered and functional. So let drivers register with it now. ++ self.for_each_descriptor(|descriptor| { ++ if let Some(irq_number) = &descriptor.irq_number { ++ if let Err(x) = descriptor ++ .device_driver ++ .register_and_enable_irq_handler(irq_number) ++ { ++ panic!( ++ "Error during driver interrupt handler registration: {}: {}", ++ descriptor.device_driver.compatible(), ++ x ++ ); + } + } + }); + +diff -uNr 12_integrated_testing/kernel/src/exception/asynchronous/null_irq_manager.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous/null_irq_manager.rs +--- 12_integrated_testing/kernel/src/exception/asynchronous/null_irq_manager.rs ++++ 13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous/null_irq_manager.rs +@@ -0,0 +1,42 @@ ++// SPDX-License-Identifier: MIT OR Apache-2.0 ++// ++// Copyright (c) 2022 Andre Richter ++ ++//! Null IRQ Manager. ++ ++use super::{interface, IRQContext, IRQHandlerDescriptor}; ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Definitions ++//-------------------------------------------------------------------------------------------------- ++ ++pub struct NullIRQManager; ++ ++//-------------------------------------------------------------------------------------------------- ++// Global instances ++//-------------------------------------------------------------------------------------------------- ++ ++pub static NULL_IRQ_MANAGER: NullIRQManager = NullIRQManager {}; ++ ++//-------------------------------------------------------------------------------------------------- ++// Public Code ++//-------------------------------------------------------------------------------------------------- ++ ++impl interface::IRQManager for NullIRQManager { ++ type IRQNumberType = super::IRQNumber; ++ ++ fn register_handler( ++ &self, ++ _descriptor: IRQHandlerDescriptor, ++ ) -> Result<(), &'static str> { ++ panic!("No IRQ Manager registered yet"); ++ } ++ ++ fn enable(&self, _irq_number: &Self::IRQNumberType) { ++ panic!("No IRQ Manager registered yet"); ++ } ++ ++ fn handle_pending_irqs<'irq_context>(&'irq_context self, _ic: &IRQContext<'irq_context>) { ++ panic!("No IRQ Manager registered yet"); ++ } ++} diff -uNr 12_integrated_testing/kernel/src/exception/asynchronous.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous.rs --- 12_integrated_testing/kernel/src/exception/asynchronous.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous.rs -@@ -8,7 +8,152 @@ +@@ -7,8 +7,184 @@ + #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/exception/asynchronous.rs"] mod arch_asynchronous; - -+use crate::bsp; -+use core::{fmt, marker::PhantomData}; ++mod null_irq_manager; + ++use crate::{bsp, synchronization}; ++use core::marker::PhantomData; + //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports //-------------------------------------------------------------------------------------------------- @@ -2249,14 +2678,23 @@ diff -uNr 12_integrated_testing/kernel/src/exception/asynchronous.rs 13_exceptio +// Public Definitions +//-------------------------------------------------------------------------------------------------- + ++/// Interrupt number as defined by the BSP. ++pub type IRQNumber = bsp::exception::asynchronous::IRQNumber; ++ +/// Interrupt descriptor. +#[derive(Copy, Clone)] -+pub struct IRQDescriptor { ++pub struct IRQHandlerDescriptor ++where ++ T: Copy, ++{ ++ /// The IRQ number. ++ number: T, ++ + /// Descriptive name. -+ pub name: &'static str, ++ name: &'static str, + + /// Reference to handler trait object. -+ pub handler: &'static (dyn interface::IRQHandler + Sync), ++ handler: &'static (dyn interface::IRQHandler + Sync), +} + +/// IRQContext token. @@ -2286,17 +2724,16 @@ diff -uNr 12_integrated_testing/kernel/src/exception/asynchronous.rs 13_exceptio + /// platform's interrupt controller. + pub trait IRQManager { + /// The IRQ number type depends on the implementation. -+ type IRQNumberType; ++ type IRQNumberType: Copy; + + /// Register a handler. + fn register_handler( + &self, -+ irq_number: Self::IRQNumberType, -+ descriptor: super::IRQDescriptor, ++ irq_handler_descriptor: super::IRQHandlerDescriptor, + ) -> Result<(), &'static str>; + + /// Enable an interrupt in the controller. -+ fn enable(&self, irq_number: Self::IRQNumberType); ++ fn enable(&self, irq_number: &Self::IRQNumberType); + + /// Handle pending interrupts. + /// @@ -2312,17 +2749,55 @@ diff -uNr 12_integrated_testing/kernel/src/exception/asynchronous.rs 13_exceptio + ); + + /// Print list of registered handlers. -+ fn print_handler(&self); ++ fn print_handler(&self) {} + } +} + -+/// A wrapper type for IRQ numbers with integrated range sanity check. -+#[derive(Copy, Clone)] -+pub struct IRQNumber(usize); ++//-------------------------------------------------------------------------------------------------- ++// Global instances ++//-------------------------------------------------------------------------------------------------- ++ ++static CUR_IRQ_MANAGER: InitStateLock< ++ &'static (dyn interface::IRQManager + Sync), ++> = InitStateLock::new(&null_irq_manager::NULL_IRQ_MANAGER); + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- ++use synchronization::{interface::ReadWriteEx, InitStateLock}; ++ ++impl IRQHandlerDescriptor ++where ++ T: Copy, ++{ ++ /// Create an instance. ++ pub const fn new( ++ number: T, ++ name: &'static str, ++ handler: &'static (dyn interface::IRQHandler + Sync), ++ ) -> Self { ++ Self { ++ number, ++ name, ++ handler, ++ } ++ } ++ ++ /// Return the number. ++ pub const fn number(&self) -> T { ++ self.number ++ } ++ ++ /// Return the name. ++ pub const fn name(&self) -> &'static str { ++ self.name ++ } ++ ++ /// Return the handler. ++ pub const fn handler(&self) -> &'static (dyn interface::IRQHandler + Sync) { ++ self.handler ++ } ++} + +impl<'irq_context> IRQContext<'irq_context> { + /// Creates an IRQContext token. @@ -2341,29 +2816,6 @@ diff -uNr 12_integrated_testing/kernel/src/exception/asynchronous.rs 13_exceptio + } +} + -+impl IRQNumber<{ MAX_INCLUSIVE }> { -+ /// The total number of IRQs this type supports. -+ pub const NUM_TOTAL: usize = MAX_INCLUSIVE + 1; -+ -+ /// Creates a new instance if number <= MAX_INCLUSIVE. -+ pub const fn new(number: usize) -> Self { -+ assert!(number <= MAX_INCLUSIVE); -+ -+ Self(number) -+ } -+ -+ /// Return the wrapped number. -+ pub const fn get(self) -> usize { -+ self.0 -+ } -+} -+ -+impl fmt::Display for IRQNumber<{ MAX_INCLUSIVE }> { -+ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { -+ write!(f, "{}", self.0) -+ } -+} -+ +/// Executes the provided closure while IRQs are masked on the executing core. +/// +/// While the function temporarily changes the HW state of the executing core, it restores it to the @@ -2377,11 +2829,18 @@ diff -uNr 12_integrated_testing/kernel/src/exception/asynchronous.rs 13_exceptio + ret +} + -+/// Return a reference to the IRQ manager. ++/// Register a new IRQ manager. ++pub fn register_irq_manager( ++ new_manager: &'static (dyn interface::IRQManager + Sync), ++) { ++ CUR_IRQ_MANAGER.write(|manager| *manager = new_manager); ++} ++ ++/// Return a reference to the currently registered IRQ manager. +/// +/// This is the IRQ manager used by the architectural interrupt handling code. -+pub fn irq_manager() -> &'static dyn interface::IRQManager { -+ bsp::exception::asynchronous::irq_manager() ++pub fn irq_manager() -> &'static dyn interface::IRQManager { ++ CUR_IRQ_MANAGER.read(|manager| *manager) +} diff -uNr 12_integrated_testing/kernel/src/lib.rs 13_exceptions_part2_peripheral_IRQs/kernel/src/lib.rs @@ -2404,7 +2863,7 @@ diff -uNr 12_integrated_testing/kernel/src/main.rs 13_exceptions_part2_periphera #![no_std] -use libkernel::{bsp, console, driver, exception, info, memory, time}; -+use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; ++use libkernel::{bsp, cpu, driver, exception, info, memory, state, time}; /// Early init code. /// @@ -2416,43 +2875,39 @@ diff -uNr 12_integrated_testing/kernel/src/main.rs 13_exceptions_part2_periphera +/// IRQSafeNullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; -@@ -43,14 +43,27 @@ - bsp::driver::driver_manager().post_device_driver_init(); - // println! is usable from here on. - -+ // Let device drivers register and enable their handlers with the interrupt controller. -+ for i in bsp::driver::driver_manager().all_device_drivers() { -+ if let Err(msg) = i.register_and_enable_irq_handler() { -+ warn!("Error registering IRQ handler: {}", msg); -+ } -+ } + use memory::mmu::interface::MMU; +@@ -40,8 +40,13 @@ + } + + // Initialize all device drivers. +- driver::driver_manager().init_drivers(); +- // println! is usable from here on. ++ driver::driver_manager().init_drivers_and_irqs(); + + // Unmask interrupts on the boot CPU core. + exception::asynchronous::local_irq_unmask(); + + // Announce conclusion of the kernel_init() phase. + state::state_manager().transition_to_single_core_main(); -+ + // Transition from unsafe to safe. kernel_main() - } +@@ -49,8 +54,6 @@ /// The main function running after the early init. fn kernel_main() -> ! { - use console::console; - use driver::interface::DriverManager; -+ use exception::asynchronous::interface::IRQManager; - +- info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); -@@ -78,12 +91,9 @@ - info!(" {}. {}", i + 1, driver.compatible()); - } + +@@ -71,12 +74,9 @@ + info!("Drivers loaded:"); + driver::driver_manager().enumerate(); - info!("Echoing input now"); + info!("Registered IRQ handlers:"); -+ bsp::exception::asynchronous::irq_manager().print_handler(); ++ exception::asynchronous::irq_manager().print_handler(); - // Discard any spurious received characters before going into echo mode. - console().clear_rx(); @@ -2721,7 +3176,7 @@ diff -uNr 12_integrated_testing/kernel/src/synchronization.rs 13_exceptions_part diff -uNr 12_integrated_testing/kernel/tests/04_exception_irq_sanity.rs 13_exceptions_part2_peripheral_IRQs/kernel/tests/04_exception_irq_sanity.rs --- 12_integrated_testing/kernel/tests/04_exception_irq_sanity.rs +++ 13_exceptions_part2_peripheral_IRQs/kernel/tests/04_exception_irq_sanity.rs -@@ -0,0 +1,67 @@ +@@ -0,0 +1,66 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020-2022 Andre Richter @@ -2734,13 +3189,12 @@ diff -uNr 12_integrated_testing/kernel/tests/04_exception_irq_sanity.rs 13_excep +#![reexport_test_harness_main = "test_main"] +#![test_runner(libkernel::test_runner)] + -+use libkernel::{bsp, cpu, driver, exception}; ++use libkernel::{bsp, cpu, exception}; +use test_macros::kernel_test; + +#[no_mangle] +unsafe fn kernel_init() -> ! { -+ use driver::interface::DriverManager; -+ bsp::driver::driver_manager().qemu_bring_up_console(); ++ bsp::driver::qemu_bring_up_console(); + + exception::handling_init(); + exception::asynchronous::local_irq_unmask(); diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2.rs index c810a59a..2623c305 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2.rs @@ -79,20 +79,25 @@ mod gicc; mod gicd; -use crate::{bsp, cpu, driver, exception, synchronization, synchronization::InitStateLock}; +use crate::{ + bsp::{self, device_driver::common::BoundedUsize}, + cpu, driver, exception, synchronization, + synchronization::InitStateLock, +}; //-------------------------------------------------------------------------------------------------- // Private Definitions //-------------------------------------------------------------------------------------------------- -type HandlerTable = [Option; IRQNumber::NUM_TOTAL]; +type HandlerTable = [Option>; + IRQNumber::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. -pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER }>; +pub type IRQNumber = BoundedUsize<{ GICv2::MAX_IRQ_NUMBER }>; /// Representation of the GIC. pub struct GICv2 { @@ -124,7 +129,7 @@ impl GICv2 { Self { gicd: gicd::GICD::new(gicd_mmio_start_addr), gicc: gicc::GICC::new(gicc_mmio_start_addr), - handler_table: InitStateLock::new([None; IRQNumber::NUM_TOTAL]), + handler_table: InitStateLock::new([None; IRQNumber::MAX_INCLUSIVE + 1]), } } } @@ -135,6 +140,8 @@ impl GICv2 { use synchronization::interface::ReadWriteEx; impl driver::interface::DeviceDriver for GICv2 { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } @@ -156,23 +163,22 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq_number.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq_number: Self::IRQNumberType) { + fn enable(&self, irq_number: &Self::IRQNumberType) { self.gicd.enable(irq_number); } @@ -195,7 +201,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } }); @@ -212,7 +218,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { self.handler_table.read(|table| { for (i, opt) in table.iter().skip(32).enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i + 32, handler.name); + info!(" {: >3}. {}", i + 32, handler.name()); } } }); diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs index 54aab60c..2d18be9c 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -170,7 +170,7 @@ impl GICD { } /// Enable an interrupt. - pub fn enable(&self, irq_num: super::IRQNumber) { + pub fn enable(&self, irq_num: &super::IRQNumber) { let irq_num = irq_num.get(); // Each bit in the u32 enable register corresponds to one IRQ number. Shift right by 5 diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 0f3e701f..336cc8f7 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -5,8 +5,8 @@ //! GPIO Driver. use crate::{ - bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, - synchronization::IRQSafeNullLock, + bsp::device_driver::common::MMIODerefWrapper, driver, exception::asynchronous::IRQNumber, + synchronization, synchronization::IRQSafeNullLock, }; use tock_registers::{ interfaces::{ReadWriteable, Writeable}, @@ -216,6 +216,8 @@ impl GPIO { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for GPIO { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs index a88899c2..22298f1c 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -6,7 +6,12 @@ mod peripheral_ic; -use crate::{driver, exception}; +use crate::{ + bsp::device_driver::common::BoundedUsize, + driver, + exception::{self, asynchronous::IRQHandlerDescriptor}, +}; +use core::fmt; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -21,10 +26,8 @@ struct PendingIRQs { // Public Definitions //-------------------------------------------------------------------------------------------------- -pub type LocalIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; -pub type PeripheralIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; +pub type LocalIRQ = BoundedUsize<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; +pub type PeripheralIRQ = BoundedUsize<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. #[derive(Copy, Clone)] @@ -67,6 +70,15 @@ impl Iterator for PendingIRQs { // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for IRQNumber { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + match self { + Self::Local(number) => write!(f, "Local({})", number), + Self::Peripheral(number) => write!(f, "Peripheral({})", number), + } + } +} + impl InterruptController { // Restrict to 3 for now. This makes future code for local_ic.rs more straight forward. const MAX_LOCAL_IRQ_NUMBER: usize = 3; @@ -91,6 +103,8 @@ impl InterruptController { //------------------------------------------------------------------------------ impl driver::interface::DeviceDriver for InterruptController { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } @@ -101,16 +115,23 @@ impl exception::asynchronous::interface::IRQManager for InterruptController { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { - match irq { + match irq_handler_descriptor.number() { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), - IRQNumber::Peripheral(pirq) => self.periph.register_handler(pirq, descriptor), + IRQNumber::Peripheral(pirq) => { + let periph_descriptor = IRQHandlerDescriptor::new( + pirq, + irq_handler_descriptor.name(), + irq_handler_descriptor.handler(), + ); + + self.periph.register_handler(periph_descriptor) + } } } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { match irq { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), IRQNumber::Peripheral(pirq) => self.periph.enable(pirq), diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 90d9dd81..1af6f22f 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -50,7 +50,8 @@ type WriteOnlyRegisters = MMIODerefWrapper; /// Abstraction for the ReadOnly parts of the associated MMIO registers. type ReadOnlyRegisters = MMIODerefWrapper; -type HandlerTable = [Option; PeripheralIRQ::NUM_TOTAL]; +type HandlerTable = [Option>; + PeripheralIRQ::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -82,7 +83,7 @@ impl PeripheralIC { Self { wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(mmio_start_addr)), ro_registers: ReadOnlyRegisters::new(mmio_start_addr), - handler_table: InitStateLock::new([None; PeripheralIRQ::NUM_TOTAL]), + handler_table: InitStateLock::new([None; PeripheralIRQ::MAX_INCLUSIVE + 1]), } } @@ -105,23 +106,22 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { self.wo_registers.lock(|regs| { let enable_reg = if irq.get() <= 31 { ®s.ENABLE_1 @@ -147,7 +147,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } } @@ -162,7 +162,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { self.handler_table.read(|table| { for (i, opt) in table.iter().enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i, handler.name); + info!(" {: >3}. {}", i, handler.name()); } } }); diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index a96ebf7e..71b3e254 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -10,8 +10,11 @@ //! - use crate::{ - bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, - synchronization, synchronization::IRQSafeNullLock, + bsp::device_driver::common::MMIODerefWrapper, + console, cpu, driver, + exception::{self, asynchronous::IRQNumber}, + synchronization, + synchronization::IRQSafeNullLock, }; use core::fmt; use tock_registers::{ @@ -229,7 +232,6 @@ struct PL011UartInner { /// Representation of the UART. pub struct PL011Uart { inner: IRQSafeNullLock, - irq_number: bsp::device_driver::IRQNumber, } //-------------------------------------------------------------------------------------------------- @@ -393,13 +395,9 @@ impl PL011Uart { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new( - mmio_start_addr: usize, - irq_number: bsp::device_driver::IRQNumber, - ) -> Self { + pub const unsafe fn new(mmio_start_addr: usize) -> Self { Self { inner: IRQSafeNullLock::new(PL011UartInner::new(mmio_start_addr)), - irq_number, } } } @@ -410,6 +408,8 @@ impl PL011Uart { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for PL011Uart { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } @@ -420,16 +420,16 @@ impl driver::interface::DeviceDriver for PL011Uart { Ok(()) } - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - use exception::asynchronous::{irq_manager, IRQDescriptor}; + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + use exception::asynchronous::{irq_manager, IRQHandlerDescriptor}; - let descriptor = IRQDescriptor { - name: Self::COMPATIBLE, - handler: self, - }; + let descriptor = IRQHandlerDescriptor::new(*irq_number, Self::COMPATIBLE, self); - irq_manager().register_handler(self.irq_number, descriptor)?; - irq_manager().enable(self.irq_number); + irq_manager().register_handler(descriptor)?; + irq_manager().enable(irq_number); Ok(()) } diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/common.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/common.rs index fd9e988e..c368534a 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/common.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/common.rs @@ -4,7 +4,7 @@ //! Common device driver code. -use core::{marker::PhantomData, ops}; +use core::{fmt, marker::PhantomData, ops}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -15,6 +15,10 @@ pub struct MMIODerefWrapper { phantom: PhantomData T>, } +/// A wrapper type for usize with integrated range bound check. +#[derive(Copy, Clone)] +pub struct BoundedUsize(usize); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -36,3 +40,25 @@ impl ops::Deref for MMIODerefWrapper { unsafe { &*(self.start_addr as *const _) } } } + +impl BoundedUsize<{ MAX_INCLUSIVE }> { + pub const MAX_INCLUSIVE: usize = MAX_INCLUSIVE; + + /// Creates a new instance if number <= MAX_INCLUSIVE. + pub const fn new(number: usize) -> Self { + assert!(number <= MAX_INCLUSIVE); + + Self(number) + } + + /// Return the wrapped number. + pub const fn get(self) -> usize { + self.0 + } +} + +impl fmt::Display for BoundedUsize<{ MAX_INCLUSIVE }> { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + write!(f, "{}", self.0) + } +} diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi.rs index 50b36762..474419f4 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi.rs @@ -4,7 +4,6 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -pub mod console; pub mod cpu; pub mod driver; pub mod exception; diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/console.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/console.rs deleted file mode 100644 index 0a630eef..00000000 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/console.rs +++ /dev/null @@ -1,16 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2022 Andre Richter - -//! BSP console facilities. - -use crate::console; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return a reference to the console. -pub fn console() -> &'static dyn console::interface::All { - &super::driver::PL011_UART -} diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/driver.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/driver.rs index 568f1096..91dd6133 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/driver.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/driver.rs @@ -5,68 +5,109 @@ //! BSP driver support. use super::{exception, memory::map::mmio}; -use crate::{bsp::device_driver, driver}; - -pub use device_driver::IRQNumber; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: [&'static (dyn DeviceDriver + Sync); 3], -} +use crate::{ + bsp::device_driver, + console, driver as generic_driver, + exception::{self as generic_exception}, +}; +use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- -pub(super) static PL011_UART: device_driver::PL011Uart = unsafe { - device_driver::PL011Uart::new( - mmio::PL011_UART_START, - exception::asynchronous::irq_map::PL011_UART, - ) -}; - +static PL011_UART: device_driver::PL011Uart = + unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; #[cfg(feature = "bsp_rpi3")] -pub(super) static INTERRUPT_CONTROLLER: device_driver::InterruptController = - unsafe { device_driver::InterruptController::new(mmio::PERIPHERAL_INTERRUPT_CONTROLLER_START) }; +static INTERRUPT_CONTROLLER: device_driver::InterruptController = + unsafe { device_driver::InterruptController::new(mmio::PERIPHERAL_IC_START) }; #[cfg(feature = "bsp_rpi4")] -pub(super) static INTERRUPT_CONTROLLER: device_driver::GICv2 = +static INTERRUPT_CONTROLLER: device_driver::GICv2 = unsafe { device_driver::GICv2::new(mmio::GICD_START, mmio::GICC_START) }; -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: [&PL011_UART, &GPIO, &INTERRUPT_CONTROLLER], -}; - //-------------------------------------------------------------------------------------------------- -// Public Code +// Private Code //-------------------------------------------------------------------------------------------------- -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER +/// This must be called only after successful init of the UART driver. +fn post_init_uart() -> Result<(), &'static str> { + console::register_console(&PL011_UART); + + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +/// This must be called only after successful init of the GPIO driver. +fn post_init_gpio() -> Result<(), &'static str> { + GPIO.map_pl011_uart(); + Ok(()) +} -impl driver::interface::DriverManager for BSPDriverManager { - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { - &self.device_drivers[..] - } +/// This must be called only after successful init of the interrupt controller driver. +fn post_init_interrupt_controller() -> Result<(), &'static str> { + generic_exception::asynchronous::register_irq_manager(&INTERRUPT_CONTROLLER); + + Ok(()) +} - fn post_device_driver_init(&self) { - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); +fn driver_uart() -> Result<(), &'static str> { + let uart_descriptor = generic_driver::DeviceDriverDescriptor::new( + &PL011_UART, + Some(post_init_uart), + Some(exception::asynchronous::irq_map::PL011_UART), + ); + generic_driver::driver_manager().register_driver(uart_descriptor); + + Ok(()) +} + +fn driver_gpio() -> Result<(), &'static str> { + let gpio_descriptor = + generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio), None); + generic_driver::driver_manager().register_driver(gpio_descriptor); + + Ok(()) +} + +fn driver_interrupt_controller() -> Result<(), &'static str> { + let interrupt_controller_descriptor = generic_driver::DeviceDriverDescriptor::new( + &INTERRUPT_CONTROLLER, + Some(post_init_interrupt_controller), + None, + ); + generic_driver::driver_manager().register_driver(interrupt_controller_descriptor); + + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self) {} + driver_uart()?; + driver_gpio()?; + driver_interrupt_controller()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) +} + +/// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps +/// than on real hardware due to QEMU's abstractions. +#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() { + console::register_console(&PL011_UART); } diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/exception/asynchronous.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/exception/asynchronous.rs index a13b2444..06a67558 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/exception/asynchronous.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/exception/asynchronous.rs @@ -4,12 +4,15 @@ //! BSP asynchronous exception handling. -use crate::{bsp, bsp::driver, exception}; +use crate::bsp; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Export for reuse in generic asynchronous.rs. +pub use bsp::device_driver::IRQNumber; + #[cfg(feature = "bsp_rpi3")] pub(in crate::bsp) mod irq_map { use super::bsp::device_driver::{IRQNumber, PeripheralIRQ}; @@ -23,14 +26,3 @@ pub(in crate::bsp) mod irq_map { pub const PL011_UART: IRQNumber = IRQNumber::new(153); } - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return a reference to the IRQ manager. -pub fn irq_manager() -> &'static impl exception::asynchronous::interface::IRQManager< - IRQNumberType = bsp::device_driver::IRQNumber, -> { - &driver::INTERRUPT_CONTROLLER -} diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/memory.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/memory.rs index a383c9fe..be13bb7a 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/memory.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/memory.rs @@ -73,11 +73,11 @@ pub(super) mod map { pub mod mmio { use super::*; - pub const START: usize = 0x3F00_0000; - pub const PERIPHERAL_INTERRUPT_CONTROLLER_START: usize = START + 0x0000_B200; - pub const GPIO_START: usize = START + GPIO_OFFSET; - pub const PL011_UART_START: usize = START + UART_OFFSET; - pub const END_INCLUSIVE: usize = 0x4000_FFFF; + pub const START: usize = 0x3F00_0000; + pub const PERIPHERAL_IC_START: usize = START + 0x0000_B200; + pub const GPIO_START: usize = START + GPIO_OFFSET; + pub const PL011_UART_START: usize = START + UART_OFFSET; + pub const END_INCLUSIVE: usize = 0x4000_FFFF; } /// Physical devices. diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/console.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/console.rs index c1fb0e53..a85bcffe 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/console.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/console.rs @@ -4,7 +4,9 @@ //! System console. -use crate::bsp; +mod null_console; + +use crate::synchronization; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -54,13 +56,26 @@ pub mod interface { pub trait All: Write + Read + Statistics {} } +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static CUR_CONSOLE: InitStateLock<&'static (dyn interface::All + Sync)> = + InitStateLock::new(&null_console::NULL_CONSOLE); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +use synchronization::{interface::ReadWriteEx, InitStateLock}; + +/// Register a new console. +pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { + CUR_CONSOLE.write(|con| *con = new_console); +} -/// Return a reference to the console. +/// Return a reference to the currently registered console. /// /// This is the global console used by all printing macros. pub fn console() -> &'static dyn interface::All { - bsp::console::console() + CUR_CONSOLE.read(|con| *con) } diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/console/null_console.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/console/null_console.rs new file mode 100644 index 00000000..2c64d499 --- /dev/null +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/console/null_console.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter + +//! Null console. + +use super::interface; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct NullConsole; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +pub static NULL_CONSOLE: NullConsole = NullConsole {}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl interface::Write for NullConsole { + fn write_char(&self, _c: char) {} + + fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { + fmt::Result::Ok(()) + } + + fn flush(&self) {} +} + +impl interface::Read for NullConsole { + fn clear_rx(&self) {} +} + +impl interface::Statistics for NullConsole {} +impl interface::All for NullConsole {} diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/driver.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/driver.rs index c39e5eaf..18066c31 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/driver.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/driver.rs @@ -4,6 +4,26 @@ //! Driver support. +use crate::{ + exception, info, + synchronization::{interface::ReadWriteEx, InitStateLock}, +}; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner +where + T: 'static, +{ + next_index: usize, + descriptors: [Option>; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -12,6 +32,9 @@ pub mod interface { /// Device Driver functions. pub trait DeviceDriver { + /// Different interrupt controllers might use different types for IRQ number. + type IRQNumberType: super::fmt::Display; + /// Return a compatibility string for identifying the driver. fn compatible(&self) -> &'static str; @@ -24,34 +47,175 @@ pub mod interface { Ok(()) } - /// Called by the kernel to register and enable the device's IRQ handlers, if any. + /// Called by the kernel to register and enable the device's IRQ handler. /// /// Rust's type system will prevent a call to this function unless the calling instance /// itself has static lifetime. - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - Ok(()) + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + panic!( + "Attempt to enable IRQ {} for device {}, but driver does not support this", + irq_number, + self.compatible() + ) + } + } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor +where + T: 'static, +{ + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager +where + T: 'static, +{ + inner: InitStateLock>, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner +where + T: 'static + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], + } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + irq_number, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager +where + T: fmt::Display + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: InitStateLock::new(DriverManagerInner::new()), } } - /// Device driver management functions. + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.write(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.read(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers and their interrupts handlers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Return a slice of references to all `BSP`-instantiated drivers. - /// - /// # Safety - /// - /// - The order of devices is the order in which `DeviceDriver::init()` is called. - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers_and_irqs(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } - /// Initialization code that runs after driver init. - /// - /// For example, device driver code that depends on other drivers already being online. - fn post_device_driver_init(&self); + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + + // 3. After all post-init callbacks were done, the interrupt controller should be + // registered and functional. So let drivers register with it now. + self.for_each_descriptor(|descriptor| { + if let Some(irq_number) = &descriptor.irq_number { + if let Err(x) = descriptor + .device_driver + .register_and_enable_irq_handler(irq_number) + { + panic!( + "Error during driver interrupt handler registration: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } + + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); - /// Minimal code needed to bring up the console in QEMU (for testing only). This is often - /// less steps than on real hardware due to QEMU's abstractions. - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self); + i += 1; + }); } } diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous.rs index d55ce642..c1f2a27b 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous.rs @@ -7,9 +7,10 @@ #[cfg(target_arch = "aarch64")] #[path = "../_arch/aarch64/exception/asynchronous.rs"] mod arch_asynchronous; +mod null_irq_manager; -use crate::bsp; -use core::{fmt, marker::PhantomData}; +use crate::{bsp, synchronization}; +use core::marker::PhantomData; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports @@ -23,14 +24,23 @@ pub use arch_asynchronous::{ // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Interrupt number as defined by the BSP. +pub type IRQNumber = bsp::exception::asynchronous::IRQNumber; + /// Interrupt descriptor. #[derive(Copy, Clone)] -pub struct IRQDescriptor { +pub struct IRQHandlerDescriptor +where + T: Copy, +{ + /// The IRQ number. + number: T, + /// Descriptive name. - pub name: &'static str, + name: &'static str, /// Reference to handler trait object. - pub handler: &'static (dyn interface::IRQHandler + Sync), + handler: &'static (dyn interface::IRQHandler + Sync), } /// IRQContext token. @@ -60,17 +70,16 @@ pub mod interface { /// platform's interrupt controller. pub trait IRQManager { /// The IRQ number type depends on the implementation. - type IRQNumberType; + type IRQNumberType: Copy; /// Register a handler. fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: super::IRQDescriptor, + irq_handler_descriptor: super::IRQHandlerDescriptor, ) -> Result<(), &'static str>; /// Enable an interrupt in the controller. - fn enable(&self, irq_number: Self::IRQNumberType); + fn enable(&self, irq_number: &Self::IRQNumberType); /// Handle pending interrupts. /// @@ -86,17 +95,55 @@ pub mod interface { ); /// Print list of registered handlers. - fn print_handler(&self); + fn print_handler(&self) {} } } -/// A wrapper type for IRQ numbers with integrated range sanity check. -#[derive(Copy, Clone)] -pub struct IRQNumber(usize); +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static CUR_IRQ_MANAGER: InitStateLock< + &'static (dyn interface::IRQManager + Sync), +> = InitStateLock::new(&null_irq_manager::NULL_IRQ_MANAGER); //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +use synchronization::{interface::ReadWriteEx, InitStateLock}; + +impl IRQHandlerDescriptor +where + T: Copy, +{ + /// Create an instance. + pub const fn new( + number: T, + name: &'static str, + handler: &'static (dyn interface::IRQHandler + Sync), + ) -> Self { + Self { + number, + name, + handler, + } + } + + /// Return the number. + pub const fn number(&self) -> T { + self.number + } + + /// Return the name. + pub const fn name(&self) -> &'static str { + self.name + } + + /// Return the handler. + pub const fn handler(&self) -> &'static (dyn interface::IRQHandler + Sync) { + self.handler + } +} impl<'irq_context> IRQContext<'irq_context> { /// Creates an IRQContext token. @@ -115,29 +162,6 @@ impl<'irq_context> IRQContext<'irq_context> { } } -impl IRQNumber<{ MAX_INCLUSIVE }> { - /// The total number of IRQs this type supports. - pub const NUM_TOTAL: usize = MAX_INCLUSIVE + 1; - - /// Creates a new instance if number <= MAX_INCLUSIVE. - pub const fn new(number: usize) -> Self { - assert!(number <= MAX_INCLUSIVE); - - Self(number) - } - - /// Return the wrapped number. - pub const fn get(self) -> usize { - self.0 - } -} - -impl fmt::Display for IRQNumber<{ MAX_INCLUSIVE }> { - fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { - write!(f, "{}", self.0) - } -} - /// Executes the provided closure while IRQs are masked on the executing core. /// /// While the function temporarily changes the HW state of the executing core, it restores it to the @@ -151,9 +175,16 @@ pub fn exec_with_irq_masked(f: impl FnOnce() -> T) -> T { ret } -/// Return a reference to the IRQ manager. +/// Register a new IRQ manager. +pub fn register_irq_manager( + new_manager: &'static (dyn interface::IRQManager + Sync), +) { + CUR_IRQ_MANAGER.write(|manager| *manager = new_manager); +} + +/// Return a reference to the currently registered IRQ manager. /// /// This is the IRQ manager used by the architectural interrupt handling code. -pub fn irq_manager() -> &'static dyn interface::IRQManager { - bsp::exception::asynchronous::irq_manager() +pub fn irq_manager() -> &'static dyn interface::IRQManager { + CUR_IRQ_MANAGER.read(|manager| *manager) } diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous/null_irq_manager.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous/null_irq_manager.rs new file mode 100644 index 00000000..438f9649 --- /dev/null +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous/null_irq_manager.rs @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter + +//! Null IRQ Manager. + +use super::{interface, IRQContext, IRQHandlerDescriptor}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct NullIRQManager; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +pub static NULL_IRQ_MANAGER: NullIRQManager = NullIRQManager {}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl interface::IRQManager for NullIRQManager { + type IRQNumberType = super::IRQNumber; + + fn register_handler( + &self, + _descriptor: IRQHandlerDescriptor, + ) -> Result<(), &'static str> { + panic!("No IRQ Manager registered yet"); + } + + fn enable(&self, _irq_number: &Self::IRQNumberType) { + panic!("No IRQ Manager registered yet"); + } + + fn handle_pending_irqs<'irq_context>(&'irq_context self, _ic: &IRQContext<'irq_context>) { + panic!("No IRQ Manager registered yet"); + } +} diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/lib.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/lib.rs index 194e2455..93b863d7 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/lib.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/lib.rs @@ -183,10 +183,8 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); test_main(); diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/src/main.rs b/13_exceptions_part2_peripheral_IRQs/kernel/src/main.rs index 0f72d17f..d3bdf394 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/src/main.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/src/main.rs @@ -13,7 +13,7 @@ #![no_main] #![no_std] -use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; +use libkernel::{bsp, cpu, driver, exception, info, memory, state, time}; /// Early init code. /// @@ -26,7 +26,6 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// IRQSafeNullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; use memory::mmu::interface::MMU; exception::handling_init(); @@ -35,21 +34,14 @@ unsafe fn kernel_init() -> ! { panic!("MMU: {}", string); } - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } - } - bsp::driver::driver_manager().post_device_driver_init(); - // println! is usable from here on. - - // Let device drivers register and enable their handlers with the interrupt controller. - for i in bsp::driver::driver_manager().all_device_drivers() { - if let Err(msg) = i.register_and_enable_irq_handler() { - warn!("Error registering IRQ handler: {}", msg); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } + // Initialize all device drivers. + driver::driver_manager().init_drivers_and_irqs(); + // Unmask interrupts on the boot CPU core. exception::asynchronous::local_irq_unmask(); @@ -62,9 +54,6 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { - use driver::interface::DriverManager; - use exception::asynchronous::interface::IRQManager; - info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); @@ -83,16 +72,10 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); info!("Registered IRQ handlers:"); - bsp::exception::asynchronous::irq_manager().print_handler(); + exception::asynchronous::irq_manager().print_handler(); info!("Echoing input now"); cpu::wait_forever(); diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/tests/00_console_sanity.rs b/13_exceptions_part2_peripheral_IRQs/kernel/tests/00_console_sanity.rs index e12a711f..69313428 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/tests/00_console_sanity.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/tests/00_console_sanity.rs @@ -11,15 +11,14 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{bsp, console, cpu, driver, exception, print}; +use libkernel::{bsp, console, cpu, exception, print}; #[no_mangle] unsafe fn kernel_init() -> ! { use console::console; - use driver::interface::DriverManager; exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/tests/01_timer_sanity.rs b/13_exceptions_part2_peripheral_IRQs/kernel/tests/01_timer_sanity.rs index a0eb732b..b86016b6 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/tests/01_timer_sanity.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/tests/01_timer_sanity.rs @@ -11,15 +11,13 @@ #![test_runner(libkernel::test_runner)] use core::time::Duration; -use libkernel::{bsp, cpu, driver, exception, time}; +use libkernel::{bsp, cpu, exception, time}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/tests/02_exception_sync_page_fault.rs b/13_exceptions_part2_peripheral_IRQs/kernel/tests/02_exception_sync_page_fault.rs index 62908377..46501960 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/tests/02_exception_sync_page_fault.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/tests/02_exception_sync_page_fault.rs @@ -17,15 +17,14 @@ /// or indirectly. mod panic_exit_success; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; use memory::mmu::interface::MMU; exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/tests/03_exception_restore_sanity.rs b/13_exceptions_part2_peripheral_IRQs/kernel/tests/03_exception_restore_sanity.rs index 3f4eae79..cba9285f 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/tests/03_exception_restore_sanity.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/tests/03_exception_restore_sanity.rs @@ -12,7 +12,7 @@ mod panic_wait_forever; use core::arch::asm; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[inline(never)] fn nested_system_call() { @@ -30,11 +30,10 @@ fn nested_system_call() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; use memory::mmu::interface::MMU; exception::handling_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing exception restore"); diff --git a/13_exceptions_part2_peripheral_IRQs/kernel/tests/04_exception_irq_sanity.rs b/13_exceptions_part2_peripheral_IRQs/kernel/tests/04_exception_irq_sanity.rs index 9deac6cc..8f2b924e 100644 --- a/13_exceptions_part2_peripheral_IRQs/kernel/tests/04_exception_irq_sanity.rs +++ b/13_exceptions_part2_peripheral_IRQs/kernel/tests/04_exception_irq_sanity.rs @@ -10,13 +10,12 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -use libkernel::{bsp, cpu, driver, exception}; +use libkernel::{bsp, cpu, exception}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); exception::handling_init(); exception::asynchronous::local_irq_unmask(); diff --git a/14_virtual_mem_part2_mmio_remap/Makefile b/14_virtual_mem_part2_mmio_remap/Makefile index 670c30c4..2d480119 100644 --- a/14_virtual_mem_part2_mmio_remap/Makefile +++ b/14_virtual_mem_part2_mmio_remap/Makefile @@ -291,7 +291,7 @@ test_boot: $(KERNEL_BIN) ## Helpers for unit and integration test targets ##------------------------------------------------------------------------------ define KERNEL_TEST_RUNNER - #!/usr/bin/env bash +#!/usr/bin/env bash # The cargo test runner seems to change into the crate under test's directory. Therefore, ensure # this script executes from the root. diff --git a/14_virtual_mem_part2_mmio_remap/README.md b/14_virtual_mem_part2_mmio_remap/README.md index c0db9938..61117358 100644 --- a/14_virtual_mem_part2_mmio_remap/README.md +++ b/14_virtual_mem_part2_mmio_remap/README.md @@ -257,81 +257,48 @@ static mut INTERRUPT_CONTROLLER: MaybeUninit static mut INTERRUPT_CONTROLLER: MaybeUninit = MaybeUninit::uninit(); ``` -`BSPDriverManager` implements the new `instantiate_drivers()` interface function, which will be -called early during `kernel_init()`, short after virtual memory has been activated: +Accordingly, new dedicated `instantiate_xyz()` functions have been added, which will be called by +the corresponding `driver_xyz()` functions. Here is an example for the `UART`: ```rust -unsafe fn instantiate_drivers(&self) -> Result<(), &'static str> { - if self.init_done.load(Ordering::Relaxed) { - return Err("Drivers already instantiated"); - } - - self.instantiate_uart()?; - self.instantiate_gpio()?; - self.instantiate_interrupt_controller()?; +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_uart() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; - self.register_drivers(); + PL011_UART.write(device_driver::PL011Uart::new(virt_addr)); - self.init_done.store(true, Ordering::Relaxed); Ok(()) } ``` -As can be seen, for each driver, this `BSP` code calls a dedicated instantiation function. In this -tutorial text, only the `UART` will be discussed in detail: - ```rust -unsafe fn instantiate_uart(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_uart() -> Result<(), &'static str> { + instantiate_uart()?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn uart_post_init() { - console::register_console(unsafe { PL011_UART.assume_init_ref() }); - } - - PL011_UART.write(device_driver::PL011Uart::new( - virt_addr, - exception::asynchronous::irq_map::PL011_UART, - uart_post_init, - )); + let uart_descriptor = generic_driver::DeviceDriverDescriptor::new( + PL011_UART.assume_init_ref(), + Some(post_init_uart), + Some(exception::asynchronous::irq_map::PL011_UART), + ); + generic_driver::driver_manager().register_driver(uart_descriptor); Ok(()) } ``` -A couple of things are happening here. First, an `MMIODescriptor` is created and then used to remap -the MMIO region using `memory::mmu::kernel_map_mmio()`. This function will be discussed in detail in -the next chapter. What's important for now is that it returns the new `Virtual Address` of the -remapped MMIO region. The constructor of the `UART` driver now also expects a virtual address. +The code shows that an `MMIODescriptor` is created first, and then used to remap the MMIO region +using `memory::mmu::kernel_map_mmio()`. This function will be discussed in detail in the next +chapter. What's important for now is that it returns the new `Virtual Address` of the remapped MMIO +region. The constructor of the `UART` driver now also expects a virtual address. Next, a new instance of the `PL011Uart` driver is created, and written into the `PL011_UART` global variable (remember, it is defined as `MaybeUninit = MaybeUninit::uninit()`). Meaning, after this line of code, `PL011_UART` is properly initialized. - -Another new feature is the function `uart_post_init()`, which is supplied to the UART constructor. -This is a callback that will be called by the UART driver when it concludes its `init()` function -(remember that the driver's init functions are called from `kernel_init()`). This callback, in turn, -registers the `PL011_UART` as the new console of the kernel. - -A look into `src/console.rs` should make clear what is happening. The classic `console::console()` -now dynamically points to whoever registered itself using the `console::register_console()` -function, instead of using a hardcoded static reference (from the `BSP`). This has been introduced -to accommodate the run-time instantiation of the device drivers (the same feature has been -implemented for the `IRQManager` as well). Until `console::register_console()` has been called for -the first time, an instance of the newly introduced `NullConsole` is used as the default. -`NullConsole` implements all the console traits, but does nothing. It discards outputs, and returns -dummy input. For example, should one of the printing macros be called before the UART driver has -been instantiated and registered, the kernel does not need to crash because the driver is not -brought up yet. Instead, it can just discard the output. With this new scheme of things, it is -possible to safely switch global references like the UART or the IRQ Manager at runtime. - -That all the post-driver-init work has now been moved to callbacks is motivated by the idea that -this fully enables a driver once it has concluded its `init()` function, and not only after all the -drivers have been init'ed and then the post-init code would be called, as earlier. In our example, -printing through the UART will now be available already before the interrupt controller driver runs -its init function. +Only then, the driver is registered with the kernel and thus becomes accessible for the first time. +This ensures that nobody can use the UART before its memory has been initialized properly. ### MMIO Virtual Address Allocation @@ -924,31 +891,18 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/g diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2.rs 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/arm/gicv2.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/gicv2.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/arm/gicv2.rs -@@ -79,7 +79,12 @@ - mod gicc; - mod gicd; +@@ -81,7 +81,9 @@ --use crate::{bsp, cpu, driver, exception, synchronization, synchronization::InitStateLock}; -+use crate::{ -+ bsp, cpu, driver, exception, + use crate::{ + bsp::{self, device_driver::common::BoundedUsize}, +- cpu, driver, exception, synchronization, ++ cpu, driver, exception, + memory::{Address, Virtual}, + synchronization, -+ synchronization::InitStateLock, -+}; - - //-------------------------------------------------------------------------------------------------- - // Private Definitions -@@ -104,6 +109,9 @@ - - /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. - handler_table: InitStateLock, -+ -+ /// Callback to be invoked after successful init. -+ post_init_callback: fn(), - } + synchronization::InitStateLock, + }; - //-------------------------------------------------------------------------------------------------- -@@ -120,11 +128,16 @@ +@@ -125,7 +127,10 @@ /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -956,49 +910,29 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/arm/g + pub const unsafe fn new( + gicd_mmio_start_addr: Address, + gicc_mmio_start_addr: Address, -+ post_init_callback: fn(), + ) -> Self { Self { gicd: gicd::GICD::new(gicd_mmio_start_addr), gicc: gicc::GICC::new(gicc_mmio_start_addr), - handler_table: InitStateLock::new([None; IRQNumber::NUM_TOTAL]), -+ post_init_callback, - } - } - } -@@ -147,6 +160,8 @@ - self.gicc.priority_accept_all(); - self.gicc.enable(); - -+ (self.post_init_callback)(); -+ - Ok(()) - } - } diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs -@@ -5,7 +5,10 @@ +@@ -5,8 +5,12 @@ //! GPIO Driver. use crate::{ -- bsp::device_driver::common::MMIODerefWrapper, driver, synchronization, +- bsp::device_driver::common::MMIODerefWrapper, driver, exception::asynchronous::IRQNumber, +- synchronization, synchronization::IRQSafeNullLock, + bsp::device_driver::common::MMIODerefWrapper, + driver, ++ exception::asynchronous::IRQNumber, + memory::{Address, Virtual}, + synchronization, - synchronization::IRQSafeNullLock, ++ synchronization::IRQSafeNullLock, }; use tock_registers::{ -@@ -119,6 +122,7 @@ - /// Representation of the GPIO HW. - pub struct GPIO { - inner: IRQSafeNullLock, -+ post_init_callback: fn(), - } - - //-------------------------------------------------------------------------------------------------- + interfaces::{ReadWriteable, Writeable}, @@ -131,7 +135,7 @@ /// # Safety /// @@ -1008,29 +942,15 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/b Self { registers: Registers::new(mmio_start_addr), } -@@ -198,9 +202,10 @@ +@@ -198,7 +202,7 @@ /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new(mmio_start_addr: usize) -> Self { -+ pub const unsafe fn new(mmio_start_addr: Address, post_init_callback: fn()) -> Self { ++ pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(GPIOInner::new(mmio_start_addr)), -+ post_init_callback, } - } - -@@ -219,4 +224,10 @@ - fn compatible(&self) -> &'static str { - Self::COMPATIBLE - } -+ -+ unsafe fn init(&self) -> Result<(), &'static str> { -+ (self.post_init_callback)(); -+ -+ Ok(()) -+ } - } diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -1046,7 +966,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/b synchronization::{IRQSafeNullLock, InitStateLock}, }; use tock_registers::{ -@@ -78,7 +80,7 @@ +@@ -79,7 +81,7 @@ /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -1059,82 +979,36 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/b diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs -@@ -6,7 +6,10 @@ - - mod peripheral_ic; - --use crate::{driver, exception}; -+use crate::{ -+ driver, exception, +@@ -10,6 +10,7 @@ + bsp::device_driver::common::BoundedUsize, + driver, + exception::{self, asynchronous::IRQHandlerDescriptor}, + memory::{Address, Virtual}, -+}; - - //-------------------------------------------------------------------------------------------------- - // Private Definitions -@@ -37,6 +40,7 @@ - /// Representation of the Interrupt Controller. - pub struct InterruptController { - periph: peripheral_ic::PeripheralIC, -+ post_init_callback: fn(), - } + }; + use core::fmt; - //-------------------------------------------------------------------------------------------------- -@@ -79,9 +83,13 @@ +@@ -91,7 +92,7 @@ /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new(periph_mmio_start_addr: usize) -> Self { -+ pub const unsafe fn new( -+ periph_mmio_start_addr: Address, -+ post_init_callback: fn(), -+ ) -> Self { ++ pub const unsafe fn new(periph_mmio_start_addr: Address) -> Self { Self { periph: peripheral_ic::PeripheralIC::new(periph_mmio_start_addr), -+ post_init_callback, } - } - } -@@ -94,6 +102,12 @@ - fn compatible(&self) -> &'static str { - Self::COMPATIBLE - } -+ -+ unsafe fn init(&self) -> Result<(), &'static str> { -+ (self.post_init_callback)(); -+ -+ Ok(()) -+ } - } - - impl exception::asynchronous::interface::IRQManager for InterruptController { diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -10,8 +10,12 @@ - //! - - - use crate::{ -- bsp, bsp::device_driver::common::MMIODerefWrapper, console, cpu, driver, exception, -- synchronization, synchronization::IRQSafeNullLock, -+ bsp, -+ bsp::device_driver::common::MMIODerefWrapper, -+ console, cpu, driver, exception, +@@ -13,6 +13,7 @@ + bsp::device_driver::common::MMIODerefWrapper, + console, cpu, driver, + exception::{self, asynchronous::IRQNumber}, + memory::{Address, Virtual}, -+ synchronization, -+ synchronization::IRQSafeNullLock, + synchronization, + synchronization::IRQSafeNullLock, }; - use core::fmt; - use tock_registers::{ -@@ -230,6 +234,7 @@ - pub struct PL011Uart { - inner: IRQSafeNullLock, - irq_number: bsp::device_driver::IRQNumber, -+ post_init_callback: fn(), - } - - //-------------------------------------------------------------------------------------------------- -@@ -242,7 +247,7 @@ +@@ -244,7 +245,7 @@ /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. @@ -1143,32 +1017,15 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/bcm/b Self { registers: Registers::new(mmio_start_addr), chars_written: 0, -@@ -393,13 +398,16 @@ +@@ -395,7 +396,7 @@ /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. -+ /// - The user must ensure to provide correct IRQ numbers. - pub const unsafe fn new( -- mmio_start_addr: usize, -+ mmio_start_addr: Address, - irq_number: bsp::device_driver::IRQNumber, -+ post_init_callback: fn(), - ) -> Self { +- pub const unsafe fn new(mmio_start_addr: usize) -> Self { ++ pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(PL011UartInner::new(mmio_start_addr)), - irq_number, -+ post_init_callback, } - } - } -@@ -416,6 +424,7 @@ - - unsafe fn init(&self) -> Result<(), &'static str> { - self.inner.lock(|inner| inner.init()); -+ (self.post_init_callback)(); - - Ok(()) - } diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/common.rs 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/common.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/common.rs @@ -1178,7 +1035,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/commo //! Common device driver code. +use crate::memory::{Address, Virtual}; - use core::{marker::PhantomData, ops}; + use core::{fmt, marker::PhantomData, ops}; //-------------------------------------------------------------------------------------------------- @@ -11,7 +12,7 @@ @@ -1190,7 +1047,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/commo phantom: PhantomData T>, } -@@ -21,7 +22,7 @@ +@@ -25,7 +26,7 @@ impl MMIODerefWrapper { /// Create an instance. @@ -1199,7 +1056,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/commo Self { start_addr, phantom: PhantomData, -@@ -33,6 +34,6 @@ +@@ -37,7 +38,7 @@ type Target = T; fn deref(&self) -> &Self::Target { @@ -1208,265 +1065,180 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/device_driver/commo } } -diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/console.rs 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/console.rs ---- 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/console.rs -+++ 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/console.rs -@@ -1,16 +0,0 @@ --// SPDX-License-Identifier: MIT OR Apache-2.0 --// --// Copyright (c) 2018-2022 Andre Richter -- --//! BSP console facilities. -- --use crate::console; -- --//-------------------------------------------------------------------------------------------------- --// Public Code --//-------------------------------------------------------------------------------------------------- -- --/// Return a reference to the console. --pub fn console() -> &'static dyn console::interface::All { -- &super::driver::PL011_UART --} diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/driver.rs 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/driver.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/driver.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/driver.rs -@@ -5,7 +5,16 @@ - //! BSP driver support. - - use super::{exception, memory::map::mmio}; --use crate::{bsp::device_driver, driver}; -+use crate::{ -+ bsp::device_driver, -+ console, driver, exception as generic_exception, memory, +@@ -9,52 +9,109 @@ + bsp::device_driver, + console, driver as generic_driver, + exception::{self as generic_exception}, ++ memory, + memory::mmu::MMIODescriptor, -+ synchronization::{interface::ReadWriteEx, InitStateLock}, +}; +use core::{ + mem::MaybeUninit, + sync::atomic::{AtomicBool, Ordering}, -+}; - - pub use device_driver::IRQNumber; - -@@ -15,35 +24,133 @@ - - /// Device Driver Manager type. - struct BSPDriverManager { -- device_drivers: [&'static (dyn DeviceDriver + Sync); 3], -+ device_drivers: InitStateLock<[Option<&'static (dyn DeviceDriver + Sync)>; NUM_DRIVERS]>, -+ init_done: AtomicBool, - } + }; +-use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- --// Global instances -+// Public Definitions + // Global instances //-------------------------------------------------------------------------------------------------- --pub(super) static PL011_UART: device_driver::PL011Uart = unsafe { -- device_driver::PL011Uart::new( -- mmio::PL011_UART_START, -- exception::asynchronous::irq_map::PL011_UART, -- ) --}; -+/// The number of active drivers provided by this BSP. -+pub const NUM_DRIVERS: usize = 3; -+ -+//-------------------------------------------------------------------------------------------------- -+// Global instances -+//-------------------------------------------------------------------------------------------------- - +-static PL011_UART: device_driver::PL011Uart = +- unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; -static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; +static mut PL011_UART: MaybeUninit = MaybeUninit::uninit(); +static mut GPIO: MaybeUninit = MaybeUninit::uninit(); #[cfg(feature = "bsp_rpi3")] --pub(super) static INTERRUPT_CONTROLLER: device_driver::InterruptController = -- unsafe { device_driver::InterruptController::new(mmio::PERIPHERAL_INTERRUPT_CONTROLLER_START) }; +-static INTERRUPT_CONTROLLER: device_driver::InterruptController = +- unsafe { device_driver::InterruptController::new(mmio::PERIPHERAL_IC_START) }; +static mut INTERRUPT_CONTROLLER: MaybeUninit = + MaybeUninit::uninit(); #[cfg(feature = "bsp_rpi4")] --pub(super) static INTERRUPT_CONTROLLER: device_driver::GICv2 = +-static INTERRUPT_CONTROLLER: device_driver::GICv2 = - unsafe { device_driver::GICv2::new(mmio::GICD_START, mmio::GICC_START) }; +static mut INTERRUPT_CONTROLLER: MaybeUninit = MaybeUninit::uninit(); - static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { -- device_drivers: [&PL011_UART, &GPIO, &INTERRUPT_CONTROLLER], -+ device_drivers: InitStateLock::new([None; NUM_DRIVERS]), -+ init_done: AtomicBool::new(false), - }; - //-------------------------------------------------------------------------------------------------- -+// Private Code -+//-------------------------------------------------------------------------------------------------- -+ -+impl BSPDriverManager { -+ unsafe fn instantiate_uart(&self) -> Result<(), &'static str> { -+ let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); -+ let virt_addr = -+ memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; -+ -+ // This is safe to do, because it is only called from the init'ed instance itself. -+ fn uart_post_init() { -+ console::register_console(unsafe { PL011_UART.assume_init_ref() }); -+ } -+ -+ PL011_UART.write(device_driver::PL011Uart::new( -+ virt_addr, -+ exception::asynchronous::irq_map::PL011_UART, -+ uart_post_init, -+ )); -+ -+ Ok(()) -+ } -+ -+ unsafe fn instantiate_gpio(&self) -> Result<(), &'static str> { -+ let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); -+ let virt_addr = -+ memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; -+ -+ // This is safe to do, because it is only called from the init'ed instance itself. -+ fn gpio_post_init() { -+ unsafe { GPIO.assume_init_ref().map_pl011_uart() }; -+ } -+ -+ GPIO.write(device_driver::GPIO::new(virt_addr, gpio_post_init)); -+ -+ Ok(()) -+ } -+ -+ #[cfg(feature = "bsp_rpi3")] -+ unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { -+ let periph_mmio_descriptor = -+ MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); -+ let periph_virt_addr = memory::mmu::kernel_map_mmio( -+ device_driver::InterruptController::COMPATIBLE, -+ &periph_mmio_descriptor, -+ )?; + // Private Code + //-------------------------------------------------------------------------------------------------- + ++/// This must be called only after successful init of the memory subsystem. ++unsafe fn instantiate_uart() -> Result<(), &'static str> { ++ let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); ++ let virt_addr = ++ memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; + -+ // This is safe to do, because it is only called from the init'ed instance itself. -+ fn interrupt_controller_post_init() { -+ generic_exception::asynchronous::register_irq_manager(unsafe { -+ INTERRUPT_CONTROLLER.assume_init_ref() -+ }); -+ } ++ PL011_UART.write(device_driver::PL011Uart::new(virt_addr)); + -+ INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new( -+ periph_virt_addr, -+ interrupt_controller_post_init, -+ )); ++ Ok(()) ++} + -+ Ok(()) -+ } + /// This must be called only after successful init of the UART driver. +-fn post_init_uart() -> Result<(), &'static str> { +- console::register_console(&PL011_UART); ++unsafe fn post_init_uart() -> Result<(), &'static str> { ++ console::register_console(PL011_UART.assume_init_ref()); + -+ #[cfg(feature = "bsp_rpi4")] -+ unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { -+ let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); -+ let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; ++ Ok(()) ++} + -+ let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); -+ let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; ++/// This must be called only after successful init of the memory subsystem. ++unsafe fn instantiate_gpio() -> Result<(), &'static str> { ++ let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); ++ let virt_addr = ++ memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; + -+ // This is safe to do, because it is only called from the init'ed instance itself. -+ fn interrupt_controller_post_init() { -+ generic_exception::asynchronous::register_irq_manager(unsafe { -+ INTERRUPT_CONTROLLER.assume_init_ref() -+ }); -+ } ++ GPIO.write(device_driver::GPIO::new(virt_addr)); + + Ok(()) + } + + /// This must be called only after successful init of the GPIO driver. +-fn post_init_gpio() -> Result<(), &'static str> { +- GPIO.map_pl011_uart(); ++unsafe fn post_init_gpio() -> Result<(), &'static str> { ++ GPIO.assume_init_ref().map_pl011_uart(); ++ Ok(()) ++} + -+ INTERRUPT_CONTROLLER.write(device_driver::GICv2::new( -+ gicd_virt_addr, -+ gicc_virt_addr, -+ interrupt_controller_post_init, -+ )); ++/// This must be called only after successful init of the memory subsystem. ++#[cfg(feature = "bsp_rpi3")] ++unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { ++ let periph_mmio_descriptor = ++ MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); ++ let periph_virt_addr = memory::mmu::kernel_map_mmio( ++ device_driver::InterruptController::COMPATIBLE, ++ &periph_mmio_descriptor, ++ )?; + -+ Ok(()) -+ } ++ INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new(periph_virt_addr)); + -+ unsafe fn register_drivers(&self) { -+ self.device_drivers.write(|drivers| { -+ drivers[0] = Some(PL011_UART.assume_init_ref()); -+ drivers[1] = Some(GPIO.assume_init_ref()); -+ drivers[2] = Some(INTERRUPT_CONTROLLER.assume_init_ref()); -+ }); -+ } ++ Ok(()) +} + -+//-------------------------------------------------------------------------------------------------- - // Public Code - //-------------------------------------------------------------------------------------------------- - -@@ -58,15 +165,34 @@ - use driver::interface::DeviceDriver; - - impl driver::interface::DriverManager for BSPDriverManager { -- fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { -- &self.device_drivers[..] -+ unsafe fn instantiate_drivers(&self) -> Result<(), &'static str> { -+ if self.init_done.load(Ordering::Relaxed) { -+ return Err("Drivers already instantiated"); -+ } ++/// This must be called only after successful init of the memory subsystem. ++#[cfg(feature = "bsp_rpi4")] ++unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { ++ let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); ++ let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; + -+ self.instantiate_uart()?; -+ self.instantiate_gpio()?; -+ self.instantiate_interrupt_controller()?; ++ let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); ++ let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; + -+ self.register_drivers(); ++ INTERRUPT_CONTROLLER.write(device_driver::GICv2::new(gicd_virt_addr, gicc_virt_addr)); + -+ self.init_done.store(true, Ordering::Relaxed); -+ Ok(()) - } - -- fn post_device_driver_init(&self) { -- // Configure PL011Uart's output pins. -- GPIO.map_pl011_uart(); -+ fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); NUM_DRIVERS] { -+ self.device_drivers -+ .read(|drivers| drivers.map(|drivers| drivers.unwrap())) - } - - #[cfg(feature = "test_build")] -- fn qemu_bring_up_console(&self) {} -+ fn qemu_bring_up_console(&self) { -+ use crate::cpu; -+ -+ unsafe { -+ self.instantiate_uart() -+ .unwrap_or_else(|_| cpu::qemu_exit_failure()); -+ console::register_console(PL011_UART.assume_init_ref()); -+ }; -+ } + Ok(()) } -diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/exception/asynchronous.rs 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/exception/asynchronous.rs ---- 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/exception/asynchronous.rs -+++ 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/exception/asynchronous.rs -@@ -4,7 +4,7 @@ + /// This must be called only after successful init of the interrupt controller driver. +-fn post_init_interrupt_controller() -> Result<(), &'static str> { +- generic_exception::asynchronous::register_irq_manager(&INTERRUPT_CONTROLLER); ++unsafe fn post_init_interrupt_controller() -> Result<(), &'static str> { ++ generic_exception::asynchronous::register_irq_manager(INTERRUPT_CONTROLLER.assume_init_ref()); - //! BSP asynchronous exception handling. + Ok(()) + } --use crate::{bsp, bsp::driver, exception}; -+use crate::bsp; +-fn driver_uart() -> Result<(), &'static str> { ++/// Function needs to ensure that driver registration happens only after correct instantiation. ++unsafe fn driver_uart() -> Result<(), &'static str> { ++ instantiate_uart()?; ++ + let uart_descriptor = generic_driver::DeviceDriverDescriptor::new( +- &PL011_UART, ++ PL011_UART.assume_init_ref(), + Some(post_init_uart), + Some(exception::asynchronous::irq_map::PL011_UART), + ); +@@ -63,17 +120,26 @@ + Ok(()) + } - //-------------------------------------------------------------------------------------------------- - // Public Definitions -@@ -23,14 +23,3 @@ +-fn driver_gpio() -> Result<(), &'static str> { +- let gpio_descriptor = +- generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio), None); ++/// Function needs to ensure that driver registration happens only after correct instantiation. ++unsafe fn driver_gpio() -> Result<(), &'static str> { ++ instantiate_gpio()?; ++ ++ let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new( ++ GPIO.assume_init_ref(), ++ Some(post_init_gpio), ++ None, ++ ); + generic_driver::driver_manager().register_driver(gpio_descriptor); + + Ok(()) + } - pub const PL011_UART: IRQNumber = IRQNumber::new(153); +-fn driver_interrupt_controller() -> Result<(), &'static str> { ++/// Function needs to ensure that driver registration happens only after correct instantiation. ++unsafe fn driver_interrupt_controller() -> Result<(), &'static str> { ++ instantiate_interrupt_controller()?; ++ + let interrupt_controller_descriptor = generic_driver::DeviceDriverDescriptor::new( +- &INTERRUPT_CONTROLLER, ++ INTERRUPT_CONTROLLER.assume_init_ref(), + Some(post_init_interrupt_controller), + None, + ); +@@ -109,5 +175,10 @@ + /// than on real hardware due to QEMU's abstractions. + #[cfg(feature = "test_build")] + pub fn qemu_bring_up_console() { +- console::register_console(&PL011_UART); ++ use crate::cpu; ++ ++ unsafe { ++ instantiate_uart().unwrap_or_else(|_| cpu::qemu_exit_failure()); ++ console::register_console(PL011_UART.assume_init_ref()); ++ }; } -- --//-------------------------------------------------------------------------------------------------- --// Public Code --//-------------------------------------------------------------------------------------------------- -- --/// Return a reference to the IRQ manager. --pub fn irq_manager() -> &'static impl exception::asynchronous::interface::IRQManager< -- IRQNumberType = bsp::device_driver::IRQNumber, --> { -- &driver::INTERRUPT_CONTROLLER --} diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/kernel.ld 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/kernel.ld --- 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/kernel.ld @@ -1910,11 +1682,11 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/memory. pub mod mmio { use super::*; -- pub const START: usize = 0x3F00_0000; -- pub const PERIPHERAL_INTERRUPT_CONTROLLER_START: usize = START + 0x0000_B200; -- pub const GPIO_START: usize = START + GPIO_OFFSET; -- pub const PL011_UART_START: usize = START + UART_OFFSET; -- pub const END_INCLUSIVE: usize = 0x4000_FFFF; +- pub const START: usize = 0x3F00_0000; +- pub const PERIPHERAL_IC_START: usize = START + 0x0000_B200; +- pub const GPIO_START: usize = START + GPIO_OFFSET; +- pub const PL011_UART_START: usize = START + UART_OFFSET; +- pub const END_INCLUSIVE: usize = 0x4000_FFFF; + pub const PERIPHERAL_IC_START: Address = Address::new(0x3F00_B200); + pub const PERIPHERAL_IC_SIZE: usize = 0x24; + @@ -2040,18 +1812,6 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi/memory. + PageAddress::from(map::END) } -diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi.rs 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi.rs ---- 13_exceptions_part2_peripheral_IRQs/kernel/src/bsp/raspberrypi.rs -+++ 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi.rs -@@ -4,7 +4,6 @@ - - //! Top-level BSP file for the Raspberry Pi 3 and 4. - --pub mod console; - pub mod cpu; - pub mod driver; - pub mod exception; - diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/common.rs 14_virtual_mem_part2_mmio_remap/kernel/src/common.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/src/common.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/src/common.rs @@ -2087,242 +1847,6 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/common.rs 14_virtual_me pub const fn size_human_readable_ceil(size: usize) -> (usize, &'static str) { const KIB: usize = 1024; -diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/console/null_console.rs 14_virtual_mem_part2_mmio_remap/kernel/src/console/null_console.rs ---- 13_exceptions_part2_peripheral_IRQs/kernel/src/console/null_console.rs -+++ 14_virtual_mem_part2_mmio_remap/kernel/src/console/null_console.rs -@@ -0,0 +1,41 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2022 Andre Richter -+ -+//! Null console. -+ -+use super::interface; -+use core::fmt; -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Definitions -+//-------------------------------------------------------------------------------------------------- -+ -+pub struct NullConsole; -+ -+//-------------------------------------------------------------------------------------------------- -+// Global instances -+//-------------------------------------------------------------------------------------------------- -+ -+pub static NULL_CONSOLE: NullConsole = NullConsole {}; -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- -+ -+impl interface::Write for NullConsole { -+ fn write_char(&self, _c: char) {} -+ -+ fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { -+ fmt::Result::Ok(()) -+ } -+ -+ fn flush(&self) {} -+} -+ -+impl interface::Read for NullConsole { -+ fn clear_rx(&self) {} -+} -+ -+impl interface::Statistics for NullConsole {} -+impl interface::All for NullConsole {} - -diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/console.rs 14_virtual_mem_part2_mmio_remap/kernel/src/console.rs ---- 13_exceptions_part2_peripheral_IRQs/kernel/src/console.rs -+++ 14_virtual_mem_part2_mmio_remap/kernel/src/console.rs -@@ -4,7 +4,9 @@ - - //! System console. - --use crate::bsp; -+mod null_console; -+ -+use crate::synchronization; - - //-------------------------------------------------------------------------------------------------- - // Public Definitions -@@ -55,12 +57,25 @@ - } - - //-------------------------------------------------------------------------------------------------- -+// Global instances -+//-------------------------------------------------------------------------------------------------- -+ -+static CUR_CONSOLE: InitStateLock<&'static (dyn interface::All + Sync)> = -+ InitStateLock::new(&null_console::NULL_CONSOLE); -+ -+//-------------------------------------------------------------------------------------------------- - // Public Code - //-------------------------------------------------------------------------------------------------- -+use synchronization::{interface::ReadWriteEx, InitStateLock}; -+ -+/// Register a new console. -+pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { -+ CUR_CONSOLE.write(|con| *con = new_console); -+} - --/// Return a reference to the console. -+/// Return a reference to the currently registered console. - /// - /// This is the global console used by all printing macros. - pub fn console() -> &'static dyn interface::All { -- bsp::console::console() -+ CUR_CONSOLE.read(|con| *con) - } - -diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/driver.rs 14_virtual_mem_part2_mmio_remap/kernel/src/driver.rs ---- 13_exceptions_part2_peripheral_IRQs/kernel/src/driver.rs -+++ 14_virtual_mem_part2_mmio_remap/kernel/src/driver.rs -@@ -10,6 +10,8 @@ - - /// Driver interfaces. - pub mod interface { -+ use crate::bsp; -+ - /// Device Driver functions. - pub trait DeviceDriver { - /// Return a compatibility string for identifying the driver. -@@ -37,17 +39,15 @@ - /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { -- /// Return a slice of references to all `BSP`-instantiated drivers. -+ /// Instantiate all drivers. - /// - /// # Safety - /// -- /// - The order of devices is the order in which `DeviceDriver::init()` is called. -- fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; -+ /// Must be called before `all_device_drivers`. -+ unsafe fn instantiate_drivers(&self) -> Result<(), &'static str>; - -- /// Initialization code that runs after driver init. -- /// -- /// For example, device driver code that depends on other drivers already being online. -- fn post_device_driver_init(&self); -+ /// Return a slice of references to all `BSP`-instantiated drivers. -+ fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); bsp::driver::NUM_DRIVERS]; - - /// Minimal code needed to bring up the console in QEMU (for testing only). This is often - /// less steps than on real hardware due to QEMU's abstractions. - -diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous/null_irq_manager.rs 14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous/null_irq_manager.rs ---- 13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous/null_irq_manager.rs -+++ 14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous/null_irq_manager.rs -@@ -0,0 +1,44 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2022 Andre Richter -+ -+//! Null IRQ Manager. -+ -+use super::{interface, IRQContext, IRQDescriptor}; -+use crate::bsp; -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Definitions -+//-------------------------------------------------------------------------------------------------- -+ -+pub struct NullIRQManager; -+ -+//-------------------------------------------------------------------------------------------------- -+// Global instances -+//-------------------------------------------------------------------------------------------------- -+ -+pub static NULL_IRQ_MANAGER: NullIRQManager = NullIRQManager {}; -+ -+//-------------------------------------------------------------------------------------------------- -+// Public Code -+//-------------------------------------------------------------------------------------------------- -+ -+impl interface::IRQManager for NullIRQManager { -+ type IRQNumberType = bsp::driver::IRQNumber; -+ -+ fn register_handler( -+ &self, -+ _irq_number: Self::IRQNumberType, -+ _descriptor: IRQDescriptor, -+ ) -> Result<(), &'static str> { -+ panic!("No IRQ Manager registered yet"); -+ } -+ -+ fn enable(&self, _irq_number: Self::IRQNumberType) { -+ panic!("No IRQ Manager registered yet"); -+ } -+ -+ fn handle_pending_irqs<'irq_context>(&'irq_context self, _ic: &IRQContext<'irq_context>) { -+ panic!("No IRQ Manager registered yet"); -+ } -+} - -diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous.rs 14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous.rs ---- 13_exceptions_part2_peripheral_IRQs/kernel/src/exception/asynchronous.rs -+++ 14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous.rs -@@ -7,8 +7,9 @@ - #[cfg(target_arch = "aarch64")] - #[path = "../_arch/aarch64/exception/asynchronous.rs"] - mod arch_asynchronous; -+mod null_irq_manager; - --use crate::bsp; -+use crate::{bsp, synchronization}; - use core::{fmt, marker::PhantomData}; - - //-------------------------------------------------------------------------------------------------- -@@ -86,7 +87,7 @@ - ); - - /// Print list of registered handlers. -- fn print_handler(&self); -+ fn print_handler(&self) {} - } - } - -@@ -95,8 +96,17 @@ - pub struct IRQNumber(usize); - - //-------------------------------------------------------------------------------------------------- -+// Global instances -+//-------------------------------------------------------------------------------------------------- -+ -+static CUR_IRQ_MANAGER: InitStateLock< -+ &'static (dyn interface::IRQManager + Sync), -+> = InitStateLock::new(&null_irq_manager::NULL_IRQ_MANAGER); -+ -+//-------------------------------------------------------------------------------------------------- - // Public Code - //-------------------------------------------------------------------------------------------------- -+use synchronization::{interface::ReadWriteEx, InitStateLock}; - - impl<'irq_context> IRQContext<'irq_context> { - /// Creates an IRQContext token. -@@ -151,9 +161,17 @@ - ret - } - --/// Return a reference to the IRQ manager. -+/// Register a new IRQ manager. -+pub fn register_irq_manager( -+ new_manager: &'static (dyn interface::IRQManager -+ + Sync), -+) { -+ CUR_IRQ_MANAGER.write(|manager| *manager = new_manager); -+} -+ -+/// Return a reference to the currently registered IRQ manager. - /// - /// This is the IRQ manager used by the architectural interrupt handling code. - pub fn irq_manager() -> &'static dyn interface::IRQManager { -- bsp::exception::asynchronous::irq_manager() -+ CUR_IRQ_MANAGER.read(|manager| *manager) - } - diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/lib.rs 14_virtual_mem_part2_mmio_remap/kernel/src/lib.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/src/lib.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/src/lib.rs @@ -2340,9 +1864,9 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/lib.rs 14_virtual_mem_p #![feature(trait_alias)] #![feature(unchecked_math)] #![no_std] -@@ -186,6 +189,17 @@ - use driver::interface::DriverManager; - +@@ -184,6 +187,17 @@ + #[no_mangle] + unsafe fn kernel_init() -> ! { exception::handling_init(); + + let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { @@ -2355,19 +1879,19 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/lib.rs 14_virtual_mem_p + } + + memory::mmu::post_enable_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); test_main(); diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/main.rs 14_virtual_mem_part2_mmio_remap/kernel/src/main.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/src/main.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/src/main.rs -@@ -27,21 +27,29 @@ +@@ -26,14 +26,19 @@ + /// IRQSafeNullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - use memory::mmu::interface::MMU; - +- exception::handling_init(); - if let Err(string) = memory::mmu::mmu().enable_mmu_and_caching() { @@ -2383,26 +1907,10 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/main.rs 14_virtual_mem_ + memory::mmu::post_enable_init(); + -+ // Instantiate and init all device drivers. -+ if let Err(x) = bsp::driver::driver_manager().instantiate_drivers() { -+ panic!("Error instantiating drivers: {}", x); -+ } - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } - } -- bsp::driver::driver_manager().post_device_driver_init(); -- // println! is usable from here on. - - // Let device drivers register and enable their handlers with the interrupt controller. - for i in bsp::driver::driver_manager().all_device_drivers() { -@@ -63,13 +71,12 @@ - /// The main function running after the early init. - fn kernel_main() -> ! { - use driver::interface::DriverManager; -- use exception::asynchronous::interface::IRQManager; - + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); +@@ -57,8 +62,8 @@ info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); @@ -2413,15 +1921,6 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/main.rs 14_virtual_mem_ let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); -@@ -92,7 +99,7 @@ - } - - info!("Registered IRQ handlers:"); -- bsp::exception::asynchronous::irq_manager().print_handler(); -+ exception::asynchronous::irq_manager().print_handler(); - - info!("Echoing input now"); - cpu::wait_forever(); diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/memory/mmu/mapping_record.rs 14_virtual_mem_part2_mmio_remap/kernel/src/memory/mmu/mapping_record.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/src/memory/mmu/mapping_record.rs @@ -3787,17 +3286,16 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/src/memory.rs 14_virtual_me diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/00_console_sanity.rs 14_virtual_mem_part2_mmio_remap/kernel/tests/00_console_sanity.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/tests/00_console_sanity.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/tests/00_console_sanity.rs -@@ -11,7 +11,7 @@ +@@ -11,13 +11,24 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; --use libkernel::{bsp, console, cpu, driver, exception, print}; -+use libkernel::{bsp, console, cpu, driver, exception, memory, print}; +-use libkernel::{bsp, console, cpu, exception, print}; ++use libkernel::{bsp, console, cpu, exception, memory, print}; #[no_mangle] unsafe fn kernel_init() -> ! { -@@ -19,6 +19,17 @@ - use driver::interface::DriverManager; + use console::console; exception::handling_init(); + @@ -3811,25 +3309,23 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/00_console_sanity.rs + } + + memory::mmu::post_enable_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Handshake diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/01_timer_sanity.rs 14_virtual_mem_part2_mmio_remap/kernel/tests/01_timer_sanity.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/tests/01_timer_sanity.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/tests/01_timer_sanity.rs -@@ -11,7 +11,7 @@ +@@ -11,12 +11,23 @@ #![test_runner(libkernel::test_runner)] use core::time::Duration; --use libkernel::{bsp, cpu, driver, exception, time}; -+use libkernel::{bsp, cpu, driver, exception, memory, time}; +-use libkernel::{bsp, cpu, exception, time}; ++use libkernel::{bsp, cpu, exception, memory, time}; use test_macros::kernel_test; #[no_mangle] -@@ -19,6 +19,17 @@ - use driver::interface::DriverManager; - + unsafe fn kernel_init() -> ! { exception::handling_init(); + + let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { @@ -3842,21 +3338,21 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/01_timer_sanity.rs 14 + } + + memory::mmu::post_enable_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/02_exception_sync_page_fault.rs 14_virtual_mem_part2_mmio_remap/kernel/tests/02_exception_sync_page_fault.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/tests/02_exception_sync_page_fault.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/tests/02_exception_sync_page_fault.rs -@@ -22,19 +22,28 @@ +@@ -21,19 +21,27 @@ + #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - use memory::mmu::interface::MMU; - +- exception::handling_init(); -- bsp::driver::driver_manager().qemu_bring_up_console(); +- bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); @@ -3877,7 +3373,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/02_exception_sync_pag } + memory::mmu::post_enable_init(); -+ bsp::driver::driver_manager().qemu_bring_up_console(); ++ bsp::driver::qemu_bring_up_console(); + info!("Writing beyond mapped area to address 9 GiB..."); let big_addr: u64 = 9 * 1024 * 1024 * 1024; @@ -3886,14 +3382,14 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/02_exception_sync_pag diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/03_exception_restore_sanity.rs 14_virtual_mem_part2_mmio_remap/kernel/tests/03_exception_restore_sanity.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/tests/03_exception_restore_sanity.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/tests/03_exception_restore_sanity.rs -@@ -31,19 +31,28 @@ +@@ -30,19 +30,27 @@ + #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - use memory::mmu::interface::MMU; - +- exception::handling_init(); -- bsp::driver::driver_manager().qemu_bring_up_console(); +- bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing exception restore"); @@ -3914,7 +3410,7 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/03_exception_restore_ } + memory::mmu::post_enable_init(); -+ bsp::driver::driver_manager().qemu_bring_up_console(); ++ bsp::driver::qemu_bring_up_console(); + info!("Making a dummy system call"); @@ -3923,20 +3419,17 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/03_exception_restore_ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/04_exception_irq_sanity.rs 14_virtual_mem_part2_mmio_remap/kernel/tests/04_exception_irq_sanity.rs --- 13_exceptions_part2_peripheral_IRQs/kernel/tests/04_exception_irq_sanity.rs +++ 14_virtual_mem_part2_mmio_remap/kernel/tests/04_exception_irq_sanity.rs -@@ -10,15 +10,27 @@ +@@ -10,14 +10,25 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] --use libkernel::{bsp, cpu, driver, exception}; -+use libkernel::{bsp, cpu, driver, exception, memory}; +-use libkernel::{bsp, cpu, exception}; ++use libkernel::{bsp, cpu, exception, memory}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; -- bsp::driver::driver_manager().qemu_bring_up_console(); - - exception::handling_init(); ++ exception::handling_init(); + + let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { + Err(string) => panic!("Error mapping kernel binary: {}", string), @@ -3948,8 +3441,9 @@ diff -uNr 13_exceptions_part2_peripheral_IRQs/kernel/tests/04_exception_irq_sani + } + + memory::mmu::post_enable_init(); -+ bsp::driver::driver_manager().qemu_bring_up_console(); -+ + bsp::driver::qemu_bring_up_console(); + +- exception::handling_init(); exception::asynchronous::local_irq_unmask(); test_main(); diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/arm/gicv2.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/arm/gicv2.rs index ec65f1b2..3cc35b5e 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/arm/gicv2.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/arm/gicv2.rs @@ -80,7 +80,8 @@ mod gicc; mod gicd; use crate::{ - bsp, cpu, driver, exception, + bsp::{self, device_driver::common::BoundedUsize}, + cpu, driver, exception, memory::{Address, Virtual}, synchronization, synchronization::InitStateLock, @@ -90,14 +91,15 @@ use crate::{ // Private Definitions //-------------------------------------------------------------------------------------------------- -type HandlerTable = [Option; IRQNumber::NUM_TOTAL]; +type HandlerTable = [Option>; + IRQNumber::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. -pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER }>; +pub type IRQNumber = BoundedUsize<{ GICv2::MAX_IRQ_NUMBER }>; /// Representation of the GIC. pub struct GICv2 { @@ -109,9 +111,6 @@ pub struct GICv2 { /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. handler_table: InitStateLock, - - /// Callback to be invoked after successful init. - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -131,13 +130,11 @@ impl GICv2 { pub const unsafe fn new( gicd_mmio_start_addr: Address, gicc_mmio_start_addr: Address, - post_init_callback: fn(), ) -> Self { Self { gicd: gicd::GICD::new(gicd_mmio_start_addr), gicc: gicc::GICC::new(gicc_mmio_start_addr), - handler_table: InitStateLock::new([None; IRQNumber::NUM_TOTAL]), - post_init_callback, + handler_table: InitStateLock::new([None; IRQNumber::MAX_INCLUSIVE + 1]), } } } @@ -148,6 +145,8 @@ impl GICv2 { use synchronization::interface::ReadWriteEx; impl driver::interface::DeviceDriver for GICv2 { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } @@ -160,8 +159,6 @@ impl driver::interface::DeviceDriver for GICv2 { self.gicc.priority_accept_all(); self.gicc.enable(); - (self.post_init_callback)(); - Ok(()) } } @@ -171,23 +168,22 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq_number.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq_number: Self::IRQNumberType) { + fn enable(&self, irq_number: &Self::IRQNumberType) { self.gicd.enable(irq_number); } @@ -210,7 +206,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } }); @@ -227,7 +223,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { self.handler_table.read(|table| { for (i, opt) in table.iter().skip(32).enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i + 32, handler.name); + info!(" {: >3}. {}", i + 32, handler.name()); } } }); diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs index a9c97a8e..8aebcf2b 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -172,7 +172,7 @@ impl GICD { } /// Enable an interrupt. - pub fn enable(&self, irq_num: super::IRQNumber) { + pub fn enable(&self, irq_num: &super::IRQNumber) { let irq_num = irq_num.get(); // Each bit in the u32 enable register corresponds to one IRQ number. Shift right by 5 diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 2281e66a..fb61a651 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -7,6 +7,7 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, + exception::asynchronous::IRQNumber, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -122,7 +123,6 @@ struct GPIOInner { /// Representation of the GPIO HW. pub struct GPIO { inner: IRQSafeNullLock, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -202,10 +202,9 @@ impl GPIO { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new(mmio_start_addr: Address, post_init_callback: fn()) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(GPIOInner::new(mmio_start_addr)), - post_init_callback, } } @@ -221,13 +220,9 @@ impl GPIO { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for GPIO { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { - (self.post_init_callback)(); - - Ok(()) - } } diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs index b4dd530d..c93a9fa1 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -7,9 +7,12 @@ mod peripheral_ic; use crate::{ - driver, exception, + bsp::device_driver::common::BoundedUsize, + driver, + exception::{self, asynchronous::IRQHandlerDescriptor}, memory::{Address, Virtual}, }; +use core::fmt; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -24,10 +27,8 @@ struct PendingIRQs { // Public Definitions //-------------------------------------------------------------------------------------------------- -pub type LocalIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; -pub type PeripheralIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; +pub type LocalIRQ = BoundedUsize<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; +pub type PeripheralIRQ = BoundedUsize<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. #[derive(Copy, Clone)] @@ -40,7 +41,6 @@ pub enum IRQNumber { /// Representation of the Interrupt Controller. pub struct InterruptController { periph: peripheral_ic::PeripheralIC, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -71,6 +71,15 @@ impl Iterator for PendingIRQs { // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for IRQNumber { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + match self { + Self::Local(number) => write!(f, "Local({})", number), + Self::Peripheral(number) => write!(f, "Peripheral({})", number), + } + } +} + impl InterruptController { // Restrict to 3 for now. This makes future code for local_ic.rs more straight forward. const MAX_LOCAL_IRQ_NUMBER: usize = 3; @@ -83,13 +92,9 @@ impl InterruptController { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new( - periph_mmio_start_addr: Address, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(periph_mmio_start_addr: Address) -> Self { Self { periph: peripheral_ic::PeripheralIC::new(periph_mmio_start_addr), - post_init_callback, } } } @@ -99,15 +104,11 @@ impl InterruptController { //------------------------------------------------------------------------------ impl driver::interface::DeviceDriver for InterruptController { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { - (self.post_init_callback)(); - - Ok(()) - } } impl exception::asynchronous::interface::IRQManager for InterruptController { @@ -115,16 +116,23 @@ impl exception::asynchronous::interface::IRQManager for InterruptController { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { - match irq { + match irq_handler_descriptor.number() { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), - IRQNumber::Peripheral(pirq) => self.periph.register_handler(pirq, descriptor), + IRQNumber::Peripheral(pirq) => { + let periph_descriptor = IRQHandlerDescriptor::new( + pirq, + irq_handler_descriptor.name(), + irq_handler_descriptor.handler(), + ); + + self.periph.register_handler(periph_descriptor) + } } } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { match irq { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), IRQNumber::Peripheral(pirq) => self.periph.enable(pirq), diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 06802d27..0a20bd87 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -52,7 +52,8 @@ type WriteOnlyRegisters = MMIODerefWrapper; /// Abstraction for the ReadOnly parts of the associated MMIO registers. type ReadOnlyRegisters = MMIODerefWrapper; -type HandlerTable = [Option; PeripheralIRQ::NUM_TOTAL]; +type HandlerTable = [Option>; + PeripheralIRQ::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -84,7 +85,7 @@ impl PeripheralIC { Self { wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(mmio_start_addr)), ro_registers: ReadOnlyRegisters::new(mmio_start_addr), - handler_table: InitStateLock::new([None; PeripheralIRQ::NUM_TOTAL]), + handler_table: InitStateLock::new([None; PeripheralIRQ::MAX_INCLUSIVE + 1]), } } @@ -107,23 +108,22 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { self.wo_registers.lock(|regs| { let enable_reg = if irq.get() <= 31 { ®s.ENABLE_1 @@ -149,7 +149,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } } @@ -164,7 +164,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { self.handler_table.read(|table| { for (i, opt) in table.iter().enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i, handler.name); + info!(" {: >3}. {}", i, handler.name()); } } }); diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index f67d70df..0ee7feb7 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -10,9 +10,9 @@ //! - use crate::{ - bsp, bsp::device_driver::common::MMIODerefWrapper, - console, cpu, driver, exception, + console, cpu, driver, + exception::{self, asynchronous::IRQNumber}, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -233,8 +233,6 @@ struct PL011UartInner { /// Representation of the UART. pub struct PL011Uart { inner: IRQSafeNullLock, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -398,16 +396,9 @@ impl PL011Uart { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - /// - The user must ensure to provide correct IRQ numbers. - pub const unsafe fn new( - mmio_start_addr: Address, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(PL011UartInner::new(mmio_start_addr)), - irq_number, - post_init_callback, } } } @@ -418,27 +409,28 @@ impl PL011Uart { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for PL011Uart { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } unsafe fn init(&self) -> Result<(), &'static str> { self.inner.lock(|inner| inner.init()); - (self.post_init_callback)(); Ok(()) } - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - use exception::asynchronous::{irq_manager, IRQDescriptor}; + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + use exception::asynchronous::{irq_manager, IRQHandlerDescriptor}; - let descriptor = IRQDescriptor { - name: Self::COMPATIBLE, - handler: self, - }; + let descriptor = IRQHandlerDescriptor::new(*irq_number, Self::COMPATIBLE, self); - irq_manager().register_handler(self.irq_number, descriptor)?; - irq_manager().enable(self.irq_number); + irq_manager().register_handler(descriptor)?; + irq_manager().enable(irq_number); Ok(()) } diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/common.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/common.rs index 69f038d4..ca7aeb76 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/common.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/device_driver/common.rs @@ -5,7 +5,7 @@ //! Common device driver code. use crate::memory::{Address, Virtual}; -use core::{marker::PhantomData, ops}; +use core::{fmt, marker::PhantomData, ops}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -16,6 +16,10 @@ pub struct MMIODerefWrapper { phantom: PhantomData T>, } +/// A wrapper type for usize with integrated range bound check. +#[derive(Copy, Clone)] +pub struct BoundedUsize(usize); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -37,3 +41,25 @@ impl ops::Deref for MMIODerefWrapper { unsafe { &*(self.start_addr.as_usize() as *const _) } } } + +impl BoundedUsize<{ MAX_INCLUSIVE }> { + pub const MAX_INCLUSIVE: usize = MAX_INCLUSIVE; + + /// Creates a new instance if number <= MAX_INCLUSIVE. + pub const fn new(number: usize) -> Self { + assert!(number <= MAX_INCLUSIVE); + + Self(number) + } + + /// Return the wrapped number. + pub const fn get(self) -> usize { + self.0 + } +} + +impl fmt::Display for BoundedUsize<{ MAX_INCLUSIVE }> { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + write!(f, "{}", self.0) + } +} diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/driver.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/driver.rs index e1db4a00..ca3435aa 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/driver.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/driver.rs @@ -7,34 +7,16 @@ use super::{exception, memory::map::mmio}; use crate::{ bsp::device_driver, - console, driver, exception as generic_exception, memory, + console, driver as generic_driver, + exception::{self as generic_exception}, + memory, memory::mmu::MMIODescriptor, - synchronization::{interface::ReadWriteEx, InitStateLock}, }; use core::{ mem::MaybeUninit, sync::atomic::{AtomicBool, Ordering}, }; -pub use device_driver::IRQNumber; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: InitStateLock<[Option<&'static (dyn DeviceDriver + Sync)>; NUM_DRIVERS]>, - init_done: AtomicBool, -} - -//-------------------------------------------------------------------------------------------------- -// Public Definitions -//-------------------------------------------------------------------------------------------------- - -/// The number of active drivers provided by this BSP. -pub const NUM_DRIVERS: usize = 3; - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- @@ -49,150 +31,154 @@ static mut INTERRUPT_CONTROLLER: MaybeUninit #[cfg(feature = "bsp_rpi4")] static mut INTERRUPT_CONTROLLER: MaybeUninit = MaybeUninit::uninit(); -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: InitStateLock::new([None; NUM_DRIVERS]), - init_done: AtomicBool::new(false), -}; - //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -impl BSPDriverManager { - unsafe fn instantiate_uart(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_uart() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn uart_post_init() { - console::register_console(unsafe { PL011_UART.assume_init_ref() }); - } + PL011_UART.write(device_driver::PL011Uart::new(virt_addr)); - PL011_UART.write(device_driver::PL011Uart::new( - virt_addr, - exception::asynchronous::irq_map::PL011_UART, - uart_post_init, - )); + Ok(()) +} - Ok(()) - } +/// This must be called only after successful init of the UART driver. +unsafe fn post_init_uart() -> Result<(), &'static str> { + console::register_console(PL011_UART.assume_init_ref()); - unsafe fn instantiate_gpio(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; + Ok(()) +} - // This is safe to do, because it is only called from the init'ed instance itself. - fn gpio_post_init() { - unsafe { GPIO.assume_init_ref().map_pl011_uart() }; - } +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_gpio() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; - GPIO.write(device_driver::GPIO::new(virt_addr, gpio_post_init)); + GPIO.write(device_driver::GPIO::new(virt_addr)); - Ok(()) - } + Ok(()) +} - #[cfg(feature = "bsp_rpi3")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let periph_mmio_descriptor = - MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); - let periph_virt_addr = memory::mmu::kernel_map_mmio( - device_driver::InterruptController::COMPATIBLE, - &periph_mmio_descriptor, - )?; - - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } - - INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new( - periph_virt_addr, - interrupt_controller_post_init, - )); - - Ok(()) - } +/// This must be called only after successful init of the GPIO driver. +unsafe fn post_init_gpio() -> Result<(), &'static str> { + GPIO.assume_init_ref().map_pl011_uart(); + Ok(()) +} + +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi3")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let periph_mmio_descriptor = + MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); + let periph_virt_addr = memory::mmu::kernel_map_mmio( + device_driver::InterruptController::COMPATIBLE, + &periph_mmio_descriptor, + )?; + + INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new(periph_virt_addr)); - #[cfg(feature = "bsp_rpi4")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); - let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; + Ok(()) +} - let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); - let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi4")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); + let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } + let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); + let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; - INTERRUPT_CONTROLLER.write(device_driver::GICv2::new( - gicd_virt_addr, - gicc_virt_addr, - interrupt_controller_post_init, - )); + INTERRUPT_CONTROLLER.write(device_driver::GICv2::new(gicd_virt_addr, gicc_virt_addr)); - Ok(()) - } + Ok(()) +} - unsafe fn register_drivers(&self) { - self.device_drivers.write(|drivers| { - drivers[0] = Some(PL011_UART.assume_init_ref()); - drivers[1] = Some(GPIO.assume_init_ref()); - drivers[2] = Some(INTERRUPT_CONTROLLER.assume_init_ref()); - }); - } +/// This must be called only after successful init of the interrupt controller driver. +unsafe fn post_init_interrupt_controller() -> Result<(), &'static str> { + generic_exception::asynchronous::register_irq_manager(INTERRUPT_CONTROLLER.assume_init_ref()); + + Ok(()) } -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_uart() -> Result<(), &'static str> { + instantiate_uart()?; + + let uart_descriptor = generic_driver::DeviceDriverDescriptor::new( + PL011_UART.assume_init_ref(), + Some(post_init_uart), + Some(exception::asynchronous::irq_map::PL011_UART), + ); + generic_driver::driver_manager().register_driver(uart_descriptor); + + Ok(()) +} + +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_gpio() -> Result<(), &'static str> { + instantiate_gpio()?; + + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new( + GPIO.assume_init_ref(), + Some(post_init_gpio), + None, + ); + generic_driver::driver_manager().register_driver(gpio_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_interrupt_controller() -> Result<(), &'static str> { + instantiate_interrupt_controller()?; -impl driver::interface::DriverManager for BSPDriverManager { - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str> { - if self.init_done.load(Ordering::Relaxed) { - return Err("Drivers already instantiated"); - } + let interrupt_controller_descriptor = generic_driver::DeviceDriverDescriptor::new( + INTERRUPT_CONTROLLER.assume_init_ref(), + Some(post_init_interrupt_controller), + None, + ); + generic_driver::driver_manager().register_driver(interrupt_controller_descriptor); - self.instantiate_uart()?; - self.instantiate_gpio()?; - self.instantiate_interrupt_controller()?; + Ok(()) +} - self.register_drivers(); +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - self.init_done.store(true, Ordering::Relaxed); - Ok(()) +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } - fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); NUM_DRIVERS] { - self.device_drivers - .read(|drivers| drivers.map(|drivers| drivers.unwrap())) - } + driver_uart()?; + driver_gpio()?; + driver_interrupt_controller()?; - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self) { - use crate::cpu; + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) +} - unsafe { - self.instantiate_uart() - .unwrap_or_else(|_| cpu::qemu_exit_failure()); - console::register_console(PL011_UART.assume_init_ref()); - }; - } +/// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps +/// than on real hardware due to QEMU's abstractions. +#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() { + use crate::cpu; + + unsafe { + instantiate_uart().unwrap_or_else(|_| cpu::qemu_exit_failure()); + console::register_console(PL011_UART.assume_init_ref()); + }; } diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/exception/asynchronous.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/exception/asynchronous.rs index ab20d86d..06a67558 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/exception/asynchronous.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/exception/asynchronous.rs @@ -10,6 +10,9 @@ use crate::bsp; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Export for reuse in generic asynchronous.rs. +pub use bsp::device_driver::IRQNumber; + #[cfg(feature = "bsp_rpi3")] pub(in crate::bsp) mod irq_map { use super::bsp::device_driver::{IRQNumber, PeripheralIRQ}; diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/driver.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/driver.rs index 98197fef..18066c31 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/driver.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/driver.rs @@ -4,16 +4,37 @@ //! Driver support. +use crate::{ + exception, info, + synchronization::{interface::ReadWriteEx, InitStateLock}, +}; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner +where + T: 'static, +{ + next_index: usize, + descriptors: [Option>; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Driver interfaces. pub mod interface { - use crate::bsp; - /// Device Driver functions. pub trait DeviceDriver { + /// Different interrupt controllers might use different types for IRQ number. + type IRQNumberType: super::fmt::Display; + /// Return a compatibility string for identifying the driver. fn compatible(&self) -> &'static str; @@ -26,32 +47,175 @@ pub mod interface { Ok(()) } - /// Called by the kernel to register and enable the device's IRQ handlers, if any. + /// Called by the kernel to register and enable the device's IRQ handler. /// /// Rust's type system will prevent a call to this function unless the calling instance /// itself has static lifetime. - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - Ok(()) + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + panic!( + "Attempt to enable IRQ {} for device {}, but driver does not support this", + irq_number, + self.compatible() + ) + } + } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor +where + T: 'static, +{ + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager +where + T: 'static, +{ + inner: InitStateLock>, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner +where + T: 'static + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], } } +} - /// Device driver management functions. +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + irq_number, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager +where + T: fmt::Display + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: InitStateLock::new(DriverManagerInner::new()), + } + } + + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.write(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.read(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers and their interrupts handlers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Instantiate all drivers. - /// - /// # Safety - /// - /// Must be called before `all_device_drivers`. - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str>; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers_and_irqs(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + + // 3. After all post-init callbacks were done, the interrupt controller should be + // registered and functional. So let drivers register with it now. + self.for_each_descriptor(|descriptor| { + if let Some(irq_number) = &descriptor.irq_number { + if let Err(x) = descriptor + .device_driver + .register_and_enable_irq_handler(irq_number) + { + panic!( + "Error during driver interrupt handler registration: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } - /// Return a slice of references to all `BSP`-instantiated drivers. - fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); bsp::driver::NUM_DRIVERS]; + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); - /// Minimal code needed to bring up the console in QEMU (for testing only). This is often - /// less steps than on real hardware due to QEMU's abstractions. - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self); + i += 1; + }); } } diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous.rs index 83ca1aa6..c1f2a27b 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous.rs @@ -10,7 +10,7 @@ mod arch_asynchronous; mod null_irq_manager; use crate::{bsp, synchronization}; -use core::{fmt, marker::PhantomData}; +use core::marker::PhantomData; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports @@ -24,14 +24,23 @@ pub use arch_asynchronous::{ // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Interrupt number as defined by the BSP. +pub type IRQNumber = bsp::exception::asynchronous::IRQNumber; + /// Interrupt descriptor. #[derive(Copy, Clone)] -pub struct IRQDescriptor { +pub struct IRQHandlerDescriptor +where + T: Copy, +{ + /// The IRQ number. + number: T, + /// Descriptive name. - pub name: &'static str, + name: &'static str, /// Reference to handler trait object. - pub handler: &'static (dyn interface::IRQHandler + Sync), + handler: &'static (dyn interface::IRQHandler + Sync), } /// IRQContext token. @@ -61,17 +70,16 @@ pub mod interface { /// platform's interrupt controller. pub trait IRQManager { /// The IRQ number type depends on the implementation. - type IRQNumberType; + type IRQNumberType: Copy; /// Register a handler. fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: super::IRQDescriptor, + irq_handler_descriptor: super::IRQHandlerDescriptor, ) -> Result<(), &'static str>; /// Enable an interrupt in the controller. - fn enable(&self, irq_number: Self::IRQNumberType); + fn enable(&self, irq_number: &Self::IRQNumberType); /// Handle pending interrupts. /// @@ -91,16 +99,12 @@ pub mod interface { } } -/// A wrapper type for IRQ numbers with integrated range sanity check. -#[derive(Copy, Clone)] -pub struct IRQNumber(usize); - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- static CUR_IRQ_MANAGER: InitStateLock< - &'static (dyn interface::IRQManager + Sync), + &'static (dyn interface::IRQManager + Sync), > = InitStateLock::new(&null_irq_manager::NULL_IRQ_MANAGER); //-------------------------------------------------------------------------------------------------- @@ -108,6 +112,39 @@ static CUR_IRQ_MANAGER: InitStateLock< //-------------------------------------------------------------------------------------------------- use synchronization::{interface::ReadWriteEx, InitStateLock}; +impl IRQHandlerDescriptor +where + T: Copy, +{ + /// Create an instance. + pub const fn new( + number: T, + name: &'static str, + handler: &'static (dyn interface::IRQHandler + Sync), + ) -> Self { + Self { + number, + name, + handler, + } + } + + /// Return the number. + pub const fn number(&self) -> T { + self.number + } + + /// Return the name. + pub const fn name(&self) -> &'static str { + self.name + } + + /// Return the handler. + pub const fn handler(&self) -> &'static (dyn interface::IRQHandler + Sync) { + self.handler + } +} + impl<'irq_context> IRQContext<'irq_context> { /// Creates an IRQContext token. /// @@ -125,29 +162,6 @@ impl<'irq_context> IRQContext<'irq_context> { } } -impl IRQNumber<{ MAX_INCLUSIVE }> { - /// The total number of IRQs this type supports. - pub const NUM_TOTAL: usize = MAX_INCLUSIVE + 1; - - /// Creates a new instance if number <= MAX_INCLUSIVE. - pub const fn new(number: usize) -> Self { - assert!(number <= MAX_INCLUSIVE); - - Self(number) - } - - /// Return the wrapped number. - pub const fn get(self) -> usize { - self.0 - } -} - -impl fmt::Display for IRQNumber<{ MAX_INCLUSIVE }> { - fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { - write!(f, "{}", self.0) - } -} - /// Executes the provided closure while IRQs are masked on the executing core. /// /// While the function temporarily changes the HW state of the executing core, it restores it to the @@ -163,8 +177,7 @@ pub fn exec_with_irq_masked(f: impl FnOnce() -> T) -> T { /// Register a new IRQ manager. pub fn register_irq_manager( - new_manager: &'static (dyn interface::IRQManager - + Sync), + new_manager: &'static (dyn interface::IRQManager + Sync), ) { CUR_IRQ_MANAGER.write(|manager| *manager = new_manager); } @@ -172,6 +185,6 @@ pub fn register_irq_manager( /// Return a reference to the currently registered IRQ manager. /// /// This is the IRQ manager used by the architectural interrupt handling code. -pub fn irq_manager() -> &'static dyn interface::IRQManager { +pub fn irq_manager() -> &'static dyn interface::IRQManager { CUR_IRQ_MANAGER.read(|manager| *manager) } diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous/null_irq_manager.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous/null_irq_manager.rs index 560e3ce4..438f9649 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous/null_irq_manager.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/exception/asynchronous/null_irq_manager.rs @@ -4,8 +4,7 @@ //! Null IRQ Manager. -use super::{interface, IRQContext, IRQDescriptor}; -use crate::bsp; +use super::{interface, IRQContext, IRQHandlerDescriptor}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -24,17 +23,16 @@ pub static NULL_IRQ_MANAGER: NullIRQManager = NullIRQManager {}; //-------------------------------------------------------------------------------------------------- impl interface::IRQManager for NullIRQManager { - type IRQNumberType = bsp::driver::IRQNumber; + type IRQNumberType = super::IRQNumber; fn register_handler( &self, - _irq_number: Self::IRQNumberType, - _descriptor: IRQDescriptor, + _descriptor: IRQHandlerDescriptor, ) -> Result<(), &'static str> { panic!("No IRQ Manager registered yet"); } - fn enable(&self, _irq_number: Self::IRQNumberType) { + fn enable(&self, _irq_number: &Self::IRQNumberType) { panic!("No IRQ Manager registered yet"); } diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/lib.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/lib.rs index 22fb5fea..25f66be3 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/lib.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/lib.rs @@ -186,8 +186,6 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { @@ -200,7 +198,7 @@ unsafe fn kernel_init() -> ! { } memory::mmu::post_enable_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); test_main(); diff --git a/14_virtual_mem_part2_mmio_remap/kernel/src/main.rs b/14_virtual_mem_part2_mmio_remap/kernel/src/main.rs index 776b28fa..045d1200 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/src/main.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/src/main.rs @@ -13,7 +13,7 @@ #![no_main] #![no_std] -use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; +use libkernel::{bsp, cpu, driver, exception, info, memory, state, time}; /// Early init code. /// @@ -26,8 +26,6 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// IRQSafeNullLocks instead of spinlocks), will fail to work (properly) on the RPi SoCs. #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { @@ -41,22 +39,13 @@ unsafe fn kernel_init() -> ! { memory::mmu::post_enable_init(); - // Instantiate and init all device drivers. - if let Err(x) = bsp::driver::driver_manager().instantiate_drivers() { - panic!("Error instantiating drivers: {}", x); - } - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - // Let device drivers register and enable their handlers with the interrupt controller. - for i in bsp::driver::driver_manager().all_device_drivers() { - if let Err(msg) = i.register_and_enable_irq_handler() { - warn!("Error registering IRQ handler: {}", msg); - } - } + // Initialize all device drivers. + driver::driver_manager().init_drivers_and_irqs(); // Unmask interrupts on the boot CPU core. exception::asynchronous::local_irq_unmask(); @@ -70,8 +59,6 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { - use driver::interface::DriverManager; - info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); @@ -90,13 +77,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); info!("Registered IRQ handlers:"); exception::asynchronous::irq_manager().print_handler(); diff --git a/14_virtual_mem_part2_mmio_remap/kernel/tests/00_console_sanity.rs b/14_virtual_mem_part2_mmio_remap/kernel/tests/00_console_sanity.rs index 9c6e9cef..b27822d5 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/tests/00_console_sanity.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/tests/00_console_sanity.rs @@ -11,12 +11,11 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{bsp, console, cpu, driver, exception, memory, print}; +use libkernel::{bsp, console, cpu, exception, memory, print}; #[no_mangle] unsafe fn kernel_init() -> ! { use console::console; - use driver::interface::DriverManager; exception::handling_init(); @@ -30,7 +29,7 @@ unsafe fn kernel_init() -> ! { } memory::mmu::post_enable_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/14_virtual_mem_part2_mmio_remap/kernel/tests/01_timer_sanity.rs b/14_virtual_mem_part2_mmio_remap/kernel/tests/01_timer_sanity.rs index dd34b06c..691b511d 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/tests/01_timer_sanity.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/tests/01_timer_sanity.rs @@ -11,13 +11,11 @@ #![test_runner(libkernel::test_runner)] use core::time::Duration; -use libkernel::{bsp, cpu, driver, exception, memory, time}; +use libkernel::{bsp, cpu, exception, memory, time}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { @@ -30,7 +28,7 @@ unsafe fn kernel_init() -> ! { } memory::mmu::post_enable_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff --git a/14_virtual_mem_part2_mmio_remap/kernel/tests/02_exception_sync_page_fault.rs b/14_virtual_mem_part2_mmio_remap/kernel/tests/02_exception_sync_page_fault.rs index 594ffb40..c3053961 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/tests/02_exception_sync_page_fault.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/tests/02_exception_sync_page_fault.rs @@ -17,12 +17,10 @@ /// or indirectly. mod panic_exit_success; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); // This line will be printed as the test header. @@ -42,7 +40,7 @@ unsafe fn kernel_init() -> ! { } memory::mmu::post_enable_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); info!("Writing beyond mapped area to address 9 GiB..."); let big_addr: u64 = 9 * 1024 * 1024 * 1024; diff --git a/14_virtual_mem_part2_mmio_remap/kernel/tests/03_exception_restore_sanity.rs b/14_virtual_mem_part2_mmio_remap/kernel/tests/03_exception_restore_sanity.rs index 32001b2a..6351e80c 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/tests/03_exception_restore_sanity.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/tests/03_exception_restore_sanity.rs @@ -12,7 +12,7 @@ mod panic_wait_forever; use core::arch::asm; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[inline(never)] fn nested_system_call() { @@ -30,8 +30,6 @@ fn nested_system_call() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); // This line will be printed as the test header. @@ -51,7 +49,7 @@ unsafe fn kernel_init() -> ! { } memory::mmu::post_enable_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); info!("Making a dummy system call"); diff --git a/14_virtual_mem_part2_mmio_remap/kernel/tests/04_exception_irq_sanity.rs b/14_virtual_mem_part2_mmio_remap/kernel/tests/04_exception_irq_sanity.rs index c9e244ce..35bf51b6 100644 --- a/14_virtual_mem_part2_mmio_remap/kernel/tests/04_exception_irq_sanity.rs +++ b/14_virtual_mem_part2_mmio_remap/kernel/tests/04_exception_irq_sanity.rs @@ -10,13 +10,11 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -use libkernel::{bsp, cpu, driver, exception, memory}; +use libkernel::{bsp, cpu, exception, memory}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { @@ -29,7 +27,7 @@ unsafe fn kernel_init() -> ! { } memory::mmu::post_enable_init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); exception::asynchronous::local_irq_unmask(); diff --git a/15_virtual_mem_part3_precomputed_tables/Makefile b/15_virtual_mem_part3_precomputed_tables/Makefile index 2e0a3462..4092ec8f 100644 --- a/15_virtual_mem_part3_precomputed_tables/Makefile +++ b/15_virtual_mem_part3_precomputed_tables/Makefile @@ -310,7 +310,7 @@ test_boot: $(KERNEL_BIN) ## Helpers for unit and integration test targets ##------------------------------------------------------------------------------ define KERNEL_TEST_RUNNER - #!/usr/bin/env bash +#!/usr/bin/env bash # The cargo test runner seems to change into the crate under test's directory. Therefore, ensure # this script executes from the root. diff --git a/15_virtual_mem_part3_precomputed_tables/README.md b/15_virtual_mem_part3_precomputed_tables/README.md index a6ebe332..00586235 100644 --- a/15_virtual_mem_part3_precomputed_tables/README.md +++ b/15_virtual_mem_part3_precomputed_tables/README.md @@ -1336,9 +1336,9 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/src/bsp/raspberrypi/memory/mmu. diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/src/lib.rs 15_virtual_mem_part3_precomputed_tables/kernel/src/lib.rs --- 14_virtual_mem_part2_mmio_remap/kernel/src/lib.rs +++ 15_virtual_mem_part3_precomputed_tables/kernel/src/lib.rs -@@ -189,17 +189,7 @@ - use driver::interface::DriverManager; - +@@ -187,17 +187,7 @@ + #[no_mangle] + unsafe fn kernel_init() -> ! { exception::handling_init(); - - let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { @@ -1352,14 +1352,14 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/src/lib.rs 15_virtual_mem_part3 - - memory::mmu::post_enable_init(); + memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); test_main(); diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/src/main.rs 15_virtual_mem_part3_precomputed_tables/kernel/src/main.rs --- 14_virtual_mem_part2_mmio_remap/kernel/src/main.rs +++ 15_virtual_mem_part3_precomputed_tables/kernel/src/main.rs -@@ -17,29 +17,18 @@ +@@ -17,27 +17,16 @@ /// Early init code. /// @@ -1375,8 +1375,6 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/src/main.rs 15_virtual_mem_part +/// - Printing will not work until the respective driver's MMIO is remapped. #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); - - let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { @@ -1391,11 +1389,11 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/src/main.rs 15_virtual_mem_part - memory::mmu::post_enable_init(); + memory::init(); - // Instantiate and init all device drivers. - if let Err(x) = bsp::driver::driver_manager().instantiate_drivers() { -@@ -58,6 +47,8 @@ - } - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { +@@ -47,6 +36,8 @@ + // Initialize all device drivers. + driver::driver_manager().init_drivers_and_irqs(); + bsp::memory::mmu::kernel_add_mapping_records_for_precomputed(); + @@ -1711,8 +1709,8 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/src/memory.rs 15_virtual_mem_pa diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/00_console_sanity.rs 15_virtual_mem_part3_precomputed_tables/kernel/tests/00_console_sanity.rs --- 14_virtual_mem_part2_mmio_remap/kernel/tests/00_console_sanity.rs +++ 15_virtual_mem_part3_precomputed_tables/kernel/tests/00_console_sanity.rs -@@ -19,17 +19,7 @@ - use driver::interface::DriverManager; +@@ -18,17 +18,7 @@ + use console::console; exception::handling_init(); - @@ -1727,16 +1725,16 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/00_console_sanity.rs 15_v - - memory::mmu::post_enable_init(); + memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Handshake diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/01_timer_sanity.rs 15_virtual_mem_part3_precomputed_tables/kernel/tests/01_timer_sanity.rs --- 14_virtual_mem_part2_mmio_remap/kernel/tests/01_timer_sanity.rs +++ 15_virtual_mem_part3_precomputed_tables/kernel/tests/01_timer_sanity.rs -@@ -19,17 +19,7 @@ - use driver::interface::DriverManager; - +@@ -17,17 +17,7 @@ + #[no_mangle] + unsafe fn kernel_init() -> ! { exception::handling_init(); - - let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { @@ -1750,19 +1748,19 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/01_timer_sanity.rs 15_vir - - memory::mmu::post_enable_init(); + memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/02_exception_sync_page_fault.rs 15_virtual_mem_part3_precomputed_tables/kernel/tests/02_exception_sync_page_fault.rs --- 14_virtual_mem_part2_mmio_remap/kernel/tests/02_exception_sync_page_fault.rs +++ 15_virtual_mem_part3_precomputed_tables/kernel/tests/02_exception_sync_page_fault.rs -@@ -24,26 +24,12 @@ - use driver::interface::DriverManager; - +@@ -22,26 +22,12 @@ + #[no_mangle] + unsafe fn kernel_init() -> ! { exception::handling_init(); + memory::init(); -+ bsp::driver::driver_manager().qemu_bring_up_console(); ++ bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); @@ -1781,7 +1779,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/02_exception_sync_page_fa - } - - memory::mmu::post_enable_init(); -- bsp::driver::driver_manager().qemu_bring_up_console(); +- bsp::driver::qemu_bring_up_console(); - info!("Writing beyond mapped area to address 9 GiB..."); let big_addr: u64 = 9 * 1024 * 1024 * 1024; @@ -1790,12 +1788,12 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/02_exception_sync_page_fa diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/03_exception_restore_sanity.rs 15_virtual_mem_part3_precomputed_tables/kernel/tests/03_exception_restore_sanity.rs --- 14_virtual_mem_part2_mmio_remap/kernel/tests/03_exception_restore_sanity.rs +++ 15_virtual_mem_part3_precomputed_tables/kernel/tests/03_exception_restore_sanity.rs -@@ -33,26 +33,12 @@ - use driver::interface::DriverManager; - +@@ -31,26 +31,12 @@ + #[no_mangle] + unsafe fn kernel_init() -> ! { exception::handling_init(); + memory::init(); -+ bsp::driver::driver_manager().qemu_bring_up_console(); ++ bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing exception restore"); @@ -1814,7 +1812,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/03_exception_restore_sani - } - - memory::mmu::post_enable_init(); -- bsp::driver::driver_manager().qemu_bring_up_console(); +- bsp::driver::qemu_bring_up_console(); - info!("Making a dummy system call"); @@ -1823,10 +1821,10 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/03_exception_restore_sani diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/04_exception_irq_sanity.rs 15_virtual_mem_part3_precomputed_tables/kernel/tests/04_exception_irq_sanity.rs --- 14_virtual_mem_part2_mmio_remap/kernel/tests/04_exception_irq_sanity.rs +++ 15_virtual_mem_part3_precomputed_tables/kernel/tests/04_exception_irq_sanity.rs -@@ -17,20 +17,10 @@ - unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; +@@ -15,20 +15,10 @@ + #[no_mangle] + unsafe fn kernel_init() -> ! { - exception::handling_init(); - - let phys_kernel_tables_base_addr = match memory::mmu::kernel_map_binary() { @@ -1840,7 +1838,7 @@ diff -uNr 14_virtual_mem_part2_mmio_remap/kernel/tests/04_exception_irq_sanity.r - - memory::mmu::post_enable_init(); + memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); + exception::handling_init(); exception::asynchronous::local_irq_unmask(); diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/arm/gicv2.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/arm/gicv2.rs index ec65f1b2..3cc35b5e 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/arm/gicv2.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/arm/gicv2.rs @@ -80,7 +80,8 @@ mod gicc; mod gicd; use crate::{ - bsp, cpu, driver, exception, + bsp::{self, device_driver::common::BoundedUsize}, + cpu, driver, exception, memory::{Address, Virtual}, synchronization, synchronization::InitStateLock, @@ -90,14 +91,15 @@ use crate::{ // Private Definitions //-------------------------------------------------------------------------------------------------- -type HandlerTable = [Option; IRQNumber::NUM_TOTAL]; +type HandlerTable = [Option>; + IRQNumber::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. -pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER }>; +pub type IRQNumber = BoundedUsize<{ GICv2::MAX_IRQ_NUMBER }>; /// Representation of the GIC. pub struct GICv2 { @@ -109,9 +111,6 @@ pub struct GICv2 { /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. handler_table: InitStateLock, - - /// Callback to be invoked after successful init. - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -131,13 +130,11 @@ impl GICv2 { pub const unsafe fn new( gicd_mmio_start_addr: Address, gicc_mmio_start_addr: Address, - post_init_callback: fn(), ) -> Self { Self { gicd: gicd::GICD::new(gicd_mmio_start_addr), gicc: gicc::GICC::new(gicc_mmio_start_addr), - handler_table: InitStateLock::new([None; IRQNumber::NUM_TOTAL]), - post_init_callback, + handler_table: InitStateLock::new([None; IRQNumber::MAX_INCLUSIVE + 1]), } } } @@ -148,6 +145,8 @@ impl GICv2 { use synchronization::interface::ReadWriteEx; impl driver::interface::DeviceDriver for GICv2 { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } @@ -160,8 +159,6 @@ impl driver::interface::DeviceDriver for GICv2 { self.gicc.priority_accept_all(); self.gicc.enable(); - (self.post_init_callback)(); - Ok(()) } } @@ -171,23 +168,22 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq_number.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq_number: Self::IRQNumberType) { + fn enable(&self, irq_number: &Self::IRQNumberType) { self.gicd.enable(irq_number); } @@ -210,7 +206,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } }); @@ -227,7 +223,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { self.handler_table.read(|table| { for (i, opt) in table.iter().skip(32).enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i + 32, handler.name); + info!(" {: >3}. {}", i + 32, handler.name()); } } }); diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs index a9c97a8e..8aebcf2b 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -172,7 +172,7 @@ impl GICD { } /// Enable an interrupt. - pub fn enable(&self, irq_num: super::IRQNumber) { + pub fn enable(&self, irq_num: &super::IRQNumber) { let irq_num = irq_num.get(); // Each bit in the u32 enable register corresponds to one IRQ number. Shift right by 5 diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 2281e66a..fb61a651 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -7,6 +7,7 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, + exception::asynchronous::IRQNumber, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -122,7 +123,6 @@ struct GPIOInner { /// Representation of the GPIO HW. pub struct GPIO { inner: IRQSafeNullLock, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -202,10 +202,9 @@ impl GPIO { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new(mmio_start_addr: Address, post_init_callback: fn()) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(GPIOInner::new(mmio_start_addr)), - post_init_callback, } } @@ -221,13 +220,9 @@ impl GPIO { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for GPIO { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { - (self.post_init_callback)(); - - Ok(()) - } } diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs index b4dd530d..c93a9fa1 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -7,9 +7,12 @@ mod peripheral_ic; use crate::{ - driver, exception, + bsp::device_driver::common::BoundedUsize, + driver, + exception::{self, asynchronous::IRQHandlerDescriptor}, memory::{Address, Virtual}, }; +use core::fmt; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -24,10 +27,8 @@ struct PendingIRQs { // Public Definitions //-------------------------------------------------------------------------------------------------- -pub type LocalIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; -pub type PeripheralIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; +pub type LocalIRQ = BoundedUsize<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; +pub type PeripheralIRQ = BoundedUsize<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. #[derive(Copy, Clone)] @@ -40,7 +41,6 @@ pub enum IRQNumber { /// Representation of the Interrupt Controller. pub struct InterruptController { periph: peripheral_ic::PeripheralIC, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -71,6 +71,15 @@ impl Iterator for PendingIRQs { // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for IRQNumber { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + match self { + Self::Local(number) => write!(f, "Local({})", number), + Self::Peripheral(number) => write!(f, "Peripheral({})", number), + } + } +} + impl InterruptController { // Restrict to 3 for now. This makes future code for local_ic.rs more straight forward. const MAX_LOCAL_IRQ_NUMBER: usize = 3; @@ -83,13 +92,9 @@ impl InterruptController { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new( - periph_mmio_start_addr: Address, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(periph_mmio_start_addr: Address) -> Self { Self { periph: peripheral_ic::PeripheralIC::new(periph_mmio_start_addr), - post_init_callback, } } } @@ -99,15 +104,11 @@ impl InterruptController { //------------------------------------------------------------------------------ impl driver::interface::DeviceDriver for InterruptController { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { - (self.post_init_callback)(); - - Ok(()) - } } impl exception::asynchronous::interface::IRQManager for InterruptController { @@ -115,16 +116,23 @@ impl exception::asynchronous::interface::IRQManager for InterruptController { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { - match irq { + match irq_handler_descriptor.number() { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), - IRQNumber::Peripheral(pirq) => self.periph.register_handler(pirq, descriptor), + IRQNumber::Peripheral(pirq) => { + let periph_descriptor = IRQHandlerDescriptor::new( + pirq, + irq_handler_descriptor.name(), + irq_handler_descriptor.handler(), + ); + + self.periph.register_handler(periph_descriptor) + } } } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { match irq { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), IRQNumber::Peripheral(pirq) => self.periph.enable(pirq), diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 06802d27..0a20bd87 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -52,7 +52,8 @@ type WriteOnlyRegisters = MMIODerefWrapper; /// Abstraction for the ReadOnly parts of the associated MMIO registers. type ReadOnlyRegisters = MMIODerefWrapper; -type HandlerTable = [Option; PeripheralIRQ::NUM_TOTAL]; +type HandlerTable = [Option>; + PeripheralIRQ::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -84,7 +85,7 @@ impl PeripheralIC { Self { wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(mmio_start_addr)), ro_registers: ReadOnlyRegisters::new(mmio_start_addr), - handler_table: InitStateLock::new([None; PeripheralIRQ::NUM_TOTAL]), + handler_table: InitStateLock::new([None; PeripheralIRQ::MAX_INCLUSIVE + 1]), } } @@ -107,23 +108,22 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { self.wo_registers.lock(|regs| { let enable_reg = if irq.get() <= 31 { ®s.ENABLE_1 @@ -149,7 +149,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } } @@ -164,7 +164,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { self.handler_table.read(|table| { for (i, opt) in table.iter().enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i, handler.name); + info!(" {: >3}. {}", i, handler.name()); } } }); diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index f67d70df..0ee7feb7 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -10,9 +10,9 @@ //! - use crate::{ - bsp, bsp::device_driver::common::MMIODerefWrapper, - console, cpu, driver, exception, + console, cpu, driver, + exception::{self, asynchronous::IRQNumber}, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -233,8 +233,6 @@ struct PL011UartInner { /// Representation of the UART. pub struct PL011Uart { inner: IRQSafeNullLock, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -398,16 +396,9 @@ impl PL011Uart { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - /// - The user must ensure to provide correct IRQ numbers. - pub const unsafe fn new( - mmio_start_addr: Address, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(PL011UartInner::new(mmio_start_addr)), - irq_number, - post_init_callback, } } } @@ -418,27 +409,28 @@ impl PL011Uart { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for PL011Uart { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } unsafe fn init(&self) -> Result<(), &'static str> { self.inner.lock(|inner| inner.init()); - (self.post_init_callback)(); Ok(()) } - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - use exception::asynchronous::{irq_manager, IRQDescriptor}; + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + use exception::asynchronous::{irq_manager, IRQHandlerDescriptor}; - let descriptor = IRQDescriptor { - name: Self::COMPATIBLE, - handler: self, - }; + let descriptor = IRQHandlerDescriptor::new(*irq_number, Self::COMPATIBLE, self); - irq_manager().register_handler(self.irq_number, descriptor)?; - irq_manager().enable(self.irq_number); + irq_manager().register_handler(descriptor)?; + irq_manager().enable(irq_number); Ok(()) } diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/common.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/common.rs index 69f038d4..ca7aeb76 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/common.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/device_driver/common.rs @@ -5,7 +5,7 @@ //! Common device driver code. use crate::memory::{Address, Virtual}; -use core::{marker::PhantomData, ops}; +use core::{fmt, marker::PhantomData, ops}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -16,6 +16,10 @@ pub struct MMIODerefWrapper { phantom: PhantomData T>, } +/// A wrapper type for usize with integrated range bound check. +#[derive(Copy, Clone)] +pub struct BoundedUsize(usize); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -37,3 +41,25 @@ impl ops::Deref for MMIODerefWrapper { unsafe { &*(self.start_addr.as_usize() as *const _) } } } + +impl BoundedUsize<{ MAX_INCLUSIVE }> { + pub const MAX_INCLUSIVE: usize = MAX_INCLUSIVE; + + /// Creates a new instance if number <= MAX_INCLUSIVE. + pub const fn new(number: usize) -> Self { + assert!(number <= MAX_INCLUSIVE); + + Self(number) + } + + /// Return the wrapped number. + pub const fn get(self) -> usize { + self.0 + } +} + +impl fmt::Display for BoundedUsize<{ MAX_INCLUSIVE }> { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + write!(f, "{}", self.0) + } +} diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/raspberrypi/driver.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/raspberrypi/driver.rs index e1db4a00..ca3435aa 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/raspberrypi/driver.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/raspberrypi/driver.rs @@ -7,34 +7,16 @@ use super::{exception, memory::map::mmio}; use crate::{ bsp::device_driver, - console, driver, exception as generic_exception, memory, + console, driver as generic_driver, + exception::{self as generic_exception}, + memory, memory::mmu::MMIODescriptor, - synchronization::{interface::ReadWriteEx, InitStateLock}, }; use core::{ mem::MaybeUninit, sync::atomic::{AtomicBool, Ordering}, }; -pub use device_driver::IRQNumber; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: InitStateLock<[Option<&'static (dyn DeviceDriver + Sync)>; NUM_DRIVERS]>, - init_done: AtomicBool, -} - -//-------------------------------------------------------------------------------------------------- -// Public Definitions -//-------------------------------------------------------------------------------------------------- - -/// The number of active drivers provided by this BSP. -pub const NUM_DRIVERS: usize = 3; - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- @@ -49,150 +31,154 @@ static mut INTERRUPT_CONTROLLER: MaybeUninit #[cfg(feature = "bsp_rpi4")] static mut INTERRUPT_CONTROLLER: MaybeUninit = MaybeUninit::uninit(); -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: InitStateLock::new([None; NUM_DRIVERS]), - init_done: AtomicBool::new(false), -}; - //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -impl BSPDriverManager { - unsafe fn instantiate_uart(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_uart() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn uart_post_init() { - console::register_console(unsafe { PL011_UART.assume_init_ref() }); - } + PL011_UART.write(device_driver::PL011Uart::new(virt_addr)); - PL011_UART.write(device_driver::PL011Uart::new( - virt_addr, - exception::asynchronous::irq_map::PL011_UART, - uart_post_init, - )); + Ok(()) +} - Ok(()) - } +/// This must be called only after successful init of the UART driver. +unsafe fn post_init_uart() -> Result<(), &'static str> { + console::register_console(PL011_UART.assume_init_ref()); - unsafe fn instantiate_gpio(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; + Ok(()) +} - // This is safe to do, because it is only called from the init'ed instance itself. - fn gpio_post_init() { - unsafe { GPIO.assume_init_ref().map_pl011_uart() }; - } +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_gpio() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; - GPIO.write(device_driver::GPIO::new(virt_addr, gpio_post_init)); + GPIO.write(device_driver::GPIO::new(virt_addr)); - Ok(()) - } + Ok(()) +} - #[cfg(feature = "bsp_rpi3")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let periph_mmio_descriptor = - MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); - let periph_virt_addr = memory::mmu::kernel_map_mmio( - device_driver::InterruptController::COMPATIBLE, - &periph_mmio_descriptor, - )?; - - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } - - INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new( - periph_virt_addr, - interrupt_controller_post_init, - )); - - Ok(()) - } +/// This must be called only after successful init of the GPIO driver. +unsafe fn post_init_gpio() -> Result<(), &'static str> { + GPIO.assume_init_ref().map_pl011_uart(); + Ok(()) +} + +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi3")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let periph_mmio_descriptor = + MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); + let periph_virt_addr = memory::mmu::kernel_map_mmio( + device_driver::InterruptController::COMPATIBLE, + &periph_mmio_descriptor, + )?; + + INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new(periph_virt_addr)); - #[cfg(feature = "bsp_rpi4")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); - let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; + Ok(()) +} - let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); - let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi4")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); + let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } + let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); + let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; - INTERRUPT_CONTROLLER.write(device_driver::GICv2::new( - gicd_virt_addr, - gicc_virt_addr, - interrupt_controller_post_init, - )); + INTERRUPT_CONTROLLER.write(device_driver::GICv2::new(gicd_virt_addr, gicc_virt_addr)); - Ok(()) - } + Ok(()) +} - unsafe fn register_drivers(&self) { - self.device_drivers.write(|drivers| { - drivers[0] = Some(PL011_UART.assume_init_ref()); - drivers[1] = Some(GPIO.assume_init_ref()); - drivers[2] = Some(INTERRUPT_CONTROLLER.assume_init_ref()); - }); - } +/// This must be called only after successful init of the interrupt controller driver. +unsafe fn post_init_interrupt_controller() -> Result<(), &'static str> { + generic_exception::asynchronous::register_irq_manager(INTERRUPT_CONTROLLER.assume_init_ref()); + + Ok(()) } -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_uart() -> Result<(), &'static str> { + instantiate_uart()?; + + let uart_descriptor = generic_driver::DeviceDriverDescriptor::new( + PL011_UART.assume_init_ref(), + Some(post_init_uart), + Some(exception::asynchronous::irq_map::PL011_UART), + ); + generic_driver::driver_manager().register_driver(uart_descriptor); + + Ok(()) +} + +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_gpio() -> Result<(), &'static str> { + instantiate_gpio()?; + + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new( + GPIO.assume_init_ref(), + Some(post_init_gpio), + None, + ); + generic_driver::driver_manager().register_driver(gpio_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_interrupt_controller() -> Result<(), &'static str> { + instantiate_interrupt_controller()?; -impl driver::interface::DriverManager for BSPDriverManager { - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str> { - if self.init_done.load(Ordering::Relaxed) { - return Err("Drivers already instantiated"); - } + let interrupt_controller_descriptor = generic_driver::DeviceDriverDescriptor::new( + INTERRUPT_CONTROLLER.assume_init_ref(), + Some(post_init_interrupt_controller), + None, + ); + generic_driver::driver_manager().register_driver(interrupt_controller_descriptor); - self.instantiate_uart()?; - self.instantiate_gpio()?; - self.instantiate_interrupt_controller()?; + Ok(()) +} - self.register_drivers(); +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - self.init_done.store(true, Ordering::Relaxed); - Ok(()) +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } - fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); NUM_DRIVERS] { - self.device_drivers - .read(|drivers| drivers.map(|drivers| drivers.unwrap())) - } + driver_uart()?; + driver_gpio()?; + driver_interrupt_controller()?; - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self) { - use crate::cpu; + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) +} - unsafe { - self.instantiate_uart() - .unwrap_or_else(|_| cpu::qemu_exit_failure()); - console::register_console(PL011_UART.assume_init_ref()); - }; - } +/// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps +/// than on real hardware due to QEMU's abstractions. +#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() { + use crate::cpu; + + unsafe { + instantiate_uart().unwrap_or_else(|_| cpu::qemu_exit_failure()); + console::register_console(PL011_UART.assume_init_ref()); + }; } diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/raspberrypi/exception/asynchronous.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/raspberrypi/exception/asynchronous.rs index ab20d86d..06a67558 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/raspberrypi/exception/asynchronous.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/bsp/raspberrypi/exception/asynchronous.rs @@ -10,6 +10,9 @@ use crate::bsp; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Export for reuse in generic asynchronous.rs. +pub use bsp::device_driver::IRQNumber; + #[cfg(feature = "bsp_rpi3")] pub(in crate::bsp) mod irq_map { use super::bsp::device_driver::{IRQNumber, PeripheralIRQ}; diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/driver.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/driver.rs index 98197fef..18066c31 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/driver.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/driver.rs @@ -4,16 +4,37 @@ //! Driver support. +use crate::{ + exception, info, + synchronization::{interface::ReadWriteEx, InitStateLock}, +}; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner +where + T: 'static, +{ + next_index: usize, + descriptors: [Option>; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Driver interfaces. pub mod interface { - use crate::bsp; - /// Device Driver functions. pub trait DeviceDriver { + /// Different interrupt controllers might use different types for IRQ number. + type IRQNumberType: super::fmt::Display; + /// Return a compatibility string for identifying the driver. fn compatible(&self) -> &'static str; @@ -26,32 +47,175 @@ pub mod interface { Ok(()) } - /// Called by the kernel to register and enable the device's IRQ handlers, if any. + /// Called by the kernel to register and enable the device's IRQ handler. /// /// Rust's type system will prevent a call to this function unless the calling instance /// itself has static lifetime. - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - Ok(()) + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + panic!( + "Attempt to enable IRQ {} for device {}, but driver does not support this", + irq_number, + self.compatible() + ) + } + } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor +where + T: 'static, +{ + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager +where + T: 'static, +{ + inner: InitStateLock>, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner +where + T: 'static + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], } } +} - /// Device driver management functions. +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + irq_number, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager +where + T: fmt::Display + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: InitStateLock::new(DriverManagerInner::new()), + } + } + + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.write(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.read(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers and their interrupts handlers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Instantiate all drivers. - /// - /// # Safety - /// - /// Must be called before `all_device_drivers`. - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str>; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers_and_irqs(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + + // 3. After all post-init callbacks were done, the interrupt controller should be + // registered and functional. So let drivers register with it now. + self.for_each_descriptor(|descriptor| { + if let Some(irq_number) = &descriptor.irq_number { + if let Err(x) = descriptor + .device_driver + .register_and_enable_irq_handler(irq_number) + { + panic!( + "Error during driver interrupt handler registration: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } - /// Return a slice of references to all `BSP`-instantiated drivers. - fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); bsp::driver::NUM_DRIVERS]; + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); - /// Minimal code needed to bring up the console in QEMU (for testing only). This is often - /// less steps than on real hardware due to QEMU's abstractions. - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self); + i += 1; + }); } } diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/exception/asynchronous.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/exception/asynchronous.rs index 83ca1aa6..c1f2a27b 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/exception/asynchronous.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/exception/asynchronous.rs @@ -10,7 +10,7 @@ mod arch_asynchronous; mod null_irq_manager; use crate::{bsp, synchronization}; -use core::{fmt, marker::PhantomData}; +use core::marker::PhantomData; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports @@ -24,14 +24,23 @@ pub use arch_asynchronous::{ // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Interrupt number as defined by the BSP. +pub type IRQNumber = bsp::exception::asynchronous::IRQNumber; + /// Interrupt descriptor. #[derive(Copy, Clone)] -pub struct IRQDescriptor { +pub struct IRQHandlerDescriptor +where + T: Copy, +{ + /// The IRQ number. + number: T, + /// Descriptive name. - pub name: &'static str, + name: &'static str, /// Reference to handler trait object. - pub handler: &'static (dyn interface::IRQHandler + Sync), + handler: &'static (dyn interface::IRQHandler + Sync), } /// IRQContext token. @@ -61,17 +70,16 @@ pub mod interface { /// platform's interrupt controller. pub trait IRQManager { /// The IRQ number type depends on the implementation. - type IRQNumberType; + type IRQNumberType: Copy; /// Register a handler. fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: super::IRQDescriptor, + irq_handler_descriptor: super::IRQHandlerDescriptor, ) -> Result<(), &'static str>; /// Enable an interrupt in the controller. - fn enable(&self, irq_number: Self::IRQNumberType); + fn enable(&self, irq_number: &Self::IRQNumberType); /// Handle pending interrupts. /// @@ -91,16 +99,12 @@ pub mod interface { } } -/// A wrapper type for IRQ numbers with integrated range sanity check. -#[derive(Copy, Clone)] -pub struct IRQNumber(usize); - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- static CUR_IRQ_MANAGER: InitStateLock< - &'static (dyn interface::IRQManager + Sync), + &'static (dyn interface::IRQManager + Sync), > = InitStateLock::new(&null_irq_manager::NULL_IRQ_MANAGER); //-------------------------------------------------------------------------------------------------- @@ -108,6 +112,39 @@ static CUR_IRQ_MANAGER: InitStateLock< //-------------------------------------------------------------------------------------------------- use synchronization::{interface::ReadWriteEx, InitStateLock}; +impl IRQHandlerDescriptor +where + T: Copy, +{ + /// Create an instance. + pub const fn new( + number: T, + name: &'static str, + handler: &'static (dyn interface::IRQHandler + Sync), + ) -> Self { + Self { + number, + name, + handler, + } + } + + /// Return the number. + pub const fn number(&self) -> T { + self.number + } + + /// Return the name. + pub const fn name(&self) -> &'static str { + self.name + } + + /// Return the handler. + pub const fn handler(&self) -> &'static (dyn interface::IRQHandler + Sync) { + self.handler + } +} + impl<'irq_context> IRQContext<'irq_context> { /// Creates an IRQContext token. /// @@ -125,29 +162,6 @@ impl<'irq_context> IRQContext<'irq_context> { } } -impl IRQNumber<{ MAX_INCLUSIVE }> { - /// The total number of IRQs this type supports. - pub const NUM_TOTAL: usize = MAX_INCLUSIVE + 1; - - /// Creates a new instance if number <= MAX_INCLUSIVE. - pub const fn new(number: usize) -> Self { - assert!(number <= MAX_INCLUSIVE); - - Self(number) - } - - /// Return the wrapped number. - pub const fn get(self) -> usize { - self.0 - } -} - -impl fmt::Display for IRQNumber<{ MAX_INCLUSIVE }> { - fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { - write!(f, "{}", self.0) - } -} - /// Executes the provided closure while IRQs are masked on the executing core. /// /// While the function temporarily changes the HW state of the executing core, it restores it to the @@ -163,8 +177,7 @@ pub fn exec_with_irq_masked(f: impl FnOnce() -> T) -> T { /// Register a new IRQ manager. pub fn register_irq_manager( - new_manager: &'static (dyn interface::IRQManager - + Sync), + new_manager: &'static (dyn interface::IRQManager + Sync), ) { CUR_IRQ_MANAGER.write(|manager| *manager = new_manager); } @@ -172,6 +185,6 @@ pub fn register_irq_manager( /// Return a reference to the currently registered IRQ manager. /// /// This is the IRQ manager used by the architectural interrupt handling code. -pub fn irq_manager() -> &'static dyn interface::IRQManager { +pub fn irq_manager() -> &'static dyn interface::IRQManager { CUR_IRQ_MANAGER.read(|manager| *manager) } diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/exception/asynchronous/null_irq_manager.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/exception/asynchronous/null_irq_manager.rs index 560e3ce4..438f9649 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/exception/asynchronous/null_irq_manager.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/exception/asynchronous/null_irq_manager.rs @@ -4,8 +4,7 @@ //! Null IRQ Manager. -use super::{interface, IRQContext, IRQDescriptor}; -use crate::bsp; +use super::{interface, IRQContext, IRQHandlerDescriptor}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -24,17 +23,16 @@ pub static NULL_IRQ_MANAGER: NullIRQManager = NullIRQManager {}; //-------------------------------------------------------------------------------------------------- impl interface::IRQManager for NullIRQManager { - type IRQNumberType = bsp::driver::IRQNumber; + type IRQNumberType = super::IRQNumber; fn register_handler( &self, - _irq_number: Self::IRQNumberType, - _descriptor: IRQDescriptor, + _descriptor: IRQHandlerDescriptor, ) -> Result<(), &'static str> { panic!("No IRQ Manager registered yet"); } - fn enable(&self, _irq_number: Self::IRQNumberType) { + fn enable(&self, _irq_number: &Self::IRQNumberType) { panic!("No IRQ Manager registered yet"); } diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/lib.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/lib.rs index 26c147d6..197b1f41 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/lib.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/lib.rs @@ -186,11 +186,9 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); test_main(); diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/src/main.rs b/15_virtual_mem_part3_precomputed_tables/kernel/src/main.rs index 9aeb4438..65905258 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/src/main.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/src/main.rs @@ -13,7 +13,7 @@ #![no_main] #![no_std] -use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; +use libkernel::{bsp, cpu, driver, exception, info, memory, state, time}; /// Early init code. /// @@ -25,27 +25,16 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// - Printing will not work until the respective driver's MMIO is remapped. #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - // Instantiate and init all device drivers. - if let Err(x) = bsp::driver::driver_manager().instantiate_drivers() { - panic!("Error instantiating drivers: {}", x); - } - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - // Let device drivers register and enable their handlers with the interrupt controller. - for i in bsp::driver::driver_manager().all_device_drivers() { - if let Err(msg) = i.register_and_enable_irq_handler() { - warn!("Error registering IRQ handler: {}", msg); - } - } + // Initialize all device drivers. + driver::driver_manager().init_drivers_and_irqs(); bsp::memory::mmu::kernel_add_mapping_records_for_precomputed(); @@ -61,8 +50,6 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { - use driver::interface::DriverManager; - info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); @@ -81,13 +68,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); info!("Registered IRQ handlers:"); exception::asynchronous::irq_manager().print_handler(); diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/tests/00_console_sanity.rs b/15_virtual_mem_part3_precomputed_tables/kernel/tests/00_console_sanity.rs index 305510ce..2c0225b7 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/tests/00_console_sanity.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/tests/00_console_sanity.rs @@ -11,16 +11,15 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{bsp, console, cpu, driver, exception, memory, print}; +use libkernel::{bsp, console, cpu, exception, memory, print}; #[no_mangle] unsafe fn kernel_init() -> ! { use console::console; - use driver::interface::DriverManager; exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/tests/01_timer_sanity.rs b/15_virtual_mem_part3_precomputed_tables/kernel/tests/01_timer_sanity.rs index 39899332..8188b942 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/tests/01_timer_sanity.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/tests/01_timer_sanity.rs @@ -11,16 +11,14 @@ #![test_runner(libkernel::test_runner)] use core::time::Duration; -use libkernel::{bsp, cpu, driver, exception, memory, time}; +use libkernel::{bsp, cpu, exception, memory, time}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/tests/02_exception_sync_page_fault.rs b/15_virtual_mem_part3_precomputed_tables/kernel/tests/02_exception_sync_page_fault.rs index ec7a365a..c4b801ce 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/tests/02_exception_sync_page_fault.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/tests/02_exception_sync_page_fault.rs @@ -17,15 +17,13 @@ /// or indirectly. mod panic_exit_success; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/tests/03_exception_restore_sanity.rs b/15_virtual_mem_part3_precomputed_tables/kernel/tests/03_exception_restore_sanity.rs index 47dd2714..f176c6a6 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/tests/03_exception_restore_sanity.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/tests/03_exception_restore_sanity.rs @@ -12,7 +12,7 @@ mod panic_wait_forever; use core::arch::asm; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[inline(never)] fn nested_system_call() { @@ -30,11 +30,9 @@ fn nested_system_call() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing exception restore"); diff --git a/15_virtual_mem_part3_precomputed_tables/kernel/tests/04_exception_irq_sanity.rs b/15_virtual_mem_part3_precomputed_tables/kernel/tests/04_exception_irq_sanity.rs index e93a4caf..e6f94c91 100644 --- a/15_virtual_mem_part3_precomputed_tables/kernel/tests/04_exception_irq_sanity.rs +++ b/15_virtual_mem_part3_precomputed_tables/kernel/tests/04_exception_irq_sanity.rs @@ -10,15 +10,13 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -use libkernel::{bsp, cpu, driver, exception, memory}; +use libkernel::{bsp, cpu, exception, memory}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); exception::handling_init(); exception::asynchronous::local_irq_unmask(); diff --git a/16_virtual_mem_part4_higher_half_kernel/Makefile b/16_virtual_mem_part4_higher_half_kernel/Makefile index 2e0a3462..4092ec8f 100644 --- a/16_virtual_mem_part4_higher_half_kernel/Makefile +++ b/16_virtual_mem_part4_higher_half_kernel/Makefile @@ -310,7 +310,7 @@ test_boot: $(KERNEL_BIN) ## Helpers for unit and integration test targets ##------------------------------------------------------------------------------ define KERNEL_TEST_RUNNER - #!/usr/bin/env bash +#!/usr/bin/env bash # The cargo test runner seems to change into the crate under test's directory. Therefore, ensure # this script executes from the root. diff --git a/16_virtual_mem_part4_higher_half_kernel/README.md b/16_virtual_mem_part4_higher_half_kernel/README.md index 494506da..8082ab45 100644 --- a/16_virtual_mem_part4_higher_half_kernel/README.md +++ b/16_virtual_mem_part4_higher_half_kernel/README.md @@ -817,7 +817,7 @@ diff -uNr 15_virtual_mem_part3_precomputed_tables/kernel/src/memory/mmu.rs 16_vi diff -uNr 15_virtual_mem_part3_precomputed_tables/kernel/tests/02_exception_sync_page_fault.rs 16_virtual_mem_part4_higher_half_kernel/kernel/tests/02_exception_sync_page_fault.rs --- 15_virtual_mem_part3_precomputed_tables/kernel/tests/02_exception_sync_page_fault.rs +++ 16_virtual_mem_part4_higher_half_kernel/kernel/tests/02_exception_sync_page_fault.rs -@@ -30,8 +30,8 @@ +@@ -28,8 +28,8 @@ // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/arm/gicv2.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/arm/gicv2.rs index ec65f1b2..3cc35b5e 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/arm/gicv2.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/arm/gicv2.rs @@ -80,7 +80,8 @@ mod gicc; mod gicd; use crate::{ - bsp, cpu, driver, exception, + bsp::{self, device_driver::common::BoundedUsize}, + cpu, driver, exception, memory::{Address, Virtual}, synchronization, synchronization::InitStateLock, @@ -90,14 +91,15 @@ use crate::{ // Private Definitions //-------------------------------------------------------------------------------------------------- -type HandlerTable = [Option; IRQNumber::NUM_TOTAL]; +type HandlerTable = [Option>; + IRQNumber::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. -pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER }>; +pub type IRQNumber = BoundedUsize<{ GICv2::MAX_IRQ_NUMBER }>; /// Representation of the GIC. pub struct GICv2 { @@ -109,9 +111,6 @@ pub struct GICv2 { /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. handler_table: InitStateLock, - - /// Callback to be invoked after successful init. - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -131,13 +130,11 @@ impl GICv2 { pub const unsafe fn new( gicd_mmio_start_addr: Address, gicc_mmio_start_addr: Address, - post_init_callback: fn(), ) -> Self { Self { gicd: gicd::GICD::new(gicd_mmio_start_addr), gicc: gicc::GICC::new(gicc_mmio_start_addr), - handler_table: InitStateLock::new([None; IRQNumber::NUM_TOTAL]), - post_init_callback, + handler_table: InitStateLock::new([None; IRQNumber::MAX_INCLUSIVE + 1]), } } } @@ -148,6 +145,8 @@ impl GICv2 { use synchronization::interface::ReadWriteEx; impl driver::interface::DeviceDriver for GICv2 { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } @@ -160,8 +159,6 @@ impl driver::interface::DeviceDriver for GICv2 { self.gicc.priority_accept_all(); self.gicc.enable(); - (self.post_init_callback)(); - Ok(()) } } @@ -171,23 +168,22 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq_number.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq_number: Self::IRQNumberType) { + fn enable(&self, irq_number: &Self::IRQNumberType) { self.gicd.enable(irq_number); } @@ -210,7 +206,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } }); @@ -227,7 +223,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { self.handler_table.read(|table| { for (i, opt) in table.iter().skip(32).enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i + 32, handler.name); + info!(" {: >3}. {}", i + 32, handler.name()); } } }); diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs index a9c97a8e..8aebcf2b 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -172,7 +172,7 @@ impl GICD { } /// Enable an interrupt. - pub fn enable(&self, irq_num: super::IRQNumber) { + pub fn enable(&self, irq_num: &super::IRQNumber) { let irq_num = irq_num.get(); // Each bit in the u32 enable register corresponds to one IRQ number. Shift right by 5 diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 2281e66a..fb61a651 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -7,6 +7,7 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, + exception::asynchronous::IRQNumber, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -122,7 +123,6 @@ struct GPIOInner { /// Representation of the GPIO HW. pub struct GPIO { inner: IRQSafeNullLock, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -202,10 +202,9 @@ impl GPIO { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new(mmio_start_addr: Address, post_init_callback: fn()) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(GPIOInner::new(mmio_start_addr)), - post_init_callback, } } @@ -221,13 +220,9 @@ impl GPIO { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for GPIO { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { - (self.post_init_callback)(); - - Ok(()) - } } diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs index b4dd530d..c93a9fa1 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -7,9 +7,12 @@ mod peripheral_ic; use crate::{ - driver, exception, + bsp::device_driver::common::BoundedUsize, + driver, + exception::{self, asynchronous::IRQHandlerDescriptor}, memory::{Address, Virtual}, }; +use core::fmt; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -24,10 +27,8 @@ struct PendingIRQs { // Public Definitions //-------------------------------------------------------------------------------------------------- -pub type LocalIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; -pub type PeripheralIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; +pub type LocalIRQ = BoundedUsize<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; +pub type PeripheralIRQ = BoundedUsize<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. #[derive(Copy, Clone)] @@ -40,7 +41,6 @@ pub enum IRQNumber { /// Representation of the Interrupt Controller. pub struct InterruptController { periph: peripheral_ic::PeripheralIC, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -71,6 +71,15 @@ impl Iterator for PendingIRQs { // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for IRQNumber { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + match self { + Self::Local(number) => write!(f, "Local({})", number), + Self::Peripheral(number) => write!(f, "Peripheral({})", number), + } + } +} + impl InterruptController { // Restrict to 3 for now. This makes future code for local_ic.rs more straight forward. const MAX_LOCAL_IRQ_NUMBER: usize = 3; @@ -83,13 +92,9 @@ impl InterruptController { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new( - periph_mmio_start_addr: Address, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(periph_mmio_start_addr: Address) -> Self { Self { periph: peripheral_ic::PeripheralIC::new(periph_mmio_start_addr), - post_init_callback, } } } @@ -99,15 +104,11 @@ impl InterruptController { //------------------------------------------------------------------------------ impl driver::interface::DeviceDriver for InterruptController { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { - (self.post_init_callback)(); - - Ok(()) - } } impl exception::asynchronous::interface::IRQManager for InterruptController { @@ -115,16 +116,23 @@ impl exception::asynchronous::interface::IRQManager for InterruptController { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { - match irq { + match irq_handler_descriptor.number() { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), - IRQNumber::Peripheral(pirq) => self.periph.register_handler(pirq, descriptor), + IRQNumber::Peripheral(pirq) => { + let periph_descriptor = IRQHandlerDescriptor::new( + pirq, + irq_handler_descriptor.name(), + irq_handler_descriptor.handler(), + ); + + self.periph.register_handler(periph_descriptor) + } } } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { match irq { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), IRQNumber::Peripheral(pirq) => self.periph.enable(pirq), diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 06802d27..0a20bd87 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -52,7 +52,8 @@ type WriteOnlyRegisters = MMIODerefWrapper; /// Abstraction for the ReadOnly parts of the associated MMIO registers. type ReadOnlyRegisters = MMIODerefWrapper; -type HandlerTable = [Option; PeripheralIRQ::NUM_TOTAL]; +type HandlerTable = [Option>; + PeripheralIRQ::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -84,7 +85,7 @@ impl PeripheralIC { Self { wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(mmio_start_addr)), ro_registers: ReadOnlyRegisters::new(mmio_start_addr), - handler_table: InitStateLock::new([None; PeripheralIRQ::NUM_TOTAL]), + handler_table: InitStateLock::new([None; PeripheralIRQ::MAX_INCLUSIVE + 1]), } } @@ -107,23 +108,22 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { self.wo_registers.lock(|regs| { let enable_reg = if irq.get() <= 31 { ®s.ENABLE_1 @@ -149,7 +149,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } } @@ -164,7 +164,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { self.handler_table.read(|table| { for (i, opt) in table.iter().enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i, handler.name); + info!(" {: >3}. {}", i, handler.name()); } } }); diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index f67d70df..0ee7feb7 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -10,9 +10,9 @@ //! - use crate::{ - bsp, bsp::device_driver::common::MMIODerefWrapper, - console, cpu, driver, exception, + console, cpu, driver, + exception::{self, asynchronous::IRQNumber}, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -233,8 +233,6 @@ struct PL011UartInner { /// Representation of the UART. pub struct PL011Uart { inner: IRQSafeNullLock, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -398,16 +396,9 @@ impl PL011Uart { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - /// - The user must ensure to provide correct IRQ numbers. - pub const unsafe fn new( - mmio_start_addr: Address, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(PL011UartInner::new(mmio_start_addr)), - irq_number, - post_init_callback, } } } @@ -418,27 +409,28 @@ impl PL011Uart { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for PL011Uart { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } unsafe fn init(&self) -> Result<(), &'static str> { self.inner.lock(|inner| inner.init()); - (self.post_init_callback)(); Ok(()) } - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - use exception::asynchronous::{irq_manager, IRQDescriptor}; + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + use exception::asynchronous::{irq_manager, IRQHandlerDescriptor}; - let descriptor = IRQDescriptor { - name: Self::COMPATIBLE, - handler: self, - }; + let descriptor = IRQHandlerDescriptor::new(*irq_number, Self::COMPATIBLE, self); - irq_manager().register_handler(self.irq_number, descriptor)?; - irq_manager().enable(self.irq_number); + irq_manager().register_handler(descriptor)?; + irq_manager().enable(irq_number); Ok(()) } diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/common.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/common.rs index 69f038d4..ca7aeb76 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/common.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/device_driver/common.rs @@ -5,7 +5,7 @@ //! Common device driver code. use crate::memory::{Address, Virtual}; -use core::{marker::PhantomData, ops}; +use core::{fmt, marker::PhantomData, ops}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -16,6 +16,10 @@ pub struct MMIODerefWrapper { phantom: PhantomData T>, } +/// A wrapper type for usize with integrated range bound check. +#[derive(Copy, Clone)] +pub struct BoundedUsize(usize); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -37,3 +41,25 @@ impl ops::Deref for MMIODerefWrapper { unsafe { &*(self.start_addr.as_usize() as *const _) } } } + +impl BoundedUsize<{ MAX_INCLUSIVE }> { + pub const MAX_INCLUSIVE: usize = MAX_INCLUSIVE; + + /// Creates a new instance if number <= MAX_INCLUSIVE. + pub const fn new(number: usize) -> Self { + assert!(number <= MAX_INCLUSIVE); + + Self(number) + } + + /// Return the wrapped number. + pub const fn get(self) -> usize { + self.0 + } +} + +impl fmt::Display for BoundedUsize<{ MAX_INCLUSIVE }> { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + write!(f, "{}", self.0) + } +} diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/raspberrypi/driver.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/raspberrypi/driver.rs index e1db4a00..ca3435aa 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/raspberrypi/driver.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/raspberrypi/driver.rs @@ -7,34 +7,16 @@ use super::{exception, memory::map::mmio}; use crate::{ bsp::device_driver, - console, driver, exception as generic_exception, memory, + console, driver as generic_driver, + exception::{self as generic_exception}, + memory, memory::mmu::MMIODescriptor, - synchronization::{interface::ReadWriteEx, InitStateLock}, }; use core::{ mem::MaybeUninit, sync::atomic::{AtomicBool, Ordering}, }; -pub use device_driver::IRQNumber; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: InitStateLock<[Option<&'static (dyn DeviceDriver + Sync)>; NUM_DRIVERS]>, - init_done: AtomicBool, -} - -//-------------------------------------------------------------------------------------------------- -// Public Definitions -//-------------------------------------------------------------------------------------------------- - -/// The number of active drivers provided by this BSP. -pub const NUM_DRIVERS: usize = 3; - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- @@ -49,150 +31,154 @@ static mut INTERRUPT_CONTROLLER: MaybeUninit #[cfg(feature = "bsp_rpi4")] static mut INTERRUPT_CONTROLLER: MaybeUninit = MaybeUninit::uninit(); -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: InitStateLock::new([None; NUM_DRIVERS]), - init_done: AtomicBool::new(false), -}; - //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -impl BSPDriverManager { - unsafe fn instantiate_uart(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_uart() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn uart_post_init() { - console::register_console(unsafe { PL011_UART.assume_init_ref() }); - } + PL011_UART.write(device_driver::PL011Uart::new(virt_addr)); - PL011_UART.write(device_driver::PL011Uart::new( - virt_addr, - exception::asynchronous::irq_map::PL011_UART, - uart_post_init, - )); + Ok(()) +} - Ok(()) - } +/// This must be called only after successful init of the UART driver. +unsafe fn post_init_uart() -> Result<(), &'static str> { + console::register_console(PL011_UART.assume_init_ref()); - unsafe fn instantiate_gpio(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; + Ok(()) +} - // This is safe to do, because it is only called from the init'ed instance itself. - fn gpio_post_init() { - unsafe { GPIO.assume_init_ref().map_pl011_uart() }; - } +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_gpio() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; - GPIO.write(device_driver::GPIO::new(virt_addr, gpio_post_init)); + GPIO.write(device_driver::GPIO::new(virt_addr)); - Ok(()) - } + Ok(()) +} - #[cfg(feature = "bsp_rpi3")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let periph_mmio_descriptor = - MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); - let periph_virt_addr = memory::mmu::kernel_map_mmio( - device_driver::InterruptController::COMPATIBLE, - &periph_mmio_descriptor, - )?; - - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } - - INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new( - periph_virt_addr, - interrupt_controller_post_init, - )); - - Ok(()) - } +/// This must be called only after successful init of the GPIO driver. +unsafe fn post_init_gpio() -> Result<(), &'static str> { + GPIO.assume_init_ref().map_pl011_uart(); + Ok(()) +} + +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi3")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let periph_mmio_descriptor = + MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); + let periph_virt_addr = memory::mmu::kernel_map_mmio( + device_driver::InterruptController::COMPATIBLE, + &periph_mmio_descriptor, + )?; + + INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new(periph_virt_addr)); - #[cfg(feature = "bsp_rpi4")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); - let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; + Ok(()) +} - let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); - let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi4")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); + let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } + let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); + let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; - INTERRUPT_CONTROLLER.write(device_driver::GICv2::new( - gicd_virt_addr, - gicc_virt_addr, - interrupt_controller_post_init, - )); + INTERRUPT_CONTROLLER.write(device_driver::GICv2::new(gicd_virt_addr, gicc_virt_addr)); - Ok(()) - } + Ok(()) +} - unsafe fn register_drivers(&self) { - self.device_drivers.write(|drivers| { - drivers[0] = Some(PL011_UART.assume_init_ref()); - drivers[1] = Some(GPIO.assume_init_ref()); - drivers[2] = Some(INTERRUPT_CONTROLLER.assume_init_ref()); - }); - } +/// This must be called only after successful init of the interrupt controller driver. +unsafe fn post_init_interrupt_controller() -> Result<(), &'static str> { + generic_exception::asynchronous::register_irq_manager(INTERRUPT_CONTROLLER.assume_init_ref()); + + Ok(()) } -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_uart() -> Result<(), &'static str> { + instantiate_uart()?; + + let uart_descriptor = generic_driver::DeviceDriverDescriptor::new( + PL011_UART.assume_init_ref(), + Some(post_init_uart), + Some(exception::asynchronous::irq_map::PL011_UART), + ); + generic_driver::driver_manager().register_driver(uart_descriptor); + + Ok(()) +} + +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_gpio() -> Result<(), &'static str> { + instantiate_gpio()?; + + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new( + GPIO.assume_init_ref(), + Some(post_init_gpio), + None, + ); + generic_driver::driver_manager().register_driver(gpio_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_interrupt_controller() -> Result<(), &'static str> { + instantiate_interrupt_controller()?; -impl driver::interface::DriverManager for BSPDriverManager { - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str> { - if self.init_done.load(Ordering::Relaxed) { - return Err("Drivers already instantiated"); - } + let interrupt_controller_descriptor = generic_driver::DeviceDriverDescriptor::new( + INTERRUPT_CONTROLLER.assume_init_ref(), + Some(post_init_interrupt_controller), + None, + ); + generic_driver::driver_manager().register_driver(interrupt_controller_descriptor); - self.instantiate_uart()?; - self.instantiate_gpio()?; - self.instantiate_interrupt_controller()?; + Ok(()) +} - self.register_drivers(); +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - self.init_done.store(true, Ordering::Relaxed); - Ok(()) +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } - fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); NUM_DRIVERS] { - self.device_drivers - .read(|drivers| drivers.map(|drivers| drivers.unwrap())) - } + driver_uart()?; + driver_gpio()?; + driver_interrupt_controller()?; - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self) { - use crate::cpu; + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) +} - unsafe { - self.instantiate_uart() - .unwrap_or_else(|_| cpu::qemu_exit_failure()); - console::register_console(PL011_UART.assume_init_ref()); - }; - } +/// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps +/// than on real hardware due to QEMU's abstractions. +#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() { + use crate::cpu; + + unsafe { + instantiate_uart().unwrap_or_else(|_| cpu::qemu_exit_failure()); + console::register_console(PL011_UART.assume_init_ref()); + }; } diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/raspberrypi/exception/asynchronous.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/raspberrypi/exception/asynchronous.rs index ab20d86d..06a67558 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/raspberrypi/exception/asynchronous.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/bsp/raspberrypi/exception/asynchronous.rs @@ -10,6 +10,9 @@ use crate::bsp; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Export for reuse in generic asynchronous.rs. +pub use bsp::device_driver::IRQNumber; + #[cfg(feature = "bsp_rpi3")] pub(in crate::bsp) mod irq_map { use super::bsp::device_driver::{IRQNumber, PeripheralIRQ}; diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/driver.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/driver.rs index 98197fef..18066c31 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/driver.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/driver.rs @@ -4,16 +4,37 @@ //! Driver support. +use crate::{ + exception, info, + synchronization::{interface::ReadWriteEx, InitStateLock}, +}; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner +where + T: 'static, +{ + next_index: usize, + descriptors: [Option>; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Driver interfaces. pub mod interface { - use crate::bsp; - /// Device Driver functions. pub trait DeviceDriver { + /// Different interrupt controllers might use different types for IRQ number. + type IRQNumberType: super::fmt::Display; + /// Return a compatibility string for identifying the driver. fn compatible(&self) -> &'static str; @@ -26,32 +47,175 @@ pub mod interface { Ok(()) } - /// Called by the kernel to register and enable the device's IRQ handlers, if any. + /// Called by the kernel to register and enable the device's IRQ handler. /// /// Rust's type system will prevent a call to this function unless the calling instance /// itself has static lifetime. - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - Ok(()) + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + panic!( + "Attempt to enable IRQ {} for device {}, but driver does not support this", + irq_number, + self.compatible() + ) + } + } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor +where + T: 'static, +{ + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager +where + T: 'static, +{ + inner: InitStateLock>, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner +where + T: 'static + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], } } +} - /// Device driver management functions. +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + irq_number, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager +where + T: fmt::Display + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: InitStateLock::new(DriverManagerInner::new()), + } + } + + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.write(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.read(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers and their interrupts handlers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Instantiate all drivers. - /// - /// # Safety - /// - /// Must be called before `all_device_drivers`. - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str>; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers_and_irqs(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + + // 3. After all post-init callbacks were done, the interrupt controller should be + // registered and functional. So let drivers register with it now. + self.for_each_descriptor(|descriptor| { + if let Some(irq_number) = &descriptor.irq_number { + if let Err(x) = descriptor + .device_driver + .register_and_enable_irq_handler(irq_number) + { + panic!( + "Error during driver interrupt handler registration: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } - /// Return a slice of references to all `BSP`-instantiated drivers. - fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); bsp::driver::NUM_DRIVERS]; + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); - /// Minimal code needed to bring up the console in QEMU (for testing only). This is often - /// less steps than on real hardware due to QEMU's abstractions. - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self); + i += 1; + }); } } diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/exception/asynchronous.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/exception/asynchronous.rs index 83ca1aa6..c1f2a27b 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/exception/asynchronous.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/exception/asynchronous.rs @@ -10,7 +10,7 @@ mod arch_asynchronous; mod null_irq_manager; use crate::{bsp, synchronization}; -use core::{fmt, marker::PhantomData}; +use core::marker::PhantomData; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports @@ -24,14 +24,23 @@ pub use arch_asynchronous::{ // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Interrupt number as defined by the BSP. +pub type IRQNumber = bsp::exception::asynchronous::IRQNumber; + /// Interrupt descriptor. #[derive(Copy, Clone)] -pub struct IRQDescriptor { +pub struct IRQHandlerDescriptor +where + T: Copy, +{ + /// The IRQ number. + number: T, + /// Descriptive name. - pub name: &'static str, + name: &'static str, /// Reference to handler trait object. - pub handler: &'static (dyn interface::IRQHandler + Sync), + handler: &'static (dyn interface::IRQHandler + Sync), } /// IRQContext token. @@ -61,17 +70,16 @@ pub mod interface { /// platform's interrupt controller. pub trait IRQManager { /// The IRQ number type depends on the implementation. - type IRQNumberType; + type IRQNumberType: Copy; /// Register a handler. fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: super::IRQDescriptor, + irq_handler_descriptor: super::IRQHandlerDescriptor, ) -> Result<(), &'static str>; /// Enable an interrupt in the controller. - fn enable(&self, irq_number: Self::IRQNumberType); + fn enable(&self, irq_number: &Self::IRQNumberType); /// Handle pending interrupts. /// @@ -91,16 +99,12 @@ pub mod interface { } } -/// A wrapper type for IRQ numbers with integrated range sanity check. -#[derive(Copy, Clone)] -pub struct IRQNumber(usize); - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- static CUR_IRQ_MANAGER: InitStateLock< - &'static (dyn interface::IRQManager + Sync), + &'static (dyn interface::IRQManager + Sync), > = InitStateLock::new(&null_irq_manager::NULL_IRQ_MANAGER); //-------------------------------------------------------------------------------------------------- @@ -108,6 +112,39 @@ static CUR_IRQ_MANAGER: InitStateLock< //-------------------------------------------------------------------------------------------------- use synchronization::{interface::ReadWriteEx, InitStateLock}; +impl IRQHandlerDescriptor +where + T: Copy, +{ + /// Create an instance. + pub const fn new( + number: T, + name: &'static str, + handler: &'static (dyn interface::IRQHandler + Sync), + ) -> Self { + Self { + number, + name, + handler, + } + } + + /// Return the number. + pub const fn number(&self) -> T { + self.number + } + + /// Return the name. + pub const fn name(&self) -> &'static str { + self.name + } + + /// Return the handler. + pub const fn handler(&self) -> &'static (dyn interface::IRQHandler + Sync) { + self.handler + } +} + impl<'irq_context> IRQContext<'irq_context> { /// Creates an IRQContext token. /// @@ -125,29 +162,6 @@ impl<'irq_context> IRQContext<'irq_context> { } } -impl IRQNumber<{ MAX_INCLUSIVE }> { - /// The total number of IRQs this type supports. - pub const NUM_TOTAL: usize = MAX_INCLUSIVE + 1; - - /// Creates a new instance if number <= MAX_INCLUSIVE. - pub const fn new(number: usize) -> Self { - assert!(number <= MAX_INCLUSIVE); - - Self(number) - } - - /// Return the wrapped number. - pub const fn get(self) -> usize { - self.0 - } -} - -impl fmt::Display for IRQNumber<{ MAX_INCLUSIVE }> { - fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { - write!(f, "{}", self.0) - } -} - /// Executes the provided closure while IRQs are masked on the executing core. /// /// While the function temporarily changes the HW state of the executing core, it restores it to the @@ -163,8 +177,7 @@ pub fn exec_with_irq_masked(f: impl FnOnce() -> T) -> T { /// Register a new IRQ manager. pub fn register_irq_manager( - new_manager: &'static (dyn interface::IRQManager - + Sync), + new_manager: &'static (dyn interface::IRQManager + Sync), ) { CUR_IRQ_MANAGER.write(|manager| *manager = new_manager); } @@ -172,6 +185,6 @@ pub fn register_irq_manager( /// Return a reference to the currently registered IRQ manager. /// /// This is the IRQ manager used by the architectural interrupt handling code. -pub fn irq_manager() -> &'static dyn interface::IRQManager { +pub fn irq_manager() -> &'static dyn interface::IRQManager { CUR_IRQ_MANAGER.read(|manager| *manager) } diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/exception/asynchronous/null_irq_manager.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/exception/asynchronous/null_irq_manager.rs index 560e3ce4..438f9649 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/exception/asynchronous/null_irq_manager.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/exception/asynchronous/null_irq_manager.rs @@ -4,8 +4,7 @@ //! Null IRQ Manager. -use super::{interface, IRQContext, IRQDescriptor}; -use crate::bsp; +use super::{interface, IRQContext, IRQHandlerDescriptor}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -24,17 +23,16 @@ pub static NULL_IRQ_MANAGER: NullIRQManager = NullIRQManager {}; //-------------------------------------------------------------------------------------------------- impl interface::IRQManager for NullIRQManager { - type IRQNumberType = bsp::driver::IRQNumber; + type IRQNumberType = super::IRQNumber; fn register_handler( &self, - _irq_number: Self::IRQNumberType, - _descriptor: IRQDescriptor, + _descriptor: IRQHandlerDescriptor, ) -> Result<(), &'static str> { panic!("No IRQ Manager registered yet"); } - fn enable(&self, _irq_number: Self::IRQNumberType) { + fn enable(&self, _irq_number: &Self::IRQNumberType) { panic!("No IRQ Manager registered yet"); } diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/lib.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/lib.rs index 59ef14ac..f595b587 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/lib.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/lib.rs @@ -181,11 +181,9 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); test_main(); diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/src/main.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/src/main.rs index 9aeb4438..65905258 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/src/main.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/src/main.rs @@ -13,7 +13,7 @@ #![no_main] #![no_std] -use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; +use libkernel::{bsp, cpu, driver, exception, info, memory, state, time}; /// Early init code. /// @@ -25,27 +25,16 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// - Printing will not work until the respective driver's MMIO is remapped. #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - // Instantiate and init all device drivers. - if let Err(x) = bsp::driver::driver_manager().instantiate_drivers() { - panic!("Error instantiating drivers: {}", x); - } - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - // Let device drivers register and enable their handlers with the interrupt controller. - for i in bsp::driver::driver_manager().all_device_drivers() { - if let Err(msg) = i.register_and_enable_irq_handler() { - warn!("Error registering IRQ handler: {}", msg); - } - } + // Initialize all device drivers. + driver::driver_manager().init_drivers_and_irqs(); bsp::memory::mmu::kernel_add_mapping_records_for_precomputed(); @@ -61,8 +50,6 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { - use driver::interface::DriverManager; - info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); @@ -81,13 +68,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); info!("Registered IRQ handlers:"); exception::asynchronous::irq_manager().print_handler(); diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/tests/00_console_sanity.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/tests/00_console_sanity.rs index 305510ce..2c0225b7 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/tests/00_console_sanity.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/tests/00_console_sanity.rs @@ -11,16 +11,15 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{bsp, console, cpu, driver, exception, memory, print}; +use libkernel::{bsp, console, cpu, exception, memory, print}; #[no_mangle] unsafe fn kernel_init() -> ! { use console::console; - use driver::interface::DriverManager; exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/tests/01_timer_sanity.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/tests/01_timer_sanity.rs index 39899332..8188b942 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/tests/01_timer_sanity.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/tests/01_timer_sanity.rs @@ -11,16 +11,14 @@ #![test_runner(libkernel::test_runner)] use core::time::Duration; -use libkernel::{bsp, cpu, driver, exception, memory, time}; +use libkernel::{bsp, cpu, exception, memory, time}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/tests/02_exception_sync_page_fault.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/tests/02_exception_sync_page_fault.rs index a6d15b69..fab44c8f 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/tests/02_exception_sync_page_fault.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/tests/02_exception_sync_page_fault.rs @@ -17,15 +17,13 @@ /// or indirectly. mod panic_exit_success; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/tests/03_exception_restore_sanity.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/tests/03_exception_restore_sanity.rs index 47dd2714..f176c6a6 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/tests/03_exception_restore_sanity.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/tests/03_exception_restore_sanity.rs @@ -12,7 +12,7 @@ mod panic_wait_forever; use core::arch::asm; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[inline(never)] fn nested_system_call() { @@ -30,11 +30,9 @@ fn nested_system_call() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing exception restore"); diff --git a/16_virtual_mem_part4_higher_half_kernel/kernel/tests/04_exception_irq_sanity.rs b/16_virtual_mem_part4_higher_half_kernel/kernel/tests/04_exception_irq_sanity.rs index e93a4caf..e6f94c91 100644 --- a/16_virtual_mem_part4_higher_half_kernel/kernel/tests/04_exception_irq_sanity.rs +++ b/16_virtual_mem_part4_higher_half_kernel/kernel/tests/04_exception_irq_sanity.rs @@ -10,15 +10,13 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -use libkernel::{bsp, cpu, driver, exception, memory}; +use libkernel::{bsp, cpu, exception, memory}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); exception::handling_init(); exception::asynchronous::local_irq_unmask(); diff --git a/17_kernel_symbols/Makefile b/17_kernel_symbols/Makefile index 9af33140..7bed51da 100644 --- a/17_kernel_symbols/Makefile +++ b/17_kernel_symbols/Makefile @@ -335,7 +335,7 @@ test_boot: $(KERNEL_BIN) ## Helpers for unit and integration test targets ##------------------------------------------------------------------------------ define KERNEL_TEST_RUNNER - #!/usr/bin/env bash +#!/usr/bin/env bash # The cargo test runner seems to change into the crate under test's directory. Therefore, ensure # this script executes from the root. diff --git a/17_kernel_symbols/kernel/src/bsp/device_driver/arm/gicv2.rs b/17_kernel_symbols/kernel/src/bsp/device_driver/arm/gicv2.rs index ec65f1b2..3cc35b5e 100644 --- a/17_kernel_symbols/kernel/src/bsp/device_driver/arm/gicv2.rs +++ b/17_kernel_symbols/kernel/src/bsp/device_driver/arm/gicv2.rs @@ -80,7 +80,8 @@ mod gicc; mod gicd; use crate::{ - bsp, cpu, driver, exception, + bsp::{self, device_driver::common::BoundedUsize}, + cpu, driver, exception, memory::{Address, Virtual}, synchronization, synchronization::InitStateLock, @@ -90,14 +91,15 @@ use crate::{ // Private Definitions //-------------------------------------------------------------------------------------------------- -type HandlerTable = [Option; IRQNumber::NUM_TOTAL]; +type HandlerTable = [Option>; + IRQNumber::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. -pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER }>; +pub type IRQNumber = BoundedUsize<{ GICv2::MAX_IRQ_NUMBER }>; /// Representation of the GIC. pub struct GICv2 { @@ -109,9 +111,6 @@ pub struct GICv2 { /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. handler_table: InitStateLock, - - /// Callback to be invoked after successful init. - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -131,13 +130,11 @@ impl GICv2 { pub const unsafe fn new( gicd_mmio_start_addr: Address, gicc_mmio_start_addr: Address, - post_init_callback: fn(), ) -> Self { Self { gicd: gicd::GICD::new(gicd_mmio_start_addr), gicc: gicc::GICC::new(gicc_mmio_start_addr), - handler_table: InitStateLock::new([None; IRQNumber::NUM_TOTAL]), - post_init_callback, + handler_table: InitStateLock::new([None; IRQNumber::MAX_INCLUSIVE + 1]), } } } @@ -148,6 +145,8 @@ impl GICv2 { use synchronization::interface::ReadWriteEx; impl driver::interface::DeviceDriver for GICv2 { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } @@ -160,8 +159,6 @@ impl driver::interface::DeviceDriver for GICv2 { self.gicc.priority_accept_all(); self.gicc.enable(); - (self.post_init_callback)(); - Ok(()) } } @@ -171,23 +168,22 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq_number.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq_number: Self::IRQNumberType) { + fn enable(&self, irq_number: &Self::IRQNumberType) { self.gicd.enable(irq_number); } @@ -210,7 +206,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } }); @@ -227,7 +223,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { self.handler_table.read(|table| { for (i, opt) in table.iter().skip(32).enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i + 32, handler.name); + info!(" {: >3}. {}", i + 32, handler.name()); } } }); diff --git a/17_kernel_symbols/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs b/17_kernel_symbols/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs index a9c97a8e..8aebcf2b 100644 --- a/17_kernel_symbols/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/17_kernel_symbols/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -172,7 +172,7 @@ impl GICD { } /// Enable an interrupt. - pub fn enable(&self, irq_num: super::IRQNumber) { + pub fn enable(&self, irq_num: &super::IRQNumber) { let irq_num = irq_num.get(); // Each bit in the u32 enable register corresponds to one IRQ number. Shift right by 5 diff --git a/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 2281e66a..fb61a651 100644 --- a/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -7,6 +7,7 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, + exception::asynchronous::IRQNumber, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -122,7 +123,6 @@ struct GPIOInner { /// Representation of the GPIO HW. pub struct GPIO { inner: IRQSafeNullLock, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -202,10 +202,9 @@ impl GPIO { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new(mmio_start_addr: Address, post_init_callback: fn()) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(GPIOInner::new(mmio_start_addr)), - post_init_callback, } } @@ -221,13 +220,9 @@ impl GPIO { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for GPIO { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { - (self.post_init_callback)(); - - Ok(()) - } } diff --git a/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs index b4dd530d..c93a9fa1 100644 --- a/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ b/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -7,9 +7,12 @@ mod peripheral_ic; use crate::{ - driver, exception, + bsp::device_driver::common::BoundedUsize, + driver, + exception::{self, asynchronous::IRQHandlerDescriptor}, memory::{Address, Virtual}, }; +use core::fmt; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -24,10 +27,8 @@ struct PendingIRQs { // Public Definitions //-------------------------------------------------------------------------------------------------- -pub type LocalIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; -pub type PeripheralIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; +pub type LocalIRQ = BoundedUsize<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; +pub type PeripheralIRQ = BoundedUsize<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. #[derive(Copy, Clone)] @@ -40,7 +41,6 @@ pub enum IRQNumber { /// Representation of the Interrupt Controller. pub struct InterruptController { periph: peripheral_ic::PeripheralIC, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -71,6 +71,15 @@ impl Iterator for PendingIRQs { // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for IRQNumber { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + match self { + Self::Local(number) => write!(f, "Local({})", number), + Self::Peripheral(number) => write!(f, "Peripheral({})", number), + } + } +} + impl InterruptController { // Restrict to 3 for now. This makes future code for local_ic.rs more straight forward. const MAX_LOCAL_IRQ_NUMBER: usize = 3; @@ -83,13 +92,9 @@ impl InterruptController { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new( - periph_mmio_start_addr: Address, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(periph_mmio_start_addr: Address) -> Self { Self { periph: peripheral_ic::PeripheralIC::new(periph_mmio_start_addr), - post_init_callback, } } } @@ -99,15 +104,11 @@ impl InterruptController { //------------------------------------------------------------------------------ impl driver::interface::DeviceDriver for InterruptController { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { - (self.post_init_callback)(); - - Ok(()) - } } impl exception::asynchronous::interface::IRQManager for InterruptController { @@ -115,16 +116,23 @@ impl exception::asynchronous::interface::IRQManager for InterruptController { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { - match irq { + match irq_handler_descriptor.number() { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), - IRQNumber::Peripheral(pirq) => self.periph.register_handler(pirq, descriptor), + IRQNumber::Peripheral(pirq) => { + let periph_descriptor = IRQHandlerDescriptor::new( + pirq, + irq_handler_descriptor.name(), + irq_handler_descriptor.handler(), + ); + + self.periph.register_handler(periph_descriptor) + } } } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { match irq { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), IRQNumber::Peripheral(pirq) => self.periph.enable(pirq), diff --git a/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 06802d27..0a20bd87 100644 --- a/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -52,7 +52,8 @@ type WriteOnlyRegisters = MMIODerefWrapper; /// Abstraction for the ReadOnly parts of the associated MMIO registers. type ReadOnlyRegisters = MMIODerefWrapper; -type HandlerTable = [Option; PeripheralIRQ::NUM_TOTAL]; +type HandlerTable = [Option>; + PeripheralIRQ::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -84,7 +85,7 @@ impl PeripheralIC { Self { wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(mmio_start_addr)), ro_registers: ReadOnlyRegisters::new(mmio_start_addr), - handler_table: InitStateLock::new([None; PeripheralIRQ::NUM_TOTAL]), + handler_table: InitStateLock::new([None; PeripheralIRQ::MAX_INCLUSIVE + 1]), } } @@ -107,23 +108,22 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { self.wo_registers.lock(|regs| { let enable_reg = if irq.get() <= 31 { ®s.ENABLE_1 @@ -149,7 +149,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } } @@ -164,7 +164,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { self.handler_table.read(|table| { for (i, opt) in table.iter().enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i, handler.name); + info!(" {: >3}. {}", i, handler.name()); } } }); diff --git a/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index f67d70df..0ee7feb7 100644 --- a/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/17_kernel_symbols/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -10,9 +10,9 @@ //! - use crate::{ - bsp, bsp::device_driver::common::MMIODerefWrapper, - console, cpu, driver, exception, + console, cpu, driver, + exception::{self, asynchronous::IRQNumber}, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -233,8 +233,6 @@ struct PL011UartInner { /// Representation of the UART. pub struct PL011Uart { inner: IRQSafeNullLock, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -398,16 +396,9 @@ impl PL011Uart { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - /// - The user must ensure to provide correct IRQ numbers. - pub const unsafe fn new( - mmio_start_addr: Address, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(PL011UartInner::new(mmio_start_addr)), - irq_number, - post_init_callback, } } } @@ -418,27 +409,28 @@ impl PL011Uart { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for PL011Uart { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } unsafe fn init(&self) -> Result<(), &'static str> { self.inner.lock(|inner| inner.init()); - (self.post_init_callback)(); Ok(()) } - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - use exception::asynchronous::{irq_manager, IRQDescriptor}; + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + use exception::asynchronous::{irq_manager, IRQHandlerDescriptor}; - let descriptor = IRQDescriptor { - name: Self::COMPATIBLE, - handler: self, - }; + let descriptor = IRQHandlerDescriptor::new(*irq_number, Self::COMPATIBLE, self); - irq_manager().register_handler(self.irq_number, descriptor)?; - irq_manager().enable(self.irq_number); + irq_manager().register_handler(descriptor)?; + irq_manager().enable(irq_number); Ok(()) } diff --git a/17_kernel_symbols/kernel/src/bsp/device_driver/common.rs b/17_kernel_symbols/kernel/src/bsp/device_driver/common.rs index 69f038d4..ca7aeb76 100644 --- a/17_kernel_symbols/kernel/src/bsp/device_driver/common.rs +++ b/17_kernel_symbols/kernel/src/bsp/device_driver/common.rs @@ -5,7 +5,7 @@ //! Common device driver code. use crate::memory::{Address, Virtual}; -use core::{marker::PhantomData, ops}; +use core::{fmt, marker::PhantomData, ops}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -16,6 +16,10 @@ pub struct MMIODerefWrapper { phantom: PhantomData T>, } +/// A wrapper type for usize with integrated range bound check. +#[derive(Copy, Clone)] +pub struct BoundedUsize(usize); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -37,3 +41,25 @@ impl ops::Deref for MMIODerefWrapper { unsafe { &*(self.start_addr.as_usize() as *const _) } } } + +impl BoundedUsize<{ MAX_INCLUSIVE }> { + pub const MAX_INCLUSIVE: usize = MAX_INCLUSIVE; + + /// Creates a new instance if number <= MAX_INCLUSIVE. + pub const fn new(number: usize) -> Self { + assert!(number <= MAX_INCLUSIVE); + + Self(number) + } + + /// Return the wrapped number. + pub const fn get(self) -> usize { + self.0 + } +} + +impl fmt::Display for BoundedUsize<{ MAX_INCLUSIVE }> { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + write!(f, "{}", self.0) + } +} diff --git a/17_kernel_symbols/kernel/src/bsp/raspberrypi/driver.rs b/17_kernel_symbols/kernel/src/bsp/raspberrypi/driver.rs index e1db4a00..ca3435aa 100644 --- a/17_kernel_symbols/kernel/src/bsp/raspberrypi/driver.rs +++ b/17_kernel_symbols/kernel/src/bsp/raspberrypi/driver.rs @@ -7,34 +7,16 @@ use super::{exception, memory::map::mmio}; use crate::{ bsp::device_driver, - console, driver, exception as generic_exception, memory, + console, driver as generic_driver, + exception::{self as generic_exception}, + memory, memory::mmu::MMIODescriptor, - synchronization::{interface::ReadWriteEx, InitStateLock}, }; use core::{ mem::MaybeUninit, sync::atomic::{AtomicBool, Ordering}, }; -pub use device_driver::IRQNumber; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: InitStateLock<[Option<&'static (dyn DeviceDriver + Sync)>; NUM_DRIVERS]>, - init_done: AtomicBool, -} - -//-------------------------------------------------------------------------------------------------- -// Public Definitions -//-------------------------------------------------------------------------------------------------- - -/// The number of active drivers provided by this BSP. -pub const NUM_DRIVERS: usize = 3; - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- @@ -49,150 +31,154 @@ static mut INTERRUPT_CONTROLLER: MaybeUninit #[cfg(feature = "bsp_rpi4")] static mut INTERRUPT_CONTROLLER: MaybeUninit = MaybeUninit::uninit(); -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: InitStateLock::new([None; NUM_DRIVERS]), - init_done: AtomicBool::new(false), -}; - //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -impl BSPDriverManager { - unsafe fn instantiate_uart(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_uart() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn uart_post_init() { - console::register_console(unsafe { PL011_UART.assume_init_ref() }); - } + PL011_UART.write(device_driver::PL011Uart::new(virt_addr)); - PL011_UART.write(device_driver::PL011Uart::new( - virt_addr, - exception::asynchronous::irq_map::PL011_UART, - uart_post_init, - )); + Ok(()) +} - Ok(()) - } +/// This must be called only after successful init of the UART driver. +unsafe fn post_init_uart() -> Result<(), &'static str> { + console::register_console(PL011_UART.assume_init_ref()); - unsafe fn instantiate_gpio(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; + Ok(()) +} - // This is safe to do, because it is only called from the init'ed instance itself. - fn gpio_post_init() { - unsafe { GPIO.assume_init_ref().map_pl011_uart() }; - } +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_gpio() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; - GPIO.write(device_driver::GPIO::new(virt_addr, gpio_post_init)); + GPIO.write(device_driver::GPIO::new(virt_addr)); - Ok(()) - } + Ok(()) +} - #[cfg(feature = "bsp_rpi3")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let periph_mmio_descriptor = - MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); - let periph_virt_addr = memory::mmu::kernel_map_mmio( - device_driver::InterruptController::COMPATIBLE, - &periph_mmio_descriptor, - )?; - - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } - - INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new( - periph_virt_addr, - interrupt_controller_post_init, - )); - - Ok(()) - } +/// This must be called only after successful init of the GPIO driver. +unsafe fn post_init_gpio() -> Result<(), &'static str> { + GPIO.assume_init_ref().map_pl011_uart(); + Ok(()) +} + +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi3")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let periph_mmio_descriptor = + MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); + let periph_virt_addr = memory::mmu::kernel_map_mmio( + device_driver::InterruptController::COMPATIBLE, + &periph_mmio_descriptor, + )?; + + INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new(periph_virt_addr)); - #[cfg(feature = "bsp_rpi4")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); - let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; + Ok(()) +} - let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); - let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi4")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); + let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } + let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); + let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; - INTERRUPT_CONTROLLER.write(device_driver::GICv2::new( - gicd_virt_addr, - gicc_virt_addr, - interrupt_controller_post_init, - )); + INTERRUPT_CONTROLLER.write(device_driver::GICv2::new(gicd_virt_addr, gicc_virt_addr)); - Ok(()) - } + Ok(()) +} - unsafe fn register_drivers(&self) { - self.device_drivers.write(|drivers| { - drivers[0] = Some(PL011_UART.assume_init_ref()); - drivers[1] = Some(GPIO.assume_init_ref()); - drivers[2] = Some(INTERRUPT_CONTROLLER.assume_init_ref()); - }); - } +/// This must be called only after successful init of the interrupt controller driver. +unsafe fn post_init_interrupt_controller() -> Result<(), &'static str> { + generic_exception::asynchronous::register_irq_manager(INTERRUPT_CONTROLLER.assume_init_ref()); + + Ok(()) } -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_uart() -> Result<(), &'static str> { + instantiate_uart()?; + + let uart_descriptor = generic_driver::DeviceDriverDescriptor::new( + PL011_UART.assume_init_ref(), + Some(post_init_uart), + Some(exception::asynchronous::irq_map::PL011_UART), + ); + generic_driver::driver_manager().register_driver(uart_descriptor); + + Ok(()) +} + +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_gpio() -> Result<(), &'static str> { + instantiate_gpio()?; + + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new( + GPIO.assume_init_ref(), + Some(post_init_gpio), + None, + ); + generic_driver::driver_manager().register_driver(gpio_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_interrupt_controller() -> Result<(), &'static str> { + instantiate_interrupt_controller()?; -impl driver::interface::DriverManager for BSPDriverManager { - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str> { - if self.init_done.load(Ordering::Relaxed) { - return Err("Drivers already instantiated"); - } + let interrupt_controller_descriptor = generic_driver::DeviceDriverDescriptor::new( + INTERRUPT_CONTROLLER.assume_init_ref(), + Some(post_init_interrupt_controller), + None, + ); + generic_driver::driver_manager().register_driver(interrupt_controller_descriptor); - self.instantiate_uart()?; - self.instantiate_gpio()?; - self.instantiate_interrupt_controller()?; + Ok(()) +} - self.register_drivers(); +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - self.init_done.store(true, Ordering::Relaxed); - Ok(()) +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } - fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); NUM_DRIVERS] { - self.device_drivers - .read(|drivers| drivers.map(|drivers| drivers.unwrap())) - } + driver_uart()?; + driver_gpio()?; + driver_interrupt_controller()?; - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self) { - use crate::cpu; + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) +} - unsafe { - self.instantiate_uart() - .unwrap_or_else(|_| cpu::qemu_exit_failure()); - console::register_console(PL011_UART.assume_init_ref()); - }; - } +/// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps +/// than on real hardware due to QEMU's abstractions. +#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() { + use crate::cpu; + + unsafe { + instantiate_uart().unwrap_or_else(|_| cpu::qemu_exit_failure()); + console::register_console(PL011_UART.assume_init_ref()); + }; } diff --git a/17_kernel_symbols/kernel/src/bsp/raspberrypi/exception/asynchronous.rs b/17_kernel_symbols/kernel/src/bsp/raspberrypi/exception/asynchronous.rs index ab20d86d..06a67558 100644 --- a/17_kernel_symbols/kernel/src/bsp/raspberrypi/exception/asynchronous.rs +++ b/17_kernel_symbols/kernel/src/bsp/raspberrypi/exception/asynchronous.rs @@ -10,6 +10,9 @@ use crate::bsp; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Export for reuse in generic asynchronous.rs. +pub use bsp::device_driver::IRQNumber; + #[cfg(feature = "bsp_rpi3")] pub(in crate::bsp) mod irq_map { use super::bsp::device_driver::{IRQNumber, PeripheralIRQ}; diff --git a/17_kernel_symbols/kernel/src/driver.rs b/17_kernel_symbols/kernel/src/driver.rs index 98197fef..18066c31 100644 --- a/17_kernel_symbols/kernel/src/driver.rs +++ b/17_kernel_symbols/kernel/src/driver.rs @@ -4,16 +4,37 @@ //! Driver support. +use crate::{ + exception, info, + synchronization::{interface::ReadWriteEx, InitStateLock}, +}; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner +where + T: 'static, +{ + next_index: usize, + descriptors: [Option>; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Driver interfaces. pub mod interface { - use crate::bsp; - /// Device Driver functions. pub trait DeviceDriver { + /// Different interrupt controllers might use different types for IRQ number. + type IRQNumberType: super::fmt::Display; + /// Return a compatibility string for identifying the driver. fn compatible(&self) -> &'static str; @@ -26,32 +47,175 @@ pub mod interface { Ok(()) } - /// Called by the kernel to register and enable the device's IRQ handlers, if any. + /// Called by the kernel to register and enable the device's IRQ handler. /// /// Rust's type system will prevent a call to this function unless the calling instance /// itself has static lifetime. - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - Ok(()) + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + panic!( + "Attempt to enable IRQ {} for device {}, but driver does not support this", + irq_number, + self.compatible() + ) + } + } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor +where + T: 'static, +{ + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager +where + T: 'static, +{ + inner: InitStateLock>, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner +where + T: 'static + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], } } +} - /// Device driver management functions. +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + irq_number, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager +where + T: fmt::Display + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: InitStateLock::new(DriverManagerInner::new()), + } + } + + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.write(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.read(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers and their interrupts handlers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Instantiate all drivers. - /// - /// # Safety - /// - /// Must be called before `all_device_drivers`. - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str>; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers_and_irqs(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + + // 3. After all post-init callbacks were done, the interrupt controller should be + // registered and functional. So let drivers register with it now. + self.for_each_descriptor(|descriptor| { + if let Some(irq_number) = &descriptor.irq_number { + if let Err(x) = descriptor + .device_driver + .register_and_enable_irq_handler(irq_number) + { + panic!( + "Error during driver interrupt handler registration: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } - /// Return a slice of references to all `BSP`-instantiated drivers. - fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); bsp::driver::NUM_DRIVERS]; + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); - /// Minimal code needed to bring up the console in QEMU (for testing only). This is often - /// less steps than on real hardware due to QEMU's abstractions. - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self); + i += 1; + }); } } diff --git a/17_kernel_symbols/kernel/src/exception/asynchronous.rs b/17_kernel_symbols/kernel/src/exception/asynchronous.rs index 83ca1aa6..c1f2a27b 100644 --- a/17_kernel_symbols/kernel/src/exception/asynchronous.rs +++ b/17_kernel_symbols/kernel/src/exception/asynchronous.rs @@ -10,7 +10,7 @@ mod arch_asynchronous; mod null_irq_manager; use crate::{bsp, synchronization}; -use core::{fmt, marker::PhantomData}; +use core::marker::PhantomData; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports @@ -24,14 +24,23 @@ pub use arch_asynchronous::{ // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Interrupt number as defined by the BSP. +pub type IRQNumber = bsp::exception::asynchronous::IRQNumber; + /// Interrupt descriptor. #[derive(Copy, Clone)] -pub struct IRQDescriptor { +pub struct IRQHandlerDescriptor +where + T: Copy, +{ + /// The IRQ number. + number: T, + /// Descriptive name. - pub name: &'static str, + name: &'static str, /// Reference to handler trait object. - pub handler: &'static (dyn interface::IRQHandler + Sync), + handler: &'static (dyn interface::IRQHandler + Sync), } /// IRQContext token. @@ -61,17 +70,16 @@ pub mod interface { /// platform's interrupt controller. pub trait IRQManager { /// The IRQ number type depends on the implementation. - type IRQNumberType; + type IRQNumberType: Copy; /// Register a handler. fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: super::IRQDescriptor, + irq_handler_descriptor: super::IRQHandlerDescriptor, ) -> Result<(), &'static str>; /// Enable an interrupt in the controller. - fn enable(&self, irq_number: Self::IRQNumberType); + fn enable(&self, irq_number: &Self::IRQNumberType); /// Handle pending interrupts. /// @@ -91,16 +99,12 @@ pub mod interface { } } -/// A wrapper type for IRQ numbers with integrated range sanity check. -#[derive(Copy, Clone)] -pub struct IRQNumber(usize); - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- static CUR_IRQ_MANAGER: InitStateLock< - &'static (dyn interface::IRQManager + Sync), + &'static (dyn interface::IRQManager + Sync), > = InitStateLock::new(&null_irq_manager::NULL_IRQ_MANAGER); //-------------------------------------------------------------------------------------------------- @@ -108,6 +112,39 @@ static CUR_IRQ_MANAGER: InitStateLock< //-------------------------------------------------------------------------------------------------- use synchronization::{interface::ReadWriteEx, InitStateLock}; +impl IRQHandlerDescriptor +where + T: Copy, +{ + /// Create an instance. + pub const fn new( + number: T, + name: &'static str, + handler: &'static (dyn interface::IRQHandler + Sync), + ) -> Self { + Self { + number, + name, + handler, + } + } + + /// Return the number. + pub const fn number(&self) -> T { + self.number + } + + /// Return the name. + pub const fn name(&self) -> &'static str { + self.name + } + + /// Return the handler. + pub const fn handler(&self) -> &'static (dyn interface::IRQHandler + Sync) { + self.handler + } +} + impl<'irq_context> IRQContext<'irq_context> { /// Creates an IRQContext token. /// @@ -125,29 +162,6 @@ impl<'irq_context> IRQContext<'irq_context> { } } -impl IRQNumber<{ MAX_INCLUSIVE }> { - /// The total number of IRQs this type supports. - pub const NUM_TOTAL: usize = MAX_INCLUSIVE + 1; - - /// Creates a new instance if number <= MAX_INCLUSIVE. - pub const fn new(number: usize) -> Self { - assert!(number <= MAX_INCLUSIVE); - - Self(number) - } - - /// Return the wrapped number. - pub const fn get(self) -> usize { - self.0 - } -} - -impl fmt::Display for IRQNumber<{ MAX_INCLUSIVE }> { - fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { - write!(f, "{}", self.0) - } -} - /// Executes the provided closure while IRQs are masked on the executing core. /// /// While the function temporarily changes the HW state of the executing core, it restores it to the @@ -163,8 +177,7 @@ pub fn exec_with_irq_masked(f: impl FnOnce() -> T) -> T { /// Register a new IRQ manager. pub fn register_irq_manager( - new_manager: &'static (dyn interface::IRQManager - + Sync), + new_manager: &'static (dyn interface::IRQManager + Sync), ) { CUR_IRQ_MANAGER.write(|manager| *manager = new_manager); } @@ -172,6 +185,6 @@ pub fn register_irq_manager( /// Return a reference to the currently registered IRQ manager. /// /// This is the IRQ manager used by the architectural interrupt handling code. -pub fn irq_manager() -> &'static dyn interface::IRQManager { +pub fn irq_manager() -> &'static dyn interface::IRQManager { CUR_IRQ_MANAGER.read(|manager| *manager) } diff --git a/17_kernel_symbols/kernel/src/exception/asynchronous/null_irq_manager.rs b/17_kernel_symbols/kernel/src/exception/asynchronous/null_irq_manager.rs index 560e3ce4..438f9649 100644 --- a/17_kernel_symbols/kernel/src/exception/asynchronous/null_irq_manager.rs +++ b/17_kernel_symbols/kernel/src/exception/asynchronous/null_irq_manager.rs @@ -4,8 +4,7 @@ //! Null IRQ Manager. -use super::{interface, IRQContext, IRQDescriptor}; -use crate::bsp; +use super::{interface, IRQContext, IRQHandlerDescriptor}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -24,17 +23,16 @@ pub static NULL_IRQ_MANAGER: NullIRQManager = NullIRQManager {}; //-------------------------------------------------------------------------------------------------- impl interface::IRQManager for NullIRQManager { - type IRQNumberType = bsp::driver::IRQNumber; + type IRQNumberType = super::IRQNumber; fn register_handler( &self, - _irq_number: Self::IRQNumberType, - _descriptor: IRQDescriptor, + _descriptor: IRQHandlerDescriptor, ) -> Result<(), &'static str> { panic!("No IRQ Manager registered yet"); } - fn enable(&self, _irq_number: Self::IRQNumberType) { + fn enable(&self, _irq_number: &Self::IRQNumberType) { panic!("No IRQ Manager registered yet"); } diff --git a/17_kernel_symbols/kernel/src/lib.rs b/17_kernel_symbols/kernel/src/lib.rs index 975bad2e..22572709 100644 --- a/17_kernel_symbols/kernel/src/lib.rs +++ b/17_kernel_symbols/kernel/src/lib.rs @@ -182,11 +182,9 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); test_main(); diff --git a/17_kernel_symbols/kernel/src/main.rs b/17_kernel_symbols/kernel/src/main.rs index 9aeb4438..65905258 100644 --- a/17_kernel_symbols/kernel/src/main.rs +++ b/17_kernel_symbols/kernel/src/main.rs @@ -13,7 +13,7 @@ #![no_main] #![no_std] -use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; +use libkernel::{bsp, cpu, driver, exception, info, memory, state, time}; /// Early init code. /// @@ -25,27 +25,16 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// - Printing will not work until the respective driver's MMIO is remapped. #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - // Instantiate and init all device drivers. - if let Err(x) = bsp::driver::driver_manager().instantiate_drivers() { - panic!("Error instantiating drivers: {}", x); - } - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - // Let device drivers register and enable their handlers with the interrupt controller. - for i in bsp::driver::driver_manager().all_device_drivers() { - if let Err(msg) = i.register_and_enable_irq_handler() { - warn!("Error registering IRQ handler: {}", msg); - } - } + // Initialize all device drivers. + driver::driver_manager().init_drivers_and_irqs(); bsp::memory::mmu::kernel_add_mapping_records_for_precomputed(); @@ -61,8 +50,6 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { - use driver::interface::DriverManager; - info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); @@ -81,13 +68,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); info!("Registered IRQ handlers:"); exception::asynchronous::irq_manager().print_handler(); diff --git a/17_kernel_symbols/kernel/tests/00_console_sanity.rs b/17_kernel_symbols/kernel/tests/00_console_sanity.rs index 305510ce..2c0225b7 100644 --- a/17_kernel_symbols/kernel/tests/00_console_sanity.rs +++ b/17_kernel_symbols/kernel/tests/00_console_sanity.rs @@ -11,16 +11,15 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{bsp, console, cpu, driver, exception, memory, print}; +use libkernel::{bsp, console, cpu, exception, memory, print}; #[no_mangle] unsafe fn kernel_init() -> ! { use console::console; - use driver::interface::DriverManager; exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/17_kernel_symbols/kernel/tests/01_timer_sanity.rs b/17_kernel_symbols/kernel/tests/01_timer_sanity.rs index 39899332..8188b942 100644 --- a/17_kernel_symbols/kernel/tests/01_timer_sanity.rs +++ b/17_kernel_symbols/kernel/tests/01_timer_sanity.rs @@ -11,16 +11,14 @@ #![test_runner(libkernel::test_runner)] use core::time::Duration; -use libkernel::{bsp, cpu, driver, exception, memory, time}; +use libkernel::{bsp, cpu, exception, memory, time}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff --git a/17_kernel_symbols/kernel/tests/02_exception_sync_page_fault.rs b/17_kernel_symbols/kernel/tests/02_exception_sync_page_fault.rs index a6d15b69..fab44c8f 100644 --- a/17_kernel_symbols/kernel/tests/02_exception_sync_page_fault.rs +++ b/17_kernel_symbols/kernel/tests/02_exception_sync_page_fault.rs @@ -17,15 +17,13 @@ /// or indirectly. mod panic_exit_success; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); diff --git a/17_kernel_symbols/kernel/tests/03_exception_restore_sanity.rs b/17_kernel_symbols/kernel/tests/03_exception_restore_sanity.rs index 47dd2714..f176c6a6 100644 --- a/17_kernel_symbols/kernel/tests/03_exception_restore_sanity.rs +++ b/17_kernel_symbols/kernel/tests/03_exception_restore_sanity.rs @@ -12,7 +12,7 @@ mod panic_wait_forever; use core::arch::asm; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[inline(never)] fn nested_system_call() { @@ -30,11 +30,9 @@ fn nested_system_call() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing exception restore"); diff --git a/17_kernel_symbols/kernel/tests/04_exception_irq_sanity.rs b/17_kernel_symbols/kernel/tests/04_exception_irq_sanity.rs index e93a4caf..e6f94c91 100644 --- a/17_kernel_symbols/kernel/tests/04_exception_irq_sanity.rs +++ b/17_kernel_symbols/kernel/tests/04_exception_irq_sanity.rs @@ -10,15 +10,13 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -use libkernel::{bsp, cpu, driver, exception, memory}; +use libkernel::{bsp, cpu, exception, memory}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); exception::handling_init(); exception::asynchronous::local_irq_unmask(); diff --git a/18_backtrace/Makefile b/18_backtrace/Makefile index 1dda95a4..789ed5cd 100644 --- a/18_backtrace/Makefile +++ b/18_backtrace/Makefile @@ -336,7 +336,7 @@ test_boot: $(KERNEL_BIN) ## Helpers for unit and integration test targets ##------------------------------------------------------------------------------ define KERNEL_TEST_RUNNER - #!/usr/bin/env bash +#!/usr/bin/env bash # The cargo test runner seems to change into the crate under test's directory. Therefore, ensure # this script executes from the root. diff --git a/18_backtrace/README.md b/18_backtrace/README.md index 77222e3a..021b8435 100644 --- a/18_backtrace/README.md +++ b/18_backtrace/README.md @@ -989,6 +989,19 @@ diff -uNr 17_kernel_symbols/kernel/src/panic_wait.rs 18_backtrace/kernel/src/pan _panic_exit() +diff -uNr 17_kernel_symbols/kernel/src/state.rs 18_backtrace/kernel/src/state.rs +--- 17_kernel_symbols/kernel/src/state.rs ++++ 18_backtrace/kernel/src/state.rs +@@ -52,7 +52,7 @@ + const SINGLE_CORE_MAIN: u8 = 1; + const MULTI_CORE_MAIN: u8 = 2; + +- /// Create a new instance. ++ /// Create an instance. + pub const fn new() -> Self { + Self(AtomicU8::new(Self::INIT)) + } + diff -uNr 17_kernel_symbols/kernel/tests/05_backtrace_sanity.rb 18_backtrace/kernel/tests/05_backtrace_sanity.rb --- 17_kernel_symbols/kernel/tests/05_backtrace_sanity.rb +++ 18_backtrace/kernel/tests/05_backtrace_sanity.rb @@ -1036,7 +1049,7 @@ diff -uNr 17_kernel_symbols/kernel/tests/05_backtrace_sanity.rb 18_backtrace/ker diff -uNr 17_kernel_symbols/kernel/tests/05_backtrace_sanity.rs 18_backtrace/kernel/tests/05_backtrace_sanity.rs --- 17_kernel_symbols/kernel/tests/05_backtrace_sanity.rs +++ 18_backtrace/kernel/tests/05_backtrace_sanity.rs -@@ -0,0 +1,33 @@ +@@ -0,0 +1,31 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter @@ -1050,7 +1063,7 @@ diff -uNr 17_kernel_symbols/kernel/tests/05_backtrace_sanity.rs 18_backtrace/ker +/// Console tests should time out on the I/O harness in case of panic. +mod panic_wait_forever; + -+use libkernel::{bsp, cpu, driver, exception, memory}; ++use libkernel::{bsp, cpu, exception, memory}; + +#[inline(never)] +fn nested() { @@ -1059,11 +1072,9 @@ diff -uNr 17_kernel_symbols/kernel/tests/05_backtrace_sanity.rs 18_backtrace/ker + +#[no_mangle] +unsafe fn kernel_init() -> ! { -+ use driver::interface::DriverManager; -+ + exception::handling_init(); + memory::init(); -+ bsp::driver::driver_manager().qemu_bring_up_console(); ++ bsp::driver::qemu_bring_up_console(); + + nested(); + @@ -1105,7 +1116,7 @@ diff -uNr 17_kernel_symbols/kernel/tests/06_backtrace_invalid_frame.rb 18_backtr diff -uNr 17_kernel_symbols/kernel/tests/06_backtrace_invalid_frame.rs 18_backtrace/kernel/tests/06_backtrace_invalid_frame.rs --- 17_kernel_symbols/kernel/tests/06_backtrace_invalid_frame.rs +++ 18_backtrace/kernel/tests/06_backtrace_invalid_frame.rs -@@ -0,0 +1,35 @@ +@@ -0,0 +1,33 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter @@ -1119,7 +1130,7 @@ diff -uNr 17_kernel_symbols/kernel/tests/06_backtrace_invalid_frame.rs 18_backtr +/// Console tests should time out on the I/O harness in case of panic. +mod panic_wait_forever; + -+use libkernel::{backtrace, bsp, cpu, driver, exception, memory}; ++use libkernel::{backtrace, bsp, cpu, exception, memory}; + +#[inline(never)] +fn nested() { @@ -1130,11 +1141,9 @@ diff -uNr 17_kernel_symbols/kernel/tests/06_backtrace_invalid_frame.rs 18_backtr + +#[no_mangle] +unsafe fn kernel_init() -> ! { -+ use driver::interface::DriverManager; -+ + exception::handling_init(); + memory::init(); -+ bsp::driver::driver_manager().qemu_bring_up_console(); ++ bsp::driver::qemu_bring_up_console(); + + nested(); + @@ -1175,7 +1184,7 @@ diff -uNr 17_kernel_symbols/kernel/tests/07_backtrace_invalid_link.rb 18_backtra diff -uNr 17_kernel_symbols/kernel/tests/07_backtrace_invalid_link.rs 18_backtrace/kernel/tests/07_backtrace_invalid_link.rs --- 17_kernel_symbols/kernel/tests/07_backtrace_invalid_link.rs +++ 18_backtrace/kernel/tests/07_backtrace_invalid_link.rs -@@ -0,0 +1,40 @@ +@@ -0,0 +1,38 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter @@ -1189,7 +1198,7 @@ diff -uNr 17_kernel_symbols/kernel/tests/07_backtrace_invalid_link.rs 18_backtra +/// Console tests should time out on the I/O harness in case of panic. +mod panic_wait_forever; + -+use libkernel::{backtrace, bsp, cpu, driver, exception, memory}; ++use libkernel::{backtrace, bsp, cpu, exception, memory}; + +#[inline(never)] +fn nested_2() -> &'static str { @@ -1205,11 +1214,9 @@ diff -uNr 17_kernel_symbols/kernel/tests/07_backtrace_invalid_link.rs 18_backtra + +#[no_mangle] +unsafe fn kernel_init() -> ! { -+ use driver::interface::DriverManager; -+ + exception::handling_init(); + memory::init(); -+ bsp::driver::driver_manager().qemu_bring_up_console(); ++ bsp::driver::qemu_bring_up_console(); + + nested_1(); + diff --git a/18_backtrace/kernel/src/bsp/device_driver/arm/gicv2.rs b/18_backtrace/kernel/src/bsp/device_driver/arm/gicv2.rs index ec65f1b2..3cc35b5e 100644 --- a/18_backtrace/kernel/src/bsp/device_driver/arm/gicv2.rs +++ b/18_backtrace/kernel/src/bsp/device_driver/arm/gicv2.rs @@ -80,7 +80,8 @@ mod gicc; mod gicd; use crate::{ - bsp, cpu, driver, exception, + bsp::{self, device_driver::common::BoundedUsize}, + cpu, driver, exception, memory::{Address, Virtual}, synchronization, synchronization::InitStateLock, @@ -90,14 +91,15 @@ use crate::{ // Private Definitions //-------------------------------------------------------------------------------------------------- -type HandlerTable = [Option; IRQNumber::NUM_TOTAL]; +type HandlerTable = [Option>; + IRQNumber::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. -pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER }>; +pub type IRQNumber = BoundedUsize<{ GICv2::MAX_IRQ_NUMBER }>; /// Representation of the GIC. pub struct GICv2 { @@ -109,9 +111,6 @@ pub struct GICv2 { /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. handler_table: InitStateLock, - - /// Callback to be invoked after successful init. - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -131,13 +130,11 @@ impl GICv2 { pub const unsafe fn new( gicd_mmio_start_addr: Address, gicc_mmio_start_addr: Address, - post_init_callback: fn(), ) -> Self { Self { gicd: gicd::GICD::new(gicd_mmio_start_addr), gicc: gicc::GICC::new(gicc_mmio_start_addr), - handler_table: InitStateLock::new([None; IRQNumber::NUM_TOTAL]), - post_init_callback, + handler_table: InitStateLock::new([None; IRQNumber::MAX_INCLUSIVE + 1]), } } } @@ -148,6 +145,8 @@ impl GICv2 { use synchronization::interface::ReadWriteEx; impl driver::interface::DeviceDriver for GICv2 { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } @@ -160,8 +159,6 @@ impl driver::interface::DeviceDriver for GICv2 { self.gicc.priority_accept_all(); self.gicc.enable(); - (self.post_init_callback)(); - Ok(()) } } @@ -171,23 +168,22 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq_number.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq_number: Self::IRQNumberType) { + fn enable(&self, irq_number: &Self::IRQNumberType) { self.gicd.enable(irq_number); } @@ -210,7 +206,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } }); @@ -227,7 +223,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { self.handler_table.read(|table| { for (i, opt) in table.iter().skip(32).enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i + 32, handler.name); + info!(" {: >3}. {}", i + 32, handler.name()); } } }); diff --git a/18_backtrace/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs b/18_backtrace/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs index a9c97a8e..8aebcf2b 100644 --- a/18_backtrace/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/18_backtrace/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -172,7 +172,7 @@ impl GICD { } /// Enable an interrupt. - pub fn enable(&self, irq_num: super::IRQNumber) { + pub fn enable(&self, irq_num: &super::IRQNumber) { let irq_num = irq_num.get(); // Each bit in the u32 enable register corresponds to one IRQ number. Shift right by 5 diff --git a/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 2281e66a..fb61a651 100644 --- a/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -7,6 +7,7 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, + exception::asynchronous::IRQNumber, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -122,7 +123,6 @@ struct GPIOInner { /// Representation of the GPIO HW. pub struct GPIO { inner: IRQSafeNullLock, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -202,10 +202,9 @@ impl GPIO { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new(mmio_start_addr: Address, post_init_callback: fn()) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(GPIOInner::new(mmio_start_addr)), - post_init_callback, } } @@ -221,13 +220,9 @@ impl GPIO { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for GPIO { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { - (self.post_init_callback)(); - - Ok(()) - } } diff --git a/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs index b4dd530d..c93a9fa1 100644 --- a/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ b/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -7,9 +7,12 @@ mod peripheral_ic; use crate::{ - driver, exception, + bsp::device_driver::common::BoundedUsize, + driver, + exception::{self, asynchronous::IRQHandlerDescriptor}, memory::{Address, Virtual}, }; +use core::fmt; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -24,10 +27,8 @@ struct PendingIRQs { // Public Definitions //-------------------------------------------------------------------------------------------------- -pub type LocalIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; -pub type PeripheralIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; +pub type LocalIRQ = BoundedUsize<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; +pub type PeripheralIRQ = BoundedUsize<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. #[derive(Copy, Clone)] @@ -40,7 +41,6 @@ pub enum IRQNumber { /// Representation of the Interrupt Controller. pub struct InterruptController { periph: peripheral_ic::PeripheralIC, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -71,6 +71,15 @@ impl Iterator for PendingIRQs { // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for IRQNumber { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + match self { + Self::Local(number) => write!(f, "Local({})", number), + Self::Peripheral(number) => write!(f, "Peripheral({})", number), + } + } +} + impl InterruptController { // Restrict to 3 for now. This makes future code for local_ic.rs more straight forward. const MAX_LOCAL_IRQ_NUMBER: usize = 3; @@ -83,13 +92,9 @@ impl InterruptController { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new( - periph_mmio_start_addr: Address, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(periph_mmio_start_addr: Address) -> Self { Self { periph: peripheral_ic::PeripheralIC::new(periph_mmio_start_addr), - post_init_callback, } } } @@ -99,15 +104,11 @@ impl InterruptController { //------------------------------------------------------------------------------ impl driver::interface::DeviceDriver for InterruptController { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { - (self.post_init_callback)(); - - Ok(()) - } } impl exception::asynchronous::interface::IRQManager for InterruptController { @@ -115,16 +116,23 @@ impl exception::asynchronous::interface::IRQManager for InterruptController { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { - match irq { + match irq_handler_descriptor.number() { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), - IRQNumber::Peripheral(pirq) => self.periph.register_handler(pirq, descriptor), + IRQNumber::Peripheral(pirq) => { + let periph_descriptor = IRQHandlerDescriptor::new( + pirq, + irq_handler_descriptor.name(), + irq_handler_descriptor.handler(), + ); + + self.periph.register_handler(periph_descriptor) + } } } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { match irq { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), IRQNumber::Peripheral(pirq) => self.periph.enable(pirq), diff --git a/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 06802d27..0a20bd87 100644 --- a/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -52,7 +52,8 @@ type WriteOnlyRegisters = MMIODerefWrapper; /// Abstraction for the ReadOnly parts of the associated MMIO registers. type ReadOnlyRegisters = MMIODerefWrapper; -type HandlerTable = [Option; PeripheralIRQ::NUM_TOTAL]; +type HandlerTable = [Option>; + PeripheralIRQ::MAX_INCLUSIVE + 1]; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -84,7 +85,7 @@ impl PeripheralIC { Self { wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(mmio_start_addr)), ro_registers: ReadOnlyRegisters::new(mmio_start_addr), - handler_table: InitStateLock::new([None; PeripheralIRQ::NUM_TOTAL]), + handler_table: InitStateLock::new([None; PeripheralIRQ::MAX_INCLUSIVE + 1]), } } @@ -107,23 +108,22 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { self.wo_registers.lock(|regs| { let enable_reg = if irq.get() <= 31 { ®s.ENABLE_1 @@ -149,7 +149,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } } @@ -164,7 +164,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { self.handler_table.read(|table| { for (i, opt) in table.iter().enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i, handler.name); + info!(" {: >3}. {}", i, handler.name()); } } }); diff --git a/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index f67d70df..0ee7feb7 100644 --- a/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -10,9 +10,9 @@ //! - use crate::{ - bsp, bsp::device_driver::common::MMIODerefWrapper, - console, cpu, driver, exception, + console, cpu, driver, + exception::{self, asynchronous::IRQNumber}, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -233,8 +233,6 @@ struct PL011UartInner { /// Representation of the UART. pub struct PL011Uart { inner: IRQSafeNullLock, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -398,16 +396,9 @@ impl PL011Uart { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - /// - The user must ensure to provide correct IRQ numbers. - pub const unsafe fn new( - mmio_start_addr: Address, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(PL011UartInner::new(mmio_start_addr)), - irq_number, - post_init_callback, } } } @@ -418,27 +409,28 @@ impl PL011Uart { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for PL011Uart { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } unsafe fn init(&self) -> Result<(), &'static str> { self.inner.lock(|inner| inner.init()); - (self.post_init_callback)(); Ok(()) } - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - use exception::asynchronous::{irq_manager, IRQDescriptor}; + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + use exception::asynchronous::{irq_manager, IRQHandlerDescriptor}; - let descriptor = IRQDescriptor { - name: Self::COMPATIBLE, - handler: self, - }; + let descriptor = IRQHandlerDescriptor::new(*irq_number, Self::COMPATIBLE, self); - irq_manager().register_handler(self.irq_number, descriptor)?; - irq_manager().enable(self.irq_number); + irq_manager().register_handler(descriptor)?; + irq_manager().enable(irq_number); Ok(()) } diff --git a/18_backtrace/kernel/src/bsp/device_driver/common.rs b/18_backtrace/kernel/src/bsp/device_driver/common.rs index 69f038d4..ca7aeb76 100644 --- a/18_backtrace/kernel/src/bsp/device_driver/common.rs +++ b/18_backtrace/kernel/src/bsp/device_driver/common.rs @@ -5,7 +5,7 @@ //! Common device driver code. use crate::memory::{Address, Virtual}; -use core::{marker::PhantomData, ops}; +use core::{fmt, marker::PhantomData, ops}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -16,6 +16,10 @@ pub struct MMIODerefWrapper { phantom: PhantomData T>, } +/// A wrapper type for usize with integrated range bound check. +#[derive(Copy, Clone)] +pub struct BoundedUsize(usize); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -37,3 +41,25 @@ impl ops::Deref for MMIODerefWrapper { unsafe { &*(self.start_addr.as_usize() as *const _) } } } + +impl BoundedUsize<{ MAX_INCLUSIVE }> { + pub const MAX_INCLUSIVE: usize = MAX_INCLUSIVE; + + /// Creates a new instance if number <= MAX_INCLUSIVE. + pub const fn new(number: usize) -> Self { + assert!(number <= MAX_INCLUSIVE); + + Self(number) + } + + /// Return the wrapped number. + pub const fn get(self) -> usize { + self.0 + } +} + +impl fmt::Display for BoundedUsize<{ MAX_INCLUSIVE }> { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + write!(f, "{}", self.0) + } +} diff --git a/18_backtrace/kernel/src/bsp/raspberrypi/driver.rs b/18_backtrace/kernel/src/bsp/raspberrypi/driver.rs index e1db4a00..ca3435aa 100644 --- a/18_backtrace/kernel/src/bsp/raspberrypi/driver.rs +++ b/18_backtrace/kernel/src/bsp/raspberrypi/driver.rs @@ -7,34 +7,16 @@ use super::{exception, memory::map::mmio}; use crate::{ bsp::device_driver, - console, driver, exception as generic_exception, memory, + console, driver as generic_driver, + exception::{self as generic_exception}, + memory, memory::mmu::MMIODescriptor, - synchronization::{interface::ReadWriteEx, InitStateLock}, }; use core::{ mem::MaybeUninit, sync::atomic::{AtomicBool, Ordering}, }; -pub use device_driver::IRQNumber; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: InitStateLock<[Option<&'static (dyn DeviceDriver + Sync)>; NUM_DRIVERS]>, - init_done: AtomicBool, -} - -//-------------------------------------------------------------------------------------------------- -// Public Definitions -//-------------------------------------------------------------------------------------------------- - -/// The number of active drivers provided by this BSP. -pub const NUM_DRIVERS: usize = 3; - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- @@ -49,150 +31,154 @@ static mut INTERRUPT_CONTROLLER: MaybeUninit #[cfg(feature = "bsp_rpi4")] static mut INTERRUPT_CONTROLLER: MaybeUninit = MaybeUninit::uninit(); -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: InitStateLock::new([None; NUM_DRIVERS]), - init_done: AtomicBool::new(false), -}; - //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -impl BSPDriverManager { - unsafe fn instantiate_uart(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_uart() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn uart_post_init() { - console::register_console(unsafe { PL011_UART.assume_init_ref() }); - } + PL011_UART.write(device_driver::PL011Uart::new(virt_addr)); - PL011_UART.write(device_driver::PL011Uart::new( - virt_addr, - exception::asynchronous::irq_map::PL011_UART, - uart_post_init, - )); + Ok(()) +} - Ok(()) - } +/// This must be called only after successful init of the UART driver. +unsafe fn post_init_uart() -> Result<(), &'static str> { + console::register_console(PL011_UART.assume_init_ref()); - unsafe fn instantiate_gpio(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; + Ok(()) +} - // This is safe to do, because it is only called from the init'ed instance itself. - fn gpio_post_init() { - unsafe { GPIO.assume_init_ref().map_pl011_uart() }; - } +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_gpio() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; - GPIO.write(device_driver::GPIO::new(virt_addr, gpio_post_init)); + GPIO.write(device_driver::GPIO::new(virt_addr)); - Ok(()) - } + Ok(()) +} - #[cfg(feature = "bsp_rpi3")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let periph_mmio_descriptor = - MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); - let periph_virt_addr = memory::mmu::kernel_map_mmio( - device_driver::InterruptController::COMPATIBLE, - &periph_mmio_descriptor, - )?; - - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } - - INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new( - periph_virt_addr, - interrupt_controller_post_init, - )); - - Ok(()) - } +/// This must be called only after successful init of the GPIO driver. +unsafe fn post_init_gpio() -> Result<(), &'static str> { + GPIO.assume_init_ref().map_pl011_uart(); + Ok(()) +} + +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi3")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let periph_mmio_descriptor = + MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); + let periph_virt_addr = memory::mmu::kernel_map_mmio( + device_driver::InterruptController::COMPATIBLE, + &periph_mmio_descriptor, + )?; + + INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new(periph_virt_addr)); - #[cfg(feature = "bsp_rpi4")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); - let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; + Ok(()) +} - let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); - let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi4")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); + let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } + let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); + let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; - INTERRUPT_CONTROLLER.write(device_driver::GICv2::new( - gicd_virt_addr, - gicc_virt_addr, - interrupt_controller_post_init, - )); + INTERRUPT_CONTROLLER.write(device_driver::GICv2::new(gicd_virt_addr, gicc_virt_addr)); - Ok(()) - } + Ok(()) +} - unsafe fn register_drivers(&self) { - self.device_drivers.write(|drivers| { - drivers[0] = Some(PL011_UART.assume_init_ref()); - drivers[1] = Some(GPIO.assume_init_ref()); - drivers[2] = Some(INTERRUPT_CONTROLLER.assume_init_ref()); - }); - } +/// This must be called only after successful init of the interrupt controller driver. +unsafe fn post_init_interrupt_controller() -> Result<(), &'static str> { + generic_exception::asynchronous::register_irq_manager(INTERRUPT_CONTROLLER.assume_init_ref()); + + Ok(()) } -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_uart() -> Result<(), &'static str> { + instantiate_uart()?; + + let uart_descriptor = generic_driver::DeviceDriverDescriptor::new( + PL011_UART.assume_init_ref(), + Some(post_init_uart), + Some(exception::asynchronous::irq_map::PL011_UART), + ); + generic_driver::driver_manager().register_driver(uart_descriptor); + + Ok(()) +} + +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_gpio() -> Result<(), &'static str> { + instantiate_gpio()?; + + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new( + GPIO.assume_init_ref(), + Some(post_init_gpio), + None, + ); + generic_driver::driver_manager().register_driver(gpio_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_interrupt_controller() -> Result<(), &'static str> { + instantiate_interrupt_controller()?; -impl driver::interface::DriverManager for BSPDriverManager { - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str> { - if self.init_done.load(Ordering::Relaxed) { - return Err("Drivers already instantiated"); - } + let interrupt_controller_descriptor = generic_driver::DeviceDriverDescriptor::new( + INTERRUPT_CONTROLLER.assume_init_ref(), + Some(post_init_interrupt_controller), + None, + ); + generic_driver::driver_manager().register_driver(interrupt_controller_descriptor); - self.instantiate_uart()?; - self.instantiate_gpio()?; - self.instantiate_interrupt_controller()?; + Ok(()) +} - self.register_drivers(); +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - self.init_done.store(true, Ordering::Relaxed); - Ok(()) +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } - fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); NUM_DRIVERS] { - self.device_drivers - .read(|drivers| drivers.map(|drivers| drivers.unwrap())) - } + driver_uart()?; + driver_gpio()?; + driver_interrupt_controller()?; - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self) { - use crate::cpu; + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) +} - unsafe { - self.instantiate_uart() - .unwrap_or_else(|_| cpu::qemu_exit_failure()); - console::register_console(PL011_UART.assume_init_ref()); - }; - } +/// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps +/// than on real hardware due to QEMU's abstractions. +#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() { + use crate::cpu; + + unsafe { + instantiate_uart().unwrap_or_else(|_| cpu::qemu_exit_failure()); + console::register_console(PL011_UART.assume_init_ref()); + }; } diff --git a/18_backtrace/kernel/src/bsp/raspberrypi/exception/asynchronous.rs b/18_backtrace/kernel/src/bsp/raspberrypi/exception/asynchronous.rs index ab20d86d..06a67558 100644 --- a/18_backtrace/kernel/src/bsp/raspberrypi/exception/asynchronous.rs +++ b/18_backtrace/kernel/src/bsp/raspberrypi/exception/asynchronous.rs @@ -10,6 +10,9 @@ use crate::bsp; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Export for reuse in generic asynchronous.rs. +pub use bsp::device_driver::IRQNumber; + #[cfg(feature = "bsp_rpi3")] pub(in crate::bsp) mod irq_map { use super::bsp::device_driver::{IRQNumber, PeripheralIRQ}; diff --git a/18_backtrace/kernel/src/driver.rs b/18_backtrace/kernel/src/driver.rs index 98197fef..18066c31 100644 --- a/18_backtrace/kernel/src/driver.rs +++ b/18_backtrace/kernel/src/driver.rs @@ -4,16 +4,37 @@ //! Driver support. +use crate::{ + exception, info, + synchronization::{interface::ReadWriteEx, InitStateLock}, +}; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner +where + T: 'static, +{ + next_index: usize, + descriptors: [Option>; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Driver interfaces. pub mod interface { - use crate::bsp; - /// Device Driver functions. pub trait DeviceDriver { + /// Different interrupt controllers might use different types for IRQ number. + type IRQNumberType: super::fmt::Display; + /// Return a compatibility string for identifying the driver. fn compatible(&self) -> &'static str; @@ -26,32 +47,175 @@ pub mod interface { Ok(()) } - /// Called by the kernel to register and enable the device's IRQ handlers, if any. + /// Called by the kernel to register and enable the device's IRQ handler. /// /// Rust's type system will prevent a call to this function unless the calling instance /// itself has static lifetime. - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - Ok(()) + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + panic!( + "Attempt to enable IRQ {} for device {}, but driver does not support this", + irq_number, + self.compatible() + ) + } + } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor +where + T: 'static, +{ + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager +where + T: 'static, +{ + inner: InitStateLock>, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner +where + T: 'static + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], } } +} - /// Device driver management functions. +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + irq_number, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager +where + T: fmt::Display + Copy, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: InitStateLock::new(DriverManagerInner::new()), + } + } + + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.write(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.read(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers and their interrupts handlers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Instantiate all drivers. - /// - /// # Safety - /// - /// Must be called before `all_device_drivers`. - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str>; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers_and_irqs(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + + // 3. After all post-init callbacks were done, the interrupt controller should be + // registered and functional. So let drivers register with it now. + self.for_each_descriptor(|descriptor| { + if let Some(irq_number) = &descriptor.irq_number { + if let Err(x) = descriptor + .device_driver + .register_and_enable_irq_handler(irq_number) + { + panic!( + "Error during driver interrupt handler registration: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); + } - /// Return a slice of references to all `BSP`-instantiated drivers. - fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); bsp::driver::NUM_DRIVERS]; + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + let mut i: usize = 1; + self.for_each_descriptor(|descriptor| { + info!(" {}. {}", i, descriptor.device_driver.compatible()); - /// Minimal code needed to bring up the console in QEMU (for testing only). This is often - /// less steps than on real hardware due to QEMU's abstractions. - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self); + i += 1; + }); } } diff --git a/18_backtrace/kernel/src/exception/asynchronous.rs b/18_backtrace/kernel/src/exception/asynchronous.rs index 83ca1aa6..c1f2a27b 100644 --- a/18_backtrace/kernel/src/exception/asynchronous.rs +++ b/18_backtrace/kernel/src/exception/asynchronous.rs @@ -10,7 +10,7 @@ mod arch_asynchronous; mod null_irq_manager; use crate::{bsp, synchronization}; -use core::{fmt, marker::PhantomData}; +use core::marker::PhantomData; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports @@ -24,14 +24,23 @@ pub use arch_asynchronous::{ // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Interrupt number as defined by the BSP. +pub type IRQNumber = bsp::exception::asynchronous::IRQNumber; + /// Interrupt descriptor. #[derive(Copy, Clone)] -pub struct IRQDescriptor { +pub struct IRQHandlerDescriptor +where + T: Copy, +{ + /// The IRQ number. + number: T, + /// Descriptive name. - pub name: &'static str, + name: &'static str, /// Reference to handler trait object. - pub handler: &'static (dyn interface::IRQHandler + Sync), + handler: &'static (dyn interface::IRQHandler + Sync), } /// IRQContext token. @@ -61,17 +70,16 @@ pub mod interface { /// platform's interrupt controller. pub trait IRQManager { /// The IRQ number type depends on the implementation. - type IRQNumberType; + type IRQNumberType: Copy; /// Register a handler. fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: super::IRQDescriptor, + irq_handler_descriptor: super::IRQHandlerDescriptor, ) -> Result<(), &'static str>; /// Enable an interrupt in the controller. - fn enable(&self, irq_number: Self::IRQNumberType); + fn enable(&self, irq_number: &Self::IRQNumberType); /// Handle pending interrupts. /// @@ -91,16 +99,12 @@ pub mod interface { } } -/// A wrapper type for IRQ numbers with integrated range sanity check. -#[derive(Copy, Clone)] -pub struct IRQNumber(usize); - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- static CUR_IRQ_MANAGER: InitStateLock< - &'static (dyn interface::IRQManager + Sync), + &'static (dyn interface::IRQManager + Sync), > = InitStateLock::new(&null_irq_manager::NULL_IRQ_MANAGER); //-------------------------------------------------------------------------------------------------- @@ -108,6 +112,39 @@ static CUR_IRQ_MANAGER: InitStateLock< //-------------------------------------------------------------------------------------------------- use synchronization::{interface::ReadWriteEx, InitStateLock}; +impl IRQHandlerDescriptor +where + T: Copy, +{ + /// Create an instance. + pub const fn new( + number: T, + name: &'static str, + handler: &'static (dyn interface::IRQHandler + Sync), + ) -> Self { + Self { + number, + name, + handler, + } + } + + /// Return the number. + pub const fn number(&self) -> T { + self.number + } + + /// Return the name. + pub const fn name(&self) -> &'static str { + self.name + } + + /// Return the handler. + pub const fn handler(&self) -> &'static (dyn interface::IRQHandler + Sync) { + self.handler + } +} + impl<'irq_context> IRQContext<'irq_context> { /// Creates an IRQContext token. /// @@ -125,29 +162,6 @@ impl<'irq_context> IRQContext<'irq_context> { } } -impl IRQNumber<{ MAX_INCLUSIVE }> { - /// The total number of IRQs this type supports. - pub const NUM_TOTAL: usize = MAX_INCLUSIVE + 1; - - /// Creates a new instance if number <= MAX_INCLUSIVE. - pub const fn new(number: usize) -> Self { - assert!(number <= MAX_INCLUSIVE); - - Self(number) - } - - /// Return the wrapped number. - pub const fn get(self) -> usize { - self.0 - } -} - -impl fmt::Display for IRQNumber<{ MAX_INCLUSIVE }> { - fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { - write!(f, "{}", self.0) - } -} - /// Executes the provided closure while IRQs are masked on the executing core. /// /// While the function temporarily changes the HW state of the executing core, it restores it to the @@ -163,8 +177,7 @@ pub fn exec_with_irq_masked(f: impl FnOnce() -> T) -> T { /// Register a new IRQ manager. pub fn register_irq_manager( - new_manager: &'static (dyn interface::IRQManager - + Sync), + new_manager: &'static (dyn interface::IRQManager + Sync), ) { CUR_IRQ_MANAGER.write(|manager| *manager = new_manager); } @@ -172,6 +185,6 @@ pub fn register_irq_manager( /// Return a reference to the currently registered IRQ manager. /// /// This is the IRQ manager used by the architectural interrupt handling code. -pub fn irq_manager() -> &'static dyn interface::IRQManager { +pub fn irq_manager() -> &'static dyn interface::IRQManager { CUR_IRQ_MANAGER.read(|manager| *manager) } diff --git a/18_backtrace/kernel/src/exception/asynchronous/null_irq_manager.rs b/18_backtrace/kernel/src/exception/asynchronous/null_irq_manager.rs index 560e3ce4..438f9649 100644 --- a/18_backtrace/kernel/src/exception/asynchronous/null_irq_manager.rs +++ b/18_backtrace/kernel/src/exception/asynchronous/null_irq_manager.rs @@ -4,8 +4,7 @@ //! Null IRQ Manager. -use super::{interface, IRQContext, IRQDescriptor}; -use crate::bsp; +use super::{interface, IRQContext, IRQHandlerDescriptor}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -24,17 +23,16 @@ pub static NULL_IRQ_MANAGER: NullIRQManager = NullIRQManager {}; //-------------------------------------------------------------------------------------------------- impl interface::IRQManager for NullIRQManager { - type IRQNumberType = bsp::driver::IRQNumber; + type IRQNumberType = super::IRQNumber; fn register_handler( &self, - _irq_number: Self::IRQNumberType, - _descriptor: IRQDescriptor, + _descriptor: IRQHandlerDescriptor, ) -> Result<(), &'static str> { panic!("No IRQ Manager registered yet"); } - fn enable(&self, _irq_number: Self::IRQNumberType) { + fn enable(&self, _irq_number: &Self::IRQNumberType) { panic!("No IRQ Manager registered yet"); } diff --git a/18_backtrace/kernel/src/lib.rs b/18_backtrace/kernel/src/lib.rs index 2545b20b..5b79ebfe 100644 --- a/18_backtrace/kernel/src/lib.rs +++ b/18_backtrace/kernel/src/lib.rs @@ -183,11 +183,9 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); test_main(); diff --git a/18_backtrace/kernel/src/main.rs b/18_backtrace/kernel/src/main.rs index 9aeb4438..65905258 100644 --- a/18_backtrace/kernel/src/main.rs +++ b/18_backtrace/kernel/src/main.rs @@ -13,7 +13,7 @@ #![no_main] #![no_std] -use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; +use libkernel::{bsp, cpu, driver, exception, info, memory, state, time}; /// Early init code. /// @@ -25,27 +25,16 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// - Printing will not work until the respective driver's MMIO is remapped. #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - // Instantiate and init all device drivers. - if let Err(x) = bsp::driver::driver_manager().instantiate_drivers() { - panic!("Error instantiating drivers: {}", x); - } - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - // Let device drivers register and enable their handlers with the interrupt controller. - for i in bsp::driver::driver_manager().all_device_drivers() { - if let Err(msg) = i.register_and_enable_irq_handler() { - warn!("Error registering IRQ handler: {}", msg); - } - } + // Initialize all device drivers. + driver::driver_manager().init_drivers_and_irqs(); bsp::memory::mmu::kernel_add_mapping_records_for_precomputed(); @@ -61,8 +50,6 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { - use driver::interface::DriverManager; - info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); @@ -81,13 +68,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); info!("Registered IRQ handlers:"); exception::asynchronous::irq_manager().print_handler(); diff --git a/18_backtrace/kernel/src/state.rs b/18_backtrace/kernel/src/state.rs index 0af3688c..6a261b34 100644 --- a/18_backtrace/kernel/src/state.rs +++ b/18_backtrace/kernel/src/state.rs @@ -52,7 +52,7 @@ impl StateManager { const SINGLE_CORE_MAIN: u8 = 1; const MULTI_CORE_MAIN: u8 = 2; - /// Create a new instance. + /// Create an instance. pub const fn new() -> Self { Self(AtomicU8::new(Self::INIT)) } diff --git a/18_backtrace/kernel/tests/00_console_sanity.rs b/18_backtrace/kernel/tests/00_console_sanity.rs index 305510ce..2c0225b7 100644 --- a/18_backtrace/kernel/tests/00_console_sanity.rs +++ b/18_backtrace/kernel/tests/00_console_sanity.rs @@ -11,16 +11,15 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{bsp, console, cpu, driver, exception, memory, print}; +use libkernel::{bsp, console, cpu, exception, memory, print}; #[no_mangle] unsafe fn kernel_init() -> ! { use console::console; - use driver::interface::DriverManager; exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/18_backtrace/kernel/tests/01_timer_sanity.rs b/18_backtrace/kernel/tests/01_timer_sanity.rs index 39899332..8188b942 100644 --- a/18_backtrace/kernel/tests/01_timer_sanity.rs +++ b/18_backtrace/kernel/tests/01_timer_sanity.rs @@ -11,16 +11,14 @@ #![test_runner(libkernel::test_runner)] use core::time::Duration; -use libkernel::{bsp, cpu, driver, exception, memory, time}; +use libkernel::{bsp, cpu, exception, memory, time}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff --git a/18_backtrace/kernel/tests/02_exception_sync_page_fault.rs b/18_backtrace/kernel/tests/02_exception_sync_page_fault.rs index a6d15b69..fab44c8f 100644 --- a/18_backtrace/kernel/tests/02_exception_sync_page_fault.rs +++ b/18_backtrace/kernel/tests/02_exception_sync_page_fault.rs @@ -17,15 +17,13 @@ /// or indirectly. mod panic_exit_success; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); diff --git a/18_backtrace/kernel/tests/03_exception_restore_sanity.rs b/18_backtrace/kernel/tests/03_exception_restore_sanity.rs index 47dd2714..f176c6a6 100644 --- a/18_backtrace/kernel/tests/03_exception_restore_sanity.rs +++ b/18_backtrace/kernel/tests/03_exception_restore_sanity.rs @@ -12,7 +12,7 @@ mod panic_wait_forever; use core::arch::asm; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[inline(never)] fn nested_system_call() { @@ -30,11 +30,9 @@ fn nested_system_call() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing exception restore"); diff --git a/18_backtrace/kernel/tests/04_exception_irq_sanity.rs b/18_backtrace/kernel/tests/04_exception_irq_sanity.rs index e93a4caf..e6f94c91 100644 --- a/18_backtrace/kernel/tests/04_exception_irq_sanity.rs +++ b/18_backtrace/kernel/tests/04_exception_irq_sanity.rs @@ -10,15 +10,13 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -use libkernel::{bsp, cpu, driver, exception, memory}; +use libkernel::{bsp, cpu, exception, memory}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); exception::handling_init(); exception::asynchronous::local_irq_unmask(); diff --git a/18_backtrace/kernel/tests/05_backtrace_sanity.rs b/18_backtrace/kernel/tests/05_backtrace_sanity.rs index ea425f68..f75c0ea3 100644 --- a/18_backtrace/kernel/tests/05_backtrace_sanity.rs +++ b/18_backtrace/kernel/tests/05_backtrace_sanity.rs @@ -11,7 +11,7 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{bsp, cpu, driver, exception, memory}; +use libkernel::{bsp, cpu, exception, memory}; #[inline(never)] fn nested() { @@ -20,11 +20,9 @@ fn nested() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); nested(); diff --git a/18_backtrace/kernel/tests/06_backtrace_invalid_frame.rs b/18_backtrace/kernel/tests/06_backtrace_invalid_frame.rs index 8769c7b6..33d3c02d 100644 --- a/18_backtrace/kernel/tests/06_backtrace_invalid_frame.rs +++ b/18_backtrace/kernel/tests/06_backtrace_invalid_frame.rs @@ -11,7 +11,7 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{backtrace, bsp, cpu, driver, exception, memory}; +use libkernel::{backtrace, bsp, cpu, exception, memory}; #[inline(never)] fn nested() { @@ -22,11 +22,9 @@ fn nested() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); nested(); diff --git a/18_backtrace/kernel/tests/07_backtrace_invalid_link.rs b/18_backtrace/kernel/tests/07_backtrace_invalid_link.rs index 28f3cdda..bcb0538a 100644 --- a/18_backtrace/kernel/tests/07_backtrace_invalid_link.rs +++ b/18_backtrace/kernel/tests/07_backtrace_invalid_link.rs @@ -11,7 +11,7 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{backtrace, bsp, cpu, driver, exception, memory}; +use libkernel::{backtrace, bsp, cpu, exception, memory}; #[inline(never)] fn nested_2() -> &'static str { @@ -27,11 +27,9 @@ fn nested_1() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); nested_1(); diff --git a/19_kernel_heap/Makefile b/19_kernel_heap/Makefile index 7b1673f3..be41c136 100644 --- a/19_kernel_heap/Makefile +++ b/19_kernel_heap/Makefile @@ -341,7 +341,7 @@ test_boot: $(KERNEL_BIN) ## Helpers for unit and integration test targets ##------------------------------------------------------------------------------ define KERNEL_TEST_RUNNER - #!/usr/bin/env bash +#!/usr/bin/env bash # The cargo test runner seems to change into the crate under test's directory. Therefore, ensure # this script executes from the root. diff --git a/19_kernel_heap/README.md b/19_kernel_heap/README.md index 4eafd344..a40d203d 100644 --- a/19_kernel_heap/README.md +++ b/19_kernel_heap/README.md @@ -131,11 +131,19 @@ wrapped allocator to the heap that we defined earlier: ```rust /// Query the BSP for the heap region and initialize the kernel's heap allocator with it. pub fn kernel_init_heap_allocator() { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + warn!("Already initialized"); + return; + } + let region = bsp::memory::mmu::virt_heap_region(); - KERNEL_HEAP_ALLOCATOR - .inner - .lock(|inner| unsafe { inner.init(region.start_addr().as_usize(), region.size()) }); + KERNEL_HEAP_ALLOCATOR.inner.lock(|inner| unsafe { + inner.init(region.start_addr().as_usize() as *mut u8, region.size()) + }); + + INIT_DONE.store(true, Ordering::Relaxed); } ``` @@ -288,7 +296,7 @@ diff -uNr 18_backtrace/kernel/Cargo.toml 19_kernel_heap/kernel/Cargo.toml diff -uNr 18_backtrace/kernel/src/bsp/device_driver/arm/gicv2.rs 19_kernel_heap/kernel/src/bsp/device_driver/arm/gicv2.rs --- 18_backtrace/kernel/src/bsp/device_driver/arm/gicv2.rs +++ 19_kernel_heap/kernel/src/bsp/device_driver/arm/gicv2.rs -@@ -85,12 +85,13 @@ +@@ -86,13 +86,13 @@ synchronization, synchronization::InitStateLock, }; @@ -298,12 +306,13 @@ diff -uNr 18_backtrace/kernel/src/bsp/device_driver/arm/gicv2.rs 19_kernel_heap/ // Private Definitions //-------------------------------------------------------------------------------------------------- --type HandlerTable = [Option; IRQNumber::NUM_TOTAL]; -+type HandlerTable = Vec>; +-type HandlerTable = [Option>; +- IRQNumber::MAX_INCLUSIVE + 1]; ++type HandlerTable = Vec>>; //-------------------------------------------------------------------------------------------------- // Public Definitions -@@ -119,7 +120,7 @@ +@@ -118,7 +118,7 @@ //-------------------------------------------------------------------------------------------------- impl GICv2 { @@ -312,21 +321,21 @@ diff -uNr 18_backtrace/kernel/src/bsp/device_driver/arm/gicv2.rs 19_kernel_heap/ pub const COMPATIBLE: &'static str = "GICv2 (ARM Generic Interrupt Controller v2)"; -@@ -136,7 +137,7 @@ +@@ -134,7 +134,7 @@ Self { gicd: gicd::GICD::new(gicd_mmio_start_addr), gicc: gicc::GICC::new(gicc_mmio_start_addr), -- handler_table: InitStateLock::new([None; IRQNumber::NUM_TOTAL]), +- handler_table: InitStateLock::new([None; IRQNumber::MAX_INCLUSIVE + 1]), + handler_table: InitStateLock::new(Vec::new()), - post_init_callback, } } -@@ -153,6 +154,9 @@ + } +@@ -152,6 +152,9 @@ } unsafe fn init(&self) -> Result<(), &'static str> { + self.handler_table -+ .write(|table| table.resize(IRQNumber::NUM_TOTAL, None)); ++ .write(|table| table.resize(IRQNumber::MAX_INCLUSIVE + 1, None)); + if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { self.gicd.boot_core_init(); @@ -343,20 +352,21 @@ diff -uNr 18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_contro use tock_registers::{ interfaces::{Readable, Writeable}, register_structs, -@@ -52,7 +53,7 @@ +@@ -52,8 +53,7 @@ /// Abstraction for the ReadOnly parts of the associated MMIO registers. type ReadOnlyRegisters = MMIODerefWrapper; --type HandlerTable = [Option; PeripheralIRQ::NUM_TOTAL]; -+type HandlerTable = Vec>; +-type HandlerTable = [Option>; +- PeripheralIRQ::MAX_INCLUSIVE + 1]; ++type HandlerTable = Vec>>; //-------------------------------------------------------------------------------------------------- // Public Definitions -@@ -84,10 +85,16 @@ +@@ -85,10 +85,16 @@ Self { wo_registers: IRQSafeNullLock::new(WriteOnlyRegisters::new(mmio_start_addr)), ro_registers: ReadOnlyRegisters::new(mmio_start_addr), -- handler_table: InitStateLock::new([None; PeripheralIRQ::NUM_TOTAL]), +- handler_table: InitStateLock::new([None; PeripheralIRQ::MAX_INCLUSIVE + 1]), + handler_table: InitStateLock::new(Vec::new()), } } @@ -364,7 +374,7 @@ diff -uNr 18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_contro + /// Called by the kernel to bring up the device. + pub fn init(&self) { + self.handler_table -+ .write(|table| table.resize(PeripheralIRQ::NUM_TOTAL, None)); ++ .write(|table| table.resize(PeripheralIRQ::MAX_INCLUSIVE + 1, None)); + } + /// Query the list of pending IRQs. @@ -374,20 +384,24 @@ diff -uNr 18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_contro diff -uNr 18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs 19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs --- 18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ 19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs -@@ -104,6 +104,8 @@ +@@ -109,6 +109,12 @@ + fn compatible(&self) -> &'static str { + Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { ++ ++ unsafe fn init(&self) -> Result<(), &'static str> { + self.periph.init(); + - (self.post_init_callback)(); ++ Ok(()) ++ } + } - Ok(()) + impl exception::asynchronous::interface::IRQManager for InterruptController { diff -uNr 18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs --- 18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ 19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs -@@ -329,6 +329,13 @@ +@@ -327,6 +327,13 @@ self.chars_written += 1; } @@ -401,7 +415,7 @@ diff -uNr 18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 19 /// Block execution until the last buffered character has been physically put on the TX wire. fn flush(&self) { // Spin until the busy bit is cleared. -@@ -451,6 +458,10 @@ +@@ -443,6 +450,10 @@ self.inner.lock(|inner| inner.write_char(c)); } @@ -413,72 +427,6 @@ diff -uNr 18_backtrace/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs 19 // Fully qualified syntax for the call to `core::fmt::Write::write_fmt()` to increase // readability. -diff -uNr 18_backtrace/kernel/src/bsp/raspberrypi/driver.rs 19_kernel_heap/kernel/src/bsp/raspberrypi/driver.rs ---- 18_backtrace/kernel/src/bsp/raspberrypi/driver.rs -+++ 19_kernel_heap/kernel/src/bsp/raspberrypi/driver.rs -@@ -11,6 +11,7 @@ - memory::mmu::MMIODescriptor, - synchronization::{interface::ReadWriteEx, InitStateLock}, - }; -+use alloc::vec::Vec; - use core::{ - mem::MaybeUninit, - sync::atomic::{AtomicBool, Ordering}, -@@ -24,18 +25,11 @@ - - /// Device Driver Manager type. - struct BSPDriverManager { -- device_drivers: InitStateLock<[Option<&'static (dyn DeviceDriver + Sync)>; NUM_DRIVERS]>, -+ device_drivers: InitStateLock>, - init_done: AtomicBool, - } - - //-------------------------------------------------------------------------------------------------- --// Public Definitions --//-------------------------------------------------------------------------------------------------- -- --/// The number of active drivers provided by this BSP. --pub const NUM_DRIVERS: usize = 3; -- --//-------------------------------------------------------------------------------------------------- - // Global instances - //-------------------------------------------------------------------------------------------------- - -@@ -50,7 +44,7 @@ - static mut INTERRUPT_CONTROLLER: MaybeUninit = MaybeUninit::uninit(); - - static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { -- device_drivers: InitStateLock::new([None; NUM_DRIVERS]), -+ device_drivers: InitStateLock::new(Vec::new()), - init_done: AtomicBool::new(false), - }; - -@@ -143,9 +137,9 @@ - - unsafe fn register_drivers(&self) { - self.device_drivers.write(|drivers| { -- drivers[0] = Some(PL011_UART.assume_init_ref()); -- drivers[1] = Some(GPIO.assume_init_ref()); -- drivers[2] = Some(INTERRUPT_CONTROLLER.assume_init_ref()); -+ drivers.push(PL011_UART.assume_init_ref()); -+ drivers.push(GPIO.assume_init_ref()); -+ drivers.push(INTERRUPT_CONTROLLER.assume_init_ref()); - }); - } - } -@@ -180,9 +174,8 @@ - Ok(()) - } - -- fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); NUM_DRIVERS] { -- self.device_drivers -- .read(|drivers| drivers.map(|drivers| drivers.unwrap())) -+ fn all_device_drivers(&self) -> &Vec<&(dyn DeviceDriver + Sync)> { -+ self.device_drivers.read(|drivers| drivers) - } - - #[cfg(feature = "test_build")] - diff -uNr 18_backtrace/kernel/src/bsp/raspberrypi/kernel.ld 19_kernel_heap/kernel/src/bsp/raspberrypi/kernel.ld --- 18_backtrace/kernel/src/bsp/raspberrypi/kernel.ld +++ 19_kernel_heap/kernel/src/bsp/raspberrypi/kernel.ld @@ -819,26 +767,198 @@ diff -uNr 18_backtrace/kernel/src/console.rs 19_kernel_heap/kernel/src/console.r diff -uNr 18_backtrace/kernel/src/driver.rs 19_kernel_heap/kernel/src/driver.rs --- 18_backtrace/kernel/src/driver.rs +++ 19_kernel_heap/kernel/src/driver.rs -@@ -10,7 +10,7 @@ +@@ -8,23 +8,10 @@ + exception, info, + synchronization::{interface::ReadWriteEx, InitStateLock}, + }; ++use alloc::vec::Vec; + use core::fmt; - /// Driver interfaces. - pub mod interface { -- use crate::bsp; -+ use alloc::vec::Vec; + //-------------------------------------------------------------------------------------------------- +-// Private Definitions +-//-------------------------------------------------------------------------------------------------- +- +-const NUM_DRIVERS: usize = 5; +- +-struct DriverManagerInner +-where +- T: 'static, +-{ +- next_index: usize, +- descriptors: [Option>; NUM_DRIVERS], +-} +- +-//-------------------------------------------------------------------------------------------------- + // Public Definitions + //-------------------------------------------------------------------------------------------------- + +@@ -68,7 +55,6 @@ + pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + + /// A descriptor for device drivers. +-#[derive(Copy, Clone)] + pub struct DeviceDriverDescriptor + where + T: 'static, +@@ -83,7 +69,7 @@ + where + T: 'static, + { +- inner: InitStateLock>, ++ descriptors: InitStateLock>>, + } + + //-------------------------------------------------------------------------------------------------- +@@ -93,23 +79,6 @@ + static DRIVER_MANAGER: DriverManager = DriverManager::new(); + + //-------------------------------------------------------------------------------------------------- +-// Private Code +-//-------------------------------------------------------------------------------------------------- +- +-impl DriverManagerInner +-where +- T: 'static + Copy, +-{ +- /// Create an instance. +- pub const fn new() -> Self { +- Self { +- next_index: 0, +- descriptors: [None; NUM_DRIVERS], +- } +- } +-} +- +-//-------------------------------------------------------------------------------------------------- + // Public Code + //-------------------------------------------------------------------------------------------------- + +@@ -135,32 +104,19 @@ + + impl DriverManager + where +- T: fmt::Display + Copy, ++ T: fmt::Display, + { + /// Create an instance. + pub const fn new() -> Self { + Self { +- inner: InitStateLock::new(DriverManagerInner::new()), ++ descriptors: InitStateLock::new(Vec::new()), + } + } - /// Device Driver functions. - pub trait DeviceDriver { -@@ -46,8 +46,8 @@ - /// Must be called before `all_device_drivers`. - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str>; + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { +- self.inner.write(|inner| { +- inner.descriptors[inner.next_index] = Some(descriptor); +- inner.next_index += 1; +- }) +- } +- +- /// Helper for iterating over registered drivers. +- fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { +- self.inner.read(|inner| { +- inner +- .descriptors +- .iter() +- .filter_map(|x| x.as_ref()) +- .for_each(f) +- }) ++ self.descriptors ++ .write(|descriptors| descriptors.push(descriptor)); + } -- /// Return a slice of references to all `BSP`-instantiated drivers. -- fn all_device_drivers(&self) -> [&(dyn DeviceDriver + Sync); bsp::driver::NUM_DRIVERS]; -+ /// Return a vector of references to all `BSP`-instantiated drivers. -+ fn all_device_drivers(&self) -> &Vec<&(dyn DeviceDriver + Sync)>; + /// Fully initialize all drivers and their interrupts handlers. +@@ -169,53 +125,54 @@ + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers_and_irqs(&self) { +- self.for_each_descriptor(|descriptor| { +- // 1. Initialize driver. +- if let Err(x) = descriptor.device_driver.init() { +- panic!( +- "Error initializing driver: {}: {}", +- descriptor.device_driver.compatible(), +- x +- ); +- } +- +- // 2. Call corresponding post init callback. +- if let Some(callback) = &descriptor.post_init_callback { +- if let Err(x) = callback() { ++ self.descriptors.read(|descriptors| { ++ for descriptor in descriptors { ++ // 1. Initialize driver. ++ if let Err(x) = descriptor.device_driver.init() { + panic!( +- "Error during driver post-init callback: {}: {}", ++ "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } ++ ++ // 2. Call corresponding post init callback. ++ if let Some(callback) = &descriptor.post_init_callback { ++ if let Err(x) = callback() { ++ panic!( ++ "Error during driver post-init callback: {}: {}", ++ descriptor.device_driver.compatible(), ++ x ++ ); ++ } ++ } + } +- }); + +- // 3. After all post-init callbacks were done, the interrupt controller should be +- // registered and functional. So let drivers register with it now. +- self.for_each_descriptor(|descriptor| { +- if let Some(irq_number) = &descriptor.irq_number { +- if let Err(x) = descriptor +- .device_driver +- .register_and_enable_irq_handler(irq_number) +- { +- panic!( +- "Error during driver interrupt handler registration: {}: {}", +- descriptor.device_driver.compatible(), +- x +- ); ++ // 3. After all post-init callbacks were done, the interrupt controller should be ++ // registered and functional. So let drivers register with it now. ++ for descriptor in descriptors { ++ if let Some(irq_number) = &descriptor.irq_number { ++ if let Err(x) = descriptor ++ .device_driver ++ .register_and_enable_irq_handler(irq_number) ++ { ++ panic!( ++ "Error during driver interrupt handler registration: {}: {}", ++ descriptor.device_driver.compatible(), ++ x ++ ); ++ } + } + } +- }); ++ }) + } - /// Minimal code needed to bring up the console in QEMU (for testing only). This is often - /// less steps than on real hardware due to QEMU's abstractions. + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { +- let mut i: usize = 1; +- self.for_each_descriptor(|descriptor| { +- info!(" {}. {}", i, descriptor.device_driver.compatible()); +- +- i += 1; ++ self.descriptors.read(|descriptors| { ++ for (i, desc) in descriptors.iter().enumerate() { ++ info!(" {}. {}", i + 1, desc.device_driver.compatible()); ++ } + }); + } + } diff -uNr 18_backtrace/kernel/src/lib.rs 19_kernel_heap/kernel/src/lib.rs --- 18_backtrace/kernel/src/lib.rs @@ -870,10 +990,10 @@ diff -uNr 18_backtrace/kernel/src/main.rs 19_kernel_heap/kernel/src/main.rs +extern crate alloc; + - use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; + use libkernel::{bsp, cpu, driver, exception, info, memory, state, time}; /// Early init code. -@@ -92,6 +94,9 @@ +@@ -73,6 +75,9 @@ info!("Registered IRQ handlers:"); exception::asynchronous::irq_manager().print_handler(); @@ -887,7 +1007,7 @@ diff -uNr 18_backtrace/kernel/src/main.rs 19_kernel_heap/kernel/src/main.rs diff -uNr 18_backtrace/kernel/src/memory/heap_alloc.rs 19_kernel_heap/kernel/src/memory/heap_alloc.rs --- 18_backtrace/kernel/src/memory/heap_alloc.rs +++ 19_kernel_heap/kernel/src/memory/heap_alloc.rs -@@ -0,0 +1,137 @@ +@@ -0,0 +1,147 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter @@ -899,8 +1019,10 @@ diff -uNr 18_backtrace/kernel/src/memory/heap_alloc.rs 19_kernel_heap/kernel/src + memory::{Address, Virtual}, + synchronization, + synchronization::IRQSafeNullLock, ++ warn, +}; +use alloc::alloc::{GlobalAlloc, Layout}; ++use core::sync::atomic::{AtomicBool, Ordering}; +use linked_list_allocator::Heap as LinkedListHeap; + +//-------------------------------------------------------------------------------------------------- @@ -1019,11 +1141,19 @@ diff -uNr 18_backtrace/kernel/src/memory/heap_alloc.rs 19_kernel_heap/kernel/src + +/// Query the BSP for the heap region and initialize the kernel's heap allocator with it. +pub fn kernel_init_heap_allocator() { ++ static INIT_DONE: AtomicBool = AtomicBool::new(false); ++ if INIT_DONE.load(Ordering::Relaxed) { ++ warn!("Already initialized"); ++ return; ++ } ++ + let region = bsp::memory::mmu::virt_heap_region(); + + KERNEL_HEAP_ALLOCATOR.inner.lock(|inner| unsafe { + inner.init(region.start_addr().as_usize() as *mut u8, region.size()) + }); ++ ++ INIT_DONE.store(true, Ordering::Relaxed); +} diff -uNr 18_backtrace/kernel/src/memory/mmu/mapping_record.rs 19_kernel_heap/kernel/src/memory/mmu/mapping_record.rs @@ -1280,6 +1410,19 @@ diff -uNr 18_backtrace/kernel/src/print.rs 19_kernel_heap/kernel/src/print.rs + }) +} +diff -uNr 18_backtrace/kernel/src/state.rs 19_kernel_heap/kernel/src/state.rs +--- 18_backtrace/kernel/src/state.rs ++++ 19_kernel_heap/kernel/src/state.rs +@@ -52,7 +52,7 @@ + const SINGLE_CORE_MAIN: u8 = 1; + const MULTI_CORE_MAIN: u8 = 2; + +- /// Create an instance. ++ /// Create a new instance. + pub const fn new() -> Self { + Self(AtomicU8::new(Self::INIT)) + } + diff -uNr 18_backtrace/Makefile 19_kernel_heap/Makefile --- 18_backtrace/Makefile +++ 19_kernel_heap/Makefile diff --git a/19_kernel_heap/kernel/src/bsp/device_driver/arm/gicv2.rs b/19_kernel_heap/kernel/src/bsp/device_driver/arm/gicv2.rs index 69ba8513..fee8bb4c 100644 --- a/19_kernel_heap/kernel/src/bsp/device_driver/arm/gicv2.rs +++ b/19_kernel_heap/kernel/src/bsp/device_driver/arm/gicv2.rs @@ -80,7 +80,8 @@ mod gicc; mod gicd; use crate::{ - bsp, cpu, driver, exception, + bsp::{self, device_driver::common::BoundedUsize}, + cpu, driver, exception, memory::{Address, Virtual}, synchronization, synchronization::InitStateLock, @@ -91,14 +92,14 @@ use alloc::vec::Vec; // Private Definitions //-------------------------------------------------------------------------------------------------- -type HandlerTable = Vec>; +type HandlerTable = Vec>>; //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. -pub type IRQNumber = exception::asynchronous::IRQNumber<{ GICv2::MAX_IRQ_NUMBER }>; +pub type IRQNumber = BoundedUsize<{ GICv2::MAX_IRQ_NUMBER }>; /// Representation of the GIC. pub struct GICv2 { @@ -110,9 +111,6 @@ pub struct GICv2 { /// Stores registered IRQ handlers. Writable only during kernel init. RO afterwards. handler_table: InitStateLock, - - /// Callback to be invoked after successful init. - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -132,13 +130,11 @@ impl GICv2 { pub const unsafe fn new( gicd_mmio_start_addr: Address, gicc_mmio_start_addr: Address, - post_init_callback: fn(), ) -> Self { Self { gicd: gicd::GICD::new(gicd_mmio_start_addr), gicc: gicc::GICC::new(gicc_mmio_start_addr), handler_table: InitStateLock::new(Vec::new()), - post_init_callback, } } } @@ -149,13 +145,15 @@ impl GICv2 { use synchronization::interface::ReadWriteEx; impl driver::interface::DeviceDriver for GICv2 { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } unsafe fn init(&self) -> Result<(), &'static str> { self.handler_table - .write(|table| table.resize(IRQNumber::NUM_TOTAL, None)); + .write(|table| table.resize(IRQNumber::MAX_INCLUSIVE + 1, None)); if bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id() { self.gicd.boot_core_init(); @@ -164,8 +162,6 @@ impl driver::interface::DeviceDriver for GICv2 { self.gicc.priority_accept_all(); self.gicc.enable(); - (self.post_init_callback)(); - Ok(()) } } @@ -175,23 +171,22 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq_number.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq_number: Self::IRQNumberType) { + fn enable(&self, irq_number: &Self::IRQNumberType) { self.gicd.enable(irq_number); } @@ -214,7 +209,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } }); @@ -231,7 +226,7 @@ impl exception::asynchronous::interface::IRQManager for GICv2 { self.handler_table.read(|table| { for (i, opt) in table.iter().skip(32).enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i + 32, handler.name); + info!(" {: >3}. {}", i + 32, handler.name()); } } }); diff --git a/19_kernel_heap/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs b/19_kernel_heap/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs index a9c97a8e..8aebcf2b 100644 --- a/19_kernel_heap/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs +++ b/19_kernel_heap/kernel/src/bsp/device_driver/arm/gicv2/gicd.rs @@ -172,7 +172,7 @@ impl GICD { } /// Enable an interrupt. - pub fn enable(&self, irq_num: super::IRQNumber) { + pub fn enable(&self, irq_num: &super::IRQNumber) { let irq_num = irq_num.get(); // Each bit in the u32 enable register corresponds to one IRQ number. Shift right by 5 diff --git a/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs b/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 2281e66a..fb61a651 100644 --- a/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs +++ b/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -7,6 +7,7 @@ use crate::{ bsp::device_driver::common::MMIODerefWrapper, driver, + exception::asynchronous::IRQNumber, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -122,7 +123,6 @@ struct GPIOInner { /// Representation of the GPIO HW. pub struct GPIO { inner: IRQSafeNullLock, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -202,10 +202,9 @@ impl GPIO { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new(mmio_start_addr: Address, post_init_callback: fn()) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(GPIOInner::new(mmio_start_addr)), - post_init_callback, } } @@ -221,13 +220,9 @@ impl GPIO { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for GPIO { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } - - unsafe fn init(&self) -> Result<(), &'static str> { - (self.post_init_callback)(); - - Ok(()) - } } diff --git a/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs b/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs index 658d3705..a03b472a 100644 --- a/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs +++ b/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller.rs @@ -7,9 +7,12 @@ mod peripheral_ic; use crate::{ - driver, exception, + bsp::device_driver::common::BoundedUsize, + driver, + exception::{self, asynchronous::IRQHandlerDescriptor}, memory::{Address, Virtual}, }; +use core::fmt; //-------------------------------------------------------------------------------------------------- // Private Definitions @@ -24,10 +27,8 @@ struct PendingIRQs { // Public Definitions //-------------------------------------------------------------------------------------------------- -pub type LocalIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; -pub type PeripheralIRQ = - exception::asynchronous::IRQNumber<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; +pub type LocalIRQ = BoundedUsize<{ InterruptController::MAX_LOCAL_IRQ_NUMBER }>; +pub type PeripheralIRQ = BoundedUsize<{ InterruptController::MAX_PERIPHERAL_IRQ_NUMBER }>; /// Used for the associated type of trait [`exception::asynchronous::interface::IRQManager`]. #[derive(Copy, Clone)] @@ -40,7 +41,6 @@ pub enum IRQNumber { /// Representation of the Interrupt Controller. pub struct InterruptController { periph: peripheral_ic::PeripheralIC, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -71,6 +71,15 @@ impl Iterator for PendingIRQs { // Public Code //-------------------------------------------------------------------------------------------------- +impl fmt::Display for IRQNumber { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + match self { + Self::Local(number) => write!(f, "Local({})", number), + Self::Peripheral(number) => write!(f, "Peripheral({})", number), + } + } +} + impl InterruptController { // Restrict to 3 for now. This makes future code for local_ic.rs more straight forward. const MAX_LOCAL_IRQ_NUMBER: usize = 3; @@ -83,13 +92,9 @@ impl InterruptController { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - pub const unsafe fn new( - periph_mmio_start_addr: Address, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(periph_mmio_start_addr: Address) -> Self { Self { periph: peripheral_ic::PeripheralIC::new(periph_mmio_start_addr), - post_init_callback, } } } @@ -99,6 +104,8 @@ impl InterruptController { //------------------------------------------------------------------------------ impl driver::interface::DeviceDriver for InterruptController { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } @@ -106,8 +113,6 @@ impl driver::interface::DeviceDriver for InterruptController { unsafe fn init(&self) -> Result<(), &'static str> { self.periph.init(); - (self.post_init_callback)(); - Ok(()) } } @@ -117,16 +122,23 @@ impl exception::asynchronous::interface::IRQManager for InterruptController { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { - match irq { + match irq_handler_descriptor.number() { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), - IRQNumber::Peripheral(pirq) => self.periph.register_handler(pirq, descriptor), + IRQNumber::Peripheral(pirq) => { + let periph_descriptor = IRQHandlerDescriptor::new( + pirq, + irq_handler_descriptor.name(), + irq_handler_descriptor.handler(), + ); + + self.periph.register_handler(periph_descriptor) + } } } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { match irq { IRQNumber::Local(_) => unimplemented!("Local IRQ controller not implemented."), IRQNumber::Peripheral(pirq) => self.periph.enable(pirq), diff --git a/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs b/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs index 90b56378..238088a8 100644 --- a/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs +++ b/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_interrupt_controller/peripheral_ic.rs @@ -53,7 +53,7 @@ type WriteOnlyRegisters = MMIODerefWrapper; /// Abstraction for the ReadOnly parts of the associated MMIO registers. type ReadOnlyRegisters = MMIODerefWrapper; -type HandlerTable = Vec>; +type HandlerTable = Vec>>; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -92,7 +92,7 @@ impl PeripheralIC { /// Called by the kernel to bring up the device. pub fn init(&self) { self.handler_table - .write(|table| table.resize(PeripheralIRQ::NUM_TOTAL, None)); + .write(|table| table.resize(PeripheralIRQ::MAX_INCLUSIVE + 1, None)); } /// Query the list of pending IRQs. @@ -114,23 +114,22 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { fn register_handler( &self, - irq: Self::IRQNumberType, - descriptor: exception::asynchronous::IRQDescriptor, + irq_handler_descriptor: exception::asynchronous::IRQHandlerDescriptor, ) -> Result<(), &'static str> { self.handler_table.write(|table| { - let irq_number = irq.get(); + let irq_number = irq_handler_descriptor.number().get(); if table[irq_number].is_some() { return Err("IRQ handler already registered"); } - table[irq_number] = Some(descriptor); + table[irq_number] = Some(irq_handler_descriptor); Ok(()) }) } - fn enable(&self, irq: Self::IRQNumberType) { + fn enable(&self, irq: &Self::IRQNumberType) { self.wo_registers.lock(|regs| { let enable_reg = if irq.get() <= 31 { ®s.ENABLE_1 @@ -156,7 +155,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { None => panic!("No handler registered for IRQ {}", irq_number), Some(descriptor) => { // Call the IRQ handler. Panics on failure. - descriptor.handler.handle().expect("Error handling IRQ"); + descriptor.handler().handle().expect("Error handling IRQ"); } } } @@ -171,7 +170,7 @@ impl exception::asynchronous::interface::IRQManager for PeripheralIC { self.handler_table.read(|table| { for (i, opt) in table.iter().enumerate() { if let Some(handler) = opt { - info!(" {: >3}. {}", i, handler.name); + info!(" {: >3}. {}", i, handler.name()); } } }); diff --git a/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs b/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index 26593039..3e7e1812 100644 --- a/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs +++ b/19_kernel_heap/kernel/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -10,9 +10,9 @@ //! - use crate::{ - bsp, bsp::device_driver::common::MMIODerefWrapper, - console, cpu, driver, exception, + console, cpu, driver, + exception::{self, asynchronous::IRQNumber}, memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, @@ -233,8 +233,6 @@ struct PL011UartInner { /// Representation of the UART. pub struct PL011Uart { inner: IRQSafeNullLock, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), } //-------------------------------------------------------------------------------------------------- @@ -405,16 +403,9 @@ impl PL011Uart { /// # Safety /// /// - The user must ensure to provide a correct MMIO start address. - /// - The user must ensure to provide correct IRQ numbers. - pub const unsafe fn new( - mmio_start_addr: Address, - irq_number: bsp::device_driver::IRQNumber, - post_init_callback: fn(), - ) -> Self { + pub const unsafe fn new(mmio_start_addr: Address) -> Self { Self { inner: IRQSafeNullLock::new(PL011UartInner::new(mmio_start_addr)), - irq_number, - post_init_callback, } } } @@ -425,27 +416,28 @@ impl PL011Uart { use synchronization::interface::Mutex; impl driver::interface::DeviceDriver for PL011Uart { + type IRQNumberType = IRQNumber; + fn compatible(&self) -> &'static str { Self::COMPATIBLE } unsafe fn init(&self) -> Result<(), &'static str> { self.inner.lock(|inner| inner.init()); - (self.post_init_callback)(); Ok(()) } - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - use exception::asynchronous::{irq_manager, IRQDescriptor}; + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + use exception::asynchronous::{irq_manager, IRQHandlerDescriptor}; - let descriptor = IRQDescriptor { - name: Self::COMPATIBLE, - handler: self, - }; + let descriptor = IRQHandlerDescriptor::new(*irq_number, Self::COMPATIBLE, self); - irq_manager().register_handler(self.irq_number, descriptor)?; - irq_manager().enable(self.irq_number); + irq_manager().register_handler(descriptor)?; + irq_manager().enable(irq_number); Ok(()) } diff --git a/19_kernel_heap/kernel/src/bsp/device_driver/common.rs b/19_kernel_heap/kernel/src/bsp/device_driver/common.rs index 69f038d4..ca7aeb76 100644 --- a/19_kernel_heap/kernel/src/bsp/device_driver/common.rs +++ b/19_kernel_heap/kernel/src/bsp/device_driver/common.rs @@ -5,7 +5,7 @@ //! Common device driver code. use crate::memory::{Address, Virtual}; -use core::{marker::PhantomData, ops}; +use core::{fmt, marker::PhantomData, ops}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -16,6 +16,10 @@ pub struct MMIODerefWrapper { phantom: PhantomData T>, } +/// A wrapper type for usize with integrated range bound check. +#[derive(Copy, Clone)] +pub struct BoundedUsize(usize); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- @@ -37,3 +41,25 @@ impl ops::Deref for MMIODerefWrapper { unsafe { &*(self.start_addr.as_usize() as *const _) } } } + +impl BoundedUsize<{ MAX_INCLUSIVE }> { + pub const MAX_INCLUSIVE: usize = MAX_INCLUSIVE; + + /// Creates a new instance if number <= MAX_INCLUSIVE. + pub const fn new(number: usize) -> Self { + assert!(number <= MAX_INCLUSIVE); + + Self(number) + } + + /// Return the wrapped number. + pub const fn get(self) -> usize { + self.0 + } +} + +impl fmt::Display for BoundedUsize<{ MAX_INCLUSIVE }> { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + write!(f, "{}", self.0) + } +} diff --git a/19_kernel_heap/kernel/src/bsp/raspberrypi/driver.rs b/19_kernel_heap/kernel/src/bsp/raspberrypi/driver.rs index 5571c392..ca3435aa 100644 --- a/19_kernel_heap/kernel/src/bsp/raspberrypi/driver.rs +++ b/19_kernel_heap/kernel/src/bsp/raspberrypi/driver.rs @@ -7,28 +7,16 @@ use super::{exception, memory::map::mmio}; use crate::{ bsp::device_driver, - console, driver, exception as generic_exception, memory, + console, driver as generic_driver, + exception::{self as generic_exception}, + memory, memory::mmu::MMIODescriptor, - synchronization::{interface::ReadWriteEx, InitStateLock}, }; -use alloc::vec::Vec; use core::{ mem::MaybeUninit, sync::atomic::{AtomicBool, Ordering}, }; -pub use device_driver::IRQNumber; - -//-------------------------------------------------------------------------------------------------- -// Private Definitions -//-------------------------------------------------------------------------------------------------- - -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: InitStateLock>, - init_done: AtomicBool, -} - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- @@ -43,149 +31,154 @@ static mut INTERRUPT_CONTROLLER: MaybeUninit #[cfg(feature = "bsp_rpi4")] static mut INTERRUPT_CONTROLLER: MaybeUninit = MaybeUninit::uninit(); -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: InitStateLock::new(Vec::new()), - init_done: AtomicBool::new(false), -}; - //-------------------------------------------------------------------------------------------------- // Private Code //-------------------------------------------------------------------------------------------------- -impl BSPDriverManager { - unsafe fn instantiate_uart(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_uart() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::PL011_UART_START, mmio::PL011_UART_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::PL011Uart::COMPATIBLE, &mmio_descriptor)?; - // This is safe to do, because it is only called from the init'ed instance itself. - fn uart_post_init() { - console::register_console(unsafe { PL011_UART.assume_init_ref() }); - } + PL011_UART.write(device_driver::PL011Uart::new(virt_addr)); - PL011_UART.write(device_driver::PL011Uart::new( - virt_addr, - exception::asynchronous::irq_map::PL011_UART, - uart_post_init, - )); + Ok(()) +} - Ok(()) - } +/// This must be called only after successful init of the UART driver. +unsafe fn post_init_uart() -> Result<(), &'static str> { + console::register_console(PL011_UART.assume_init_ref()); - unsafe fn instantiate_gpio(&self) -> Result<(), &'static str> { - let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); - let virt_addr = - memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; + Ok(()) +} - // This is safe to do, because it is only called from the init'ed instance itself. - fn gpio_post_init() { - unsafe { GPIO.assume_init_ref().map_pl011_uart() }; - } +/// This must be called only after successful init of the memory subsystem. +unsafe fn instantiate_gpio() -> Result<(), &'static str> { + let mmio_descriptor = MMIODescriptor::new(mmio::GPIO_START, mmio::GPIO_SIZE); + let virt_addr = + memory::mmu::kernel_map_mmio(device_driver::GPIO::COMPATIBLE, &mmio_descriptor)?; - GPIO.write(device_driver::GPIO::new(virt_addr, gpio_post_init)); + GPIO.write(device_driver::GPIO::new(virt_addr)); - Ok(()) - } + Ok(()) +} - #[cfg(feature = "bsp_rpi3")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let periph_mmio_descriptor = - MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); - let periph_virt_addr = memory::mmu::kernel_map_mmio( - device_driver::InterruptController::COMPATIBLE, - &periph_mmio_descriptor, - )?; - - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } - - INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new( - periph_virt_addr, - interrupt_controller_post_init, - )); - - Ok(()) - } +/// This must be called only after successful init of the GPIO driver. +unsafe fn post_init_gpio() -> Result<(), &'static str> { + GPIO.assume_init_ref().map_pl011_uart(); + Ok(()) +} - #[cfg(feature = "bsp_rpi4")] - unsafe fn instantiate_interrupt_controller(&self) -> Result<(), &'static str> { - let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); - let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi3")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let periph_mmio_descriptor = + MMIODescriptor::new(mmio::PERIPHERAL_IC_START, mmio::PERIPHERAL_IC_SIZE); + let periph_virt_addr = memory::mmu::kernel_map_mmio( + device_driver::InterruptController::COMPATIBLE, + &periph_mmio_descriptor, + )?; - let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); - let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; + INTERRUPT_CONTROLLER.write(device_driver::InterruptController::new(periph_virt_addr)); - // This is safe to do, because it is only called from the init'ed instance itself. - fn interrupt_controller_post_init() { - generic_exception::asynchronous::register_irq_manager(unsafe { - INTERRUPT_CONTROLLER.assume_init_ref() - }); - } + Ok(()) +} - INTERRUPT_CONTROLLER.write(device_driver::GICv2::new( - gicd_virt_addr, - gicc_virt_addr, - interrupt_controller_post_init, - )); +/// This must be called only after successful init of the memory subsystem. +#[cfg(feature = "bsp_rpi4")] +unsafe fn instantiate_interrupt_controller() -> Result<(), &'static str> { + let gicd_mmio_descriptor = MMIODescriptor::new(mmio::GICD_START, mmio::GICD_SIZE); + let gicd_virt_addr = memory::mmu::kernel_map_mmio("GICv2 GICD", &gicd_mmio_descriptor)?; - Ok(()) - } + let gicc_mmio_descriptor = MMIODescriptor::new(mmio::GICC_START, mmio::GICC_SIZE); + let gicc_virt_addr = memory::mmu::kernel_map_mmio("GICV2 GICC", &gicc_mmio_descriptor)?; - unsafe fn register_drivers(&self) { - self.device_drivers.write(|drivers| { - drivers.push(PL011_UART.assume_init_ref()); - drivers.push(GPIO.assume_init_ref()); - drivers.push(INTERRUPT_CONTROLLER.assume_init_ref()); - }); - } + INTERRUPT_CONTROLLER.write(device_driver::GICv2::new(gicd_virt_addr, gicc_virt_addr)); + + Ok(()) } -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +/// This must be called only after successful init of the interrupt controller driver. +unsafe fn post_init_interrupt_controller() -> Result<(), &'static str> { + generic_exception::asynchronous::register_irq_manager(INTERRUPT_CONTROLLER.assume_init_ref()); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_uart() -> Result<(), &'static str> { + instantiate_uart()?; + + let uart_descriptor = generic_driver::DeviceDriverDescriptor::new( + PL011_UART.assume_init_ref(), + Some(post_init_uart), + Some(exception::asynchronous::irq_map::PL011_UART), + ); + generic_driver::driver_manager().register_driver(uart_descriptor); -impl driver::interface::DriverManager for BSPDriverManager { - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str> { - if self.init_done.load(Ordering::Relaxed) { - return Err("Drivers already instantiated"); - } + Ok(()) +} - self.instantiate_uart()?; - self.instantiate_gpio()?; - self.instantiate_interrupt_controller()?; +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_gpio() -> Result<(), &'static str> { + instantiate_gpio()?; - self.register_drivers(); + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new( + GPIO.assume_init_ref(), + Some(post_init_gpio), + None, + ); + generic_driver::driver_manager().register_driver(gpio_descriptor); - self.init_done.store(true, Ordering::Relaxed); - Ok(()) - } + Ok(()) +} - fn all_device_drivers(&self) -> &Vec<&(dyn DeviceDriver + Sync)> { - self.device_drivers.read(|drivers| drivers) - } +/// Function needs to ensure that driver registration happens only after correct instantiation. +unsafe fn driver_interrupt_controller() -> Result<(), &'static str> { + instantiate_interrupt_controller()?; + + let interrupt_controller_descriptor = generic_driver::DeviceDriverDescriptor::new( + INTERRUPT_CONTROLLER.assume_init_ref(), + Some(post_init_interrupt_controller), + None, + ); + generic_driver::driver_manager().register_driver(interrupt_controller_descriptor); - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self) { - use crate::cpu; + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - unsafe { - self.instantiate_uart() - .unwrap_or_else(|_| cpu::qemu_exit_failure()); - console::register_console(PL011_UART.assume_init_ref()); - }; +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } + + driver_uart()?; + driver_gpio()?; + driver_interrupt_controller()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) +} + +/// Minimal code needed to bring up the console in QEMU (for testing only). This is often less steps +/// than on real hardware due to QEMU's abstractions. +#[cfg(feature = "test_build")] +pub fn qemu_bring_up_console() { + use crate::cpu; + + unsafe { + instantiate_uart().unwrap_or_else(|_| cpu::qemu_exit_failure()); + console::register_console(PL011_UART.assume_init_ref()); + }; } diff --git a/19_kernel_heap/kernel/src/bsp/raspberrypi/exception/asynchronous.rs b/19_kernel_heap/kernel/src/bsp/raspberrypi/exception/asynchronous.rs index ab20d86d..06a67558 100644 --- a/19_kernel_heap/kernel/src/bsp/raspberrypi/exception/asynchronous.rs +++ b/19_kernel_heap/kernel/src/bsp/raspberrypi/exception/asynchronous.rs @@ -10,6 +10,9 @@ use crate::bsp; // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Export for reuse in generic asynchronous.rs. +pub use bsp::device_driver::IRQNumber; + #[cfg(feature = "bsp_rpi3")] pub(in crate::bsp) mod irq_map { use super::bsp::device_driver::{IRQNumber, PeripheralIRQ}; diff --git a/19_kernel_heap/kernel/src/driver.rs b/19_kernel_heap/kernel/src/driver.rs index d884d471..2f22fd20 100644 --- a/19_kernel_heap/kernel/src/driver.rs +++ b/19_kernel_heap/kernel/src/driver.rs @@ -4,16 +4,24 @@ //! Driver support. +use crate::{ + exception, info, + synchronization::{interface::ReadWriteEx, InitStateLock}, +}; +use alloc::vec::Vec; +use core::fmt; + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- /// Driver interfaces. pub mod interface { - use alloc::vec::Vec; - /// Device Driver functions. pub trait DeviceDriver { + /// Different interrupt controllers might use different types for IRQ number. + type IRQNumberType: super::fmt::Display; + /// Return a compatibility string for identifying the driver. fn compatible(&self) -> &'static str; @@ -26,32 +34,145 @@ pub mod interface { Ok(()) } - /// Called by the kernel to register and enable the device's IRQ handlers, if any. + /// Called by the kernel to register and enable the device's IRQ handler. /// /// Rust's type system will prevent a call to this function unless the calling instance /// itself has static lifetime. - fn register_and_enable_irq_handler(&'static self) -> Result<(), &'static str> { - Ok(()) + fn register_and_enable_irq_handler( + &'static self, + irq_number: &Self::IRQNumberType, + ) -> Result<(), &'static str> { + panic!( + "Attempt to enable IRQ {} for device {}, but driver does not support this", + irq_number, + self.compatible() + ) + } + } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; + +/// A descriptor for device drivers. +pub struct DeviceDriverDescriptor +where + T: 'static, +{ + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager +where + T: 'static, +{ + descriptors: InitStateLock>>, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + irq_number: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + irq_number, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager +where + T: fmt::Display, +{ + /// Create an instance. + pub const fn new() -> Self { + Self { + descriptors: InitStateLock::new(Vec::new()), } } - /// Device driver management functions. + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.descriptors + .write(|descriptors| descriptors.push(descriptor)); + } + + /// Fully initialize all drivers and their interrupts handlers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Instantiate all drivers. - /// - /// # Safety - /// - /// Must be called before `all_device_drivers`. - unsafe fn instantiate_drivers(&self) -> Result<(), &'static str>; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers_and_irqs(&self) { + self.descriptors.read(|descriptors| { + for descriptor in descriptors { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } - /// Return a vector of references to all `BSP`-instantiated drivers. - fn all_device_drivers(&self) -> &Vec<&(dyn DeviceDriver + Sync)>; + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + } + + // 3. After all post-init callbacks were done, the interrupt controller should be + // registered and functional. So let drivers register with it now. + for descriptor in descriptors { + if let Some(irq_number) = &descriptor.irq_number { + if let Err(x) = descriptor + .device_driver + .register_and_enable_irq_handler(irq_number) + { + panic!( + "Error during driver interrupt handler registration: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + } + }) + } - /// Minimal code needed to bring up the console in QEMU (for testing only). This is often - /// less steps than on real hardware due to QEMU's abstractions. - #[cfg(feature = "test_build")] - fn qemu_bring_up_console(&self); + /// Enumerate all registered device drivers. + pub fn enumerate(&self) { + self.descriptors.read(|descriptors| { + for (i, desc) in descriptors.iter().enumerate() { + info!(" {}. {}", i + 1, desc.device_driver.compatible()); + } + }); } } diff --git a/19_kernel_heap/kernel/src/exception/asynchronous.rs b/19_kernel_heap/kernel/src/exception/asynchronous.rs index 83ca1aa6..c1f2a27b 100644 --- a/19_kernel_heap/kernel/src/exception/asynchronous.rs +++ b/19_kernel_heap/kernel/src/exception/asynchronous.rs @@ -10,7 +10,7 @@ mod arch_asynchronous; mod null_irq_manager; use crate::{bsp, synchronization}; -use core::{fmt, marker::PhantomData}; +use core::marker::PhantomData; //-------------------------------------------------------------------------------------------------- // Architectural Public Reexports @@ -24,14 +24,23 @@ pub use arch_asynchronous::{ // Public Definitions //-------------------------------------------------------------------------------------------------- +/// Interrupt number as defined by the BSP. +pub type IRQNumber = bsp::exception::asynchronous::IRQNumber; + /// Interrupt descriptor. #[derive(Copy, Clone)] -pub struct IRQDescriptor { +pub struct IRQHandlerDescriptor +where + T: Copy, +{ + /// The IRQ number. + number: T, + /// Descriptive name. - pub name: &'static str, + name: &'static str, /// Reference to handler trait object. - pub handler: &'static (dyn interface::IRQHandler + Sync), + handler: &'static (dyn interface::IRQHandler + Sync), } /// IRQContext token. @@ -61,17 +70,16 @@ pub mod interface { /// platform's interrupt controller. pub trait IRQManager { /// The IRQ number type depends on the implementation. - type IRQNumberType; + type IRQNumberType: Copy; /// Register a handler. fn register_handler( &self, - irq_number: Self::IRQNumberType, - descriptor: super::IRQDescriptor, + irq_handler_descriptor: super::IRQHandlerDescriptor, ) -> Result<(), &'static str>; /// Enable an interrupt in the controller. - fn enable(&self, irq_number: Self::IRQNumberType); + fn enable(&self, irq_number: &Self::IRQNumberType); /// Handle pending interrupts. /// @@ -91,16 +99,12 @@ pub mod interface { } } -/// A wrapper type for IRQ numbers with integrated range sanity check. -#[derive(Copy, Clone)] -pub struct IRQNumber(usize); - //-------------------------------------------------------------------------------------------------- // Global instances //-------------------------------------------------------------------------------------------------- static CUR_IRQ_MANAGER: InitStateLock< - &'static (dyn interface::IRQManager + Sync), + &'static (dyn interface::IRQManager + Sync), > = InitStateLock::new(&null_irq_manager::NULL_IRQ_MANAGER); //-------------------------------------------------------------------------------------------------- @@ -108,6 +112,39 @@ static CUR_IRQ_MANAGER: InitStateLock< //-------------------------------------------------------------------------------------------------- use synchronization::{interface::ReadWriteEx, InitStateLock}; +impl IRQHandlerDescriptor +where + T: Copy, +{ + /// Create an instance. + pub const fn new( + number: T, + name: &'static str, + handler: &'static (dyn interface::IRQHandler + Sync), + ) -> Self { + Self { + number, + name, + handler, + } + } + + /// Return the number. + pub const fn number(&self) -> T { + self.number + } + + /// Return the name. + pub const fn name(&self) -> &'static str { + self.name + } + + /// Return the handler. + pub const fn handler(&self) -> &'static (dyn interface::IRQHandler + Sync) { + self.handler + } +} + impl<'irq_context> IRQContext<'irq_context> { /// Creates an IRQContext token. /// @@ -125,29 +162,6 @@ impl<'irq_context> IRQContext<'irq_context> { } } -impl IRQNumber<{ MAX_INCLUSIVE }> { - /// The total number of IRQs this type supports. - pub const NUM_TOTAL: usize = MAX_INCLUSIVE + 1; - - /// Creates a new instance if number <= MAX_INCLUSIVE. - pub const fn new(number: usize) -> Self { - assert!(number <= MAX_INCLUSIVE); - - Self(number) - } - - /// Return the wrapped number. - pub const fn get(self) -> usize { - self.0 - } -} - -impl fmt::Display for IRQNumber<{ MAX_INCLUSIVE }> { - fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { - write!(f, "{}", self.0) - } -} - /// Executes the provided closure while IRQs are masked on the executing core. /// /// While the function temporarily changes the HW state of the executing core, it restores it to the @@ -163,8 +177,7 @@ pub fn exec_with_irq_masked(f: impl FnOnce() -> T) -> T { /// Register a new IRQ manager. pub fn register_irq_manager( - new_manager: &'static (dyn interface::IRQManager - + Sync), + new_manager: &'static (dyn interface::IRQManager + Sync), ) { CUR_IRQ_MANAGER.write(|manager| *manager = new_manager); } @@ -172,6 +185,6 @@ pub fn register_irq_manager( /// Return a reference to the currently registered IRQ manager. /// /// This is the IRQ manager used by the architectural interrupt handling code. -pub fn irq_manager() -> &'static dyn interface::IRQManager { +pub fn irq_manager() -> &'static dyn interface::IRQManager { CUR_IRQ_MANAGER.read(|manager| *manager) } diff --git a/19_kernel_heap/kernel/src/exception/asynchronous/null_irq_manager.rs b/19_kernel_heap/kernel/src/exception/asynchronous/null_irq_manager.rs index 560e3ce4..438f9649 100644 --- a/19_kernel_heap/kernel/src/exception/asynchronous/null_irq_manager.rs +++ b/19_kernel_heap/kernel/src/exception/asynchronous/null_irq_manager.rs @@ -4,8 +4,7 @@ //! Null IRQ Manager. -use super::{interface, IRQContext, IRQDescriptor}; -use crate::bsp; +use super::{interface, IRQContext, IRQHandlerDescriptor}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -24,17 +23,16 @@ pub static NULL_IRQ_MANAGER: NullIRQManager = NullIRQManager {}; //-------------------------------------------------------------------------------------------------- impl interface::IRQManager for NullIRQManager { - type IRQNumberType = bsp::driver::IRQNumber; + type IRQNumberType = super::IRQNumber; fn register_handler( &self, - _irq_number: Self::IRQNumberType, - _descriptor: IRQDescriptor, + _descriptor: IRQHandlerDescriptor, ) -> Result<(), &'static str> { panic!("No IRQ Manager registered yet"); } - fn enable(&self, _irq_number: Self::IRQNumberType) { + fn enable(&self, _irq_number: &Self::IRQNumberType) { panic!("No IRQ Manager registered yet"); } diff --git a/19_kernel_heap/kernel/src/lib.rs b/19_kernel_heap/kernel/src/lib.rs index 8e4195e0..bba0fcc7 100644 --- a/19_kernel_heap/kernel/src/lib.rs +++ b/19_kernel_heap/kernel/src/lib.rs @@ -186,11 +186,9 @@ pub fn test_runner(tests: &[&test_types::UnitTest]) { #[cfg(test)] #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); test_main(); diff --git a/19_kernel_heap/kernel/src/main.rs b/19_kernel_heap/kernel/src/main.rs index d2044c29..9a5fd58a 100644 --- a/19_kernel_heap/kernel/src/main.rs +++ b/19_kernel_heap/kernel/src/main.rs @@ -15,7 +15,7 @@ extern crate alloc; -use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; +use libkernel::{bsp, cpu, driver, exception, info, memory, state, time}; /// Early init code. /// @@ -27,27 +27,16 @@ use libkernel::{bsp, cpu, driver, exception, info, memory, state, time, warn}; /// - Printing will not work until the respective driver's MMIO is remapped. #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - // Instantiate and init all device drivers. - if let Err(x) = bsp::driver::driver_manager().instantiate_drivers() { - panic!("Error instantiating drivers: {}", x); - } - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - // Let device drivers register and enable their handlers with the interrupt controller. - for i in bsp::driver::driver_manager().all_device_drivers() { - if let Err(msg) = i.register_and_enable_irq_handler() { - warn!("Error registering IRQ handler: {}", msg); - } - } + // Initialize all device drivers. + driver::driver_manager().init_drivers_and_irqs(); bsp::memory::mmu::kernel_add_mapping_records_for_precomputed(); @@ -63,8 +52,6 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { - use driver::interface::DriverManager; - info!("{}", libkernel::version()); info!("Booting on: {}", bsp::board_name()); @@ -83,13 +70,7 @@ fn kernel_main() -> ! { ); info!("Drivers loaded:"); - for (i, driver) in bsp::driver::driver_manager() - .all_device_drivers() - .iter() - .enumerate() - { - info!(" {}. {}", i + 1, driver.compatible()); - } + driver::driver_manager().enumerate(); info!("Registered IRQ handlers:"); exception::asynchronous::irq_manager().print_handler(); diff --git a/19_kernel_heap/kernel/src/memory/heap_alloc.rs b/19_kernel_heap/kernel/src/memory/heap_alloc.rs index c290be4d..c0f56d8d 100644 --- a/19_kernel_heap/kernel/src/memory/heap_alloc.rs +++ b/19_kernel_heap/kernel/src/memory/heap_alloc.rs @@ -9,8 +9,10 @@ use crate::{ memory::{Address, Virtual}, synchronization, synchronization::IRQSafeNullLock, + warn, }; use alloc::alloc::{GlobalAlloc, Layout}; +use core::sync::atomic::{AtomicBool, Ordering}; use linked_list_allocator::Heap as LinkedListHeap; //-------------------------------------------------------------------------------------------------- @@ -129,9 +131,17 @@ unsafe impl GlobalAlloc for HeapAllocator { /// Query the BSP for the heap region and initialize the kernel's heap allocator with it. pub fn kernel_init_heap_allocator() { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + warn!("Already initialized"); + return; + } + let region = bsp::memory::mmu::virt_heap_region(); KERNEL_HEAP_ALLOCATOR.inner.lock(|inner| unsafe { inner.init(region.start_addr().as_usize() as *mut u8, region.size()) }); + + INIT_DONE.store(true, Ordering::Relaxed); } diff --git a/19_kernel_heap/kernel/tests/00_console_sanity.rs b/19_kernel_heap/kernel/tests/00_console_sanity.rs index 305510ce..2c0225b7 100644 --- a/19_kernel_heap/kernel/tests/00_console_sanity.rs +++ b/19_kernel_heap/kernel/tests/00_console_sanity.rs @@ -11,16 +11,15 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{bsp, console, cpu, driver, exception, memory, print}; +use libkernel::{bsp, console, cpu, exception, memory, print}; #[no_mangle] unsafe fn kernel_init() -> ! { use console::console; - use driver::interface::DriverManager; exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Handshake assert_eq!(console().read_char(), 'A'); diff --git a/19_kernel_heap/kernel/tests/01_timer_sanity.rs b/19_kernel_heap/kernel/tests/01_timer_sanity.rs index 39899332..8188b942 100644 --- a/19_kernel_heap/kernel/tests/01_timer_sanity.rs +++ b/19_kernel_heap/kernel/tests/01_timer_sanity.rs @@ -11,16 +11,14 @@ #![test_runner(libkernel::test_runner)] use core::time::Duration; -use libkernel::{bsp, cpu, driver, exception, memory, time}; +use libkernel::{bsp, cpu, exception, memory, time}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // Depending on CPU arch, some timer bring-up code could go here. Not needed for the RPi. diff --git a/19_kernel_heap/kernel/tests/02_exception_sync_page_fault.rs b/19_kernel_heap/kernel/tests/02_exception_sync_page_fault.rs index a6d15b69..fab44c8f 100644 --- a/19_kernel_heap/kernel/tests/02_exception_sync_page_fault.rs +++ b/19_kernel_heap/kernel/tests/02_exception_sync_page_fault.rs @@ -17,15 +17,13 @@ /// or indirectly. mod panic_exit_success; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing synchronous exception handling by causing a page fault"); diff --git a/19_kernel_heap/kernel/tests/03_exception_restore_sanity.rs b/19_kernel_heap/kernel/tests/03_exception_restore_sanity.rs index 47dd2714..f176c6a6 100644 --- a/19_kernel_heap/kernel/tests/03_exception_restore_sanity.rs +++ b/19_kernel_heap/kernel/tests/03_exception_restore_sanity.rs @@ -12,7 +12,7 @@ mod panic_wait_forever; use core::arch::asm; -use libkernel::{bsp, cpu, driver, exception, info, memory, println}; +use libkernel::{bsp, cpu, exception, info, memory, println}; #[inline(never)] fn nested_system_call() { @@ -30,11 +30,9 @@ fn nested_system_call() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); // This line will be printed as the test header. println!("Testing exception restore"); diff --git a/19_kernel_heap/kernel/tests/04_exception_irq_sanity.rs b/19_kernel_heap/kernel/tests/04_exception_irq_sanity.rs index e93a4caf..e6f94c91 100644 --- a/19_kernel_heap/kernel/tests/04_exception_irq_sanity.rs +++ b/19_kernel_heap/kernel/tests/04_exception_irq_sanity.rs @@ -10,15 +10,13 @@ #![reexport_test_harness_main = "test_main"] #![test_runner(libkernel::test_runner)] -use libkernel::{bsp, cpu, driver, exception, memory}; +use libkernel::{bsp, cpu, exception, memory}; use test_macros::kernel_test; #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); exception::handling_init(); exception::asynchronous::local_irq_unmask(); diff --git a/19_kernel_heap/kernel/tests/05_backtrace_sanity.rs b/19_kernel_heap/kernel/tests/05_backtrace_sanity.rs index ea425f68..f75c0ea3 100644 --- a/19_kernel_heap/kernel/tests/05_backtrace_sanity.rs +++ b/19_kernel_heap/kernel/tests/05_backtrace_sanity.rs @@ -11,7 +11,7 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{bsp, cpu, driver, exception, memory}; +use libkernel::{bsp, cpu, exception, memory}; #[inline(never)] fn nested() { @@ -20,11 +20,9 @@ fn nested() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); nested(); diff --git a/19_kernel_heap/kernel/tests/06_backtrace_invalid_frame.rs b/19_kernel_heap/kernel/tests/06_backtrace_invalid_frame.rs index 8769c7b6..33d3c02d 100644 --- a/19_kernel_heap/kernel/tests/06_backtrace_invalid_frame.rs +++ b/19_kernel_heap/kernel/tests/06_backtrace_invalid_frame.rs @@ -11,7 +11,7 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{backtrace, bsp, cpu, driver, exception, memory}; +use libkernel::{backtrace, bsp, cpu, exception, memory}; #[inline(never)] fn nested() { @@ -22,11 +22,9 @@ fn nested() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); nested(); diff --git a/19_kernel_heap/kernel/tests/07_backtrace_invalid_link.rs b/19_kernel_heap/kernel/tests/07_backtrace_invalid_link.rs index 28f3cdda..bcb0538a 100644 --- a/19_kernel_heap/kernel/tests/07_backtrace_invalid_link.rs +++ b/19_kernel_heap/kernel/tests/07_backtrace_invalid_link.rs @@ -11,7 +11,7 @@ /// Console tests should time out on the I/O harness in case of panic. mod panic_wait_forever; -use libkernel::{backtrace, bsp, cpu, driver, exception, memory}; +use libkernel::{backtrace, bsp, cpu, exception, memory}; #[inline(never)] fn nested_2() -> &'static str { @@ -27,11 +27,9 @@ fn nested_1() { #[no_mangle] unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - exception::handling_init(); memory::init(); - bsp::driver::driver_manager().qemu_bring_up_console(); + bsp::driver::qemu_bring_up_console(); nested_1(); diff --git a/X1_JTAG_boot/src/bsp/raspberrypi.rs b/X1_JTAG_boot/src/bsp/raspberrypi.rs index eb6be81a..7bda8a4d 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi.rs @@ -4,7 +4,6 @@ //! Top-level BSP file for the Raspberry Pi 3 and 4. -pub mod console; pub mod cpu; pub mod driver; pub mod memory; diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/console.rs b/X1_JTAG_boot/src/bsp/raspberrypi/console.rs deleted file mode 100644 index 0a630eef..00000000 --- a/X1_JTAG_boot/src/bsp/raspberrypi/console.rs +++ /dev/null @@ -1,16 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2022 Andre Richter - -//! BSP console facilities. - -use crate::console; - -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- - -/// Return a reference to the console. -pub fn console() -> &'static dyn console::interface::All { - &super::driver::PL011_UART -} diff --git a/X1_JTAG_boot/src/bsp/raspberrypi/driver.rs b/X1_JTAG_boot/src/bsp/raspberrypi/driver.rs index 8b683ed8..ea843066 100644 --- a/X1_JTAG_boot/src/bsp/raspberrypi/driver.rs +++ b/X1_JTAG_boot/src/bsp/raspberrypi/driver.rs @@ -5,51 +5,67 @@ //! BSP driver support. use super::memory::map::mmio; -use crate::{bsp::device_driver, driver}; +use crate::{bsp::device_driver, console, driver as generic_driver}; +use core::sync::atomic::{AtomicBool, Ordering}; //-------------------------------------------------------------------------------------------------- -// Private Definitions +// Global instances //-------------------------------------------------------------------------------------------------- -/// Device Driver Manager type. -struct BSPDriverManager { - device_drivers: [&'static (dyn DeviceDriver + Sync); 2], -} +static PL011_UART: device_driver::PL011Uart = + unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; //-------------------------------------------------------------------------------------------------- -// Global instances +// Private Code //-------------------------------------------------------------------------------------------------- -pub(super) static PL011_UART: device_driver::PL011Uart = - unsafe { device_driver::PL011Uart::new(mmio::PL011_UART_START) }; +/// This must be called only after successful init of the UART driver. +fn post_init_uart() -> Result<(), &'static str> { + console::register_console(&PL011_UART); -static GPIO: device_driver::GPIO = unsafe { device_driver::GPIO::new(mmio::GPIO_START) }; + Ok(()) +} -static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { - device_drivers: [&PL011_UART, &GPIO], -}; +/// This must be called only after successful init of the GPIO driver. +fn post_init_gpio() -> Result<(), &'static str> { + GPIO.map_pl011_uart(); + Ok(()) +} -//-------------------------------------------------------------------------------------------------- -// Public Code -//-------------------------------------------------------------------------------------------------- +fn driver_uart() -> Result<(), &'static str> { + let uart_descriptor = + generic_driver::DeviceDriverDescriptor::new(&PL011_UART, Some(post_init_uart)); + generic_driver::driver_manager().register_driver(uart_descriptor); -/// Return a reference to the driver manager. -pub fn driver_manager() -> &'static impl driver::interface::DriverManager { - &BSP_DRIVER_MANAGER + Ok(()) } -//------------------------------------------------------------------------------ -// OS Interface Code -//------------------------------------------------------------------------------ -use driver::interface::DeviceDriver; +fn driver_gpio() -> Result<(), &'static str> { + let gpio_descriptor = generic_driver::DeviceDriverDescriptor::new(&GPIO, Some(post_init_gpio)); + generic_driver::driver_manager().register_driver(gpio_descriptor); -impl driver::interface::DriverManager for BSPDriverManager { - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { - &self.device_drivers[..] - } + Ok(()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- - fn post_device_driver_init(&self) { - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); +/// Initialize the driver subsystem. +/// +/// # Safety +/// +/// See child function calls. +pub unsafe fn init() -> Result<(), &'static str> { + static INIT_DONE: AtomicBool = AtomicBool::new(false); + if INIT_DONE.load(Ordering::Relaxed) { + return Err("Init already done"); } + + driver_uart()?; + driver_gpio()?; + + INIT_DONE.store(true, Ordering::Relaxed); + Ok(()) } diff --git a/X1_JTAG_boot/src/console.rs b/X1_JTAG_boot/src/console.rs index c1fb0e53..02b43df9 100644 --- a/X1_JTAG_boot/src/console.rs +++ b/X1_JTAG_boot/src/console.rs @@ -4,7 +4,9 @@ //! System console. -use crate::bsp; +mod null_console; + +use crate::synchronization::{self, NullLock}; //-------------------------------------------------------------------------------------------------- // Public Definitions @@ -54,13 +56,26 @@ pub mod interface { pub trait All: Write + Read + Statistics {} } +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static CUR_CONSOLE: NullLock<&'static (dyn interface::All + Sync)> = + NullLock::new(&null_console::NULL_CONSOLE); + //-------------------------------------------------------------------------------------------------- // Public Code //-------------------------------------------------------------------------------------------------- +use synchronization::interface::Mutex; + +/// Register a new console. +pub fn register_console(new_console: &'static (dyn interface::All + Sync)) { + CUR_CONSOLE.lock(|con| *con = new_console); +} -/// Return a reference to the console. +/// Return a reference to the currently registered console. /// /// This is the global console used by all printing macros. pub fn console() -> &'static dyn interface::All { - bsp::console::console() + CUR_CONSOLE.lock(|con| *con) } diff --git a/X1_JTAG_boot/src/console/null_console.rs b/X1_JTAG_boot/src/console/null_console.rs new file mode 100644 index 00000000..2c64d499 --- /dev/null +++ b/X1_JTAG_boot/src/console/null_console.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2022 Andre Richter + +//! Null console. + +use super::interface; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +pub struct NullConsole; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +pub static NULL_CONSOLE: NullConsole = NullConsole {}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl interface::Write for NullConsole { + fn write_char(&self, _c: char) {} + + fn write_fmt(&self, _args: fmt::Arguments) -> fmt::Result { + fmt::Result::Ok(()) + } + + fn flush(&self) {} +} + +impl interface::Read for NullConsole { + fn clear_rx(&self) {} +} + +impl interface::Statistics for NullConsole {} +impl interface::All for NullConsole {} diff --git a/X1_JTAG_boot/src/driver.rs b/X1_JTAG_boot/src/driver.rs index 2fcc7562..fb44bbd9 100644 --- a/X1_JTAG_boot/src/driver.rs +++ b/X1_JTAG_boot/src/driver.rs @@ -4,6 +4,19 @@ //! Driver support. +use crate::synchronization::{interface::Mutex, NullLock}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +const NUM_DRIVERS: usize = 5; + +struct DriverManagerInner { + next_index: usize, + descriptors: [Option; NUM_DRIVERS], +} + //-------------------------------------------------------------------------------------------------- // Public Definitions //-------------------------------------------------------------------------------------------------- @@ -24,21 +37,118 @@ pub mod interface { Ok(()) } } +} + +/// Tpye to be used as an optional callback after a driver's init() has run. +pub type DeviceDriverPostInitCallback = unsafe fn() -> Result<(), &'static str>; - /// Device driver management functions. +/// A descriptor for device drivers. +#[derive(Copy, Clone)] +pub struct DeviceDriverDescriptor { + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, +} + +/// Provides device driver management functions. +pub struct DriverManager { + inner: NullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static DRIVER_MANAGER: DriverManager = DriverManager::new(); + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DriverManagerInner { + /// Create an instance. + pub const fn new() -> Self { + Self { + next_index: 0, + descriptors: [None; NUM_DRIVERS], + } + } +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl DeviceDriverDescriptor { + /// Create an instance. + pub fn new( + device_driver: &'static (dyn interface::DeviceDriver + Sync), + post_init_callback: Option, + ) -> Self { + Self { + device_driver, + post_init_callback, + } + } +} + +/// Return a reference to the global DriverManager. +pub fn driver_manager() -> &'static DriverManager { + &DRIVER_MANAGER +} + +impl DriverManager { + /// Create an instance. + pub const fn new() -> Self { + Self { + inner: NullLock::new(DriverManagerInner::new()), + } + } + + /// Register a device driver with the kernel. + pub fn register_driver(&self, descriptor: DeviceDriverDescriptor) { + self.inner.lock(|inner| { + inner.descriptors[inner.next_index] = Some(descriptor); + inner.next_index += 1; + }) + } + + /// Helper for iterating over registered drivers. + fn for_each_descriptor<'a>(&'a self, f: impl FnMut(&'a DeviceDriverDescriptor)) { + self.inner.lock(|inner| { + inner + .descriptors + .iter() + .filter_map(|x| x.as_ref()) + .for_each(f) + }) + } + + /// Fully initialize all drivers. /// - /// The `BSP` is supposed to supply one global instance. - pub trait DriverManager { - /// Return a slice of references to all `BSP`-instantiated drivers. - /// - /// # Safety - /// - /// - The order of devices is the order in which `DeviceDriver::init()` is called. - fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + /// # Safety + /// + /// - During init, drivers might do stuff with system-wide impact. + pub unsafe fn init_drivers(&self) { + self.for_each_descriptor(|descriptor| { + // 1. Initialize driver. + if let Err(x) = descriptor.device_driver.init() { + panic!( + "Error initializing driver: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } - /// Initialization code that runs after driver init. - /// - /// For example, device driver code that depends on other drivers already being online. - fn post_device_driver_init(&self); + // 2. Call corresponding post init callback. + if let Some(callback) = &descriptor.post_init_callback { + if let Err(x) = callback() { + panic!( + "Error during driver post-init callback: {}: {}", + descriptor.device_driver.compatible(), + x + ); + } + } + }); } } diff --git a/X1_JTAG_boot/src/main.rs b/X1_JTAG_boot/src/main.rs index a38b1ef5..8d244c77 100644 --- a/X1_JTAG_boot/src/main.rs +++ b/X1_JTAG_boot/src/main.rs @@ -133,14 +133,13 @@ mod time; /// - Only a single core must be active and running this function. /// - The init calls in this function must appear in the correct order. unsafe fn kernel_init() -> ! { - use driver::interface::DriverManager; - - for i in bsp::driver::driver_manager().all_device_drivers().iter() { - if let Err(x) = i.init() { - panic!("Error loading driver: {}: {}", i.compatible(), x); - } + // Initialize the BSP driver subsystem. + if let Err(x) = bsp::driver::init() { + panic!("Error initializing BSP driver subsystem: {}", x); } - bsp::driver::driver_manager().post_device_driver_init(); + + // Initialize all device drivers. + driver::driver_manager().init_drivers(); // println! is usable from here on. // Transition from unsafe to safe. diff --git a/rust-toolchain.toml b/rust-toolchain.toml index 0a525545..a554c0a9 100644 --- a/rust-toolchain.toml +++ b/rust-toolchain.toml @@ -1,4 +1,4 @@ [toolchain] -channel = "nightly-2022-08-01" +channel = "nightly-2022-10-13" components = ["rust-src", "llvm-tools-preview", "rustfmt"] targets = ["aarch64-unknown-none-softfloat"]