- 21 Apr, 2023 9 commits
-
-
Wedson Almeida Filho authored
This is the traditional condition variable or monitor synchronisation primitive. It is implemented with C's `wait_queue_head_t`. It allows users to release a lock and go to sleep while guaranteeing that notifications won't be missed. This is achieved by enqueuing a wait entry before releasing the lock. Cc: Peter Zijlstra <peterz@infradead.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Will Deacon <will@kernel.org> Cc: Waiman Long <longman@redhat.com> Reviewed-by:
Martin Rodriguez Reboredo <yakoyoku@gmail.com> Signed-off-by:
Wedson Almeida Filho <walmeida@microsoft.com> Reviewed-by:
Alice Ryhl <aliceryhl@google.com> Link: https://lore.kernel.org/r/20230411054543.21278-12-wedsonaf@gmail.comSigned-off-by:
Miguel Ojeda <ojeda@kernel.org>
-
Wedson Almeida Filho authored
It releases the lock, executes some function provided by the caller, then reacquires the lock. This is preparation for the implementation of condvars, which will sleep after between unlocking and relocking. We need an explicit `relock` method for primitives like `SpinLock` that have an irqsave variant: we use the guard state to determine if the lock was originally acquired with the regular `lock` function or `lock_irqsave`. Reviewed-by:
-
Wedson Almeida Filho authored
This allows us to have data protected by a lock despite not being wrapped by it. Access is granted by providing evidence that the lock is held by the caller. Reviewed-by:
Benno Lossin <benno.lossin@proton.me> Link: https://lore.kernel.org/r/20230411054543.21278-13-wedsonaf@gmail.comSigned-off-by:
-
Wedson Almeida Filho authored
This allows Rust code to get a reference to the current task without having to increment the refcount, but still guaranteeing memory safety. Cc: Ingo Molnar <mingo@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Reviewed-by:
-
Wedson Almeida Filho authored
It is an abstraction for C's `struct task_struct`. It implements `AlwaysRefCounted`, so the refcount of the wrapped object is managed safely on the Rust side. Cc: Ingo Molnar <mingo@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Reviewed-by:
-
Wedson Almeida Filho authored
This is an owned reference to an object that is always ref-counted. This is meant to be used in wrappers for C types that have their own ref counting functions, for example, tasks, files, inodes, dentries, etc. Reviewed-by:
Gary Guo <gary@garyguo.net> Link: https://lore.kernel.org/r/20230411054543.21278-8-wedsonaf@gmail.comSigned-off-by:
-
Wedson Almeida Filho authored
This is the `spinlock_t` lock backend and allows Rust code to use the kernel spinlock idiomatically. Cc: Peter Zijlstra <peterz@infradead.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Will Deacon <will@kernel.org> Cc: Waiman Long <longman@redhat.com> Reviewed-by:
-
Wedson Almeida Filho authored
This is the `struct mutex` lock backend and allows Rust code to use the kernel mutex idiomatically. Cc: Peter Zijlstra <peterz@infradead.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Will Deacon <will@kernel.org> Cc: Waiman Long <longman@redhat.com> Reviewed-by:
-
Wedson Almeida Filho authored
They are generic Rust implementations of a lock and a lock guard that contain code that is common to all locks. Different backends will be introduced in subsequent commits. Reviewed-by:
-
- 20 Apr, 2023 3 commits
-
-
Wedson Almeida Filho authored
It is a wrapper around C's `lock_class_key`, which is used by the synchronisation primitives that are checked with lockdep. This is in preparation for introducing Rust abstractions for these primitives. Cc: Peter Zijlstra <peterz@infradead.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Will Deacon <will@kernel.org> Cc: Waiman Long <longman@redhat.com> Reviewed-by:
Boqun Feng <boqun.feng@gmail.com> Signed-off-by:
-
Miguel Ojeda authored
Benno has been involved with the Rust for Linux project for the better part of a year now. He has been working on solving the safe pinned initialization problem [1], which resulted in the pin-init API patch series [2] that allows to reduce the need for `unsafe` code in the kernel. He is also working on the field projection RFC for Rust [3] to bring pin-init as a language feature. His expertise with the language will be very useful to have around in the future if Rust grows within the kernel, thus add him to the `RUST` entry as reviewer. Link: https://rust-for-linux.com/the-safe-pinned-initialization-problem [1] Link: https://lore.kernel.org/rust-for-linux/20230408122429.1103522-1-y86-dev@protonmail.com/ [2] Link: https://github.com/rust-lang/rfcs/pull/3318 [3] Cc: Benno Lossin <benno.lossin@proton.me> Reviewed-by:
Alex Gaynor <alex.gaynor@gmail.com> Link: https://lore.kernel.org/r/20230412221823.830135-1-ojeda@kernel.orgSigned-off-by:
-
Benno Lossin authored
This makes it possible to use `T` as a `impl Init<T, E>` for every error type `E` instead of just `Infallible`. Signed-off-by:
-
- 12 Apr, 2023 21 commits
-
-
Benno Lossin authored
Add two functions `init_with` and `pin_init_with` to `UniqueArc<MaybeUninit<T>>` to initialize the memory of already allocated `UniqueArc`s. This is useful when you want to allocate memory check some condition inside of a context where allocation is forbidden and then conditionally initialize an object. Signed-off-by:
Andreas Hindborg <a.hindborg@samsung.com> Link: https://lore.kernel.org/r/20230408122429.1103522-16-y86-dev@protonmail.comSigned-off-by:
-
Benno Lossin authored
`UniqueArc::try_new_uninit` calls `Arc::try_new(MaybeUninit::uninit())`. This results in the uninitialized memory being placed on the stack, which may be arbitrarily large due to the generic `T` and thus could cause a stack overflow for large types. Change the implementation to use the pin-init API which enables in-place initialization. In particular it avoids having to first construct and then move the uninitialized memory from the stack into the final location. Signed-off-by:
-
Benno Lossin authored
This function allows to easily initialize `Opaque` with the pin-init API. `Opaque::ffi_init` takes a closure and returns a pin-initializer. This pin-initiailizer calls the given closure with a pointer to the inner `T`. Co-developed-by:
-
Benno Lossin authored
Add `pin-init` API macros and traits to the prelude. Signed-off-by:
-
Benno Lossin authored
Add the `Zeroable` trait which marks types that can be initialized by writing `0x00` to every byte of the type. Also add the `init::zeroed` function that creates an initializer for a `Zeroable` type that writes `0x00` to every byte. Signed-off-by:
-
Benno Lossin authored
The `stack_pin_init!` macro allows pin-initializing a value on the stack. It accepts a `impl PinInit<T, E>` to initialize a `T`. It allows propagating any errors via `?` or handling it normally via `match`. Signed-off-by:
-
Benno Lossin authored
The `PinnedDrop` trait that facilitates destruction of pinned types. It has to be implemented via the `#[pinned_drop]` macro, since the `drop` function should not be called by normal code, only by other destructors. It also only works on structs that are annotated with `#[pin_data(PinnedDrop)]`. Co-developed-by:
-
Benno Lossin authored
The `InPlaceInit` trait that provides two functions, for initializing using `PinInit<T, E>` and `Init<T>`. It is implemented by `Arc<T>`, `UniqueArc<T>` and `Box<T>`. Signed-off-by:
-
Benno Lossin authored
Add the following initializer macros: - `#[pin_data]` to annotate structurally pinned fields of structs, needed for `pin_init!` and `try_pin_init!` to select the correct initializer of fields. - `pin_init!` create a pin-initializer for a struct with the `Infallible` error type. - `try_pin_init!` create a pin-initializer for a struct with a custom error type (`kernel::error::Error` is the default). - `init!` create an in-place-initializer for a struct with the `Infallible` error type. - `try_init!` create an in-place-initializer for a struct with a custom error type (`kernel::error::Error` is the default). Also add their needed internal helper traits and structs. Co-developed-by:
-
Benno Lossin authored
This API is used to facilitate safe pinned initialization of structs. It replaces cumbersome `unsafe` manual initialization with elegant safe macro invocations. Due to the size of this change it has been split into six commits: 1. This commit introducing the basic public interface: traits and functions to represent and create initializers. 2. Adds the `#[pin_data]`, `pin_init!`, `try_pin_init!`, `init!` and `try_init!` macros along with their internal types. 3. Adds the `InPlaceInit` trait that allows using an initializer to create an object inside of a `Box<T>` and other smart pointers. 4. Adds the `PinnedDrop` trait and adds macro support for it in the `#[pin_data]` macro. 5. Adds the `stack_pin_init!` macro allowing to pin-initialize a struct on the stack. 6. Adds the `Zeroable` trait and `init::zeroed` function to initialize types that have `0x00` in all bytes as a valid bit pattern. -- In this section the problem that the new pin-init API solves is outlined. This message describes the entirety of the API, not just the parts introduced in this commit. For a more granular explanation and additional information on pinning and this issue, view [1]. Pinning is Rust's way of enforcing the address stability of a value. When a value gets pinned it will be impossible for safe code to move it to another location. This is done by wrapping pointers to said object with `Pin<P>`. This wrapper prevents safe code from creating mutable references to the object, preventing mutable access, which is needed to move the value. `Pin<P>` provides `unsafe` functions to circumvent this and allow modifications regardless. It is then the programmer's responsibility to uphold the pinning guarantee. Many kernel data structures require a stable address, because there are foreign pointers to them which would get invalidated by moving the structure. Since these data structures are usually embedded in structs to use them, this pinning property propagates to the container struct. Resulting in most structs in both Rust and C code needing to be pinned. So if we want to have a `mutex` field in a Rust struct, this struct also needs to be pinned, because a `mutex` contains a `list_head`. Additionally initializing a `list_head` requires already having the final memory location available, because it is initialized by pointing it to itself. But this presents another challenge in Rust: values have to be initialized at all times. There is the `MaybeUninit<T>` wrapper type, which allows handling uninitialized memory, but this requires using the `unsafe` raw pointers and a casting the type to the initialized variant. This problem gets exacerbated when considering encapsulation and the normal safety requirements of Rust code. The fields of the Rust `Mutex<T>` should not be accessible to normal driver code. After all if anyone can modify the fields, there is no way to ensure the invariants of the `Mutex<T>` are upheld. But if the fields are inaccessible, then initialization of a `Mutex<T>` needs to be somehow achieved via a function or a macro. Because the `Mutex<T>` must be pinned in memory, the function cannot return it by value. It also cannot allocate a `Box` to put the `Mutex<T>` into, because that is an unnecessary allocation and indirection which would hurt performance. The solution in the rust tree (e.g. this commit: [2]) that is replaced by this API is to split this function into two parts: 1. A `new` function that returns a partially initialized `Mutex<T>`, 2. An `init` function that requires the `Mutex<T>` to be pinned and that fully initializes the `Mutex<T>`. Both of these functions have to be marked `unsafe`, since a call to `new` needs to be accompanied with a call to `init`, otherwise using the `Mutex<T>` could result in UB. And because calling `init` twice also is not safe. While `Mutex<T>` initialization cannot fail, other structs might also have to allocate memory, which would result in conditional successful initialization requiring even more manual accommodation work. Combine this with the problem of pin-projections -- the way of accessing fields of a pinned struct -- which also have an `unsafe` API, pinned initialization is riddled with `unsafe` resulting in very poor ergonomics. Not only that, but also having to call two functions possibly multiple lines apart makes it very easy to forget it outright or during refactoring. Here is an example of the current way of initializing a struct with two synchronization primitives (see [3] for the full example): struct SharedState { state_changed: CondVar, inner: Mutex<SharedStateInner>, } impl SharedState { fn try_new() -> Result<Arc<Self>> { let mut state = Pin::from(UniqueArc::try_new(Self { // SAFETY: `condvar_init!` is called below. state_changed: unsafe { CondVar::new() }, // SAFETY: `mutex_init!` is called below. inner: unsafe { Mutex::new(SharedStateInner { token_count: 0 }) }, })?); // SAFETY: `state_changed` is pinned when `state` is. let pinned = unsafe { state.as_mut().map_unchecked_mut(|s| &mut s.state_changed) }; kernel::condvar_init!(pinned, "SharedState::state_changed"); // SAFETY: `inner` is pinned when `state` is. let pinned = unsafe { state.as_mut().map_unchecked_mut(|s| &mut s.inner) }; kernel::mutex_init!(pinned, "SharedState::inner"); Ok(state.into()) } } The pin-init API of this patch solves this issue by providing a comprehensive solution comprised of macros and traits. Here is the example from above using the pin-init API: #[pin_data] struct SharedState { #[pin] state_changed: CondVar, #[pin] inner: Mutex<SharedStateInner>, } impl SharedState { fn new() -> impl PinInit<Self> { pin_init!(Self { state_changed <- new_condvar!("SharedState::state_changed"), inner <- new_mutex!( SharedStateInner { token_count: 0 }, "SharedState::inner", ), }) } } Notably the way the macro is used here requires no `unsafe` and thus comes with the usual Rust promise of safe code not introducing any memory violations. Additionally it is now up to the caller of `new()` to decide the memory location of the `SharedState`. They can choose at the moment `Arc<T>`, `Box<T>` or the stack. -- The API has the following architecture: 1. Initializer traits `PinInit<T, E>` and `Init<T, E>` that act like closures. 2. Macros to create these initializer traits safely. 3. Functions to allow manually writing initializers. The initializers (an `impl PinInit<T, E>`) receive a raw pointer pointing to uninitialized memory and their job is to fully initialize a `T` at that location. If initialization fails, they return an error (`E`) by value. This way of initializing cannot be safely exposed to the user, since it relies upon these properties outside of the control of the trait: - the memory location (slot) needs to be valid memory, - if initialization fails, the slot should not be read from, - the value in the slot should be pinned, so it cannot move and the memory cannot be deallocated until the value is dropped. This is why using an initializer is facilitated by another trait that ensures these requirements. These initializers can be created manually by just supplying a closure that fulfills the same safety requirements as `PinInit<T, E>`. But this is an `unsafe` operation. To allow safe initializer creation, the `pin_init!` is provided along with three other variants: `try_pin_init!`, `try_init!` and `init!`. These take a modified struct initializer as a parameter and generate a closure that initializes the fields in sequence. The macros take great care in upholding the safety requirements: - A shadowed struct type is used as the return type of the closure instead of `()`. This is to prevent early returns, as these would prevent full initialization. - To ensure every field is only initialized once, a normal struct initializer is placed in unreachable code. The type checker will emit errors if a field is missing or specified multiple times. - When initializing a field fails, the whole initializer will fail and automatically drop fields that have been initialized earlier. - Only the correct initializer type is allowed for unpinned fields. You cannot use a `impl PinInit<T, E>` to initialize a structurally not pinned field. To ensure the last point, an additional macro `#[pin_data]` is needed. This macro annotates the struct itself and the user specifies structurally pinned and not pinned fields. Because dropping a pinned struct is also not allowed to break the pinning invariants, another macro attribute `#[pinned_drop]` is needed. This macro is introduced in a following commit. These two macros also have mechanisms to ensure the overall safety of the API. Additionally, they utilize a combined proc-macro, declarative macro design: first a proc-macro enables the outer attribute syntax `#[...]` and does some important pre-parsing. Notably this prepares the generics such that the declarative macro can handle them using token trees. Then the actual parsing of the structure and the emission of code is handled by a declarative macro. For pin-projections the crates `pin-project` [4] and `pin-project-lite` [5] had been considered, but were ultimately rejected: - `pin-project` depends on `syn` [6] which is a very big dependency, around 50k lines of code. - `pin-project-lite` is a more reasonable 5k lines of code, but contains a very complex declarative macro to parse generics. On top of that it would require modification that would need to be maintained independently. Link: https://rust-for-linux.com/the-safe-pinned-initialization-problem [1] Link: https://github.com/Rust-for-Linux/linux/tree/0a04dc4ddd671efb87eef54dde0fb38e9074f4be [2] Link: https://github.com/Rust-for-Linux/linux/blob/f509ede33fc10a07eba3da14aa00302bd4b5dddd/samples/rust/rust_miscdev.rs [3] Link: https://crates.io/crates/pin-project [4] Link: https://crates.io/crates/pin-project-lite [5] Link: https://crates.io/crates/syn [6] Co-developed-by:Wedson Almeida Filho <wedsonaf@gmail.com> Reviewed-by:
-
Benno Lossin authored
This function mirrors `UnsafeCell::raw_get`. It avoids creating a reference and allows solely using raw pointers. The `pin-init` API will be using this, since uninitialized memory requires raw pointers. Signed-off-by:
-
Benno Lossin authored
Change the error type of the constructors of `Arc` and `UniqueArc` to be `AllocError` instead of `Error`. This makes the API more clear as to what can go wrong when calling `try_new` or its variants. Signed-off-by:
-
Gary Guo authored
Add the `quote!` macro for creating `TokenStream`s directly via the given Rust tokens. It also supports repetitions using iterators. It will be used by the pin-init API proc-macros to generate code. Signed-off-by:
-
Benno Lossin authored
This feature enables the use of the `pin!` macro for the `stack_pin_init!` macro. This feature is already stabilized in Rust version 1.68. Signed-off-by:
-
Wedson Almeida Filho authored
Add a helper function to easily return C result codes from a Rust function that calls functions which return a Result<T>. Lina: Imported from rust-for-linux/rust, originally developed by Wedson as part of file_operations.rs. Added the allow() flags since there is no user in the kernel crate yet and fixed a typo in a comment. Replaced the macro with a function taking a closure, per discussion on the ML. Co-developed-by:
Fox Chen <foxhlchen@gmail.com> Signed-off-by:
Asahi Lina <lina@asahilina.net> Link: https://lore.kernel.org/r/20230224-rust-error-v3-6-03779bddc02b@asahilina.netSigned-off-by:
-
Sven Van Asbroeck authored
Some kernel C API functions return a pointer which embeds an optional `errno`. Callers are supposed to check the returned pointer with `IS_ERR()` and if this returns `true`, retrieve the `errno` using `PTR_ERR()`. Create a Rust helper function to implement the Rust equivalent: transform a `*mut T` to `Result<*mut T>`. Lina: Imported from rust-for-linux/linux, with subsequent refactoring and contributions squashed in and attributed below. Renamed the function to from_err_ptr(). Co-developed-by:
Sven Van Asbroeck <thesven73@gmail.com> Reviewed-by:
-
Wedson Almeida Filho authored
Add a to_result() helper to convert kernel C return values to a Rust Result, mapping >=0 values to Ok(()) and negative values to Err(...), with Error::from_errno() ensuring that the errno is within range. Lina: Imported from rust-for-linux/rust, originally developed by Wedson as part of the AMBA device driver support. Signed-off-by:
-
Miguel Ojeda authored
Add a function to create `Error` values out of a kernel error return, which safely upholds the invariant that the error code is well-formed (negative and greater than -MAX_ERRNO). If a malformed code is passed in, it will be converted to EINVAL. Lina: Imported from rust-for-linux/rust as authored by Miguel and Fox with refactoring from Wedson, renamed from_kernel_errno() to from_errno(). Co-developed-by:
-
Asahi Lina authored
This is the Rust equivalent to ERR_PTR(), for use in C callbacks. Marked as #[allow(dead_code)] for now, since it does not have any consumers yet. Reviewed-by:
-
Asahi Lina authored
This is kernel code, so specifying "kernel" is redundant. Let's simplify things and just call it to_errno(). Reviewed-by:
-
Asahi Lina authored
We can already create `UniqueArc<MaybeUninit<T>>` instances with `UniqueArc::try_new_uninit()` and write to them with `write()`. Add the missing unsafe `assume_init()` function to promote it to `UniqueArc<T>`, so users can do piece-wise initialization of the contents instead of doing it all at once as long as they keep the invariants (the same requirements as `MaybeUninit::assume_init()`). This mirrors the std `Arc::assume_init()` function. In the kernel, since we have `UniqueArc`, arguably this only belongs there since most use cases will initialize it immediately after creating it, before demoting it to `Arc` to share it. [ Miguel: The "Rust pin-init API for pinned initialization of structs" patch series [1] from Benno Lossin contains a very similar patch: rust: sync: add `assume_init` to `UniqueArc` Adds the `assume_init` function to `UniqueArc<MaybeUninit<T>>` that unsafely assumes the value to be initialized and yields a value of type `UniqueArc<T>`. This function is used when manually initializing the pointee of an `UniqueArc`. To make that patch a noop and thus drop it, I adjusted the `SAFETY` comment here to be the same as in the current latest version of that series (v7). I have also brought the `Reviewed-by`s there into here, and reworded the `Co-authored-by` into `Co-developed-by`. ] Link: https://lore.kernel.org/r/20230408122429.1103522-5-y86-dev@protonmail.com [1] Co-developed-by:
-
- 10 Apr, 2023 7 commits
-
-
Asahi Lina authored
This mirrors the standard library's alloc::sync::Arc::downcast(). Based on the Rust standard library implementation, ver 1.62.0, licensed under "Apache-2.0 OR MIT", from: https://github.com/rust-lang/rust/tree/1.62.0/library/alloc/src For copyright details, please see: https://github.com/rust-lang/rust/blob/1.62.0/COPYRIGHTReviewed-by:Vincenzo Palazzo <vincenzopalazzodev@gmail.com> Reviewed-by:
-
Asahi Lina authored
Modules can (and usually do) have multiple alias tags, in order to specify multiple possible device matches for autoloading. Allow this by changing the alias ModuleInfo field to an Option<Vec<String>>. Note: For normal device IDs this is autogenerated by modpost (which is not properly integrated with Rust support yet), so it is useful to be able to manually add device match aliases for now, and should still be useful in the future for corner cases that modpost does not handle. This pulls in the expect_group() helper from the rfl/rust branch (with credit to authors). Co-developed-by:
Finn Behrens <me@kloenk.dev> Signed-off-by:
Sumera Priyadarsini <sylphrenadin@gmail.com> Signed-off-by:
-
Miguel Ojeda authored
Add some missing fallible methods that we need. They are all marked as: #[stable(feature = "kernel", since = "1.0.0")] for easy identification. Lina: Extracted from commit 487d7578bd03 ("rust: alloc: add some `try_*` methods we need") in rust-for-linux/rust. Signed-off-by: -
Asahi Lina authored
Add the missing SPDX headers to these modules, which were just imported from the Rust stdlib. Doing this in a separate commit makes it easier to audit that the files have not been modified in the original import. See the preceding two commits for attribution and licensing details. Signed-off-by:
-
Asahi Lina authored
This is a subset of the Rust standard library `alloc` crate, version 1.62.0, licensed under "Apache-2.0 OR MIT", from: https://github.com/rust-lang/rust/tree/1.62.0/library/alloc/src The file is copied as-is, with no modifications whatsoever (not even adding the SPDX identifiers). For copyright details, please see: https://github.com/rust-lang/rust/blob/1.62.0/COPYRIGHTSigned-off-by: -
Asahi Lina authored
This is a subset of the Rust standard library `alloc` crate, version 1.62.0, licensed under "Apache-2.0 OR MIT", from: https://github.com/rust-lang/rust/tree/1.62.0/library/alloc/src The file is copied as-is, with no modifications whatsoever (not even adding the SPDX identifiers). For copyright details, please see: https://github.com/rust-lang/rust/blob/1.62.0/COPYRIGHTSigned-off-by: -
Asahi Lina authored
The unstable new_uninit feature enables various library APIs to create uninitialized containers, such as `Box::assume_init()`. This is necessary to build abstractions that directly initialize memory at the target location, instead of doing copies through the stack. Will be used by the DRM scheduler abstraction in the kernel crate, and by field-wise initialization (e.g. using `place!()` or a future replacement macro which may itself live in `kernel`) in driver crates. Link: https://github.com/Rust-for-Linux/linux/issues/879 Link: https://github.com/Rust-for-Linux/linux/issues/2 Link: https://github.com/rust-lang/rust/issues/63291Signed-off-by:
-
