Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Traits for crates (or: canonical API implementations and how to interchange them) #3757

Closed
Cel-Service opened this issue Jan 3, 2025 · 8 comments

Comments

@Cel-Service
Copy link

It would be nice if, for any given API, I could specify my program's canonical implementation of that API in such a way that all of my dependencies can statically call that canonical implementation (without them having to know about and consciously accomodate my preference beforehand).

This proposal fills a gap that traits alone can't (yet) fill both ergonomically and performantly.

Overview

I have two (mutually-exclusive) design ideas that can provide this capability:

  1. One which prioritizes flexibility and long-term maintainability at the expense of encoding its design choices in ways that could be hard to change our minds about later (and therefore requiring more up-front debate and discussion).

  2. One which prioritizes making the absolute minimum number of changes necessary to rustc, Cargo, and the Rust language at the expense of being less ergonomic and more of a headache to implement.

That said, I want to make it clear early on that I'm not committed to these ideas in particular. I just want to communicate the capabilities I'm looking for.

Further, I will also note that (while I do try to go into quite a lot of detail) these ideas are not intended to be complete specifications (or RFCs). I'm not a Rust compiler dev; I'd just like to see this document create discussion and debate.

Why?

The Rust ecosystem is starting to settle on de-facto standard APIs for solving various problems. Further, some APIs are shaped so directly by their domain that they may as well be a standard.

This proposal would allow the Rust ecosystem to innovate around the implementation of a specific API while:

  1. Not requiring intermediary library authors to think about portability. (Better inter- and intra-ecosystem compatibility)

  2. Not having to reimplement or port intermediary dependencies if you switch to a new crate. (Lower switching costs, less duplicated work)

  3. Not having to pay the dyn Trait tax. (Better performance)

  4. Not having to pass an extra <T: Trait> or (implementation: T) around everywhere. (Better ergonomics)

Why not <something we already have>?

Traits alone do not fulfill this proposal's needs: You can't always just use Trait to access a guaranteed complete canonical implementation, nor can you swap out the default implementation from outside of the crate within which it was defined, nor can you simply use Trait as a concrete type (you must always use it through <T: Trait> or impl Trait or dyn Trait, with all the limitations those options impose).

Feature gates alone do not fulfill this proposal's needs: They don't allow for arbitrary, out-of-tree implementations; only those implementations foreseen by the crate author are available.

Enums alone do not fulfill this proposal's needs: Like feature gates, they require the library author to either foresee every option or pay the dyn Trait tax for options they didn't account for.

An API-only crate nearly fulfills this proposal's needs, but:

  1. Any library that wants to build on top of an API-only crate needs to either (A) sacrifice performance and pay the dyn Trait tax to get a dynamically-dispatched implementation of that API, (B) sacrifice ergonomics and juggle a <T: Trait> and (in many cases) require that the user explicitly provide an (implementation: T), or (C) sacrifice portability and explicitly pull in one concrete implementation of that API from another crate.

  2. As a consumer of an API-only crate, you're beholden to any dependencies that build on top of said API-only crate. If a library that you need:

    • Hard-codes a specific struct instead of offering Box<dyn Trait> or <T: Trait> when you need to customize the implementation, or

    • Hard-codes Box<dyn Trait> in a way that causes measurable performance degredation (or when you don't have an allocator, or when you need to use a specific allocator, etc.),

    ..then you have to either (A) convince the maintainers to modify that crate, (B) modify the crate yourself and then convince the maintainers to merge your change, or (C) modify the crate yourself and then maintain your own fork.

  3. At some point, some other crate needs to decide on an implementation to depend on. An API-only crate cannot (by definition) provide a meaningful default implementation.

What are you actually proposing?

Option 1: API modules as a first-class concept

API modules could become their own independent, first-class concept, similar to how traits work: they specify the shape of (but also the canonical path by which one may use) an API, while any other crate can offer an implementation of that API.

  • Any module could be marked as being a definition (or implementation) of an API, not just the top-level module of a crate.

  • Implementations would never conflict; one could still implement any API in a way that picks between any number of concrete implementations.

  • Any crate may recommend dependencies to guarantee that there will always be an implementation of the APIs that it defines within itself.


Click here to show/hide the specifics for this design.

Similar to a trait, API modules would be able to:

  • Declare a pub fn name(with_arguments: ...); but not (necessarily) define it.
  • Declare a pub type Name: Bounds; or pub type Name; but not (necessarily) define it.
  • Declare a pub const Name: Type; but not (necessarily) define it.
  • Use types from anywhere as part of their API surface, e.g. structs coming from third-party crates.
  • Feature-gate any part of their API (for example, to add OS-specific extensions).

Unlike a trait, API modules would also be able to:

  • Define anything that's not pub (to help organize provided-implementations).
  • Organize their contents into pub mods (just like any other module).
  • Define a pub enum Name {...}
  • Declare a pub struct Name: Bounds; or pub struct Name; (would work exactly like pub type Name: Bounds, but would imply Send, Sync, and other "normal" traits by default).
  • Define a pub struct Name {...} or pub struct Name(...) (but that struct's members then become set in stone forever).
  • impl structs (and declare or define pub methods freely).
  • Define traits (and use them freely).
  • Define tests (which would not be part of the crate's public API, but would get added to any implementation of the API, and which would allow implementors to easily verify (just by running cargo test) that their implementation matches the behavior expected by the API author).

Critically, API modules would not be able to:

  • Declare enums without defining them (as that would make no sense).
  • Declare traits without defining them (as that would make no sense).

An API module could be treated by the compiler almost like a trait used as part of a generic bound: its implementation would be statically dispatched to whichever crate canonically provides said API for the current program.

The syntax could look like this:

In api-crate/src/lib.rs:

#![api]
//
// This directive would inform the compiler that this module should not be
// usable without an implementation having been provided (and thus that
// declarations without definitions should be allowed).

pub fn do_something();

pub fn do_this_then_do_something(this: impl FnOnce() -> ()) {
    this();
    do_something();
}

In api-crate/Cargo.toml:

[recommends]
implementation-crate = { version = "...", implements = ["::"] }
#
# The following restrictions apply:
#
# 1. Each recommended dependency _must_ be marked with the API(s) that it
#    `implements`.
#
# 2. All paths in `implements` must start with `::` (i.e. a crate may only
#    recommend dependencies if they implement its own APIs).
#
# 3. Only one dependency may be recommended per API (but that crate may then
#    in turn have its own dependencies).
#
# Presumably, [recommends] would also work as [target.'cfg(...)'.recommends]
# whenever one needs to recommend a different implementation crate in specific
# situations.

[[lib]]
apis = ["::"]
#
# When necessary, the above is what it could look like to explicitly inform
# Cargo that this crate defines an API module (in this case, that the crate
# itself _is_ an API module).
#
# Ideally, mentioning `implements` in a recommended dependency would imply the
# existence of said API module.
#
# Even more ideally, simply annotating a module as `#[api]` should be enough
# (just like declaring a trait).

In implementation-crate/src/lib.rs:

#![implements(api_crate)]
//
// This directive would cause the compiler to type-check this module against
// the API module accessible from the path 'api_crate' in a very similar way
// to a trait (e.g. "missing function", "unexpected public function not
// defined in api_crate" and other such errors) and to fill out any missing
// definitions with provided ones where available.
//
// Ideally, it could be applied to any module; that would allow one crate to
// implement several conflicting APIs as various submodules if so required.
// Since callers would (normally) be expected to access the implementation
// through the path where the API module was defined, it wouldn't matter at
// all where the implementation module is (so long as the compiler is
// eventually told which module to use as the _canonical_ implementation).

pub fn do_something() {
    println!("Did something!");
}

In implementation-crate/Cargo.toml:

[[lib]]
implements = ["api_crate"]

[dependencies]
api-crate = "..."

# When necessary, a single crate could implement many different APIs without
# pulling in any unnecessary dependencies like so:
#
#api-crate = { version = "...", when-implementing = ["::"] }
#
# `when-implementing` could act as shorthand for:
#
# - Defining some kind of faux-feature like
#   `#[cfg(implementing = api::module::path)]` for all API modules listed
#   (where `::` resolves to the dependency itself, and `::path` resolves to a
#   path inside that dependency; any bare `name` would imply an API module
#   defined by some other dependency).
#
# - Adding `<cratename>` as conditional on that faux-feature, such that it
#   only gets pulled in when either (A) this crate is providing the canonical
#   implementation for one of those APIs, or (B) this crate was explicitly
#   marked with a matching `implements = ["api::path"]` in the binary/dynamic
#   library's Cargo.toml.
#
# This approach could also be used in combination with `package = "..."` to
# implement multiple incompatible versions of a given API (while still not
# pulling in unnecessary dependencies).

To override a recommended dependency, in binary-crate/Cargo.toml:

[dependencies]
api-crate = "..."
custom-implementation-crate = { version = "...", implements = ["api_crate"] }
#
# Note that explicitly specifying which crate to import an API from should
# disable the dependency that was recommended to implement that API (if one
# _was_ recommended).

# When a crate needs to pull in multiple implementations of the same API, it
# can add `{ canonical = ["path"] }` to one of its dependencies (instead of
# or in addition to `implements`) in order to mark that dependency's
# implementation as the canonical one.

To keep things sane, when resolving dependencies, Cargo (probably) shouldn't actually be aware of paths in crates; it should just see APIs as opaque keys (which just so happen to look like module paths) that are defined within a scope owned by a specific crate (where :: is simply a key that maps to "this crate itself", and ::path::to::api maps to the key path::to::api defined by "this crate itself").

Any name assigned to a dependency would (as with normal dependency resolution) simply the local crate's label for that dependency crate.

For example, if a crate named alpha contains these lines:

[[lib]]
implements = ["a"]

[dependencies]
a = { package = "api-crate" }

the following is how one should use it from one's binary crate:

[dependencies]
api-crate = "..."
alpha = { version = "...", implements = ["api_crate"] }

Click here to show/hide the upsides/downsides of this design.

Special considerations required for API modules as a first-class concept

  1. We need a standard, well-documented solution for the community to make backwards-compatible changes to an API module, otherwise this entire concept will just create more and more dependency management headaches as crates age and APIs evolve.

    A (potentially) simple solution could be to have some directive like #[added_in("1.1")] and require that anything with said directive be fully defined by the API crate.

    If we were to also hide newer APIs from any crate that depends on e.g. "1.0" (and provide a useful compiler error if such a crate tries to access an API newer than its minimum required version of that API), then we could guarantee that API modules could be updated fully independently: crates that depend on an outdated version of the API wouldn't be able to accidentally depend on newer additions to that API, while crates which require a newer version of the API would be able to seamlessly interoperate with those older crates.

    This concept could be extended further for convenience: #[deprecated_in("version")] could be used to trigger deprecation warnings when depending on API version or later and trying to use the annotated part of said API, while #[removed_in("version")] might be a useful concept to allow API authors to make part of an API invisible to crates which require version or later (as opposed to making said change as part of a major version bump).

  2. Bounds would need to become much more explicit when defining an API module.

    If we use the default assumptions (i.e. everything is Send and Sync unless it contains a type that indicates otherwise) then it won't be possible for implementers anywhere to define a struct as containing !Send members (without unsafe impl Send, at least).

    If we assume nothing about any type but the explicitly-provided bounds, then no APIs will be Send or Sync unless explicitly marked as such. That would be a lot of boilerplate to expect people to include in their bounds.

    Perhaps we could ensure that API module authors will think about this (if only briefly) by requiring them to explicitly encode their assumptions into their API's definition: something like #[api(default = Send + Sync))] (or some variant thereof, e.g. !Send + Sync).

Benefits of API modules as a first-class concept

Compared to the other design proposal:

  1. This design makes it much easier to leave the "happy path": switching implementations just requires adding a dependency and marking it as implements = ["api_crate::path::to::api"].

  2. This design is more ergonomic, especially for crates with APIs centered around crate::functions() rather than crate::Structs and crate::Traits.

  3. This design is more flexible; the other design proposal's capabilities could be implemented in terms of this one.

  4. This design completely sidesteps the circular-dependency complication: since APIs become a Cargo concept, the API crate can simply recommend a crate that implements its API rather than depending on such a crate.

Drawbacks of API modules as a first-class concept

As hinted before, compared to the other design proposal, I imagine that this approach would require touching a much wider number of places between cargo, rustc, rustdoc, the Rust language itself, and the official documentation.

While it would be very nice, I don't have a meaningful frame of reference for what all it would take. The pieces are in place, but the devil is in the details.


Option 2: pub(import) and pub(export)

In effect, this design could be implemented as just another generic parameter (bound to a type name instead of a <T>).

One could pub(import) type Interchangeable: Trait; in one crate to reserve a name for some type that will meet the given bounds, and then pub(export) type cratename::Interchangeable = Concrete; in another crate to define the type associated with that name.

API crates could then implement their APIs in terms of one or more concrete implementations of one or more traits.


Click here to show/hide the specifics for this design.

In api-crate/src/lib.rs:

pub trait Trait {
    fn do_something();
}

pub struct FallbackImpl {}

impl Trait for FallbackImpl {
    fn do_something() {
        println!("Did nothing.");
    }
}

pub(import) type Interchangeable: Trait = FallbackImpl;
//
// Providing a fallback implementation would be optional (just like a default
// concrete type is optional in `<T: Trait = Type>`).
//
// A fallback implementation would be provided in situations where a library
// just wants to create an _opportunity_ for configuration, but doesn't want
// to _impose_ configuration on callers who don't have the need for it and
// shouldn't have to think about it.
//
// The API-only crate could also refer to another crate's type as its fallback
// implementation; see my notes later on about that faux-circular dependency.

In any library crate that depends on api-crate:

use api_crate::Interchangeable;

pub fn component() {
    // Statically call the implementation provided by the binary crate, or (if
    // one was provided) call the fallback implementation.
    //
    // If no fallback was defined and the binary didn't
    // `pub(export) api_crate::Interchangeable`, the compiler should complain
    // that `api_crate::Interchangeable` needs to be `pub(export)`ed.
    //
    Interchangeable::do_something();
}

In binary-crate/src/main.rs, when one needs to (re)define a pub(import) type:

struct MyStruct;

impl api_crate::Trait for MyStruct {
    fn do_something() {
        println!("Did something!");
    }
}

pub(export) type api_crate::Interchangeable = MyStruct;
//
// To begin with, it would be more than enough to only allow binary/dynamic
// library crates to `pub(export)` types like this.
//
// It would be slick to let a framework implementation crate take care of this
// step for you, but I imagine that could require a lot more effort to
// implement (properly), since the binary will already be guaranteed to be the
// last crate to get compiled.

A possible alternative for pub(import) and pub(export)

For the sake of argument, let's say that even allowing pub(export) in main.rs would be completely impractical; I could imagine that may require a much more involved rethink of compilation and/or adding an additional step before linking, so I wouldn't consider that to be an unreasonable initial response.

I can think of at least one reasonable alternative that could make this design even easier to implement (at the cost of being an even worse UX).


Click here to show/hide this design's main alternative.

A (binary/dynamic library-only) Cargo.toml option similar to:

[exports.api_crate]
dependencies = [{ preferred-implementation = { ... }}]

and (in the binary/dynamic library crate) an exports/api_crate.rs file like:

pub(export) type api_crate::Interchangeable = preferred_implementation::Implementation;

where exports/api_crate.rs is guaranteed by Cargo to compile alongside api_crate at the stage where it needs those types to be defined, and Cargo will then add preferred-implementation to api_crate's dependency list.

This is not quite as ergonomic as being able to "just" add a pub(export) line if and when you need it, but it's a very solid start, and seems like it could be pretty practical to implement.


Click here to show/hide the upsides/downsides of this design.

Benefits of pub(import) and pub(export)

Compared to first-class API modules, this design should require as few changes as possible (across the lowest number of places possible) while still achieving the goal of this proposal.

This would still offer a (reasonably) smooth UX until you need to override an API's implementation.

Drawbacks of pub(import) and pub(export)

Rough edges

The UX (both for library consumers and for library authors) isn't as smooth as what first-class API modules could offer:

  • From an outsider's perspective, specifying these explicit "reverse dependency" links would be an undeniably weird way to accomplish this proposal's goal, and definitely looks like the compromise that it is.

    This puts the design in a very similar situation to extern crate right out of the gate. If this were to be released, the community would almost certainly want something nicer than this sooner than later.

  • Library authors would be forced to implement their API in terms of one or more traits. This isn't always going to feel natural, especially for crate::functions(); it's an added level of indirection that will make it (just a little bit) harder to understand a library's code.

  • Since this design is fundamentally based on traits, this would prevent API-only crates from offering (reimplementable) const fn in their API if they want to take advantage of the benefits offered by this proposal.

  • Anyone who wants to override an API's implementation would (at best) need to explicitly manage pub(export) type lines in their crate, or (at worst) manage (and mentally keep track of) a whole separate file.

    (One could, of course, argue that having to add implements = ["..."] to a dependency would be a similar inconvenience, but that could be debated.)

None of these problems are even close to being dealbreakers, but they aren't ideal, either.

Circular dependency

I have to imagine that this would complicate things.

While logically, the dependency between main and api_crate (or api_crate and any implementation_crate) shouldn't actually be circular*, it will likely cause Cargo to complain and will require some finesse to guarantee that only pub(import) allows referring circularly to a dependent crate.

*: Ideally pub(import) should be implemented as just reserving a name for a bounded generic type that will eventually be defined, pub(export) as just defining a concrete type for a name that was already reserved, and = Fallback as just suggesting a path where the compiler can look later if none was provided.

Cargo integration

This design doesn't have any integration with Cargo's dependency system. Without some additional thought on how to gate the crate's default implementation on a dependency, this design (on its own) would leave API authors with three options:

  1. Expect the binary author to pub(export) type manually for every type in every dependency. This would be a horrifying downgrade in UX.

  2. Expect the binary author to crate::export_macro! manually for every dependency. This is nearly as bad as option 1.

  3. Try to paper over the gaps with feature gates:

    #[cfg(feature = "implementation")]
    pub(import) type DefaultImplementation: Trait = other_crate::DefaultImplementation;
    
    #[cfg(not(feature = "implementation"))]
    pub(import) type DefaultImplementation: Trait;

This is obviously not ideal, so it would be preferable to make some further changes in order to smooth this out:

  1. Rust should "just know" that the = Fallback (if present) should be completely ignored by dependency crates and only applied by the binary/dynamic library crate (and even then, only if it didn't pub(export) that path).

    (Alternatively, we could use a syntax like #[implementation("path::to::Fallback")] instead of = path::to::Fallback for this purpose if it would make this integration easier.)

  2. Cargo functionality could be added to make implementation plumbing more ergonomic:

    In the API-only crate, it could be as simple as (e.g.):

    [imports-dependencies]
    current-frontend-implementation-crate = { version = "..." }
    
    # Has the exact same semantics as [features], but every 'import' is enabled by default.
    #
    # Disabling all imports that depend on a crate in [imports-dependencies], disables that crate.
    #
    [imports]
    frontend = ["current-frontend-implementation-crate"]

    And turning off unnecessary dependencies could be as simple as (e.g.):

    [dependencies]
    api-crate = { version = "...", disable-imports = ["frontend"] }
    # Or 'no-default-imports = true' to disable _all_ imports.

    Only the binary/dynamic library crate should be allowed to specify default-imports or imports (as half of the point is to make libraries agnostic over the implementation of API-only crates; if they really want to go out of the way to use a specific implementation, they can just add that implementation as a dependency).


Shared upsides and downsides of these designs


Click here to show/hide the shared upsides and downsides.

Benefits common to both design proposals

  1. Any API crate can guarantee that there will always be a default implementation, so any crate can (seamlessly) become API-only when deemed worth doing so.

  2. Since any crate can become API-only seamlessly, code written before this feature came out will continue to compile against existing crates that choose to become API-only.

  3. Any implementation crate can (where necessary) expose yet more API modules (or pub(import) types) to abstract over some of its own specifics.

  4. Library crates can depend directly on the API crate (without naming a specific implementation) and trust that it will "just work" for users.

  5. Binary and dynamic library crates can swap between implementations of any API without having to coordinate with the maintainers of the intermediary libraries that they depend on.

Downsides common to both design proposals

  1. Incompatible implementations.

    From a technical perspective, this is part of why I mentioned tests being a part of the "API modules" design above. A high-quality API crate can provide tests to verify an implementation, and then intermediary library authors can trust that most implementations should meet their needs.

    From a social perspective, it would be unreasonable to expect support from intermediary library maintainers when bugs arise due to a non-compliant implementation of an API that they rely on.

    That said, this is really no different than the status quo with traits: you're already expected to implement traits correctly (where correct is defined both by the trait's bounds and by its documentation).


Practical uses

Any API which:

  1. Aims to be portable in some way (e.g. runtime environment, OS, hardware, service provider) but
  2. Really only needs one canonical implementation per compiled program/dynamic library,
  3. Wants to support a stable ecosystem of libraries that build on top of it, and
  4. Cares about both performance and API ergonomics,

would benefit from this functionality.

Further, this means that basically every existing library could benefit from this functionality: the simplest possible way for a crate to become API-only would involve simply moving its implementation into another crate and having its API-only crate mirror its original API one-to-one.

Case: winit

winit is looking to stabilize their main API into its own crate. If they do so, winit will truly become the de facto Rust standard API for cross-platform window management.

For windowing, in a given program, there's almost always going to be one canonical (and platform-dependent) way to accomplish a given task. (Further, dual X11 and Wayland compatibility could easily be handled by an intermediary implementation winit-impl-linux that swaps between calling the implementations provided by hypothetical winit-impl-x11 and winit-impl-wayland crates at runtime.)

If winit itself could become an API-only crate without losing backwards-compatibility, then (without having to think about it) any library that depends on winit can be made portable to your needs, even if your needs aren't met by winit's default implementation.

If winit-impl (or the like) could also be made portable over the specific means of accessing specific windowing APIs, then one can benefit from all of the portability work winit provides to the ecosystem while still being able to (for example) proxy otherwise-standards-compliant events to and from a more privileged process when necessary.

Case: wgpu

wgpu is becoming the de facto Rust standard API for cross-platform GPU access, and does seem to have some interest in splitting wgpu into API-only and implementation crates.

Similar to winit, in a given program, there's almost always going to be one canonical (and platform-dependent) way to access Vulkan, OpenGL, and other graphics APIs. Further, there are exceedingly few situations where one would want to mix-and-match multiple methods of (for example) accessing Vulkan in the same program; thus, a trait isn't really the right tool for the job.

If wgpu itself could become an API-only crate while neither losing backwards-compatibility, nor performance, then (without having to think about it) any library that depends on wgpu can be made portable to your needs, even if your needs aren't met by wgpu's default implementation.

If wgpu-impl (or whatever it may be called) could also expose an API to override how it accesses (for example) Vulkan or OpenGL, then one can benefit from all of the portability work that wgpu provides to your programs while still being able to e.g. proxy rendering commands to a more privileged process.

Bonus: Imagine being able to expose fully ecosystem-compatible wgpu directly to Rust code running in a standalone (non-browser, non-WASI) WASM runtime by compiling it with an implementation of the wgpu API that's backed by calls to standard, cross-platform wgpu in the host process.

: It also seems like they explicitly mention frustration with the way that dyn Trait affects their debugging experience. I don't know for a fact whether this proposal would help with that, but I do hope so!

Case: SDL

While SDL is not, itself, a Rust project, it is (at minimum) close to being the de facto standard C API in its field, and would (if it were written in Rust) greatly benefit from this proposal.

For those unfamiliar, SDL is a (relatively) low-level library that provides abstract, platform-independent access to audio, input, and other resources that makes implementing cross-platform games (and game engines) easier.

Each type of resource must be acquired in different ways depending on the operating system, available supporting services, runtime environment, attached hardware, and so on; further, you might need to acquire access to some of these resources indirectly. However, for any given program, there's usually only one to a handful of ways any given resource may be acquired; furthermore, all of these methods are abstracted away from the public API.

Any Rust-based library that wants to offer some (or all) of SDL's features would, similarly, greatly benefit from being transparently portable (at multiple layers) over alternative implementations of its API or subsystems.

Case: hidapi

Similar to SDL, hidapi is the de facto standard, cross-platform C API for interfacing with Human Interface Devices.

hidapi is implemented as a common header implemented by many (in-tree) libraries; each platform has a number of means by which one can access HIDs, including a backend implemented in terms of a cross-platform USB abstraction library (that I will refrain from including as a case, since the value proposition is so similar).

Any Rust-based alternative to hidapi would, similarly, greatly benefit from this proposal.

Case: Algorithm crates

While traits will continue to be Rust's best (and most flexible) tool for interchanging different algorithms in an API, it would be nice if one could seamlessly swap out their program's canonical implementation of (for example) Flate or SHA-256 such that if one has different needs (e.g. working with a hardware accelerator, or decoding media out-of-process for security) all of their existing dependencies just work without ever having thought about portability.

In summary

I hope I've managed to convince you, reader, that there's a compelling value proposition for these capabilities across the entire Rust ecosystem!

I hope that this proposal starts the ball rolling and gets gears turning in people's heads. What I've written here could always be improved upon, so it would be nice to see what people think.

Note: be sure to expand the dropdowns if you missed them:


They look like this!

@Cel-Service
Copy link
Author

In relation to portability over (the canonical implementation of) a given algorithm, it looks like someone in the embedded world presented a similar use case.

Indeed, I'm of the opinion that extremely low-level crates (e.g. crates that implement just one specific algorithm) could ideally be defined as API-only, with a canonical implementation provided by default.

As I already mentioned in brief, this would introduce new portability: if you need to swap out the implementation of (for example) BLAKE2b, you need only ensure that your dependencies use the blake2 crate and aren't explicitly requiring e.g. blake2-impl-rust or blake2-impl-simd crate. From there, regardless of either above design, one may swap canonical implementations without other crates needing to accommodate your needs.

@Cel-Service
Copy link
Author

It appears that a small subset of the above-proposed functionality has been approved for unstable implementation in the first of this year.

@kennytm
Copy link
Member

kennytm commented Jan 4, 2025

meta-point, given the current stance of T-lang in #3756 and that you have basically written the details perhaps it's better just publish the RFC as an actual PR?

@programmerjake
Copy link
Member

meta-point, given the current stance of T-lang in #3756

I don't think that's the current stance yet, just proposed to be. that said, I agree this issue should just be a PR

@Cel-Service
Copy link
Author

meta-point, given the current stance of T-lang in #3756 and that you have basically written the details perhaps it's better just publish the RFC as an actual PR?

I'd be more than alright with writing out an RFC, but I must set expectations beforehand: I am a Rust user first and foremost. Compiler development is not my field of expertise. I won't be able to provide meaningful insight into (for example) the exact monomorphization strategy by which this UX could be obtained.

I don't wish to waste people's time, so I'd first like to get a feel for which of my proposals (if either) there is enough appetite for to be worth specifying in complete and total detail.

Would you be able to recommend a method by which I can gauge interest before committing to an RFC, or do you feel the level of detail I've gone into here is worth (for example) simply submitting my ideas as two competing RFCs?

@Nadrieril
Copy link
Member

Nadrieril commented Jan 4, 2025

The typical way to gauge interest and get feedback is to open a "pre-RFC" post on https://internals.rust-lang.org . You could open one post for each of your proposals for example, or just one post that explains the problem-space and presents your two solutions. For https://internals.rust-lang.org you don't at all need the level of detail of an official RFC; what you wrote in the OP is enough.

@Cel-Service
Copy link
Author

Cel-Service commented Jan 4, 2025

The typical way to gauge interest and get feedback is to open a "pre-RFC" post on https://internals.rust-lang.org . You could open one post for each of your proposals for example, or just one post that explains the problem-space and presents your two solutions. For https://internals.rust-lang.org you don't at all need the level of detail of an official RFC; what you wrote in the OP is enough.

Thank you!

I'd managed to miss this line in "before creating an RFC":

You may file issues on this repo for discussion, but these are not actively looked at by the teams.

@Cel-Service
Copy link
Author

Cel-Service commented Jan 4, 2025

For those interested, I've filed this proposal as a pre-RFC at internals.rust-lang.org.

I had additionally previously started a topic on Zulip for this proposal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants