RFC: Allow packages to specify a set of supported targets

[carloskiki]

Summary

The addition of supported-targets to Cargo.toml. This field is an array of target-triple/cfg specifications that restricts the set of targets which a package supports. Packages must meet the supported-targets of their dependencies, and they can only be built for targets that satisfy their supported-targets.

Rendered

[Lokathor]

I think that there should be a little more explanation about how docs generation works with this. Specifically: Can I build docs for a target that's unsupported, such as if my host machine isn't supported, can i cargo doc to still just see the documentation?

[opeik]

This would be a godsend for crates that link against third party libraries, thus limiting supported targets.

[carloskiki]

I think that there should be a little more explanation about how docs generation works with this. Specifically: Can I build docs for a target that's unsupported, such as if my host machine isn't supported, can i cargo doc to still just see the documentation?

Indeed, and this is especially important since docs.rs needs to be able to generate the docs for all crates. I added it here.

[Lokathor]

Crates assuming that they're running on one of several targets could even end up making unsafe code decisions based on that fact. Forcing the code to "just build anyway" would naturally lead to problems.

And crates can already force themselves to only build only on a specific target, this would not be a new ability, but instead it's only a way to better organize that information.

[workingjubilee]

The author already mentions in the Prior art that the following Rust can be written:

#[cfg(target_arch = "lol"))]
compile_error!("experience bij)";

Please do not make comments on the RFC which do not engage with the RFC's content.

[joshtriplett]

This seems helpful, but at the same time, it may prove annoying to have to copy these across, if a dependency has a very specific list. And it may be non-trivial to enforce.

I think we should downgrade this to a lint, and say that it's best-effort, not mandatory.

[Mark-Simulacrum]

This also seems complicated by the fact that I might have a dependency that only supports (say) wasm, but I'm using it solely as a dev-dep (e.g., in tests). I don't think that case merits also setting supported targets on the containing package as a whole, since downstream consumers might not care about that limitation for running on tests.

[Lokathor]

If it's just a lint then unsafe coders can't depend on this, and they will still need to just use a compile_error! or something if they're really trying to avoid unsoundness.

[epage]

This is effectively saying that if you have a dependency that can't build on a target you claim to support, the build will fail and you should instead move it to a target.*.dependencies table.

imo that seems like something that should be a hard error to me.

I could see loosening the restriction on

When supported-targets is not specified, any target is accepted, so all dependencies must support all targets.

To me "we don't check", like package.rust-version, requiring supported-targets = ["cfg(true)"] to get the checking.

[carloskiki]

I totally agree with @epage here.

Using the same logic as package.rust-version is also something I have not thought of but is a great idea.

[carloskiki]

A problem I could see however is that if a package foo does not use supported-targets but one of its dependencies does, then when depending on foo errors of incompatible targets can be hard to solve since they come from transitive dependencies.

[epage]

Not quiet sure the problem. When building foo, you are building for a specific target and the error would be for that target. With package.rust-version, we check what all packages aren't compatible with the current toolchain and provide a single error message, see https://github.com/rust-lang/cargo/blob/9589831f61a8259919e64c6d68c1a36efc6efd20/src/cargo/ops/cargo_compile/mod.rs#L494-L543

[joshtriplett]

If it's just a lint then unsafe coders can't depend on this, and they will still need to just use a compile_error! or something if they're really trying to avoid unsoundness.

supported-targets would still be enforced (modulo some kind of force option); I'm talking about softening the enforcement that a crate's supported-targets is a subset of its dependencies.

[joshtriplett]

I don't think this should be a requirement. It's sometimes reasonable for tests to only run on some targets.

[epage]

But we'll try to build the dev-dependency on an unsupported target, independent of what cfg is on the test, which will fail. The dev-dependency should be moved to a target.*.dev-dependencies table, bypassing this.

[carloskiki]

The dev-dependency should be moved to a target.*.dev-dependencies table, bypassing this.

The RFC does not do a good job of "teaching" how to use this feature (Could be added to guide-level explanation), but this is what I have in mind.

[joshtriplett]

This looks excellent!

[joshtriplett]

Let's go ahead and start the process of asynchronously checking for consensus.

@rfcbot merge

[rfcbot]

Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Eh2406
• @Muscraft
• @Rustin170506
• @arlosi
• @ehuss
• @epage
• @joshtriplett
• @weihanglo

Concerns:

• field-name (RFC: Allow packages to specify a set of supported targets #3759 (comment))
• should-crates-have-to-set-supported-targets-to-match-their-dependencies (RFC: Allow packages to specify a set of supported targets #3759 (comment))
• should-this-be-in-manifest-or-updatable-metadata (RFC: Allow packages to specify a set of supported targets #3759 (comment))

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[joshtriplett]

@rfcbot concern should-crates-have-to-set-supported-targets-to-match-their-dependencies

[epage]

For me the important precedence is that the field means "skip if the qualification is not met" and for that reason I favor using this name.

[epage]

Apparently, this is different than required-features as this errors, rather than skips.

I raised this at https://github.com/rust-lang/rfcs/pull/3759/files#r1909208702

[epage]

I also feel like using the word "support" carries a lot of unnecessary connotations that complicate the conversation. For example, with MSRV, the Cargo team has been leaning in the direction that "support" is an active process that gets tested. However, applying that here would lead to people over-constraining their targets.

[carloskiki]

The difference with this and required-features is that supported-targets is at the package level, not at the cargo-target level.

So if I run cargo build in a package with a binary foo which does not have its required-features met, then cargo can still possibly "do work" if there is another cargo-target which has its required-features met (for instance a library cannot have any required-features).

However if the supported-targets are not met for a package, then there is no chance of cargo doing compilation work for that package. That is why the packages are skipped when in a workspace, but an error is raised in a single package. This is just how if I ran cargo build --bin foo in the previous example, then cargo errors instead of skipping.

[carloskiki]

I do not have a preference for the name, I kept it as is to not confuse people who had read the Pre-RFC. I would not be against changing it.

[epage]

I don't think this being at the package or build-target level makes much of a difference. The RFC is starting at the package level and has build-target as a future possibility. In that case, we can treat this as the package is providing the default for all build-targets like package.edition

[epage]

@rfcbot concern field-name

See https://github.com/rust-lang/rfcs/pull/3759/files#r1909195446

[epage]

The fact that the alternatives call this out I feel like this needs to be under its own heading with a bit more explanation behind it

[epage]

This should call out the reason why it was originally proposed

• Easier, more exact relations logic
• Easier for the resolver to leverage this information

[CryZe]

I think the chances that this whitelist is far too restrictive is incredibly high. I've experienced a bunch of cases where the author of a crate says "I only test for x86 and ARM so these are the only officially supported targets" when the crate works perfectly fine on many other architectures (the code doesn't use anything architecture specific, the author just doesn't care about setting up CI for e.g. PowerPC). I would want to see some fallback mechanism that the author of a crate can specify such that in the example x86 and ARM can be the "officially" supported targets but other targets can still build just fine, but possibly emit a warning that they are not officially supported by the author.

Alternatively the documentation should strongly suggest not using the supported-targets in these cases.

Update: I just noticed another example of this. I'm the author of a hotkey listening crate. It supports specific operating systems. However, there's a dummy implementation for all other targets. It compiles, but no hotkeys are ever triggered. Here it would also be great to advertise in the Cargo.toml that Windows, Linux and co. are the main supported targets, but that it's perfectly fine (with possibly a warning to the user) to just use the crate across all targets.

[kpreid]

As per some non-threaded comments on this PR by @CryZe and @ahicks92: this feature has the risk that package authors will list a too narrow set of supported-targets. They can already do this with a conditional compile_error!, but I am particularly concerned that they might do so on the optimistic principle “filling in all of the metadata is good, right?”, whereas if the manifest field did not exist, they wouldn’t have thought to write a compile_error!.

I think that, in order to reduce the risk of gratuitous breakage that requires users of a not-eagerly-maintained package to fork/vendor it or find an alternative, this should be advisory for dependents: it does not force a build to fail unconditionally, but causes a lint that might error by default but can be opted out of in some way (perhaps as a field in the [dependencies] table).

(By “not eagerly maintained”, I mean: the code is generally good quality, but it takes weeks/months to get a PR in upstream, or for a release to happen after the PR. I think that it is an important virtue of Rust that you can usually successfully use such packages, rather than various kinds of churn making it hard.)

In order to support the “it’s UB to run this code on (not) this target” use case, there can be a way to require it to be a hard error (or at least, not bypassable by dependents, only by the build command configuration). But that should be opt-in, so that the feature can also be used for the softer use case of offering machine-readable requirements documentation and avoiding messes of “item/symbol not found” error messages.

[hanna-kruppe]

For the "it's UB to run this code on (not) this target" use cases, the existing tools of conditional compile_error!(...) / const { assert!(...); } seems better than overloading the supported-targets field. Putting each assumption right next to the unsafe code that relies on it means:

• It's easier to audit the code because it's more localized -- even if the safety comment on a block justifies something by reference to Cargo.toml, I as reviewer have to check that.
• It's harder to forget about reviewing/updating the code when changing the required-targets definition. You could put a comment in the relevant part of the manifest, but that's further removed from the relevant code so it's easier for the comment to go out of sync.
• If I depend on a library that has (in my view) an overly restrictive supported-targets, I can try to work around this by patching it locally. Editing the manifest is easy, bypasses any "enforcement" by Cargo, and makes it easy to miss that some unsafe code in the guts of the library actually relies on it for soundness. Having to dive into the source code and remove static assertions next to unsafe blocks is higher friction, which is good when soundness hinges on it.

[epage]

One of unresolved questions is

• Some crates will inevitably have target requirements that are too strict, how
  do we make users bypass the error (probably with some --ignore-** flag)? Do we want to allow this?

The approach taken with package.rust-version is the --ignore-rust-version flag. We have approved but not implemented turning the hard error into a deny-by-default lint.

The problem with either situation is that one of the intended use cases is for this to trim down Cargo.lock / vendor/. If its advisory only, then we can't do that anymore.

[Mark-Simulacrum]

If the opt-out is encoded in Cargo.toml, then Cargo can see that when generating the lockfile, right? So CLI flags do make it impossible to trim the lockfile, but explicit override (similar to how default / non-default features work in principle) seems doable?

[epage]

Cargo resolves Cargo.lock assuming all features of workspace members may be activated. It only trims features from transitive dependencies where there is no way to activate them through the command-line.

An --ignore-supported-targets flag is a transient state and the lockfile generally would need to take into account all transient states in its creation so it doesn't change based on how Cargo commands invoke it. The only thing it could trim are transitive dependencies through target.*.dependencies tables or artifact dependencies with a target

In the past, I have toyed with the idea that the lockfile could remember some types of resolution inputs to re-apply them in the future. This is a very dodgy area that would need a lot of detail to figure out and is questionable whether it would ever get approved. This is the type of thing that could slow down or derail the RFC and would ideally be split out into a future possibility to make this RFC go more smoothly.

[joshtriplett]

It sounds like in theory we could provide a non-transient force/override option in the manifest, which could then disable trimming the dependencies. But 👍 for not blocking on that in any case.

[zackw]

I share the concern that people might write target requirements that are too strict, or that they might write target requirements that were correct at the time but things have changed and the crate hasn't been updated. I want to suggest that:

• When directly building a package (i.e. not as a dependency), attempting to build for a target that it declares it doesn't support should trigger a deny-by-default lint.
• When a package is being built as a dependency, attempting to build for a target that it declares it doesn't support should trigger an allow-by-default lint.

This is analogous to honoring Cargo.lock for the package being directly built but not for its dependencies. If, hypothetically speaking, the io_uring API becomes available on FreeBSD at some time in the future, but the maintainer of some io_uring interface package neglects to add FreeBSD to the list of supported targets for that package, binary crates that use that package can still be built for FreeBSD despite the exclusion.

[epage]

1. I think we need to keep in mind that the framing of this RFC can be improved which affects the conversation, e.g. RFC: Allow packages to specify a set of supported targets #3759 (comment)
2. We need to make sure we are clear in communicating how this should be used, see RFC: Allow packages to specify a set of supported targets #3759 (comment)

That said, there are likely still times to override. I think we're being a bit limited in exploring our options

The discussed CLI override

• Affect the entire dependency tree
• Prevents Cargo.lock trimming
• Every dependent must do it

Lint overrides

• See CLI

A single manifest override (bespoke version of lint override)

• See CLI

Switching to mutable metadata, see #3759 (comment)

• Whole new process that would need to be designed and explored

Other options

• Patching (existing)
  • Requires copying the entire repo somewhere (maybe cargo override can help?)
  • Every dependent must do it
• Unidiff patch (proposed)
  • More limited in what needs to be in your repo
  • Has some design questions that need working out
  • Every dependent must do it
• Manifest override on specific dependencies
  • Every dependent must do it

[epage]

As posted at #3759 (comment)

one of the intended use cases is for this to trim down Cargo.lock / vendor/. If its advisory only, then we can't do that anymore.

[epage]

@workingjubilee at #3759 (comment)

Maybe if we adopt this it should include a way to describe whether the use of the crate on an unsupported target is one of

• actively dangerous (implementations depend on target-specific details that must be ported)

• probably bad? but not automatically unsound (implementation depends on target-specific details that work if another target is similar enough)

• maybe it works, maybe it doesn't? glwt (the common "I've only tested it on..." situation)

(moved here to move this off the main thread so the discussion can be followed)

[epage]

Looking at the Motivation section (and from my understanding of the problem), this is meant to help with

• cargo check --workspace when some packages aren't supported
• Smaller Cargo.lock / vendor/

In those situations, the main reason for a package to limit the targets is

• a workspace member can't build on all targets
• An application will only ever be built for specific targets and they want dependencies stripped that aren't for that target
• A library is platform-specific (e.g. windows-sys) and they want to allow themselves to be stripped for the above.

imo this shouldn't be about "I only tested on this platform". I'm not even sure if we should get nuanced about the other two cases. I also don't think distinguishing these cases is worth the data format and end-user complexity.

I do think we should be clear in communicating abut this how this is intended to be used to reduce problems with it. For example, the name alone invites a lot of this kind of thinking which I noted at #3759 (comment)

[workingjubilee]

Perhaps it should be clearer that it's allowed-targets, i.e. that all others are implicitly banned.

[carloskiki]

Maybe required-targets makes it a bit more clear?

[programmerjake]

a more obtuse option that might make desired usage clearer would be to have known-broken-targets and require specifying where it doesn't work, instead of where it works.

[hanna-kruppe]

Does this section just describe conversion to disjunctive normal form? The array of strings is the top-level logical OR, and each entry is the AND of (possibly negated) atoms. If this correct, it would be helpful to make this explicit because it saves anyone who has encountered DNF before some reverse-engineering. This also makes it easier to see that this conversion can cause exponential blowup in the worst case (not a fatal problem, since dependency resolution is also NP-hard, but good to keep in mind).

[carloskiki]

I have no clue what that is, I just decided on a representation that "felt right" (I have very little academic knowledge on SAT solving). I used this representation because I thought that no-one would ever have crazy nested cfg specifications. I'll read up on it and link it there.

[ahicks92]

@workingjubilee
I have read most of the comments (new ones are coming in too rapidly) and cannot find a thread to reply to where this point is appropriate. I'll be honest that I'm very surprised how much attention this RFC is getting.

I feel the need to push back on you (I believe it was you but GH won't let me see this?...) dismissing my previous comment as off topic. I am aware that you can technically do this today. I have now read the RFC in detail. The ability to do it today if you really want is not by itself a justification for encouraging doing so. Many targets are supersets of previous targets. I do not want crates.io to slowly build up this cruft. You can model this new metadata as effectively a gradual deletion of the package, and if you do so it's concerning. This kind of thing cascades to all downstream packages, without a good mechanism for direct control, and as far as I can tell introduces the first "blessed" mechanism into the language which in effect requires maintainers to remain around. Add this, encourage people to use it, decide that such things are good code, and the ecosystem is one popular package away from Rust not being able to easily get new targets anymore. Looking at the output of rustc --print target-list even today, we can identify groups like this:

arm-linux-androideabi
arm-unknown-linux-gnueabi
arm-unknown-linux-gnueabihf
arm-unknown-linux-musleabi
arm-unknown-linux-musleabihf

Obviously the correct thing in such a case is to use cfg predicates to account for the entire groups..but will everyone do that in practice? I doubt it.

What about Nintendo Switch? Who's going to even be able to test that one given that you need an SDK from Nintendo? There's targets like that in here where the original authors of packages literally cannot get access but their code might very well have been fine.

That said this is way too noisy and you're the mod and I've made my viewpoint clear, so I'm unsubscribing for now. But I ask that if you still think I'm off topic, please give more of an explanation why this isn't a legitimate, important concern. I personally consider most of the rest of the RFC to have low relevance until this question is answered, and am quite surprised that it wasn't answered before posting it. I may check back in later once this has had a chance to settle, but the chances of me properly keeping up at this point are very low so it doesn't seem worth me trying to do so now given that the nature of this concern boils down to a yes-or-no question more than a long drawn-out discussion.

[workingjubilee]

• getrandom: https://github.com/rust-random/getrandom/blob/9fb4a9a2481018e4ab58d597ecd167a609033149/src/backends.rs#L156-L160 - This one is particularly notable because their implementation for many targets is actually identical due to using either getrandom or getentropy.

[hanna-kruppe]

How does this interact with crate features and optional dependencies? In some cases, these impose additional platform requirements, e.g., because the alloc::sync types don't exist on platforms without atomics. How could this be modeled, if at all? Platform-specific dependencies aren't allowed to mention cfg(feature = "..."), but that would be required to express e.g., "if the std feature is enabled, I require cfg(target_has_atomic="ptr")".

[joshtriplett]

That sounds like a good addition to future work, at a minimum: expressing requirements that vary by feature. There are a few possible designs I could imagine for that.

[epage]

As this affects dependency resolution, we'll also need to discuss the relevant changes to the Index.

For the existing schema, see https://doc.rust-lang.org/cargo/reference/registry-index.html#json-schema

[epage]

What does this mean? If its affecting Cargo.lock, then its affecting the resolver.

[joshtriplett]

@rfcbot concern should-this-be-in-manifest-or-updatable-metadata

We might want to consider the possibility of using the proposed mechanism for updatable crate metadata. That would allow updating this list without uploading a new version of the crate.

On the other hand, that would be less convenient and less self-contained.

[programmerjake]

We might want to consider the possibility of using the proposed mechanism for updatable crate metadata. That would allow updating this list without uploading a new version of the crate.

If that metadata is update-able, what happens when some popular crate's maintainer suddenly decides to go rogue and they replace the set of supported targets with cfg(false) or equivalent? That would effectively be another leftpad incident.

[joshtriplett]

There are several comments here that suggest it might be a problem if a crate sets this metadata and someone wants to run the crate on an unsupported target. I think it's worth having some clear documentation about recommended usage of this mechanism.

This should not, in general, be used for "I haven't tested this on other targets and I don't know if it works". This should be used for "I have good reason to believe it doesn't work as expected".

[joshtriplett]

We might want to consider the possibility of using the proposed mechanism for updatable crate metadata. That would allow updating this list without uploading a new version of the crate.

If that metadata is update-able, what happens when some popular crate's maintainer suddenly decides to go rogue and they replace the set of supported targets with cfg(false) or equivalent? That would effectively be another leftpad incident.

"What happens if a crate maintainer goes rogue" is not a question in any way specific to this mechanism. A crate maintainer could also replace the entire code of a crate with a compilation error, or worse.

[programmerjake]

If that metadata is update-able, what happens when some popular crate's maintainer suddenly decides to go rogue and they replace the set of supported targets with cfg(false) or equivalent? That would effectively be another leftpad incident.

"What happens if a crate maintainer goes rogue" is not a question in any way specific to this mechanism. A crate maintainer could also replace the entire code of a crate with a compilation error, or worse.

except that currently they can't -- they can upload a new rogue version, but they can't do anything to existing versions except yank them (which afaik doesn't break users who have it already in their lockfile). update-able metadata allows them to break existing versions -- though if that metadata was also stored in the lockfile and wasn't updated unless running cargo update or equivalent, that could fix most of the problem.

[joshtriplett]

@programmerjake I would expect it to be tied to cargo update, yes. But either way I don't think it's an issue we need to worry about.

[zackw]

I'm putting this as a top-level comment because there are some discussion threads above that might be talking about it but I'm not sure and I don't want to derail them if they aren't.

As a crate author one of the most important reasons why I want this feature is because it might enable pruning of my crate's transitive dependencies. Concrete example: I'm working on a crate that is inherently Linux-specific (it provides an interface to a Linux-specific kernel API). This crate uses rustix to make system calls. rustix itself is not Linux-specific, and in fact, despite the name, it can be used on non-Unix systems like Windows and WASI. For a while last year, rustix had transitive dependencies on two different versions of the windows-sys crate (directly and via libc) which caused clippy's "multiple-crate-versions" lint to flag my crate, even though my crate is never going to work on Windows. (See bytecodealliance/rustix#1233 for more detail. This specific problem has been fixed but you can see how things like it are liable to recur in the future.)

So, because of that experience, I really want supported-targets = [...] in my crate to mean that Cargo prunes the dependency tree very early -- during the initial computation of Cargo.lock, probably -- such that cargo tree --target all and cargo tree both report only the transitive dependency set that is relevant to the declared supported targets, and clippy doesn't even consider what rustix depends on when it's being used on other targets.

At the same time, I also think it's very important for supported-targets to be override-able. It should be possible for someone to force a build of my crate on Windows, even though this isn't going to accomplish anything useful, with no more ceremony than e.g. cargo build -- -A cargo::unsupported-target. And it should be possible for some other crate to declare a dependency on my crate even if there is no overlap whatsoever between that crate's supported targets and mine. I may not think that's a useful thing to do, but I shouldn't get the last word here! Maybe it's 30 years from now and I'm dead and never going to update my crate again, but meanwhile "Windows" and "Linux" have converged to such an extent that you can usefully use it on Windows! Of course the long-term solution in that hypothetical is for someone to take over maintenance of my crate and update the supported targets list, but in the short term, it's the depending crate that should get the last word.

I bring these two things up at the same time because I see potential for a conflict between them. Cargo should prune the dependency tree early -- but it needs to do the pruning in a way that honors any override that is happening, and I'm not sure exactly what that turns out looking like.

[epage]

Should this be moved to a future possibility to better focus this RFC?