Stabilize target_feature_11

[veluca93]

Stabilization report

This is an updated version of #116114, which is itself a redo of #99767. Most of this commit and report were copied from those PRs. Thanks @LeSeulArtichaut and @calebzulawski!

Summary

Allows for safe functions to be marked with #[target_feature] attributes.

Functions marked with #[target_feature] are generally considered as unsafe functions: they are unsafe to call, cannot generally be assigned to safe function pointers, and don't implement the Fn* traits.

However, calling them from other #[target_feature] functions with a superset of features is safe.

// Demonstration function
#[target_feature(enable = "avx2")]
fn avx2() {}

fn foo() {
    // Calling `avx2` here is unsafe, as we must ensure
    // that AVX is available first.
    unsafe {
        avx2();
    }
}

#[target_feature(enable = "avx2")]
fn bar() {
    // Calling `avx2` here is safe.
    avx2();
}

Moreover, once #135504 is merged, they can be converted to safe function pointers in a context in which calling them is safe:

// Demonstration function
#[target_feature(enable = "avx2")]
fn avx2() {}

fn foo() -> fn() {
    // Converting `avx2` to fn() is a compilation error here.
    avx2
}

#[target_feature(enable = "avx2")]
fn bar() -> fn() {
    // `avx2` coerces to fn() here
    avx2
}

See the section "Closures" below for justification of this behaviour.

Test cases

Tests for this feature can be found in tests/ui/target_feature/.

Edge cases

Closures

• target-feature 1.1: should closures inherit target-feature annotations? #73631

Closures defined inside functions marked with #[target_feature] inherit the target features of their parent function. They can still be assigned to safe function pointers and implement the appropriate Fn* traits.

#[target_feature(enable = "avx2")]
fn qux() {
    let my_closure = || avx2(); // this call to `avx2` is safe
    let f: fn() = my_closure;
}

This means that in order to call a function with #[target_feature], you must guarantee that the target-feature is available while the function, any closures defined inside it, as well as any safe function pointers obtained from target-feature functions inside it, execute.

This is usually ensured because target features are assumed to never disappear, and:

• on any unsafe call to a #[target_feature] function, presence of the target feature is guaranteed by the programmer through the safety requirements of the unsafe call.
• on any safe call, this is guaranteed recursively by the caller.

If you work in an environment where target features can be disabled, it is your responsibility to ensure that no code inside a target feature function (including inside a closure) runs after this (until the feature is enabled again).

Note: this has an effect on existing code, as nowadays closures do not inherit features from the enclosing function, and thus this strengthens a safety requirement. It was originally proposed in #73631 to solve this by adding a new type of UB: “taking a target feature away from your process after having run code that uses that target feature is UB” .
This was motivated by userspace code already assuming in a few places that CPU features never disappear from a program during execution (see i.e. /~https://github.com/rust-lang/stdarch/blob/2e29bdf90832931ea499755bb4ad7a6b0809295a/crates/std_detect/src/detect/arch/x86.rs); however, concerns were raised in the context of the Linux kernel; thus, we propose to relax that requirement to "causing the set of usable features to be reduced is unsafe; when doing so, the programmer is required to ensure that no closures or safe fn pointers that use removed features are still in scope".

• Fix #[inline(always)] on closures with target feature 1.1 #111836

Closures accept #[inline(always)], even within functions marked with #[target_feature]. Since these attributes conflict, #[inline(always)] wins out to maintain compatibility.

ABI concerns

• The extern "C" ABI of SIMD vector types depends on target features #116558

The ABI of some types can change when compiling a function with different target features. This could have introduced unsoundness with target_feature_11, but recent fixes (#133102, #132173) either make those situations invalid or make the ABI no longer dependent on features. Thus, those issues should no longer occur.

Special functions

The #[target_feature] attribute is forbidden from a variety of special functions, such as main, current and future lang items (e.g. #[start], #[panic_handler]), safe default trait implementations and safe trait methods.

This was not disallowed at the time of the first stabilization PR for target_features_11, and resulted in the following issues/PRs:

• #[target_feature] is allowed on main #108645
• #[target_feature] is allowed on default implementations #108646
• #[target_feature] is allowed on #[panic_handler] with target_feature 1.1 #109411
• Prevent using #[target_feature] on lang item functions #115910

Documentation

• Reference: Document the target_feature_11 feature reference#1181

---

cc tracking issue #69098
cc @workingjubilee
cc @RalfJung
r? @rust-lang/lang

[RalfJung]

Closures defined inside functions marked with #[target_feature] inherit the target features of their parent function. They can still be assigned to safe function pointers and implement the appropriate Fn* traits.

Oddly, this means this compiles

#[target_feature(enable = "avx2")]
fn bar() -> fn() {
    || avx2()
}

but this does not

#[target_feature(enable = "avx2")]
fn bar() -> fn() {
    avx2
}

That's not a blocker but might be worth an issue. (But changing this may mean we have to refine the safety condition.)

This means that in order to call a function with #[target_feature], you must guarantee that the target-feature is available while the function executes and for as long as any returned closures (or function pointers) that inherit features from that function live.

The definition of when something "live"s is subtle. I would make it about execution: you must guarantee that the target feature is available while the function or any closure defined inside that function executes. This is generally ensured because target features, once available, cannot usually be taken back; if you work in an environment where they can be taken back, it is your responsibility to ensure that no code inside a target feature function (including inside a closure) runs after this (until the feature is enabled again).

The #[target_feature] attribute is forbidden from a variety of special functions

There is an enumeration saying that the attribute is allowed on main etc, only to then state that it is forbidden. This is confusing.

Have we covered all special functions?

[tmandry]

My feeling is that this is a limited kind of effects/contexts system and we will want to build it into a broader picture one day. But it's okay to do something one-off to unblock the use case, and either make it fit in a bigger picture later, or worst case, deprecate #[target_feature] and replace it with something else.

@rfcbot reviewed

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[joshtriplett]

I think, for the moment, treating target features as permanently available once detected is consistent with what the vast majority of programs assume.

Notable exceptions include the Linux kernel, where some features are available in a "scoped" fashion (e.g. kernel_fpu_begin/kernel_fpu_end or equivalent). I think we should support that as well, but I think it's reasonable for that support to come later.

[RalfJung]

And of course we already "support" this unsafely. "You can do it if you are careful" is generally not a great answer, but it's something in particular the Linux folks are very used to and know how to deal with.

[lcnr]

as discussed in https://rust-lang.zulipchat.com/#narrow/channel/144729-t-types/topic/request.20for.20breaking.20soundness.20of.20target.20feature.201.2E1/near/490224983

This relies on closures (and likely fn-ptrs as I also believe #134090 (comment) should compile) do not implement Default (or any trait like it which allows creating a callable ZST/fn-ptr without already haivng one available)

I believe this to be otherwise alright from a type system pov

[RalfJung]

What would go wrong if a closure type (I assume you mean the unnamed type generated during lowering) would implement Default?

For function item types, having a Default impl does not seem unreasonable, and that should suffice for whatever you want to do here, doesn't it?

[oli-obk]

What would go wrong if a closure type (I assume you mean the unnamed type generated during lowering) would implement Default?

For function item types, having a Default impl does not seem unreasonable, and that should suffice for whatever you want to do here, doesn't it?

#![target_feature(foo)]
fn foo() {}

fn main() {
    let f = if true {
        None
    } else {
        unsafe{Some(|| foo())}
    }.unwrap_or_default();
    f();
}

[RalfJung]

Oh, that is nasty... so let's assume we did have impl Default for fn item types, and we do something like #135504 for function item types. Both of these sound pretty reasonable. Then we could do this, no?

#[target_feature(foo)]
fn with_foo() {}

#[target_feature(foo)]
fn mk_foo() -> impl FnOnce() + Default { with_foo }

fn main() {
    let f = if true {
        None
    } else {
        // SAFETY: this is dead code.
        unsafe{Some(mk_foo())}
    }.unwrap_or_default();
    f();
}

I am increasingly leaning towards not inheriting target features for closures, fn ptrs, or fn items...

[veluca93]

Oh, that is nasty... so let's assume we did have impl Default for fn item types , and we do something like #135504 for function item types. Both of these sound pretty reasonable. Then we could do this, no?

#[target_feature(foo)]
fn with_foo() {}

#[target_feature(foo)]
fn mk_foo() -> impl FnOnce() + Default { with_foo }

fn main() {
    let f = if true {
        None
    } else {
        // SAFETY: this is dead code.
        unsafe{Some(mk_foo())}
    }.unwrap_or_default();
    f();
}

I am increasingly leaning towards not inheriting target features...

I don't believe the with_foo fn item type could ever reasonably implement FnOnce, no? That would be unsound in other ways.

The fn pointer type does implement FnOnce, but it's unclear how it would implement Default then.

[RalfJung]

Good point, "#[target_feature] functions do not implement the Fn traits".

For function pointers, Default clearly makes no sense, so the only possibility is something like "captureless closure could implement Default" or even "closures where all captures are Default could implement Default". The latter is a terrible idea IMO, but we do the same for Clone which IMO is also a terrible idea so who knows what else we'll do in the future.^^

I guess worst-case we could say that closures which inherit #[target_feature] do not implement Default even if other closures do.

Still I feel like the motivation for this target feature inheritance was not spelled out very well, all I could find is this one example. The RFC does not mention closures, and this seems like a major feature with subtle interactions.

[veluca93]

Good point, "#[target_feature] functions do not implement the Fn traits".

For function pointers, Default clearly makes no sense, so the only possibility is something like "captureless closure could implement Default" or even "closures where all captures are Default could implement Default". The latter is a terrible idea IMO, but we do the same for Clone which IMO is also a terrible idea so who knows what else we'll do in the future.^^

The latter does sound like a terrible idea, I would be surprised if that would not easily cause unsoundness in existing code...

I guess worst-case we could say that closures which inherit #[target_feature] do not implement Default even if other closures do.

Still I feel like the motivation for this target feature inheritance was not spelled out very well, all I could find is this one example. The RFC does not mention closures, and this seems like a major feature with subtle interactions.

Doesn't the initial post in that issue spell out a clear reason for inheriting feature annotations?

#[target_feature(enable="avx")]
fn also_use_avx() {
    println!("Hello from AVX")
}

#[target_feature(enable="avx")]
fn use_avx() {
    let f = || also_use_avx();
    f();
}

This code would fail to compile, and require unsafe, otherwise -- that would seem surprising to me.

[RalfJung]

The latter does sound like a terrible idea, I would be surprised if that would not easily cause unsoundness in existing code...

The latter being Clone? That's long stable.

This code would fail to compile, and require unsafe, otherwise -- that would seem surprising to me.

It wouldn't require unsafe, it'd require extra annotations:

#[target_feature(enable="avx")]
fn also_use_avx() {
    println!("Hello from AVX")
}

#[target_feature(enable="avx")]
fn use_avx() {
    let f = #[target_feature(enable="avx")] || also_use_avx();
    f();
}

We allow #[inline] on closures so we might as well allow target_feature as well.

[RalfJung]

Oh wait, that would not work since a closure with those attributes wouldn't implement the Fn* traits...

So really this target feature inheritance is a poor substitute for "I want to call these higher-order functions with something that is not quite a regular safe function"... we don't have the effect system required to properly typecheck this so we use a somewhat crude but probably effective hack. That makes sense, it could have been spelled out more explicitly. :) @tmandry alluded to this but I didn't get it when I read their message the first time.

[oli-obk]

I don't think target_feature is special here. You have the same "issue" for unsafe functions:

unsafe fn foo() {}

fn main() {
    let f = if true {
        None
    } else {
        unsafe{Some(|| foo())}
    }.unwrap_or_default();
    f();
}

[veluca93]

The latter does sound like a terrible idea, I would be surprised if that would not easily cause unsoundness in existing code...

The latter being Clone? That's long stable.

No, I meant closures implicitly implementing default if they could :-)

This code would fail to compile, and require unsafe, otherwise -- that would seem surprising to me.

It wouldn't require unsafe, it'd require extra annotations:

#[target_feature(enable="avx")]
fn also_use_avx() {
    println!("Hello from AVX")
}

#[target_feature(enable="avx")]
fn use_avx() {
    let f = #[target_feature(enable="avx")] || also_use_avx();
    f();
}

We allow #[inline] on closures so we might as well allow target_feature as well.

This would indeed fix the requirement for unsafe, but I am not sure it actually does anything for the actual "this is potentially risky" concerns.
Arguably, for the closures to be useful they need to be used wherever one would expect Fn traits, as otherwise i.e. all Iterator methods would not be usable with them. Thus, one could still return a impl Fn, and the concerns here become somewhat orthogonal from whether features are inherited or explicitly annotated.

[RalfJung]

I don't think target_feature is special here. You have the same "issue" for unsafe functions:

Ah right, unsafe fn cannot have Default on their item type. Good point.

[lcnr]

My worry here is not even necessary having the unsafe in dead code and then reconstructing the closure otherwise.

I expect it to be likely that you're able to extract the closure type out of a target feature fn without ever using unsafe using opaque types. I am not confident that the type system does not allow that for RPIT and it definitely allows it with TAITs. So if closures implement Default, you don't need a single call-site of its unsafe parent

[veluca93]

My worry here is not even necessary having the unsafe in dead code and then reconstructing the closure otherwise.

I expect it to be likely that you're able to extract the closure type out of a target feature fn without ever using unsafe using opaque types. I am not confident that the type system does not allow that for RPIT and it definitely allows it with TAITs. So if closures implement Default, you don't need a single call-site of its unsafe parent

Sure, but as Oli was pointing out, that situation would be broken already, right?

In particular it would allow you to synthetize closures generated in unsafe blocks, bypassing checks people might have put there:

fn f() -> impl FnOnce() {
   // do some check
   unsafe { || some_unsafe_fn() } 
}

fn bar() {
  if check() {
    unsafe { Some(|| some_unsafe_fn()) }
  } else {
    None
  }.unwrap_or_default()
}

If closure types ever implemented Default, both of those examples would be unsound.

[lcnr]

If closure types ever implemented Default, both of those examples would be unsound.

No, because the function only returns impl FnOnce(), so even if the underlying type implements Default, you cannot use it without adding a + Default bound to the opaque type (ignoring specialization, which is yet another issue with that feature). With target_feature_11 you can get that behavior without using an unsafe block anywhere. i.e. if you've got

// Unsound without ever actually using `unsafe`
type Tait = impl FnOnce() + Default;
#[target_feature(whatever)]
fn f() -> Tait {
   || target_feature_fn()
}

[veluca93]

Even if target_feature would create a slightly worse situation were Default to be implemented for all closures that could implement Default, I am not convinced that doing so would ever be a good idea, because it would make it significantly harder to write sound unsafe code.

If the eventual decision is to make closures implement Default if possible, it would be fairly easy to have closures that do inherit target_features not implement Default, as Ralf suggested, no?

[RalfJung]

"No unsafe block" and "unsafe block only in dead code" does not make much of a difference IMO.