-C linker flag is misleading

[m4b]

from rustc -C help we're told that:

    -C            linker=val -- system linker to link outputs with

I mistakenly thought that this accepted a path to a system linker (e.g., /usr/bin/ld on GNU/Linux or OSX); instead it really seems to want a compiler driver like cc or clang.

For example, one would expect the following two invocations to be the same (on a typical GNU/Linux system) if it expected an actual linker:

rustc
rustc -C linker=/usr/bin/ld

but the second errors out because ld does not understand the emulation mode "64":

note: /usr/bin/ld: unrecognised emulation mode: 64

(but cc does)

Besides being misleading, the current invocation seems to require the hard-coded command line arguments to be understood by whatever "linker" (compiler driver) is passed.

Perhaps if you're only targetting cc derivatives this might suffice. But if the flag is for switching out the linker (e.g., using gold instead of ld, or even lld), the current implementation simply will not work.

For example, suppose we're on GNU/Linux and we have our main.rs, and we compile it with the new shiny --target switches from nightly like so:

rustc --target=x86_64-apple-darwin main.rs

we'll error out in the linking phase, since cc is trying to link a mach-o binary using the system linker, /usr/bin/ld, which only understands ELF binaries.

I naively thought this was what the -C linker flag was for, for the above reasons, and so tried:

rustc --target=x86_64-apple-darwin -C linker=lld main.rs

But this of course won't work because:

1. lld actually requires -flavor darwin, etc. flags passed and the current linker= does not facilitate calling it appropriately
2. the hard-coded flags are for cc-esque driver.

So if the purpose of the -C linker flag is really a compiler driver, then probably -C driver would be a better name (and at the very least change the -C help output for the flag), but if not, then there's some work cut out... I'm thinking in the latter case it might almost be best if when the user specifies a -C linker then all linking flags are cleared, and it's up to the user to pass the appropriate flags via -C linker-flags.

[alexcrichton]

Right we currently expect linker to look like gcc on Unix platforms and link.exe on Windows platforms. We don't currently support switching to something like lld as it's got a different interface than gcc/clang. I'd be fine adding a -C linker-flavor flag or something like that which allows you to configure how we pass arguments to the linker.

[retep998]

Since lld does have a flavor where it matches link.exe style arguments, we almost support calling lld directly, for -msvc targets anyway. At the moment you just need a small shim to insert the -flavor link flag.

[m4b]

So for lld -flavor link will expect object files of input type COFF iiuc...

To input macho binaries you use the darwin flavor, for ELF you'd use gnu.

I think this is what @alexcrichton was suggesting wrt a flavor flag, that we'd need a new flag to allow fine grained control of the linker?

Note the flags lld implements are the system linker flags (/usr/bin/ld), not the cc flags:http://lld.llvm.org/open_projects.html#gnu-ld-driver

Similarly for osx.

I think this is the part I'm confused about, on the windows side it looks like the system linker, eg link.exe is used for this flag, but on the UNIX side, the compiler driver is used...

This is why as @retep998 suggested, using -flavor link almost works (for Windows), because lld implements the system linker flags for each OS/platform, which for Windows is link.exe, and Rust seems to use the system linker for Windows, but not for nix in this flag.

It's just seems a little inconsistent, and hence my suggestion the flag is misleading.

I'm not sure what the best approach is though without breaking stuff, but I suspect as cross compilation gets more and more momentum people still want finer and finer control of their linker selection on rust (and not just a compiler driver like GCC/clang/cc)... :/

[alexcrichton]

@m4b yeah to clarify I was thinking of something like -C link-flavor=(ld,cc,link.exe) or something like that, and we'd tailor appropriately. We'd continue to default to cc for most platforms but we could perhaps do a bit of detection based on -C linker as well.

We call cc instead of ld because invoking ld directly is pretty unlikely to work. It's missing information like:

• System -L paths
• Startup object files (things like crtX.o)

We currently don't probe for that information, but we'd have to do so to start calling ld directly (which I'd love to do by the way!)

[m4b]

So on the one hand, for a potentially quick win, I like the idea of a -C link-flavor flag, where if i'm understanding correctly, we'd pass it the "kind" of linker-qua-command-line style, e.g., darwin-ld, unix-ld, link.exe, cc, etc., which are then configured perhaps through -C link-args (notwithstanding some of the current hardcoded flag issues)?

On the other I'm a little worried it might be confusing? However, I think what we're really getting at here is advanced linking in rust, so I'm thinking it might be useful to outline or survey some use cases that will/are coming up w.r.t. rust linking, or what we all have in mind.

So, I actually submitted a bug report on lld because I can't seem to link rustc produced macho binaries (but I realize the error of my ways now, although it shouldn't be segfaulting), because I wanted to use it + rustc cross-compiled files to cross-link. (Side note: we could also eventually ship our own cross-platform static linker, written in Rust of course, and not have to worry about any of these issues 😈)

So for me, in particular, one of the things I'd like to use the -C link* flags for is linking the original example, say with lld, but without the error step, and in a single rustc invocation.

After much derping about, I've actually managed to use lld to fully link a mach-o object file on GNU/Linux, with a script like the following:

#!/bin/bash

# new rustup.rs nightly darwin toolchain
DIR=/home/m4b/.multirust/toolchains/nightly/lib/rustlib/x86_64-apple-darwin/lib
# assumes you've got the darwin system binaries from `/usr/lib/` hanging out here (change this to wherever they might be)
DARWIN_PATH=/home/m4b/binaries/darwin/usr/lib
SYS=$DARWIN_PATH/system
# i built lld from source, and this is the path to it
LLD=/home/m4b/Downloads/llvm-3.8.0.src/build/bin/lld

MAIN=$1
NAME=${MAIN%.*}

rustc --target=x86_64-apple-darwin $MAIN

$LLD -flavor darwin -arch x86_64 -t "$SYS/libsystem_m.dylib" "$SYS/libsystem_kernel.dylib" "$SYS/libsystem_info.dylib" "$SYS/libsystem_malloc.dylib" "$SYS/libsystem_platform.dylib" "$SYS/libsystem_c.dylib" "$SYS/libsystem_pthread.dylib" "$SYS/libunwind.dylib" "$SYS/libdyld.dylib" "$DIR/liblibc-18402db3.rlib" "$DIR/libstd-18402db3.rlib" "$DIR/libcollections-18402db3.rlib" "$DIR/librustc_unicode-18402db3.rlib" "$DIR/librand-18402db3.rlib" "$DIR/liballoc-18402db3.rlib" "$DIR/liballoc_jemalloc-18402db3.rlib" "$DIR/libcore-18402db3.rlib" "$DIR/libcompiler-rt.a"  ${NAME}.0.o -o ${NAME}_darwin

(note: while the final produced binary is "syntactically" correct, it segfaults if I attempt to run it on OSX in the getenv call, which is just a whole other can of worms :P)

So ideally, one use case would be if I could transfer the lld invocation into something like -C link-flavor=lld, which say clears all linking flags, and then the rest of the link-line, in addition to the -flavor darwin flags, etc., are passed in perhaps -C link-args=<all that crazy stuff>.

So the final rustc invocation is perhaps something like:

rustc --target=x86_64-apple-darwin main.rs -C link-flavor=lld -C link-args=<all the crazy stuff>

and it:

1. doesn't error out like in the above script, stopping with an object file artifact for the target triple,
2. is a single invocation from rustc

It's also occurred to me, depending on how mature lld becomes, it might even be easiest to ship lld within the rustup/multirust toolchain, say sitting in .multirust/toolchains/nightly/tools/lld or something, which is then standardized and rust knows about, etc. The advantage of this is that we could depend on the command line arguments, so if we're on windows, and we do:

rustc --target=x86_64-unknown-linux-gnu main.rs -C link-flavor=gnu -C link-system-home=/path/to/gnu/elf/system/binaries

(where link-system-home is a new flag I just invented to pass the system deps in a more uniform way to the linker, which rustc would then populate with known required link candidates, etc.)

The same command would work on OSX (or GNU/Linux) too, just that the path to the needed system binaries is variable, i.e., the local cross compiling paths to stuff like mscvrt.dll, libc.so.6, libsystem_c.dylib, etc.

Or again, we build our own cross-platform linker in Rust 💯 😆

Ok, that was really a mouthful, sorry, I'll let someone else talk :)

[alexcrichton]

I know I'd personally love to "just use lld", but I think as you're discovering it unfortunately seems like it's not quite ready for prime time just yet.

My recommendation for advanced linking in Rust would be to just not use the compiler altogether. You're essentially just fighting a long uphill battle trying to pack so much into the compiler, it'll be easier if you just produce a staticlib and then link that yourself (passing all the fancy flags you'd like). There are downsides to doing that, unfortunately, but the "long tail of fixing bugs" seems like it may be smaller in that case!

[m4b]

@alexcrichton

We currently don't probe for that information, but we'd have to do so to start calling ld directly (which I'd love to do by the way!)

My recommendation for advanced linking in Rust would be to just not use the compiler altogether

Now I'm a little confused, as these two statements seem to be slightly at odds.

Are you in favor of a flavor flag for calling different linkers or not? Also since rustc/cargo aren't just compilers, but also compiler drivers (otherwise why link args, and linker flags?), why wouldn't we want to add a more robust linker selection functionality?

(And yes, lld is definitely not ready, but it was simply an exampl/use case of how someone might cross link, which is an issue I see repeatedly asked, and which I mistakenly thought the solution to was the -C linker flag)

Lastly, so we don't lose sight, the original impetus for this issue is that the linker flag is misleading, which I still maintain. And especially in light of whether the official answer to advanced linking with rust is "don't use rustc"- why provide these flags at all?

[alexcrichton]

I would be fine adding flags to control the flavor of the linker, but those will take some time to stabilize and work out the kinks. In general if you're doing crazy shenanigans with the linker it's just better to do them directly because you probably want control over what's going on rather than trying to work with rustc randomly injecting arguments.

The flag is exposed because in many situations you just need to specify a different executable than the default (for example in cross-compiling situations).

[m4b]

Linker flavor seems to have been implemented in #40018

I still think -linker is misleading :P but it's too late to change I think.

Closing this issue, tho I suspect as more people experiment with cross linking more people will be confused by the name choices ;)

Anyway the cross compilation story in rust is getting really exciting, awesome work everyone !