Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multiple definition link error when using multiple codegen units #32518

Closed
MagaTailor opened this issue Mar 26, 2016 · 10 comments · Fixed by #32557
Closed

multiple definition link error when using multiple codegen units #32518

MagaTailor opened this issue Mar 26, 2016 · 10 comments · Fixed by #32557
Assignees
Labels
regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@MagaTailor
Copy link

I wanted to use @dotdash's LLVM assertions patch but now my stage1 libstd fails to link. Probably unrelated but here it is:

rustc: arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libstd
error: linking with `cc` failed: exit code: 1
note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-L" "/rust-master/arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib" "arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.0.o" "arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.1.o" "arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.2.o" "arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.3.o" "-o" "arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libstd-9026086f.so" "arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.metadata.o" "-Wl,-O1" "-nodefaultlibs" "-L" "arm-unknown-linux-gnueabihf/rt" "-L" "/rust-master/arm-unknown-linux-gnueabihf/llvm/Release+Asserts/lib" "-L" "/rust-master/arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib" "-Wl,-Bstatic" "-Wl,--whole-archive" "-l" "backtrace" "-Wl,--no-whole-archive" "-Wl,-Bdynamic" "-l" "dl" "-l" "pthread" "-l" "gcc_s" "-Wl,--whole-archive" "/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FHfUQVlQsuoG/liballoc-9026086f.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FHfUQVlQsuoG/librustc_unicode-9026086f.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FHfUQVlQsuoG/liballoc_system-9026086f.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FHfUQVlQsuoG/liblibc-9026086f.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FHfUQVlQsuoG/librand-9026086f.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.FHfUQVlQsuoG/libcore-9026086f.rlib" "-Wl,--no-whole-archive" "-l" "c" "-l" "m" "-l" "rt" "-shared" "-Wl,-rpath,/home/odroid/rust-nightly-sysalloc-assert/lib/rustlib/arm-unknown-linux-gnueabihf/lib" "-Wl,--enable-new-dtags" "-s" "-l" "compiler-rt"
note: /tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `_$LT$core..option..Option$LT$T$GT$$GT$::map::h58518f67af9bf04e':
collections.0.rs:(.text._ZN38_$LT$core..option..Option$LT$T$GT$$GT$3map17h58518f67af9bf04eE+0x0): multiple definition of `_$LT$core..option..Option$LT$T$GT$$GT$::map::h58518f67af9bf04e'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.0.o:std.0.rs:(.text._ZN38_$LT$core..option..Option$LT$T$GT$$GT$3map17h58518f67af9bf04eE+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$::new::h0c1cdac7fb2b416c':
collections.0.rs:(.text._ZN40_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$3new17h0c1cdac7fb2b416cE+0x0): multiple definition of `_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$::new::h0c1cdac7fb2b416c'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.3.o:std.3.rs:(.text._ZN40_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$3new17h0c1cdac7fb2b416cE+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$::ptr::h4e14ee1392d1411e':
collections.0.rs:(.text._ZN40_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$3ptr17h4e14ee1392d1411eE+0x0): multiple definition of `_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$::ptr::h4e14ee1392d1411e'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.0.o:std.0.rs:(.text._ZN40_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$3ptr17h4e14ee1392d1411eE+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$::double::heaf5b97a8495eb9e':
collections.0.rs:(.text._ZN40_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$6double17heaf5b97a8495eb9eE+0x0): multiple definition of `_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$::double::heaf5b97a8495eb9e'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.3.o:std.3.rs:(.text._ZN40_$LT$alloc..raw_vec..RawVec$LT$T$GT$$GT$6double17heaf5b97a8495eb9eE+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `core::mem::uninitialized::he8ba150ad18eedcf':
collections.0.rs:(.text._ZN4core3mem13uninitialized17he8ba150ad18eedcfE+0x0): multiple definition of `core::mem::uninitialized::he8ba150ad18eedcf'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.0.o:std.0.rs:(.text._ZN4core3mem13uninitialized17he8ba150ad18eedcfE+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `core::mem::uninitialized::hfa1a47f766d26f10':
collections.0.rs:(.text._ZN4core3mem13uninitialized17hfa1a47f766d26f10E+0x0): multiple definition of `core::mem::uninitialized::hfa1a47f766d26f10'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.3.o:std.3.rs:(.text._ZN4core3mem13uninitialized17hfa1a47f766d26f10E+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `core::mem::transmute_copy::h5f22bd4094d00101':
collections.0.rs:(.text._ZN4core3mem14transmute_copy17h5f22bd4094d00101E+0x0): multiple definition of `core::mem::transmute_copy::h5f22bd4094d00101'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.0.o:std.0.rs:(.text._ZN4core3mem14transmute_copy17h5f22bd4094d00101E+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `core::mem::transmute_copy::h75f84324b8ee3166':
collections.0.rs:(.text._ZN4core3mem14transmute_copy17h75f84324b8ee3166E+0x0): multiple definition of `core::mem::transmute_copy::h75f84324b8ee3166'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.3.o:std.3.rs:(.text._ZN4core3mem14transmute_copy17h75f84324b8ee3166E+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `core::mem::forget::h502109c553961f51':
collections.0.rs:(.text._ZN4core3mem6forget17h502109c553961f51E+0x0): multiple definition of `core::mem::forget::h502109c553961f51'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.0.o:std.0.rs:(.text._ZN4core3mem6forget17h502109c553961f51E+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `core::mem::size_of::hfb509be3c362d215':
collections.0.rs:(.text._ZN4core3mem7size_of17hfb509be3c362d215E+0x0): multiple definition of `core::mem::size_of::hfb509be3c362d215'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.0.o:std.0.rs:(.text._ZN4core3mem7size_of17hfb509be3c362d215E+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `core::mem::align_of::h2a452ce69533a17c':
collections.0.rs:(.text._ZN4core3mem8align_of17h2a452ce69533a17cE+0x0): multiple definition of `core::mem::align_of::h2a452ce69533a17c'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.0.o:std.0.rs:(.text._ZN4core3mem8align_of17h2a452ce69533a17cE+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `core::ptr::_$LT$impl$u20$$BP$mut$u20$T$GT$::is_null::h3fe8e4e4a8815d84':
collections.0.rs:(.text._ZN4core3ptr31_$LT$impl$u20$$BP$mut$u20$T$GT$7is_null17h3fe8e4e4a8815d84E+0x0): multiple definition of `core::ptr::_$LT$impl$u20$$BP$mut$u20$T$GT$::is_null::h3fe8e4e4a8815d84'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.0.o:std.0.rs:(.text._ZN4core3ptr31_$LT$impl$u20$$BP$mut$u20$T$GT$7is_null17h3fe8e4e4a8815d84E+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `core::ptr::_$LT$impl$u20$$BP$const$u20$T$GT$::offset::he051aa5e5b8f8bfb':
collections.0.rs:(.text._ZN4core3ptr33_$LT$impl$u20$$BP$const$u20$T$GT$6offset17he051aa5e5b8f8bfbE+0x0): multiple definition of `core::ptr::_$LT$impl$u20$$BP$const$u20$T$GT$::offset::he051aa5e5b8f8bfb'
arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib/std-9026086f.0.o:std.0.rs:(.text._ZN4core3ptr33_$LT$impl$u20$$BP$const$u20$T$GT$6offset17he051aa5e5b8f8bfbE+0x0): first defined here
/tmp/rustc.FHfUQVlQsuoG/libcollections-9026086f.rlib(collections-9026086f.0.o): In function `core::ptr::_$LT$impl$u20$$BP$const$u20$T$GT$::is_null::h630d425861f40502':
collections.0.rs:(.text._ZN4core3ptr33_$LT$impl$u20$$BP$const$u20$T$GT$7is_null17h630d425861f40502E+0x0): multiple definition of `core::ptr::_$LT$impl$u20$$BP$const$u20$T$GT$::is_null::h630d425861f40502'

and so on. Configured with --enable-optimize --disable-jemalloc --enable-llvm-assertions and RUSTFLAGS='-C codegen-units=4 -C link-args=-s'

Are all these errors about memory management functions? (--disable-jemalloc broken again?)

@MagaTailor MagaTailor changed the title Stage1 libstd doesn't link on ARM any more multiple definition link error in libstd on ARM Mar 26, 2016
@sanxiyn sanxiyn added the O-Arm Target: 32-bit Arm processors (armv6, armv7, thumb...), including 64-bit Arm in AArch32 state label Mar 27, 2016
@sanxiyn
Copy link
Member

sanxiyn commented Mar 27, 2016

Does it happen without codegen-units option?

@MagaTailor
Copy link
Author

Ha, good question, let me check! It used to work fine even with codegen units maybe a week ago.

@MagaTailor
Copy link
Author

@sanxiyn You were right, it was parallel codegen causing problems here.

However, when it used to work, the llvm builds were just the regular Release ones, never with assertions enabled. (probably irrelevant)

My guess is one of the recent llvm updates is causing all that pain on ARM. (here and here)

@alexcrichton
Copy link
Member

cc @nikomatsakis, perhaps this is a regression from #32469

@alexcrichton alexcrichton added regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. I-nominated T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Mar 27, 2016
@alexcrichton
Copy link
Member

This error also isn't related to ARM, you can reproduce with a bootstrap with some number of codegen units (I used 8 locally)

@alexcrichton alexcrichton removed the O-Arm Target: 32-bit Arm processors (armv6, armv7, thumb...), including 64-bit Arm in AArch32 state label Mar 27, 2016
@dotdash dotdash changed the title multiple definition link error in libstd on ARM multiple definition link error when using multiple codegen units Mar 27, 2016
@dotdash
Copy link
Contributor

dotdash commented Mar 27, 2016

Minimized:

cgu.rs

#![crate_type="lib"]
pub fn id<T>(t: T) {
}

a.rs

#![crate_type="lib"]
extern crate cgu;
pub mod a {
    pub fn a() {
        ::cgu::id(());
    }
}
pub mod b {
    pub fn a() {
        ::cgu::id(());
    }
}

b.rs

#![crate_type="lib"]
extern crate cgu;
pub mod a {
    pub fn a() {
        ::cgu::id(());
    }
}
pub mod b {
    pub fn a() {
        ::cgu::id(());
    }
}

main.rs

extern crate a;
extern crate b;
fn main() {
  a::a::a();
  b::a::a();
}

The problem here is that the id function no longer gets internal linkage when it's used in more than one codegen unit.

@dotdash dotdash self-assigned this Mar 28, 2016
@nikomatsakis
Copy link
Contributor

Seems likely to be a regression, yes. @dotdash, I take it from the assignment that you are fixing it? (If so, how do you plan to do so?)

@nikomatsakis
Copy link
Contributor

(I imagine that whatever we do for drop glue ought to work here, since I believe those are also shared across CGU.)

@MagaTailor
Copy link
Author

OK, the assertions build (rustc 1.9.0-dev (346d0d517 2016-03-26)) produces the following ICE, compiling the igd 0.4.2 crate:

thread 'rustc' panicked at 'assertion failed: declare::get_defined_value(ccx, &symbol).is_none()', src/librustc_trans/trans/monomorphize.rs:95

Posting here in case it's related to the PR that'd been linked above.

@sanxiyn
Copy link
Member

sanxiyn commented Mar 28, 2016

igd ICE is #32530.

dotdash added a commit to dotdash/rust that referenced this issue Mar 29, 2016
When reuing a definition across codegen units, we obviously cannot use
internal linkage, but using external linkage means that we can end up
with multiple conflicting definitions of a single symbol across
multiple crates. Since the definitions should all be equal
semantically, we can use weak_odr linkage to resolve the situation.

Fixes rust-lang#32518
bors added a commit that referenced this issue Mar 29, 2016
Use weak_odr linkage when reusing definitions across codegen units

When reuing a definition across codegen units, we obviously cannot use
internal linkage, but using external linkage means that we can end up
with multiple conflicting definitions of a single symbol across
multiple crates. Since the definitions should all be equal
semantically, we can use weak_odr linkage to resolve the situation.

Fixes #32518

r? @nikomatsakis
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants