Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustbuild: Switch back to using hard links #44260

Merged
merged 1 commit into from
Sep 10, 2017

Conversation

alexcrichton
Copy link
Member

The copy function historically in rustbuild used hard links to speed up the
copy operations that it does. This logic was backed out, however, in #39518 due
to a bug that only showed up on Windows, described in #39504. The cause
described in #39504 happened because Cargo, on a fresh build, would overwrite
the previous artifacts with new hard links that Cargo itself manages.

This behavior in Cargo was fixed in rust-lang/cargo#4390 where it no longer
should overwrite files on fresh builds, opportunistically leaving the filesystem
intact and not touching it.

Hopefully this can help speed up local builds by doing fewer copies all over the
place!

@alexcrichton
Copy link
Member Author

r? @Mark-Simulacrum

@rust-highfive
Copy link
Collaborator

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

@Mark-Simulacrum
Copy link
Member

@bors r+ p=1 (may speed up builds)

@bors
Copy link
Contributor

bors commented Sep 2, 2017

📌 Commit 53f6404 has been approved by Mark-Simulacrum

@bors
Copy link
Contributor

bors commented Sep 2, 2017

⌛ Testing commit 53f64045c9b5fee475c41ddf271a8fa799609102 with merge 3bc47abcfba717e2172768a4f2e0a93157b260e2...

@bors
Copy link
Contributor

bors commented Sep 2, 2017

💔 Test failed - status-appveyor

@kennytm
Copy link
Member

kennytm commented Sep 2, 2017

dist x86_64-pc-windows-msvc failed. std::fs::copy errored. Maybe caused by a process trying to override a hard-linked file?

Dist rustc stage2 (x86_64-pc-windows-msvc)
thread 'main' panicked at 'fs::copy(src, &dst) failed with The process cannot access the file because it is being used by another process. (os error 32)', src\bootstrap\dist.rs:897:4
note: Run with `RUST_BACKTRACE=1` for a backtrace.
failed to run: C:\projects\rust\build\bootstrap\debug\bootstrap dist

@arielb1 arielb1 added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Sep 5, 2017
@Mark-Simulacrum
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Sep 6, 2017

📌 Commit 5381c40 has been approved by Mark-Simulacrum

@Mark-Simulacrum Mark-Simulacrum added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 6, 2017
@alexcrichton
Copy link
Member Author

@bors: r-

er sorry accidentally pushed, still investigating

@aidanhs aidanhs added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 7, 2017
@alexcrichton
Copy link
Member Author

@bors: r=Mark-Simulacrum

@bors
Copy link
Contributor

bors commented Sep 9, 2017

📌 Commit 5381c40 has been approved by Mark-Simulacrum

@alexcrichton
Copy link
Member Author

@bors: r=Mark-Simulacrum

@bors
Copy link
Contributor

bors commented Sep 9, 2017

📌 Commit 71318a6 has been approved by Mark-Simulacrum

@bors
Copy link
Contributor

bors commented Sep 9, 2017

⌛ Testing commit 71318a65c7b9b485dd92e34074ec85f79c875754 with merge ab0b9195df04f09f53af8ab224446272dc2c0c10...

@bors
Copy link
Contributor

bors commented Sep 9, 2017

💔 Test failed - status-travis

The `copy` function historically in rustbuild used hard links to speed up the
copy operations that it does. This logic was backed out, however, in rust-lang#39518 due
to a bug that only showed up on Windows, described in rust-lang#39504. The cause
described in rust-lang#39504 happened because Cargo, on a fresh build, would overwrite
the previous artifacts with new hard links that Cargo itself manages.

This behavior in Cargo was fixed in rust-lang/cargo#4390 where it no longer
should overwrite files on fresh builds, opportunistically leaving the filesystem
intact and not touching it.

Hopefully this can help speed up local builds by doing fewer copies all over the
place!
@alexcrichton
Copy link
Member Author

@bors: r=Mark-Simulacrum

@bors
Copy link
Contributor

bors commented Sep 10, 2017

📌 Commit 38bedfa has been approved by Mark-Simulacrum

@bors
Copy link
Contributor

bors commented Sep 10, 2017

⌛ Testing commit 38bedfa with merge 6d445e1...

bors added a commit that referenced this pull request Sep 10, 2017
rustbuild: Switch back to using hard links

The `copy` function historically in rustbuild used hard links to speed up the
copy operations that it does. This logic was backed out, however, in #39518 due
to a bug that only showed up on Windows, described in #39504. The cause
described in #39504 happened because Cargo, on a fresh build, would overwrite
the previous artifacts with new hard links that Cargo itself manages.

This behavior in Cargo was fixed in rust-lang/cargo#4390 where it no longer
should overwrite files on fresh builds, opportunistically leaving the filesystem
intact and not touching it.

Hopefully this can help speed up local builds by doing fewer copies all over the
place!
@bors
Copy link
Contributor

bors commented Sep 10, 2017

☀️ Test successful - status-appveyor, status-travis
Approved by: Mark-Simulacrum
Pushing 6d445e1 to master...

@bors bors merged commit 38bedfa into rust-lang:master Sep 10, 2017
@alexcrichton alexcrichton deleted the hardlink-no-copy branch September 10, 2017 16:30
{
let mut s = t!(fs::File::open(&src));
let mut d = t!(fs::File::create(&dst));
io::copy(&mut s, &mut d).expect("failed to copy");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious; is this construction different from fs::copy?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently on Windows yeah, I didn't dig too deeply.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants