cargo run cannot accept non-unicode arguments

[oconnor663]

Say I write a small program that just prints its args:

fn main() {
    for arg in std::env::args_os() {
        println!("{:?}", arg);
    }
}

Because it uses args_os() instead of args(), this program can handle command line args that aren't valid unicode. However, cargo run chokes:

$ cargo run $(printf '\xff')             
invalid unicode in argument: "\u{fffd}"

I have to invoke the binary directly to test the program:

$ ./target/debug/scratch $(printf '\xff')
"./target/debug/scratch"
"\u{fffd}"

Does Cargo need to inspect the command line args at all? It not, it would be nice if it didn't require them to be utf8.

[jonas-schievink]

You might want to try cargo run -- $(printf '\xff') - without the --, cargo parses the 0xff as an argument for cargo run

[oconnor663]

Oh I hadn't tried that. But it does look like I get the same result:

$ cargo run -- $(printf '\xff')
invalid unicode in argument: "\u{fffd}"

[alexcrichton]

cc docopt/docopt.rs#152 and @BurntSushi, unfortunately this is a limitation of the underlying docopt library that Cargo is currently using.

[alexcrichton]

Now that being said I think we're also storing Vec<String> in a few places instead of Vec<OsString>, so there's also changes to be had on the Cargo side as well

[BurntSushi]

Yeah, docopt should grow support for this. The arg parser itself is written to work on &str, so there are a fair amount of guts that will have to change, but shouldn't be too bad.

[johnbartholomew]

This issue still seems valid in September 2018, after cargo switched to a different flag parsing library (clap).

// main.rs
fn main() {
    for input in std::env::args_os().skip(1) {
        println!("input: {}", input.to_str().unwrap_or("<not a UTF-8 path>"));
    }
}

$ cargo build
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s                                                                                                   

$ ./target/debug/example test $(printf '\xff')
input: "test"
input: "<not a UTF-8 path>"

$ cargo run -- test $(printf '\xff')
thread 'main' panicked at 'unexpected invalid UTF-8 code point', libcore/option.rs:989:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.