ripgrep: migrate to libripgrep

This commit does the work to delete the old `grep` crate and effectively
rewrite most of ripgrep core to use the new libripgrep crates. The new
`grep` crate is now a facade that collects the various crates that make
up libripgrep.

The most complex part of ripgrep core is now arguably the translation
between command line parameters and the library options, which is
ultimately where we want to be.

.travis.yml

@@ -17,6 +17,8 @@ addons:
       # Needed for testing decompression search.
       - xz-utils
       - liblz4-tool
+      # For building MUSL static builds on Linux.
+      - musl-tools
 matrix:
   fast_finish: true
   include:

Cargo.toml

@@ -35,7 +35,6 @@ path = "tests/tests.rs"
 members = [
   "globset",
   "grep",
-  "grep2",
   "grep-matcher",
   "grep-pcre2",
   "grep-printer",
@@ -46,20 +45,15 @@ members = [
 
 [dependencies]
 atty = "0.2.11"
-bytecount = "0.3.2"
-encoding_rs = "0.8"
-encoding_rs_io = "0.1"
 globset = { version = "0.4.0", path = "globset" }
-grep = { version = "0.1.8", path = "grep" }
+grep = { version = "0.2.0", path = "grep" }
 ignore = { version = "0.4.0", path = "ignore" }
 lazy_static = "1"
-libc = "0.2"
 log = "0.4"
-memchr = "2"
-memmap = "0.6"
 num_cpus = "1"
 regex = "1"
 same-file = "1"
+serde_json = "1"
 termcolor = "1"
 
 [dependencies.clap]
@@ -69,7 +63,7 @@ features = ["suggestions", "color"]
 
 [target.'cfg(windows)'.dependencies.winapi]
 version = "0.3"
-features = ["std", "winnt"]
+features = ["std", "fileapi", "winnt"]
 
 [build-dependencies]
 lazy_static = "1"
@@ -80,15 +74,9 @@ default-features = false
 features = ["suggestions", "color"]
 
 [features]
-avx-accel = [
-  "bytecount/avx-accel",
-  "grep2/avx-accel",
-]
-simd-accel = [
-  "bytecount/simd-accel",
-  "encoding_rs/simd-accel",
-  "grep2/simd-accel",
-]
+avx-accel = ["grep/avx-accel"]
+simd-accel = ["grep/simd-accel"]
+pcre2 = ["grep/pcre2"]
 
 [profile.release]
-debug = true
+debug = 1

FAQ.md

@@ -157,13 +157,37 @@ tool. With that said,
 How do I use lookaround and/or backreferences?
 </h3>
 
-This isn't currently possible. ripgrep uses finite automata to implement
-regular expression search, and in turn, guarantees linear time searching on all
-inputs. It is difficult to efficiently support lookaround and backreferences in
-finite automata engines, so ripgrep does not provide these features.
+ripgrep's default regex engine does not support lookaround or backreferences.
+This is primarily because the default regex engine is implemented using finite
+state machines in order to guarantee a linear worst case time complexity on all
+inputs. Backreferences are not possible to implement in this paradigm, and
+lookaround appears difficult to do efficiently.
 
-If a production quality regular expression engine with these features is ever
-written in Rust, then it is possible ripgrep will provide it as an opt-in
+However, ripgrep optionally supports using PCRE2 as the regex engine instead of
+the default one based on finite state machines. You can enable PCRE2 with the
+`-P/--pcre2` flag. For example, in the root of the ripgrep repo, you can easily
+find all palindromes:
+
+```
+$ rg -P '(\w{10})\1'
+tests/misc.rs
+483:    cmd.arg("--max-filesize").arg("44444444444444444444");
+globset/src/glob.rs
+1206:    matches!(match7, "a*a*a*a*a*a*a*a*a", "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa");
+```
+
+If your version of ripgrep doesn't support PCRE2, then you'll get an error
+message when you try to use the `-P/--pcre2` flag:
+
+```
+$ rg -P '(\w{10})\1'
+PCRE2 is not available in this build of ripgrep
+```
+
+Most of the releases distributed by the ripgrep project here on GitHub will
+come bundled with PCRE2 enabled. If you installed ripgrep through a different
+means (like your system's package manager), then please reach out to the
+maintainer of that package to see whether it's possible to enable the PCRE2
 feature.