Skip to content

Commit

Permalink
Merge pull request #37 from wookietreiber/autotools-build
Browse files Browse the repository at this point in the history
autotools build
  • Loading branch information
wookietreiber committed Aug 7, 2015
2 parents afd3914 + 00bad4e commit 2ea45c3
Show file tree
Hide file tree
Showing 19 changed files with 639 additions and 146 deletions.
45 changes: 38 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,40 @@
pixz
pixz.1
dist
pixz-*.tgz
# created by autoscan

*.o
/autom4te.cache
/autoscan.log
/configure.scan

# created by autoreconf -fi (includes autoconf, automake, libtool)

Makefile.in
/aclocal.m4
/autom4te.cache
/compile
/config.guess
/config.h.in*
/config.sub
/configure
/depcomp
/install-sh
/ltmain.sh
/missing

# created by configure

libs
tars
Makefile
/config.h
/config.log
/config.status
/libtool
/stamp-h1
.deps

# created by make

*.la
*.lo
*.o
.libs
pixz
pixz.1
*.tar.*
56 changes: 0 additions & 56 deletions Makefile

This file was deleted.

5 changes: 5 additions & 0 deletions Makefile.am
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
ACLOCAL_AMFLAGS = -I m4

SUBDIRS = src

EXTRA_DIST = LICENSE m4 NEWS README test.sh TODO
80 changes: 0 additions & 80 deletions README

This file was deleted.

182 changes: 182 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
pixz
====

Pixz (pronounced *pixie*) is a parallel, indexing version of `xz`.

Repository: /~https://github.com/vasi/pixz

Downloads: /~https://github.com/vasi/pixz/releases

pixz vs xz
----------

The existing [XZ Utils](http://tukaani.org/xz/) provide great compression in the `.xz` file format,
but they have two significant problems:

- they are single-threaded, while most users nowadays have multi-core computers
- the `.xz` files they produce are just one big block of compressed data, rather than a collection
of smaller blocks which makes random access to the original data impossible

With pixz, both these problems are solved.

Building pixz
-------------

General help about the building process's configuration step can be acquired via:

```
./configure --help
```

### Dependencies

- pthreads
- liblzma 4.999.9-beta-212 or later (from the xz distribution)
- libarchive 2.8 or later
- AsciiDoc to generate the man page

### Build from Release Tarball

```
./configure
make
make install
```

You many need `sudo` permissions to run `make install`.

### Build from GitHub

```
git clone /~https://github.com/vasi/pixz.git
cd pixz
./autogen.sh
./configure
make
make install
```

You many need `sudo` permissions to run `make install`.

Usage
-----

### Single Files

Compress a single file (no tarball, just compression), multi-core:

pixz bar bar.xz

Decompress it, multi-core:

pixz -d bar.xz bar

### Tarballs

Compress and index a tarball, multi-core:

pixz foo.tar foo.tpxz

Very quickly list the contents of the compressed tarball:

pixz -l foo.tpxz

Decompress the tarball, multi-core:

pixz -d foo.tpxz foo.tar

Very quickly extract a single file, multi-core, also verifies that contents match index:

pixz -x dir/file < foo.tpxz | tar x

Create a tarball using pixz for multi-core compression:

tar -Ipixz -cf foo.tpxz foo/

### Specifying Input and Output

These are the same (also work for `-x`, `-d` and `-l` as well):

pixz foo.tar foo.tpxz
pixz < foo.tar > foo.tpxz
pixz -i foo.tar -o foo.tpxz

Extract the files from `foo.tpxz` into `foo.tar`:

pixz -x -i foo.tpxz -o foo.tar file1 file2 ...

Compress to `foo.tpxz`, removing the original:

pixz foo.tar

Extract to `foo.tar`, removing the original:

pixz -d foo.tpxz

### Other Flags

Faster, worse compression:

pixz -1 foo.tar

Better, slower compression:

pixz -9 foo.tar

Use exactly 2 threads:

pixz -p 2 foo.tar

Compress, but do not treat it as a tarball, i.e. do not index it:

pixz -t foo.tar

Decompress, but do not check that contents match index:

pixz -d -t foo.tpxz

List the xz blocks instead of files:

pixz -l -t foo.tpxz

For even more tuning flags, check the manual page:

man pixz

Comparison to other Tools
-------------------------

### plzip

- about equally complex and efficient
- lzip format seems less-used
- version 1 is theoretically indexable, I think

### ChopZip

- written in Python, much simpler
- more flexible, supports arbitrary compression programs
- uses streams instead of blocks, not indexable
- splits input and then combines output, much higher disk usage

### pxz

- simpler code
- uses OpenMP instead of pthreads
- uses streams instead of blocks, not indexable
- uses temporary files and does not combine them until the whole file is compressed, high disk and
memory usage

### pbzip2

- not indexable
- appears slow
- bzip2 algorithm is non-ideal

### pigz

- not indexable

### dictzip, idzip

- not parallel
1 change: 0 additions & 1 deletion TODO
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ CLEANUP
* error handling
* signal handling
* globals
* autoconf
* optimized settings
* memory limit
* cpu number
Expand Down
3 changes: 3 additions & 0 deletions autogen.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh

autoreconf -fi
Loading

0 comments on commit 2ea45c3

Please sign in to comment.