There are some dedicated backup tools that successfully run multiple threads, but lots more that just split the workload by directory at a high level.
Usage: lbzip2 -d <2> pbzip2 -d <2> -d option is used for decompression.There was a hack for Solaris Tar for use on ZFS: 4 Answers Sorted by: 24 lbzip2 and pbzip2 are the tools which you can use for parallel compression and decompression. Actual embbeded version for tar command on Qnap is from Stone. Brian L at 17:30 I would still recommend using LZMA/Deflate even though it's only single threaded. Tar compressor with multithreading compression, replace the old version from Qnap firmware. All files compressed in multi-threaded mode meet this condition. According to /7zip/MANUAL/switches/method.htmZipMultiThread, you can specify the number of threads to use with, -mmtN. For instance: tar -I pbzip2 -cf 2 pathstoarchive tar -use-compress-programpigz -cf pathstoarchive. Most popular multithread archivers are pigz (instead of gzip) and pbzip2 (instead of bzip2). I'm not sure of any actual parallel archivers that are mainstream. It will only work on files that contain multiple blocks with size information in block headers. You can use multithread version of archiver or compressor utility. If you want a fast single-threaded compressor you can use lz4. Examples: tar -I 'zstd -T0' or tar -use-compress-programpigz. This is done using the -I argument to tar, which tells tar what program to use to compress the tar archive, and what arguments to pass to it. It does so by specifying the arguments to be passed to the xz subprocess, which compresses the tar archive. In gnu tar you can specify which compressor with a flag. This compresses a list of files and directories into an.
#Tar compress multithread archive
The meaningful question becomes, what does the output from a parallel archive look like? It's no longer just a single file descriptor / stdout, but a file descriptor per thread.Īn example of this so far is the parallel dump mode of Postgresql pg_dump, wherein it dumps to a directory, with threads working over the set of tables to back up (work queue w/ multiple threads consuming the queue). A very fast compressor is lbzip2 with multithreading, but it cannot be run from tar itself.
If the source material is coming from a filesystem where different directories/files might be on different disks, or even a single fast disk that exceeds the input speed of the compression tool(s), then could indeed be beneficial to have multiple streams of input going into the compression layers. The OP asked about parallel archiving, not parallel compression.
0 kommentar(er)
