How to compress data for archival - Revision history

Lleung: Added navbox

2023-09-21T20:28:07Z

Added navbox

← Older revision		Revision as of 20:28, 21 September 2023
Line 86:		Line 86:
	[[Category:Guides]]		[[Category:Guides]]
	[[Category:How-Tos]]		[[Category:How-Tos]]
			{{Navbox Guides}}

Lleung: Added categories

2023-09-20T21:53:25Z

Added categories

← Older revision		Revision as of 21:53, 20 September 2023
Line 83:		Line 83:
	-rw-r--r-- 1 drozmano drozmano 22M Jul 4 09:17 archive.tar.xz		-rw-r--r-- 1 drozmano drozmano 22M Jul 4 09:17 archive.tar.xz
	</pre>		</pre>

			[[Category:Guides]]
			[[Category:How-Tos]]

Dmitri: /* One directory */

2022-07-04T22:27:08Z

One directory

← Older revision		Revision as of 22:27, 4 July 2022
Line 24:		Line 24:
	....		....
	$ ls -lh archive.tar		$ ls -lh archive.tar
	-rw-r--r-- 1 drozmano drozmano 150M Jul 4 08:42 ~~gromacs~~.tar		-rw-r--r-- 1 drozmano drozmano 150M Jul 4 08:42 archive.tar
	</pre>		</pre>
	Individual files use some integral number of storage blocks,		Individual files use some integral number of storage blocks,

Dmitri: /* One directory */

2022-07-04T15:32:47Z

One directory

← Older revision		Revision as of 15:32, 4 July 2022
Line 21:		Line 21:
	Archival without compression takes 8.5 sec and produces a file of 150MB in size.		Archival without compression takes 8.5 sec and produces a file of 150MB in size.
	<pre>		<pre>
	~~# 8.5 sec~~
	$ tar cvf archive.tar my_data		$ tar cvf archive.tar my_data
	....		....
Line 36:		Line 35:
	The archive size is 33MB, that is '''20% of the original size'''.		The archive size is 33MB, that is '''20% of the original size'''.
	<pre>		<pre>
	~~# 5.9 sec~~
	$ tar czvf archive.tar.gz my_data		$ tar czvf archive.tar.gz my_data
	....		....
Line 46:		Line 44:
	The archive size is 26MB, that is '''16% of the original size'''.		The archive size is 26MB, that is '''16% of the original size'''.
	<pre>		<pre>
	~~# 5.9 sec~~
	$ tar cjvf archive.tar.bz2 my_data		$ tar cjvf archive.tar.bz2 my_data
	....		....
Line 56:		Line 53:
	The archive size is the smallest, 22MB, which is '''13% of the original size'''.		The archive size is the smallest, 22MB, which is '''13% of the original size'''.
	<pre>		<pre>
	~~# 5.9 sec~~
	$ tar cJvf archive.tar.xz my_data		$ tar cJvf archive.tar.xz my_data
	....		....

Dmitri: /* One directory */

2022-07-04T15:32:00Z

One directory

← Older revision		Revision as of 15:32, 4 July 2022
Line 13:		Line 13:

	== One directory ==		== One directory ==
	For this example we use a directory <code>my_data</code> that contains '''6172 files ~~containing 165 MB~~''' ~~of data~~.		For this example we use a directory <code>my_data</code> that contains '''165 MB of data in 6172 files'''.

	To create an archive, the '''cvf''' options are used,		To create an archive, the '''cvf''' options are used,

Dmitri: /* One directory */

2022-07-04T15:31:16Z

One directory

← Older revision		Revision as of 15:31, 4 July 2022
Line 13:		Line 13:

	== One directory ==		== One directory ==
	For this example we use a directory that contains '''6172 files containing 165 MB''' of data.		For this example we use a directory <code>my_data</code> that contains '''6172 files containing 165 MB''' of data.

	To create an archive, the '''cvf''' options are used,		To create an archive, the '''cvf''' options are used,

Dmitri: /* One directory */

2022-07-04T15:29:52Z

One directory

← Older revision		Revision as of 15:29, 4 July 2022
Line 33:		Line 33:

	Archival with compression using the <code>gzip</code> is requested by using the '''z''' option,		Archival with compression using the <code>gzip</code> is requested by using the '''z''' option,
	and it is actually '''faster''' (5.9 sec) due to less data that ~~needs~~ to be written to the compressed archive.		and it is actually '''faster''' (5.9 sec) due to less data that need to be written to the compressed archive.
	The archive size is 33MB, that is '''20% of the original size'''.		The archive size is 33MB, that is '''20% of the original size'''.
	<pre>		<pre>

Dmitri: /* Several files */

2022-07-04T15:22:40Z

Several files

← Older revision		Revision as of 15:22, 4 July 2022
Line 87:		Line 87:
	-rw-r--r-- 1 drozmano drozmano 22M Jul 4 09:17 archive.tar.xz		-rw-r--r-- 1 drozmano drozmano 22M Jul 4 09:17 archive.tar.xz
	</pre>		</pre>

	~~== Several files ==~~

Dmitri: /* One file */

2022-07-04T15:21:01Z

One file

← Older revision		Revision as of 15:21, 4 July 2022
Line 83:		Line 83:
	$ ls -lh		$ ls -lh
	-rw-r--r-- 1 drozmano drozmano 150M Jul 4 08:57 archive.tar		-rw-r--r-- 1 drozmano drozmano 150M Jul 4 08:57 archive.tar
	-rw-r--r-- 1 drozmano drozmano 26M Jul 4 09:16 archive.tar.~~bzip2~~		-rw-r--r-- 1 drozmano drozmano 26M Jul 4 09:16 archive.tar.bz2
	-rw-r--r-- 1 drozmano drozmano 33M Jul 4 09:15 archive.tar.gz		-rw-r--r-- 1 drozmano drozmano 33M Jul 4 09:15 archive.tar.gz
	-rw-r--r-- 1 drozmano drozmano 22M Jul 4 09:17 archive.tar.xz		-rw-r--r-- 1 drozmano drozmano 22M Jul 4 09:17 archive.tar.xz

Dmitri: /* Using tar command */

2022-07-04T15:20:04Z

Using tar command

← Older revision		Revision as of 15:20, 4 July 2022
Line 11:		Line 11:

	<code>bzip2</code> compresses better, but significantly slower. In many cases, <code>gzip</code> is good enough.		<code>bzip2</code> compresses better, but significantly slower. In many cases, <code>gzip</code> is good enough.

	~~== One file ==~~

	== One directory ==		== One directory ==
	For this example we use a directory that contains '''6172 files containing 165 MB''' of data.		For this example we use a directory that contains '''6172 files containing 165 MB''' of data.

	To create an archive, the '''-cvf''' options are used,		To create an archive, the '''cvf''' options are used,
	which means '''Create Verbose a File'''.		which means '''Create Verbose a File'''.
	These options instruct <code>tar</code> to create a new file for the archive and		These options instruct <code>tar</code> to create a new file for the archive and
Line 24:		Line 22:
	<pre>		<pre>
	# 8.5 sec		# 8.5 sec
	$ tar -cvf archive.tar my_data		$ tar cvf archive.tar my_data
	....		....
	$ ls -lh archive.tar		$ ls -lh archive.tar
Line 34:		Line 32:


	Archival with compression using the <code>gzip</code> is requested by using the '''-z''' option,		Archival with compression using the <code>gzip</code> is requested by using the '''z''' option,
	and it is actually '''faster''' (5.9 sec) due to less data that needs to be written to the compressed archive.		and it is actually '''faster''' (5.9 sec) due to less data that needs to be written to the compressed archive.
	The archive size is 33MB, that is '''20% of the original size'''.		The archive size is 33MB, that is '''20% of the original size'''.
	<pre>		<pre>
	# 5.9 sec		# 5.9 sec
	$ tar -czvf archive.tar.gz my_data		$ tar czvf archive.tar.gz my_data
	....		....
	$ ls -lh archive.tar.gz		$ ls -lh archive.tar.gz
Line 45:		Line 43:
	</pre>		</pre>

	Archival with the <code>bzip2</code> compression is requested by the '''-j''' option, and takes about '''twice as long''', 16.8 sec.		Archival with the <code>bzip2</code> compression is requested by the '''j''' option, and takes about '''twice as long''', 16.8 sec.
	The archive size is 26MB, that is '''16% of the original size'''.		The archive size is 26MB, that is '''16% of the original size'''.
	<pre>		<pre>
	# 5.9 sec		# 5.9 sec
	$ tar -cjvf archive.tar.bz2 my_data		$ tar cjvf archive.tar.bz2 my_data
	....		....
	$ ls -lh archive.tar.bz2		$ ls -lh archive.tar.bz2
Line 55:		Line 53:
	</pre>		</pre>

	Archival with the newer <code>XZ</code> compression is requested by the '''-J''' option, and takes the longest, '''4-times as long''', 30.7 sec:		Archival with the newer <code>XZ</code> compression is requested by the '''J''' option, and takes the longest, '''4-times as long''', 30.7 sec:
	The archive size is the smallest, 22MB, which is '''13% of the original size'''.		The archive size is the smallest, 22MB, which is '''13% of the original size'''.
	<pre>		<pre>
	# 5.9 sec		# 5.9 sec
	$ tar -cJvf archive.tar.xz my_data		$ tar cJvf archive.tar.xz my_data
	....		....
	$ ls -lh archive.tar.xz		$ ls -lh archive.tar.xz
	-rw-r--r-- 1 drozmano drozmano 22M Jul 4 09:02 archive.tar.xz		-rw-r--r-- 1 drozmano drozmano 22M Jul 4 09:02 archive.tar.xz
			</pre>

			== One file ==
			If one wants to compress just one file.
			For example, there is an uncompressed archive, <code>archive.tar</code> and we want to compress it.
			Here are the possible commands:

			<pre>
			$ls -lh
			-rw-r--r-- 1 drozmano drozmano 150M Jul 4 08:57 archive.tar

			$ tar czvf archive.tar.gz archive.tar
			....

			$ tar cjvf archive.tar.bz2 archive.tar
			....

			$ tar cJvf archive.tar.xz archive.tar
			.....

			$ ls -lh
			-rw-r--r-- 1 drozmano drozmano 150M Jul 4 08:57 archive.tar
			-rw-r--r-- 1 drozmano drozmano 26M Jul 4 09:16 archive.tar.bzip2
			-rw-r--r-- 1 drozmano drozmano 33M Jul 4 09:15 archive.tar.gz
			-rw-r--r-- 1 drozmano drozmano 22M Jul 4 09:17 archive.tar.xz
	</pre>		</pre>

	== Several files ==		== Several files ==