What Is an .XZ File? (LZMA2 Compression Built to Replace the Old .LZMA Format)

What the .XZ Format Actually Is and Why It Replaced .LZMA

The .xz file format was released in 2009 by the Tukaani Project, designed by Lasse Collin, as the successor to the older, more limited .lzma format. Both formats compress data using variants of Igor Pavlov's LZMA algorithm, but .xz wraps that compression in a genuinely improved container: it adds magic bytes at the start of the file so tools can reliably identify it, supports several different integrity-check options, and allows multiple independent compressed streams to be concatenated together in one file, none of which the older .lzma format handled as robustly.

Structurally, an .xz file holds one or more compressed streams, each built from one or more independently compressed blocks. Like a .gz or .bz2 file, .xz only compresses a single data stream on its own — bundling multiple files together means running tar first to combine them into one archive, then xz second to compress the result, producing the familiar .tar.xz extension used throughout the Linux world for source releases, packages, and kernel distributions.

That block structure isn't just a formality — it's what makes multithreaded compression possible at all. The xz command's -T option can split input data into independent blocks and compress several of them simultaneously across multiple CPU cores, trading a small amount of compression ratio, since each block loses some of the cross-block matching a single continuous stream could exploit, for meaningfully faster compression on multi-core hardware. This option didn't exist in the older, single-stream .lzma format, which had no equivalent block boundary to split work across in the first place.

The Dictionary-Size Setting That Trades Memory for Compression

LZMA2, the compression method .xz uses internally, works by finding repeated sequences across a sliding dictionary window and encoding matches using range coding, a more precise form of entropy coding than the Huffman coding gzip or bzip2 rely on. The size of that dictionary window is directly configurable, and it's the single setting with the biggest effect on both compression ratio and memory usage: a larger dictionary lets the encoder find matches across a wider span of the file, generally producing a smaller result, but it also means the decompressor needs at least that same amount of memory just to hold the dictionary while decoding.

At the highest standard preset, xz -9, the dictionary size is 64 MiB, and compressing a file at that setting takes roughly 674 MiB of memory, while decompressing it afterward needs about 65 MiB — a real, documented gap between what compressing and decompressing the same file actually cost. That 65 MiB decompression requirement is precisely why extracting a maximally-compressed .xz archive can be slow or fail outright on genuinely memory-constrained devices, such as older routers or minimal embedded Linux systems, even though the file itself might be quite small.

Xz's encoder can create dictionaries as large as 1.5 GiB, and the decoder is specified to support dictionaries up to just under 4 GiB, though userspace tools in practice typically use dictionary sizes in the range of a few megabytes to 64 MiB. This dictionary-size trade-off is a genuinely different mechanism from anything in gzip's DEFLATE, which uses a fixed, much smaller 32 KiB window regardless of settings, and it's the direct reason xz can compress noticeably smaller than gzip on the same data at the cost of needing considerably more memory and time to do it.

Integrity Checks the Old .LZMA Format Never Offered

• None: integrity checking can be disabled entirely, trading corruption detection for a small reduction in file size and processing overhead.
• CRC32: a lighter, faster checksum offering weaker corruption detection than the alternatives, useful when speed matters more than certainty.
• CRC64 (the default): xz's standard choice, offering meaningfully stronger corruption detection than CRC32 at a modest additional cost.
• SHA-256: the strongest option, a cryptographic hash offering the most reliable corruption detection of the available choices, at the highest computational cost.
• Stream concatenation support: multiple independently compressed .xz streams can be joined into one file, something the .lzma format's simpler single-stream design didn't support the same way.

Why Linux Distributions Adopted XZ So Broadly

Xz Utils' own documentation and independent compression comparisons consistently report that xz produces meaningfully smaller output than gzip on typical files — figures around 30% smaller are commonly cited — which is precisely why the Linux kernel, most major package managers, and countless open-source project release archives moved to .tar.xz as their default distribution format over the past decade and a half. Smaller downloads matter directly for mirror bandwidth costs and for users on slower connections, and the compression gain over gzip is large enough to be worth the extra compression time on the publishing side, since compression only needs to happen once per release while decompression happens on every download.

That asymmetry is deliberate and well-documented: xz compression is genuinely slow at higher settings compared with gzip, sometimes dramatically so, but decompression remains reasonably fast even at maximum compression settings, which is exactly the trade-off a software distribution scenario wants — pay the compression cost once when building a release, and let every downstream user benefit from the smaller file and still-fast extraction.

Package managers on several major Linux distributions, along with source tarballs for large projects like the Linux kernel itself, standardized on .tar.xz specifically because that one-time compression cost during release building is negligible against the cumulative bandwidth savings across millions of downloads, a calculation that tips clearly in xz's favor for anything distributed at real scale.

Real Problems That Show Up Around Large Dictionary Sizes

A documented, recurring complaint on embedded-Linux and low-memory-device forums involves an .xz-compressed file that decompresses fine on a desktop system but fails outright or runs extremely slowly on a router, single-board computer, or other RAM-constrained device — the root cause traced consistently back to the dictionary size used during compression exceeding what the target device can allocate for decompression, not to any actual file corruption. The documented fix is recompressing with a smaller dictionary size setting, matched to what the target hardware can actually handle, rather than assuming the default or maximum preset is universally safe.

A second recurring issue involves confusion over the different integrity-check levels: users who explicitly disabled the checksum for a small size or speed gain sometimes later have no way to confirm whether a downloaded or transferred .xz file arrived intact, since there's no checksum stored in the file to check against at all in that configuration. Xz's own tooling reports which integrity check level a file used, and choosing at least CRC32 rather than disabling checks entirely is the documented recommendation for anything where verifying integrity later might matter.

A third pattern shows up specifically around the Linux kernel's own use of xz for compressing kernel images: the kernel's in-tree xz decoder, used for tasks like decompressing a compressed kernel image at boot, is deliberately built to work within tighter memory constraints than desktop xz tooling assumes by default, and kernel documentation explicitly notes that reasonable dictionary sizes for that kind of embedded, in-kernel use might be as small as 64 KiB to 1 MiB, far below the multi-megabyte or 64 MiB dictionaries common in general-purpose desktop compression.

LZMA2-Based XZ Set Beside the Legacy LZMA Format It Replaced

Common Questions About XZ Compression and Its Dictionary Setting

Why does my .xz file take so long to extract on some devices?
It's usually the dictionary size used during compression. At the -9 preset, decompression needs about 65 MiB of memory; on a low-memory embedded device, that can cause slow decompression or an outright failure.

Is .xz just a newer name for .lzma?
Not exactly. Both use variants of the LZMA algorithm, but .xz is a genuinely improved container format released in 2009 with magic bytes, integrity checks, and stream concatenation that the older .lzma format lacked.

Why do Linux distributions prefer .tar.xz over .tar.gz?
Mainly file size. Xz typically produces files around 30% smaller than gzip on the same data, which matters for download bandwidth, even though xz compression itself takes noticeably longer to run.

What does disabling the integrity check actually do?
It removes the built-in checksum that would otherwise let a decompressor detect data corruption, trading that safety check for a very small reduction in file size and processing time.

Can I make an .xz file use less memory to decompress?
Yes, by compressing it with a smaller dictionary size setting rather than the default or maximum preset, which reduces how much memory a decompressor needs later.

Can xz compression use multiple CPU cores?
Yes, using the -T option, which splits data into independent blocks compressed in parallel. This trades a small amount of compression ratio for meaningfully faster compression on multi-core hardware.