LinearReader

# LinearReader !(https://raw.githubusercontent.com/Bugfunbug/LinearReader/refs/heads/main/.github/assets/badges/minecraft.svg) !(https://raw.githubusercontent.com/Bugfunbug/LinearReader/refs/heads/main/.github/assets/badges/mod-loaders.svg) !(https://raw.githubusercontent.com/Bugfunbug/LinearReader/refs/heads/main/.github/assets/badges/server-side.svg)

!(https://raw.githubusercontent.com/Bugfunbug/LinearReader/refs/heads/main/.github/assets/badges/compression.svg) [!(https://raw.githubusercontent.com/Bugfunbug/LinearReader/refs/heads/main/.github/assets/badges/wiki.svg)](https://github.com/Bugfunbug/LinearReader/wiki)

LinearReader is a server-side Minecraft mod that reduces your world size without negatively impacting gameplay. This is done mainly through using Xymb's (https://github.com/xymb-endcrystalme/LinearRegionFileFormatTools) in place of Minecraft's Anvil file format, and through utilizing Zstd for compression instead of the Zlib compression algorithm used by vanilla Minecraft.

## How is World Size Reduced? Xymb's (https://github.com/xymb-endcrystalme/LinearRegionFileFormatTools) is already a large step up from vanilla Minecraft's region file format, and greatly reduces world sizes.

The other necessary improvement is the compression algorithm and level of compression. Higher levels of compression require more resources and more time, which is not ideal for an active server, and lower levels obviously do not save as much storage. So the challenge was, how can LinearReader provide the speed of low-level compression and the storage savings of high-level compression?

LinearReader solves this by splitting the workload into two separate stages. Live writes are compressed quickly with low-level Zstd compression levels, leaving the CPU with plenty of resources to run Minecraft. Then later, when the server is idle, LinearReader will safely recompress the region files with much deeper compression.

## Compression Benchmarks To show LinearReader's storage gains and why high level compression is not used during live writes, I ran benchmarks for the Anvil file format with Zlib using Java’s default compression level (which is what Vanilla Minecraft uses), Zstd level 4 (which is what LinearReader usually uses for live writes), and Zstd level 22 (which LinearReader uses for recompression). For a deeper dive on the benchmarks, see the `benchmarks/` directory.

| Dataset | Format | Size | Compression Ratio | Compression Speed | Decompression Speed | |----------------------|------------------------|---------|-------------------|-------------------|---------------------| | Overworld | Anvil/Zlib 6 | 8. 38 GB | 5. 44x | 75. 7 MB/s | 1271. 3 MB/s | | Overworld | LinearReader (Zstd 4) | 6. 21 GB | 7. 39x | 585. 5 MB/s | 1412. 8 MB/s | | Overworld | LinearReader (Zstd 22) | 4. 28 GB | 10. 71x | 4. 4 MB/s | 1589. 2 MB/s | | End (mostly void) | Anvil/Zlib 6 | 2. 16 GB | 3. 66x | 188. 4 MB/s | 1576. 4 MB/s | | End (mostly void) | LinearReader (Zstd 4) | 244 MB | 33. 20x | 1811. 1 MB/s | 4062. 7 MB/s | | End (mostly void) | LinearReader (Zstd 22) | 141 MB | 57. 50x | 10. 1 MB/s | 6166. 9 MB/s | | Tectonic + Terralith | Anvil/Zlib 6 | 2. 56 GB | 5. 97x | 84. 1 MB/s | 1338. 1 MB/s | | Tectonic + Terralith | LinearReader (Zstd 4) | 1. 80 GB | 8. 49x | 735. 8 MB/s | 1742. 8 MB/s | | Tectonic + Terralith | LinearReader (Zstd 22) | 1. 24 GB | 12. 37x | 4. 4 MB/s | 1787. 3 MB/s |

As is shown in the above table, LinearReader + Zstd achieves better compression while still maintaining fast compression and decompression speeds during normal gameplay, (Zstd 22 is used for the recompressor, which is not active during normal gameplay).

## Other Features LinearReader doesn't just replace Minecraft's file format and compression algorithm, it also includes a bunch of other features, such as: A backup system: LinearReader can automatically create backups of region files, which can be used if need be. A cache system: A configurable number of recently used region files remain cached in memory, reducing disk reads and decompression. A chunk pruning system: LinearReader can search for empty, worthless chunks that can safely be deleted. Strict rules are in place to ensure that only truly worthless chunks are pruned. An adaptive runtime policy: LinearReader can tweak certain settings to keep the game running smooth.

## Made with Servers in Mind Being a server-side mod, LinearReader tries to be as helpful and informative for server admins as possible. There are commands, such as `benc