IOUtilities.jl

Note

An earlier version used the GZip package. The current version switch to TranscodingStreams and has some codecs as dependencies.

Read and write compressed files as with, e.g., Base.open.

Note

This package does not export any methods! Use, e.g., IOUtilities.open(...).

Installation

julia> using Pkg
julia> Pkg.Registry.add("https://visual2.cs.ovgu.de/roessl/VCJuliaRegistry.git")
julia> Pkg.add("IOUtilities")
julia> using IOUtilities

Compressed files

Note

Compression algorithms are selected only by the file name extension and not contents of streams.

@assert IOUtilities.isgzipped("/path/to/file.txt.gz")
@assert IOUtilities.iscompressed("/path/to/file.txt.gz")
@assert IOUtiscompressed.!iscompressed("/path/to/file.txt")

isgzipped and iscompressed determine if a file is compressed (and the compression algorithm is supported).

@assert IOUtilities.iscompressionext(".gz")
@assert IOUtilities.splitcompressionext("/path/to/file.txt.gz") == ("/path/to/file.txt", ".gz")
@assert IOUtilities.splitcompressionext("/path/to/file.txt") == ("/path/to/file.txt", "")

iscompressionext and splitcompressionext handle file extensions

@assert IOUtilities.splitrealext("/path/to/file.txt.gz") == ("/path/to/file", ".txt", ".gz")
@assert IOUtilities.splitrealext("/path/to/file.txt") == ("/path/to/file", ".txt", "")
@assert IOUtilities.splitrealext("/path/to/file.gz") == ("/path/to/file", "", ".gz")
@assert IOUtilities.splitrealext("/path/to/file") == ("/path/to/file", "", "")

splitrealext splits into base path, "real extension" and "compression extension".

Working with compressed files

IOUtilities.open("/path/to/file.zst", "w+") do f
    print(f, "Hello!")
end

IOUtilities.open("/path/to/file.zst") do f
    @assert readline(f) == "Hello!"
end

open files for reading and writing and apply compression or decompression based on the file name extension.

Conveniently reading and writing a file

IOUtilities.write("/path/to/file.txt", "No compression!") # see file extension

IOUtilities.write("/path/to/file.txt.xz", "Hello!")
@assert IOUtilities.read("/path/to/file.txt.xz", String) == "Hello!"
@assert IOUtilities.readlines("/path/to/file.txt.xz", String) |> join == "Hello!"

read, readlines and write open a file given a path and read/write contents. (You don't need to open the file explicitly.)

Compress and uncompress files

IOUtilities.compress("/path/to/file.txt", ".bz2") # writes /path/to/file.txt.bz2
IOUtilities.uncompress("/path/to/file.txt.bz2")   # writes /path/to/file.txt


IOUtilities.gzip("/path/to/file.txt") # writes /path/to/file.txt.gz
IOUtilities.gunzip("/path/to/file.txt.gz")   # writes /path/to/file.txt

compress and uncompress read a file and write compressed or uncompressed contents to a file with file name extension added or removed, respectively.

gzip and gunzip are provided for convenience.

IOUtilities.transcode("/path/to/file.txt.gz", "/path/to/file.txt.lz4")

Use transcode to "change" the compression algorithm.

Note

Existing input files remain unchanged. Existing output files are overwritten.

Note

All methods return the path of the written file. There is no action if input and output file paths are identical.

zip-archives

files = unzip("archive.zip"[, what=:all])      # all files
files = unzip("archive.zip", what=:first)      # first file
files = unzip(==("this.file"), "archive.zip")  # files matchin predicate

The returned files::Vector{String} stores the file names of extracted files as they appear in the zip-archive.

Internals

makeistream and [makeostream] construct input and output streams <:IO that decode and encode, respectively. These streams are passed to _open which ensures that streams will be closed.

Methods

IOUtilities.compress — Function

newfile = compress(filepath, ext)

Compress filepath using an algorithm determined by the file extension ext (e.g., ".gz") and return name of compressed file with ext appended.

If filepath is already a compressed file, return filepath without action. The compression method may differ from ext. Use transcode changing compression schemes.

Errors if ext is unknown (by iscompressionext).

source

IOUtilities.gunzip — Method

newfile = ungzip(filepath)

Convenient for uncompress(filepath).

precondition

filepath must have extension ".gz".