Query and fetching datasets

Query datasets

candidates = VCDataSets.query(category=nothing, source=nothing,
                              name=nothing, filename=nothing)

candidates = VCDataSets.query(filename="bunny.off")
candidates = VCDataSets.query(name="bunny")
candidates = VCDataSets.query(name=:bunny)

for d in VCDataSets.query(category="geometry")
    @show(d)
end

for d in VCDataSets.query(category="flow/netcdf")
    @show(d)
end

VCDataSets.query returns a potentially empty Vector of datasets, which math the search criteria.

Fetch a dataset

path = VCDataSets.withdir("./data") do
    path1 = VCDataSets.file(name=:bunny)

    candidates = VCDataSets.query(name="bunny")
    path2 = VCDataSets.file(candidates)

    @assert path1 == path2
    @assert isfile(path1)

    @assert VCDataSets.indirs("./data")

    path1
end

# open file in path
Note

VCDataSets.file provides the option force=true to enforce a download.

Fetch into a specific file

thisfile("mydir/bunny.off") == "mydir/bunny.off"
thisfile("mydir/data.off", "bunny.off")  == "mydir/data.off"
thisfile("mydir/data.off", query(...))  == "mydir/data.off"
thisfile("mydir/bunny.off"[, ...; force=true, copy=false)  == "mydir/bunny.off"

VCDataSet.thisfile is like VCDataSets.file but stores the dataset in path: dirs is only relevant for copying an already downloaded file from local storage (unless copy=false).

VCDataSets.dirs — Method
list = dirs()

Get list of directories where datasets are stored. Data that is not present in any of dirs() and needs to be fetched from a source is stored in dirs()[end].

Note

dirs() is initialized with abspath(joinpath(tempdir(), "VCDataSets")).

See also pushdir!, withdir, indirs

source
VCDataSets.pushdir! — Method
pushdir!(path[; absolute=false])

Push path onto dirs and make it the current path for stroring data that needs to be fetched.

Make sure path is an abosulte path ifabsolute=true, otherwise relative paths can be pushed!

Note

pushdir! always pushes and does not check if a path already exists in dirs.

See also withdir, dirs, indirs

source
VCDataSets.query — Method
list = query(category=nothing, source=nothing,
             name=nothing, filename=nothing)

Example

content = read(open(file(first(query(name=:bunny)))), String)

Query database for a dataset and return a potentially empty list of candidates.

See also file

source
VCDataSets.thisfile — Function
dataset = query(...)

path = thisfile(path; dataset=dataset[; force=false, copy=true)

path = thisfile(path[; force=false, copy=true,
                filename=basename(path), name=nothing,
                category=nothing, source=nothing])

Similar to file but store file in path. The dataset is specified either by dataset = query(...) or by name, category, source, and filename as for query.

The default filename is the basename(path) if path is not a directory. If a different filename is used to specify the dataset, the data is still saved to the file given as path!.

If path refers to a directory, the filename is determined from the dataset.

Note

The returned path is either identical to the input path, or the filename is joined if isdir(path). I.e., the returned path is can be a relative path (other than for file.

As for [file(@ref), force=true enforces a download from the external source.

If copy=true, the file is not downloaded by copied from local dirs if available.

Note

dirs is relevant only for copying a file, not for storing a downloaded file!

See also file, query, dirs

source
VCDataSets.withdir — Method
withdir(path) do
    @path == dirs()[end]
end

Apply pushdir! with path temporarily while evaluating the given function.

The method returns the value returned by f, i.e., f may just return the path.

See also pushdir!, dirs, indirs

source