Local datasets

In addition to global datasets, the dataloader also looks for local datasets in the local_datasets_dir directory. This directory can either be set in the config or specified when initializing the loader. If you have generated an hdf5 file it is sufficient to place it in the local dataset directory; the file name will be used as the dataset name.

You can use local datasets just like global datasets:

from pbdl.loader import Dataloader

# assuming a dataset `my-dataset` has been created and stored in the local dataset directory

dataset = Dataloader(
    "my-dataset",
    time_steps=10,
    local_datasets_dir="./local_datasets/" # overwrite config
)