super_gradients.training.datasets.dataset_interfaces package
Submodules
super_gradients.training.datasets.dataset_interfaces.dataset_interface module
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface(dataset_params={}, train_loader=None, val_loader=None, test_loader=None, classes=None)[source]
Bases:
object
DatasetInterface - This class manages all of the “communiation” the Model has with the Data Sets
- build_data_loaders(batch_size_factor=1, num_workers=8, train_batch_size=None, val_batch_size=None, test_batch_size=None, distributed_sampler: bool = False)[source]
define train, val (and optionally test) loaders. The method deals separately with distributed training and standard (non distributed, or parallel training). In the case of distributed training we need to rely on distributed samplers. :param batch_size_factor: int - factor to multiply the batch size (usually for multi gpu) :param num_workers: int - number of workers (parallel processes) for dataloaders :param train_batch_size: int - batch size for train loader, if None will be taken from dataset_params :param val_batch_size: int - batch size for val loader, if None will be taken from dataset_params :param distributed_sampler: boolean flag for distributed training mode :return: train_loader, val_loader, classes: list of classes
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.ExternalDatasetInterface(train_loader, val_loader, num_classes, dataset_params={})[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- get_data_loaders(batch_size_factor: int = 1, num_workers: int = 8, train_batch_size: Optional[int] = None, val_batch_size: Optional[int] = None, distributed_sampler: bool = False)[source]
Get self.train_loader, self.test_loader, self.classes.
If the data loaders haven’t been initialized yet, build them first.
- Parameters
kwargs – kwargs are passed to build_data_loaders.
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.LibraryDatasetInterface(name='cifar10', dataset_params={}, to_cutout=False)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.Cifar10DatasetInterface(dataset_params={})[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.LibraryDatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.Cifar100DatasetInterface(dataset_params={})[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.LibraryDatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.TestDatasetInterface(trainset, dataset_params={}, classes=None)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- get_data_loaders(batch_size_factor=1, num_workers=8, train_batch_size=None, val_batch_size=None, distributed_sampler=False)[source]
Get self.train_loader, self.test_loader, self.classes.
If the data loaders haven’t been initialized yet, build them first.
- Parameters
kwargs – kwargs are passed to build_data_loaders.
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.ClassificationTestDatasetInterface(dataset_params={}, image_size=32, batch_size=5, classes=None)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.TestDatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.SegmentationTestDatasetInterface(dataset_params={}, image_size=512, batch_size=4)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.TestDatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.DetectionTestDatasetInterface(dataset_params={}, image_size=320, batch_size=4)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.TestDatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.TestYoloDetectionDatasetInterface(dataset_params={}, input_dims=(3, 32, 32), batch_size=5)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
note: the output size is (batch_size, 6) in the test while in real training the size of axis 0 can vary (the number of bounding boxes)
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.ImageNetDatasetInterface(dataset_params={}, data_dir='/data/Imagenet')[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.TinyImageNetDatasetInterface(dataset_params={}, data_dir='/data/TinyImagenet')[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.ClassificationDatasetInterface(normalization_mean=(0, 0, 0), normalization_std=(1, 1, 1), resolution=64, dataset_params={})[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.PascalVOC2012DetectionDataSetInterface(dataset_params=None, cache_labels=False, cache_images=False)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.PascalVOC2012SegmentationDataSetInterface(dataset_params=None, cache_labels=False, cache_images=False)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.PascalAUG2012SegmentationDataSetInterface(dataset_params=None, cache_labels=False, cache_images=False)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.CoCoDataSetInterfaceBase(dataset_params=None)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.CoCoDetectionDatasetInterface(dataset_params=None, cache_labels=False, cache_images=False, train_list_file='train2017.txt', val_list_file='val2017.txt')[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.CoCoDataSetInterfaceBase
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.CoCoSegmentationDatasetInterface(dataset_params=None, cache_labels: bool = False, cache_images: bool = False, dataset_classes_inclusion_tuples_list: Optional[list] = None)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.CoCoDataSetInterfaceBase
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.CoCo2014DetectionDatasetInterface(dataset_params=None, cache_labels=False, cache_images=False, train_list_file='train2014.txt', val_list_file='val2014.txt')[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.CoCoDetectionDatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.dataset_interface.CityscapesDatasetInterface(dataset_params=None, cache_labels: bool = False, cache_images: bool = False)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
Module contents
- class super_gradients.training.datasets.dataset_interfaces.DatasetInterface(dataset_params={}, train_loader=None, val_loader=None, test_loader=None, classes=None)[source]
Bases:
object
DatasetInterface - This class manages all of the “communiation” the Model has with the Data Sets
- build_data_loaders(batch_size_factor=1, num_workers=8, train_batch_size=None, val_batch_size=None, test_batch_size=None, distributed_sampler: bool = False)[source]
define train, val (and optionally test) loaders. The method deals separately with distributed training and standard (non distributed, or parallel training). In the case of distributed training we need to rely on distributed samplers. :param batch_size_factor: int - factor to multiply the batch size (usually for multi gpu) :param num_workers: int - number of workers (parallel processes) for dataloaders :param train_batch_size: int - batch size for train loader, if None will be taken from dataset_params :param val_batch_size: int - batch size for val loader, if None will be taken from dataset_params :param distributed_sampler: boolean flag for distributed training mode :return: train_loader, val_loader, classes: list of classes
- class super_gradients.training.datasets.dataset_interfaces.TestDatasetInterface(trainset, dataset_params={}, classes=None)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- get_data_loaders(batch_size_factor=1, num_workers=8, train_batch_size=None, val_batch_size=None, distributed_sampler=False)[source]
Get self.train_loader, self.test_loader, self.classes.
If the data loaders haven’t been initialized yet, build them first.
- Parameters
kwargs – kwargs are passed to build_data_loaders.
- class super_gradients.training.datasets.dataset_interfaces.LibraryDatasetInterface(name='cifar10', dataset_params={}, to_cutout=False)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.ClassificationDatasetInterface(normalization_mean=(0, 0, 0), normalization_std=(1, 1, 1), resolution=64, dataset_params={})[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.Cifar10DatasetInterface(dataset_params={})[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.LibraryDatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.Cifar100DatasetInterface(dataset_params={})[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.LibraryDatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.ImageNetDatasetInterface(dataset_params={}, data_dir='/data/Imagenet')[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.TinyImageNetDatasetInterface(dataset_params={}, data_dir='/data/TinyImagenet')[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.CoCoDetectionDatasetInterface(dataset_params=None, cache_labels=False, cache_images=False, train_list_file='train2017.txt', val_list_file='val2017.txt')[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.CoCoDataSetInterfaceBase
- class super_gradients.training.datasets.dataset_interfaces.CoCo2014DetectionDatasetInterface(dataset_params=None, cache_labels=False, cache_images=False, train_list_file='train2014.txt', val_list_file='val2014.txt')[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.CoCoDetectionDatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.CoCoSegmentationDatasetInterface(dataset_params=None, cache_labels: bool = False, cache_images: bool = False, dataset_classes_inclusion_tuples_list: Optional[list] = None)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.CoCoDataSetInterfaceBase
- class super_gradients.training.datasets.dataset_interfaces.PascalAUG2012SegmentationDataSetInterface(dataset_params=None, cache_labels=False, cache_images=False)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.PascalVOC2012SegmentationDataSetInterface(dataset_params=None, cache_labels=False, cache_images=False)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.TestYoloDetectionDatasetInterface(dataset_params={}, input_dims=(3, 32, 32), batch_size=5)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.DatasetInterface
note: the output size is (batch_size, 6) in the test while in real training the size of axis 0 can vary (the number of bounding boxes)
- class super_gradients.training.datasets.dataset_interfaces.SegmentationTestDatasetInterface(dataset_params={}, image_size=512, batch_size=4)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.TestDatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.DetectionTestDatasetInterface(dataset_params={}, image_size=320, batch_size=4)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.TestDatasetInterface
- class super_gradients.training.datasets.dataset_interfaces.ClassificationTestDatasetInterface(dataset_params={}, image_size=32, batch_size=5, classes=None)[source]
Bases:
super_gradients.training.datasets.dataset_interfaces.dataset_interface.TestDatasetInterface