gradient.api_sdk package¶
Subpackages¶
- gradient.api_sdk.clients package
- Submodules
- gradient.api_sdk.clients.base_client module
- gradient.api_sdk.clients.clusters module
- gradient.api_sdk.clients.dataset_client module
- gradient.api_sdk.clients.dataset_tag_client module
- gradient.api_sdk.clients.dataset_version_client module
- gradient.api_sdk.clients.deployment_client module
- gradient.api_sdk.clients.experiment_client module
- gradient.api_sdk.clients.http_client module
- gradient.api_sdk.clients.hyperparameter_client module
- gradient.api_sdk.clients.job_client module
- gradient.api_sdk.clients.machine_types_client module
- gradient.api_sdk.clients.machines_client module
- gradient.api_sdk.clients.model_client module
- gradient.api_sdk.clients.notebook_client module
- gradient.api_sdk.clients.project_client module
- gradient.api_sdk.clients.sdk_client module
- gradient.api_sdk.clients.secret_client module
- gradient.api_sdk.clients.storage_provider_client module
- gradient.api_sdk.clients.tensorboards_client module
- gradient.api_sdk.clients.workflow_client module
- Module contents
- gradient.api_sdk.models package
- Submodules
- gradient.api_sdk.models.artifact module
- gradient.api_sdk.models.cluster module
- gradient.api_sdk.models.dataset module
- gradient.api_sdk.models.dataset_tag module
- gradient.api_sdk.models.dataset_version module
- gradient.api_sdk.models.deployment module
- gradient.api_sdk.models.experiment module
- gradient.api_sdk.models.experiment_dataset module
- gradient.api_sdk.models.hyperparameter module
- gradient.api_sdk.models.job module
- gradient.api_sdk.models.log module
- gradient.api_sdk.models.machine module
- gradient.api_sdk.models.model module
- gradient.api_sdk.models.notebook module
- gradient.api_sdk.models.pagination module
- gradient.api_sdk.models.project module
- gradient.api_sdk.models.secret module
- gradient.api_sdk.models.storage_provider module
- gradient.api_sdk.models.tag module
- gradient.api_sdk.models.tensorboard module
- gradient.api_sdk.models.vm_type module
- gradient.api_sdk.models.workflows module
- Module contents
- gradient.api_sdk.repositories package
- Submodules
- gradient.api_sdk.repositories.clusters module
- gradient.api_sdk.repositories.common module
- gradient.api_sdk.repositories.dataset_tags module
- gradient.api_sdk.repositories.dataset_versions module
- gradient.api_sdk.repositories.datasets module
- gradient.api_sdk.repositories.deployments module
- gradient.api_sdk.repositories.experiments module
- gradient.api_sdk.repositories.hyperparameter module
- gradient.api_sdk.repositories.jobs module
- gradient.api_sdk.repositories.machine_types module
- gradient.api_sdk.repositories.machines module
- gradient.api_sdk.repositories.models module
- gradient.api_sdk.repositories.notebooks module
- gradient.api_sdk.repositories.projects module
- gradient.api_sdk.repositories.secrets module
- gradient.api_sdk.repositories.storage_providers module
- gradient.api_sdk.repositories.tags module
- gradient.api_sdk.repositories.tensorboards module
- gradient.api_sdk.repositories.workflows module
- Module contents
- gradient.api_sdk.serializers package
- Submodules
- gradient.api_sdk.serializers.artifact module
- gradient.api_sdk.serializers.base module
- gradient.api_sdk.serializers.cluster module
- gradient.api_sdk.serializers.dataset module
- gradient.api_sdk.serializers.dataset_tag module
- gradient.api_sdk.serializers.dataset_version module
- gradient.api_sdk.serializers.deployment module
- gradient.api_sdk.serializers.experiment module
- gradient.api_sdk.serializers.experiment_dataset module
- gradient.api_sdk.serializers.hyperparameter module
- gradient.api_sdk.serializers.job module
- gradient.api_sdk.serializers.log module
- gradient.api_sdk.serializers.machine module
- gradient.api_sdk.serializers.model module
- gradient.api_sdk.serializers.notebook module
- gradient.api_sdk.serializers.project module
- gradient.api_sdk.serializers.secret module
- gradient.api_sdk.serializers.storage_provider module
- gradient.api_sdk.serializers.tag module
- gradient.api_sdk.serializers.tensorboard module
- gradient.api_sdk.serializers.utils module
- gradient.api_sdk.serializers.vm_type module
- gradient.api_sdk.serializers.workflows module
- Module contents
Submodules¶
gradient.api_sdk.archivers module¶
-
class
gradient.api_sdk.archivers.
ZipArchiver
(logger=None)¶ Bases:
object
-
DEFAULT_EXCLUDED_PATHS
= ['.git/*', '.idea/*', '.pytest_cache/*']¶
-
__init__
(logger=None)¶ Initialize self. See help(type(self)) for accurate signature.
-
archive
(input_dir_path, output_file_path, overwrite_existing_archive=True, exclude=None)¶ - Parameters
input_dir_path (str) –
output_file_path (str) –
overwrite_existing_archive (bool) –
exclude (list|tuple|None) –
-
get_excluded_paths
(exclude=None)¶ - Parameters
exclude (list|tuple|None) –
- Return type
set
-
static
get_file_paths
(input_path, excluded_paths=None)¶ Get a dictionary of all files in input_dir excluding specified in excluded_paths
- Parameters
input_path (str) –
excluded_paths (list|tuple|set|None) –
- Returns
dictionary with full paths as values as keys and relative paths
- Return type
dict[str,str]
-
-
class
gradient.api_sdk.archivers.
ZipArchiverWithProgressbar
(logger=None)¶
gradient.api_sdk.config module¶
-
gradient.api_sdk.config.
get_api_key
(config_dir_path, config_file_name)¶
-
gradient.api_sdk.config.
get_help_colors_dict
(use_colors, help_headers_color, help_options_color)¶
-
class
gradient.api_sdk.config.
config
¶ Bases:
object
-
DEBUG
= False¶
-
WEB_URL
= 'https://console.paperspace.com'¶
-
CONFIG_HOST
= 'https://api.paperspace.io'¶
-
CONFIG_LOG_HOST
= 'https://logs.paperspace.io'¶
-
CONFIG_EXPERIMENTS_HOST
= 'https://services.paperspace.io/experiments/v1/'¶
-
CONFIG_EXPERIMENTS_HOST_V2
= 'https://services.paperspace.io/experiments/v2/'¶
-
CONFIG_SERVICE_HOST
= 'https://services.paperspace.io'¶
-
CONFIG_DIR_PATH
= '/Users/dillon/.paperspace'¶
-
CONFIG_FILE_NAME
= 'config.json'¶
-
PAPERSPACE_API_KEY
= 'c9d19179a29378da35e0cf70ba70c9'¶
-
HELP_HEADERS_COLOR
= 'yellow'¶
-
HELP_OPTIONS_COLOR
= 'green'¶
-
USE_CONSOLE_COLORS
= True¶
-
HELP_COLORS_DICT
= {'help_headers_color': 'yellow', 'help_options_color': 'green'}¶
-
gradient.api_sdk.constants module¶
-
class
gradient.api_sdk.constants.
ExperimentType
¶ Bases:
object
-
SINGLE_NODE
= 1¶
-
GRPC_MULTI_NODE
= 2¶
-
MPI_MULTI_NODE
= 3¶
-
HYPERPARAMETER_TUNING
= 4¶
-
classmethod
get_type_str
(state_int)¶
-
-
class
gradient.api_sdk.constants.
ExperimentState
¶ Bases:
object
-
PENDING
= 1¶
-
PROVISIONED
= 2¶
-
NETWORK_SETUP
= 3¶
-
RUNNING
= 4¶
-
STOPPED
= 5¶
-
ERROR
= 6¶
-
FAILED
= 7¶
-
CANCELLED
= 8¶
-
NETWORK_TEARDOWN
= 9¶
-
CREATED
= 10¶
-
PROVISIONING
= 11¶
-
NETWORK_SETTING_UP
= 12¶
-
NETWORK_TEARING_DOWN
= 13¶
-
ABORTING
= 14¶
-
classmethod
get_state_str
(state_int)¶
-
-
class
gradient.api_sdk.constants.
Region
¶ Bases:
object
-
CA1
= 'West Coast (CA1)'¶
-
NY2
= 'East Coast (NY2)'¶
-
AMS1
= 'Europe (AMS1)'¶
-
-
class
gradient.api_sdk.constants.
RunMode
¶ Bases:
object
-
RUN_MODE_DEFAULT
= 1¶
-
RUN_MODE_PYTHON_COMMAND
= 2¶
-
RUN_MODE_SHELL_COMMAND
= 3¶
-
RUN_MODE_PYTHON_MODULE
= 4¶
-
-
class
gradient.api_sdk.constants.
ModelTypes
¶ Bases:
object
-
ONNX
= 'ONNX'¶
-
TENSORFLOW
= 'Tensorflow'¶
-
GENERIC
= 'Custom'¶
-
-
class
gradient.api_sdk.constants.
BuiltinMetrics
¶ Bases:
object
-
cpu_percentage
= 'cpuPercentage'¶
-
memory_usage
= 'memoryUsage'¶
-
gpu_memory_free
= 'gpuMemoryFree'¶
-
gpu_memory_used
= 'gpuMemoryUsed'¶
-
gpu_power_draw
= 'gpuPowerDraw'¶
-
gpu_temp
= 'gpuTemp'¶
-
gpu_utilization
= 'gpuUtilization'¶
-
gpu_memory_utilization
= 'gpuMemoryUtilization'¶
-
gradient.api_sdk.logger module¶
-
class
gradient.api_sdk.logger.
Logger
¶ Bases:
object
-
abstract
log
(msg, *args, **kwargs)¶
-
abstract
warning
(msg, *args, **kwargs)¶
-
abstract
error
(msg, *args, **kwargs)¶
-
debug
(msg, *args, **kwargs)¶
-
abstract
-
class
gradient.api_sdk.logger.
MuteLogger
¶ Bases:
gradient.api_sdk.logger.Logger
-
log
(msg, *args, **kwargs)¶
-
warning
(msg, *args, **kwargs)¶
-
error
(msg, *args, **kwargs)¶
-
gradient.api_sdk.s3_downloader module¶
-
class
gradient.api_sdk.s3_downloader.
S3FilesDownloader
(logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
object
-
__init__
(logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Initialize self. See help(type(self)) for accurate signature.
-
download_list
(sources, destination_dir)¶ - Parameters
sources (tuple[tuple[str,str]]) – tuple/list of (file_path, file_url) pairs
destination_dir (str) –
-
download_file
(source, destination_dir, max_retries=0)¶
-
-
class
gradient.api_sdk.s3_downloader.
ResourceDownloader
(api_key, logger=<gradient.api_sdk.logger.MuteLogger object>, ps_client_name=None)¶ Bases:
object
-
CLIENT_CLASS
= None¶
-
__init__
(api_key, logger=<gradient.api_sdk.logger.MuteLogger object>, ps_client_name=None)¶ Initialize self. See help(type(self)) for accurate signature.
-
download
(job_id, destination)¶
-
-
class
gradient.api_sdk.s3_downloader.
JobArtifactsDownloader
(api_key, logger=<gradient.api_sdk.logger.MuteLogger object>, ps_client_name=None)¶ Bases:
gradient.api_sdk.s3_downloader.ResourceDownloader
-
CLIENT_CLASS
¶
-
-
class
gradient.api_sdk.s3_downloader.
ModelFilesDownloader
(api_key, logger=<gradient.api_sdk.logger.MuteLogger object>, ps_client_name=None)¶ Bases:
gradient.api_sdk.s3_downloader.ResourceDownloader
-
CLIENT_CLASS
¶
-
gradient.api_sdk.s3_uploader module¶
-
class
gradient.api_sdk.s3_uploader.
S3FileUploader
(multipart_encoder_cls=None, logger=None, ps_client_name=None)¶ Bases:
object
-
DEFAULT_MULTIPART_ENCODER_CLS
¶
-
__init__
(multipart_encoder_cls=None, logger=None, ps_client_name=None)¶ - Parameters
multipart_encoder_cls (type(MultipartEncoder)) –
logger (Logger) –
-
upload
(file_path, url, s3_fields=None)¶ Upload a file to S3
- Parameters
file_path (str) –
url (str) –
bucket_name (str) –
s3_fields (dict[str,str]) –
-
static
get_bucket_url
(bucket_name, s3_fields)¶ - Parameters
bucket_name (str) –
s3_fields (dict) –
- Return type
str
-
-
class
gradient.api_sdk.s3_uploader.
S3PutFileUploader
(multipart_encoder_cls=None, logger=None, ps_client_name=None)¶
-
class
gradient.api_sdk.s3_uploader.
ExperimentFileUploader
(api_key, uploader=None, logger=None, ps_client_name=None)¶ Bases:
object
-
__init__
(api_key, uploader=None, logger=None, ps_client_name=None)¶ - Parameters
api_key (str) –
uploader (S3FileUploader) –
logger (Logger) –
-
upload
(file_path, project_id, cluster_id=None)¶ Upload file to S3 bucket for a project
- Parameters
file_path (str) –
project_id (str) –
cluster_id (str) –
- Return type
str
- Returns
S3 bucket’s URL
-
-
class
gradient.api_sdk.s3_uploader.
S3ProjectFileUploader
(api_key, uploader=None, logger=None, ps_client_name=None)¶ Bases:
gradient.api_sdk.s3_uploader.ExperimentFileUploader
DEPRECATED: This class will be renamed to ExperimentFileUploader in release v0.8
-
class
gradient.api_sdk.s3_uploader.
S3ModelFileUploader
(api_key, multipart_encoder_cls=None, logger=None, ps_client_name=None, s3uploader=None)¶ Bases:
object
-
DEFAULT_MULTIPART_ENCODER_CLS
¶ alias of
gradient.api_sdk.utils.MultipartEncoderWithProgressbar
-
__init__
(api_key, multipart_encoder_cls=None, logger=None, ps_client_name=None, s3uploader=None)¶ - Parameters
api_key (str) –
logger (Logger) –
-
upload
(file_path, model_id, cluster_id=None)¶ Upload file to S3 bucket for a project
- Parameters
file_path (str) –
model_id (str) –
- Return type
str
- Returns
S3 bucket’s URL
-
-
class
gradient.api_sdk.s3_uploader.
S3ModelUploader
(api_key, multipart_encoder_cls=None, logger=None, ps_client_name=None, s3uploader=None)¶ Bases:
gradient.api_sdk.s3_uploader.S3ModelFileUploader
-
upload
(file_path, model_id, cluster_id=None)¶ Upload file to S3 bucket for a project
- Parameters
file_path (str) –
model_id (str) –
- Return type
str
- Returns
S3 bucket’s URL
-
-
class
gradient.api_sdk.s3_uploader.
ExperimentWorkspaceDirectoryUploader
(api_key, temp_dir=None, archiver=None, project_uploader=None, ps_client_name=None)¶ Bases:
object
-
__init__
(api_key, temp_dir=None, archiver=None, project_uploader=None, ps_client_name=None)¶ - Parameters
api_key (str) –
temp_dir (str) –
archiver (ZipArchiver) –
project_uploader (ExperimentFileUploader) –
-
upload
(workspace_dir_path, project_id, exclude=None, temp_file_name='temp.zip')¶ Archive and upload a workspace directory
- Parameters
workspace_dir_path (str) –
project_id (str) –
exclude (list|tuple|None) –
temp_file_name (str) –
- Return type
str
- Returns
URL to the S3 bucket
-
get_archive_path
(temp_file_name)¶
-
-
class
gradient.api_sdk.s3_uploader.
S3WorkspaceDirectoryUploader
(api_key, temp_dir=None, archiver=None, project_uploader=None, ps_client_name=None)¶ Bases:
gradient.api_sdk.s3_uploader.ExperimentWorkspaceDirectoryUploader
DEPRECATED: This class will be renamed to ExperimentWorkspaceDirectoryUploader in release v0.8
-
class
gradient.api_sdk.s3_uploader.
DeploymentWorkspaceDirectoryUploader
(api_key, uploader=None, logger=None, ps_client_name=None)¶ Bases:
object
-
__init__
(api_key, uploader=None, logger=None, ps_client_name=None)¶ - Parameters
api_key (str) –
uploader (S3PutFileUploader) –
logger (Logger) –
-
upload
(file_path, project_id=None, cluster_id=None, **kwargs)¶ Upload file to S3 bucket for a project
- Parameters
file_path (str) –
project_id (str) –
cluster_id (str) –
- Return type
str
- Returns
S3 bucket’s URL
-
gradient.api_sdk.sdk_exceptions module¶
-
exception
gradient.api_sdk.sdk_exceptions.
GradientSdkError
¶ Bases:
Exception
-
exception
gradient.api_sdk.sdk_exceptions.
ResourceFetchingError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
ResourceCreatingError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
MalformedResponseError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
ResourceCreatingDataError
¶ Bases:
gradient.api_sdk.sdk_exceptions.ResourceCreatingError
-
exception
gradient.api_sdk.sdk_exceptions.
ArchiveUploadError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
PresignedUrlMalformedResponseError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
PresignedUrlError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
S3UploadFailedError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
ProjectAccessDeniedError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
ReceivingDataFailedError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
WrongPathError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
PresignedUrlUnreachableError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
PresignedUrlAccessDeniedError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
PresignedUrlConnectionError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
InvalidParametersError
¶
-
exception
gradient.api_sdk.sdk_exceptions.
EndWebsocketStream
¶ Bases:
Exception
gradient.api_sdk.utils module¶
-
class
gradient.api_sdk.utils.
MessageExtractor
¶ Bases:
object
-
get_message_from_response_data
(response_data, sep='\n')¶ - Return type
str
-
get_error_messages
(data, add_prefix=False)¶
-
-
gradient.api_sdk.utils.
print_dict_recursive
(input_dict, logger, indent=0, tabulator=' ')¶
-
class
gradient.api_sdk.utils.
ExperimentsClientHelpersMixin
¶ Bases:
object
-
gradient.api_sdk.utils.
validate_auth_options
(auth_username, auth_password, generate_auth)¶
-
gradient.api_sdk.utils.
generate_credential
(n)¶
-
gradient.api_sdk.utils.
generate_credentials_pair
(n)¶
-
gradient.api_sdk.utils.
base64_encode
(s)¶
-
gradient.api_sdk.utils.
base64_encode_attribute
(data, name)¶
-
gradient.api_sdk.utils.
concatenate_urls
(fst_part, snd_part)¶
-
class
gradient.api_sdk.utils.
MultipartEncoder
(fields)¶ Bases:
object
-
__init__
(fields)¶ Initialize self. See help(type(self)) for accurate signature.
-
get_monitor
()¶
-
-
class
gradient.api_sdk.utils.
MultipartEncoderWithProgressbar
(fields)¶
-
class
gradient.api_sdk.utils.
PathParser
¶ Bases:
object
-
LOCAL_DIR
= 0¶
-
LOCAL_FILE
= 1¶
-
GIT_URL
= 2¶
-
S3_URL
= 3¶
-
HTTP_URL
= 4¶
-
classmethod
parse_path
(path)¶
-
static
is_local_dir
(path)¶
-
static
is_local_zip_file
(path)¶
-
static
is_git_url
(path)¶
-
static
is_s3_url
(path)¶
-
static
is_http_url
(path)¶
-
classmethod
is_remote_path
(path)¶
-
classmethod
is_local_path
(path)¶
-
gradient.api_sdk.validation_messages module¶
gradient.api_sdk.workspace module¶
-
class
gradient.api_sdk.workspace.
WorkspaceHandler
(logger_=None, archiver_cls=None)¶ Bases:
object
-
WORKSPACE_ARCHIVER_CLS
¶
-
__init__
(logger_=None, archiver_cls=None)¶ - Parameters
logger – gradient.logger
-
handle
(input_data)¶
-
-
class
gradient.api_sdk.workspace.
S3WorkspaceHandler
(api_key, client_name=None, uploader_cls=None, *args, **kwargs)¶ Bases:
gradient.api_sdk.workspace.WorkspaceHandler
-
WORKSPACE_UPLOADER_CLS
¶ alias of
gradient.api_sdk.s3_uploader.ExperimentFileUploader
-
__init__
(api_key, client_name=None, uploader_cls=None, *args, **kwargs)¶ - Parameters
api_key (str) –
client_name (str) –
uploader_cls (object) –
logger (gradient.logger) –
-
handle
(input_data)¶
-
-
class
gradient.api_sdk.workspace.
S3WorkspaceHandlerWithProgressbar
(api_key, client_name=None, uploader_cls=None, *args, **kwargs)¶ Bases:
gradient.api_sdk.workspace.S3WorkspaceHandler
-
WORKSPACE_ARCHIVER_CLS
¶ alias of
gradient.api_sdk.archivers.ZipArchiverWithProgressbar
-