gradient.api_sdk.clients package¶
Submodules¶
gradient.api_sdk.clients.base_client module¶
-
class
gradient.api_sdk.clients.base_client.
BaseClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
object
-
__init__
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Base class. All client classes inherit from it.
- Parameters
api_key (str) – your API key
ps_client_name (str) –
logger (sdk_logger.Logger) –
-
build_repository
(repository_class, *args, **kwargs)¶ - Parameters
repository_class (type[BaseRepository]) –
- Return type
-
-
class
gradient.api_sdk.clients.base_client.
TagsSupportMixin
¶ Bases:
object
-
entity
= ''¶
Add tags to entity. :param entity_id: :param entity: :param tags: :return:
Remove tags from entity. :param str entity_id: :param list[str] tags: list of tags to remove from entity :return:
List tags for entity :param list[str] entity_ids: :return:
-
gradient.api_sdk.clients.clusters module¶
-
class
gradient.api_sdk.clients.clusters.
ClustersClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.BaseClient
-
list
(limit=11, offset=0, **kwargs)¶ Get a list of clusters for your team
- Parameters
limit (int) – how many element to return on request
offset (int) – from what position we should return clusters
- Returns
clusters
- Return type
list
-
gradient.api_sdk.clients.dataset_client module¶
-
class
gradient.api_sdk.clients.dataset_client.
DatasetsClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.BaseClient
-
list
(limit=20, offset=0)¶ Get a list of datasets
- Parameters
limit (int) – Limit results
offset (int) – Skip results
- Returns
List of datasets
- Return type
list[models.Dataset]
-
delete
(dataset_id)¶ Delete a dataset
- Parameters
dataset_id (str) – Dataset ID [required]
- Returns
- Return type
None
-
get
(dataset_id)¶ Get a dataset
- Parameters
dataset_id (str) – Dataset ID [required]
- Returns
Dataset
- Return type
models.Dataset
-
get_ref
(dataset_ref)¶ Get dataset with resolved version by reference
- Parameters
dataset_ref (str) – Dataset reference [required]
- Returns
Dataset with resolved version
- Return type
models.DatasetRef
-
create
(name, storage_provider_id, description=None)¶ Create a new dataset
- Parameters
name (str) – Name of dataset [required]
storage_provider_id (str) – Storage provider ID [required]
description (str) – Brief description of the dataset
- Returns
Dataset ID
- Return type
str
-
update
(dataset_id, name=None, description=None)¶ Update an existing dataset
- Parameters
dataset_id (str) – Dataset ID [required]
name (str) – Name of dataset
description (str) – Brief description of the dataset
- Returns
- Return type
None
-
gradient.api_sdk.clients.dataset_tag_client module¶
-
class
gradient.api_sdk.clients.dataset_tag_client.
DatasetTagsClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.BaseClient
-
list
(dataset_id, limit=20, offset=0)¶ Get a list of dataset tags
- Parameters
dataset_id (str) – Dataset ID [required]
limit (int) – Limit results
offset (int) – Skip results
- Returns
List of dataset tags
- Return type
list[models.DatasetTag]
-
delete
(dataset_tag_id)¶ Delete a dataset tag
- Parameters
dataset_tag_id (str) – Dataset tag ID (ex: dataset_id:tag) [required]
- Returns
- Return type
None
-
get
(dataset_tag_id)¶ Get a dataset tag
- Parameters
dataset_tag_id (str) – Dataset tag ID (ex: dataset_id:tag) [required]
- Returns
Dataset tag
- Return type
models.DatasetTag
-
set
(dataset_version_id, name)¶ Set a dataset tag
- Parameters
dataset_version_id (str) – Dataset version ID (ex: dataset_id:version) [required]
name (str) – Dataset tag name [required]
- Returns
Dataset tag
- Return type
models.DatasetTag
-
gradient.api_sdk.clients.dataset_version_client module¶
-
class
gradient.api_sdk.clients.dataset_version_client.
DatasetVersionsClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.BaseClient
-
list
(dataset_id, is_committed=True, limit=20, offset=0)¶ Get list of your dataset versions
- Parameters
dataset_id (str) – Dataset ID [required]
is_committed (bool) – Filter versions by commit status
limit (int) – Limit results
offset (int) – Skip results
- Returns
List of dataset versions
- Return type
list[models.DatasetVersion]
-
delete
(dataset_version_id)¶ Delete a dataset version
- Parameters
dataset_version_id (str) – Dataset version ID (ex: dataset_id:version) [required]
- Returns
- Return type
None
-
get
(dataset_version_id)¶ Delete a dataset version
- Parameters
dataset_version_id (str) – Dataset version ID (ex: dataset_id:version) [required]
- Returns
dataset
- Return type
models.DatasetVersion
-
create
(dataset_id, message=None)¶ Create a new dataset version
- Parameters
id (str) – Dataset ID [required]
message (str) – Dataset version message
- Returns
dataset ID
- Return type
str
-
update
(dataset_version_id, message=None, is_committed=None)¶ Update an existing S3 dataset
- Parameters
dataset_version_id (str) – Dataset version ID (ex: dataset_id:version)
message (str) – Dataset version message
is_committed (bool) – Mark dataset version as committed
- Returns
- Return type
None
-
generate_pre_signed_s3_url
(dataset_version_id, method, params=None)¶ Generate pre-signed URL for S3 storage providers
- Parameters
dataset_version_id (str) – Dataset version ID (ex: dataset_id:version)
method (str) – S3 method
params (dict) – S3 params
- Returns
- Return type
-
generate_pre_signed_s3_urls
(dataset_version_id, calls)¶ Generate pre-signed URLs for S3 storage providers
- Parameters
dataset_version_id (str) – Dataset version ID (ex: dataset_id:version)
calls (list[dict]) – List of S3 calls
- Returns
- Return type
list[models.DatasetVersionPreSignedURL]
-
gradient.api_sdk.clients.deployment_client module¶
Deployment related client handler logic.
Remember that in code snippets all highlighted lines are required other lines are optional.
-
class
gradient.api_sdk.clients.deployment_client.
DeploymentsClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.TagsSupportMixin
,gradient.api_sdk.clients.base_client.BaseClient
Client to handle deployment related actions.
How to create instance of deployment client:
1 2 3 4 5
from gradient import DeploymentsClient deployment_client = DeploymentsClient( api_key='your_api_key_here' )
-
HOST_URL
= 'https://api.paperspace.io'¶
-
entity
= 'deployment'¶
-
create
(deployment_type, name, machine_type, image_url, instance_count=None, model_id=None, container_model_path=None, image_username=None, image_password=None, image_server=None, container_url_path=None, endpoint_url_path=None, method=None, docker_args=None, env=None, api_type=None, ports=None, cluster_id=None, auth_username=None, auth_password=None, tags=None, command=None, workspace_url=None, workspace_ref=None, workspace_username=None, workspace_password=None, project_id=None, autoscaling=None)¶ Method to create a Deployment instance.
To create a new Deployment, you must first create a Model. With a Model available, use the
create
subcommand and specify all of the following parameters: deployment type, base image, name, machine type, and container image for serving, as well as the instance count:1 2 3 4 5
from gradient import DeploymentsClient deployment_client = DeploymentsClient( api_key='your_api_key_here' )
To obtain your Model ID, you can run
command gradient models list
and copy the target Model ID from your available Models.- Parameters
deployment_type (str) – Model deployment type. Only TensorFlow Model deployment type is currently supported [required]
model_id (str) – ID of a trained model [required]
name (str) – Human-friendly name for new model deployment [required]
machine_type (str) – [G1|G6|G12|K80|P100|GV100] Type of machine for new deployment [required]
image_url (str) – Docker image for model deployment [required]
instance_count (int) – Number of machine instances [required]
container_model_path (str) – Container model path
image_username (str) – Username used to access docker image
image_password (str) – Password used to access docker image
image_server (str) – Docker image server
container_url_path (str) – Container URL path
endpoint_url_path (str) – Endpoint URL path
method (str) – Method
docker_args (list[str]|tuple[str]) – List of docker args
env (dict[str,str]) – Environmental variables
api_type (str) – Type of API (REST/GRPC)
ports (str) – Ports
cluster_id (str) – cluster ID
auth_username (str) – Username
auth_password (str) – Password
tags (list[str]) – List of tags
command (str) – Deployment command
workspace_url (str) – Project git or s3repository url
workspace_ref (str) – Git commit hash, branch name or tag
workspace_username (str) – Project git repository username
workspace_password (str) – Project git repository password
project_id (str) – Project ID
autoscaling (models.AutoscalingDefinition) – Deployment autoscaling definition
- Returns
Created deployment id
- Return type
str
-
get
(deployment_id)¶ Get deployment instance
- Parameters
deployment_id (str) – Deployment ID
- Returns
Deployment instance
- Return type
models.Deployment
-
start
(deployment_id)¶ Start deployment
EXAMPLE:
gradient deployments start --id <your-deployment-id>
- Parameters
deployment_id (str) – Deployment ID
-
stop
(deployment_id)¶ Stop deployment
EXAMPLE:
gradient deployments stop --id <your-deployment-id>
- Parameters
deployment_id – Deployment ID
-
list
(state=None, project_id=None, model_id=None, tags=None)¶ List deployments with optional filtering
- Parameters
state (str) – state to filter deployments
project_id (str) – project ID to filter deployments
model_id (str) – model ID to filter deployments
tags (list[str]|tuple[str]) – tags to filter deployments with OR
- Returns
List of Deployment model instances
- Return type
list[models.Deployment]
-
delete
(deployment_id)¶
-
update
(deployment_id, deployment_type=None, model_id=None, name=None, machine_type=None, image_url=None, instance_count=None, container_model_path=None, image_username=None, image_password=None, image_server=None, container_url_path=None, endpoint_url_path=None, method=None, docker_args=None, env=None, api_type=None, ports=None, cluster_id=None, auth_username=None, auth_password=None, workspace_url=None, workspace_ref=None, workspace_username=None, workspace_password=None, project_id=None, command=None, autoscaling=None)¶
-
get_metrics
(deployment_id, start=None, end=None, interval='30s', built_in_metrics=None)¶ Get model deployment metrics
- Parameters
deployment_id (str) – ID of deployment
start (datetime.datetime|str) –
end (datetime.datetime|str) –
interval (str) –
built_in_metrics (list[str]) –
List of metrics to get if different than default Available builtin metrics: cpuPercentage, memoryUsage, gpuMemoryFree, gpuMemoryUsed, gpuPowerDraw,
gpuTemp, gpuUtilization, gpuMemoryUtilization
- Returns
Metrics of a model deployment job
- Return type
dict[str,dict[str,list[dict]]]
-
list_metrics
(deployment_id, start=None, end=None, interval='30s')¶ List model deployment metrics
- Parameters
deployment_id (str) – ID of deployment
start (datetime.datetime|str) –
end (datetime.datetime|str) –
interval (str) –
- Returns
Metrics of a model deployment job
- Return type
dict[str,dict[str,list[dict]]]
-
stream_metrics
(deployment_id, interval='30s', built_in_metrics=None)¶ Stream live model deployment metrics
- Parameters
deployment_id (str) – ID of model deployment
interval (str) –
built_in_metrics (list[str]) –
List of metrics to get if different than default Available builtin metrics: cpuPercentage, memoryUsage, gpuMemoryFree, gpuMemoryUsed, gpuPowerDraw,
gpuTemp, gpuUtilization, gpuMemoryUtilization
- Returns
Generator object yielding live model deployment metrics
- Return type
Iterable[dict]
-
logs
(deployment_id, line=1, limit=10000)¶ Show list of latest logs from the specified deployment.
- Parameters
deployment_id (str) – Deployment Id
line (int) – line number at which logs starts to display on screen
limit (int) – maximum lines displayed on screen, default set to 10 000
- Returns
list of LogRows
- Return type
list[models.LogRow]
-
yield_logs
(deployment_id, line=1, limit=10000)¶ Get log generator. Polls the API for new logs
- Parameters
deployment_id (str) – Deployment Id
line (int) – line number at which logs starts to display on screen
limit (int) – maximum lines displayed on screen, default set to 10 000
- Returns
generator yielding LogRow instances
- Return type
Iterator[models.LogRow]
-
gradient.api_sdk.clients.experiment_client module¶
-
class
gradient.api_sdk.clients.experiment_client.
ExperimentsClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.TagsSupportMixin
,gradient.api_sdk.utils.ExperimentsClientHelpersMixin
,gradient.api_sdk.clients.base_client.BaseClient
-
entity
= 'experiment'¶
-
create_single_node
(project_id, machine_type, command, ports=None, name=None, workspace_url=None, workspace_ref=None, workspace_username=None, workspace_password=None, datasets=None, working_directory=None, artifact_directory=None, cluster_id=None, experiment_env=None, model_type=None, model_path=None, is_preemptible=False, container=None, container_user=None, registry_username=None, registry_password=None, registry_url=None, tags=None)¶ Create single node experiment
- Parameters
name (str) – Name of new experiment [required]
project_id (str) – Project ID [required]
machine_type (str) – Machine type [required]
command (str) – Container entrypoint command [required]
ports (str) – Port to use in new experiment
workspace_url (str) – Project git repository url
workspace_ref (str) – Git commit hash, branch name or tag
workspace_username (str) – Project git repository username
workspace_password (str) – Project git repository password
datasets (dict|list[dict]|tuple[dict]) – Dict or list of dicts describing dataset(s) used in experiment. Required keys: “id” or “url” Optional keys: “tag” for S3 tag and “auth” for S3 token
working_directory (str) – Working directory for the experiment
artifact_directory (str) – Artifacts directory
cluster_id (str) – Cluster ID
experiment_env (dict) – Environment variables in a JSON
model_type (str) – defines the type of model that is being generated by the experiment. Model type must be one of Tensorflow, ONNX, or Custom
model_path (str) – Model path
is_preemptible (bool) – Is preemptible
container (str) – Container (dockerfile) [required]
container_user (str) – Container user for running the specified command in the container. If no containerUser is specified, the user will default to ‘root’ in the container.
registry_username (str) – Registry username for accessing private docker registry container if necessary
registry_password (str) – Registry password for accessing private docker registry container if necessary
registry_url (str) – Registry server URL for accessing private docker registry container if necessary
tags (list[str]) – List of tags
- Returns
experiment handle
- Return type
str
-
create_multi_node
(project_id, worker_container, worker_machine_type, worker_command, worker_count, parameter_server_container, parameter_server_machine_type, parameter_server_command, parameter_server_count, name=None, experiment_type_id=2, ports=None, workspace_url=None, workspace_ref=None, workspace_username=None, workspace_password=None, datasets=None, working_directory=None, artifact_directory=None, cluster_id=None, experiment_env=None, model_type=None, model_path=None, is_preemptible=False, worker_container_user=None, worker_registry_username=None, worker_registry_password=None, worker_registry_url=None, parameter_server_container_user=None, parameter_server_registry_username=None, parameter_server_registry_password=None, parameter_server_registry_url=None, tags=None)¶ Create multinode experiment
- Parameters
name (str) – Name of new experiment [required]
project_id (str) – Project ID [required]
worker_container (str) – Worker container (dockerfile) [required]
worker_machine_type (str) – Worker machine type [required]
worker_command (str) – Worker command [required]
worker_count (int) – Worker count [required]
parameter_server_container (str) – Parameter server container [required]
parameter_server_machine_type (str) – Parameter server machine type [required]
parameter_server_command (str) – Parameter server command [required]
parameter_server_count (int) – Parameter server count [required]
experiment_type_id (int|str) – Experiment Type ID
ports (str) – Port to use in new experiment
workspace_url (str) – Project git repository url
workspace_ref (str) – Git commit hash, branch name or tag
workspace_username (str) – Project git repository username
workspace_password (str) – Project git repository password
datasets (dict|list[dict]|tuple[dict]) – Dict or list of dicts describing dataset(s) used in experiment. Required keys: “url” Optional keys: “tag” for S3 tag and “auth” for S3 token
working_directory (str) – Working directory for the experiment
artifact_directory (str) – Artifacts directory
cluster_id (str) – Cluster ID
experiment_env (dict) – Environment variables in a JSON
model_type (str) – defines the type of model that is being generated by the experiment. Model type must be one of Tensorflow, ONNX, or Custom
model_path (str) – Model path
is_preemptible (bool) – Is preemptible
worker_container_user (str) – Worker container user
worker_registry_username (str) – Registry username for accessing private docker registry container if necessary
worker_registry_password (str) – Registry password for accessing private docker registry container if necessary
worker_registry_url (str) – Registry server URL for accessing private docker registry container if necessary
parameter_server_container_user (str) – Parameter server container user
parameter_server_registry_username (str) – Registry username for accessing private docker registry container if necessary
parameter_server_registry_password (str) – Registry password for accessing private docker registry container if necessary
parameter_server_registry_url (str) – Registry server URL for accessing private docker registry container if necessary
tags (list[str]) – List of tags
- Returns
experiment handle
- Return type
str
-
create_mpi_multi_node
(project_id, worker_container, worker_machine_type, worker_command, worker_count, name=None, master_container=None, master_machine_type=None, master_command=None, master_count=None, ports=None, workspace_url=None, workspace_ref=None, workspace_username=None, workspace_password=None, datasets=None, working_directory=None, artifact_directory=None, cluster_id=None, experiment_env=None, model_type=None, model_path=None, is_preemptible=False, worker_container_user=None, worker_registry_username=None, worker_registry_password=None, worker_registry_url=None, master_container_user=None, master_registry_username=None, master_registry_password=None, master_registry_url=None, tags=None)¶ Create multinode experiment using MPI
- Parameters
name (str) – Name of new experiment [required]
project_id (str) – Project ID [required]
worker_container (str) – Worker container (dockerfile) [required]
worker_machine_type (str) – Worker machine type [required]
worker_command (str) – Worker command [required]
worker_count (int) – Worker count [required]
master_container (str) – Master container [required]
master_machine_type (str) – Master machine type [required]
master_command (str) – Master command [required]
master_count (int) – Master count [required]
ports (str) – Port to use in new experiment
workspace_url (str) – Project git repository url
workspace_ref (str) – Git commit hash, branch name or tag
workspace_username (str) – Project git repository username
workspace_password (str) – Project git repository password
datasets (dict|list[dict]|tuple[dict]) – Dict or list of dicts describing dataset(s) used in experiment. Required keys: “url” Optional keys: “tag” for S3 tag and “auth” for S3 token
working_directory (str) – Working directory for the experiment
artifact_directory (str) – Artifacts directory
cluster_id (str) – Cluster ID
experiment_env (dict) – Environment variables in a JSON
model_type (str) – defines the type of model that is being generated by the experiment. Model type must be one of Tensorflow, ONNX, or Custom
model_path (str) – Model path
is_preemptible (bool) – Is preemptible
worker_container_user (str) – Worker container user
worker_registry_username (str) – Registry username for accessing private docker registry container if necessary
worker_registry_password (str) – Registry password for accessing private docker registry container if necessary
worker_registry_url (str) – Registry server URL for accessing private docker registry container if necessary
master_container_user (str) – Master container user
master_registry_username (str) – Registry username for accessing private docker registry container if necessary
master_registry_password (str) – Registry password for accessing private docker registry container if necessary
master_registry_url (str) – Registry server URL for accessing private docker registry container if necessary
tags (list[str]) – List of tags
- Returns
experiment handle
- Return type
str
-
run_single_node
(project_id, machine_type, command, name=None, ports=None, workspace_url=None, workspace_ref=None, workspace_username=None, workspace_password=None, datasets=None, working_directory=None, artifact_directory=None, cluster_id=None, experiment_env=None, model_type=None, model_path=None, is_preemptible=False, container=None, container_user=None, registry_username=None, registry_password=None, registry_url=None, tags=None)¶ Create and start single node experiment
- Parameters
name (str) – Name of new experiment [required]
project_id (str) – Project ID [required]
machine_type (str) – Machine type [required]
command (str) – Container entrypoint command [required]
ports (str) – Port to use in new experiment
workspace_url (str) – Project git repository url
workspace_ref (str) – Git commit hash, branch name or tag
workspace_username (str) – Project git repository username
workspace_password (str) – Project git repository password
datasets (dict|list[dict]|tuple[dict]) – Dict or list of dicts describing dataset(s) used in experiment. Required keys: “url” Optional keys: “tag” for S3 tag and “auth” for S3 token
working_directory (str) – Working directory for the experiment
artifact_directory (str) – Artifacts directory
cluster_id (str) – Cluster ID
experiment_env (dict) – Environment variables in a JSON
model_type (str) – defines the type of model that is being generated by the experiment. Model type must be one of Tensorflow, ONNX, or Custom
model_path (str) – Model path
is_preemptible (bool) – Is preemptible
container (str) – Container (dockerfile) [required]
container_user (str) – Container user for running the specified command in the container. If no containerUser is specified, the user will default to ‘root’ in the container.
registry_username (str) – Registry username for accessing private docker registry container if necessary
registry_password (str) – Registry password for accessing private docker registry container if necessary
registry_url (str) – Registry server URL for accessing private docker registry container if necessary
tags (list[str]) – List of tags
- Returns
experiment handle
- Return type
str
-
run_multi_node
(project_id, worker_container, worker_machine_type, worker_command, worker_count, parameter_server_container, parameter_server_machine_type, parameter_server_command, parameter_server_count, name=None, experiment_type_id=2, ports=None, workspace_url=None, workspace_ref=None, workspace_username=None, workspace_password=None, datasets=None, working_directory=None, artifact_directory=None, cluster_id=None, experiment_env=None, model_type=None, model_path=None, is_preemptible=False, worker_container_user=None, worker_registry_username=None, worker_registry_password=None, worker_registry_url=None, parameter_server_container_user=None, parameter_server_registry_username=None, parameter_server_registry_password=None, parameter_server_registry_url=None, tags=None)¶ Create and start multinode experiment
- Parameters
name (str) – Name of new experiment [required]
project_id (str) – Project ID [required]
worker_container (str) – Worker container (dockerfile) [required]
worker_machine_type (str) – Worker machine type [required]
worker_command (str) – Worker command [required]
worker_count (int) – Worker count [required]
parameter_server_container (str) – Parameter server container [required]
parameter_server_machine_type (str) – Parameter server machine type [required]
parameter_server_command (str) – Parameter server command [required]
parameter_server_count (int) – Parameter server count [required]
experiment_type_id (int|str) – Experiment Type ID [required]
ports (str) – Port to use in new experiment
workspace_url (str) – Project git repository url
workspace_ref (str) – Git commit hash, branch name or tag
workspace_username (str) – Project git repository username
workspace_password (str) – Project git repository password
datasets (dict|list[dict]|tuple[dict]) – Dict or list of dicts describing dataset(s) used in experiment. Required keys: “url” Optional keys: “tag” for S3 tag and “auth” for S3 token
working_directory (str) – Working directory for the experiment
artifact_directory (str) – Artifacts directory
cluster_id (str) – Cluster ID
experiment_env (dict) – Environment variables in a JSON
model_type (str) – defines the type of model that is being generated by the experiment. Model type must be one of Tensorflow, ONNX, or Custom
model_path (str) – Model path
is_preemptible (bool) – Is preemptible
worker_container_user (str) – Worker container user
worker_registry_username (str) – Registry username for accessing private docker registry container if necessary
worker_registry_password (str) – Registry password for accessing private docker registry container if necessary
worker_registry_url (str) – Registry server URL for accessing private docker registry container if necessary
parameter_server_container_user (str) – Parameter server container user
parameter_server_registry_username (str) – Registry username for accessing private docker registry container if necessary
parameter_server_registry_password (str) – Registry password for accessing private docker registry container if necessary
parameter_server_registry_url (str) – Registry server URL for accessing private docker registry container if necessary
tags (list[str]) – List of tags
- Returns
experiment handle
- Return type
str
-
run_mpi_multi_node
(project_id, worker_container, worker_machine_type, worker_command, worker_count, master_container, master_machine_type, master_command, master_count, name=None, ports=None, workspace_url=None, workspace_ref=None, workspace_username=None, workspace_password=None, datasets=None, working_directory=None, artifact_directory=None, cluster_id=None, experiment_env=None, model_type=None, model_path=None, is_preemptible=False, worker_container_user=None, worker_registry_username=None, worker_registry_password=None, worker_registry_url=None, master_container_user=None, master_registry_username=None, master_registry_password=None, master_registry_url=None, tags=None)¶ Create and start multinode experiment using MPI
- Parameters
name (str) – Name of new experiment [required]
project_id (str) – Project ID [required]
worker_container (str) – Worker container (dockerfile) [required]
worker_machine_type (str) – Worker machine type [required]
worker_command (str) – Worker command [required]
worker_count (int) – Worker count [required]
master_container (str) – Master container [required]
master_machine_type (str) – Master machine type [required]
master_command (str) – Master command [required]
master_count (int) – Master count [required]
ports (str) – Port to use in new experiment
workspace_url (str) – Project git repository url
workspace_ref (str) – Git commit hash, branch name or tag
workspace_username (str) – Project git repository username
workspace_password (str) – Project git repository password
datasets (dict|list[dict]|tuple[dict]) – Dict or list of dicts describing dataset(s) used in experiment. Required keys: “url” Optional keys: “tag” for S3 tag and “auth” for S3 token
working_directory (str) – Working directory for the experiment
artifact_directory (str) – Artifacts directory
cluster_id (str) – Cluster ID
experiment_env (dict) – Environment variables in a JSON
model_type (str) – defines the type of model that is being generated by the experiment. Model type must be one of Tensorflow, ONNX, or Custom
model_path (str) – Model path
is_preemptible (bool) – Is preemptible
worker_container_user (str) – Worker container user
worker_registry_username (str) – Registry username for accessing private docker registry container if necessary
worker_registry_password (str) – Registry password for accessing private docker registry container if necessary
worker_registry_url (str) – Registry server URL for accessing private docker registry container if necessary
master_container_user (str) – Master container user
master_registry_username (str) – Registry username for accessing private docker registry container if necessary
master_registry_password (str) – Registry password for accessing private docker registry container if necessary
master_registry_url (str) – Registry server URL for accessing private docker registry container if necessary
tags (list[str]) – List of tags
- Returns
experiment handle
- Return type
str
-
start
(experiment_id)¶ Start existing experiment that has not run
- Parameters
experiment_id (str) – Experiment ID
- Raises
exceptions.GradientSdkError
-
stop
(experiment_id)¶ Stop running experiment
- Parameters
experiment_id (str) – Experiment ID
- Raises
exceptions.GradientSdkError
-
list
(project_id=None, offset=None, limit=None, get_meta=False, tags=None)¶ Get a list of experiments. Optionally filter by project ID
- Parameters
project_id (str|list|None) –
offset (int) –
limit (int) –
get_meta (bool) – get dict of metadata like number of total items, etc. Setting to True changes rtype
tags (list[str]|tuple[str]) – tags to filter with OR
- Returns
experiments
- Return type
list[models.SingleNodeExperiment|models.MultiNodeExperiment]|tuple[list[models.SingleNodeExperiment|models.MultiNodeExperiment],dict]
-
get
(experiment_id)¶ Get experiment instance
- Parameters
experiment_id (str) – Experiment ID
- Return type
models.SingleNodeExperiment|models.MultiNodeExperiment|MpiMultiNodeExperiment
-
logs
(experiment_id, line=1, limit=10000)¶ Show list of latest logs from the specified experiment.
- Parameters
experiment_id (str) – Experiment ID
line (int) – line number at which logs starts to display on screen
limit (int) – maximum lines displayed on screen, default set to 10 000
- Returns
list of LogRows
- Return type
list[models.LogRow]
-
yield_logs
(experiment_id, line=1, limit=10000)¶ Get log generator. Polls the API for new logs
- Parameters
experiment_id (str) –
line (int) – line number at which logs starts to display on screen
limit (int) – maximum lines displayed on screen, default set to 10 000
- Returns
generator yielding LogRow instances
- Return type
Iterator[models.LogRow]
-
delete
(experiment_id)¶
-
get_metrics
(experiment_id, start=None, end=None, interval='30s', built_in_metrics=None)¶ Get experiment metrics
- Parameters
experiment_id (str) – ID of experiment
start (datetime.datetime|str) –
end (datetime.datetime|str) –
interval (str) –
built_in_metrics (list[str]) –
List of metrics to get if different than default Available builtin metrics: cpuPercentage, memoryUsage, gpuMemoryFree, gpuMemoryUsed, gpuPowerDraw,
gpuTemp, gpuUtilization, gpuMemoryUtilization
- Returns
Metrics of and experiment
- Return type
dict[str,dict[str,list[dict]]]
-
list_metrics
(experiment_id, start=None, end=None, interval='30s')¶ List experiment metrics
- Parameters
experiment_id (str) – ID of experiment
start (datetime.datetime|str) –
end (datetime.datetime|str) –
interval (str) –
- Returns
Metrics of and experiment
- Return type
dict[str,dict[str,list[dict]]]
-
stream_metrics
(experiment_id, interval='30s', built_in_metrics=None)¶ Stream live experiment metrics
- Parameters
experiment_id (str) – ID of experiment
interval (str) –
built_in_metrics (list[str]) –
List of metrics to get if different than default Available builtin metrics: cpuPercentage, memoryUsage, gpuMemoryFree, gpuMemoryUsed, gpuPowerDraw,
gpuTemp, gpuUtilization, gpuMemoryUtilization
- Returns
Generator object yielding live experiment metrics
- Return type
Iterable[dict]
-
gradient.api_sdk.clients.http_client module¶
-
class
gradient.api_sdk.clients.http_client.
API
(api_url, headers=None, api_key=None, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
object
-
__init__
(api_url, headers=None, api_key=None, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ - Parameters
api_url (str) – url you want to connect
headers (dict) – headers
api_key (str) – your API key
ps_client_name (str) – Client name
logger (sdk_logger.Logger) –
-
property
api_key
¶
-
property
ps_client_name
¶
-
get_path
(url)¶
-
post
(url, json=None, params=None, files=None, data=None)¶
-
put
(url, json=None, params=None, data=None)¶
-
get
(url, json=None, params=None)¶
-
delete
(url, json=None, params=None)¶
-
gradient.api_sdk.clients.hyperparameter_client module¶
-
class
gradient.api_sdk.clients.hyperparameter_client.
HyperparameterJobsClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.TagsSupportMixin
,gradient.api_sdk.clients.base_client.BaseClient
-
entity
= 'experiment'¶
-
create
(name, project_id, tuning_command, worker_container, worker_machine_type, worker_command, worker_count, worker_container_user=None, worker_registry_username=None, worker_registry_password=None, is_preemptible=False, ports=None, workspace_url=None, artifact_directory=None, cluster_id=None, experiment_env=None, trigger_event_id=None, model_type=None, model_path=None, dockerfile_path=None, hyperparameter_server_registry_username=None, hyperparameter_server_registry_password=None, hyperparameter_server_container=None, hyperparameter_server_container_user=None, hyperparameter_server_machine_type=None, working_directory=None, use_dockerfile=False, tags=None)¶ Create hyperparameter tuning job :param str name: Name of new experiment [required] :param str project_id: Project ID [required] :param str tuning_command: Tuning command [required] :param str worker_container: Worker container [required] :param str worker_machine_type: Worker machine type [required] :param str worker_command: Worker command [required] :param int worker_count: Worker count [required] :param str worker_container_user: Worker Container user :param str worker_registry_username: Worker registry username :param str worker_registry_password: Worker registry password :param bool is_preemptible: Flag: is preemptible :param str ports: Port to use in new experiment :param str workspace_url: Project git repository url :param str artifact_directory: Artifacts directory :param str cluster_id: Cluster ID :param dict experiment_env: Environment variables (in JSON) :param str trigger_event_id: GradientCI trigger event id :param str model_type: Model type :param str model_path: Model path :param str dockerfile_path: Path to dockerfile in project :param str hyperparameter_server_registry_username: Hyperparameter server registry username :param str hyperparameter_server_registry_password: Hyperparameter server registry password :param str hyperparameter_server_container: Hyperparameter server container :param str hyperparameter_server_container_user: Hyperparameter server container user :param str hyperparameter_server_machine_type: Hyperparameter server machine type :param str working_directory: Working directory for the experiment :param bool use_dockerfile: Flag: use dockerfile :param list[str] tags: List of tags
- Returns
ID of a new job
- Return type
str
-
run
(name, project_id, tuning_command, worker_container, worker_machine_type, worker_command, worker_count, worker_registry_username=None, worker_registry_password=None, worker_container_user=None, is_preemptible=False, ports=None, workspace_url=None, artifact_directory=None, cluster_id=None, experiment_env=None, trigger_event_id=None, model_type=None, model_path=None, dockerfile_path=None, hyperparameter_server_registry_username=None, hyperparameter_server_registry_password=None, hyperparameter_server_container_user=None, hyperparameter_server_container=None, hyperparameter_server_machine_type=None, working_directory=None, use_dockerfile=False, tags=None)¶ Create and start hyperparameter tuning job
- Parameters
name (str) – Name of new experiment [required]
project_id (str) – Project ID [required]
tuning_command (str) – Tuning command [required]
worker_container (str) – Worker container [required]
worker_machine_type (str) – Worker machine type [required]
worker_command (str) – Worker command [required]
worker_count (int) – Worker count [required]
worker_container_user (str) – Worker container user
worker_registry_password – Worker registry password
worker_registry_username – Worker registry username
is_preemptible (bool) – Flag: is preemptible
ports (str) – Port to use in new experiment
workspace_url (str) – Project git repository url
artifact_directory (str) – Artifacts directory
cluster_id (str) – Cluster ID
experiment_env (dict) – Environment variables (in JSON)
trigger_event_id (str) – GradientCI trigger event id
model_type (str) – Model type
model_path (str) – Model path
dockerfile_path (str) – Path to dockerfile
hyperparameter_server_registry_username (str) – container registry username
hyperparameter_server_registry_password (str) – container registry password
hyperparameter_server_container_user (str) – hps container user
hyperparameter_server_container (str) – hps container
hyperparameter_server_machine_type (str) – hps machine type
working_directory (str) – Working directory for the experiment
use_dockerfile (bool) – Flag: use dockerfile
tags (list[str]) – List of tags
- Returns
ID of a new job
- Return type
str
-
get
(id)¶ Get Hyperparameter tuning job’s instance
- Parameters
id (str) – Hyperparameter job id
- Returns
instance of Hyperparameter
- Return type
models.Hyperparameter
-
start
(id)¶ Start existing hyperparameter tuning job
- Parameters
id (str) – Hyperparameter job id
- Raises
exceptions.GradientSdkError
-
list
()¶ Get a list of hyperparameter tuning jobs
- Return type
list[models.Hyperparameter]
-
gradient.api_sdk.clients.job_client module¶
Jobs related client handler logic.
Remember that in code snippets all highlighted lines are required other lines are optional.
-
class
gradient.api_sdk.clients.job_client.
JobsClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.TagsSupportMixin
,gradient.api_sdk.clients.base_client.BaseClient
Client to handle job related actions.
How to create instance of job client:
1 2 3 4 5
from gradient import JobsClient job_client = JobClient( api_key='your_api_key_here' )
-
entity
= 'job'¶
-
create
(machine_type, container, project_id, data=None, name=None, command=None, ports=None, is_public=None, working_directory=None, experiment_id=None, env_vars=None, use_dockerfile=None, is_preemptible=None, project=None, started_by_user_id=None, rel_dockerfile_path=None, registry_username=None, registry_password=None, cluster=None, cluster_id=None, node_attrs=None, workspace_file_name=None, registry_target=None, registry_target_username=None, registry_target_password=None, build_only=False, tags=None, datasets=None)¶ Method to create and start job in paperspace gradient.
Example create job:
1 2 3 4 5 6 7 8 9 10
job = job_client.create( machine_type='K80', container='tensorflow/tensorflow:1.13.1-gpu-py3', project_id='Som3ProjecTiD', data=data, name='Example job', command='pip install -r requirements.txt && python mnist.py', ports='5000:5000', env_vars={'CUSTOM_ENV': 'Some value that will be set as system environment'} )
- Parameters
machine_type (str) –
Type of machine on which job should run. This field is required.
We recommend to choose one of this:
K80 P100 TPU GV100 GV100x8 G1 G6 G12
container (str) –
name of docker container that should be used to run job. This field is required.
Example value:
tensorflow/tensorflow:1.13.1-gpu-py3
project_id (str) – Identify to which project job should be connected. This field is required.
data (None|MultipartEncoderMonitor) – None if there are no data to upload or encoded multipart data information with files to upload.
name (str) – name for job that creator wish to have. If not provided it will be autogenerated.
command (str) – custom command that should be run against command from docker image
ports (str) –
string with comma , separated mapped ports.
Example value:
5000:5000,8080:8080
is_public (bool) – bool flag to select if job should be available by default None
working_directory (str) – location of code to run. By default
/paperspace
experiment_id (str) – Id of experiment to which job should be connected. If not provided there will be created new experiment for this job.
env_vars (dict) – key value collection of envs that are used in code
use_dockerfile (bool) – determines whether to build from Dockerfile (default false). Do not include a –container argument when using this flag.
is_preemptible (bool) – flag if we you want to use spot instance. By default False
project (str) – name of project that job is linked to.
started_by_user_id (str) – id of user that started job. By default it take user id from access token or api key.
rel_dockerfile_path (str) – relative location to your dockerfile. Default set to
./Dockerfile
registry_username (str) – username for custom docker registry
registry_password (str) – password for custom docker registry
cluster (str) – name of cluster that job should be run on.
cluster_id (str) – id of cluster that job should be run on. If you use one of recommended machine type cluster will be chosen so you do not need to provide it.
node_attrs (dict) –
workspace_file_name (str) –
registry_target (str) – custom docker registry
registry_target_username (str) – username for custom docker registry
registry_target_password (str) – password for custom docker registry
build_only (bool) – determines whether to only build and not run image
tags (list[str]) – List of tags
datasets (list[object]) – List of input/output datasets
- Returns
Job handle
- Return type
str
-
delete
(job_id)¶ Method to remove job.
1 2 3
job_client.delete( job_id='Your_job_id_here' )
- Parameters
job_id (str) – id of job that you want to remove
- Raises
exceptions.GradientSdkError
-
stop
(job_id)¶ Method to stop working job
1 2 3
job_client.stop( job_id='Your_job_id_here' )
- Parameters
job_id – id of job that we want to stop
- Raises
exceptions.GradientSdkError
-
list
(project_id=None, project=None, experiment_id=None, tags=None)¶ Method to list jobs.
To retrieve all user jobs:
1
jobs = job_client.list()
To list jobs from project:
1 2 3
job = job_client.list( project_id="Your_project_id_here", )
- Parameters
project_id (str) – id of project that you want to list jobs
project (str) – name of project that you want to list jobs
experiment_id (str) – id of experiment that you want to list jobs
tags (list[str]|tuple[str]) – tags to filter jobs with OR
- Returns
list of job models
- Return type
list[Job]
-
logs
(job_id, line=1, limit=10000)¶ Method to retrieve job logs.
1 2 3 4 5
job_logs = job_client.logs( job_id='Your_job_id_here', line=100, limit=100 )
- Parameters
job_id (str) – id of job that we want to retrieve logs
line (int) – from what line you want to retrieve logs. Default 0
limit (int) – how much lines you want to retrieve logs. Default 10000
- Returns
list of formatted logs lines
- Return type
list
-
yield_logs
(job_id, line=1, limit=10000)¶ Get log generator. Polls the API for new logs
1 2 3 4 5
job_logs_generator = job_client.yield_logs( job_id='Your_job_id_here', line=100, limit=100 )
- Parameters
job_id (str) –
line (int) – line number at which logs starts to display on screen
limit (int) – maximum lines displayed on screen, default set to 10 000
- Returns
generator yielding LogRow instances
- Return type
Iterator[models.LogRow]
-
artifacts_delete
(job_id, files=None)¶ Method to delete job artifact.
1 2 3 4
job_client.artifacts_delete( job_id='Your_job_id_here', files=files, )
- Parameters
job_id (str) – Id of job which artifact you want to delete
files (str) – if you wish to remove only few files from artifact pass string with names of this files separated by comma
,
- Raises
exceptions.GradientSdkError
-
artifacts_get
(job_id)¶ Method to retrieve federated access information for job artifacts.
1 2 3
artifacts = job_client.artifacts_get( job_id='your_job_id_here', )
- Parameters
job_id – Id of job from which you want to retrieve artifacts information about location
- Returns
Information about artifact place
- Return type
dict
-
artifacts_list
(job_id, files=None, size=False, links=True, start_after=None)¶ Method to retrieve all artifacts files.
1 2 3 4 5 6 7
artifacts = job_client.artifacts_list( job_id='your_job_id_here', files='your_files,here', size=False, links=True, start_after='key', )
- Parameters
job_id (str) – to limit artifact from this job.
files (str) – to limit result only to file names provided. You can use wildcard option
*
.size (bool) – flag to show file size. Default value is set to False.
links (bool) – flag to show file url. Default value is set to True.
- Params str start_after
key to list after
- Returns
list of files with description if specified from job artifacts.
- Return type
-
get_metrics
(job_id, start=None, end=None, interval='30s', built_in_metrics=None)¶ Get job metrics
- Parameters
job_id (str) – ID of a job
start (datetime.datetime|str) –
end (datetime.datetime|str) –
interval (str) –
built_in_metrics (list[str]) –
List of metrics to get if different than default Available builtin metrics: cpuPercentage, memoryUsage, gpuMemoryFree, gpuMemoryUsed, gpuPowerDraw,
gpuTemp, gpuUtilization, gpuMemoryUtilization
- Returns
Metrics of a job
- Return type
dict[str,dict[str,list[dict]]]
-
list_metrics
(job_id, start=None, end=None, interval='30s')¶ List job metrics
- Parameters
job_id (str) – ID of a job
start (datetime.datetime|str) –
end (datetime.datetime|str) –
interval (str) –
- Returns
Metrics of a job
- Return type
dict[str,dict[str,list[dict]]]
-
stream_metrics
(job_id, interval='30s', built_in_metrics=None)¶ Stream live job metrics
- Parameters
job_id (str) – ID of a job
interval (str) –
built_in_metrics (list[str]) –
List of metrics to get if different than default Available builtin metrics: cpuPercentage, memoryUsage, gpuMemoryFree, gpuMemoryUsed, gpuPowerDraw,
gpuTemp, gpuUtilization, gpuMemoryUtilization
- Returns
Generator object yielding live job metrics
- Return type
Iterable[dict]
-
gradient.api_sdk.clients.machine_types_client module¶
-
class
gradient.api_sdk.clients.machine_types_client.
MachineTypesClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.BaseClient
-
list
(cluster_id=None)¶
-
gradient.api_sdk.clients.machines_client module¶
-
class
gradient.api_sdk.clients.machines_client.
MachinesClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.TagsSupportMixin
,gradient.api_sdk.clients.base_client.BaseClient
-
entity
= 'machine'¶
-
create
(name, machine_type, region, size, billing_type, template_id, assign_public_ip=None, dynamic_public_ip=None, network_id=None, team_id=None, user_id=None, email=None, password=None, first_name=None, last_name=None, notification_email=None, script_id=None, tags=None)¶ Create new machine
- Parameters
name (str) – A memorable name for this machine [required]
machine_type (str) – Machine type [required]
region (str) – Name of the region [required]
size (str) – Storage size for the machine in GB [required]
billing_type (str) – Either ‘monthly’ or ‘hourly’ billing [required]
template_id (str) – Template id of the template to use for creating this machine [required]
assign_public_ip (bool) – Assign a new public ip address. Cannot be used with dynamic_public_ip
dynamic_public_ip (bool) – Temporarily assign a new public ip address on machine. Cannot be used with assign_public_ip
network_id (str) – If creating on a specific network, specify its id
team_id (str) – If creating the machine for a team, specify the team id
user_id (str) – If assigning to an existing user other than yourself, specify the user id (mutually exclusive with email, password, first_name, last_name)
email (str) – If creating a new user for this machine, specify their email address (mutually exclusive with user_id)
password (str) – If creating a new user, specify their password (mutually exclusive with user_id)
first_name (str) – If creating a new user, specify their first name (mutually exclusive with user_id)
last_name (str) – If creating a new user, specify their last name (mutually exclusive with user_id)
notification_email (str) – Send a notification to this email address when complete
script_id (str) – The script id of a script to be run on startup
tags (list[str]) – List of tags
- Returns
ID of created machine
- Return type
str
-
get
(id)¶ Get machine instance
- Parameters
id (str) – ID of a machine [required]
- Returns
Machine instance
- Return type
models.Machine
-
is_available
(machine_type, region)¶ Check if specified machine is available in certain region
- Parameters
machine_type (str) – Machine type [required]
region (str) – Name of the region [required]
- Returns
If specified machine is available in the region
- Return type
bool
-
restart
(id)¶ Restart machine
- Parameters
id (str) – ID of a machine [required]
-
start
(id)¶ Start machine
- Parameters
id (str) – ID of a machine [required]
-
stop
(id)¶ Stop machine
- Parameters
id (str) – ID of a machine [required]
-
update
(id, name=None, shutdown_timeout_in_hours=None, shutdown_timeout_forces=None, perform_auto_snapshot=None, auto_snapshot_frequency=None, auto_snapshot_save_count=None, dynamic_public_ip=None)¶ Update machine instance
- Parameters
id (str) – Id of the machine to update [required]
name (str) – New name for the machine
shutdown_timeout_in_hours (int) – Number of hours before machine is shutdown if no one is logged in via the Paperspace client
shutdown_timeout_forces (bool) – Force shutdown at shutdown timeout, even if there is a Paperspace client connection
perform_auto_snapshot (bool) – Perform auto snapshots
auto_snapshot_frequency (str) – One of ‘hour’, ‘day’, ‘week’, or None
auto_snapshot_save_count (int) – Number of snapshots to save
dynamic_public_ip (str) – If true, assigns a new public ip address on machine start and releases it from the account on machine stop
-
get_utilization
(id, billing_month)¶ - Parameters
id – ID of the machine
billing_month – Billing month in “YYYY-MM” format
- Returns
Machine utilization info
- Return type
models.MachineUtilization
-
delete
(machine_id, release_public_ip=False)¶ Destroy machine with given ID
- Parameters
machine_id (str) – ID of the machine
release_public_ip (bool) – If the assigned public IP should be released
-
wait_for_state
(machine_id, state, interval=5)¶ Wait for defined machine state
- Parameters
machine_id (str) – ID of the machine
state (str) – State of machine to wait for
interval (int) – interval between polls
-
list
(id=None, name=None, os=None, ram=None, cpus=None, gpu=None, storage_total=None, storage_used=None, usage_rate=None, shutdown_timeout_in_hours=None, perform_auto_snapshot=None, auto_snapshot_frequency=None, auto_snapshot_save_count=None, agent_type=None, created_timestamp=None, state=None, updates_pending=None, network_id=None, private_ip_address=None, public_ip_address=None, region=None, user_id=None, team_id=None, last_run_timestamp=None)¶ - Parameters
id (str) – Optional machine id to match on
name (str) – Filter by machine name
os (str) – Filter by os used
ram (int) – Filter by machine RAM (in bytes)
cpus (int) – Filter by CPU count
gpu (str) – Filter by GPU type
storage_total (str) – Filter by total storage
storage_used (str) – Filter by storage used
usage_rate (str) – Filter by usage rate
shutdown_timeout_in_hours (int) – Filter by shutdown timeout
perform_auto_snapshot (bool) – Filter by performAutoSnapshot flag
auto_snapshot_frequency (str) – Filter by autoSnapshotFrequency flag
auto_snapshot_save_count (int) – Filter by auto shapshots count
agent_type (str) – Filter by agent type
created_timestamp (datetime) – Filter by date created
state (str) – Filter by state
updates_pending (str) – Filter by updates pending
network_id (str) – Filter by network ID
private_ip_address (str) – Filter by private IP address
public_ip_address (str) – Filter by public IP address
region (str) – Filter by region. One of {CA, NY2, AMS1}
user_id (str) – Filter by user ID
team_id (str) – Filter by team ID
last_run_timestamp (str) – Filter by last run date
- Returns
List of machines
- Return type
list[models.Machine]
-
gradient.api_sdk.clients.model_client module¶
-
class
gradient.api_sdk.clients.model_client.
ModelsClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.TagsSupportMixin
,gradient.api_sdk.clients.base_client.BaseClient
-
entity
= 'mlModel'¶
-
list
(experiment_id=None, project_id=None, tags=None)¶ Get list of models
- Parameters
experiment_id (str) – Experiment ID to filter models
project_id (str) – Project ID to filter models
tags (list[str]|tuple[str]) – tags to filter models
- Returns
List of Model instances
- Return type
list[models.Model]
-
delete
(model_id)¶ Delete a model
- Parameters
model_id (str) – Model ID
-
upload
(path, name, model_type, model_summary=None, notes=None, tags=None, project_id=None, cluster_id=None)¶ Upload model
- Parameters
path (file) – path to Model
name (str) – Model name
model_type (str) – Model Type
model_summary (dict|None) – Dictionary describing model parameters like loss, accuracy, etc.
notes (str|None) – Optional model description
tags (list[str]) – List of tags
project_id (str|None) – ID of a project
cluster_id (str|None) – ID of a cluster
- Returns
ID of new model
- Return type
str
-
get
(model_id)¶ Get model instance
- Parameters
model_id (str) –
- Returns
Model instance
- Return type
models.Model
-
get_model_files
(model_id, links=False, size=False)¶ Get list of models
- Parameters
model_id (str) – Model ID
links (bool) – Get links to model files
size (bool) – Get sizes of each file in bytes
- Return type
list[models.ModelFile]
-
gradient.api_sdk.clients.notebook_client module¶
-
class
gradient.api_sdk.clients.notebook_client.
NotebooksClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.TagsSupportMixin
,gradient.api_sdk.clients.base_client.BaseClient
-
entity
= 'notebook'¶
-
create
(machine_type, project_id=None, cluster_id=None, container=None, container_id=None, name=None, registry_username=None, registry_password=None, command=None, container_user=None, shutdown_timeout=None, is_preemptible=None, is_public=None, workspace=None, workspace_ref=None, workspace_username=None, workspace_password=None, tags=None, environment=None)¶ Create new notebook
- Parameters
machine_type (str) –
container_id (int) –
project_id (str|int) –
cluster_id (str) –
container (str) –
name (str) –
registry_username (str) –
registry_password (str) –
command (str) –
container_user (str) –
shutdown_timeout (int) –
is_preemptible (bool) –
is_public (bool) –
tags (list[str]) – List of tags
workspace (str) – Project git repository url
workspace_ref (str) – Git commit hash, branch name or tag
workspace_username (str) – Project git repository username
workspace_password (str) – Project git repository password
environment (dict) – key value collection of envs that are used in notebook
- Returns
Notebook ID
- Rtype str
-
start
(id, machine_type, cluster_id=None, name=None, shutdown_timeout=None, is_preemptible=None, tags=None)¶ Start existing notebook :param str|int id: :param str machine_type: :param str cluster_id: :param str name: :param int shutdown_timeout: :param bool is_preemptible: :param list[str] tags: List of tags
- Returns
Notebook ID
- Rtype str
-
fork
(id, project_id, tags=None)¶ Fork an existing notebook :param str|int id: :param str project_id: :param list[str] tags: List of tags
- Returns
Notebook ID
- Rtype str
-
get
(id)¶ Get Notebook
- Parameters
id (str) – Notebook ID
- Return type
models.Notebook
-
delete
(id)¶ Delete existing notebook
- Parameters
id (str) – Notebook ID
-
list
(tags=None, limit=None, offset=None, get_meta=False)¶ Get list of Notebooks
- Return type
list[models.Notebook]
-
get_metrics
(notebook_id, start=None, end=None, interval='30s', built_in_metrics=None)¶ Get notebook metrics
- Parameters
notebook_id (str) – notebook ID
start (datetime.datetime|str) –
end (datetime.datetime|str) –
interval (str) –
built_in_metrics (list[str]) –
List of metrics to get if different than default Available builtin metrics: cpuPercentage, memoryUsage, gpuMemoryFree, gpuMemoryUsed, gpuPowerDraw,
gpuTemp, gpuUtilization, gpuMemoryUtilization
- Returns
Metrics of a notebook
- Return type
dict[str,dict[str,list[dict]]]
-
list_metrics
(notebook_id, start=None, end=None, interval='30s')¶ List notebook metrics
- Parameters
notebook_id (str) – notebook ID
start (datetime.datetime|str) –
end (datetime.datetime|str) –
interval (str) –
- Returns
Metrics of a notebook
- Return type
dict[str,dict[str,list[dict]]]
-
stream_metrics
(notebook_id, interval='30s', built_in_metrics=None)¶ Stream live notebook metrics
- Parameters
notebook_id (str) – notebook ID
interval (str) –
built_in_metrics (list[str]) –
List of metrics to get if different than default Available builtin metrics: cpuPercentage, memoryUsage, gpuMemoryFree, gpuMemoryUsed, gpuPowerDraw,
gpuTemp, gpuUtilization, gpuMemoryUtilization
- Returns
Generator object yielding live notebook metrics
- Return type
Iterable[dict]
-
stop
(id)¶ Stop existing notebook
- Parameters
id (str|int) – Notebook ID
-
artifacts_list
(notebook_id, files=None, size=False, links=True)¶ Method to retrieve all artifacts files.
1 2 3 4 5 6
artifacts = notebook_client.artifacts_list( notebook_id='your_notebook_id_here', files='your_files,here', size=False, links=True )
- Parameters
notebook_id (str) – to limit artifact from this notebook.
files (str) – to limit result only to file names provided. You can use wildcard option
*
.size (bool) – flag to show file size. Default value is set to False.
links (bool) – flag to show file url. Default value is set to True.
- Returns
list of files with description if specified from notebook artifacts.
- Return type
list[Artifact]
-
logs
(notebook_id, line=1, limit=10000)¶ Method to retrieve notebook logs.
1 2 3 4 5
notebook_logs = notebook_client.logs( notebook_id='Your_job_id_here', line=100, limit=100 )
- Parameters
notebook_id (str) – id of notebook that we want to retrieve logs
line (int) – from what line you want to retrieve logs. Default 0
limit (int) – how much lines you want to retrieve logs. Default 10000
- Returns
list of formatted logs lines
- Return type
list
-
yield_logs
(notebook_id, line=1, limit=10000)¶ Get log generator. Polls the API for new logs
1 2 3 4 5
notebook_logs_generator = notebook_client.yield_logs( notebook_id='Your_job_id_here', line=100, limit=100 )
- Parameters
notebook_id (str) –
line (int) – line number at which logs starts to display on screen
limit (int) – maximum lines displayed on screen, default set to 10 000
- Returns
generator yielding LogRow instances
- Return type
Iterator[models.LogRow]
-
gradient.api_sdk.clients.project_client module¶
-
class
gradient.api_sdk.clients.project_client.
ProjectsClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.TagsSupportMixin
,gradient.api_sdk.clients.base_client.BaseClient
-
entity
= 'project'¶
-
create
(name, repository_name=None, repository_url=None, tags=None)¶ Create new project
EXAMPLE:
gradient projects create --name new-project
EXAMPLE RETURN:
Project created with ID: <your-project-id>
in sdk:
from gradient.api_sdk.clients import ProjectsClient api_key = 'your-api-key' projects_client = ProjectsClient(api_key) new_project = projects_client.create('your-project-name') print(new_project)
- Parameters
name (str) – Name of new project [required]
repository_name (str) – Name of the repository
repository_url (str) – URL to the repository
tags (list[str]) – List of tags
- Returns
project ID
- Return type
str
-
list
(tags=None)¶ Get list of your projects
- Parameters
tags (list[str]|tuple[str]) – tags to filter with OR
- Returns
list of projects
- Return type
list[models.Project]
-
delete
(project_id)¶
-
get
(project_id)¶
-
gradient.api_sdk.clients.sdk_client module¶
gradient.api_sdk.clients.secret_client module¶
-
class
gradient.api_sdk.clients.secret_client.
SecretsClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.BaseClient
-
list
(entity, entity_id)¶ List secrets by entity type and ID.
- Parameters
entity (str) – entity type (ex: team, cluster, project)
entity_id (str) – entity ID
- Returns
list of secrets
- Return type
list[models.Secret]
-
set
(entity, entity_id, name, value)¶ Set entity secret.
- Parameters
entity (str) – entity type (ex: team, cluster, project)
entity_id (str) – entity ID
name (str) – secret name
value (str) – secret value
- Returns
- Return type
None
-
delete
(entity, entity_id, name)¶ Delete entity secret.
- Parameters
entity (str) – entity type (ex: team, cluster, project)
entity_id (str) – entity ID
name (str) – secret name
- Returns
- Return type
None
-
ephemeral
(key, value, expires_in)¶ Create ephemeral secret.
- Parameters
key (str) – secret key
value (str) – secret value
expires_in (str) – seconds secrets expire in
- Returns
- Return type
str
-
gradient.api_sdk.clients.storage_provider_client module¶
-
class
gradient.api_sdk.clients.storage_provider_client.
StorageProvidersClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.BaseClient
-
list
(limit=20, offset=0)¶ Get list of your storage providers
- Parameters
limit (int) – limit of the number results
offset (int) – skip number of results
- Returns
list of storage providers
- Return type
list[models.StorageProvider]
-
delete
(storage_provider_id)¶ Delete a storage provider
- Parameters
storage_provider_id (int) – ID [required]
- Returns
- Return type
None
-
get
(storage_provider_id)¶ Delete a storage provider
- Parameters
storage_provider_id (int) – ID [required]
- Returns
storage provider
- Return type
models.StorageProvider
-
create_s3
(name, bucket, access_key, secret_access_key, endpoint=None, region=None, signature_version=None)¶ Create a new S3 storage provider
- Parameters
name (str) – Name of new AWS storage provider [required]
bucket (str) – S3 bucket [required]
access_key (str) – S3 access key ID [required]
secret_access_key (str) – S3 access key ID [required]
endpoint (str) – S3 endpoint URL
region (str) – S3 region
signature_version (str) – S3 signature version (ex: v4)
- Returns
storage provider ID
- Return type
str
-
update_s3
(storage_provider_id, name=None, bucket=None, access_key=None, secret_access_key=None, endpoint=None, region=None, signature_version=None)¶ Update an existing S3 storage provider
- Parameters
storage_provider_id (str) – Storage provider ID
name (str) – Storage provider name
bucket (str) – S3 bucket
access_key (str) – S3 access key ID
secret_access_key (str) – S3 access key ID
endpoint (str) – S3 endpoint URL
region (str) – S3 region
signature_version (str) – S3 signature version (ex: v4)
- Returns
- Return type
None
-
gradient.api_sdk.clients.tensorboards_client module¶
Tensorboard logic related client handler.
Remember that in code snippets all highlighted lines are required other lines are optional.
-
class
gradient.api_sdk.clients.tensorboards_client.
TensorboardClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.BaseClient
Client to handle tensorboard related actions.
How to create instance of tensorboard client:
1 2 3 4 5
from gradient import TensorboardClient tb_client = TensorboardClient( api_key='your_api_key_here' )
-
create
(image=None, username=None, password=None, instance_type=None, instance_size=None, instances_count=None, experiments=None)¶ Method to create tensorboard in paperspace gradient.
Example create tensorboard:
1 2 3 4 5 6 7 8 9
tb_id = tb_client.create( experiments=['some_experiment_id'], image='tensorflow/tensorflow:latest-py3', username='your_username', password='your_password', instance_type='cpu', instance_size='small', instance_count=1 )
- Parameters
image (str) – your tensorboard will run with this image. By default it will be run with
tensorflow/tensorflow:latest-py3
username (str) – if you wish to limit access to your tensorboard with base auth then provide username
password (str) – if you wish to limit access to your tensorboard with base auth then provide password
instance_type (str) –
type of instance on which you want to run tensorboard. Available choices:
cpu gpu
By default we use
cpu
instance type.instance_size (str) –
size of instance on which you want to run tensorboard. Available choices:
small medium large
By default we use
small
instance size.instances_count (int) – on how many machines you want to run tensorboard. By default
1
is used.experiments (list) – list of experiments that you wish to add to tensorboard. To create tensorboard you need to provide at least one experiment id. This field is required.
- Returns
Return tensorboard id
- Return type
str
- Raises
ResourceFetchingError: When there is problem with response from API
-
get
(id)¶ Method to get tensorboard details.
Example get tensorboard details:
1 2 3
tb = tb_client.get( id='your_tb_id' )
- Parameters
id (str) – Tensorboard id of which you want to get details
- Returns
Tensorbord object if found
- Return type
None|Tensorboard
- Raises
ResourceFetchingError: When there is problem with response from API
-
list
()¶ Method to list your active tensorboards.
Example usage:
1
tb_list = tb_client.list()
- Returns
list of active tensorboards
- Return type
list[models.Tensorboard]
- Raises
ResourceFetchingError: When there is problem with response from API
-
add_experiments
(id, added_experiments)¶ Method to add experiments to existing tensorboard.
Example usage:
1 2 3 4
tb = tb_client.add_experiments( id='your_tb_id', added_experiments=['new_experiment_id', 'next_new_experiment_id'] )
- Parameters
id (str) – tensorboard id to which you want to add experiments
added_experiments (list) – list of experiment ids which you want to add to tensroboard
- Returns
updated tensorboard
- Return type
- Raises
ResourceFetchingError: When there is problem with response from API
-
remove_experiments
(id, removed_experiments)¶ Method to remove experiments from existing tensorboard.
Example usage:
1 2 3 4
tb = tb_client.remove_experiments( id='your_tb_id', removed_experiments=['experiment_id', 'next_experiment_id'] )
- Parameters
id (str) – tensorboard id from which you want to remove experiments
removed_experiments (list) – list of experiment ids which you want to remove from tensroboard
- Returns
updated tensorboard
- Return type
- Raises
ResourceFetchingError: When there is problem with response from API
-
delete
(id)¶ Method to delete tensorboard.
Example usage:
1 2 3
tb_client.delete( id='your_tb_id' )
- Parameters
id (str) – Tensoboard id which you want to delete
-
gradient.api_sdk.clients.workflow_client module¶
-
class
gradient.api_sdk.clients.workflow_client.
WorkflowsClient
(api_key, ps_client_name=None, logger=<gradient.api_sdk.logger.MuteLogger object>)¶ Bases:
gradient.api_sdk.clients.base_client.BaseClient
-
create
(name, project_id)¶ Create workflow with spec
- Parameters
name (str) – workflow name
project_id (str) – project id
- Returns
workflow create response
- Return type
list[models.Workflow]
-
run_workflow
(spec, inputs, workflow_id, cluster_id)¶ Create workflow with spec
- Parameters
spec (obj) – workflow spec
inputs (obj) – workflow inputs
workflow_id (str) – workflow id
cluster_id (str) – cluster id
- Returns
workflow create response
- Return type
list[models.Workflow]
-
list
(project_id)¶ List workflows by project
- Parameters
project_id (str) – project ID
- Returns
list of workflows
- Return type
list[models.Workflow]
-
get
(workflow_id)¶ Get a Workflow
- Parameters
workflow_id (str) – Workflow ID [required]
- Returns
workflow
- Return type
models.Workflow
-
list_runs
(workflow_id)¶ List workflows runs by workflow id
- Parameters
workflow_id (str) – workflow ID
- Returns
list of workflow runs
-
get_run
(workflow_id, run)¶ List workflows runs by workflow id
- Parameters
workflow_id (str) – workflow ID
run (str) – run count
- Returns
list of workflow runs
-
yield_logs
(job_id, line=1, limit=10000)¶ Get log generator. Polls the API for new logs
1 2 3 4 5
job_logs_generator = job_client.yield_logs( job_id='Your_job_id_here', line=100, limit=100 )
- Parameters
job_id (str) –
line (int) – line number at which logs starts to display on screen
limit (int) – maximum lines displayed on screen, default set to 10 000
- Returns
generator yielding LogRow instances
- Return type
Iterator[models.LogRow]
-
logs
(job_id, line=1, limit=10000)¶ Get log generator. Polls the API for new logs
1 2 3 4 5
job_logs_generator = job_client.yield_logs( job_id='Your_job_id_here', line=100, limit=100 )
- Parameters
job_id (str) –
line (int) – line number at which logs starts to display on screen
limit (int) – maximum lines displayed on screen, default set to 10 000
- Returns
generator yielding LogRow instances
- Return type
Iterator[models.LogRow]
-