Katib API
Field | Type | Label | Description |
name | string |
|
|
value | string |
|
Field | Type | Label | Description |
algorithm_name | string |
|
|
algorithm_setting | AlgorithmSetting | repeated |
|
early_stopping_spec | EarlyStoppingSpec |
|
Field | Type | Label | Description |
experiment_name | string |
|
Field | Type | Label | Description |
trial_name | string |
|
TODO
Field | Type | Label | Description |
name | string | Name of Experiment. This is unique in DB. |
|
spec | ExperimentSpec |
|
|
status | ExperimentStatus |
|
Spec of a Experiment. Experiment represents a single optimization run over a feasible space.
Each Experiment contains a configuration describing the feasible space, as well as a set of Trials.
It is assumed that objective function f(x) does not change in the course of a Experiment.
Field | Type | Label | Description |
parameter_specs | ExperimentSpec.ParameterSpecs |
|
|
objective | ObjectiveSpec |
|
|
algorithm | AlgorithmSpec |
|
|
trial_template | string |
|
|
metrics_collector_spec | string |
|
|
parallel_trial_count | int32 |
|
|
max_trial_count | int32 |
|
|
nas_config | NasConfig |
|
List of ParameterSpec
Field | Type | Label | Description |
parameters | ParameterSpec | repeated |
|
Field | Type | Label | Description |
start_time | string | RFC3339 format |
|
completion_time | string | RFC3339 format |
|
condition | ExperimentStatus.ExperimentConditionType |
|
Field | Type | Label | Description |
experiment_name | string |
|
|
status | ExperimentStatus |
|
Feasible space for optimization.
Int and Double type use Max/Min.
Discrete and Categorical type use List.
Field | Type | Label | Description |
max | string | Max Value |
|
min | string | Minimum Value |
|
list | string | repeated | List of Values. |
step | string | Step for double or int parameter |
Field | Type | Label | Description |
extra_algorithm_settings | AlgorithmSetting | repeated |
|
Field | Type | Label | Description |
experiment_name | string |
|
Field | Type | Label | Description |
experiment_summaries | ExperimentSummary | repeated |
|
Field | Type | Label | Description |
experiment | Experiment |
|
Field | Type | Label | Description |
experiment_name | string |
|
Field | Type | Label | Description |
observation_log | ObservationLog |
|
Field | Type | Label | Description |
trial_name | string |
|
|
metric_name | string |
|
|
start_time | string | The start of the time range. RFC3339 format |
|
end_time | string | The end of the time range. RFC3339 format |
Field | Type | Label | Description |
trials | Trial | repeated |
|
Field | Type | Label | Description |
experiment_name | string |
|
|
algorithm_name | string |
|
|
request_number | int32 | The number of Suggestion you request at one time. When you set 3 to request_number, you can get three Suggestions at one time. |
Field | Type | Label | Description |
trials | Trial | repeated |
|
Field | Type | Label | Description |
experiment_name | string |
|
|
filter | string |
|
Field | Type | Label | Description |
trial | Trial |
|
Field | Type | Label | Description |
trial_name | string |
|
GraphConfig contains a config of DAG
Field | Type | Label | Description |
num_layers | int32 | Number of layers |
|
input_sizes | int32 | repeated | Dimensions of input size |
output_sizes | int32 | repeated | Dimensions of output size |
Field | Type | Label | Description |
name | string |
|
|
value | string |
|
Field | Type | Label | Description |
time_stamp | string | RFC3339 format |
|
metric | Metric |
|
NasConfig contains a config of NAS job
Field | Type | Label | Description |
graph_config | GraphConfig | Config of DAG |
|
operations | NasConfig.Operations | List of Operation |
Field | Type | Label | Description |
operation | Operation | repeated |
|
Field | Type | Label | Description |
type | ObjectiveType |
|
|
goal | float |
|
|
objective_metric_name | string |
|
|
additional_metric_names | string | repeated | This can be empty if we only care about the objective metric. |
Field | Type | Label | Description |
metrics | Metric | repeated |
|
Field | Type | Label | Description |
metric_logs | MetricLog | repeated |
|
Config for operations in DAG
Field | Type | Label | Description |
operation_type | string | Type of operation in DAG |
|
parameter_specs | Operation.ParameterSpecs |
|
List of ParameterSpec
Field | Type | Label | Description |
parameters | ParameterSpec | repeated |
|
Field | Type | Label | Description |
name | string |
|
|
value | string |
|
Config for a Hyper parameter.
Katib will create each Hyper parameter from this config.
Field | Type | Label | Description |
name | string | Name of the parameter. |
|
parameter_type | ParameterType | Type of the parameter. |
|
feasible_space | FeasibleSpace | FeasibleSpace for the parameter. |
Field | Type | Label | Description |
can_register | bool |
|
Field | Type | Label | Description |
experiment | Experiment |
|
Field | Type | Label | Description |
trial | Trial |
|
Field | Type | Label | Description |
trial_name | string |
|
|
observation_log | ObservationLog |
|
Field | Type | Label | Description |
name | string |
|
|
spec | TrialSpec |
|
|
status | TrialStatus |
|
Field | Type | Label | Description |
experiment_name | string |
|
|
objective | ObjectiveSpec |
|
|
parameter_assignments | TrialSpec.ParameterAssignments |
|
|
run_spec | string |
|
|
metrics_collector_spec | string |
|
List of ParameterAssignment
Field | Type | Label | Description |
assignments | ParameterAssignment | repeated |
|
Field | Type | Label | Description |
start_time | string | RFC3339 format |
|
completion_time | string | RFC3339 format |
|
condition | TrialStatus.TrialConditionType |
|
|
observation | Observation | The best observation in logs. |
Field | Type | Label | Description |
experiment_name | string |
|
|
extra_algorithm_settings | AlgorithmSetting | repeated |
|
Field | Type | Label | Description |
experiment_name | string |
|
|
new_status | ExperimentStatus |
|
Field | Type | Label | Description |
trial_name | string |
|
|
new_status | TrialStatus |
|
Return INVALID_ARGUMENT Error if Algorithm Settings are not Valid
Field | Type | Label | Description |
experiment_spec | ExperimentSpec |
|
|
algorithm_name | string |
|
Name | Number | Description |
CREATED | 0 | |
RUNNING | 1 | |
RESTARTING | 2 | |
SUCCEEDED | 3 | |
FAILED | 4 | |
UNKNOWN | 5 |
Direction of optimization. Minimize or Maximize.
Name | Number | Description |
UNKNOWN | 0 | Undefined type and not used. |
MINIMIZE | 1 | Minimize |
MAXIMIZE | 2 | Maximize |
Types of value for HyperParameter.
Name | Number | Description |
UNKNOWN_TYPE | 0 | Undefined type and not used. |
DOUBLE | 1 | Double float type. Use "Max/Min". |
INT | 2 | Int type. Use "Max/Min". |
DISCRETE | 3 | Discrete number type. Use "List" as float. |
CATEGORICAL | 4 | Categorical type. Use "List" as string. |
Name | Number | Description |
CREATED | 0 | |
RUNNING | 1 | |
SUCCEEDED | 2 | |
KILLED | 3 | |
FAILED | 4 | |
UNKNOWN | 5 |
TODO
Method Name | Request Type | Response Type | Description |
Service for Main API for Katib
For each RPC service, we define mapping to HTTP REST API method.
The mapping includes the URL path, query parameters and request body.
https://cloud.google.com/service-infrastructure/docs/service-management/reference/rpc/google.api#http
Method Name | Request Type | Response Type | Description |
RegisterExperiment | RegisterExperimentRequest | RegisterExperimentReply | Register a Experiment to DB. |
PreCheckRegisterExperiment | RegisterExperimentRequest | PreCheckRegisterExperimentReply | PreCheck to register a Experiment to DB. |
DeleteExperiment | DeleteExperimentRequest | DeleteExperimentReply | Delete a Experiment from DB by name. |
GetExperiment | GetExperimentRequest | GetExperimentReply | Get a Experiment from DB by name. |
GetExperimentList | GetExperimentListRequest | GetExperimentListReply | Get a summary list of Experiment from DB. The summary includes name and condition. |
UpdateExperimentStatus | UpdateExperimentStatusRequest | UpdateExperimentStatusReply | Update Status of a experiment. |
UpdateAlgorithmExtraSettings | UpdateAlgorithmExtraSettingsRequest | UpdateAlgorithmExtraSettingsReply | Update AlgorithmExtraSettings. The ExtraSetting is created if it does not exist, otherwise it is overwrited. |
GetAlgorithmExtraSettings | GetAlgorithmExtraSettingsRequest | GetAlgorithmExtraSettingsReply | Get all AlgorithmExtraSettings. |
RegisterTrial | RegisterTrialRequest | RegisterTrialReply | Register a Trial to DB. ID will be filled by manager automatically. |
DeleteTrial | DeleteTrialRequest | DeleteTrialReply | Delete a Trial from DB by ID. |
GetTrialList | GetTrialListRequest | GetTrialListReply | Get a list of Trial from DB by name of a Experiment. |
GetTrial | GetTrialRequest | GetTrialReply | Get a Trial from DB by ID of Trial. |
UpdateTrialStatus | UpdateTrialStatusRequest | UpdateTrialStatusReply | Update Status of a trial. |
ReportObservationLog | ReportObservationLogRequest | ReportObservationLogReply | Report a log of Observations for a Trial. The log consists of timestamp and value of metric. Katib store every log of metrics. You can see accuracy curve or other metric logs on UI. |
GetObservationLog | GetObservationLogRequest | GetObservationLogReply | Get all log of Observations for a Trial. |
GetSuggestions | GetSuggestionsRequest | GetSuggestionsReply | Get Suggestions from a Suggestion service. |
ValidateAlgorithmSettings | ValidateAlgorithmSettingsRequest | ValidateAlgorithmSettingsReply | Validate AlgorithmSettings in an Experiment. Suggestion service should return INVALID_ARGUMENT Error when the parameter is invalid |
Method Name | Request Type | Response Type | Description |
GetSuggestions | GetSuggestionsRequest | GetSuggestionsReply | |
ValidateAlgorithmSettings | ValidateAlgorithmSettingsRequest | ValidateAlgorithmSettingsReply |
.proto Type | Notes | C++ Type | Java Type | Python Type |
double | double | double | float | |
float | float | float | float | |
int32 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. | int32 | int | int |
int64 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. | int64 | long | int/long |
uint32 | Uses variable-length encoding. | uint32 | int | int/long |
uint64 | Uses variable-length encoding. | uint64 | long | int/long |
sint32 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. | int32 | int | int |
sint64 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. | int64 | long | int/long |
fixed32 | Always four bytes. More efficient than uint32 if values are often greater than 2^28. | uint32 | int | int |
fixed64 | Always eight bytes. More efficient than uint64 if values are often greater than 2^56. | uint64 | long | int/long |
sfixed32 | Always four bytes. | int32 | int | int |
sfixed64 | Always eight bytes. | int64 | long | int/long |
bool | bool | boolean | boolean | |
string | A string must always contain UTF-8 encoded or 7-bit ASCII text. | string | String | str/unicode |
bytes | May contain any arbitrary sequence of bytes. | string | ByteString | str |