Protocol Documentation

Table of Contents

api.proto

Top

Katib API

AlgorithmSetting

FieldTypeLabelDescription
name string

value string

AlgorithmSpec

FieldTypeLabelDescription
algorithm_name string

algorithm_setting AlgorithmSetting repeated

early_stopping_spec EarlyStoppingSpec

DeleteExperimentReply

DeleteExperimentRequest

FieldTypeLabelDescription
experiment_name string

DeleteTrialReply

DeleteTrialRequest

FieldTypeLabelDescription
trial_name string

EarlyStoppingSpec

TODO

Experiment

FieldTypeLabelDescription
name string

Name of Experiment. This is unique in DB.

spec ExperimentSpec

status ExperimentStatus

ExperimentSpec

Spec of a Experiment. Experiment represents a single optimization run over a feasible space.

Each Experiment contains a configuration describing the feasible space, as well as a set of Trials.

It is assumed that objective function f(x) does not change in the course of a Experiment.

FieldTypeLabelDescription
parameter_specs ExperimentSpec.ParameterSpecs

objective ObjectiveSpec

algorithm AlgorithmSpec

trial_template string

metrics_collector_spec string

parallel_trial_count int32

max_trial_count int32

nas_config NasConfig

ExperimentSpec.ParameterSpecs

List of ParameterSpec

FieldTypeLabelDescription
parameters ParameterSpec repeated

ExperimentStatus

FieldTypeLabelDescription
start_time string

RFC3339 format

completion_time string

RFC3339 format

condition ExperimentStatus.ExperimentConditionType

ExperimentSummary

FieldTypeLabelDescription
experiment_name string

status ExperimentStatus

FeasibleSpace

Feasible space for optimization.

Int and Double type use Max/Min.

Discrete and Categorical type use List.

FieldTypeLabelDescription
max string

Max Value

min string

Minimum Value

list string repeated

List of Values.

step string

Step for double or int parameter

GetAlgorithmExtraSettingsReply

FieldTypeLabelDescription
extra_algorithm_settings AlgorithmSetting repeated

GetAlgorithmExtraSettingsRequest

FieldTypeLabelDescription
experiment_name string

GetExperimentListReply

FieldTypeLabelDescription
experiment_summaries ExperimentSummary repeated

GetExperimentListRequest

GetExperimentReply

FieldTypeLabelDescription
experiment Experiment

GetExperimentRequest

FieldTypeLabelDescription
experiment_name string

GetObservationLogReply

FieldTypeLabelDescription
observation_log ObservationLog

GetObservationLogRequest

FieldTypeLabelDescription
trial_name string

metric_name string

start_time string

The start of the time range. RFC3339 format

end_time string

The end of the time range. RFC3339 format

GetSuggestionsReply

FieldTypeLabelDescription
trials Trial repeated

GetSuggestionsRequest

FieldTypeLabelDescription
experiment_name string

algorithm_name string

request_number int32

The number of Suggestion you request at one time. When you set 3 to request_number, you can get three Suggestions at one time.

GetTrialListReply

FieldTypeLabelDescription
trials Trial repeated

GetTrialListRequest

FieldTypeLabelDescription
experiment_name string

filter string

GetTrialReply

FieldTypeLabelDescription
trial Trial

GetTrialRequest

FieldTypeLabelDescription
trial_name string

GraphConfig

GraphConfig contains a config of DAG

FieldTypeLabelDescription
num_layers int32

Number of layers

input_sizes int32 repeated

Dimensions of input size

output_sizes int32 repeated

Dimensions of output size

Metric

FieldTypeLabelDescription
name string

value string

MetricLog

FieldTypeLabelDescription
time_stamp string

RFC3339 format

metric Metric

NasConfig

NasConfig contains a config of NAS job

FieldTypeLabelDescription
graph_config GraphConfig

Config of DAG

operations NasConfig.Operations

List of Operation

NasConfig.Operations

FieldTypeLabelDescription
operation Operation repeated

ObjectiveSpec

FieldTypeLabelDescription
type ObjectiveType

goal float

objective_metric_name string

additional_metric_names string repeated

This can be empty if we only care about the objective metric.

Observation

FieldTypeLabelDescription
metrics Metric repeated

ObservationLog

FieldTypeLabelDescription
metric_logs MetricLog repeated

Operation

Config for operations in DAG

FieldTypeLabelDescription
operation_type string

Type of operation in DAG

parameter_specs Operation.ParameterSpecs

Operation.ParameterSpecs

List of ParameterSpec

FieldTypeLabelDescription
parameters ParameterSpec repeated

ParameterAssignment

FieldTypeLabelDescription
name string

value string

ParameterSpec

Config for a Hyper parameter.

Katib will create each Hyper parameter from this config.

FieldTypeLabelDescription
name string

Name of the parameter.

parameter_type ParameterType

Type of the parameter.

feasible_space FeasibleSpace

FeasibleSpace for the parameter.

PreCheckRegisterExperimentReply

FieldTypeLabelDescription
can_register bool

RegisterExperimentReply

RegisterExperimentRequest

FieldTypeLabelDescription
experiment Experiment

RegisterTrialReply

RegisterTrialRequest

FieldTypeLabelDescription
trial Trial

ReportObservationLogReply

ReportObservationLogRequest

FieldTypeLabelDescription
trial_name string

observation_log ObservationLog

Trial

FieldTypeLabelDescription
name string

spec TrialSpec

status TrialStatus

TrialSpec

FieldTypeLabelDescription
experiment_name string

objective ObjectiveSpec

parameter_assignments TrialSpec.ParameterAssignments

run_spec string

metrics_collector_spec string

TrialSpec.ParameterAssignments

List of ParameterAssignment

FieldTypeLabelDescription
assignments ParameterAssignment repeated

TrialStatus

FieldTypeLabelDescription
start_time string

RFC3339 format

completion_time string

RFC3339 format

condition TrialStatus.TrialConditionType

observation Observation

The best observation in logs.

UpdateAlgorithmExtraSettingsReply

UpdateAlgorithmExtraSettingsRequest

FieldTypeLabelDescription
experiment_name string

extra_algorithm_settings AlgorithmSetting repeated

UpdateExperimentStatusReply

UpdateExperimentStatusRequest

FieldTypeLabelDescription
experiment_name string

new_status ExperimentStatus

UpdateTrialStatusReply

UpdateTrialStatusRequest

FieldTypeLabelDescription
trial_name string

new_status TrialStatus

ValidateAlgorithmSettingsReply

Return INVALID_ARGUMENT Error if Algorithm Settings are not Valid

ValidateAlgorithmSettingsRequest

FieldTypeLabelDescription
experiment_spec ExperimentSpec

algorithm_name string

ExperimentStatus.ExperimentConditionType

NameNumberDescription
CREATED 0

RUNNING 1

RESTARTING 2

SUCCEEDED 3

FAILED 4

UNKNOWN 5

ObjectiveType

Direction of optimization. Minimize or Maximize.

NameNumberDescription
UNKNOWN 0

Undefined type and not used.

MINIMIZE 1

Minimize

MAXIMIZE 2

Maximize

ParameterType

Types of value for HyperParameter.

NameNumberDescription
UNKNOWN_TYPE 0

Undefined type and not used.

DOUBLE 1

Double float type. Use "Max/Min".

INT 2

Int type. Use "Max/Min".

DISCRETE 3

Discrete number type. Use "List" as float.

CATEGORICAL 4

Categorical type. Use "List" as string.

TrialStatus.TrialConditionType

NameNumberDescription
CREATED 0

RUNNING 1

SUCCEEDED 2

KILLED 3

FAILED 4

UNKNOWN 5

EarlyStopping

TODO

Method NameRequest TypeResponse TypeDescription

Manager

Service for Main API for Katib

For each RPC service, we define mapping to HTTP REST API method.

The mapping includes the URL path, query parameters and request body.

https://cloud.google.com/service-infrastructure/docs/service-management/reference/rpc/google.api#http

Method NameRequest TypeResponse TypeDescription
RegisterExperiment RegisterExperimentRequest RegisterExperimentReply

Register a Experiment to DB.

PreCheckRegisterExperiment RegisterExperimentRequest PreCheckRegisterExperimentReply

PreCheck to register a Experiment to DB.

DeleteExperiment DeleteExperimentRequest DeleteExperimentReply

Delete a Experiment from DB by name.

GetExperiment GetExperimentRequest GetExperimentReply

Get a Experiment from DB by name.

GetExperimentList GetExperimentListRequest GetExperimentListReply

Get a summary list of Experiment from DB. The summary includes name and condition.

UpdateExperimentStatus UpdateExperimentStatusRequest UpdateExperimentStatusReply

Update Status of a experiment.

UpdateAlgorithmExtraSettings UpdateAlgorithmExtraSettingsRequest UpdateAlgorithmExtraSettingsReply

Update AlgorithmExtraSettings. The ExtraSetting is created if it does not exist, otherwise it is overwrited.

GetAlgorithmExtraSettings GetAlgorithmExtraSettingsRequest GetAlgorithmExtraSettingsReply

Get all AlgorithmExtraSettings.

RegisterTrial RegisterTrialRequest RegisterTrialReply

Register a Trial to DB. ID will be filled by manager automatically.

DeleteTrial DeleteTrialRequest DeleteTrialReply

Delete a Trial from DB by ID.

GetTrialList GetTrialListRequest GetTrialListReply

Get a list of Trial from DB by name of a Experiment.

GetTrial GetTrialRequest GetTrialReply

Get a Trial from DB by ID of Trial.

UpdateTrialStatus UpdateTrialStatusRequest UpdateTrialStatusReply

Update Status of a trial.

ReportObservationLog ReportObservationLogRequest ReportObservationLogReply

Report a log of Observations for a Trial. The log consists of timestamp and value of metric. Katib store every log of metrics. You can see accuracy curve or other metric logs on UI.

GetObservationLog GetObservationLogRequest GetObservationLogReply

Get all log of Observations for a Trial.

GetSuggestions GetSuggestionsRequest GetSuggestionsReply

Get Suggestions from a Suggestion service.

ValidateAlgorithmSettings ValidateAlgorithmSettingsRequest ValidateAlgorithmSettingsReply

Validate AlgorithmSettings in an Experiment. Suggestion service should return INVALID_ARGUMENT Error when the parameter is invalid

Suggestion

Method NameRequest TypeResponse TypeDescription
GetSuggestions GetSuggestionsRequest GetSuggestionsReply

ValidateAlgorithmSettings ValidateAlgorithmSettingsRequest ValidateAlgorithmSettingsReply

Scalar Value Types

.proto TypeNotesC++ TypeJava TypePython Type
double double double float
float float float float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long
uint32 Uses variable-length encoding. uint32 int int/long
uint64 Uses variable-length encoding. uint64 long int/long
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long
sfixed32 Always four bytes. int32 int int
sfixed64 Always eight bytes. int64 long int/long
bool bool boolean boolean
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode
bytes May contain any arbitrary sequence of bytes. string ByteString str