Properties - modules for verbosity, workflows and their properties. This contains structures common to all workflows
Copyright (C) 2017-2018, Colin P Stark and Gavin J Stark. All rights reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * * @file properties.ml * @brief Workflow and properties libraries *
Verbosity is a basic part of the properties of a workflow. The verbosity can be set to one of many different levels, and the properties for a workflow can each have its own verbosity. Functions are provided to permit the execution of code dependent on the verbosity level.
val
pv_level : t_verbosity
‑> int
pv_level verbosity
converts the
verbosity to an integer level.
This function is for internal use - it is best to use the pv_verbose/pv_debug etc functions to control execution of code dependent on the verbosity level.
This function is used internally to grade verbosity, and hence to select the 'more verbose level' verbosity from a pair of verbositys.
val
pv_of_int : int ‑> t_verbosity
pv_of_int level
converts an integer
level to a verbosity
This function is used by argument parsing to convert a command line verbosity to a level for execution of the code.
This is the inverse function to pv_level
val pv_str
: t_verbosity
‑> string
pv_str verbosity
produces a string
corresponding to the verbosity level
This function is used to show how verbose a workflow is when it starts
val
pv_verbosity : ?verbose:bool ‑>
?debug:bool ‑> ?noisy:bool
‑> t_verbosity ‑> t_verbosity
pv_verbosity
?verbose ?debug ?noisy
verbosity - upgrade verbosity level
The verbosity level of a Workflow is the larger of that specified by its properties and that for the whole execution; this function permits 'upgrading' the verbosity based on the three properties 'verbose', 'noisy' and 'debug'.
val pv_if :
t_verbosity ‑> t_verbosity ‑> (unit ‑>
unit) ‑> unit
pv_if verbosity level f
- if verbosity
exceeds level then invoke
f ()
This is an internal function, for use in pv_info, pv_verbose, etc.
It invokes the function f with a unit argument if the verbosity at least matches level
val pv_info
: t_verbosity
‑> (unit ‑> unit) ‑>
unit
pv_info pv
Predicate function used with
fun _ -> ...
to invoke the function only if the verbosity level is at least PV_Info. Use, for example, as
pv_info verbosity (fun _ -> Printf.printf "Verbosity is at least info")
.
val
pv_verbose : t_verbosity
‑> (unit ‑> unit) ‑>
unit
pv_verbose pv
Predicate function used with
fun _ -> ...
to invoke the function only if the verbosity level is at least PV_Verbose. Use, for example, as
pv_verbose verbosity (fun _ -> Printf.printf "Verbosity is at least verbose")
.
val
pv_noisy : t_verbosity
‑> (unit ‑> unit) ‑>
unit
pv_noisy pv
Predicate function used with
fun _ -> ...
to invoke the function only if the verbosity level is at least PV_Noisy. Use, for example, as
pv_noisy verbosity (fun _ -> Printf.printf "Verbosity is at least noisy")
.
val
pv_debug : t_verbosity
‑> (unit ‑> unit) ‑>
unit
pv_debug pv
Predicate function used with
fun _ -> ...
to invoke the function only if the verbosity level is at least PV_Debug. Use, for example, as
pv_debug verbosity (fun _ -> Printf.printf "Verbosity is at least debug")
.
module
Properties : sig ... end
module
Workflow : sig ... end
type
t_cmdline_overrides
=
{
verbosity : t_verbosity; |
do_reload_state : bool option; |
do_geodata : bool option; |
do_preprocess : bool option; |
do_condition : bool option; |
do_trace : bool option; |
do_analysis : bool option; |
do_mapping : bool option; |
do_plot : bool option; |
do_save_state : bool option; |
do_export : bool option; |
}
t_cmdline_overrides
Structure containing command line settings, which can override the properties from the JSON files.
If the options are 'None' then there is no override; else it is 'Some <value>', and the value should override that from the JSON properties.
This structure is filled out in streamlines.ml, and used in read_state
type
t_props_state
=
{
verbosity : t_verbosity; |
verbose : bool; |
debug : bool; |
noisy : bool; |
cl_platform : int; |
cl_device : int; |
gpu_memory_limit_pc : float; |
array_order : string; |
do_geodata : bool; |
do_preprocess : bool; |
do_condition : bool; |
do_trace : bool; |
do_postprocess : bool; |
do_analysis : bool; |
do_mapping : bool; |
do_plot : bool; |
do_save_state : bool; |
do_export : bool; |
do_rw_savez : bool; |
do_rw_hdf5 : bool; |
do_reload_state : bool; |
}
t_props_state
Properties for the 'toplevel', which is specified in the 'state' section of the JSON files.
Some of these can be overridden on the command line
type
t_props_pocl
=
{
cl_platform : int; |
cl_device : int; |
gpu_memory_limit_pc : float; |
workflow : Workflow.t; |
}
t_props_pocl
Properties for the pocl, these are specified in the 'state' section of the JSON files.
type
t_props_geodata
=
{
filename : string; |
dtm_file : string; |
dtm_path : string list; |
no_data_values : float list; |
pad_width : int; |
title : string; |
roi_x_bounds : int array; |
roi_y_bounds : int array; |
basins : int list; |
basins_file : string; |
do_basin_masking : bool; |
workflow : Workflow.t; |
}
t_props_geodata
Properties for the geodata workflow.
type
t_props_preprocess
=
{
do_simple_gradient_vector_field :
bool; |
do_normalize_speed : bool; |
vecsum_threshold : float; |
divergence_threshold : float; |
curl_threshold : float; |
workflow : Workflow.t; |
}
t_props_preprocess
Properties for the preprocess workflow.
type
t_props_trace
=
{
integrator_step_factor : float; |
do_trace_upstream : bool; |
do_trace_downstream : bool; |
max_integration_step_error : float; |
max_length : float; |
integration_halt_threshold : float; |
trajectory_resolution : int; |
subpixel_seed_point_density : int; |
jitter_magnitude : float; |
interchannel_max_n_steps : int; |
segmentation_threshold : int; |
left_flank_addition : int32; |
workflow : Workflow.t; |
}
t_props_trace
Properties for the trace workflow.
type
t_props_analysis
=
{
do_marginal_distbn_dsla : bool; |
do_marginal_distbn_usla : bool; |
do_marginal_distbn_dslt : bool; |
do_marginal_distbn_uslt : bool; |
do_marginal_distbn_dslc : bool; |
do_marginal_distbn_uslc : bool; |
marginal_distbn_kde_kernel : string; |
marginal_distbn_kde_bw_method :
string; |
marginal_distbn_kde_bandwidth :
float; |
marginal_distbn_kde_nx_samples :
int; |
pdf_slt_min : float; |
do_joint_distribn_dsla_dslt : bool; |
do_joint_distribn_usla_uslt : bool; |
do_joint_distribn_uslt_dslt : bool; |
do_joint_distribn_dsla_usla : bool; |
do_joint_distribn_dsla_dslc : bool; |
do_joint_distribn_usla_uslc : bool; |
do_joint_distribn_uslc_dslc : bool; |
joint_distbn_kde_kernel : string; |
joint_distbn_kde_bw_method : string; |
joint_distbn_kde_bandwidth : float; |
joint_distbn_kde_nxy_samples : int; |
joint_distbn_y_shear_factor : float; |
joint_distbn_mode2_tilt : float; |
joint_distbn_mode2_nearness_factor :
float; |
workflow : Workflow.t; |
}
t_props_analysis
Properties for the analysis workflow.
type
t_props_plot
=
{
do_plot_dtm : bool; |
do_plot_roi : bool; |
do_plot_streamlines : bool; |
do_plot_flow_maps : bool; |
do_plot_segments : bool; |
do_plot_channels : bool; |
do_plot_hillslope_lengths : bool; |
do_plot_hillslope_lengths_contoured :
bool; |
do_plot_hillslope_distributions :
bool; |
do_plot_downstreamlines : bool; |
do_plot_upstreamlines : bool; |
do_plot_seed_points : bool; |
do_plot_flow_vectors : bool; |
do_plot_blockages : bool; |
do_plot_loops : bool; |
plot_interpolation_method : string; |
plot_streamline_limit : int; |
do_plot_color_shaded_relief : bool; |
do_plot_curvature_roi : bool; |
do_plot_colorized_streamlines :
bool; |
do_plot_merged_streamline_density_bands :
bool; |
plot_window_size_factor : float; |
plot_window_pdf_size_factor : float; |
plot_window_width : float; |
plot_window_height : float; |
hillshade_azimuth : float; |
hillshade_angle : float; |
downstreamline_color : string; |
upstreamline_color : string; |
streamline_point_marker : string; |
streamline_point_size : float; |
streamline_point_alpha : float; |
shaded_relief_hillshade_alpha :
float; |
shaded_relief_color_alpha : float; |
streamline_shaded_relief_hillshade_alpha :
float; |
streamline_shaded_relief_color_alpha :
float; |
streamline_density_alpha : float; |
streamline_density_cmap : string; |
grid_shaded_relief_hillshade_alpha :
float; |
grid_shaded_relief_color_alpha :
float; |
seed_point_marker : string; |
seed_point_marker_size : float; |
seed_point_marker_color : string; |
seed_point_marker_alpha : float; |
channel_head_marker : string; |
channel_head_marker_alpha : float; |
channel_shaded_relief_hillshade_alpha :
float; |
gradient_vector_color : string; |
gradient_vector_alpha : float; |
gradient_vector_scale : float; |
blockage_marker_size : float; |
loops_marker_size : float; |
classical_streamplot_density :
float; |
terrain_cmap : string; |
shuffle_random_seed : int; |
random_cmap_seed : int; |
do_plot_maps : bool; |
do_plot_distributions : bool; |
do_plot_marginal_pdf_dsla : bool; |
do_plot_marginal_pdf_usla : bool; |
do_plot_marginal_pdf_dslt : bool; |
do_plot_marginal_pdf_uslt : bool; |
do_plot_marginal_pdf_dslc : bool; |
do_plot_marginal_pdf_uslc : bool; |
do_plot_joint_pdf_dsla_usla : bool; |
do_plot_joint_pdf_dsla_dslt : bool; |
do_plot_joint_pdf_usla_uslt : bool; |
do_plot_joint_pdf_uslt_dslt : bool; |
do_plot_joint_pdf_dsla_dslc : bool; |
do_plot_joint_pdf_usla_uslc : bool; |
do_plot_joint_pdf_uslc_dslc : bool; |
joint_distbn_n_contours : float; |
marginal_distbn_viz_tilt : float; |
marginal_distbn_viz_scale : float; |
joint_distbn_viz_tilt : float; |
joint_distbn_viz_scale : float; |
workflow : Workflow.t; |
}
t_props_plot
Properties for the plot workflow.
type
t_props
=
{
state : t_props_state; |
pocl : t_props_pocl; |
geodata : t_props_geodata; |
preprocess : t_props_preprocess; |
trace : t_props_trace; |
analysis : t_props_analysis; |
plot : t_props_plot; |
}
t_props
Structure pulling together the properties for all the workflows
val
verbosity_from_properties : Workflow.t ‑> t_verbosity ‑> t_verbosity
verbosity_from_properties properties
verbosity
Upgrade verbosity based on 'verbose', 'noisy' and 'debug' properties
val
read_state : Properties.t ‑> t_cmdline_overrides
‑> t_props_state * t_verbosity
read_state properties
Read the properties of 'state'
val
read_pocl : t_verbosity
‑> Properties.t ‑> t_props_pocl
read_pocl verbosity properties
Read the properties of 'pocl' - actually from state - to set it up for workflow
val
geodata_validate_properties : t_props_geodata ‑> unit
geodata_validate_properties props
Validate that the properties for the geodata workflow are supported
val
read_geodata : t_verbosity ‑> Properties.t ‑> t_props_geodata
read_geodata verbosity properties
Set up properties for the Geodata workflow, given a higher level verbosity and a properties
val
read_preprocess : t_verbosity ‑> Properties.t ‑> t_props_preprocess
read_preprocess verbosity
properties
Set up properties for the Preprocess workflow, given a higher level verbosity and a properties
val
read_trace : t_verbosity
‑> Properties.t ‑> t_props_trace
read_trace verbosity properties
Set up properties for the Trace workflow, given a higher level verbosity and a properties
val
read_analysis : t_verbosity ‑> Properties.t ‑> t_props_analysis
read_analysis verbosity properties
Set up properties for the Analysis workflow, given a higher level verbosity and a properties
val
read_plot : t_verbosity
‑> Properties.t ‑> t_props_plot
read_plot verbosity properties
Set up properties for the Plot workflow, given a higher level verbosity and a properties
val
read_properties : (string list *
string) list ‑> t_cmdline_overrides
‑> t_props
read_properties file_path_list
cmdline_overrides
Read properties from a list of JSON files, and incorporate overrides from the command line