Module Streamlines__.Properties

Copyright (C) 2017-2018,  Colin P Stark and Gavin J Stark.  All rights reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *   http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 *
 * @file   properties.ml
 * @brief  Workflow and properties libraries
 *
module Json = Yojson.Basic
module Option = Batteries.Option

Verbosity

Verbosity is a basic part of the properties of a workflow. The verbosity can be set to one of many different levels, and the properties for a workflow can each have its own verbosity. Functions are provided to permit the execution of code dependent on the verbosity level.

type t_verbosity =
| PV_Quiet
| PV_Info
| PV_Verbose
| PV_Noisy
| PV_Debug

t_verbosity enumeration for the different levels of verbosity

val pv_level : t_verbosity ‑> int

pv_level verbosity converts the verbosity to an integer level.

This function is for internal use - it is best to use the pv_verbose/pv_debug etc functions to control execution of code dependent on the verbosity level.

This function is used internally to grade verbosity, and hence to select the 'more verbose level' verbosity from a pair of verbositys.

val pv_of_int : int ‑> t_verbosity

pv_of_int level converts an integer level to a verbosity

This function is used by argument parsing to convert a command line verbosity to a level for execution of the code.

This is the inverse function to pv_level

val pv_str : t_verbosity ‑> string

pv_str verbosity produces a string corresponding to the verbosity level

This function is used to show how verbose a workflow is when it starts

val pv_verbosity : ?⁠verbose:bool ‑> ?⁠debug:bool ‑> ?⁠noisy:bool ‑> t_verbosity ‑> t_verbosity

pv_verbosity ?verbose ?debug ?noisy verbosity - upgrade verbosity level

The verbosity level of a Workflow is the larger of that specified by its properties and that for the whole execution; this function permits 'upgrading' the verbosity based on the three properties 'verbose', 'noisy' and 'debug'.

val pv_if : t_verbosity ‑> t_verbosity ‑> (unit ‑> unit) ‑> unit

pv_if verbosity level f - if verbosity exceeds level then invoke

f ()

This is an internal function, for use in pv_info, pv_verbose, etc.

It invokes the function f with a unit argument if the verbosity at least matches level

val pv_info : t_verbosity ‑> (unit ‑> unit) ‑> unit

pv_info pv

Predicate function used with

fun _ -> ...

to invoke the function only if the verbosity level is at least PV_Info. Use, for example, as

pv_info verbosity (fun _ -> Printf.printf "Verbosity is at least info")

.

val pv_verbose : t_verbosity ‑> (unit ‑> unit) ‑> unit

pv_verbose pv

Predicate function used with

fun _ -> ...

to invoke the function only if the verbosity level is at least PV_Verbose. Use, for example, as

pv_verbose verbosity (fun _ -> Printf.printf "Verbosity is at least verbose")

.

val pv_noisy : t_verbosity ‑> (unit ‑> unit) ‑> unit

pv_noisy pv

Predicate function used with

fun _ -> ...

to invoke the function only if the verbosity level is at least PV_Noisy. Use, for example, as

pv_noisy verbosity (fun _ -> Printf.printf "Verbosity is at least noisy")

.

val pv_debug : t_verbosity ‑> (unit ‑> unit) ‑> unit

pv_debug pv

Predicate function used with

fun _ -> ...

to invoke the function only if the verbosity level is at least PV_Debug. Use, for example, as

pv_debug verbosity (fun _ -> Printf.printf "Verbosity is at least debug")

.

Properties module

module Properties : sig ... end

Workflow module

module Workflow : sig ... end

Properties types

exception Geodata of string

Geodata exception

type t_cmdline_overrides = {
verbosity : t_verbosity;
do_reload_state : bool option;
do_geodata : bool option;
do_preprocess : bool option;
do_condition : bool option;
do_trace : bool option;
do_analysis : bool option;
do_mapping : bool option;
do_plot : bool option;
do_save_state : bool option;
do_export : bool option;
}

t_cmdline_overrides

Structure containing command line settings, which can override the properties from the JSON files.

If the options are 'None' then there is no override; else it is 'Some <value>', and the value should override that from the JSON properties.

This structure is filled out in streamlines.ml, and used in read_state

type t_props_state = {
verbosity : t_verbosity;
verbose : bool;
debug : bool;
noisy : bool;
cl_platform : int;
cl_device : int;
gpu_memory_limit_pc : float;
array_order : string;
do_geodata : bool;
do_preprocess : bool;
do_condition : bool;
do_trace : bool;
do_postprocess : bool;
do_analysis : bool;
do_mapping : bool;
do_plot : bool;
do_save_state : bool;
do_export : bool;
do_rw_savez : bool;
do_rw_hdf5 : bool;
do_reload_state : bool;
}

t_props_state

Properties for the 'toplevel', which is specified in the 'state' section of the JSON files.

Some of these can be overridden on the command line

type t_props_pocl = {
cl_platform : int;
cl_device : int;
gpu_memory_limit_pc : float;
workflow : Workflow.t;
}

t_props_pocl

Properties for the pocl, these are specified in the 'state' section of the JSON files.

type t_props_geodata = {
filename : string;
dtm_file : string;
dtm_path : string list;
no_data_values : float list;
pad_width : int;
title : string;
roi_x_bounds : int array;
roi_y_bounds : int array;
basins : int list;
basins_file : string;
do_basin_masking : bool;
workflow : Workflow.t;
}

t_props_geodata

Properties for the geodata workflow.

type t_props_preprocess = {
do_simple_gradient_vector_field : bool;
do_normalize_speed : bool;
vecsum_threshold : float;
divergence_threshold : float;
curl_threshold : float;
workflow : Workflow.t;
}

t_props_preprocess

Properties for the preprocess workflow.

type t_props_trace = {
integrator_step_factor : float;
do_trace_upstream : bool;
do_trace_downstream : bool;
max_integration_step_error : float;
max_length : float;
integration_halt_threshold : float;
trajectory_resolution : int;
subpixel_seed_point_density : int;
jitter_magnitude : float;
interchannel_max_n_steps : int;
segmentation_threshold : int;
left_flank_addition : int32;
workflow : Workflow.t;
}

t_props_trace

Properties for the trace workflow.

type t_props_analysis = {
do_marginal_distbn_dsla : bool;
do_marginal_distbn_usla : bool;
do_marginal_distbn_dslt : bool;
do_marginal_distbn_uslt : bool;
do_marginal_distbn_dslc : bool;
do_marginal_distbn_uslc : bool;
marginal_distbn_kde_kernel : string;
marginal_distbn_kde_bw_method : string;
marginal_distbn_kde_bandwidth : float;
marginal_distbn_kde_nx_samples : int;
pdf_slt_min : float;
do_joint_distribn_dsla_dslt : bool;
do_joint_distribn_usla_uslt : bool;
do_joint_distribn_uslt_dslt : bool;
do_joint_distribn_dsla_usla : bool;
do_joint_distribn_dsla_dslc : bool;
do_joint_distribn_usla_uslc : bool;
do_joint_distribn_uslc_dslc : bool;
joint_distbn_kde_kernel : string;
joint_distbn_kde_bw_method : string;
joint_distbn_kde_bandwidth : float;
joint_distbn_kde_nxy_samples : int;
joint_distbn_y_shear_factor : float;
joint_distbn_mode2_tilt : float;
joint_distbn_mode2_nearness_factor : float;
workflow : Workflow.t;
}

t_props_analysis

Properties for the analysis workflow.

type t_props_plot = {
do_plot_dtm : bool;
do_plot_roi : bool;
do_plot_streamlines : bool;
do_plot_flow_maps : bool;
do_plot_segments : bool;
do_plot_channels : bool;
do_plot_hillslope_lengths : bool;
do_plot_hillslope_lengths_contoured : bool;
do_plot_hillslope_distributions : bool;
do_plot_downstreamlines : bool;
do_plot_upstreamlines : bool;
do_plot_seed_points : bool;
do_plot_flow_vectors : bool;
do_plot_blockages : bool;
do_plot_loops : bool;
plot_interpolation_method : string;
plot_streamline_limit : int;
do_plot_color_shaded_relief : bool;
do_plot_curvature_roi : bool;
do_plot_colorized_streamlines : bool;
do_plot_merged_streamline_density_bands : bool;
plot_window_size_factor : float;
plot_window_pdf_size_factor : float;
plot_window_width : float;
plot_window_height : float;
hillshade_azimuth : float;
hillshade_angle : float;
downstreamline_color : string;
upstreamline_color : string;
streamline_point_marker : string;
streamline_point_size : float;
streamline_point_alpha : float;
shaded_relief_hillshade_alpha : float;
shaded_relief_color_alpha : float;
streamline_shaded_relief_hillshade_alpha : float;
streamline_shaded_relief_color_alpha : float;
streamline_density_alpha : float;
streamline_density_cmap : string;
grid_shaded_relief_hillshade_alpha : float;
grid_shaded_relief_color_alpha : float;
seed_point_marker : string;
seed_point_marker_size : float;
seed_point_marker_color : string;
seed_point_marker_alpha : float;
channel_head_marker : string;
channel_head_marker_alpha : float;
channel_shaded_relief_hillshade_alpha : float;
gradient_vector_color : string;
gradient_vector_alpha : float;
gradient_vector_scale : float;
blockage_marker_size : float;
loops_marker_size : float;
classical_streamplot_density : float;
terrain_cmap : string;
shuffle_random_seed : int;
random_cmap_seed : int;
do_plot_maps : bool;
do_plot_distributions : bool;
do_plot_marginal_pdf_dsla : bool;
do_plot_marginal_pdf_usla : bool;
do_plot_marginal_pdf_dslt : bool;
do_plot_marginal_pdf_uslt : bool;
do_plot_marginal_pdf_dslc : bool;
do_plot_marginal_pdf_uslc : bool;
do_plot_joint_pdf_dsla_usla : bool;
do_plot_joint_pdf_dsla_dslt : bool;
do_plot_joint_pdf_usla_uslt : bool;
do_plot_joint_pdf_uslt_dslt : bool;
do_plot_joint_pdf_dsla_dslc : bool;
do_plot_joint_pdf_usla_uslc : bool;
do_plot_joint_pdf_uslc_dslc : bool;
joint_distbn_n_contours : float;
marginal_distbn_viz_tilt : float;
marginal_distbn_viz_scale : float;
joint_distbn_viz_tilt : float;
joint_distbn_viz_scale : float;
workflow : Workflow.t;
}

t_props_plot

Properties for the plot workflow.

type t_props = {
state : t_props_state;
pocl : t_props_pocl;
geodata : t_props_geodata;
preprocess : t_props_preprocess;
trace : t_props_trace;
analysis : t_props_analysis;
plot : t_props_plot;
}

t_props

Structure pulling together the properties for all the workflows

Properties functions

val verbosity_from_properties : Workflow.t ‑> t_verbosity ‑> t_verbosity

verbosity_from_properties properties verbosity

Upgrade verbosity based on 'verbose', 'noisy' and 'debug' properties

val read_state : Properties.t ‑> t_cmdline_overrides ‑> t_props_state * t_verbosity

read_state properties

Read the properties of 'state'

val read_pocl : t_verbosity ‑> Properties.t ‑> t_props_pocl

read_pocl verbosity properties

Read the properties of 'pocl' - actually from state - to set it up for workflow

val geodata_validate_properties : t_props_geodata ‑> unit

geodata_validate_properties props

Validate that the properties for the geodata workflow are supported

val read_geodata : t_verbosity ‑> Properties.t ‑> t_props_geodata

read_geodata verbosity properties

Set up properties for the Geodata workflow, given a higher level verbosity and a properties

val read_preprocess : t_verbosity ‑> Properties.t ‑> t_props_preprocess

read_preprocess verbosity properties

Set up properties for the Preprocess workflow, given a higher level verbosity and a properties

val read_trace : t_verbosity ‑> Properties.t ‑> t_props_trace

read_trace verbosity properties

Set up properties for the Trace workflow, given a higher level verbosity and a properties

val read_analysis : t_verbosity ‑> Properties.t ‑> t_props_analysis

read_analysis verbosity properties

Set up properties for the Analysis workflow, given a higher level verbosity and a properties

val read_plot : t_verbosity ‑> Properties.t ‑> t_props_plot

read_plot verbosity properties

Set up properties for the Plot workflow, given a higher level verbosity and a properties

val read_properties : (string list * string) list ‑> t_cmdline_overrides ‑> t_props

read_properties file_path_list cmdline_overrides

Read properties from a list of JSON files, and incorporate overrides from the command line