sparkling.destructuring
Contains wrapper-functions to destructure scala/spark data structures
key-seq-seq-fn
(key-seq-seq-fn f)
wraps a function f [k seq1 seq2] to untuple a key/value tuple with two partial values both being seqs. Useful e.g. on map after a cogroup with two RDDs.
key-val-val-fn
(key-val-val-fn f & {:keys [optional-second-value?], :or {optional-second-value? false}})
wraps a function f [k val1 val2] to untuple a key/value tuple with two partial values. Useful e.g. on map after a join. If optional-second-value? is true (default is false), the second tuple-value might not be there (by use of Optional), e.g. in joins, where the right side might be empty. We’ll just call the wrapped function with nil as second value then.
key-value-fn
(key-value-fn f)
wraps a function f [k v] to untuple a key/value tuple. Useful e.g. on map for PairRDD.
seq-seq-fn
(seq-seq-fn f)
wraps a function f [seq1 seq2] to untuple a tuple-value with two partial values all being seqs. Useful e.g. on map-values after a cogroup with two RDDs.
seq-seq-seq-fn
(seq-seq-seq-fn f)
wraps a function f [seq1 seq2 seq3] to untuple a triple-value with three partial values all being seqs. Useful e.g. on map-values after a cogroup with three RDDs.
val-val-fn
(val-val-fn f & {:keys [optional-second-value?], :or {optional-second-value? false}})
wraps a function f [val1 val2] to untuple a value tuple with two partial values. Useful e.g. on map-values after a join. If optional-second-value? is true (default is false), the second tuple-value might not be there (by use of Optional), e.g. in joins, where the right side might be empty. We’ll just call the wrapped function with nil as second value then.