nrgpy.read package#

Submodules#

nrgpy.read.channel_info_arrays module#

nrgpy.read.channel_info_arrays.return_array(data_file_type)[source]#

return data file header parameter array based on data_file_type

nrgpy.read.channel_info_arrays.return_sp3_ch_info()[source]#

returns array of sensor info parameters for Symphonie, PLUS, and PLUS3 txt export files

nrgpy.read.channel_info_arrays.return_spro_ch_info()[source]#

returns array of possible channel parameters for SymphoniePRO txt export files

nrgpy.read.spidar_txt module#

class nrgpy.read.spidar_txt.spidar_data_read(filename='')[source]#

Bases: object

reads in CSV file(s) using pandas and creates

Parameters
data_filestr

path to single CSV or ZIP to be read

Returns
dataobj

pandas dataframe of all available data

heightslist

list of measurement heights

Examples

Read a spidar data file into an object:

>>> import nrgpy
>>> reader = nrgpy.spidar_data_read(filename="1922AG0070_CAG70-SPPP-LPPP_PENT_AVGWND_2019-07-04_1.zip")
>>> reader.heights
['40', '60', '80', '90', '100', '120', '130', '160', '180', '200']
>>> reader.data
        Timestamp  pressure[mmHg]  temperature[C]  ...  dir_200_mean[Deg]  dir_200_std[Deg]  wind_measure_200_quality[%]
0   2019-07-03 23:40:00          753.55           23.68  ...             342.36             63.63                           48
1   2019-07-03 23:50:00          753.47           23.76  ...             345.70             57.59                           38
2   2019-07-04 00:00:00          753.46           23.96  ...             314.16             82.73                           20
...

Ex. read a directory of spidar data files into an object:

>>> reader = nrgpy.spidar_data_read()
>>> reader.concat_txt(
        txt_dir="/path/to/spidardata/",
        file_filter="2020-01",
        progress_bar=False
    )
Adding 1/8  ...  /home/user/spidardata/1922AG0070_CAG70-SPPP-LPPP_PENT_AVGWND_2019-07-01_1.zip [OK]
Adding 2/8  ...  /home/user/spidardata/1922AG0070_CAG70-SPPP-LPPP_PENT_AVGWND_2019-07-01_2.csv [OK]
Adding 3/8  ...  /home/user/spidardata/1922AG0070_CAG70-SPPP-LPPP_PENT_AVGWND_2019-07-02_1.zip [OK]
Adding 4/8  ...  /home/user/spidardata/1922AG0070_CAG70-SPPP-LPPP_PENT_AVGWND_2019-07-03_1.zip [OK]
Adding 5/8  ...  /home/user/spidardata/1922AG0070_CAG70-SPPP-LPPP_PENT_AVGWND_2019-07-04_1.zip [OK]
Adding 6/8  ...  /home/user/spidardata/1922AG0070_CAG70-SPPP-LPPP_PENT_AVGWND_2019-07-05_1.zip [OK]
Adding 7/8  ...  /home/user/spidardata/1922AG0070_CAG70-SPPP-LPPP_PENT_AVGWND_2019-07-06_1.zip [OK]
Adding 8/8  ...  /home/user/spidardata/1922AG0070_CAG70-SPPP-LPPP_PENT_AVGWND_2019-07-07_1.zip [OK]
>>> reader.serial_number
'1922AG0070'
concat_txt(txt_dir='', output_txt=False, out_file='', file_filter='', file_filter2='', start_date='1970-01-01', end_date='2150-12-31', progress_bar=True)[source]#

concatenate files in a folder

Parameters
txt_dirstr

path to csv or csv.zip files

output_txtboolean

export concatenated data

out_filestr

optional, filename of text export

start_datestr

yyy-mm-dd formatted string

end_datestr

yyy-mm-dd formatted string

progress_barboolean

show progress bar instead of each file being concatenated

Returns
None

adds data dataframe to reader object

get_heights()[source]#
read_file(f)[source]#

nrgpy.read.sympro_txt module#

nrgpy.read.sympro_txt.shift_timestamps(txt_folder='', out_folder='', file_filter='', start_date='1970-01-01', end_date='2150-12-31', seconds=3600)[source]#

Takes as input a folder of exported standard text files and time to shift in seconds.

Parameters
txt_folderstr

path to folder with txt files to shift

out_folderstr

where to put the shifted files (in subfolder by default)

file_filterstr

filter for restricting file set

start_datestr

date filter “YYYY-mm-dd”

end_datestr

date filter “YYYY-mm-dd”

secondsint

time in seconds to shift timestamps (default 3600)

Returns
obj

text files with shifted timestamps; new file names include shifted timestamp.

class nrgpy.read.sympro_txt.sympro_txt_read(filename='', out_file='', text_timestamps=False, **kwargs)[source]#

Bases: object

arrange_ch_info()[source]#

creates ch_info dataframe and ch_list array

calculate_soiling_ratio(method='IEC', T0=25, G0=1000, alpha=0.0004, I_clean_SC_0=0.9, I_soiled_SC_0=0.9)[source]#
concat_txt(txt_dir='', file_type='meas', file_filter='', filter2='', start_date='1970-01-01', end_date='2150-12-31', ch_details=False, output_txt=False, out_file='', progress_bar=True, **kwargs)[source]#

Will concatenate all text files in the txt_dir

files must match the site_filter argument. Note these are both blank by default.

Parameters
txt_dirstr

directory holding txt files

file_typestr

type of export (meas, event, comm, sample, etc…)

file_filterstr

text filter for txt files, like site number, etc.

filter2str

secondary text filter

start_datestr

for filtering files to concat based on date “YYYY-mm-dd”

end_datestr

for filtering files to concat based on date “YYYY-mm-dd”

ch_detailsbool

show additional info in ch_info dataframe

output_txtbool

create a txt output of data df

out_filestr

filename to write data dataframe too if output_txt = True

progress_barbool

show bar on concat [True] or list of files [False]

Returns
ch_infoobj

pandas dataframe of ch_list (below) pulled out of file with sympro_txt_read.arrange_ch_info()

ch_listlist

list of channel info; can be converted to json w/ import json … json.dumps(fut.ch_info)

dataobj

pandas dataframe of all data

headobj

lines at the top of the txt file…, used when rebuilding timeshifted files

site_infoobj

pandas dataframe of site information

logger_snstr
ipack_snstr
logger_typestr
ipack_typestr
latitudefloat
longitudefloat
elevationint
site_numberstr
site_descriptionstr
start_datestr
txt_file_nameslist

list of files included in concatenation

Examples

Read files into nrgpy reader object

>>> import nrgpy
>>> reader = nrgpy.sympro_txt_read()
>>> reader.concat_txt(
        txt_dir='/path/to/txt/files/',
        file_filter='123456', # site 123456
        start_date='2020-01-01',
        end_date='2020-01-31',
    )
Time elapsed: 2 s | 33 / 33 [=============================================] 100%
Queue processed
>>> reader.logger_sn
'820600019'
>>> reader.ch_info
        Bearing:        Channel:        Description:    Effective Date:         Height:         Offset:         Scale Factor:   Serial Number:  Type:   Units:
0       50.00       1           NRG S1          2020-01-31 00:00:00     33.00   0.13900         0.09350             94120000059         Anemometer      m/s
1       230.00      2           NRG S1          2020-01-31 00:00:00     0.00    0.13900         0.09350             94120000058         Anemometer      m/s
2       50.00       3           NRG S1          2020-01-31 00:00:00     22.00   0.13900         0.09350             94120000057         Anemometer      m/s
3       230.00      4           NRG 40C Anem    2020-01-31 00:00:00     22.00   0.35000         0.76500             179500324860        Anemometer      m/s
4       50.00       5           NRG 40C Anem    2020-01-31 00:00:00     12.00   0.35000         0.76500             179500324859        Anemometer      m/s
5       230.00      6           NRG S1          2020-01-31 00:00:00     12.00   0.13900         0.09350             94120000056         Anemometer      m/s
6       320.00      13          NRG 200M Vane   2020-01-31 00:00:00     32.00   -1.46020        147.91100           10700000125         Vane            Deg
7       320.00      14          NRG 200M Vane   2020-01-31 00:00:00     21.00   -1.46020        147.91100           10700000124         Vane            Deg
8       0.00        15          NRG T60 Temp    2020-01-31 00:00:00     34.00   -40.85550       44.74360            9400000705          Analog          C
9       0.00        16          NRG T60 Temp    2020-01-31 00:00:00     2.00    -40.85550       44.74360            9400000xxx          Analog          C
10      0.00        17          NRG RH5X Humi   2020-01-31 00:00:00     0.00    0.00000         20.00000            NaN                 Analog          %RH
11      0.00        20          NRG BP60 Baro   2020-01-31 00:00:00     0.00    495.27700       243.91400           NaN                 Analog          hPa
12      0.00        21          NRG BP60 Baro   2020-01-31 00:00:00     2.00    495.04400       244.23900           9396FT1937          Analog          hPa
format_data_for_epe()[source]#
format_site_data()[source]#

take txt header to create oject data

insert_blank_header_rows(filename)[source]#

insert blank rows when using shift_timestamps()

ensures the resulting text file looks and feels like an original Sympro Desktop exported

make_header_for_epe()[source]#
output_txt_file(epe=False, soiling=False, standard=True, shift_timestamps=False, out_file='', **kwargs)[source]#
select_channels_for_reformat(epe=False, soiling=False)[source]#

determines which of the channel headers fit those required for post-processing for either

  1. EPE formatting

  2. soiling ratio calculation

Note that this formatting requires the the channel headers to be full (requires Local export of text files, as of 0.1.8.

nrgpy.read.txt_utils module#

nrgpy.read.txt_utils.format_sympro_site_data(reader)[source]#

adds formatted site dataframe to reader object

class nrgpy.read.txt_utils.read_text_data(filename='', data_type='sp3', txt_dir='', file_filter='', filter2='', file_ext='', sep='\t')[source]#

Bases: object

class for handling known csv-style text data files with header information

Parameters
filenamestr, optional

perform a single file read (takes precedence over txt_dir)

data_typestr

specify instrument that the data file came from

sepstr

‘ ‘; csv separator

txt_dirstr

folder path of text files to read and concatenate

file_filterstr, optional

use when using txt_dir to filter on subset of files

file_extstr, optional

secondary file filter

arrange_ch_info()[source]#

generates list and dataframe of channel information

concat(output_txt=False, out_file='', file_filter='', filter2='', progress_bar=True)[source]#

combine exported rwd files (in txt format)

Parameters
output_txtbool

set to True to save a concatenated text file

out_filestr

filepath, absolute or relative

file_filterstr
filter2str
progress_barbool
format_rwd_site_data()[source]#

adds formatted site dataframe to reader object

get_data(_file)[source]#

create dataframe of tabulated data

get_head(_file)[source]#

get the first lines of the file

excluding those without tabs up to the self.skip_rows line

get_site_info(_file)[source]#

create dataframe of site info

Module contents#