ldcpy package¶
Submodules¶
ldcpy.metrics module¶
ldcpy.plot module¶
- class ldcpy.plot.calcsPlot(ds, varname, calc, sets, group_by=None, scale='linear', calc_type='raw', plot_type='spatial', transform='none', subset=None, approx_lat=None, approx_lon=None, lev=0, color='coolwarm', standardized_err=False, quantile=None, calc_ssim=False, contour_levs=24, short_title=False, axes_symmetric=False, legend_loc='upper right', vert_plot=False, tex_format=False, legend_offset=None, weighted=True)[source]¶
Bases:
object
This class contains code to plot calcs in an xarray Dataset that has either ‘lat’ and ‘lon’ dimensions, or a ‘time’ dimension.
- ldcpy.plot.plot(ds, varname, calc, sets, group_by=None, scale='linear', calc_type='raw', plot_type='spatial', transform='none', subset=None, lat=None, lon=None, lev=0, color='coolwarm', quantile=None, start=None, end=None, calc_ssim=False, short_title=False, axes_symmetric=False, legend_loc='upper right', vert_plot=False, tex_format=False, legend_offset=None, weighted=True)[source]¶
Plots the data given an xarray dataset
- Parameters
ds (xarray.Dataset) – The input dataset
varname (str) – The name of the variable to be plotted
calc (str) –
The name of the calc to be plotted (must match a property name in the Datasetcalcs class in ldcpy.plot, for more information about the available calcs see ldcpy.Datasetcalcs) Acceptable values include:
ns_con_var
ew_con_var
mean
std
variance
prob_positive
prob_negative
odds_positive
zscore
mean_abs
mean_squared
rms
sum
sum_squared
corr_lag1
quantile
lag1
standardized_mean
ann_harmonic_ratio
pooled_variance_ratio
sets (list <str>) – The labels of the dataset to gather calcs from
group_by (str) –
how to group the data in time series plots. Valid groupings:
time.day
time.dayofyear
time.month
time.year
scale (str, optional) –
time-series y-axis plot transformation. (default “linear”) Valid options:
linear
log
calc_type (str, optional) –
The type of operation to be performed on the calcs. (default ‘raw’) Valid options:
raw: the unaltered calc values
diff: the difference between the calc values in the first set and every other set
ratio: the ratio of the calc values in (2nd, 3rd, 4th… sets/1st set)
calc_of_diff: the calc value computed on the difference between the first set and every other set
plot_type (str , optional) –
The type of plot to be created. (default ‘spatial’) Valid options:
spatial: a plot of the world with values at each lat and lon point (takes the mean across the time dimension)
time-series: A time-series plot of the data (computed by taking the mean across the lat and lon dimensions)
histogram: A histogram of the time-series data
transform (str, optional) –
data transformation. (default ‘none’) Valid options:
none
log
subset (str, optional) –
subset of the data to gather calcs on (default None). Valid options:
first5: the first 5 days of data
DJF: data from the months December, January, February
MAM: data from the months March, April, May
JJA: data from the months June, July, August
SON: data from the months September, October, November
lat (float, optional) – The latitude of the data to gather calcs on (default None).
lon (float , optional) – The longitude of the data to gather calcs on (default None).
lev (float, optional) – The level of the data to gather calcs on (used if plotting from a 3d data set), (default 0).
color (str, optional) – The color scheme for spatial plots, (default ‘coolwarm’). see https://matplotlib.org/3.1.1/gallery/color/colormap_reference.html for more options
quantile (float, optional) – A value between 0 and 1 required if calc=”quantile”, corresponding to the desired quantile to gather, (default 0.5).
start (int, optional) – A value between 0 and the number of time slices indicating the start time of a subset, (default None).
end (int, optional) – A value between 0 and the number of time slices indicating the end time of a subset, (default None)
calc_ssim (bool, optional) – Whether or not to calculate the ssim (structural similarity index) between two plots (only applies to plot_type = ‘spatial’), (default False)
short_title (bool, optional) – If True, use a shortened title in the plot output (default False).
axes_symmetric (bool, optional) – Whether or not to make the colorbar axes symmetric about zero (used in a spatial plot) (default False)
legend_loc (str, optional) – The location to put the legend in a time-series plot in single-column format (plot_type = “time_series”, vert_plot=True) (default “upper right”)
vert_plot (bool, optional) – If true, forces plots into a single column format and enlarges text. (default False)
tex_format (bool, optional) – Whether to interpret all plot output strings as latex formatting (default False)
legend_offset (2-tuple, optional) – The x- and y- offset of the legend. Moves the corner of the legend specified by legend_loc to the specified location specified (where (0,0) is the bottom left corner of the plot and (1,1) is the top right corner). Only affects time-series, histogram, and periodogram plots.
- Returns
out
- Return type
ldcpy.util module¶
- ldcpy.util.check_metrics(ds, varname, set1, set2, ks_tol=0.05, pcc_tol=0.99999, spre_tol=5.0, ssim_tol=0.995, **calcs_kwargs)[source]¶
Check the K-S, Pearson Correlation, and Spatial Relative Error calcs
- Parameters
ds (xarray.Dataset) – An xarray dataset containing multiple netCDF files concatenated across a ‘collection’ dimension
varname (str) – The variable of interest in the dataset
set1 (str) – The collection label of the “control” data
set2 (str) – The collection label of the (1st) data to compare
ks_tol (float, optional) – The p-value threshold (significance level) for the K-S test (default = .05)
pcc_tol (float, optional) – The default Pearson corrolation coefficient (default = .99999)
spre_tol (float, optional) – The percentage threshold for failing grid points in the spatial relative error test (default = 5.0).
ssim_tol (float, optional) – The threshold for the data ssim test (default = .995
**calcs_kwargs – Additional keyword arguments passed through to the
Datasetcalcs
instance.
- Returns
out
- Return type
Number of failing calcs
Notes
Check the K-S, Pearson Correlation, and Spatial Relative Error calcs from:
A. H. Baker, H. Xu, D. M. Hammerling, S. Li, and J. Clyne, “Toward a Multi-method Approach: Lossy Data Compression for Climate Simulation Data”, in J.M. Kunkel et al. (Eds.): ISC High Performance Workshops 2017, Lecture Notes in Computer Science 10524, pp. 30–42, 2017 (doi:10.1007/978-3-319-67630-2_3).
Check the Data SSIM, which is a modification of SSIM calc from:
A.H. Baker, D.M. Hammerling, and T.L. Turton. “Evaluating image quality measures to assess the impact of lossy data compression applied to climate simulation data”, Computer Graphics Forum 38(3), June 2019, pp. 517-528 (doi:10.1111/cgf.13707).
K-S: fail if p-value < .05 (significance level) Pearson correlation coefficient: fail if coefficient < .99999 Spatial relative error: fail if > 5% of grid points fail relative error Data SSIM: fail if Data SSIM < .995
- ldcpy.util.collect_datasets(data_type, varnames, list_of_ds, labels, **kwargs)[source]¶
Concatonate several different xarray datasets across a new “collection” dimension, which can be accessed with the specified labels. Stores them in an xarray dataset which can be passed to the ldcpy plot functions (Call this OR open_datasets() before plotting.)
- Parameters
varnames (list) – The variable(s) of interest to combine across input files (usually just one)
list_of_datasets (list) – The datasets to be concatonated into a collection
labels (list) –
The respective label to access data from each dataset (also used in plotting fcns)
**kwargs : (optional) – Additional arguments passed on to xarray.concat(). A list of available arguments can be found here: https://xarray-test.readthedocs.io/en/latest/generated/xarray.concat.html
- Returns
out – a collection containing all the data from the list datasets
- Return type
- ldcpy.util.compare_stats(ds, varname: str, sets, significant_digits: int = 5, include_ssim: bool = False, weighted: bool = True, **calcs_kwargs)[source]¶
Print error summary statistics for multiple DataArrays (should just be a single time slice)
- Parameters
ds (xarray.Dataset) – An xarray dataset containing multiple netCDF files concatenated across a ‘collection’ dimension
varname (str) – The variable of interest in the dataset
sets (list of str) – The labels of the collection to compare (all will be compared to the first set)
significant_digits (int, optional) – The number of significant digits to use when printing stats (default 5)
include_ssim (bool, optional) – Whether or not to compute the image ssim - slow for 3D vars (default: False)
weighted (bool, optional) – Whether or not weight the means (default = True)
**calcs_kwargs – Additional keyword arguments passed through to the
Datasetcalcs
instance.
- Returns
out
- Return type
- ldcpy.util.open_datasets(data_type, varnames, list_of_files, labels, **kwargs)[source]¶
Open several different netCDF files, concatenate across a new ‘collection’ dimension, which can be accessed with the specified labels. Stores them in an xarray dataset which can be passed to the ldcpy plot functions.
- Parameters
varnames (list) – The variable(s) of interest to combine across input files (usually just one)
list_of_files (list) – The file paths for the netCDF file(s) to be opened
labels (list) – The respective label to access data from each netCDF file (also used in plotting fcns)
**kwargs – (optional) – Additional arguments passed on to xarray.open_mfdataset(). A list of available arguments can be found here: http://xarray.pydata.org/en/stable/generated/xarray.open_dataset.html
- Returns
out – a collection containing all the data from the list of files
- Return type
Module contents¶
Top-level module for ldcpy.