Tempest Extremes

Tempest Extremes is a popular software for detecting and tracking tropical cyclones and other weather systems in climate datasets. It is written by Paul Ullrich and released under a BSD 2-clause license.

For an overview of the functionalities, installation, and usage see the Tempest Extremes section of the documentation.

class tctrack.tempest_extremes.TETracker(detect_parameters=None, stitch_parameters=None)[source]

Bases: TCTracker

Class containing bindings to the Tempest Extremes code.

detect_parameters

Class containing the parameters for the detection step

Type:: TEDetectParameters

stitch_parameters

Class containing the parameters for the stitching step

Type:: TEStitchParameters | None

detect()[source]

Call the DetectNodes utility of Tempest Extremes.

This will make a system call out to the DetectNodes method from Tempest Extremes (provided it has been installed as an external dependency). DetectNodes will be run according to the parameters in the detect_parameters attribute that were set when the TETracker instance was created.

The output file is a plain text file containing each of the TC candidates at each time from the input files. If output_dir is None this will be a temporary file lasting the lifetime of the TETracker instance. If out_header is True the first two lines of the file will be a header describing the structure of the data. After this each time is listed in the format:

<year> <month> <day> <count> <hour>
       <i> <j> <lon> <lat> <var1> <var2> ...
       ...
       <i> <j> <lon> <lat> <var1> <var2> ...

count is the number of nodes at that time.
i, j are the grid indices of the node.
var1, var2, etc., are scalar variables as defined by output_commands (typically, psl, orog).

Returns:

dict of subprocess output corresponding to stdout, stderr, and returncode.

Return type:

dict

Raises:

FileNotFoundError – If the DetectNodes executeable from TempestExtremes cannot be found.
RuntimeError – If Tempest Extremes DetectNodes returns a non-zero exit code.

References

TempestExtremes Documentation and the DetectNodes Source

Examples

To set the parameters, instantiate a TETracker instance and run DetectNodes:

>>> my_params = TEDetectParameters(...)
>>> my_tracker = TETracker(detect_parameters=my_params)
>>> my_tracker.set_input_files("input.nc")
>>> result = my_tracker.detect()

stitch()[source]

Call the StitchNodes utility in Tempest Extremes.

This will make a system call out to the StitchNodes method from Tempest Extremes (provided it has been installed as an external dependency). StitchNodes will be run according to the parameters in the stitch_parameters attribute that were set when the TETracker instance was created.

The output is a file containing the data for each node of each trajectory. If output_dir is None this will be a temporary file lasting the lifetime of the TETracker instance. The format of the file depends on the out_file_format parameter. The default "gfdl" output is a plain-text “nodefile” format which contains a number of track trajectories, each of which in the form.

start <N> <year> <month> <day> <hour>
      <i> <j> <var1> <var2> ... <year> <month> <day> <hour>
      ...
      <i> <j> <var1> <var2> ... <year> <month> <day> <hour>

N is number of nodes in the trajectory (and number of lines below header).
i, j are grid indices.
var1, var2, etc., are scalar variables as defined by in_fmt (typically, lon, lat, psl, orog).
hour may instead be seconds if out_seconds is True.

Returns:

dict of subprocess output corresponding to stdout, stderr, and returncode.

Return type:

dict

Raises:

FileNotFoundError – If the StitchNodes executeable from TempestExtremes cannot be found.
RuntimeError – If Tempest Extremes StitchNodes returns a non-zero exit code.

References

TempestExtremes Documentation and the StitchNodes Source

Examples

To set the parameters, instantiate a TETracker instance and perform stitching:

>>> my_params = TEStitchParameters(...)
>>> my_tracker = TETracker(stitch_parameters=my_params)
>>> result = my_tracker.stitch()

read_trajectories()[source]

Parse outputs from Tempest Extremes to list of tctrack.core.Trajectory.

The file to be read and its properties are based on the values in the stitch_parameters attribute.

Returns:: A list of tctrack.core.Trajectory objects.
Return type:: list[Trajectory]

property global_metadata: dict

Read-only property containing metadata for tctrack / tracker instance.

Raises:: AttributeError – If _global_metadata has not been initialized.
Type:: dict

run_tracker_subprocess(command_name, command_list, input_file=None, input_str=None, cwd=None, verbosity=1)

Run a subprocess command for a cyclone tracking algorithm.

Parameters:

command_name (str) – The name of the command, used in logging and error messages.
command_list (list[str]) – The command and its arguments to execute.
input_file (str | None) – Path to a file to pass to the process via stdin (e.g. a namelist). Cannot be used together with input_str. Defaults to None.
input_str (str | None) – A string to pass to the process via stdin (e.g. interactive inputs). Cannot be used together with input_file. Defaults to None.
cwd (str | None) – Working directory in which to execute the command. Defaults to None.
verbosity (int) – Controls how much output is shown: 0 = No output gets printed. 1 = summary, first and last 12 lines printed (default). 2 = Entire output is streamed in real-time. Defaults to 1.

Returns:

Dictionary of subprocess output to ‘stdout’, ‘stderr’, and ‘returncode’.

Return type:

dict

Raises:

ValueError – If both input_file and input_str are provided simultaneously.
ValueError – If verbosity is not 0, 1, or 2.

set_input_files(input_files): Store the input file(s) as a list in a class attribute.

set_metadata()

Initialise the metadata attributes used in the CF-netcdf output.

This is called from the to_netcdf() method.

This sets the _global_metadata attribute with the TCTrack parameters and name of the tracker. It then calls the subclass-specific _set_metadata() to set _variable_metadata and _time_metadata.

Return type:: None

property time_metadata: TCTrackerTimeMetadata

Read-only property containing time metadata for this Tracker run.

Raises:: AttributeError – If _time_metadata has not been initialized.
Type:: dict

to_netcdf(output_file, trajectories=None)

Write track trajectories to CF-compliant NetCDF trajectory file.

Reads in trajectories based on the parameters set for a specific implementation of the class and writes them to a CF-Conventions compliant NetCDF trajectory file using cf-python. Trajectories are assumed to contain as a minimum data for lat, lon, and timestep.

An ancillary field variable is added to the output file indicating any tracks that start/end within 1 day of the input dataset boundaries.

Parameters:

output_file (str) – filename for the output netCDF file Note: This will be placed in the local directory unless a full path is given
trajectories (list[Trajectory] | None) – An optional list of trajectories to write. If the tracking algorithm produces an intermediate output file then this can be skipped as it will instead read from there.

Return type:

None

Warning

UserWarning: If there are no trajectories read in from the tracker outputs by read_trajectories() meaning no NetCDF output file can be written.

References

CF-Conventions v1.1 - H.4. Trajectory Data cf-python documentation

Examples

Instantiate a TCTracker subclass instance with appropriate parameters, run the relevant methods to generate cyclone track trajectories, and then save the results to a CF-compliant trajectory file:

>>> my_tracker = TCTracker(...)
>>> ...
>>> my_tracker.to_netcdf("my_netcdf_file.nc")

property variable_metadata: dict

Read-only property containing NetCDF metadata for variables.

Raises:: AttributeError – If _variable_metadata has not been initialized.
Type:: dict

run_tracker(input_files, output_file)[source]

Run TempestExtremes tracker to obtain tropical cyclone track trajectories.

This first runs detect() to get TC candidates at each time. Then these are combined into trajectories using stitch(). The output is then saved as a CF-compliant NetCDF trajectory file.

Parameters:

input_files (str | Iterable[str]) – A (list of) file path(s) containing NetCDF input data to use in the tracker.
output_file (str) – Filename to which the tropical cyclone trajectories are saved.

Raises:

FileNotFoundError –
- If the TempestExtremes executables cannot be found. - If the stitch output file does not exist.
RuntimeError – If the TempestExtremes commands return a non-zero exit code.

Examples

To create the tracker instance, then use run_tracker to perform the detection, stitching, and generate output.

>>> detect_params = TEDetectParameters(...)
>>> stitch_params = TEStitchParameters(...)
>>> my_tracker = TETracker(detect_params, stitch_params)
>>> my_tracker.run_tracker("input.nc", "output.nc")

class tctrack.tempest_extremes.TEDetectParameters(out_header=False, output_dir='', output_file='nodes.txt', search_by_min=None, search_by_max=None, closed_contours=None, no_closed_contours=None, thresholds=None, merge_dist=0.0, time_filter=None, lat_name='latitude', lon_name='longitude', min_lat=0.0, max_lat=0.0, min_lon=0.0, max_lon=0.0, regional=False, output_commands=None)[source]

Bases: TCTrackerParameters

Dataclass containing values used by the detection operation of TE.

See also

TEContour: The class used to define contour criteria
TEOutputCommand: The class used to define additional outputs

References

TempestExtremes Documentation and the DetectNodes Source

out_header: bool = False: Include header at the top of the output file?

output_dir: str = '': File path of the directory to output intermediate files. If left empty, it will use a temporary directory that will last only for the lifetime of the TETracker instance.

output_file: str = 'nodes.txt': Name of output nodefile to write to in the output_dir directory.

search_by_min: str | None = None: Input variable in NetCDF files for selecting candidate points (defined as local minima). If None, then uses "PSL" in Tempest Extremes.

search_by_max: str | None = None: Input variable in NetCDF files for selecting candidate points (defined as local maxima).

closed_contours: list[TEContour] | None = None: Criteria for candidates to be eliminated if they do not have a closed contour as a list of separate TEContour criteria.

no_closed_contours: list[TEContour] | None = None: Criteria for candidates to be eliminated if they do have a closed contour as a list of separate TEContour criteria.

thresholds: list[TEDetectThreshold] | None = None: Threshold criteria for candidates to be eliminated. I.e. if there is not a point within a given distance that satisfies the condition. Given as a list of separate TEDetectThreshold criteria.

merge_dist: float = 0.0: DetectNodes merges candidate points with a distance (in degrees) shorter than the specified value. Among two candidates within the merge distance, only the candidate with the lowest value of the search_by_min field or highest value of the search_by_max field are retained.

time_filter: str | None = None

“3hr”, “6hr”, “daily”. Alternatively, can be a regex for the datetime using format “YYYY-MM-DD HH:MM:SS”.

Type:: Filter for the input data frequency. Options are

lat_name: str = 'latitude': String for the latitude dimension in the NetCDF files.

lon_name: str = 'longitude': String for the longitude dimension in the NetCDF files.

min_lat: float = 0.0: Minimum latitude for candidate points.

max_lat: float = 0.0: Maximum latitude for candidate points. If max_lat and min_lat are equal then these arguments are ignored.

min_lon: float = 0.0: Minimum longitude for candidate points.

max_lon: float = 0.0: Maximum longitude for candidate points. If max_lon and min_lon are equal then these arguments are ignored.

regional: bool = False: Should lat-lon grid be periodic in longitude.

output_commands: list[TEOutputCommand] | None = None: Criteria for any additional columns to be added to the output. Criteria are provided as a list of separate TEOutputCommand criteria.

class tctrack.tempest_extremes.TEStitchParameters(output_dir='', output_file='trajectories.txt', in_file=None, in_list=None, in_fmt=None, allow_repeated_times=False, caltype='standard', time_begin=None, time_end=None, max_sep=5, max_gap=0, min_time=1, min_endpoint_dist=0, min_path_dist=0, threshold_filters=None, prioritize=None, add_velocity=False, out_file_format='gfdl', out_seconds=False)[source]

Bases: TCTrackerParameters

Dataclass containing values used by the stitch operation of TE.

References

TempestExtremes Documentation and the StitchNodes Source

output_dir: str = '': File path of the directory to output intermediate files. If left empty, it will use the TEDetectParameters.output_dir value if one is provided. Otherwise, it will use a temporary directory that will last only for the lifetime of the TETracker instance.

output_file: str = 'trajectories.txt': The output filename to write the trajectories to in the output_dir directory.

in_file: str | None = None: Filepath of the DetectNodes output file. If this and in_list are None, it will be determined from TEDetectParameters.output_dir and TEDetectParameters.output_file. Called “in” in TempestExtremes.

in_list: str | None = None: File containing a list of input files to be processed together. This is unadvised to use at present as it is likely to be changed.

in_fmt: list[str] | None = None: List of the variables in the order they appear in the input file. If None, it will be ["lon", "lat", ...], ending in variables defined in TEDetectParameters.output_commands.

allow_repeated_times: bool = False: If False, an error is thrown if there are multiple sections in the input nodefile with the same time.

caltype: str = 'standard': The type of calendar to use. Options are "standard" (365 days with leap years), "noleap", "360_day".

time_begin: str | None = None: Starting date/time for stitching trajectories. Earlier times will be ignored.

time_end: str | None = None: Ending date/time for stitching trajectories. Later times will be ignored.

max_sep: float = 5: The maximum distance allowed between candidates (degrees). Called “range” in TempestExtremes.

max_gap: int | str = 0: The number of missing points allowed between candidates, as an integer. Or as a string for the maximum time (inclusive) between points, e.g. "24hr".

min_time: int | str = 1: The minimum required length of a path. Either as an integer for the number of candidates, or a string for total duration, e.g. "24h".

min_endpoint_dist: float = 0: The minimum required distance between the first and last candidates (degrees).

min_path_dist: float = 0: The minimum required acumulated distance along the path (degrees).

threshold_filters: list[TEThreshold] | None = None: Filters for paths based on the number of nodes that satisfy a threshold. Uses a list of TEThreshold objects. Called “thresholdcmd” in TempestExtremes.

prioritize: str | None = None: The variable to use to determine the precedence (lowest to highest) of nodes for matching to the next position.

add_velocity: bool = False: Whether to include the velocity components (m/s) of the movement of the TC to the output file.

out_file_format: str = 'gfdl': Format of the output file. "gfdl", "csv", or "csvnohead". See TETracker.stitch() for details.

out_seconds: bool = False: For GFDL output file types, determines whether to report the sub-daily time in seconds (True) or hours (False).

tctrack.tempest_extremes.parameter_set(name, nc_names=None)[source]

Return a default TempestExtremes parameter set.

Returns:

Detection and stitching parameters.

Return type:

tuple[TEDetectParameters, TEStitchParameters]

Parameters:

name (str) –
The parameter set to return. Valid values are:
- "default" or "UZ" for parameter_set_uz().
- "OWZ" for parameter_set_owz().
nc_names (dict[str, str]) – An optional dictionary of netcdf variable names to use instead of the defaults. The defaults typically correspond to ECMWF/ERA5 names or CMIP names where these don’t exist. See the individual parameter_set functions for more details.

Raises:

ValueError – If name is not a supported parameter set.

tctrack.tempest_extremes.parameter_set_uz(nc_names)[source]

Return parameters for the UZ algorithm for detecting tropical cyclones.

Presented in [Ullrich2017].

This uses sea-level pressure for detection

Parameters:

nc_names (dict[str, str]) –

An optional dictionary of NetCDF variable names.

The dictionary keys match the default variable names, which are shown in the table below. The difference in geopotential (height) uses the 300 and 500 hPa levels by default, but can be overridden, for example: {"zgdiff": "_DIFF(zg(250hPa),zg(500hPa))"}.

By default geopotential height is used. But if either z or zdiff are set it will use geopotential instead.

NetCDF variable name defaults
Variable	Key / Default	Notes
Longitude	`longitude`
Latitude	`latitude`
Sea-level pressure	`msl`
Surface elevation	`orog`
Surface wind speed	`si10`
Geopotential height	`zg`	Used to calculate `zgdiff` at 300 hPa and 500 hPa.
Geopotential height difference	`zgdiff`	Default: `_DIFF(zg(300hPa),zg(500hPa))`.
Geopotential	`z`	Used to calculate `zdiff` at 300 hPa and 500 hPa.
Geopotential difference	`zdiff`	Default: `_DIFF(z(300hPa),z(500hPa))`.

Return type:

tuple[TEDetectParameters, TEStitchParameters]

tctrack.tempest_extremes.parameter_set_owz(nc_names)[source]

Return parameters for the OWZ algorithm for detecting tropical cyclones.

Presented in [Tory2013]. Adapted to TempestExtremes in [Bourdin2022].

This performs detection using the Okubo-Weiss-Zeta (OWZ) parameter which should be precomputed:

\[OWZ = \eta \ \mathrm{sign}(f) \times \max\left(\frac{\zeta^2 - (E^2 + F^2)}{\zeta^2}, 0\right)\]

where \(\eta\) is the absolute vorticity, \(\zeta\) is the sum of the relative vorticity, \(f\) is the coriolis parameter, and \(E\) and \(F\) are the stretching and the shearing deformation, given by

\[E = \frac{\partial u}{\partial x} - \frac{\partial v}{\partial y}, \quad F = \frac{\partial v}{\partial x} + \frac{\partial u}{\partial y}.\]

Returns:

Detection and stitching parameters.

Return type:

tuple[TEDetectParameters, TEStitchParameters]

Parameters:

nc_names (dict[str, str]) –

An optional dictionary of NetCDF variable names.

The dictionary keys match the default variable names, which are shown in the table below. Some variables require specific vertical levels. These can also be overriden, for example if there is data for OWZ at 900 hPa but not 850 hPa: {"owz850": "owz(900hPa)"}.

NetCDF variable name defaults
Variable	Key / Default	Levels	Notes
Longitude	`longitude`
Latitude	`latitude`
OWZ parameter	`owz`	`owz850` 850 hPa `owz500` 500 hPa
Relative humidity	`r`	`r950` 950 hPa `r700` 700 hPa
Specific humidity	`q`	`q950` 950 hPa
Relative vorticity	`vo`	`vo850` 850 hPa
Wind components	`u` / `v`	`u925` / `v925` 925 hPa `u850` / `v850` 850 hPa `u200` / `v200` 200 hPa	Not neeeded if `ws` and `vws` are set.
Wind speed	`ws`	`ws925` 925 hPa	If not set, calculated from `u925` / `v925`.
Vertical wind shear	`vws`		If not set, calculated from `u850` / `v850` and `u200` / `v200`.
Surface wind components (10m)	`u10` / `v10`		Not neeeded if `si10` is set.
Surface wind speed (10m)	`si10`		If not set, calculated from `u10` / `v10`.
Sea-level pressure	`msl`

class tctrack.tempest_extremes.TEContour[source]

Bases: TypedDict

Data required for checking a contour of a single variable during detection.

Points will be eliminated in a detection search if they fail this criterion. The closed contour is determined by breadth first search: if any paths exist from the candidate point (or nearby minima/maxima if minmaxdist is specified) that reach the specified distance before achieving the specified delta then we say no closed contour is present. Each contour takes the form of a dict with keys "var", "delta", "dist", and "minmaxdist".

See also

TETracker.detect: The DetectNodes call from the TETracker object
TEDetectParameters: The detection parameter class

References

TempestExtremes Documentation and the DetectNodes Source

Examples

To add a contour requirement on "psl" with a change of 200.0 within 5.5 degrees of the candidate we create a TEContour as follows:

>>> TEContour(var="psl", delta=200.0, dist=5.5, minmaxdist=0.0)
{'var': 'psl', 'delta': 200.0, 'dist': 5.5, 'minmaxdist': 0.0}

var: str: Name of the variable to contour in NetCDF files.

delta: float: Amount by which the field must change from the pivot value. If positive (negative) the field must increase (decrease) by this value along the contour.

dist: float: Lesser-circle radius (degrees) from the pivot within which the criteria must be satisfied.

minmaxdist: float: Lesser-circle radius away from the candidate to search for the minima/maxima. If delta is positive (negative), the pivot is a local minimum (maximum).

class tctrack.tempest_extremes.TEDetectThreshold[source]

Bases: TypedDict

Data for a threshold filter during the detection of candidate points.

Any candidates that do not satisfy the threshold value within a certain distance are filtered out.

When calling TempestExtremes each filter is mapped to the form “var,op,value,dist” and multiple conditions are separated by “;”.

See also

TETracker.detect: The DetectNodes call from the TETracker object
TEDetectParameters: The detection parameter class

References

TempestExtremes Documentation and the DetectNodes Source

Examples

To add a filter requiring surface windspeed "sfcWind" to be greater than 16 m/s somewhere within 1 degree of the centre:

>>> TEDetectThreshold(var="sfcWind", op=">", value=16, dist=1)
{"var": "sfcWind", "op": ">", "value": 16, "dist": 1},

var: str: Name of the variable being tested.

op: str: Operator used for the comparison (options include >,>=,<,<=,=,!=).

value: float: Value on the right-hand-side of the comparison.

dist: float | str: Great-circle distance within which the threshold must be satisfied.

class tctrack.tempest_extremes.TEOutputCommand[source]

Bases: TypedDict

Data required to specify an additional column in the detection output.

Each output command takes the form of a dict with keys "var", "op", and "dist".

See also

TETracker.detect: The DetectNodes call from the TETracker object
TEDetectParameters: The detection parameter class

References

TempestExtremes Documentation and the DetectNodes Source

Examples

To add an output column with the minimum "psl" at the candidate point:

>>> TEOutputCommand(var="psl", operator="min", dist=0.0)
{'var': 'psl', 'operator': 'min', 'dist': 0.0}

var: str: Name of the variable to write in output files.

operator: str: Operator that is applied over all points within the specified distance of the candidate (options include "max", "min", "avg", "maxdist", and "mindist").

dist: float: Lesser-circle radius (degrees) from the candidate within which the operator is applied.

class tctrack.tempest_extremes.TEThreshold[source]

Bases: TypedDict

Data required for a threshold filter for a track trajectory when stitching.

Any storm track trajectories that do not satisfy the threshold value for a given number of points will be filtered out. Each condition is of the form “var,op,value,count” and multiple conditions are separated by “;”.

If using latitude or longitude these should be specified as "lat" or "lon" regardless of their names in the input files.

See also

TETracker.stitch: The StitchNodes call from the TETracker object.
TEStitchParameters: The stitching parameter class.

References

TempestExtremes Documentation and the StitchNodes Source

Examples

To add a filter requiring latitude "lat" to be less than 40 degrees for 10 or more points in each track trajectory:

>>> TEThreshold(var="lat", op="<=", value=40, count=10)
{"var": "lat", "op": "<=", "value": 40, "count": 10},

var: str: Name of the variable being tested. Called “col” in TempestExtremes.

op: str: Operator used for the comparison (options include >,>=,<,<=,=,!=,|>=,|<=).

value: float: Value on the right-hand-side of the comparison.

count: int | str: Either the minimum number of points where the threshold must be satisfied or the instruction "all", "first", or "last". "all" for all points along the path, "first" for just the first point, and "last" for only the last point.