climix issueshttps://git.smhi.se/climix/climix/-/issues2023-09-13T12:23:39Zhttps://git.smhi.se/climix/climix/-/issues/311Write tests for index functions FirstOccurrence and LastOccurrence2023-09-13T12:23:39ZCarolina NilssonWrite tests for index functions FirstOccurrence and LastOccurrenceFirstOccurence and LastOccurrence index functions needs tests.FirstOccurence and LastOccurrence index functions needs tests.0.19 (Poco Mas)Carolina NilssonCarolina Nilssonhttps://git.smhi.se/climix/climix/-/issues/306Update clix-meta files to v0.5.2 when ready2023-04-21T15:56:13ZLars BärringUpdate clix-meta files to v0.5.2 when readyThe new maintenance version of clix-meta will soon be ready, and it would be good to have it in Climix v.0.18.The new maintenance version of clix-meta will soon be ready, and it would be good to have it in Climix v.0.18.0.18 (Gull Olle)Joakim LöwJoakim Löwhttps://git.smhi.se/climix/climix/-/issues/304Modernize build infrastructure2023-04-19T15:05:32ZKlaus ZimmermannModernize build infrastructureWe rely on traditional `setup.py`, `setup.cfg`, and various tool specific config files. We should follow modern Python packaging standards and unify things in `pyproject.toml`.We rely on traditional `setup.py`, `setup.cfg`, and various tool specific config files. We should follow modern Python packaging standards and unify things in `pyproject.toml`.0.18 (Gull Olle)Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/299Add unit tests for index functions2023-04-21T15:56:13ZCarolina NilssonAdd unit tests for index functionsImplemented tests:
- [x] CountLevelCrossings
- [x] CountOccurrences
- [x] CountJointOccurrencesPrecipitationTemperature
- [x] CountJointOccurrencesTemperature
- [x] DiurnalTemperatureRange
- [x] ExtremeTemperatureRange
- [ ] FirstOccurre...Implemented tests:
- [x] CountLevelCrossings
- [x] CountOccurrences
- [x] CountJointOccurrencesPrecipitationTemperature
- [x] CountJointOccurrencesTemperature
- [x] DiurnalTemperatureRange
- [x] ExtremeTemperatureRange
- [ ] FirstOccurrence (moved to later release)
- [x] InterdayDiurnalTemperatureRange
- [ ] LastOccurrence (moved to later release)
- [x] Percentile
- [x] Statistics
- [x] ThresholdedPercentile
- [x] ThresholdedStatistics
- [x] RunningStatistics
- [x] ThresholdedRunningStatistics
- [x] TemperatureSum
The following unit tests needs to be reviewed from a scientific point of view:
* [x] CountLevelCrossings - Gustav
* [ ] CountOccurrences - Gustav
* [ ] CountJointOccurrencesPrecipitationTemperature
* [ ] CountJointOccurrencesTemperature
* [x] DiurnalTemperatureRange - Renate
* [x] ExtremeTemperatureRange - Renate
* [x] InterdayDiurnalTemperatureRange - Renate
* [ ] Statistics
* [x] ThresholdedStatistics - Renate
* [ ] RunningStatistics (complex output with post-processing included)
* [ ] ThresholdedRunningStatistics (complex output with post-processing included)
* [ ] TemperatureSum
E.g.,
```
TEST_COUNT_LEVEL_CROSSINGS_PARAMETERS = [
(
{"data": (-1) * np.arange(12).reshape(2, 2, 3), "units": "degree_Celsius"},
{"data": np.arange(12).reshape(2, 2, 3), "units": "degree_Celsius"},
{"data": 0, "units": "degree_Celsius", "standard_name": "air_temperature"},
np.array([[1, 2, 2], [2, 2, 2]]),
), # ordinary np
]
parameter_names = "f_cube_tasmin, f_cube_tasmax, f_first_threshold, expected"
```
The index function is count_level_crossings, bellow the parameter list with the data we can see the parameter names. Such that,
```
f_cube_tasmin = {"data": (-1) * np.arange(12).reshape(2, 2, 3), "units": "degree_Celsius"}
f_cube_tasmax = {"data": np.arange(12).reshape(2, 2, 3), "units": "degree_Celsius"}
f_first_threshold = {"data": 0, "units": "degree_Celsius", "standard_name": "air_temperature"}
expected = np.array([[1, 2, 2], [2, 2, 2]])
```0.18 (Gull Olle)Carolina NilssonCarolina Nilssonhttps://git.smhi.se/climix/climix/-/issues/298Update clix-meta files to v0.5.12023-03-14T09:52:37ZJoakim LöwUpdate clix-meta files to v0.5.1Replace `index_definitions.yml` and `variables.yml` with files released with clix-meta v0.5.1 and update Climix to handle it.Replace `index_definitions.yml` and `variables.yml` with files released with clix-meta v0.5.1 and update Climix to handle it.0.18 (Gull Olle)Joakim LöwJoakim Löwhttps://git.smhi.se/climix/climix/-/issues/296Index requests of combined threshold indices (2 variables)2023-09-12T09:20:54ZRenate WilckeIndex requests of combined threshold indices (2 variables)New indices:
I need three new combined threshold indices.
1. Number of days(Temperature between two values (-2, 2) and precipitation above 0.1 mm/d)
2. ~~Number of days(Temperature below 0 and precipitation above 0.1 mm/d)~~
3. Akkumulat...New indices:
I need three new combined threshold indices.
1. Number of days(Temperature between two values (-2, 2) and precipitation above 0.1 mm/d)
2. ~~Number of days(Temperature below 0 and precipitation above 0.1 mm/d)~~
3. Akkumulated precip with Temperature below 0
I don't have names for them. If you like to get suggestions, let me know. ~~There is no rush, though it would be nice to work with them after summer.~~ Now is after summer and I would need those.https://git.smhi.se/climix/climix/-/issues/294Function to parse date and timerange in iso 8601 format2023-04-19T15:05:32ZJoakim LöwFunction to parse date and timerange in iso 8601 formatDates and time ranges will be required to be given in iso 8601 format in index definition yaml (reference period, see #273) and for command line arguments (reference period and computational period, see #289).
The end date of a time ran...Dates and time ranges will be required to be given in iso 8601 format in index definition yaml (reference period, see #273) and for command line arguments (reference period and computational period, see #289).
The end date of a time range should be interpreted as the upper bound of the date, i.e. `1961/1990` should mean `1961-01-01T00:00:00` to `1991-01-01T00:00:00` and `1961-01/1990-01` should mean `1961-01-01T00:00:00` to `1990-02-01T00:00:00` (see https://git.smhi.se/climix/climix/-/issues/273#note_34689)
The time range parser should have support for duration as start or end, i.e. `P20Y/2100` should produce same as `2081/2100`.0.17 (Black Lion)Joakim LöwJoakim Löwhttps://git.smhi.se/climix/climix/-/issues/291Make use of guvectorize in spell kernels2023-02-24T07:32:52ZErik HolmgrenMake use of guvectorize in spell kernelsTracking the progress of the investigation, and possible implementation, of using `@guvectorize` for `chunk_column` in spell kernels.
Essentially, this means that we could remove the loop over the cube dimensions in `chunk` and let numb...Tracking the progress of the investigation, and possible implementation, of using `@guvectorize` for `chunk_column` in spell kernels.
Essentially, this means that we could remove the loop over the cube dimensions in `chunk` and let numba vectorize it.Erik HolmgrenErik Holmgrenhttps://git.smhi.se/climix/climix/-/issues/290Handeling of non-CMORized input data2023-11-13T12:07:05ZCarolina NilssonHandeling of non-CMORized input dataIn issue #15 there was a discussion on how climix would handle input data that is not CMORized, i.e., does not follows the CF-conventions and CMIP standards. This is moved to a separate issue since a decision needs to be taken about if ...In issue #15 there was a discussion on how climix would handle input data that is not CMORized, i.e., does not follows the CF-conventions and CMIP standards. This is moved to a separate issue since a decision needs to be taken about if we should rely on running some other tool first (ESMValTool) or if we should implement some solution on our own possibly using ESMValTools API or similar.https://git.smhi.se/climix/climix/-/issues/289Command line option to specify computational period2023-12-12T09:57:28ZJoakim LöwCommand line option to specify computational periodAdd a command line option to specify the period to perform index calculation on. Reasons:
1. User may only want to compute the index for a sub period of the data.
2. Beginning and end of input data may be incomplete with respect to time ...Add a command line option to specify the period to perform index calculation on. Reasons:
1. User may only want to compute the index for a sub period of the data.
2. Beginning and end of input data may be incomplete with respect to time period. E.g. if 'annual' and first and/or last year of data does not contain entries for all days of year, climix may raise error for some indices.
See also #273 and #2570.20 (Urbane Goat)Carolina NilssonCarolina Nilssonhttps://git.smhi.se/climix/climix/-/issues/287Produce a nice summary of which indices are available in climix2023-04-23T13:41:47ZLars BärringProduce a nice summary of which indices are available in climixUsers have requested to have access, possibly published via SMHI internal web pages to which indices that climix can produce.
A starting point may be the output from `climix -x list`:
``` bash
~ >climix -x list
INFO:root:Activating sen...Users have requested to have access, possibly published via SMHI internal web pages to which indices that climix can produce.
A starting point may be the output from `climix -x list`:
``` bash
~ >climix -x list
INFO:root:Activating sentry (automatic error reporting)
INFO:root:Looking for metadata in directory /home/sm_lbarr/CODE/climix/climix/etc
INFO:root:Looking for metadata in directory /etc/climix
INFO:root:Looking for metadata in directory /home/sm_lbarr/.config/climix
INFO:root:Reading index definitions from file /home/sm_lbarr/CODE/climix/climix/etc/metadata.yml
INFO:root:Reading index definitions from file /home/sm_lbarr/CODE/climix/climix/etc/variables.yml
INFO:root:Reading index definitions from file /home/sm_lbarr/CODE/climix/climix/etc/index_definitions.yml
Available indices are:
['fd', 'tnlt2', 'tnltm2', 'tnltm20', 'id', 'su', 'txge30', 'txge35', 'tr', 'tmge5', 'tmlt5', 'tmge10', 'tmlt10',
'tngt{TT}', 'tnlt{TT}', 'tnge{TT}', 'tnle{TT}', 'txgt{TT}', 'txlt{TT}', 'txge{TT}', 'txle{TT}', 'tmgt{TT}',
'tmlt{TT}', 'tmge{TT}', 'tmle{TT}', 'ctngt{TT}', 'cfd', 'csu', 'ctnlt{TT}', 'ctnge{TT}', 'ctnle{TT}', 'ctxgt{TT}',
'ctxlt{TT}', 'ctxge{TT}', 'ctxle{TT}', 'ctmgt{TT}', 'ctmlt{TT}', 'ctmge{TT}', 'ctmle{TT}', 'txx', 'tnx', 'txn', '
tnn', 'txm', 'tnm', 'tmx', 'tmn', 'tmm', 'txmax', 'tnmax', 'txmin', 'tnmin', 'txmean', 'tnmean', 'tmmax', 'tmmin',
'tmmean', 'tn10p', 'tx10p', 'tn90p', 'tx90p', 'tg10p', 'tg90p', 'txgt50p', 'txgt{PRC}p', 'tngt{PRC}p', 'tmgt{PRC}p',
'txlt{PRC}p', 'tnlt{PRC}p', 'tmlt{PRC}p', 'dtr', 'vdtr', 'etr', 'tx{PRC}pctl', 'tn{PRC}pctl', 'tm{PRC}pctl',
'hd17', 'hddheat{TT}', 'ddgt{TT}', 'cddcold{TT}', 'ddlt{TT}', 'gddgrow{TT}', 'gd4', 'r10mm', 'r20mm', 'r{RT}mm',
'wetdays', 'rr1', 'cdd', 'cwd', 'prcptot', 'sdii', 'r{PRC}pctl', 'r{PRC}pDAYS', 'rx1day', 'rx5day', 'rx{ND}day',
'rh', 'rr', 'pp', 'tg', 'tn', 'tx', 'sd', 'sd1', 'sd5cm', 'sd50cm', 'sd{D}cm', 'ss', 'fxx', 'fg6bft', 'fgcalm',
'fg', 'nzero', 'maxdtr']
```
This can of course be expanded in various directions, e.g. to
* list the indices per file that is scanned,
* also give the `long_name`, `OUTPUT_unit`, input variable(s), ...
* see #57 for another and much more complex idea0.18 (Gull Olle)Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/284Amend the yaml reader function to be aware of the CLIX-META version number2023-04-19T15:05:32ZLars BärringAmend the yaml reader function to be aware of the CLIX-META version number0.18 (Gull Olle)Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/275Add integration tests using NGCD dataset2023-04-21T15:56:13ZJoakim LöwAdd integration tests using NGCD datasetAdd integration tests running climix on the NGCD dataset, and compare results with NGCD reference indicators.Add integration tests running climix on the NGCD dataset, and compare results with NGCD reference indicators.0.18 (Gull Olle)Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/273Reference period definition in index_definitions.yml2023-04-19T15:05:32ZCarolina NilssonReference period definition in index_definitions.ymlTo conclude after offline conversation. Some index definitions utilises a reference period in the definition for the calculation and there is no obvious way in how this should be handled. A suggestion would be to included the reference p...To conclude after offline conversation. Some index definitions utilises a reference period in the definition for the calculation and there is no obvious way in how this should be handled. A suggestion would be to included the reference period in the clix-meta index definition, given the following example:
```
index_function:
name: count_thresholded_percentile_occurrences
parameters:
data_threshold:
kind: quantity
standard_name: lwe_precipitation_rate
long_name: "Wet day threshold"
data: 1
units: "mm day-1"
data_condition:
kind: operator
operator: ">="
percentile:
kind: quantity
standard_name:
proposed_standard_name: quantile
long_name: "Percentile value"
data: "{PRC}"
units: "%"
percentile_condition:
kind: operator
operator: ">"
reference_period:
kind: time_period
data: ['1961-01-01 00:00:00', '1991-01-01 00:00:00']
```
the reference period could then be used in the index calculation and the output should contain a global attribute giving information about which reference period that was used.
```
global attributes:
:reference_period = "1961-01-01 00:00:00 to 1991-01-01 00:00:00"
```
or something similar.
Notes there were also a discussion about having an additional Auxiliary coordinate variable with the following information:
```
reference_period:(var_name: "reference_period", standard_name: "reference_epoch", value: mean(n1,n2), bounds: [n1, n2], unit: "days since 1961-01-01 00:00:00")
```
where n1 and n2 are the point values for the start and end of the time period defined by the unit.0.18 (Gull Olle)Carolina NilssonCarolina Nilssonhttps://git.smhi.se/climix/climix/-/issues/272tn10p, tn90p, tx10p, tx90p with NGCD data results in ZeroDivisionError2023-01-30T14:51:02ZJoakim Löwtn10p, tn90p, tx10p, tx90p with NGCD data results in ZeroDivisionErrorCalculating `tn10p`, `tn90p`, `tx10p` and `tx90p` with input from NGCD dataset results in a `ZeroDivisionError`. E.g.:
```
climix -v -e -x tn10p /home/rossby/joint_exp/climix/1/testdata/NGCD/input_data/SverigeScaled/NGCD_TN_type2_*.nc
``...Calculating `tn10p`, `tn90p`, `tx10p` and `tx90p` with input from NGCD dataset results in a `ZeroDivisionError`. E.g.:
```
climix -v -e -x tn10p /home/rossby/joint_exp/climix/1/testdata/NGCD/input_data/SverigeScaled/NGCD_TN_type2_*.nc
```
generates the following traceback:
```
Traceback (most recent call last):
File "/home/sm_joalo/.conda/envs/climix-devel/bin/climix", line 8, in <module>
sys.exit(main())
File "/home/sm_joalo/dev/repos/climix/climix/main.py", line 293, in main
do_main(
File "/home/sm_joalo/dev/repos/climix/climix/main.py", line 271, in do_main
result = index(input_data, client=scheduler.client, sliced_mode=sliced_mode)
File "/home/sm_joalo/dev/repos/climix/climix/index.py", line 24, in __call__
self.index_function.preprocess(cubes, client)
File "/home/sm_joalo/dev/repos/climix/climix/index_functions/percentile_functions.py", line 232, in preprocess
all_data = all_data.rechunk(("auto",) * (all_data.ndim - 1) + (-1,))
File "/home/sm_joalo/.conda/envs/climix-devel/lib/python3.10/site-packages/dask/array/core.py", line 2745, in rechunk
return rechunk(self, chunks, threshold, block_size_limit, balance)
File "/home/sm_joalo/.conda/envs/climix-devel/lib/python3.10/site-packages/dask/array/rechunk.py", line 297, in rechunk
chunks = normalize_chunks(
File "/home/sm_joalo/.conda/envs/climix-devel/lib/python3.10/site-packages/dask/array/core.py", line 3073, in normalize_chunks
chunks = auto_chunks(chunks, shape, limit, dtype, previous_chunks)
File "/home/sm_joalo/.conda/envs/climix-devel/lib/python3.10/site-packages/dask/array/core.py", line 3206, in auto_chunks
multiplier = _compute_multiplier(limit, dtype, largest_block, result)
File "/home/sm_joalo/.conda/envs/climix-devel/lib/python3.10/site-packages/dask/array/core.py", line 3119, in _compute_multiplier
limit
ZeroDivisionError: float division by zero
```Joakim LöwJoakim Löwhttps://git.smhi.se/climix/climix/-/issues/271rx5day index fails with NGCD input2023-01-30T14:51:14ZJoakim Löwrx5day index fails with NGCD inputRunning the following command line:
```
climix -v -e -x rx5day /home/rossby/joint_exp/climix/1/testdata/NGCD/input_data/SverigeScaled/NGCD_RR_type2_*.nc
```
results in the following traceback:
```
Traceback (most recent call last):
Fil...Running the following command line:
```
climix -v -e -x rx5day /home/rossby/joint_exp/climix/1/testdata/NGCD/input_data/SverigeScaled/NGCD_RR_type2_*.nc
```
results in the following traceback:
```
Traceback (most recent call last):
File "/home/sm_joalo/.conda/envs/climix-devel/bin/climix", line 8, in <module>
sys.exit(main())
File "/home/sm_joalo/dev/repos/climix/climix/main.py", line 293, in main
do_main(
File "/home/sm_joalo/dev/repos/climix/climix/main.py", line 271, in do_main
result = index(input_data, client=scheduler.client, sliced_mode=sliced_mode)
File "/home/sm_joalo/dev/repos/climix/climix/index.py", line 52, in __call__
aggregated = multicube_aggregated_by(
File "/home/sm_joalo/dev/repos/climix/climix/iris.py", line 126, in multicube_aggregated_by
result = list(map(agg, groupby_subcubes))
File "/home/sm_joalo/dev/repos/climix/climix/iris.py", line 109, in agg
result = aggregate(data, axis=dimension_to_groupby, cube=cube, **kwargs)
File "/home/sm_joalo/.conda/envs/climix-devel/lib/python3.10/site-packages/iris/analysis/__init__.py", line 549, in lazy_aggregate
return self.lazy_func(data, axis=axis, **kwargs)
File "/home/sm_joalo/dev/repos/climix/climix/index_functions/index_functions.py", line 449, in lazy_func
return super().call_func(thresholded_data, axis, **kwargs)
File "/home/sm_joalo/dev/repos/climix/climix/index_functions/index_functions.py", line 354, in call_func
rolling_view = np.lib.stride_tricks.sliding_window_view(
File "<__array_function__ internals>", line 180, in sliding_window_view
File "/home/sm_joalo/.conda/envs/climix-devel/lib/python3.10/site-packages/dask/array/core.py", line 1760, in __array_function__
return da_func(*args, **kwargs)
File "/home/sm_joalo/.conda/envs/climix-devel/lib/python3.10/site-packages/dask/array/overlap.py", line 815, in sliding_window_view
safe_chunks = tuple(
File "/home/sm_joalo/.conda/envs/climix-devel/lib/python3.10/site-packages/dask/array/overlap.py", line 816, in <genexpr>
ensure_minimum_chunksize(d + 1, c) for d, c in zip(depths, x.chunks)
File "/home/sm_joalo/.conda/envs/climix-devel/lib/python3.10/site-packages/dask/array/overlap.py", line 353, in ensure_minimum_chunksize
raise ValueError(
ValueError: The overlapping depth 5 is larger than your array 1.
```Joakim LöwJoakim Löwhttps://git.smhi.se/climix/climix/-/issues/269Update clix-meta files to clix-meta release 0.4.12022-12-09T13:30:11ZJoakim LöwUpdate clix-meta files to clix-meta release 0.4.1The YAML files from Clix-meta need to be updated to the ones in the latest Clix-meta (0.4.1), which contains a fix for a bug that caused some indices to get the wrong condition operator.The YAML files from Clix-meta need to be updated to the ones in the latest Clix-meta (0.4.1), which contains a fix for a bug that caused some indices to get the wrong condition operator.0.16https://git.smhi.se/climix/climix/-/issues/266Add option to calculate index with annual single month2023-01-25T16:38:05ZJoakim LöwAdd option to calculate index with annual single monthSpecifying monthly range will be possible with #95, but there is no way to specify a single month because of ambiguity in the initial letter of the months (see https://git.smhi.se/climix/climix/-/issues/255#note_31992). Add option to use...Specifying monthly range will be possible with #95, but there is no way to specify a single month because of ambiguity in the initial letter of the months (see https://git.smhi.se/climix/climix/-/issues/255#note_31992). Add option to use single month ranges by specifying the first three letters in a month. Examples:
```
-p annual[feb]
-p annual[oct]
```0.16Joakim LöwJoakim Löwhttps://git.smhi.se/climix/climix/-/issues/2643. Handle the situation where climix cannot merge and we do not know what the...2023-01-25T16:41:53ZCarolina Nilsson3. Handle the situation where climix cannot merge and we do not know what the problem is.https://git.smhi.se/climix/climix/-/issues/259Occasional 'Heartbeat to scheduler failed' error near end of execution2022-10-24T13:03:10ZJoakim LöwOccasional 'Heartbeat to scheduler failed' error near end of executionI get an occasional error when running climix locally (have not tried to reproduce with mpi/srun on multiple nodes). It seems to happen after output file has been written. I get an output `distributed.worker - WARNING - Heartbeat to sche...I get an occasional error when running climix locally (have not tried to reproduce with mpi/srun on multiple nodes). It seems to happen after output file has been written. I get an output `distributed.worker - WARNING - Heartbeat to scheduler failed` ... `tornado.iostream.StreamClosedError: Stream is closed`.
Full log (info):
```
$ climix -v -e -x dtr -o /nobackup/rossby27/users/sm_joalo/data/test_climix/mpi_221019/output/\{var_name\}_\{frequency\}.nc /nobackup/rossby27/users/sm_joalo/data/test_climix/mpi_221019/tasmin_EUR-11_MPI-M-MPI-ESM-LR_historical_r1i1p1_SMHI-RCA4_v1a_day_20010101-20051231.nc /nobackup/rossby27/users/sm_joalo/data/test_climix/mpi_221019/tasmax_EUR-11_MPI-M-MPI-ESM-LR_historical_r1i1p1_SMHI-RCA4_v1a_day_20010101-20051231.nc
2470ms:main.py:main() INFO:root:Loading metadata
4771ms:main.py:main() INFO:root:Scheduler ready; starting main program.
4771ms:main.py:do_main() INFO:root:Preparing indices
4814ms:metadata.py:build_index_function() INFO:root:Found implementation for index_function <diurnal_temperature_range> from distribution <<importlib.metadata.PathDistribution object at 0x2aecf40cdf90>>
4823ms:main.py:do_main() INFO:root:Starting calculations for index <climix.index.Index object at 0x2aeceb0a5ab0>
4823ms:main.py:do_main() INFO:root:Building output filename
4823ms:main.py:do_main() INFO:root:Preparing input data
/home/sm_joalo/dev/repos/climix/climix/datahandling.py:59: FutureWarning: Ignoring a datum in netCDF load for consistency with existing behaviour. In a future version of Iris, this datum will be applied. To apply the datum when loading, use the iris.FUTURE.datum_support flag.
datacubes = iris.load_raw(datafiles, callback=ignore_cb)
/home/sm_joalo/dev/repos/climix/climix/datahandling.py:59: FutureWarning: Ignoring a datum in netCDF load for consistency with existing behaviour. In a future version of Iris, this datum will be applied. To apply the datum when loading, use the iris.FUTURE.datum_support flag.
datacubes = iris.load_raw(datafiles, callback=ignore_cb)
17320ms:main.py:do_main() INFO:root:Calculating index
17320ms:index.py:__call__() INFO:root:Starting preprocess
17321ms:index.py:__call__() INFO:root:Finished preprocess
17321ms:index.py:__call__() INFO:root:Data found for input low_data
17321ms:index.py:__call__() INFO:root:Data found for input high_data
17321ms:index.py:__call__() INFO:root:Adding coord categorisation.
18909ms:index.py:__call__() INFO:root:Preparing cubes
18909ms:index.py:__call__() INFO:root:Setting up aggregation
19178ms:aggregators.py:compute_pre_result() INFO:root:Setting up pre-result in aggregate mode
19198ms:aggregators.py:compute_pre_result() INFO:root:Setup completed in 0
19198ms:main.py:do_main() INFO:root:Saving result
19199ms:datahandling.py:save() INFO:climix.datahandling:Storing non-iteratively
19199ms:datahandling.py:save() INFO:climix.datahandling:Computing result
[########################################] | 100% Completed | 4.3s
23548ms:datahandling.py:save() INFO:climix.datahandling:Storing result
23624ms:datahandling.py:save() INFO:climix.datahandling:Calculation complete
23627ms:main.py:main() INFO:root:Calculation took 18.8558 seconds.
2022-10-24 08:59:20,501 - distributed.worker - WARNING - Heartbeat to scheduler failed
Traceback (most recent call last):
File "/home/sm_joalo/.conda/envs/omni/lib/python3.10/site-packages/distributed/comm/tcp.py", line 225, in read
frames_nbytes = await stream.read_bytes(fmt_size)
tornado.iostream.StreamClosedError: Stream is closed
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/sm_joalo/.conda/envs/omni/lib/python3.10/site-packages/distributed/worker.py", line 1158, in heartbeat
response = await retry_operation(
File "/home/sm_joalo/.conda/envs/omni/lib/python3.10/site-packages/distributed/utils_comm.py", line 383, in retry_operation
return await retry(
File "/home/sm_joalo/.conda/envs/omni/lib/python3.10/site-packages/distributed/utils_comm.py", line 368, in retry
return await coro()
File "/home/sm_joalo/.conda/envs/omni/lib/python3.10/site-packages/distributed/core.py", line 1154, in send_recv_from_rpc
return await send_recv(comm=comm, op=key, **kwargs)
File "/home/sm_joalo/.conda/envs/omni/lib/python3.10/site-packages/distributed/core.py", line 919, in send_recv
response = await comm.read(deserializers=deserializers)
File "/home/sm_joalo/.conda/envs/omni/lib/python3.10/site-packages/distributed/comm/tcp.py", line 241, in read
convert_stream_closed_error(self, e)
File "/home/sm_joalo/.conda/envs/omni/lib/python3.10/site-packages/distributed/comm/tcp.py", line 144, in convert_stream_closed_error
raise CommClosedError(f"in {obj}: {exc}") from exc
distributed.comm.core.CommClosedError: in <TCP (closed) ConnectionPool.heartbeat_worker local=tcp://127.0.0.1:45440 remote=tcp://127.0.0.1:45998>: Stream is closed
```