climix issueshttps://git.smhi.se/climix/climix/-/issues2020-02-12T13:43:57Zhttps://git.smhi.se/climix/climix/-/issues/165Update changelog for release 0.11.02020-02-12T13:43:57ZKlaus ZimmermannUpdate changelog for release 0.11.00.11.0Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/164Update metadata with new master table and editor2020-02-12T13:13:23ZKlaus ZimmermannUpdate metadata with new master table and editorPartly due to #135, the master table has changed significantly.
This is reflected in changes to the editor in #161 and to climix in #163.
Finally, this needs to be put into the actual metadata, ie index and variable definitions.Partly due to #135, the master table has changed significantly.
This is reflected in changes to the editor in #161 and to climix in #163.
Finally, this needs to be put into the actual metadata, ie index and variable definitions.0.11.0Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/163Update climix to deal with new metadata characteristics2020-02-12T13:13:23ZKlaus ZimmermannUpdate climix to deal with new metadata characteristicsPartly due to #135 indices coming from the master table can now have empty `cell_methods` and parameters with `proposed_standard_names`. This has been integrated into the editor in #161, but needs to be reflected in climix itself before ...Partly due to #135 indices coming from the master table can now have empty `cell_methods` and parameters with `proposed_standard_names`. This has been integrated into the editor in #161, but needs to be reflected in climix itself before the changed definitions can be integrated.0.11.0Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/161Update editor to deal with new master table2020-02-12T13:13:23ZKlaus ZimmermannUpdate editor to deal with new master tableIn part stemming from the discussion in #135, the master table has undergone significant changes.
The editor component of climix needs to be adapted to this.In part stemming from the discussion in #135, the master table has undergone significant changes.
The editor component of climix needs to be adapted to this.0.11.0Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/162Improve metadata error reporting2020-02-12T12:01:01ZKlaus ZimmermannImprove metadata error reportingThe metadata error reporting could be improved. For example, if data is missing in the provided input data, we get to know which argument is missing, but not which variables are needed.The metadata error reporting could be improved. For example, if data is missing in the provided input data, we get to know which argument is missing, but not which variables are needed.0.11.0Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/155Input variables with alias names2020-02-07T13:18:30ZKlaus ZimmermannInput variables with alias namesIt is common to find input data that is perfectly acceptable but uses a different variable name.
It would be nice to be able to use this data unchanged.It is common to find input data that is perfectly acceptable but uses a different variable name.
It would be nice to be able to use this data unchanged.0.11.0Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/158Question: parameter long name2020-02-07T11:41:21ZLars BärringQuestion: parameter long name@a002160 I saw that there is a reasonable long name for the threshold parameter of `nzero`. Currently this have to be added manually in the yaml file. Could we devise a ways to have it already in the master table? This issue resurfaced n...@a002160 I saw that there is a reasonable long name for the threshold parameter of `nzero`. Currently this have to be added manually in the yaml file. Could we devise a ways to have it already in the master table? This issue resurfaced now when I am working on the running_statistics [and thresholded_running_statistics] index function. Here we have a parameter `window_size` of `kind: quantity` and `units: days`. And I cannot find a relevant standard name (other than `time`) that has the right canonical units. So, either we ...
* ... are stuck with adding this manually to the yaml file
* ... accept that the auxiliary coordinate will have neither standard_name nor long_name, only var_name "window_size"
* ... find a way to write this into in the master table
Any thoughts? For now, I would be reluctant to open a CF ticket to add a standard name involving anything resembling time/duration or similar...https://git.smhi.se/climix/climix/-/issues/143Update logging messages2020-02-05T11:56:08ZLars BärringUpdate logging messagesSuggestion is to update the logging message to include the following:
* Timing info (either wall clock or run time) --- to help identify which parts are slow.
* Source file and function/module --- would help me to better understand the...Suggestion is to update the logging message to include the following:
* Timing info (either wall clock or run time) --- to help identify which parts are slow.
* Source file and function/module --- would help me to better understand the code structure.0.11.0https://git.smhi.se/climix/climix/-/issues/115Machinery to "concatenate" or combine indices2020-02-05T07:49:53ZLars BärringMachinery to "concatenate" or combine indicesSome very useful and user-oriented indices can be viewed as a combination, or sequence, of simpler indices. E.g. the ETCCDI Growing Season Length (gsl), which is the difference between the End and Start of the Growing Season (which are i...Some very useful and user-oriented indices can be viewed as a combination, or sequence, of simpler indices. E.g. the ETCCDI Growing Season Length (gsl), which is the difference between the End and Start of the Growing Season (which are interesting in their own right. There are many other ones.....https://git.smhi.se/climix/climix/-/issues/149Clean up the master table (Libreoffice)2020-02-04T16:09:41ZLars BärringClean up the master table (Libreoffice)0.11.0Lars BärringLars Bärringhttps://git.smhi.se/climix/climix/-/issues/146nzero index2020-02-04T15:50:20ZKlaus Zimmermannnzero indexLars to specify yaml for the nzero index.Lars to specify yaml for the nzero index.0.12.0Lars BärringLars Bärringhttps://git.smhi.se/climix/climix/-/issues/151Index function for nzero index2020-02-04T15:46:33ZLars BärringIndex function for nzero index0.12.0Lars BärringLars Bärringhttps://git.smhi.se/climix/climix/-/issues/82Machinery to allow index_functions based on two or more input streams2020-02-04T15:38:45ZLars BärringMachinery to allow index_functions based on two or more input streamsSome of the more advanced indices are based on multiple input variables. E.g. dtr tn#tx# et al. all based on (tasmax,tasmin), and the ECAD ww,wd,cw,cd (tas, pr), and several more complex indices like spi, spei ....Some of the more advanced indices are based on multiple input variables. E.g. dtr tn#tx# et al. all based on (tasmax,tasmin), and the ECAD ww,wd,cw,cd (tas, pr), and several more complex indices like spi, spei ....0.11.0Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/152Improve input variable handling2020-02-04T14:16:46ZKlaus ZimmermannImprove input variable handlingAt the moment, all metadata is for input variables is replicated every time an index uses this input variable. Given that we only use a couple of different input variables, such as `tas`, `pr`, `tasmin`, and `tasmax`, this duplication of...At the moment, all metadata is for input variables is replicated every time an index uses this input variable. Given that we only use a couple of different input variables, such as `tas`, `pr`, `tasmin`, and `tasmax`, this duplication of metadata like cell methods and standard names is unnecessary.
To better deal with input variables and prepare for future extensions, we should add a new, separate registry of input variables and only refer to that where we need the input variables in the indices.0.11.0Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/145Climix currently cannot handle NGCD2020-01-24T15:59:34ZLars BärringClimix currently cannot handle NGCDclimix totally cannot handle NGCD.
* start file `/home/sm_lbarr/PROJ/test_climix/climix_run_fn_ngcd.sh`
* slurm log ` /home/sm_lbarr/PROJ/test_climix/slurm-10723857.out`
* output directory (now deleted) `/home/sm_lbarr//PROJ/test_clim...climix totally cannot handle NGCD.
* start file `/home/sm_lbarr/PROJ/test_climix/climix_run_fn_ngcd.sh`
* slurm log ` /home/sm_lbarr/PROJ/test_climix/slurm-10723857.out`
* output directory (now deleted) `/home/sm_lbarr//PROJ/test_climix/outdata/NGCD/slurm-10723857/` only contained the `dask-worker-space` subdirectory, which in turn contained several subdirectories each one holding **~150 000 (!!) files**Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/101Gitlab license upgrade2020-01-21T15:38:10ZLars BärringGitlab license upgradeInitiate dialogue to support TFO/Utveckling (Cecile) to upgrade Gitlab licene to get better functionalityInitiate dialogue to support TFO/Utveckling (Cecile) to upgrade Gitlab licene to get better functionality0.13Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/140Update changelog for release 0.10.02019-11-21T17:36:39ZKlaus ZimmermannUpdate changelog for release 0.10.00.10.0Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/139update autogenerated matadata2019-11-21T17:20:53ZLars Bärringupdate autogenerated matadata0.10.0Lars BärringLars Bärringhttps://git.smhi.se/climix/climix/-/issues/138editor broken2019-11-21T14:21:02ZLars Bärringeditor broken!95 seems to have broken the editor component:
```
(climix-devel-3) ~/CODE/climix/climix/etc >climix-editor /home/sm_lbarr/CODE/cf-index-meta/climate_indices_CV22.fods >climate_indices_CV22__2.yml
Traceback (most recent call last):
Fi...!95 seems to have broken the editor component:
```
(climix-devel-3) ~/CODE/climix/climix/etc >climix-editor /home/sm_lbarr/CODE/cf-index-meta/climate_indices_CV22.fods >climate_indices_CV22__2.yml
Traceback (most recent call last):
File "/home/sm_lbarr/.conda/envs/climix-devel-3/bin/climix-editor", line 11, in <module>
load_entry_point('climix', 'console_scripts', 'climix-editor')()
File "/home/sm_lbarr/.conda/envs/climix-devel-3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/home/sm_lbarr/.conda/envs/climix-devel-3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2852, in load_entry_point
return ep.load()
File "/home/sm_lbarr/.conda/envs/climix-devel-3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2443, in load
return self.resolve()
File "/home/sm_lbarr/.conda/envs/climix-devel-3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2449, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/home/sm_lbarr/CODE/climix/climix/editor/__init__.py", line 1, in <module>
from .main import main
File "/home/sm_lbarr/CODE/climix/climix/editor/main.py", line 9, in <module>
from .alttable import build_index_definitions as build_alt
File "/home/sm_lbarr/CODE/climix/climix/editor/alttable.py", line 8, in <module>
from climix.index_functions import SUPPORTED_OPERATORS, SUPPORTED_REDUCERS
ImportError: cannot import name 'SUPPORTED_OPERATORS' from 'climix.index_functions' (/home/sm_lbarr/CODE/climix/climix/index_functions/__init__.py)
```Klaus ZimmermannKlaus Zimmermannhttps://git.smhi.se/climix/climix/-/issues/136Bug in threshold data type conversion2019-11-21T13:02:45ZLars BärringBug in threshold data type conversionFor all tested precip indices climix crashes, see below. Probably the threshold should be casted as `np.float32` already when read into climix?
```
climix -s -e -x cdd /home/sm_lbarr/PROJ/test_climix/indata/CLIMPACT2_TEST/precip_AUS_...For all tested precip indices climix crashes, see below. Probably the threshold should be casted as `np.float32` already when read into climix?
```
climix -s -e -x cdd /home/sm_lbarr/PROJ/test_climix/indata/CLIMPACT2_TEST/precip_AUS_climpact2-sampledata_gridded-obs_v1_ET-SCI_v1_1991-2010.nc
INFO:root:Loading metadata
INFO:root:Scheduler ready; starting main program.
INFO:root:Preparing indices
INFO:root:Found implementation for index_function <spell_length> from distribution <climix 0.9.0+14.g29e95e2>
INFO:root:Starting calculations for index <climix.index.Index object at 0x2ab6840234a8>
INFO:root:Building output filename
INFO:root:Preparing input data
INFO:root:Calculating index
INFO:root:Adding coord categorisation.
INFO:root:Extracting period cube
INFO:root:Preparing cube
INFO:root:Calculation took 3.6670 seconds.
Traceback (most recent call last):
File "/home/sm_lbarr/.conda/envs/climix-devel-3/bin/climix", line 11, in <module>
load_entry_point('climix', 'console_scripts', 'climix')()
File "/home/sm_lbarr/CODE/climix/climix/main.py", line 126, in main
args.output_template, args.sliced_mode)
File "/home/sm_lbarr/CODE/climix/climix/main.py", line 105, in do_main
result = index(input_data)
File "/home/sm_lbarr/CODE/climix/climix/index.py", line 23, in __call__
self.index_function.prepare(sub_cube)
File "/home/sm_lbarr/CODE/climix/climix/index_functions.py", line 200, in prepare
input_cube.standard_name)
File "/home/sm_lbarr/CODE/climix/climix/util/__init__.py", line 9, in change_units
change_pr_units(cube_or_coord, new_units, new_standard_name)
File "/home/sm_lbarr/CODE/climix/climix/util/change_pr_units.py", line 118, in change_pr_units
cube_or_coord.points *= float(conv_factor)
numpy.core._exceptions.UFuncTypeError: Cannot cast ufunc 'multiply' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'
```0.10.0Klaus ZimmermannKlaus Zimmermann