Running climix 0.17 with new env throws hdf5 error messages
Creating a new env using climix 0.17 environment.yml and running climix throws hdf5 error messages.
HDF5-DIAG: Error detected in HDF5 (1.14.0) thread 1:
#000: H5A.c line 679 in H5Aopen_by_name(): unable to synchronously open attribute
major: Attribute
minor: Can't open object
#001: H5A.c line 641 in H5A__open_by_name_api_common(): unable to open attribute: '_QuantizeBitRoundNumberOfSignificantBits'
major: Attribute
minor: Can't open object
#002: H5A.c line 464 in H5A__open_common(): unable to open attribute: '_QuantizeBitRoundNumberOfSignificantBits'
major: Attribute
minor: Can't open object
#003: H5VLcallback.c line 1138 in H5VL_attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#004: H5VLcallback.c line 1105 in H5VL__attr_open(): attribute open failed
major: Virtual Object Layer
minor: Can't open object
#005: H5VLnative_attr.c line 161 in H5VL__native_attr_open(): can't open attribute
major: Attribute
minor: Can't open object
#006: H5Aint.c line 658 in H5A__open_by_name(): unable to load attribute info from object header
major: Attribute
minor: Unable to initialize object
#007: H5Oattribute.c line 502 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeBitRoundNumberOfSignificantBits'
major: Attribute
minor: Object not found
.
.
.
This can be solved by adding a constraint in the environment.yml libnetcdf<4.9.1
Which makes the following changes:
──────────────────────────────────────────────────────────────────────────────────────
Install:
──────────────────────────────────────────────────────────────────────────────────────
+ jpeg 9e h166bdaf_2 conda-forge/linux-64 Cached
Change:
──────────────────────────────────────────────────────────────────────────────────────
- hdf4 4.2.15 h501b40f_6 conda-forge
+ hdf4 4.2.15 h9772cbc_5 conda-forge/linux-64 Cached
- lcms2 2.15 haa2dc70_1 conda-forge
+ lcms2 2.15 hfd0df8a_0 conda-forge/linux-64 Cached
- libtiff 4.5.0 ha587672_6 conda-forge
+ libtiff 4.5.0 h6adf6a1_2 conda-forge/linux-64 Cached
- pillow 9.4.0 py310h065c6d2_2 conda-forge
+ pillow 9.4.0 py310h023d228_1 conda-forge/linux-64 Cached
Downgrade:
──────────────────────────────────────────────────────────────────────────────────────
- hdf5 1.14.0 nompi_hb72d44e_103 conda-forge
+ hdf5 1.12.2 nompi_h4df4325_101 conda-forge/linux-64 Cached
- libdeflate 1.18 h0b41bf4_0 conda-forge
+ libdeflate 1.17 h0b41bf4_0 conda-forge/linux-64 Cached
- libjpeg-turbo 2.1.5.1 h0b41bf4_0 conda-forge
+ libjpeg-turbo 2.1.4 h166bdaf_0 conda-forge/linux-64 Cached
- libnetcdf 4.9.2 nompi_hf3f8848_103 conda-forge
+ libnetcdf 4.8.1 nompi_h261ec11_106 conda-forge/linux-64 Cached
- netcdf4 1.6.3 nompi_py310h2d0b64f_102 conda-forge
+ netcdf4 1.6.2 nompi_py310h55e1e36_100 conda-forge/linux-64 Cached
Summary:
Install: 1 packages
Change: 4 packages
Downgrade: 5 packages
However constraining libnetcdf, seems to create a new error with dask:
INFO:distributed.batched:Batched Comm Closed <TCP (closed) Scheduler connection to worker local=tcp://127.0.0.1:49048 remote=tcp://127.0.0.1:53328>
Traceback (most recent call last):
File "/home/sm_carni/.conda/envs/climix_error_test/lib/python3.10/site-packages/distributed/batched.py", line 115, in _background_send
nbytes = yield coro
File "/home/sm_carni/.conda/envs/climix_error_test/lib/python3.10/site-packages/tornado/gen.py", line 769, in run
value = future.result()
File "/home/sm_carni/.conda/envs/climix_error_test/lib/python3.10/site-packages/distributed/comm/tcp.py", line 269, in write
raise CommClosedError()
distributed.comm.core.CommClosedError
can be solved by constraining netCDF4==1.6.0
but I am not sure if this is the right approach.