In [1]:
%matplotlib inline
import pandas as pd
import socket
host = socket.getfqdn()

from core import  load, zoom, calc, save,plots,monitor
In [2]:
#reload funcs after updating ./core/*.py
import importlib
importlib.reload(load)
importlib.reload(zoom)
importlib.reload(calc)
importlib.reload(save)
importlib.reload(plots)
importlib.reload(monitor)
Out[2]:
<module 'core.monitor' from '/ccc/work/cont003/gen7420/odakatin/monitor-sedna/notebook/core/monitor.py'>

If you submit the job with job scheduler, above¶

below are list of enviroment variable one can pass

%env local='2"¶

local : if True run dask local cluster, if not true, put number of workers setted in the 'local' if no 'local ' given, local will be setted automatically to 'True'

%env ychunk='2'¶

%env tchunk='2'¶

controls chunk. 'False' sets no modification from original netcdf file's chunk.¶

ychunk=10 will group the original netcdf file to 10 by 10¶

tchunk=1 will chunk the time coordinate one by one¶

%env file_exp=¶

'file_exp': Which 'experiment' name is it?¶

. this corresopnds to intake catalog name without path and .yaml¶

%env year=¶

for Validation, this correspoinds to path/year/month 's year¶

for monitoring, this corresponids to 'date' having * means do all files in the monitoring directory¶

setting it as 0[0-9] &1[0-9]& *[2-3][0-9], the job can be separated in three lots.¶

%env month=¶

for monitoring this corresponds to file path path-XIOS.{month}/¶

#

%env control=FWC_SSH¶

name of control file to be used for computation/plots/save/ & how it is called from Monitor.sh¶

Monitor.sh calls M_MLD_2D

and AWTD.sh, Fluxnet.sh, Siconc.sh, IceClim.sh, FWC_SSH.sh

  • AWTD.sh M_AWTMD

  • Fluxnet.sh M_Fluxnet

  • Siconc.sh M_Ice_quantities
  • IceClim.sh M_IceClim M_IceConce M_IceThick

FWC_SSH.sh M_FWC_2D M_FWC_integrals M_FWC_SSH M_SSH_anomaly

Integrals.sh M_Mean_temp_velo M_Mooring M_Sectionx M_Sectiony

%env save= proceed saving? True or False , Default is setted as True¶

%env plot= proceed plotting? True or False , Default is setted as True¶

%env calc= proceed computation? or just load computed result? True or False , Default is setted as True¶

%env save=False¶

%env lazy=False¶

For debugging this cell can help¶

%env file_exp=SEDNA_DELTA_MONITOR %env year=2012 %env month=01

0[1-2]¶

%env ychunk=10 %env ychunk=False %env save=False %env plot=True %env calc=True # %env lazy=False

False¶

%env control=M_Fluxnet

M_Sectiony ok with ychunk=False local=True lazy=False¶

In [3]:
%%time
# 'savefig': Do we save output in html? or not. keep it true. 
savefig=True
client,cluster,control,catalog_url,month,year,daskreport,outputpath = load.set_control(host)
!mkdir -p $outputpath
!mkdir -p $daskreport
client

local True
using host= irene4356.c-irene.mg1.tgcc.ccc.cea.fr starting dask cluster on local= True workers 16
10000000000
False
tgcc local cluster starting
This code is running on  irene4356.c-irene.mg1.tgcc.ccc.cea.fr using  SEDNA_DELTA_MONITOR file experiment, read from  ../lib/SEDNA_DELTA_MONITOR.yaml  on year= 2012  on month= 02  outputpath= ../results/SEDNA_DELTA_MONITOR/ daskreport= ../results/dask/6419280irene4356.c-irene.mg1.tgcc.ccc.cea.fr_SEDNA_DELTA_MONITOR_02M_Mooring/
CPU times: user 3.86 s, sys: 746 ms, total: 4.61 s
Wall time: 1min 36s
Out[3]:

Client

Client-bc588fcd-13da-11ed-bf8c-080038b938b9

Connection method: Cluster object Cluster type: distributed.LocalCluster
Dashboard: http://127.0.0.1:8787/status

Cluster Info

LocalCluster

006523d7

Dashboard: http://127.0.0.1:8787/status Workers: 64
Total threads: 256 Total memory: 251.06 GiB
Status: running Using processes: True

Scheduler Info

Scheduler

Scheduler-eea656d9-7dd4-480d-8467-96a4262aa046

Comm: tcp://127.0.0.1:40395 Workers: 64
Dashboard: http://127.0.0.1:8787/status Total threads: 256
Started: 1 minute ago Total memory: 251.06 GiB

Workers

Worker: 0
Comm: tcp://127.0.0.1:43873 Total threads: 4
Dashboard: http://127.0.0.1:36304/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:37929
Local directory: /tmp/dask-worker-space/worker-gsps5lxy

Worker: 1
Comm: tcp://127.0.0.1:34917 Total threads: 4
Dashboard: http://127.0.0.1:33278/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:41306
Local directory: /tmp/dask-worker-space/worker-2xxyo4tz

Worker: 2
Comm: tcp://127.0.0.1:46259 Total threads: 4
Dashboard: http://127.0.0.1:35371/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:37264
Local directory: /tmp/dask-worker-space/worker-gbfues0p

Worker: 3
Comm: tcp://127.0.0.1:35679 Total threads: 4
Dashboard: http://127.0.0.1:45210/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:36200
Local directory: /tmp/dask-worker-space/worker-v7x_xh00

Worker: 4
Comm: tcp://127.0.0.1:44246 Total threads: 4
Dashboard: http://127.0.0.1:35131/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:44139
Local directory: /tmp/dask-worker-space/worker-x550wzrr

Worker: 5
Comm: tcp://127.0.0.1:40916 Total threads: 4
Dashboard: http://127.0.0.1:41198/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45219
Local directory: /tmp/dask-worker-space/worker-1xrl0tmp

Worker: 6
Comm: tcp://127.0.0.1:44335 Total threads: 4
Dashboard: http://127.0.0.1:41942/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45783
Local directory: /tmp/dask-worker-space/worker-o3uekeil

Worker: 7
Comm: tcp://127.0.0.1:33086 Total threads: 4
Dashboard: http://127.0.0.1:36925/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40292
Local directory: /tmp/dask-worker-space/worker-rtv7s76f

Worker: 8
Comm: tcp://127.0.0.1:34165 Total threads: 4
Dashboard: http://127.0.0.1:37001/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:39652
Local directory: /tmp/dask-worker-space/worker-6u5zct47

Worker: 9
Comm: tcp://127.0.0.1:40647 Total threads: 4
Dashboard: http://127.0.0.1:37088/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45393
Local directory: /tmp/dask-worker-space/worker-10q8rpa2

Worker: 10
Comm: tcp://127.0.0.1:46529 Total threads: 4
Dashboard: http://127.0.0.1:46847/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33316
Local directory: /tmp/dask-worker-space/worker-tidzr9_k

Worker: 11
Comm: tcp://127.0.0.1:44942 Total threads: 4
Dashboard: http://127.0.0.1:42428/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:37661
Local directory: /tmp/dask-worker-space/worker-9qs9t37f

Worker: 12
Comm: tcp://127.0.0.1:32807 Total threads: 4
Dashboard: http://127.0.0.1:44030/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33102
Local directory: /tmp/dask-worker-space/worker-_vb411q0

Worker: 13
Comm: tcp://127.0.0.1:41065 Total threads: 4
Dashboard: http://127.0.0.1:41160/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:39523
Local directory: /tmp/dask-worker-space/worker-3uy24j35

Worker: 14
Comm: tcp://127.0.0.1:37007 Total threads: 4
Dashboard: http://127.0.0.1:41560/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43200
Local directory: /tmp/dask-worker-space/worker-28m_gfgw

Worker: 15
Comm: tcp://127.0.0.1:39799 Total threads: 4
Dashboard: http://127.0.0.1:45664/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:37112
Local directory: /tmp/dask-worker-space/worker-hk0kcs7h

Worker: 16
Comm: tcp://127.0.0.1:36741 Total threads: 4
Dashboard: http://127.0.0.1:34752/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43036
Local directory: /tmp/dask-worker-space/worker-i5zk06a0

Worker: 17
Comm: tcp://127.0.0.1:41355 Total threads: 4
Dashboard: http://127.0.0.1:37228/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40208
Local directory: /tmp/dask-worker-space/worker-izrqhmws

Worker: 18
Comm: tcp://127.0.0.1:45424 Total threads: 4
Dashboard: http://127.0.0.1:36623/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33470
Local directory: /tmp/dask-worker-space/worker-m9yf1u11

Worker: 19
Comm: tcp://127.0.0.1:35270 Total threads: 4
Dashboard: http://127.0.0.1:41856/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:44057
Local directory: /tmp/dask-worker-space/worker-uur9c7h7

Worker: 20
Comm: tcp://127.0.0.1:44072 Total threads: 4
Dashboard: http://127.0.0.1:43369/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45224
Local directory: /tmp/dask-worker-space/worker-yepbnf6g

Worker: 21
Comm: tcp://127.0.0.1:40406 Total threads: 4
Dashboard: http://127.0.0.1:46578/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40187
Local directory: /tmp/dask-worker-space/worker-zz7nzino

Worker: 22
Comm: tcp://127.0.0.1:45705 Total threads: 4
Dashboard: http://127.0.0.1:42905/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:38673
Local directory: /tmp/dask-worker-space/worker-1l485c7z

Worker: 23
Comm: tcp://127.0.0.1:42284 Total threads: 4
Dashboard: http://127.0.0.1:34538/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43139
Local directory: /tmp/dask-worker-space/worker-tstb6rfy

Worker: 24
Comm: tcp://127.0.0.1:33648 Total threads: 4
Dashboard: http://127.0.0.1:44188/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:36359
Local directory: /tmp/dask-worker-space/worker-dlp84nih

Worker: 25
Comm: tcp://127.0.0.1:37069 Total threads: 4
Dashboard: http://127.0.0.1:34058/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:39432
Local directory: /tmp/dask-worker-space/worker-jbi5bi6v

Worker: 26
Comm: tcp://127.0.0.1:41802 Total threads: 4
Dashboard: http://127.0.0.1:40119/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40209
Local directory: /tmp/dask-worker-space/worker-cdscl_f0

Worker: 27
Comm: tcp://127.0.0.1:45420 Total threads: 4
Dashboard: http://127.0.0.1:44687/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:34891
Local directory: /tmp/dask-worker-space/worker-u9w572ry

Worker: 28
Comm: tcp://127.0.0.1:32993 Total threads: 4
Dashboard: http://127.0.0.1:38153/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40141
Local directory: /tmp/dask-worker-space/worker-d4pqhf0e

Worker: 29
Comm: tcp://127.0.0.1:38918 Total threads: 4
Dashboard: http://127.0.0.1:33733/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:36902
Local directory: /tmp/dask-worker-space/worker-6du4pxv4

Worker: 30
Comm: tcp://127.0.0.1:43501 Total threads: 4
Dashboard: http://127.0.0.1:38047/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40848
Local directory: /tmp/dask-worker-space/worker-xniob00n

Worker: 31
Comm: tcp://127.0.0.1:34859 Total threads: 4
Dashboard: http://127.0.0.1:43471/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:34625
Local directory: /tmp/dask-worker-space/worker-a562q39s

Worker: 32
Comm: tcp://127.0.0.1:46651 Total threads: 4
Dashboard: http://127.0.0.1:46280/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43965
Local directory: /tmp/dask-worker-space/worker-k2efes12

Worker: 33
Comm: tcp://127.0.0.1:36659 Total threads: 4
Dashboard: http://127.0.0.1:36152/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:39630
Local directory: /tmp/dask-worker-space/worker-jrsm7j_5

Worker: 34
Comm: tcp://127.0.0.1:36267 Total threads: 4
Dashboard: http://127.0.0.1:44642/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:35133
Local directory: /tmp/dask-worker-space/worker-h1mg34mq

Worker: 35
Comm: tcp://127.0.0.1:44103 Total threads: 4
Dashboard: http://127.0.0.1:36203/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:35605
Local directory: /tmp/dask-worker-space/worker-hbr4u5zu

Worker: 36
Comm: tcp://127.0.0.1:33080 Total threads: 4
Dashboard: http://127.0.0.1:44888/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:38956
Local directory: /tmp/dask-worker-space/worker-5c4u7az1

Worker: 37
Comm: tcp://127.0.0.1:43730 Total threads: 4
Dashboard: http://127.0.0.1:46232/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:44332
Local directory: /tmp/dask-worker-space/worker-4f009lf_

Worker: 38
Comm: tcp://127.0.0.1:39132 Total threads: 4
Dashboard: http://127.0.0.1:38214/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:39712
Local directory: /tmp/dask-worker-space/worker-uy3j7z_u

Worker: 39
Comm: tcp://127.0.0.1:46272 Total threads: 4
Dashboard: http://127.0.0.1:34264/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:46630
Local directory: /tmp/dask-worker-space/worker-z02gfcr1

Worker: 40
Comm: tcp://127.0.0.1:38061 Total threads: 4
Dashboard: http://127.0.0.1:38912/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:39954
Local directory: /tmp/dask-worker-space/worker-f5ftjg85

Worker: 41
Comm: tcp://127.0.0.1:40291 Total threads: 4
Dashboard: http://127.0.0.1:38893/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:37739
Local directory: /tmp/dask-worker-space/worker-l7crxdig

Worker: 42
Comm: tcp://127.0.0.1:39023 Total threads: 4
Dashboard: http://127.0.0.1:37791/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:41613
Local directory: /tmp/dask-worker-space/worker-vps5l9r6

Worker: 43
Comm: tcp://127.0.0.1:42528 Total threads: 4
Dashboard: http://127.0.0.1:40980/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:35660
Local directory: /tmp/dask-worker-space/worker-pg4cso3a

Worker: 44
Comm: tcp://127.0.0.1:44847 Total threads: 4
Dashboard: http://127.0.0.1:46332/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:44808
Local directory: /tmp/dask-worker-space/worker-za1ityxo

Worker: 45
Comm: tcp://127.0.0.1:38618 Total threads: 4
Dashboard: http://127.0.0.1:37591/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33331
Local directory: /tmp/dask-worker-space/worker-k8lqeds_

Worker: 46
Comm: tcp://127.0.0.1:44428 Total threads: 4
Dashboard: http://127.0.0.1:41655/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45990
Local directory: /tmp/dask-worker-space/worker-2fsafk_p

Worker: 47
Comm: tcp://127.0.0.1:40346 Total threads: 4
Dashboard: http://127.0.0.1:38437/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:37788
Local directory: /tmp/dask-worker-space/worker-b93tlm03

Worker: 48
Comm: tcp://127.0.0.1:46326 Total threads: 4
Dashboard: http://127.0.0.1:42471/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:41346
Local directory: /tmp/dask-worker-space/worker-ctskrqya

Worker: 49
Comm: tcp://127.0.0.1:33373 Total threads: 4
Dashboard: http://127.0.0.1:39313/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33934
Local directory: /tmp/dask-worker-space/worker-9bp6yvmi

Worker: 50
Comm: tcp://127.0.0.1:38222 Total threads: 4
Dashboard: http://127.0.0.1:40443/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40570
Local directory: /tmp/dask-worker-space/worker-h47a2py2

Worker: 51
Comm: tcp://127.0.0.1:41993 Total threads: 4
Dashboard: http://127.0.0.1:43207/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33883
Local directory: /tmp/dask-worker-space/worker-c2r634t9

Worker: 52
Comm: tcp://127.0.0.1:38730 Total threads: 4
Dashboard: http://127.0.0.1:37188/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:32771
Local directory: /tmp/dask-worker-space/worker-8iu03e3n

Worker: 53
Comm: tcp://127.0.0.1:41106 Total threads: 4
Dashboard: http://127.0.0.1:45396/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45890
Local directory: /tmp/dask-worker-space/worker-4u0nf8ha

Worker: 54
Comm: tcp://127.0.0.1:37817 Total threads: 4
Dashboard: http://127.0.0.1:36239/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:39847
Local directory: /tmp/dask-worker-space/worker-d17y3bn1

Worker: 55
Comm: tcp://127.0.0.1:45621 Total threads: 4
Dashboard: http://127.0.0.1:46711/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:42371
Local directory: /tmp/dask-worker-space/worker-nfbd8m88

Worker: 56
Comm: tcp://127.0.0.1:38738 Total threads: 4
Dashboard: http://127.0.0.1:39498/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43755
Local directory: /tmp/dask-worker-space/worker-9ynz1pw2

Worker: 57
Comm: tcp://127.0.0.1:46443 Total threads: 4
Dashboard: http://127.0.0.1:39686/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:46539
Local directory: /tmp/dask-worker-space/worker-a5ulepo0

Worker: 58
Comm: tcp://127.0.0.1:42248 Total threads: 4
Dashboard: http://127.0.0.1:38289/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45073
Local directory: /tmp/dask-worker-space/worker-ns8ppp25

Worker: 59
Comm: tcp://127.0.0.1:33795 Total threads: 4
Dashboard: http://127.0.0.1:40254/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43215
Local directory: /tmp/dask-worker-space/worker-w51gumr8

Worker: 60
Comm: tcp://127.0.0.1:42498 Total threads: 4
Dashboard: http://127.0.0.1:33607/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:46448
Local directory: /tmp/dask-worker-space/worker-ldy4wkvi

Worker: 61
Comm: tcp://127.0.0.1:36150 Total threads: 4
Dashboard: http://127.0.0.1:44960/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:35471
Local directory: /tmp/dask-worker-space/worker-1phlej79

Worker: 62
Comm: tcp://127.0.0.1:41694 Total threads: 4
Dashboard: http://127.0.0.1:45500/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33674
Local directory: /tmp/dask-worker-space/worker-d34legau

Worker: 63
Comm: tcp://127.0.0.1:45979 Total threads: 4
Dashboard: http://127.0.0.1:45270/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:44296
Local directory: /tmp/dask-worker-space/worker-5btf_o46

read plotting information from a csv file¶

In [4]:
df=load.controlfile(control)
#Take out 'later' tagged computations
#df=df[~df['Value'].str.contains('later')]
df
Out[4]:
Value Inputs Equation Zone Plot Colourmap MinMax Unit Oldname Unnamed: 10
Mooring_Arc_B gridS.vosaline,gridT.votemper,param.depth,para... data Arc_B Mooring rainbow {'vosaline': (28.0,34.4), 'votemper': (-2.0,2.0)} None x
Mooring_Eur_B gridS.vosaline,gridT.votemper,param.depth,para... data Eur_B Mooring rainbow {'vosaline': (32.0,35.0), 'votemper': (-2.0,3.0)} None x

Computation starts here¶

Each computation consists of

  1. Load NEMO data set
  2. Zoom data set
  3. Compute (or load computed data set)
  4. Save
  5. Plot
  6. Close
In [5]:
%%time
import os
calcswitch=os.environ.get('calc', 'True') 
lazy=os.environ.get('lazy','False' )
loaddata=((df.Inputs != '').any()) 
print('calcswitch=',calcswitch,'df.Inputs != nothing',loaddata, 'lazy=',lazy)
data = load.datas(catalog_url,df.Inputs,month,year,daskreport,lazy=lazy) if ((calcswitch=='True' )*loaddata) else 0 
data

calcswitch= True df.Inputs != nothing True lazy= False
../lib/SEDNA_DELTA_MONITOR.yaml
using param_xios reading  ../lib/SEDNA_DELTA_MONITOR.yaml
using param_xios reading  <bound method DataSourceBase.describe of sources:
  param_xios:
    args:
      combine: nested
      concat_dim: y
      urlpath: /ccc/work/cont003/gen7420/odakatin/CONFIGS/SEDNA/SEDNA-I/SEDNA_Domain_cfg_Tgt_20210423_tsh10m_L1/param_f32/x_*.nc
      xarray_kwargs:
        compat: override
        coords: minimal
        data_vars: minimal
        parallel: true
    description: SEDNA NEMO parameters from MPI output  nav_lon lat fails
    driver: intake_xarray.netcdf.NetCDFSource
    metadata:
      catalog_dir: /ccc/work/cont003/gen7420/odakatin/monitor-sedna/notebook/../lib/
>
{'name': 'param_xios', 'container': 'xarray', 'plugin': ['netcdf'], 'driver': ['netcdf'], 'description': 'SEDNA NEMO parameters from MPI output  nav_lon lat fails', 'direct_access': 'forbid', 'user_parameters': [{'name': 'path', 'description': 'file coordinate', 'type': 'str', 'default': '/ccc/work/cont003/gen7420/odakatin/CONFIGS/SEDNA/MESH/SEDNA_mesh_mask_Tgt_20210423_tsh10m_L1/param'}], 'metadata': {}, 'args': {'urlpath': '/ccc/work/cont003/gen7420/odakatin/CONFIGS/SEDNA/SEDNA-I/SEDNA_Domain_cfg_Tgt_20210423_tsh10m_L1/param_f32/x_*.nc', 'combine': 'nested', 'concat_dim': 'y'}}
0 read gridS ['vosaline']
lazy= False
using load_data_xios_kerchunk reading  gridS
using load_data_xios_kerchunk reading  <bound method DataSourceBase.describe of sources:
  data_xios_kerchunk:
    args:
      consolidated: false
      storage_options:
        fo: file:////ccc/cont003/home/ra5563/ra5563/catalogue/DELTA/201202/gridS_0[0-5][0-9][0-9].json
        target_protocol: file
      urlpath: reference://
    description: CREG025 NEMO outputs from different xios server in kerchunk format
    driver: intake_xarray.xzarr.ZarrSource
    metadata:
      catalog_dir: /ccc/work/cont003/gen7420/odakatin/monitor-sedna/notebook/../lib/
>
      took 23.494372844696045 seconds
0 merging gridS ['vosaline']
1 read gridT ['votemper']
lazy= False
using load_data_xios_kerchunk reading  gridT
using load_data_xios_kerchunk reading  <bound method DataSourceBase.describe of sources:
  data_xios_kerchunk:
    args:
      consolidated: false
      storage_options:
        fo: file:////ccc/cont003/home/ra5563/ra5563/catalogue/DELTA/201202/gridT_0[0-5][0-9][0-9].json
        target_protocol: file
      urlpath: reference://
    description: CREG025 NEMO outputs from different xios server in kerchunk format
    driver: intake_xarray.xzarr.ZarrSource
    metadata:
      catalog_dir: /ccc/work/cont003/gen7420/odakatin/monitor-sedna/notebook/../lib/
>
      took 21.156111478805542 seconds
1 merging gridT ['votemper']
      took 0.8706753253936768 seconds
param mask2d will be included in data
param nav_lat will be included in data
param mask will be included in data
param nav_lon will be included in data
param depth will be included in data
ychunk= 5 calldatas_y_rechunk
sum_num (13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12)
start rechunking with (65, 65, 62, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 48)
end of y_rechunk
before rechunking t item (1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1)
start rechunking t with 1
end of t_rechunk
CPU times: user 47.5 s, sys: 12.5 s, total: 60 s
Wall time: 1min 40s
Out[5]:
<xarray.Dataset>
Dimensions:        (t: 28, z: 150, y: 6540, x: 6560)
Coordinates:
    time_centered  (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t              (t) object 2012-02-01 12:00:00 ... 2012-02-28 12:00:00
  * y              (y) int64 1 2 3 4 5 6 7 ... 6535 6536 6537 6538 6539 6540
  * x              (x) int64 1 2 3 4 5 6 7 ... 6555 6556 6557 6558 6559 6560
  * z              (z) int64 1 2 3 4 5 6 7 8 ... 143 144 145 146 147 148 149 150
    mask2d         (y, x) bool dask.array<chunksize=(65, 6560), meta=np.ndarray>
    nav_lat        (y, x) float32 dask.array<chunksize=(65, 6560), meta=np.ndarray>
    mask           (z, y, x) bool dask.array<chunksize=(150, 65, 6560), meta=np.ndarray>
    nav_lon        (y, x) float32 dask.array<chunksize=(65, 6560), meta=np.ndarray>
    depth          (z, y, x) float32 dask.array<chunksize=(150, 65, 6560), meta=np.ndarray>
Data variables:
    vosaline       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 65, 6560), meta=np.ndarray>
    votemper       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 65, 6560), meta=np.ndarray>
Attributes: (12/26)
    CASE:                    DELTA
    CONFIG:                  SEDNA
    Conventions:             CF-1.6
    DOMAIN_dimensions_ids:   [2, 3]
    DOMAIN_halo_size_end:    [0, 0]
    DOMAIN_halo_size_start:  [0, 0]
    ...                      ...
    nj:                      13
    output_frequency:        1d
    start_date:              20090101
    timeStamp:               2022-Jan-18 16:51:26 GMT
    title:                   ocean T grid variables
    uuid:                    6ca3a74a-269a-44e2-91db-2aea875dbf84
xarray.Dataset
    • t: 28
    • z: 150
    • y: 6540
    • x: 6560
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 224 B 8 B
      Shape (28,) (1,)
      Count 171 Tasks 28 Chunks
      Type object numpy.ndarray
      		28 1
    • t
      (t)
      object
      2012-02-01 12:00:00 ... 2012-02-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 2, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 28, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      (y)
      int64
      1 2 3 4 5 ... 6537 6538 6539 6540
      array([   1,    2,    3, ..., 6538, 6539, 6540])
    • x
      (x)
      int64
      1 2 3 4 5 ... 6557 6558 6559 6560
      array([   1,    2,    3, ..., 6558, 6559, 6560])
    • z
      (z)
      int64
      1 2 3 4 5 6 ... 146 147 148 149 150
      array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,  14,
        15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,  28,
        29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,  42,
        43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,  56,
        57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,  70,
        71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,  84,
        85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,  98,
        99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112,
       113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126,
       127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,
       141, 142, 143, 144, 145, 146, 147, 148, 149, 150])
    • mask2d
      (y, x)
      bool
      dask.array<chunksize=(65, 6560), meta=np.ndarray>
      Array Chunk
      Bytes 40.91 MiB 416.41 kiB
      Shape (6540, 6560) (65, 6560)
      Count 1741 Tasks 109 Chunks
      Type bool numpy.ndarray
      		6560 6540
    • nav_lat
      (y, x)
      float32
      dask.array<chunksize=(65, 6560), meta=np.ndarray>
      Array Chunk
      Bytes 163.66 MiB 1.63 MiB
      Shape (6540, 6560) (65, 6560)
      Count 1741 Tasks 109 Chunks
      Type float32 numpy.ndarray
      		6560 6540
    • mask
      (z, y, x)
      bool
      dask.array<chunksize=(150, 65, 6560), meta=np.ndarray>
      Array Chunk
      Bytes 5.99 GiB 61.00 MiB
      Shape (150, 6540, 6560) (150, 65, 6560)
      Count 1741 Tasks 109 Chunks
      Type bool numpy.ndarray
      		6560 6540 150
    • nav_lon
      (y, x)
      float32
      dask.array<chunksize=(65, 6560), meta=np.ndarray>
      Array Chunk
      Bytes 163.66 MiB 1.63 MiB
      Shape (6540, 6560) (65, 6560)
      Count 1741 Tasks 109 Chunks
      Type float32 numpy.ndarray
      		6560 6540
    • depth
      (z, y, x)
      float32
      dask.array<chunksize=(150, 65, 6560), meta=np.ndarray>
      Array Chunk
      Bytes 23.97 GiB 243.99 MiB
      Shape (150, 6540, 6560) (150, 65, 6560)
      Count 1741 Tasks 109 Chunks
      Type float32 numpy.ndarray
      		6560 6540 150
    • vosaline
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 65, 6560), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      salinity
      online_operation :
      average
      standard_name :
      sea_water_practical_salinity
      units :
      1e-3
      Array Chunk
      Bytes 671.26 GiB 243.99 MiB
      Shape (28, 150, 6540, 6560) (1, 150, 65, 6560)
      Count 34060 Tasks 3052 Chunks
      Type float32 numpy.ndarray
      		28 1 6560 6540 150
    • votemper
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 65, 6560), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      temperature
      online_operation :
      average
      standard_name :
      sea_water_potential_temperature
      units :
      degC
      Array Chunk
      Bytes 671.26 GiB 243.99 MiB
      Shape (28, 150, 6540, 6560) (1, 150, 65, 6560)
      Count 34060 Tasks 3052 Chunks
      Type float32 numpy.ndarray
      		28 1 6560 6540 150
  • CASE :
    DELTA
    CONFIG :
    SEDNA
    Conventions :
    CF-1.6
    DOMAIN_dimensions_ids :
    [2, 3]
    DOMAIN_halo_size_end :
    [0, 0]
    DOMAIN_halo_size_start :
    [0, 0]
    DOMAIN_number :
    0
    DOMAIN_number_total :
    544
    DOMAIN_position_first :
    [1, 1]
    DOMAIN_position_last :
    [6560, 13]
    DOMAIN_size_global :
    [6560, 6540]
    DOMAIN_size_local :
    [6560, 13]
    DOMAIN_type :
    box
    NCO :
    netCDF Operators version 4.9.1 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)
    description :
    ocean T grid variables
    history :
    Wed Jan 19 12:40:52 2022: ncks -4 -L 1 SEDNA-DELTA_1d_gridS_201202-201202_NOZIP_0000.nc /ccc/scratch/cont003/gen7420/talandel/SEDNA/SEDNA-DELTA-S/SPLIT/1d/2012/02/SEDNA-DELTA_1d_gridS_201202-201202_0000.nc Wed Jan 19 12:40:28 2022: ncrcat -n 28,2,1 SEDNA-DELTA_1d_gridS_0000_01.nc SEDNA-DELTA_1d_gridS_201202-201202_NOZIP_0000.nc
    ibegin :
    0
    jbegin :
    0
    name :
    /ccc/scratch/cont003/ra5563/talandel/ONGOING-RUNS/SEDNA-DELTA-XIOS.47/SEDNA-DELTA_1d_gridS
    ni :
    6560
    nj :
    13
    output_frequency :
    1d
    start_date :
    20090101
    timeStamp :
    2022-Jan-18 16:51:26 GMT
    title :
    ocean T grid variables
    uuid :
    6ca3a74a-269a-44e2-91db-2aea875dbf84
In [6]:
%%time
monitor.auto(df,data,savefig,daskreport,outputpath,file_exp='SEDNA'
            )

#calc= True
#save= True
#plot= False
Value='Mooring_Arc_B'
Zone='Arc_B'
Plot='Mooring'
cmap='rainbow'
clabel='None'
clim= {'vosaline': (28.0, 34.4), 'votemper': (-2.0, 2.0)}
outputpath='../results/SEDNA_DELTA_MONITOR/'
nc_outputpath='../nc_results/SEDNA_DELTA_MONITOR/'
filename='SEDNA_Mooring_Arc_B_Mooring_Arc_B'
data=monitor.optimize_dataset(data)
#2 Zooming Data
data= zoom.Arc_B(data)
data=monitor.optimize_dataset(data)

<xarray.Dataset>
Dimensions:        (t: 28, z: 102)
Coordinates:
    time_centered  (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t              (t) object 2012-02-01 12:00:00 ... 2012-02-28 12:00:00
    y              int64 4967
    x              int64 2471
  * z              (z) int64 1 2 3 4 5 6 7 8 9 ... 94 95 96 97 98 99 100 101 102
    mask2d         bool dask.array<chunksize=(), meta=np.ndarray>
    nav_lat        float32 dask.array<chunksize=(), meta=np.ndarray>
    mask           (z) bool dask.array<chunksize=(102,), meta=np.ndarray>
    nav_lon        float32 dask.array<chunksize=(), meta=np.ndarray>
    depth          (z) float32 dask.array<chunksize=(102,), meta=np.ndarray>
Data variables:
    vosaline       (t, z) float32 dask.array<chunksize=(1, 102), meta=np.ndarray>
    votemper       (t, z) float32 dask.array<chunksize=(1, 102), meta=np.ndarray>
Attributes: (12/26)
    CASE:                    DELTA
    CONFIG:                  SEDNA
    Conventions:             CF-1.6
    DOMAIN_dimensions_ids:   [2, 3]
    DOMAIN_halo_size_end:    [0, 0]
    DOMAIN_halo_size_start:  [0, 0]
    ...                      ...
    nj:                      13
    output_frequency:        1d
    start_date:              20090101
    timeStamp:               2022-Jan-18 16:51:26 GMT
    title:                   ocean T grid variables
    uuid:                    6ca3a74a-269a-44e2-91db-2aea875dbf84
xarray.Dataset
    • t: 28
    • z: 102
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 224 B 8 B
      Shape (28,) (1,)
      Count 171 Tasks 28 Chunks
      Type object numpy.ndarray
      		28 1
    • t
      (t)
      object
      2012-02-01 12:00:00 ... 2012-02-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 2, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 28, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      ()
      int64
      4967
      array(4967)
    • x
      ()
      int64
      2471
      array(2471)
    • z
      (z)
      int64
      1 2 3 4 5 6 ... 98 99 100 101 102
      array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,  14,
        15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,  28,
        29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,  42,
        43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,  56,
        57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,  70,
        71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,  84,
        85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,  98,
        99, 100, 101, 102])
    • mask2d
      ()
      bool
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 1 B 1 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type bool numpy.ndarray
    • nav_lat
      ()
      float32
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 4 B 4 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type float32 numpy.ndarray
    • mask
      (z)
      bool
      dask.array<chunksize=(102,), meta=np.ndarray>
      Array Chunk
      Bytes 102 B 102 B
      Shape (102,) (102,)
      Count 1744 Tasks 1 Chunks
      Type bool numpy.ndarray
      		102 1
    • nav_lon
      ()
      float32
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 4 B 4 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type float32 numpy.ndarray
    • depth
      (z)
      float32
      dask.array<chunksize=(102,), meta=np.ndarray>
      Array Chunk
      Bytes 408 B 408 B
      Shape (102,) (102,)
      Count 1744 Tasks 1 Chunks
      Type float32 numpy.ndarray
      		102 1
    • vosaline
      (t, z)
      float32
      dask.array<chunksize=(1, 102), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      salinity
      online_operation :
      average
      standard_name :
      sea_water_practical_salinity
      units :
      1e-3
      Array Chunk
      Bytes 11.16 kiB 408 B
      Shape (28, 102) (1, 102)
      Count 386 Tasks 28 Chunks
      Type float32 numpy.ndarray
      		102 28
    • votemper
      (t, z)
      float32
      dask.array<chunksize=(1, 102), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      temperature
      online_operation :
      average
      standard_name :
      sea_water_potential_temperature
      units :
      degC
      Array Chunk
      Bytes 11.16 kiB 408 B
      Shape (28, 102) (1, 102)
      Count 386 Tasks 28 Chunks
      Type float32 numpy.ndarray
      		102 28
  • CASE :
    DELTA
    CONFIG :
    SEDNA
    Conventions :
    CF-1.6
    DOMAIN_dimensions_ids :
    [2, 3]
    DOMAIN_halo_size_end :
    [0, 0]
    DOMAIN_halo_size_start :
    [0, 0]
    DOMAIN_number :
    0
    DOMAIN_number_total :
    544
    DOMAIN_position_first :
    [1, 1]
    DOMAIN_position_last :
    [6560, 13]
    DOMAIN_size_global :
    [6560, 6540]
    DOMAIN_size_local :
    [6560, 13]
    DOMAIN_type :
    box
    NCO :
    netCDF Operators version 4.9.1 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)
    description :
    ocean T grid variables
    history :
    Wed Jan 19 12:40:52 2022: ncks -4 -L 1 SEDNA-DELTA_1d_gridS_201202-201202_NOZIP_0000.nc /ccc/scratch/cont003/gen7420/talandel/SEDNA/SEDNA-DELTA-S/SPLIT/1d/2012/02/SEDNA-DELTA_1d_gridS_201202-201202_0000.nc Wed Jan 19 12:40:28 2022: ncrcat -n 28,2,1 SEDNA-DELTA_1d_gridS_0000_01.nc SEDNA-DELTA_1d_gridS_201202-201202_NOZIP_0000.nc
    ibegin :
    0
    jbegin :
    0
    name :
    /ccc/scratch/cont003/ra5563/talandel/ONGOING-RUNS/SEDNA-DELTA-XIOS.47/SEDNA-DELTA_1d_gridS
    ni :
    6560
    nj :
    13
    output_frequency :
    1d
    start_date :
    20090101
    timeStamp :
    2022-Jan-18 16:51:26 GMT
    title :
    ocean T grid variables
    uuid :
    6ca3a74a-269a-44e2-91db-2aea875dbf84
#3 Start computing 
data= data
monitor.optimize_dataset(data)
add optimise here once otimise can recognise

<xarray.Dataset>
Dimensions:        (t: 28, z: 102)
Coordinates:
    time_centered  (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t              (t) object 2012-02-01 12:00:00 ... 2012-02-28 12:00:00
    y              int64 4967
    x              int64 2471
  * z              (z) int64 1 2 3 4 5 6 7 8 9 ... 94 95 96 97 98 99 100 101 102
    mask2d         bool dask.array<chunksize=(), meta=np.ndarray>
    nav_lat        float32 dask.array<chunksize=(), meta=np.ndarray>
    mask           (z) bool dask.array<chunksize=(102,), meta=np.ndarray>
    nav_lon        float32 dask.array<chunksize=(), meta=np.ndarray>
    depth          (z) float32 dask.array<chunksize=(102,), meta=np.ndarray>
Data variables:
    vosaline       (t, z) float32 dask.array<chunksize=(1, 102), meta=np.ndarray>
    votemper       (t, z) float32 dask.array<chunksize=(1, 102), meta=np.ndarray>
Attributes: (12/26)
    CASE:                    DELTA
    CONFIG:                  SEDNA
    Conventions:             CF-1.6
    DOMAIN_dimensions_ids:   [2, 3]
    DOMAIN_halo_size_end:    [0, 0]
    DOMAIN_halo_size_start:  [0, 0]
    ...                      ...
    nj:                      13
    output_frequency:        1d
    start_date:              20090101
    timeStamp:               2022-Jan-18 16:51:26 GMT
    title:                   ocean T grid variables
    uuid:                    6ca3a74a-269a-44e2-91db-2aea875dbf84
xarray.Dataset
    • t: 28
    • z: 102
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 224 B 8 B
      Shape (28,) (1,)
      Count 171 Tasks 28 Chunks
      Type object numpy.ndarray
      		28 1
    • t
      (t)
      object
      2012-02-01 12:00:00 ... 2012-02-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 2, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 28, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      ()
      int64
      4967
      array(4967)
    • x
      ()
      int64
      2471
      array(2471)
    • z
      (z)
      int64
      1 2 3 4 5 6 ... 98 99 100 101 102
      array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,  14,
        15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,  28,
        29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,  42,
        43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,  56,
        57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,  70,
        71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,  84,
        85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,  98,
        99, 100, 101, 102])
    • mask2d
      ()
      bool
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 1 B 1 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type bool numpy.ndarray
    • nav_lat
      ()
      float32
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 4 B 4 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type float32 numpy.ndarray
    • mask
      (z)
      bool
      dask.array<chunksize=(102,), meta=np.ndarray>
      Array Chunk
      Bytes 102 B 102 B
      Shape (102,) (102,)
      Count 1744 Tasks 1 Chunks
      Type bool numpy.ndarray
      		102 1
    • nav_lon
      ()
      float32
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 4 B 4 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type float32 numpy.ndarray
    • depth
      (z)
      float32
      dask.array<chunksize=(102,), meta=np.ndarray>
      Array Chunk
      Bytes 408 B 408 B
      Shape (102,) (102,)
      Count 1744 Tasks 1 Chunks
      Type float32 numpy.ndarray
      		102 1
    • vosaline
      (t, z)
      float32
      dask.array<chunksize=(1, 102), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      salinity
      online_operation :
      average
      standard_name :
      sea_water_practical_salinity
      units :
      1e-3
      Array Chunk
      Bytes 11.16 kiB 408 B
      Shape (28, 102) (1, 102)
      Count 386 Tasks 28 Chunks
      Type float32 numpy.ndarray
      		102 28
    • votemper
      (t, z)
      float32
      dask.array<chunksize=(1, 102), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      temperature
      online_operation :
      average
      standard_name :
      sea_water_potential_temperature
      units :
      degC
      Array Chunk
      Bytes 11.16 kiB 408 B
      Shape (28, 102) (1, 102)
      Count 386 Tasks 28 Chunks
      Type float32 numpy.ndarray
      		102 28
  • CASE :
    DELTA
    CONFIG :
    SEDNA
    Conventions :
    CF-1.6
    DOMAIN_dimensions_ids :
    [2, 3]
    DOMAIN_halo_size_end :
    [0, 0]
    DOMAIN_halo_size_start :
    [0, 0]
    DOMAIN_number :
    0
    DOMAIN_number_total :
    544
    DOMAIN_position_first :
    [1, 1]
    DOMAIN_position_last :
    [6560, 13]
    DOMAIN_size_global :
    [6560, 6540]
    DOMAIN_size_local :
    [6560, 13]
    DOMAIN_type :
    box
    NCO :
    netCDF Operators version 4.9.1 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)
    description :
    ocean T grid variables
    history :
    Wed Jan 19 12:40:52 2022: ncks -4 -L 1 SEDNA-DELTA_1d_gridS_201202-201202_NOZIP_0000.nc /ccc/scratch/cont003/gen7420/talandel/SEDNA/SEDNA-DELTA-S/SPLIT/1d/2012/02/SEDNA-DELTA_1d_gridS_201202-201202_0000.nc Wed Jan 19 12:40:28 2022: ncrcat -n 28,2,1 SEDNA-DELTA_1d_gridS_0000_01.nc SEDNA-DELTA_1d_gridS_201202-201202_NOZIP_0000.nc
    ibegin :
    0
    jbegin :
    0
    name :
    /ccc/scratch/cont003/ra5563/talandel/ONGOING-RUNS/SEDNA-DELTA-XIOS.47/SEDNA-DELTA_1d_gridS
    ni :
    6560
    nj :
    13
    output_frequency :
    1d
    start_date :
    20090101
    timeStamp :
    2022-Jan-18 16:51:26 GMT
    title :
    ocean T grid variables
    uuid :
    6ca3a74a-269a-44e2-91db-2aea875dbf84
#4 Saving  SEDNA_Mooring_Arc_B_Mooring_Arc_B
data=save.datas(data,plot=Plot,path=nc_outputpath,filename=filename)
start saving data
saving data in a  csv file ../nc_results/SEDNA_DELTA_MONITOR/SEDNA_Mooring_Arc_B_Mooring_Arc_B2012-02-01_2012-02-28.nc
save computed data at ../nc_results/SEDNA_DELTA_MONITOR/SEDNA_Mooring_Arc_B_Mooring_Arc_B2012-02-01_2012-02-28.nc completed
Value='Mooring_Eur_B'
Zone='Eur_B'
Plot='Mooring'
cmap='rainbow'
clabel='None'
clim= {'vosaline': (32.0, 35.0), 'votemper': (-2.0, 3.0)}
outputpath='../results/SEDNA_DELTA_MONITOR/'
nc_outputpath='../nc_results/SEDNA_DELTA_MONITOR/'
filename='SEDNA_Mooring_Eur_B_Mooring_Eur_B'
data=monitor.optimize_dataset(data)
#2 Zooming Data
data= zoom.Eur_B(data)
data=monitor.optimize_dataset(data)

<xarray.Dataset>
Dimensions:        (t: 28, z: 102)
Coordinates:
    time_centered  (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t              (t) object 2012-02-01 12:00:00 ... 2012-02-28 12:00:00
    y              int64 3621
    x              int64 2495
  * z              (z) int64 1 2 3 4 5 6 7 8 9 ... 94 95 96 97 98 99 100 101 102
    mask2d         bool dask.array<chunksize=(), meta=np.ndarray>
    nav_lat        float32 dask.array<chunksize=(), meta=np.ndarray>
    mask           (z) bool dask.array<chunksize=(102,), meta=np.ndarray>
    nav_lon        float32 dask.array<chunksize=(), meta=np.ndarray>
    depth          (z) float32 dask.array<chunksize=(102,), meta=np.ndarray>
Data variables:
    vosaline       (t, z) float32 dask.array<chunksize=(1, 102), meta=np.ndarray>
    votemper       (t, z) float32 dask.array<chunksize=(1, 102), meta=np.ndarray>
Attributes: (12/26)
    CASE:                    DELTA
    CONFIG:                  SEDNA
    Conventions:             CF-1.6
    DOMAIN_dimensions_ids:   [2, 3]
    DOMAIN_halo_size_end:    [0, 0]
    DOMAIN_halo_size_start:  [0, 0]
    ...                      ...
    nj:                      13
    output_frequency:        1d
    start_date:              20090101
    timeStamp:               2022-Jan-18 16:51:26 GMT
    title:                   ocean T grid variables
    uuid:                    6ca3a74a-269a-44e2-91db-2aea875dbf84
xarray.Dataset
    • t: 28
    • z: 102
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 224 B 8 B
      Shape (28,) (1,)
      Count 171 Tasks 28 Chunks
      Type object numpy.ndarray
      		28 1
    • t
      (t)
      object
      2012-02-01 12:00:00 ... 2012-02-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 2, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 28, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      ()
      int64
      3621
      array(3621)
    • x
      ()
      int64
      2495
      array(2495)
    • z
      (z)
      int64
      1 2 3 4 5 6 ... 98 99 100 101 102
      array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,  14,
        15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,  28,
        29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,  42,
        43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,  56,
        57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,  70,
        71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,  84,
        85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,  98,
        99, 100, 101, 102])
    • mask2d
      ()
      bool
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 1 B 1 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type bool numpy.ndarray
    • nav_lat
      ()
      float32
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 4 B 4 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type float32 numpy.ndarray
    • mask
      (z)
      bool
      dask.array<chunksize=(102,), meta=np.ndarray>
      Array Chunk
      Bytes 102 B 102 B
      Shape (102,) (102,)
      Count 1744 Tasks 1 Chunks
      Type bool numpy.ndarray
      		102 1
    • nav_lon
      ()
      float32
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 4 B 4 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type float32 numpy.ndarray
    • depth
      (z)
      float32
      dask.array<chunksize=(102,), meta=np.ndarray>
      Array Chunk
      Bytes 408 B 408 B
      Shape (102,) (102,)
      Count 1744 Tasks 1 Chunks
      Type float32 numpy.ndarray
      		102 1
    • vosaline
      (t, z)
      float32
      dask.array<chunksize=(1, 102), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      salinity
      online_operation :
      average
      standard_name :
      sea_water_practical_salinity
      units :
      1e-3
      Array Chunk
      Bytes 11.16 kiB 408 B
      Shape (28, 102) (1, 102)
      Count 386 Tasks 28 Chunks
      Type float32 numpy.ndarray
      		102 28
    • votemper
      (t, z)
      float32
      dask.array<chunksize=(1, 102), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      temperature
      online_operation :
      average
      standard_name :
      sea_water_potential_temperature
      units :
      degC
      Array Chunk
      Bytes 11.16 kiB 408 B
      Shape (28, 102) (1, 102)
      Count 386 Tasks 28 Chunks
      Type float32 numpy.ndarray
      		102 28
  • CASE :
    DELTA
    CONFIG :
    SEDNA
    Conventions :
    CF-1.6
    DOMAIN_dimensions_ids :
    [2, 3]
    DOMAIN_halo_size_end :
    [0, 0]
    DOMAIN_halo_size_start :
    [0, 0]
    DOMAIN_number :
    0
    DOMAIN_number_total :
    544
    DOMAIN_position_first :
    [1, 1]
    DOMAIN_position_last :
    [6560, 13]
    DOMAIN_size_global :
    [6560, 6540]
    DOMAIN_size_local :
    [6560, 13]
    DOMAIN_type :
    box
    NCO :
    netCDF Operators version 4.9.1 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)
    description :
    ocean T grid variables
    history :
    Wed Jan 19 12:40:52 2022: ncks -4 -L 1 SEDNA-DELTA_1d_gridS_201202-201202_NOZIP_0000.nc /ccc/scratch/cont003/gen7420/talandel/SEDNA/SEDNA-DELTA-S/SPLIT/1d/2012/02/SEDNA-DELTA_1d_gridS_201202-201202_0000.nc Wed Jan 19 12:40:28 2022: ncrcat -n 28,2,1 SEDNA-DELTA_1d_gridS_0000_01.nc SEDNA-DELTA_1d_gridS_201202-201202_NOZIP_0000.nc
    ibegin :
    0
    jbegin :
    0
    name :
    /ccc/scratch/cont003/ra5563/talandel/ONGOING-RUNS/SEDNA-DELTA-XIOS.47/SEDNA-DELTA_1d_gridS
    ni :
    6560
    nj :
    13
    output_frequency :
    1d
    start_date :
    20090101
    timeStamp :
    2022-Jan-18 16:51:26 GMT
    title :
    ocean T grid variables
    uuid :
    6ca3a74a-269a-44e2-91db-2aea875dbf84
#3 Start computing 
data= data
monitor.optimize_dataset(data)
add optimise here once otimise can recognise

<xarray.Dataset>
Dimensions:        (t: 28, z: 102)
Coordinates:
    time_centered  (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t              (t) object 2012-02-01 12:00:00 ... 2012-02-28 12:00:00
    y              int64 3621
    x              int64 2495
  * z              (z) int64 1 2 3 4 5 6 7 8 9 ... 94 95 96 97 98 99 100 101 102
    mask2d         bool dask.array<chunksize=(), meta=np.ndarray>
    nav_lat        float32 dask.array<chunksize=(), meta=np.ndarray>
    mask           (z) bool dask.array<chunksize=(102,), meta=np.ndarray>
    nav_lon        float32 dask.array<chunksize=(), meta=np.ndarray>
    depth          (z) float32 dask.array<chunksize=(102,), meta=np.ndarray>
Data variables:
    vosaline       (t, z) float32 dask.array<chunksize=(1, 102), meta=np.ndarray>
    votemper       (t, z) float32 dask.array<chunksize=(1, 102), meta=np.ndarray>
Attributes: (12/26)
    CASE:                    DELTA
    CONFIG:                  SEDNA
    Conventions:             CF-1.6
    DOMAIN_dimensions_ids:   [2, 3]
    DOMAIN_halo_size_end:    [0, 0]
    DOMAIN_halo_size_start:  [0, 0]
    ...                      ...
    nj:                      13
    output_frequency:        1d
    start_date:              20090101
    timeStamp:               2022-Jan-18 16:51:26 GMT
    title:                   ocean T grid variables
    uuid:                    6ca3a74a-269a-44e2-91db-2aea875dbf84
xarray.Dataset
    • t: 28
    • z: 102
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 224 B 8 B
      Shape (28,) (1,)
      Count 171 Tasks 28 Chunks
      Type object numpy.ndarray
      		28 1
    • t
      (t)
      object
      2012-02-01 12:00:00 ... 2012-02-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 2, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 2, 28, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      ()
      int64
      3621
      array(3621)
    • x
      ()
      int64
      2495
      array(2495)
    • z
      (z)
      int64
      1 2 3 4 5 6 ... 98 99 100 101 102
      array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,  14,
        15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,  28,
        29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,  42,
        43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,  56,
        57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,  70,
        71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,  84,
        85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,  98,
        99, 100, 101, 102])
    • mask2d
      ()
      bool
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 1 B 1 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type bool numpy.ndarray
    • nav_lat
      ()
      float32
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 4 B 4 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type float32 numpy.ndarray
    • mask
      (z)
      bool
      dask.array<chunksize=(102,), meta=np.ndarray>
      Array Chunk
      Bytes 102 B 102 B
      Shape (102,) (102,)
      Count 1744 Tasks 1 Chunks
      Type bool numpy.ndarray
      		102 1
    • nav_lon
      ()
      float32
      dask.array<chunksize=(), meta=np.ndarray>
      Array Chunk
      Bytes 4 B 4 B
      Shape () ()
      Count 1743 Tasks 1 Chunks
      Type float32 numpy.ndarray
    • depth
      (z)
      float32
      dask.array<chunksize=(102,), meta=np.ndarray>
      Array Chunk
      Bytes 408 B 408 B
      Shape (102,) (102,)
      Count 1744 Tasks 1 Chunks
      Type float32 numpy.ndarray
      		102 1
    • vosaline
      (t, z)
      float32
      dask.array<chunksize=(1, 102), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      salinity
      online_operation :
      average
      standard_name :
      sea_water_practical_salinity
      units :
      1e-3
      Array Chunk
      Bytes 11.16 kiB 408 B
      Shape (28, 102) (1, 102)
      Count 386 Tasks 28 Chunks
      Type float32 numpy.ndarray
      		102 28
    • votemper
      (t, z)
      float32
      dask.array<chunksize=(1, 102), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      temperature
      online_operation :
      average
      standard_name :
      sea_water_potential_temperature
      units :
      degC
      Array Chunk
      Bytes 11.16 kiB 408 B
      Shape (28, 102) (1, 102)
      Count 386 Tasks 28 Chunks
      Type float32 numpy.ndarray
      		102 28
  • CASE :
    DELTA
    CONFIG :
    SEDNA
    Conventions :
    CF-1.6
    DOMAIN_dimensions_ids :
    [2, 3]
    DOMAIN_halo_size_end :
    [0, 0]
    DOMAIN_halo_size_start :
    [0, 0]
    DOMAIN_number :
    0
    DOMAIN_number_total :
    544
    DOMAIN_position_first :
    [1, 1]
    DOMAIN_position_last :
    [6560, 13]
    DOMAIN_size_global :
    [6560, 6540]
    DOMAIN_size_local :
    [6560, 13]
    DOMAIN_type :
    box
    NCO :
    netCDF Operators version 4.9.1 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)
    description :
    ocean T grid variables
    history :
    Wed Jan 19 12:40:52 2022: ncks -4 -L 1 SEDNA-DELTA_1d_gridS_201202-201202_NOZIP_0000.nc /ccc/scratch/cont003/gen7420/talandel/SEDNA/SEDNA-DELTA-S/SPLIT/1d/2012/02/SEDNA-DELTA_1d_gridS_201202-201202_0000.nc Wed Jan 19 12:40:28 2022: ncrcat -n 28,2,1 SEDNA-DELTA_1d_gridS_0000_01.nc SEDNA-DELTA_1d_gridS_201202-201202_NOZIP_0000.nc
    ibegin :
    0
    jbegin :
    0
    name :
    /ccc/scratch/cont003/ra5563/talandel/ONGOING-RUNS/SEDNA-DELTA-XIOS.47/SEDNA-DELTA_1d_gridS
    ni :
    6560
    nj :
    13
    output_frequency :
    1d
    start_date :
    20090101
    timeStamp :
    2022-Jan-18 16:51:26 GMT
    title :
    ocean T grid variables
    uuid :
    6ca3a74a-269a-44e2-91db-2aea875dbf84
#4 Saving  SEDNA_Mooring_Eur_B_Mooring_Eur_B
data=save.datas(data,plot=Plot,path=nc_outputpath,filename=filename)
start saving data
saving data in a  csv file ../nc_results/SEDNA_DELTA_MONITOR/SEDNA_Mooring_Eur_B_Mooring_Eur_B2012-02-01_2012-02-28.nc
save computed data at ../nc_results/SEDNA_DELTA_MONITOR/SEDNA_Mooring_Eur_B_Mooring_Eur_B2012-02-01_2012-02-28.nc completed
CPU times: user 13 s, sys: 2.29 s, total: 15.3 s
Wall time: 25.3 s