In [1]:
%matplotlib inline
import pandas as pd
import socket
host = socket.getfqdn()

from core import  load, zoom, calc, save,plots,monitor
In [2]:
#reload funcs after updating ./core/*.py
import importlib
importlib.reload(load)
importlib.reload(zoom)
importlib.reload(calc)
importlib.reload(save)
importlib.reload(plots)
importlib.reload(monitor)
Out[2]:
<module 'core.monitor' from '/ccc/work/cont003/gen7420/odakatin/monitor-sedna/notebook/core/monitor.py'>

If you submit the job with job scheduler, above¶

below are list of enviroment variable one can pass

%env local='2"¶

local : if True run dask local cluster, if not true, put number of workers setted in the 'local' if no 'local ' given, local will be setted automatically to 'True'

%env ychunk='2'¶

%env tchunk='2'¶

controls chunk. 'False' sets no modification from original netcdf file's chunk.¶

ychunk=10 will group the original netcdf file to 10 by 10¶

tchunk=1 will chunk the time coordinate one by one¶

%env file_exp=¶

'file_exp': Which 'experiment' name is it?¶

. this corresopnds to intake catalog name without path and .yaml¶

%env year=¶

for Validation, this correspoinds to path/year/month 's year¶

for monitoring, this corresponids to 'date' having * means do all files in the monitoring directory¶

setting it as 0[0-9] &1[0-9]& *[2-3][0-9], the job can be separated in three lots.¶

%env month=¶

for monitoring this corresponds to file path path-XIOS.{month}/¶

#

%env control=FWC_SSH¶

name of control file to be used for computation/plots/save/¶

AWTD.sh M_AWTMD

Fluxnet.sh M_Fluxnet

FWC_SSH.sh M_FWC_2D M_FWC_integrals M_FWC_SSH M_SSH_anomaly

Siconc.sh M_Ice_quantities

IceClim.sh M_IceClim M_IceConce M_IceThick

M_Mean_temp_velo M_MLD_2D M_Mooring M_Sectionx M_Sectiony

%env save= proceed saving? True or False , Default is setted as True¶

%env plot= proceed plotting? True or False , Default is setted as True¶

%env calc= proceed computation? or just load computed result? True or False , Default is setted as True¶

%env save=False¶

%env lazy=False¶

For debugging this cell can help¶

%env file_exp=SEDNA_DELTA_MONITOR %env year=2012 %env month=01

0[1-2]¶

%env ychunk=10 %env ychunk=False %env save=False %env plot=True %env calc=True # %env lazy=False

False¶

%env control=M_Fluxnet

M_Sectiony ok with ychunk=False local=True lazy=False¶

In [3]:
%%time
# 'savefig': Do we save output in html? or not. keep it true. 
savefig=True
client,cluster,control,catalog_url,month,year,daskreport,outputpath = load.set_control(host)
!mkdir -p $outputpath
!mkdir -p $daskreport
client

local True
using host= irene5428.c-irene.mg1.tgcc.ccc.cea.fr starting dask cluster on local= True
This code is running on  irene5428.c-irene.mg1.tgcc.ccc.cea.fr using  SEDNA_DELTA_MONITOR file experiment, read from  ../lib/SEDNA_DELTA_MONITOR.yaml  on year= 2012  on month= 01  outputpath= ../results/SEDNA_DELTA_MONITOR/ daskreport= ../results/dask/6418599irene5428.c-irene.mg1.tgcc.ccc.cea.fr_SEDNA_DELTA_MONITOR_01M_Fluxnet/
CPU times: user 3.73 s, sys: 710 ms, total: 4.44 s
Wall time: 1min 37s
Out[3]:

Client

Client-193e6dbc-13d2-11ed-a022-080038b94039

Connection method: Cluster object Cluster type: distributed.LocalCluster
Dashboard: http://127.0.0.1:8787/status

Cluster Info

LocalCluster

519e59b6

Dashboard: http://127.0.0.1:8787/status Workers: 64
Total threads: 256 Total memory: 251.06 GiB
Status: running Using processes: True

Scheduler Info

Scheduler

Scheduler-4bbbd1d9-80d7-419d-9a28-31a458a90b0a

Comm: tcp://127.0.0.1:35610 Workers: 64
Dashboard: http://127.0.0.1:8787/status Total threads: 256
Started: 1 minute ago Total memory: 251.06 GiB

Workers

Worker: 0
Comm: tcp://127.0.0.1:35304 Total threads: 4
Dashboard: http://127.0.0.1:36174/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:36038
Local directory: /tmp/dask-worker-space/worker-n0usz453

Worker: 1
Comm: tcp://127.0.0.1:34587 Total threads: 4
Dashboard: http://127.0.0.1:45008/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:41689
Local directory: /tmp/dask-worker-space/worker-ald9supo

Worker: 2
Comm: tcp://127.0.0.1:44553 Total threads: 4
Dashboard: http://127.0.0.1:33794/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:41282
Local directory: /tmp/dask-worker-space/worker-349adje5

Worker: 3
Comm: tcp://127.0.0.1:46298 Total threads: 4
Dashboard: http://127.0.0.1:45552/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:44241
Local directory: /tmp/dask-worker-space/worker-6m3ev1ph

Worker: 4
Comm: tcp://127.0.0.1:37802 Total threads: 4
Dashboard: http://127.0.0.1:40100/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:36466
Local directory: /tmp/dask-worker-space/worker-13bpnpzn

Worker: 5
Comm: tcp://127.0.0.1:45929 Total threads: 4
Dashboard: http://127.0.0.1:35902/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:37843
Local directory: /tmp/dask-worker-space/worker-8twfvpwa

Worker: 6
Comm: tcp://127.0.0.1:33080 Total threads: 4
Dashboard: http://127.0.0.1:45009/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:34242
Local directory: /tmp/dask-worker-space/worker-yttblwpb

Worker: 7
Comm: tcp://127.0.0.1:34590 Total threads: 4
Dashboard: http://127.0.0.1:36077/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:44750
Local directory: /tmp/dask-worker-space/worker-u_c_mtin

Worker: 8
Comm: tcp://127.0.0.1:33616 Total threads: 4
Dashboard: http://127.0.0.1:37661/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:36217
Local directory: /tmp/dask-worker-space/worker-ltgzy9yj

Worker: 9
Comm: tcp://127.0.0.1:42378 Total threads: 4
Dashboard: http://127.0.0.1:36862/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:34933
Local directory: /tmp/dask-worker-space/worker-6zpe881p

Worker: 10
Comm: tcp://127.0.0.1:36519 Total threads: 4
Dashboard: http://127.0.0.1:34001/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33293
Local directory: /tmp/dask-worker-space/worker-qiid5et4

Worker: 11
Comm: tcp://127.0.0.1:35726 Total threads: 4
Dashboard: http://127.0.0.1:36971/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:37907
Local directory: /tmp/dask-worker-space/worker-1b3dkuqs

Worker: 12
Comm: tcp://127.0.0.1:40580 Total threads: 4
Dashboard: http://127.0.0.1:37376/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45684
Local directory: /tmp/dask-worker-space/worker-5i0imi6e

Worker: 13
Comm: tcp://127.0.0.1:46054 Total threads: 4
Dashboard: http://127.0.0.1:37439/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45013
Local directory: /tmp/dask-worker-space/worker-wps_15d9

Worker: 14
Comm: tcp://127.0.0.1:36787 Total threads: 4
Dashboard: http://127.0.0.1:38157/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:34638
Local directory: /tmp/dask-worker-space/worker-2y8yl8x0

Worker: 15
Comm: tcp://127.0.0.1:41646 Total threads: 4
Dashboard: http://127.0.0.1:45738/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45279
Local directory: /tmp/dask-worker-space/worker-xmq10icf

Worker: 16
Comm: tcp://127.0.0.1:39348 Total threads: 4
Dashboard: http://127.0.0.1:37723/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:35557
Local directory: /tmp/dask-worker-space/worker-m2je2yke

Worker: 17
Comm: tcp://127.0.0.1:37812 Total threads: 4
Dashboard: http://127.0.0.1:46433/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:46728
Local directory: /tmp/dask-worker-space/worker-h5_77_lk

Worker: 18
Comm: tcp://127.0.0.1:39100 Total threads: 4
Dashboard: http://127.0.0.1:40734/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:42124
Local directory: /tmp/dask-worker-space/worker-eld9k0sm

Worker: 19
Comm: tcp://127.0.0.1:45213 Total threads: 4
Dashboard: http://127.0.0.1:43802/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:35036
Local directory: /tmp/dask-worker-space/worker-li138e4l

Worker: 20
Comm: tcp://127.0.0.1:44264 Total threads: 4
Dashboard: http://127.0.0.1:39116/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:46008
Local directory: /tmp/dask-worker-space/worker-gcmnqjwd

Worker: 21
Comm: tcp://127.0.0.1:43556 Total threads: 4
Dashboard: http://127.0.0.1:36327/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33974
Local directory: /tmp/dask-worker-space/worker-a7jqr6wq

Worker: 22
Comm: tcp://127.0.0.1:38685 Total threads: 4
Dashboard: http://127.0.0.1:46623/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:36573
Local directory: /tmp/dask-worker-space/worker-7i2nbcir

Worker: 23
Comm: tcp://127.0.0.1:37055 Total threads: 4
Dashboard: http://127.0.0.1:38242/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45347
Local directory: /tmp/dask-worker-space/worker-tblx3ttc

Worker: 24
Comm: tcp://127.0.0.1:41923 Total threads: 4
Dashboard: http://127.0.0.1:39588/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:46426
Local directory: /tmp/dask-worker-space/worker-neek_pv9

Worker: 25
Comm: tcp://127.0.0.1:35224 Total threads: 4
Dashboard: http://127.0.0.1:40128/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:39854
Local directory: /tmp/dask-worker-space/worker-cbkzhr8p

Worker: 26
Comm: tcp://127.0.0.1:39080 Total threads: 4
Dashboard: http://127.0.0.1:37590/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:45466
Local directory: /tmp/dask-worker-space/worker-xe5y17kr

Worker: 27
Comm: tcp://127.0.0.1:33444 Total threads: 4
Dashboard: http://127.0.0.1:40610/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43394
Local directory: /tmp/dask-worker-space/worker-78pqcvb0

Worker: 28
Comm: tcp://127.0.0.1:39910 Total threads: 4
Dashboard: http://127.0.0.1:38299/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:42083
Local directory: /tmp/dask-worker-space/worker-blsdaxzn

Worker: 29
Comm: tcp://127.0.0.1:41910 Total threads: 4
Dashboard: http://127.0.0.1:42987/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40247
Local directory: /tmp/dask-worker-space/worker-48fpsbfa

Worker: 30
Comm: tcp://127.0.0.1:36855 Total threads: 4
Dashboard: http://127.0.0.1:35156/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43793
Local directory: /tmp/dask-worker-space/worker-uunuz589

Worker: 31
Comm: tcp://127.0.0.1:39902 Total threads: 4
Dashboard: http://127.0.0.1:35431/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:41257
Local directory: /tmp/dask-worker-space/worker-b7y73xic

Worker: 32
Comm: tcp://127.0.0.1:36408 Total threads: 4
Dashboard: http://127.0.0.1:42685/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43245
Local directory: /tmp/dask-worker-space/worker-ck5l5003

Worker: 33
Comm: tcp://127.0.0.1:46225 Total threads: 4
Dashboard: http://127.0.0.1:43766/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:46306
Local directory: /tmp/dask-worker-space/worker-ej4kgiz9

Worker: 34
Comm: tcp://127.0.0.1:40895 Total threads: 4
Dashboard: http://127.0.0.1:33310/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:37526
Local directory: /tmp/dask-worker-space/worker-mgyin_mg

Worker: 35
Comm: tcp://127.0.0.1:37625 Total threads: 4
Dashboard: http://127.0.0.1:42338/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:34472
Local directory: /tmp/dask-worker-space/worker-mr5m6938

Worker: 36
Comm: tcp://127.0.0.1:33705 Total threads: 4
Dashboard: http://127.0.0.1:44648/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40226
Local directory: /tmp/dask-worker-space/worker-x5jwcuq6

Worker: 37
Comm: tcp://127.0.0.1:39919 Total threads: 4
Dashboard: http://127.0.0.1:43164/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43548
Local directory: /tmp/dask-worker-space/worker-ua8liniq

Worker: 38
Comm: tcp://127.0.0.1:44352 Total threads: 4
Dashboard: http://127.0.0.1:34120/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:38026
Local directory: /tmp/dask-worker-space/worker-c7c8f6f0

Worker: 39
Comm: tcp://127.0.0.1:46732 Total threads: 4
Dashboard: http://127.0.0.1:46259/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40693
Local directory: /tmp/dask-worker-space/worker-n6aofhd_

Worker: 40
Comm: tcp://127.0.0.1:46557 Total threads: 4
Dashboard: http://127.0.0.1:43123/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:44804
Local directory: /tmp/dask-worker-space/worker-2v02i25j

Worker: 41
Comm: tcp://127.0.0.1:37147 Total threads: 4
Dashboard: http://127.0.0.1:38040/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:37127
Local directory: /tmp/dask-worker-space/worker-o6gz25z9

Worker: 42
Comm: tcp://127.0.0.1:34072 Total threads: 4
Dashboard: http://127.0.0.1:37669/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:42988
Local directory: /tmp/dask-worker-space/worker-j0v9on8i

Worker: 43
Comm: tcp://127.0.0.1:45304 Total threads: 4
Dashboard: http://127.0.0.1:37204/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:42251
Local directory: /tmp/dask-worker-space/worker-_eamfq06

Worker: 44
Comm: tcp://127.0.0.1:40604 Total threads: 4
Dashboard: http://127.0.0.1:42786/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33781
Local directory: /tmp/dask-worker-space/worker-qcsq4gl6

Worker: 45
Comm: tcp://127.0.0.1:36637 Total threads: 4
Dashboard: http://127.0.0.1:34700/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:46209
Local directory: /tmp/dask-worker-space/worker-xx_52x2o

Worker: 46
Comm: tcp://127.0.0.1:45021 Total threads: 4
Dashboard: http://127.0.0.1:40935/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:44747
Local directory: /tmp/dask-worker-space/worker-e94fm1xd

Worker: 47
Comm: tcp://127.0.0.1:36922 Total threads: 4
Dashboard: http://127.0.0.1:37963/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:35859
Local directory: /tmp/dask-worker-space/worker-ojr287wf

Worker: 48
Comm: tcp://127.0.0.1:34865 Total threads: 4
Dashboard: http://127.0.0.1:45463/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40750
Local directory: /tmp/dask-worker-space/worker-klgi0fsu

Worker: 49
Comm: tcp://127.0.0.1:43646 Total threads: 4
Dashboard: http://127.0.0.1:37566/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33023
Local directory: /tmp/dask-worker-space/worker-5n_yxneh

Worker: 50
Comm: tcp://127.0.0.1:37287 Total threads: 4
Dashboard: http://127.0.0.1:39893/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:37286
Local directory: /tmp/dask-worker-space/worker-fylu7xa9

Worker: 51
Comm: tcp://127.0.0.1:45201 Total threads: 4
Dashboard: http://127.0.0.1:45522/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43900
Local directory: /tmp/dask-worker-space/worker-88e2eac3

Worker: 52
Comm: tcp://127.0.0.1:33757 Total threads: 4
Dashboard: http://127.0.0.1:39897/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43096
Local directory: /tmp/dask-worker-space/worker-xhs42tys

Worker: 53
Comm: tcp://127.0.0.1:33615 Total threads: 4
Dashboard: http://127.0.0.1:44887/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:40384
Local directory: /tmp/dask-worker-space/worker-yyxkqp5e

Worker: 54
Comm: tcp://127.0.0.1:41702 Total threads: 4
Dashboard: http://127.0.0.1:38251/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:38528
Local directory: /tmp/dask-worker-space/worker-37p37i67

Worker: 55
Comm: tcp://127.0.0.1:44992 Total threads: 4
Dashboard: http://127.0.0.1:44827/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:44918
Local directory: /tmp/dask-worker-space/worker-dlyur5ga

Worker: 56
Comm: tcp://127.0.0.1:46148 Total threads: 4
Dashboard: http://127.0.0.1:33315/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:33885
Local directory: /tmp/dask-worker-space/worker-b5c1hgq4

Worker: 57
Comm: tcp://127.0.0.1:46484 Total threads: 4
Dashboard: http://127.0.0.1:34487/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:38498
Local directory: /tmp/dask-worker-space/worker-66xx1x7u

Worker: 58
Comm: tcp://127.0.0.1:38996 Total threads: 4
Dashboard: http://127.0.0.1:41794/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:32907
Local directory: /tmp/dask-worker-space/worker-myzf_0dp

Worker: 59
Comm: tcp://127.0.0.1:37441 Total threads: 4
Dashboard: http://127.0.0.1:37668/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:46569
Local directory: /tmp/dask-worker-space/worker-cngfiaur

Worker: 60
Comm: tcp://127.0.0.1:36471 Total threads: 4
Dashboard: http://127.0.0.1:40569/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:43676
Local directory: /tmp/dask-worker-space/worker-47r8bhoh

Worker: 61
Comm: tcp://127.0.0.1:35623 Total threads: 4
Dashboard: http://127.0.0.1:40380/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:46059
Local directory: /tmp/dask-worker-space/worker-f4cn41on

Worker: 62
Comm: tcp://127.0.0.1:36800 Total threads: 4
Dashboard: http://127.0.0.1:37386/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:36604
Local directory: /tmp/dask-worker-space/worker-3y92fzc0

Worker: 63
Comm: tcp://127.0.0.1:43465 Total threads: 4
Dashboard: http://127.0.0.1:46572/status Memory: 3.92 GiB
Nanny: tcp://127.0.0.1:34784
Local directory: /tmp/dask-worker-space/worker-v59y_k8a

read plotting information from a csv file¶

In [4]:
df=load.controlfile(control)
#Take out 'later' tagged computations
#df=df[~df['Value'].str.contains('later')]
df
Out[4]:
Value Inputs Equation Zone Plot Colourmap MinMax Unit Oldname Unnamed: 10
Fluxnet gridV.vomecrty,param.e3v_0,param.e1v,param.mas... calc.Fluxnet(data) FramS_All Fluxnet_integrals None ((-10,10),(-10,50) ,(-150,50),(-25,5) ) (Sv,TW, mSv,10^-2 Sv) I-6
Fluxnet gridV.vomecrty,param.e3v_0,param.e1v,param.mas... calc.Fluxnet(data) Davis Fluxnet_integrals None ((-5.0,5.0),(-25,27) ,(-200,50),(-9,5) ) (Sv,TW, mSv,10^-2 Sv) I-6
Fluxnet gridV.vomecrty,param.e3v_0,param.e1v,param.mas... calc.Fluxnet(data) Bering Fluxnet_integrals None ((-2,2),(-10,50) ,(-150,50),(-2,4) ) (Sv,TW, mSv,10^-2 Sv) I-6

Computation starts here¶

Each computation consists of

  1. Load NEMO data set
  2. Zoom data set
  3. Compute (or load computed data set)
  4. Save
  5. Plot
  6. Close
In [5]:
%%time
import os
calcswitch=os.environ.get('calc', 'True') 
lazy=os.environ.get('lazy','False' )
loaddata=((df.Inputs != '').any()) 
print('calcswitch=',calcswitch,'df.Inputs != nothing',loaddata, 'lazy=',lazy)
data = load.datas(catalog_url,df.Inputs,month,year,daskreport,lazy=lazy) if ((calcswitch=='True' )*loaddata) else 0 
data

calcswitch= True df.Inputs != nothing True lazy= False
../lib/SEDNA_DELTA_MONITOR.yaml
using param_xios reading  ../lib/SEDNA_DELTA_MONITOR.yaml
using param_xios reading  <bound method DataSourceBase.describe of sources:
  param_xios:
    args:
      combine: nested
      concat_dim: y
      urlpath: /ccc/work/cont003/gen7420/odakatin/CONFIGS/SEDNA/SEDNA-I/SEDNA_Domain_cfg_Tgt_20210423_tsh10m_L1/param_f32/x_*.nc
      xarray_kwargs:
        compat: override
        coords: minimal
        data_vars: minimal
        parallel: true
    description: SEDNA NEMO parameters from MPI output  nav_lon lat fails
    driver: intake_xarray.netcdf.NetCDFSource
    metadata:
      catalog_dir: /ccc/work/cont003/gen7420/odakatin/monitor-sedna/notebook/../lib/
>
{'name': 'param_xios', 'container': 'xarray', 'plugin': ['netcdf'], 'driver': ['netcdf'], 'description': 'SEDNA NEMO parameters from MPI output  nav_lon lat fails', 'direct_access': 'forbid', 'user_parameters': [{'name': 'path', 'description': 'file coordinate', 'type': 'str', 'default': '/ccc/work/cont003/gen7420/odakatin/CONFIGS/SEDNA/MESH/SEDNA_mesh_mask_Tgt_20210423_tsh10m_L1/param'}], 'metadata': {}, 'args': {'urlpath': '/ccc/work/cont003/gen7420/odakatin/CONFIGS/SEDNA/SEDNA-I/SEDNA_Domain_cfg_Tgt_20210423_tsh10m_L1/param_f32/x_*.nc', 'combine': 'nested', 'concat_dim': 'y'}}
0 read gridS ['vosaline']
lazy= False
using load_data_xios_kerchunk reading  gridS
using load_data_xios_kerchunk reading  <bound method DataSourceBase.describe of sources:
  data_xios_kerchunk:
    args:
      consolidated: false
      storage_options:
        fo: file:////ccc/cont003/home/ra5563/ra5563/catalogue/DELTA/201201/gridS_0[0-5][0-9][0-9].json
        target_protocol: file
      urlpath: reference://
    description: CREG025 NEMO outputs from different xios server in kerchunk format
    driver: intake_xarray.xzarr.ZarrSource
    metadata:
      catalog_dir: /ccc/work/cont003/gen7420/odakatin/monitor-sedna/notebook/../lib/
>
      took 21.701594591140747 seconds
0 merging gridS ['vosaline']
1 read gridT ['votemper']
lazy= False
using load_data_xios_kerchunk reading  gridT
using load_data_xios_kerchunk reading  <bound method DataSourceBase.describe of sources:
  data_xios_kerchunk:
    args:
      consolidated: false
      storage_options:
        fo: file:////ccc/cont003/home/ra5563/ra5563/catalogue/DELTA/201201/gridT_0[0-5][0-9][0-9].json
        target_protocol: file
      urlpath: reference://
    description: CREG025 NEMO outputs from different xios server in kerchunk format
    driver: intake_xarray.xzarr.ZarrSource
    metadata:
      catalog_dir: /ccc/work/cont003/gen7420/odakatin/monitor-sedna/notebook/../lib/
>
      took 22.256991863250732 seconds
1 merging gridT ['votemper']
      took 0.9503800868988037 seconds
2 read gridV ['vomecrty']
lazy= False
using load_data_xios_kerchunk reading  gridV
using load_data_xios_kerchunk reading  <bound method DataSourceBase.describe of sources:
  data_xios_kerchunk:
    args:
      consolidated: false
      storage_options:
        fo: file:////ccc/cont003/home/ra5563/ra5563/catalogue/DELTA/201201/gridV_0[0-5][0-9][0-9].json
        target_protocol: file
      urlpath: reference://
    description: CREG025 NEMO outputs from different xios server in kerchunk format
    driver: intake_xarray.xzarr.ZarrSource
    metadata:
      catalog_dir: /ccc/work/cont003/gen7420/odakatin/monitor-sedna/notebook/../lib/
>
      took 44.17451572418213 seconds
2 merging gridV ['vomecrty']
      took 1.3552777767181396 seconds
3 read icemod ['sivolu', 'sivelv']
lazy= False
using load_data_xios_kerchunk reading  icemod
using load_data_xios_kerchunk reading  <bound method DataSourceBase.describe of sources:
  data_xios_kerchunk:
    args:
      consolidated: false
      storage_options:
        fo: file:////ccc/cont003/home/ra5563/ra5563/catalogue/DELTA/201201/icemod_0[0-5][0-9][0-9].json
        target_protocol: file
      urlpath: reference://
    description: CREG025 NEMO outputs from different xios server in kerchunk format
    driver: intake_xarray.xzarr.ZarrSource
    metadata:
      catalog_dir: /ccc/work/cont003/gen7420/odakatin/monitor-sedna/notebook/../lib/
>
      took 30.671555757522583 seconds
3 merging icemod ['sivolu', 'sivelv']
      took 0.9720711708068848 seconds
param e3v_0 will be included in data
param mask2d will be included in data
param nav_lat will be included in data
param mask will be included in data
param e1v will be included in data
param nav_lon will be included in data
CPU times: user 1min 41s, sys: 20.6 s, total: 2min 2s
Wall time: 3min 1s
Out[5]:
<xarray.Dataset>
Dimensions:        (t: 31, z: 150, y: 6540, x: 6560)
Coordinates:
    time_centered  (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t              (t) object 2012-01-01 12:00:00 ... 2012-01-31 12:00:00
  * y              (y) int64 1 2 3 4 5 6 7 ... 6535 6536 6537 6538 6539 6540
  * x              (x) int64 1 2 3 4 5 6 7 ... 6555 6556 6557 6558 6559 6560
    nav_lat        (y, x) float32 dask.array<chunksize=(13, 6560), meta=np.ndarray>
    nav_lon        (y, x) float32 dask.array<chunksize=(13, 6560), meta=np.ndarray>
  * z              (z) int64 1 2 3 4 5 6 7 8 ... 143 144 145 146 147 148 149 150
    e3v_0          (z, y, x) float64 dask.array<chunksize=(150, 13, 6560), meta=np.ndarray>
    mask2d         (y, x) bool dask.array<chunksize=(13, 6560), meta=np.ndarray>
    mask           (z, y, x) bool dask.array<chunksize=(150, 13, 6560), meta=np.ndarray>
    e1v            (y, x) float64 dask.array<chunksize=(13, 6560), meta=np.ndarray>
Data variables:
    vosaline       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 13, 6560), meta=np.ndarray>
    votemper       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 13, 6560), meta=np.ndarray>
    vomecrty       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 13, 6560), meta=np.ndarray>
    sivolu         (t, y, x) float32 dask.array<chunksize=(1, 13, 6560), meta=np.ndarray>
    sivelv         (t, y, x) float32 dask.array<chunksize=(1, 13, 6560), meta=np.ndarray>
Attributes: (12/26)
    CASE:                    DELTA
    CONFIG:                  SEDNA
    Conventions:             CF-1.6
    DOMAIN_dimensions_ids:   [2, 3]
    DOMAIN_halo_size_end:    [0, 0]
    DOMAIN_halo_size_start:  [0, 0]
    ...                      ...
    nj:                      13
    output_frequency:        1d
    start_date:              20090101
    timeStamp:               2022-Jan-17 19:00:16 GMT
    title:                   ocean T grid variables
    uuid:                    d8db76f6-a436-451a-9ab1-72dc892753af
xarray.Dataset
    • t: 31
    • z: 150
    • y: 6540
    • x: 6560
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 189 Tasks 31 Chunks
      Type object numpy.ndarray
      		31 1
    • t
      (t)
      object
      2012-01-01 12:00:00 ... 2012-01-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 1, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 28, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 29, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 30, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 31, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      (y)
      int64
      1 2 3 4 5 ... 6537 6538 6539 6540
      array([   1,    2,    3, ..., 6538, 6539, 6540])
    • x
      (x)
      int64
      1 2 3 4 5 ... 6557 6558 6559 6560
      array([   1,    2,    3, ..., 6558, 6559, 6560])
    • nav_lat
      (y, x)
      float32
      dask.array<chunksize=(13, 6560), meta=np.ndarray>
      Array Chunk
      Bytes 163.66 MiB 333.12 kiB
      Shape (6540, 6560) (13, 6560)
      Count 1632 Tasks 544 Chunks
      Type float32 numpy.ndarray
      		6560 6540
    • nav_lon
      (y, x)
      float32
      dask.array<chunksize=(13, 6560), meta=np.ndarray>
      Array Chunk
      Bytes 163.66 MiB 333.12 kiB
      Shape (6540, 6560) (13, 6560)
      Count 1632 Tasks 544 Chunks
      Type float32 numpy.ndarray
      		6560 6540
    • z
      (z)
      int64
      1 2 3 4 5 6 ... 146 147 148 149 150
      array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,  14,
        15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,  28,
        29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,  42,
        43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,  56,
        57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,  70,
        71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,  84,
        85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,  98,
        99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112,
       113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126,
       127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,
       141, 142, 143, 144, 145, 146, 147, 148, 149, 150])
    • e3v_0
      (z, y, x)
      float64
      dask.array<chunksize=(150, 13, 6560), meta=np.ndarray>
      Array Chunk
      Bytes 47.95 GiB 97.60 MiB
      Shape (150, 6540, 6560) (150, 13, 6560)
      Count 1632 Tasks 544 Chunks
      Type float64 numpy.ndarray
      		6560 6540 150
    • mask2d
      (y, x)
      bool
      dask.array<chunksize=(13, 6560), meta=np.ndarray>
      Array Chunk
      Bytes 40.91 MiB 83.28 kiB
      Shape (6540, 6560) (13, 6560)
      Count 1632 Tasks 544 Chunks
      Type bool numpy.ndarray
      		6560 6540
    • mask
      (z, y, x)
      bool
      dask.array<chunksize=(150, 13, 6560), meta=np.ndarray>
      Array Chunk
      Bytes 5.99 GiB 12.20 MiB
      Shape (150, 6540, 6560) (150, 13, 6560)
      Count 1632 Tasks 544 Chunks
      Type bool numpy.ndarray
      		6560 6540 150
    • e1v
      (y, x)
      float64
      dask.array<chunksize=(13, 6560), meta=np.ndarray>
      Array Chunk
      Bytes 327.32 MiB 666.25 kiB
      Shape (6540, 6560) (13, 6560)
      Count 1632 Tasks 544 Chunks
      Type float64 numpy.ndarray
      		6560 6540
    • vosaline
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 13, 6560), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      salinity
      online_operation :
      average
      standard_name :
      sea_water_practical_salinity
      units :
      1e-3
      Array Chunk
      Bytes 743.18 GiB 48.80 MiB
      Shape (31, 150, 6540, 6560) (1, 150, 13, 6560)
      Count 34272 Tasks 16864 Chunks
      Type float32 numpy.ndarray
      		31 1 6560 6540 150
    • votemper
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 13, 6560), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      temperature
      online_operation :
      average
      standard_name :
      sea_water_potential_temperature
      units :
      degC
      Array Chunk
      Bytes 743.18 GiB 48.80 MiB
      Shape (31, 150, 6540, 6560) (1, 150, 13, 6560)
      Count 34272 Tasks 16864 Chunks
      Type float32 numpy.ndarray
      		31 1 6560 6540 150
    • vomecrty
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 13, 6560), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      ocean current along j-axis
      online_operation :
      average
      standard_name :
      sea_water_y_velocity
      units :
      m/s
      Array Chunk
      Bytes 743.18 GiB 48.80 MiB
      Shape (31, 150, 6540, 6560) (1, 150, 13, 6560)
      Count 34272 Tasks 16864 Chunks
      Type float32 numpy.ndarray
      		31 1 6560 6540 150
    • sivolu
      (t, y, x)
      float32
      dask.array<chunksize=(1, 13, 6560), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      ice volume
      online_operation :
      average
      standard_name :
      sea_ice_thickness
      units :
      m
      Array Chunk
      Bytes 4.95 GiB 333.12 kiB
      Shape (31, 6540, 6560) (1, 13, 6560)
      Count 34272 Tasks 16864 Chunks
      Type float32 numpy.ndarray
      		6560 6540 31
    • sivelv
      (t, y, x)
      float32
      dask.array<chunksize=(1, 13, 6560), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      Y-component of sea ice velocity
      online_operation :
      average
      standard_name :
      sea_ice_y_velocity
      units :
      m/s
      Array Chunk
      Bytes 4.95 GiB 333.12 kiB
      Shape (31, 6540, 6560) (1, 13, 6560)
      Count 34272 Tasks 16864 Chunks
      Type float32 numpy.ndarray
      		6560 6540 31
  • CASE :
    DELTA
    CONFIG :
    SEDNA
    Conventions :
    CF-1.6
    DOMAIN_dimensions_ids :
    [2, 3]
    DOMAIN_halo_size_end :
    [0, 0]
    DOMAIN_halo_size_start :
    [0, 0]
    DOMAIN_number :
    0
    DOMAIN_number_total :
    544
    DOMAIN_position_first :
    [1, 1]
    DOMAIN_position_last :
    [6560, 13]
    DOMAIN_size_global :
    [6560, 6540]
    DOMAIN_size_local :
    [6560, 13]
    DOMAIN_type :
    box
    NCO :
    netCDF Operators version 4.9.1 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)
    description :
    ocean T grid variables
    history :
    Tue Jan 18 17:23:11 2022: ncks -4 -L 1 SEDNA-DELTA_1d_gridS_201201-201201_NOZIP_0000.nc /ccc/scratch/cont003/gen7420/talandel/SEDNA/SEDNA-DELTA-S/SPLIT/1d/2012/01/SEDNA-DELTA_1d_gridS_201201-201201_0000.nc Tue Jan 18 17:22:45 2022: ncrcat -n 31,2,1 SEDNA-DELTA_1d_gridS_0000_01.nc SEDNA-DELTA_1d_gridS_201201-201201_NOZIP_0000.nc
    ibegin :
    0
    jbegin :
    0
    name :
    /ccc/scratch/cont003/ra5563/talandel/ONGOING-RUNS/SEDNA-DELTA-XIOS.46/SEDNA-DELTA_1d_gridS
    ni :
    6560
    nj :
    13
    output_frequency :
    1d
    start_date :
    20090101
    timeStamp :
    2022-Jan-17 19:00:16 GMT
    title :
    ocean T grid variables
    uuid :
    d8db76f6-a436-451a-9ab1-72dc892753af
In [6]:
%%time
monitor.auto(df,data,savefig,daskreport,outputpath,file_exp='SEDNA'
            )

#calc= True
#save= True
#plot= False
Value='Fluxnet'
Zone='FramS_All'
Plot='Fluxnet_integrals'
cmap='None'
clabel='(Sv,TW, mSv,10^-2 Sv)'
clim= ((-10, 10), (-10, 50), (-150, 50), (-25, 5))
outputpath='../results/SEDNA_DELTA_MONITOR/'
nc_outputpath='../nc_results/SEDNA_DELTA_MONITOR/'
filename='SEDNA_Fluxnet_integrals_FramS_All_Fluxnet'
data=monitor.optimize_dataset(data)
#2 Zooming Data
data= zoom.FramS_All(data)
data=monitor.optimize_dataset(data)

<xarray.Dataset>
Dimensions:        (t: 31, z: 150, y: 2, x: 601)
Coordinates:
    time_centered  (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t              (t) object 2012-01-01 12:00:00 ... 2012-01-31 12:00:00
  * y              (y) int64 2608 2609
  * x              (x) int64 3734 3735 3736 3737 3738 ... 4331 4332 4333 4334
    nav_lat        (y, x) float32 dask.array<chunksize=(2, 601), meta=np.ndarray>
    nav_lon        (y, x) float32 dask.array<chunksize=(2, 601), meta=np.ndarray>
  * z              (z) int64 1 2 3 4 5 6 7 8 ... 143 144 145 146 147 148 149 150
    e3v_0          (z, y, x) float64 dask.array<chunksize=(150, 2, 601), meta=np.ndarray>
    mask2d         (y, x) bool dask.array<chunksize=(2, 601), meta=np.ndarray>
    mask           (z, y, x) bool dask.array<chunksize=(150, 2, 601), meta=np.ndarray>
    e1v            (y, x) float64 dask.array<chunksize=(2, 601), meta=np.ndarray>
Data variables:
    vosaline       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 2, 601), meta=np.ndarray>
    votemper       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 2, 601), meta=np.ndarray>
    vomecrty       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 2, 601), meta=np.ndarray>
    sivolu         (t, y, x, z) float32 dask.array<chunksize=(1, 2, 601, 150), meta=np.ndarray>
    sivelv         (t, y, x, z) float32 dask.array<chunksize=(1, 2, 601, 150), meta=np.ndarray>
Attributes: (12/26)
    CASE:                    DELTA
    CONFIG:                  SEDNA
    Conventions:             CF-1.6
    DOMAIN_dimensions_ids:   [2, 3]
    DOMAIN_halo_size_end:    [0, 0]
    DOMAIN_halo_size_start:  [0, 0]
    ...                      ...
    nj:                      13
    output_frequency:        1d
    start_date:              20090101
    timeStamp:               2022-Jan-17 19:00:16 GMT
    title:                   ocean T grid variables
    uuid:                    d8db76f6-a436-451a-9ab1-72dc892753af
xarray.Dataset
    • t: 31
    • z: 150
    • y: 2
    • x: 601
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 189 Tasks 31 Chunks
      Type object numpy.ndarray
      		31 1
    • t
      (t)
      object
      2012-01-01 12:00:00 ... 2012-01-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 1, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 28, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 29, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 30, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 31, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      (y)
      int64
      2608 2609
      array([2608, 2609])
    • x
      (x)
      int64
      3734 3735 3736 ... 4332 4333 4334
      array([3734, 3735, 3736, ..., 4332, 4333, 4334])
    • nav_lat
      (y, x)
      float32
      dask.array<chunksize=(2, 601), meta=np.ndarray>
      Array Chunk
      Bytes 4.70 kiB 4.70 kiB
      Shape (2, 601) (2, 601)
      Count 1633 Tasks 1 Chunks
      Type float32 numpy.ndarray
      		601 2
    • nav_lon
      (y, x)
      float32
      dask.array<chunksize=(2, 601), meta=np.ndarray>
      Array Chunk
      Bytes 4.70 kiB 4.70 kiB
      Shape (2, 601) (2, 601)
      Count 1633 Tasks 1 Chunks
      Type float32 numpy.ndarray
      		601 2
    • z
      (z)
      int64
      1 2 3 4 5 6 ... 146 147 148 149 150
      array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,  14,
        15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,  28,
        29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,  42,
        43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,  56,
        57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,  70,
        71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,  84,
        85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,  98,
        99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112,
       113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126,
       127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,
       141, 142, 143, 144, 145, 146, 147, 148, 149, 150])
    • e3v_0
      (z, y, x)
      float64
      dask.array<chunksize=(150, 2, 601), meta=np.ndarray>
      Array Chunk
      Bytes 1.38 MiB 1.38 MiB
      Shape (150, 2, 601) (150, 2, 601)
      Count 1633 Tasks 1 Chunks
      Type float64 numpy.ndarray
      		601 2 150
    • mask2d
      (y, x)
      bool
      dask.array<chunksize=(2, 601), meta=np.ndarray>
      Array Chunk
      Bytes 1.17 kiB 1.17 kiB
      Shape (2, 601) (2, 601)
      Count 1633 Tasks 1 Chunks
      Type bool numpy.ndarray
      		601 2
    • mask
      (z, y, x)
      bool
      dask.array<chunksize=(150, 2, 601), meta=np.ndarray>
      Array Chunk
      Bytes 176.07 kiB 176.07 kiB
      Shape (150, 2, 601) (150, 2, 601)
      Count 1633 Tasks 1 Chunks
      Type bool numpy.ndarray
      		601 2 150
    • e1v
      (y, x)
      float64
      dask.array<chunksize=(2, 601), meta=np.ndarray>
      Array Chunk
      Bytes 9.39 kiB 9.39 kiB
      Shape (2, 601) (2, 601)
      Count 1633 Tasks 1 Chunks
      Type float64 numpy.ndarray
      		601 2
    • vosaline
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 2, 601), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      salinity
      online_operation :
      average
      standard_name :
      sea_water_practical_salinity
      units :
      1e-3
      Array Chunk
      Bytes 21.32 MiB 704.30 kiB
      Shape (31, 150, 2, 601) (1, 150, 2, 601)
      Count 97 Tasks 31 Chunks
      Type float32 numpy.ndarray
      		31 1 601 2 150
    • votemper
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 2, 601), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      temperature
      online_operation :
      average
      standard_name :
      sea_water_potential_temperature
      units :
      degC
      Array Chunk
      Bytes 21.32 MiB 704.30 kiB
      Shape (31, 150, 2, 601) (1, 150, 2, 601)
      Count 97 Tasks 31 Chunks
      Type float32 numpy.ndarray
      		31 1 601 2 150
    • vomecrty
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 2, 601), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      ocean current along j-axis
      online_operation :
      average
      standard_name :
      sea_water_y_velocity
      units :
      m/s
      Array Chunk
      Bytes 21.32 MiB 704.30 kiB
      Shape (31, 150, 2, 601) (1, 150, 2, 601)
      Count 97 Tasks 31 Chunks
      Type float32 numpy.ndarray
      		31 1 601 2 150
    • sivolu
      (t, y, x, z)
      float32
      dask.array<chunksize=(1, 2, 601, 150), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      ice volume
      online_operation :
      average
      standard_name :
      sea_ice_thickness
      units :
      m
      Array Chunk
      Bytes 21.32 MiB 704.30 kiB
      Shape (31, 2, 601, 150) (1, 2, 601, 150)
      Count 97 Tasks 31 Chunks
      Type float32 numpy.ndarray
      		31 1 150 601 2
    • sivelv
      (t, y, x, z)
      float32
      dask.array<chunksize=(1, 2, 601, 150), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      Y-component of sea ice velocity
      online_operation :
      average
      standard_name :
      sea_ice_y_velocity
      units :
      m/s
      Array Chunk
      Bytes 21.32 MiB 704.30 kiB
      Shape (31, 2, 601, 150) (1, 2, 601, 150)
      Count 97 Tasks 31 Chunks
      Type float32 numpy.ndarray
      		31 1 150 601 2
  • CASE :
    DELTA
    CONFIG :
    SEDNA
    Conventions :
    CF-1.6
    DOMAIN_dimensions_ids :
    [2, 3]
    DOMAIN_halo_size_end :
    [0, 0]
    DOMAIN_halo_size_start :
    [0, 0]
    DOMAIN_number :
    0
    DOMAIN_number_total :
    544
    DOMAIN_position_first :
    [1, 1]
    DOMAIN_position_last :
    [6560, 13]
    DOMAIN_size_global :
    [6560, 6540]
    DOMAIN_size_local :
    [6560, 13]
    DOMAIN_type :
    box
    NCO :
    netCDF Operators version 4.9.1 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)
    description :
    ocean T grid variables
    history :
    Tue Jan 18 17:23:11 2022: ncks -4 -L 1 SEDNA-DELTA_1d_gridS_201201-201201_NOZIP_0000.nc /ccc/scratch/cont003/gen7420/talandel/SEDNA/SEDNA-DELTA-S/SPLIT/1d/2012/01/SEDNA-DELTA_1d_gridS_201201-201201_0000.nc Tue Jan 18 17:22:45 2022: ncrcat -n 31,2,1 SEDNA-DELTA_1d_gridS_0000_01.nc SEDNA-DELTA_1d_gridS_201201-201201_NOZIP_0000.nc
    ibegin :
    0
    jbegin :
    0
    name :
    /ccc/scratch/cont003/ra5563/talandel/ONGOING-RUNS/SEDNA-DELTA-XIOS.46/SEDNA-DELTA_1d_gridS
    ni :
    6560
    nj :
    13
    output_frequency :
    1d
    start_date :
    20090101
    timeStamp :
    2022-Jan-17 19:00:16 GMT
    title :
    ocean T grid variables
    uuid :
    d8db76f6-a436-451a-9ab1-72dc892753af
#3 Start computing 
data= calc.Fluxnet(data)
monitor.optimize_dataset(data)
add optimise here once otimise can recognise

<xarray.Dataset>
Dimensions:                (t: 31)
Coordinates:
    time_centered          (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t                      (t) object 2012-01-01 12:00:00 ... 2012-01-31 12:0...
    y                      int64 2608
Data variables:
    Volume flux Net        (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Volume flux Northward  (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Heat flux Net          (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Heat flux Northward    (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Freshwater Net         (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Freshwater Northward   (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Ice export             (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Volume flux South      (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Heat flux South        (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Freshwater South       (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
xarray.Dataset
    • t: 31
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 189 Tasks 31 Chunks
      Type object numpy.ndarray
      		31 1
    • t
      (t)
      object
      2012-01-01 12:00:00 ... 2012-01-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 1, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 28, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 29, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 30, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 31, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      ()
      int64
      2608
      array(2608)
    • Volume flux Net
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 5312 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Volume flux Northward
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 5374 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Heat flux Net
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6461 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Heat flux Northward
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6523 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Freshwater Net
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6461 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Freshwater Northward
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6523 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Ice export
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 4736 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Volume flux South
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 5560 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Heat flux South
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6740 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Freshwater South
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6740 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1 
#4 Saving  SEDNA_Fluxnet_integrals_FramS_All_Fluxnet
data=save.datas(data,plot=Plot,path=nc_outputpath,filename=filename)
start saving data
saving data in a  csv file ../nc_results/SEDNA_DELTA_MONITOR/SEDNA_Fluxnet_integrals_FramS_All_Fluxnet2012-01-01_2012-01-31.nc
save computed data at ../nc_results/SEDNA_DELTA_MONITOR/SEDNA_Fluxnet_integrals_FramS_All_Fluxnet2012-01-01_2012-01-31.nc completed
Value='Fluxnet'
Zone='Davis'
Plot='Fluxnet_integrals'
cmap='None'
clabel='(Sv,TW, mSv,10^-2 Sv)'
clim= ((-5.0, 5.0), (-25, 27), (-200, 50), (-9, 5))
outputpath='../results/SEDNA_DELTA_MONITOR/'
nc_outputpath='../nc_results/SEDNA_DELTA_MONITOR/'
filename='SEDNA_Fluxnet_integrals_Davis_Fluxnet'
data=monitor.optimize_dataset(data)
#2 Zooming Data
data= zoom.Davis(data)
data=monitor.optimize_dataset(data)

<xarray.Dataset>
Dimensions:        (t: 31, z: 150, y: 2, x: 421)
Coordinates:
    time_centered  (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t              (t) object 2012-01-01 12:00:00 ... 2012-01-31 12:00:00
  * y              (y) int64 1308 1309
  * x              (x) int64 1749 1750 1751 1752 1753 ... 2166 2167 2168 2169
    nav_lat        (y, x) float32 dask.array<chunksize=(1, 421), meta=np.ndarray>
    nav_lon        (y, x) float32 dask.array<chunksize=(1, 421), meta=np.ndarray>
  * z              (z) int64 1 2 3 4 5 6 7 8 ... 143 144 145 146 147 148 149 150
    e3v_0          (z, y, x) float64 dask.array<chunksize=(150, 1, 421), meta=np.ndarray>
    mask2d         (y, x) bool dask.array<chunksize=(1, 421), meta=np.ndarray>
    mask           (z, y, x) bool dask.array<chunksize=(150, 1, 421), meta=np.ndarray>
    e1v            (y, x) float64 dask.array<chunksize=(1, 421), meta=np.ndarray>
Data variables:
    vosaline       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 1, 421), meta=np.ndarray>
    votemper       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 1, 421), meta=np.ndarray>
    vomecrty       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 1, 421), meta=np.ndarray>
    sivolu         (t, y, x, z) float32 dask.array<chunksize=(1, 1, 421, 150), meta=np.ndarray>
    sivelv         (t, y, x, z) float32 dask.array<chunksize=(1, 1, 421, 150), meta=np.ndarray>
Attributes: (12/26)
    CASE:                    DELTA
    CONFIG:                  SEDNA
    Conventions:             CF-1.6
    DOMAIN_dimensions_ids:   [2, 3]
    DOMAIN_halo_size_end:    [0, 0]
    DOMAIN_halo_size_start:  [0, 0]
    ...                      ...
    nj:                      13
    output_frequency:        1d
    start_date:              20090101
    timeStamp:               2022-Jan-17 19:00:16 GMT
    title:                   ocean T grid variables
    uuid:                    d8db76f6-a436-451a-9ab1-72dc892753af
xarray.Dataset
    • t: 31
    • z: 150
    • y: 2
    • x: 421
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 189 Tasks 31 Chunks
      Type object numpy.ndarray
      		31 1
    • t
      (t)
      object
      2012-01-01 12:00:00 ... 2012-01-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 1, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 28, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 29, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 30, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 31, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      (y)
      int64
      1308 1309
      array([1308, 1309])
    • x
      (x)
      int64
      1749 1750 1751 ... 2167 2168 2169
      array([1749, 1750, 1751, ..., 2167, 2168, 2169])
    • nav_lat
      (y, x)
      float32
      dask.array<chunksize=(1, 421), meta=np.ndarray>
      Array Chunk
      Bytes 3.29 kiB 1.64 kiB
      Shape (2, 421) (1, 421)
      Count 1634 Tasks 2 Chunks
      Type float32 numpy.ndarray
      		421 2
    • nav_lon
      (y, x)
      float32
      dask.array<chunksize=(1, 421), meta=np.ndarray>
      Array Chunk
      Bytes 3.29 kiB 1.64 kiB
      Shape (2, 421) (1, 421)
      Count 1634 Tasks 2 Chunks
      Type float32 numpy.ndarray
      		421 2
    • z
      (z)
      int64
      1 2 3 4 5 6 ... 146 147 148 149 150
      array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,  14,
        15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,  28,
        29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,  42,
        43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,  56,
        57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,  70,
        71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,  84,
        85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,  98,
        99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112,
       113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126,
       127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,
       141, 142, 143, 144, 145, 146, 147, 148, 149, 150])
    • e3v_0
      (z, y, x)
      float64
      dask.array<chunksize=(150, 1, 421), meta=np.ndarray>
      Array Chunk
      Bytes 0.96 MiB 493.36 kiB
      Shape (150, 2, 421) (150, 1, 421)
      Count 1634 Tasks 2 Chunks
      Type float64 numpy.ndarray
      		421 2 150
    • mask2d
      (y, x)
      bool
      dask.array<chunksize=(1, 421), meta=np.ndarray>
      Array Chunk
      Bytes 842 B 421 B
      Shape (2, 421) (1, 421)
      Count 1634 Tasks 2 Chunks
      Type bool numpy.ndarray
      		421 2
    • mask
      (z, y, x)
      bool
      dask.array<chunksize=(150, 1, 421), meta=np.ndarray>
      Array Chunk
      Bytes 123.34 kiB 61.67 kiB
      Shape (150, 2, 421) (150, 1, 421)
      Count 1634 Tasks 2 Chunks
      Type bool numpy.ndarray
      		421 2 150
    • e1v
      (y, x)
      float64
      dask.array<chunksize=(1, 421), meta=np.ndarray>
      Array Chunk
      Bytes 6.58 kiB 3.29 kiB
      Shape (2, 421) (1, 421)
      Count 1634 Tasks 2 Chunks
      Type float64 numpy.ndarray
      		421 2
    • vosaline
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 1, 421), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      salinity
      online_operation :
      average
      standard_name :
      sea_water_practical_salinity
      units :
      1e-3
      Array Chunk
      Bytes 14.94 MiB 246.68 kiB
      Shape (31, 150, 2, 421) (1, 150, 1, 421)
      Count 194 Tasks 62 Chunks
      Type float32 numpy.ndarray
      		31 1 421 2 150
    • votemper
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 1, 421), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      temperature
      online_operation :
      average
      standard_name :
      sea_water_potential_temperature
      units :
      degC
      Array Chunk
      Bytes 14.94 MiB 246.68 kiB
      Shape (31, 150, 2, 421) (1, 150, 1, 421)
      Count 194 Tasks 62 Chunks
      Type float32 numpy.ndarray
      		31 1 421 2 150
    • vomecrty
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 1, 421), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      ocean current along j-axis
      online_operation :
      average
      standard_name :
      sea_water_y_velocity
      units :
      m/s
      Array Chunk
      Bytes 14.94 MiB 246.68 kiB
      Shape (31, 150, 2, 421) (1, 150, 1, 421)
      Count 194 Tasks 62 Chunks
      Type float32 numpy.ndarray
      		31 1 421 2 150
    • sivolu
      (t, y, x, z)
      float32
      dask.array<chunksize=(1, 1, 421, 150), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      ice volume
      online_operation :
      average
      standard_name :
      sea_ice_thickness
      units :
      m
      Array Chunk
      Bytes 14.94 MiB 246.68 kiB
      Shape (31, 2, 421, 150) (1, 1, 421, 150)
      Count 194 Tasks 62 Chunks
      Type float32 numpy.ndarray
      		31 1 150 421 2
    • sivelv
      (t, y, x, z)
      float32
      dask.array<chunksize=(1, 1, 421, 150), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      Y-component of sea ice velocity
      online_operation :
      average
      standard_name :
      sea_ice_y_velocity
      units :
      m/s
      Array Chunk
      Bytes 14.94 MiB 246.68 kiB
      Shape (31, 2, 421, 150) (1, 1, 421, 150)
      Count 194 Tasks 62 Chunks
      Type float32 numpy.ndarray
      		31 1 150 421 2
  • CASE :
    DELTA
    CONFIG :
    SEDNA
    Conventions :
    CF-1.6
    DOMAIN_dimensions_ids :
    [2, 3]
    DOMAIN_halo_size_end :
    [0, 0]
    DOMAIN_halo_size_start :
    [0, 0]
    DOMAIN_number :
    0
    DOMAIN_number_total :
    544
    DOMAIN_position_first :
    [1, 1]
    DOMAIN_position_last :
    [6560, 13]
    DOMAIN_size_global :
    [6560, 6540]
    DOMAIN_size_local :
    [6560, 13]
    DOMAIN_type :
    box
    NCO :
    netCDF Operators version 4.9.1 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)
    description :
    ocean T grid variables
    history :
    Tue Jan 18 17:23:11 2022: ncks -4 -L 1 SEDNA-DELTA_1d_gridS_201201-201201_NOZIP_0000.nc /ccc/scratch/cont003/gen7420/talandel/SEDNA/SEDNA-DELTA-S/SPLIT/1d/2012/01/SEDNA-DELTA_1d_gridS_201201-201201_0000.nc Tue Jan 18 17:22:45 2022: ncrcat -n 31,2,1 SEDNA-DELTA_1d_gridS_0000_01.nc SEDNA-DELTA_1d_gridS_201201-201201_NOZIP_0000.nc
    ibegin :
    0
    jbegin :
    0
    name :
    /ccc/scratch/cont003/ra5563/talandel/ONGOING-RUNS/SEDNA-DELTA-XIOS.46/SEDNA-DELTA_1d_gridS
    ni :
    6560
    nj :
    13
    output_frequency :
    1d
    start_date :
    20090101
    timeStamp :
    2022-Jan-17 19:00:16 GMT
    title :
    ocean T grid variables
    uuid :
    d8db76f6-a436-451a-9ab1-72dc892753af
#3 Start computing 
data= calc.Fluxnet(data)
monitor.optimize_dataset(data)
add optimise here once otimise can recognise

<xarray.Dataset>
Dimensions:                (t: 31)
Coordinates:
    time_centered          (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t                      (t) object 2012-01-01 12:00:00 ... 2012-01-31 12:0...
    y                      int64 1308
Data variables:
    Volume flux Net        (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Volume flux Northward  (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Heat flux Net          (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Heat flux Northward    (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Freshwater Net         (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Freshwater Northward   (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Ice export             (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Volume flux South      (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Heat flux South        (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Freshwater South       (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
xarray.Dataset
    • t: 31
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 189 Tasks 31 Chunks
      Type object numpy.ndarray
      		31 1
    • t
      (t)
      object
      2012-01-01 12:00:00 ... 2012-01-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 1, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 28, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 29, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 30, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 31, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      ()
      int64
      1308
      array(1308)
    • Volume flux Net
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 5412 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Volume flux Northward
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 5474 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Heat flux Net
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6813 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Heat flux Northward
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6875 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Freshwater Net
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6813 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Freshwater Northward
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6875 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Ice export
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 5087 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Volume flux South
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 5660 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Heat flux South
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 7092 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Freshwater South
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 7092 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1 
#4 Saving  SEDNA_Fluxnet_integrals_Davis_Fluxnet
data=save.datas(data,plot=Plot,path=nc_outputpath,filename=filename)
start saving data
saving data in a  csv file ../nc_results/SEDNA_DELTA_MONITOR/SEDNA_Fluxnet_integrals_Davis_Fluxnet2012-01-01_2012-01-31.nc
save computed data at ../nc_results/SEDNA_DELTA_MONITOR/SEDNA_Fluxnet_integrals_Davis_Fluxnet2012-01-01_2012-01-31.nc completed
Value='Fluxnet'
Zone='Bering'
Plot='Fluxnet_integrals'
cmap='None'
clabel='(Sv,TW, mSv,10^-2 Sv)'
clim= ((-2, 2), (-10, 50), (-150, 50), (-2, 4))
outputpath='../results/SEDNA_DELTA_MONITOR/'
nc_outputpath='../nc_results/SEDNA_DELTA_MONITOR/'
filename='SEDNA_Fluxnet_integrals_Bering_Fluxnet'
data=monitor.optimize_dataset(data)
#2 Zooming Data
data= zoom.Bering(data)
data=monitor.optimize_dataset(data)

<xarray.Dataset>
Dimensions:        (t: 31, z: 150, y: 2, x: 146)
Coordinates:
    time_centered  (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t              (t) object 2012-01-01 12:00:00 ... 2012-01-31 12:00:00
  * y              (y) int64 6538 6539
  * x              (x) int64 2421 2422 2423 2424 2425 ... 2563 2564 2565 2566
    nav_lat        (y, x) float32 dask.array<chunksize=(2, 146), meta=np.ndarray>
    nav_lon        (y, x) float32 dask.array<chunksize=(2, 146), meta=np.ndarray>
  * z              (z) int64 1 2 3 4 5 6 7 8 ... 143 144 145 146 147 148 149 150
    e3v_0          (z, y, x) float64 dask.array<chunksize=(150, 2, 146), meta=np.ndarray>
    mask2d         (y, x) bool dask.array<chunksize=(2, 146), meta=np.ndarray>
    mask           (z, y, x) bool dask.array<chunksize=(150, 2, 146), meta=np.ndarray>
    e1v            (y, x) float64 dask.array<chunksize=(2, 146), meta=np.ndarray>
Data variables:
    vosaline       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 2, 146), meta=np.ndarray>
    votemper       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 2, 146), meta=np.ndarray>
    vomecrty       (t, z, y, x) float32 dask.array<chunksize=(1, 150, 2, 146), meta=np.ndarray>
    sivolu         (t, y, x, z) float32 dask.array<chunksize=(1, 2, 146, 150), meta=np.ndarray>
    sivelv         (t, y, x, z) float32 dask.array<chunksize=(1, 2, 146, 150), meta=np.ndarray>
Attributes: (12/26)
    CASE:                    DELTA
    CONFIG:                  SEDNA
    Conventions:             CF-1.6
    DOMAIN_dimensions_ids:   [2, 3]
    DOMAIN_halo_size_end:    [0, 0]
    DOMAIN_halo_size_start:  [0, 0]
    ...                      ...
    nj:                      13
    output_frequency:        1d
    start_date:              20090101
    timeStamp:               2022-Jan-17 19:00:16 GMT
    title:                   ocean T grid variables
    uuid:                    d8db76f6-a436-451a-9ab1-72dc892753af
xarray.Dataset
    • t: 31
    • z: 150
    • y: 2
    • x: 146
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 189 Tasks 31 Chunks
      Type object numpy.ndarray
      		31 1
    • t
      (t)
      object
      2012-01-01 12:00:00 ... 2012-01-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 1, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 28, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 29, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 30, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 31, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      (y)
      int64
      6538 6539
      array([6538, 6539])
    • x
      (x)
      int64
      2421 2422 2423 ... 2564 2565 2566
      array([2421, 2422, 2423, 2424, 2425, 2426, 2427, 2428, 2429, 2430, 2431, 2432,
       2433, 2434, 2435, 2436, 2437, 2438, 2439, 2440, 2441, 2442, 2443, 2444,
       2445, 2446, 2447, 2448, 2449, 2450, 2451, 2452, 2453, 2454, 2455, 2456,
       2457, 2458, 2459, 2460, 2461, 2462, 2463, 2464, 2465, 2466, 2467, 2468,
       2469, 2470, 2471, 2472, 2473, 2474, 2475, 2476, 2477, 2478, 2479, 2480,
       2481, 2482, 2483, 2484, 2485, 2486, 2487, 2488, 2489, 2490, 2491, 2492,
       2493, 2494, 2495, 2496, 2497, 2498, 2499, 2500, 2501, 2502, 2503, 2504,
       2505, 2506, 2507, 2508, 2509, 2510, 2511, 2512, 2513, 2514, 2515, 2516,
       2517, 2518, 2519, 2520, 2521, 2522, 2523, 2524, 2525, 2526, 2527, 2528,
       2529, 2530, 2531, 2532, 2533, 2534, 2535, 2536, 2537, 2538, 2539, 2540,
       2541, 2542, 2543, 2544, 2545, 2546, 2547, 2548, 2549, 2550, 2551, 2552,
       2553, 2554, 2555, 2556, 2557, 2558, 2559, 2560, 2561, 2562, 2563, 2564,
       2565, 2566])
    • nav_lat
      (y, x)
      float32
      dask.array<chunksize=(2, 146), meta=np.ndarray>
      Array Chunk
      Bytes 1.14 kiB 1.14 kiB
      Shape (2, 146) (2, 146)
      Count 1633 Tasks 1 Chunks
      Type float32 numpy.ndarray
      		146 2
    • nav_lon
      (y, x)
      float32
      dask.array<chunksize=(2, 146), meta=np.ndarray>
      Array Chunk
      Bytes 1.14 kiB 1.14 kiB
      Shape (2, 146) (2, 146)
      Count 1633 Tasks 1 Chunks
      Type float32 numpy.ndarray
      		146 2
    • z
      (z)
      int64
      1 2 3 4 5 6 ... 146 147 148 149 150
      array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,  14,
        15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,  28,
        29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,  42,
        43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,  56,
        57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,  70,
        71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,  84,
        85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,  98,
        99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112,
       113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126,
       127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,
       141, 142, 143, 144, 145, 146, 147, 148, 149, 150])
    • e3v_0
      (z, y, x)
      float64
      dask.array<chunksize=(150, 2, 146), meta=np.ndarray>
      Array Chunk
      Bytes 342.19 kiB 342.19 kiB
      Shape (150, 2, 146) (150, 2, 146)
      Count 1633 Tasks 1 Chunks
      Type float64 numpy.ndarray
      		146 2 150
    • mask2d
      (y, x)
      bool
      dask.array<chunksize=(2, 146), meta=np.ndarray>
      Array Chunk
      Bytes 292 B 292 B
      Shape (2, 146) (2, 146)
      Count 1633 Tasks 1 Chunks
      Type bool numpy.ndarray
      		146 2
    • mask
      (z, y, x)
      bool
      dask.array<chunksize=(150, 2, 146), meta=np.ndarray>
      Array Chunk
      Bytes 42.77 kiB 42.77 kiB
      Shape (150, 2, 146) (150, 2, 146)
      Count 1633 Tasks 1 Chunks
      Type bool numpy.ndarray
      		146 2 150
    • e1v
      (y, x)
      float64
      dask.array<chunksize=(2, 146), meta=np.ndarray>
      Array Chunk
      Bytes 2.28 kiB 2.28 kiB
      Shape (2, 146) (2, 146)
      Count 1633 Tasks 1 Chunks
      Type float64 numpy.ndarray
      		146 2
    • vosaline
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 2, 146), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      salinity
      online_operation :
      average
      standard_name :
      sea_water_practical_salinity
      units :
      1e-3
      Array Chunk
      Bytes 5.18 MiB 171.09 kiB
      Shape (31, 150, 2, 146) (1, 150, 2, 146)
      Count 97 Tasks 31 Chunks
      Type float32 numpy.ndarray
      		31 1 146 2 150
    • votemper
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 2, 146), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      temperature
      online_operation :
      average
      standard_name :
      sea_water_potential_temperature
      units :
      degC
      Array Chunk
      Bytes 5.18 MiB 171.09 kiB
      Shape (31, 150, 2, 146) (1, 150, 2, 146)
      Count 97 Tasks 31 Chunks
      Type float32 numpy.ndarray
      		31 1 146 2 150
    • vomecrty
      (t, z, y, x)
      float32
      dask.array<chunksize=(1, 150, 2, 146), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      ocean current along j-axis
      online_operation :
      average
      standard_name :
      sea_water_y_velocity
      units :
      m/s
      Array Chunk
      Bytes 5.18 MiB 171.09 kiB
      Shape (31, 150, 2, 146) (1, 150, 2, 146)
      Count 97 Tasks 31 Chunks
      Type float32 numpy.ndarray
      		31 1 146 2 150
    • sivolu
      (t, y, x, z)
      float32
      dask.array<chunksize=(1, 2, 146, 150), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      ice volume
      online_operation :
      average
      standard_name :
      sea_ice_thickness
      units :
      m
      Array Chunk
      Bytes 5.18 MiB 171.09 kiB
      Shape (31, 2, 146, 150) (1, 2, 146, 150)
      Count 97 Tasks 31 Chunks
      Type float32 numpy.ndarray
      		31 1 150 146 2
    • sivelv
      (t, y, x, z)
      float32
      dask.array<chunksize=(1, 2, 146, 150), meta=np.ndarray>
      cell_methods :
      time: mean (interval: 40 s)
      interval_operation :
      40 s
      interval_write :
      1 d
      long_name :
      Y-component of sea ice velocity
      online_operation :
      average
      standard_name :
      sea_ice_y_velocity
      units :
      m/s
      Array Chunk
      Bytes 5.18 MiB 171.09 kiB
      Shape (31, 2, 146, 150) (1, 2, 146, 150)
      Count 97 Tasks 31 Chunks
      Type float32 numpy.ndarray
      		31 1 150 146 2
  • CASE :
    DELTA
    CONFIG :
    SEDNA
    Conventions :
    CF-1.6
    DOMAIN_dimensions_ids :
    [2, 3]
    DOMAIN_halo_size_end :
    [0, 0]
    DOMAIN_halo_size_start :
    [0, 0]
    DOMAIN_number :
    0
    DOMAIN_number_total :
    544
    DOMAIN_position_first :
    [1, 1]
    DOMAIN_position_last :
    [6560, 13]
    DOMAIN_size_global :
    [6560, 6540]
    DOMAIN_size_local :
    [6560, 13]
    DOMAIN_type :
    box
    NCO :
    netCDF Operators version 4.9.1 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)
    description :
    ocean T grid variables
    history :
    Tue Jan 18 17:23:11 2022: ncks -4 -L 1 SEDNA-DELTA_1d_gridS_201201-201201_NOZIP_0000.nc /ccc/scratch/cont003/gen7420/talandel/SEDNA/SEDNA-DELTA-S/SPLIT/1d/2012/01/SEDNA-DELTA_1d_gridS_201201-201201_0000.nc Tue Jan 18 17:22:45 2022: ncrcat -n 31,2,1 SEDNA-DELTA_1d_gridS_0000_01.nc SEDNA-DELTA_1d_gridS_201201-201201_NOZIP_0000.nc
    ibegin :
    0
    jbegin :
    0
    name :
    /ccc/scratch/cont003/ra5563/talandel/ONGOING-RUNS/SEDNA-DELTA-XIOS.46/SEDNA-DELTA_1d_gridS
    ni :
    6560
    nj :
    13
    output_frequency :
    1d
    start_date :
    20090101
    timeStamp :
    2022-Jan-17 19:00:16 GMT
    title :
    ocean T grid variables
    uuid :
    d8db76f6-a436-451a-9ab1-72dc892753af
#3 Start computing 
data= calc.Fluxnet(data)
monitor.optimize_dataset(data)
add optimise here once otimise can recognise

<xarray.Dataset>
Dimensions:                (t: 31)
Coordinates:
    time_centered          (t) object dask.array<chunksize=(1,), meta=np.ndarray>
  * t                      (t) object 2012-01-01 12:00:00 ... 2012-01-31 12:0...
    y                      int64 6538
Data variables:
    Volume flux Net        (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Volume flux Northward  (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Heat flux Net          (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Heat flux Northward    (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Freshwater Net         (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Freshwater Northward   (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Ice export             (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Volume flux South      (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Heat flux South        (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
    Freshwater South       (t) float64 dask.array<chunksize=(1,), meta=np.ndarray>
xarray.Dataset
    • t: 31
    • time_centered
      (t)
      object
      dask.array<chunksize=(1,), meta=np.ndarray>
      bounds :
      time_centered_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 189 Tasks 31 Chunks
      Type object numpy.ndarray
      		31 1
    • t
      (t)
      object
      2012-01-01 12:00:00 ... 2012-01-...
      axis :
      T
      bounds :
      time_counter_bounds
      long_name :
      Time axis
      standard_name :
      time
      time_origin :
      1900-01-01 00:00:00
      array([cftime.DatetimeNoLeap(2012, 1, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 3, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 4, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 5, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 6, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 7, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 8, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 9, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 10, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 11, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 12, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 13, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 14, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 15, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 16, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 17, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 18, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 19, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 20, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 21, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 22, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 23, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 24, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 25, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 26, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 28, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 29, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 30, 12, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2012, 1, 31, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
    • y
      ()
      int64
      6538
      array(6538)
    • Volume flux Net
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 5312 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Volume flux Northward
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 5374 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Heat flux Net
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6461 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Heat flux Northward
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6523 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Freshwater Net
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6461 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Freshwater Northward
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6523 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Ice export
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 4736 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Volume flux South
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 5560 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Heat flux South
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6740 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1
    • Freshwater South
      (t)
      float64
      dask.array<chunksize=(1,), meta=np.ndarray>
      Array Chunk
      Bytes 248 B 8 B
      Shape (31,) (1,)
      Count 6740 Tasks 31 Chunks
      Type float64 numpy.ndarray
      		31 1 
#4 Saving  SEDNA_Fluxnet_integrals_Bering_Fluxnet
data=save.datas(data,plot=Plot,path=nc_outputpath,filename=filename)
start saving data
saving data in a  csv file ../nc_results/SEDNA_DELTA_MONITOR/SEDNA_Fluxnet_integrals_Bering_Fluxnet2012-01-01_2012-01-31.nc
save computed data at ../nc_results/SEDNA_DELTA_MONITOR/SEDNA_Fluxnet_integrals_Bering_Fluxnet2012-01-01_2012-01-31.nc completed
CPU times: user 29.3 s, sys: 2.86 s, total: 32.1 s
Wall time: 36.1 s