Repository navigation
Expand file tree
/
Copy pathcalibration_tutorial.html
More file actions
566 lines (548 loc) · 39.3 KB
/
Copy pathcalibration_tutorial.html
File metadata and controls
566 lines (548 loc) · 39.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>12. Calibration tutorial — Community Water Model</title>
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="_static/graphviz.css" type="text/css" />
<link rel="stylesheet" href="_static/iiasa.css" type="text/css" />
<!--[if lt IE 9]>
<script src="_static/js/html5shiv.min.js"></script>
<![endif]-->
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
<script src="_static/doctools.js"></script>
<script src="_static/sphinx_highlight.js"></script>
<script src="_static/js/theme.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="13. Increasing resolution" href="resolution.html" />
<link rel="prev" title="11. Calibration tool" href="calibration.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="index.html" class="icon icon-home">
Community Water Model
<img src="_static/IIASA_logo_white.png" class="logo" alt="Logo"/>
</a>
<div class="version">
1.0.6
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="intro.html">1. Introduction</a></li>
<li class="toctree-l1"><a class="reference internal" href="modeldesign.html">2. Model Design</a></li>
<li class="toctree-l1"><a class="reference internal" href="publication.html">3. Publication</a></li>
<li class="toctree-l1"><a class="reference internal" href="setup.html">4. Setup of the model</a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorial.html">5. Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="errorHandling.html">6. Error handling</a></li>
<li class="toctree-l1"><a class="reference internal" href="listVariables.html">7. List of output variables</a></li>
<li class="toctree-l1"><a class="reference internal" href="results.html">8. Demo of the model</a></li>
<li class="toctree-l1"><a class="reference internal" href="todo.html">9. The Model Itself</a></li>
<li class="toctree-l1"><a class="reference internal" href="data.html">10. Data</a></li>
<li class="toctree-l1"><a class="reference internal" href="calibration.html">11. Calibration tool</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">12. Calibration tutorial</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#what-you-need">What you need</a></li>
<li class="toctree-l2"><a class="reference internal" href="#running-calibration">Running calibration</a></li>
<li class="toctree-l2"><a class="reference internal" href="#run-runcalibration-bat">Run runCalibration.bat</a></li>
<li class="toctree-l2"><a class="reference internal" href="#for-testing">For testing</a></li>
<li class="toctree-l2"><a class="reference internal" href="#running-it-on-your-computer">Running it on your computer</a></li>
<li class="toctree-l2"><a class="reference internal" href="#preparation-for-another-catchment">Preparation for another catchment</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#preparing-the-observed-dataset-discharge">Preparing the observed dataset – discharge</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#creating-an-initial-netcdf-file-for-warm-start">Creating an initial netcdf file for warm start</a></li>
<li class="toctree-l2"><a class="reference internal" href="#cutting-out-a-catchment-as-mask-map">Cutting out a catchment as mask map</a></li>
<li class="toctree-l2"><a class="reference internal" href="#calibration-of-a-downstream-catchment">Calibration of a downstream catchment</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#cutting-the-mask-map">Cutting the mask map</a></li>
<li class="toctree-l3"><a class="reference internal" href="#detecting-the-downstream-point">Detecting the downstream point</a></li>
<li class="toctree-l3"><a class="reference internal" href="#run-the-best-calibration-scenario-upstream">Run the best calibration scenario upstream</a></li>
<li class="toctree-l3"><a class="reference internal" href="#create-an-inflow-file-from-the-long-timeseries-of-outlet-s">Create an inflow file from the long timeseries of outlet(s)</a></li>
<li class="toctree-l3"><a class="reference internal" href="#create-a-downstream-calibration-settings-directories-templates-etc">Create a downstream calibration settings (directories, templates etc.)</a></li>
<li class="toctree-l3"><a class="reference internal" href="#change-the-settings-file">Change the settings file</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#joining-best-sub-basin-results-to-calibration-maps">Joining best sub-basin results to calibration maps</a></li>
<li class="toctree-l2"><a class="reference internal" href="#calibration-tool-structure">Calibration tool structure</a></li>
<li class="toctree-l2"><a class="reference internal" href="#references">References</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="resolution.html">13. Increasing resolution</a></li>
<li class="toctree-l1"><a class="reference internal" href="zambezi.html">14. Example Zambezi</a></li>
<li class="toctree-l1"><a class="reference internal" href="license.html">15. License and download info</a></li>
<li class="toctree-l1"><a class="reference internal" href="sourcecode.html">16. Source code</a></li>
<li class="toctree-l1"><a class="reference internal" href="forum.html">17. Forum</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">Community Water Model</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item active"><span class="section-number">12. </span>Calibration tutorial</li>
<li class="wy-breadcrumbs-aside">
<a href="_sources/calibration_tutorial.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<section id="calibration-tutorial">
<h1><span class="section-number">12. </span>Calibration tutorial<a class="headerlink" href="#calibration-tutorial" title="Permalink to this heading"></a></h1>
<section id="what-you-need">
<h2>What you need<a class="headerlink" href="#what-you-need" title="Permalink to this heading"></a></h2>
<p>Python 2.7.x 64 bit and a running CWatM (libraries netCDF4, numpy, scipy, GDAL)
In addition: <strong>library deap</strong></p>
<p>Calibration is using a distributed evolutionary algorithms in python: DEAP library</p>
<p>Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner, Marc Parizeau and Christian Gagné, “DEAP: Evolutionary Algorithms Made Easy”, Journal of Machine Learning Research, vol. 13, pp. 2171-2175</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Deap can also be used in Python 3.7 but at the moment we did not translate the scripts to Python3.7. it is on the TODO list</p>
</div>
<p>You can install it with:
Pip install deap (you might change into the folder ../python/Scripts/)</p>
<ul class="simple">
<li><p>Make sure that python 2.7.x is working</p></li>
<li><p>Make sure that CWatM is running in non calibration mode</p></li>
<li><p>For some of the following steps it is easier to have PCRaster installed: <a class="reference external" href="http://pcraster.geo.uu.nl/">http://pcraster.geo.uu.nl/</a></p></li>
</ul>
</section>
<section id="running-calibration">
<h2>Running calibration<a class="headerlink" href="#running-calibration" title="Permalink to this heading"></a></h2>
<ol class="arabic">
<li><p>Look into the settings file of the calibration folder.</p></li>
<li><p>look into runCalibration.bat. If python is in your computer path everything should be ok, otherwise put in the path to python</p></li>
<li><p>look into templates/runpy.bat. Put the path to python in if necessary</p></li>
<li><p>look into templates/settings.ini. Put the pathes in a right way that it fits to your computer:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>[FILE_PATHS]
#-------------------------------------------------------
PathRoot = P:/watmodel/CWatM/calibration_tutorial
PathOut = $(PathRoot)/output
PathMaps = $(PathRoot)/CWatM_data/cwatm_input
PathMeteo = $(PathRoot)/climate
</pre></div>
</div>
</li>
<li><p>in observed_data/yukon2001.cvs you find the observed data:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">-</span> <span class="n">make</span> <span class="n">sure</span> <span class="n">the</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">the</span> <span class="n">header</span> <span class="ow">is</span> <span class="n">the</span> <span class="n">same</span> <span class="k">as</span> <span class="ow">in</span> <span class="p">[</span><span class="n">ObservedData</span><span class="p">]</span> <span class="n">Column</span>
<span class="o">-</span> <span class="n">make</span> <span class="n">sure</span> <span class="n">that</span> <span class="n">there</span> <span class="n">are</span> <span class="n">enough</span> <span class="n">data</span> <span class="ow">in</span> <span class="p">(</span><span class="kn">from</span> <span class="nn">ForcingStart</span> <span class="n">to</span> <span class="n">ForcingEnd</span><span class="p">)</span>
</pre></div>
</div>
</li>
<li><p>make sure the folder catchments is empty! Before each try this folder has to be empty</p></li>
</ol>
</section>
<section id="run-runcalibration-bat">
<h2>Run runCalibration.bat<a class="headerlink" href="#run-runcalibration-bat" title="Permalink to this heading"></a></h2>
<ol class="arabic simple">
<li><p>go for testing (see below)</p></li>
<li><p>go for testing again (see below)</p></li>
<li><p>Change use_multiprocessing =1 in settings.txt</p></li>
<li><p>Run runCalibration.bat and after some time something should appear on your window</p></li>
</ol>
</section>
<section id="for-testing">
<h2>For testing<a class="headerlink" href="#for-testing" title="Permalink to this heading"></a></h2>
<ul class="simple">
<li><p>Change use_multiprocessing = 0 in settings.txt</p></li>
<li><p>Delete catchments but keep the empty folder</p></li>
<li><p>Run runCalibration.bat and wait till catchment/00_001 gets filled, then interrupt</p></li>
</ul>
<ol class="arabic simple">
<li><p>Change to catchments/00_001</p></li>
<li><p>Run runpy00_001.bat</p></li>
<li><p>See what errors come up and change settings-Run00_001.ini</p></li>
<li><p>Change template/setting.ini in the same way</p></li>
<li><p>Do this again and again till no error</p></li>
</ol>
</section>
<section id="running-it-on-your-computer">
<h2>Running it on your computer<a class="headerlink" href="#running-it-on-your-computer" title="Permalink to this heading"></a></h2>
<p>It will be really slow on Windows using data on the the server – next step run it on your PC</p>
<ul>
<li><p>copy the whole folder P:watmodelCWatMcalibration_tutorial to your PC (only 15 GB)</p></li>
<li><p>(but maybe you have already parts of it on your computer – like the big climate input files)</p></li>
<li><p>Make it work on your computer:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Changing</span> <span class="n">file</span> <span class="n">paths</span> <span class="ow">in</span> <span class="n">templates</span><span class="o">/</span><span class="n">settings</span><span class="o">.</span><span class="n">ini</span><span class="p">,</span> <span class="n">setting</span><span class="o">.</span><span class="n">txt</span>
<span class="n">Changing</span> <span class="n">the</span> <span class="n">path</span> <span class="k">for</span> <span class="n">python</span> <span class="ow">in</span> <span class="n">runCalibration</span><span class="o">.</span><span class="n">bat</span> <span class="ow">and</span> <span class="n">templates</span><span class="o">/</span><span class="n">runpy</span><span class="o">.</span><span class="n">bat</span>
</pre></div>
</div>
</li>
</ul>
</section>
<section id="preparation-for-another-catchment">
<h2>Preparation for another catchment<a class="headerlink" href="#preparation-for-another-catchment" title="Permalink to this heading"></a></h2>
<section id="preparing-the-observed-dataset-discharge">
<h3>Preparing the observed dataset – discharge<a class="headerlink" href="#preparing-the-observed-dataset-discharge" title="Permalink to this heading"></a></h3>
<p>Calibration works by comparing simulated discharge with observed discharge using an objective function:
Here we use the Kling-Gupta Efficiency but we can also use Nash-Sutcliffe Efficiency . Please find some more information on the objective function an on the evolutionary computation framework used for calibration on: <a class="reference external" href="https://cwatm.github.io/calibration.html">https://cwatm.github.io/calibration.html</a></p>
<ul>
<li><p>The observed values can be stored as daily values or monthly values</p></li>
<li><p>The observed values should be at least cover 5 years (best is 10-15 years)</p></li>
<li><p>The observed discharge has to be stored as textfile in:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>./observed_data/nameofstation.cvs
And has to look like this:
date,yukon_pilot_station
2001-04-01,1302.6
2001-04-02,1302.6
2001-04-03,1302.6
2001-04-04,1302.6
…
…
2013-12-31,2647.6
</pre></div>
</div>
</li>
<li><p>Or:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">date</span> <span class="p">,</span><span class="n">zhutuo</span>
<span class="mi">2002</span><span class="o">-</span><span class="mi">01</span><span class="o">-</span><span class="mi">01</span><span class="p">,</span><span class="mf">3229.0</span>
<span class="mi">2002</span><span class="o">-</span><span class="mi">02</span><span class="o">-</span><span class="mi">01</span><span class="p">,</span><span class="mf">2979.2</span>
<span class="mi">2002</span><span class="o">-</span><span class="mi">03</span><span class="o">-</span><span class="mi">01</span><span class="p">,</span><span class="mf">3229.0</span>
</pre></div>
</div>
</li>
</ul>
<p><strong>Format:</strong></p>
<ul>
<li><p>Date format like this year-month-day [yyyy-mm-dd]</p></li>
<li><p>Separated by a comma</p></li>
<li><p>Discharge in [m3/s]</p></li>
<li><p>If a value is missing that is not a problem (as long as the time series is long enough):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">it</span> <span class="n">should</span> <span class="n">like</span> <span class="n">this</span><span class="p">:</span> <span class="p">(</span><span class="n">no</span> <span class="n">value</span> <span class="n">after</span> <span class="n">the</span> <span class="n">comma</span><span class="p">)</span>
<span class="mi">2002</span><span class="o">-</span><span class="mi">01</span><span class="o">-</span><span class="mi">12</span><span class="p">,</span>
</pre></div>
</div>
</li>
<li><p>For each day (or month) a line</p></li>
</ul>
<p><strong>Settings.txt</strong></p>
<p>In the settings file the lines:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">ObservedData</span><span class="p">]</span>
<span class="n">Qtss</span> <span class="o">=</span> <span class="n">observed_data</span><span class="o">/</span><span class="n">zhutuo_2002month</span><span class="o">.</span><span class="n">csv</span>
<span class="n">Column</span> <span class="o">=</span> <span class="n">zhutuo</span>
<span class="n">Header</span> <span class="o">=</span> <span class="n">River</span><span class="p">:</span> <span class="n">Yangtze</span> <span class="n">station</span><span class="p">:</span> <span class="n">Zhutuo</span>
</pre></div>
</div>
<p>Should correspondent to the name and header in the observed discharge.cvs</p>
<p>The lines:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ForcingStart</span> <span class="o">=</span> <span class="mi">1</span><span class="o">/</span><span class="mi">1</span><span class="o">/</span><span class="mi">2002</span>
<span class="n">ForcingEnd</span> <span class="o">=</span> <span class="mi">31</span><span class="o">/</span><span class="mi">12</span><span class="o">/</span><span class="mi">2013</span>
</pre></div>
</div>
<p>Should correspondent to the amount of lines in the observed discharge.cvs</p>
</section>
</section>
<section id="creating-an-initial-netcdf-file-for-warm-start">
<h2>Creating an initial netcdf file for warm start<a class="headerlink" href="#creating-an-initial-netcdf-file-for-warm-start" title="Permalink to this heading"></a></h2>
<p>It is best to have a long warm up phase especially for groundwater:
See also: <a class="reference external" href="https://cwatm.github.io/setup.html#initialisation">https://cwatm.github.io/setup.html#initialisation</a></p>
<p>You can run CWatM for a couple of years (20 years or more) and store the last days storage values in a file. This file can be read in to enable a ‘warm” start</p>
<ul class="simple">
<li><p>change use_multiprocessing = 0 in settings.txt</p></li>
<li><p>Delete catchments but keep the empty folder</p></li>
<li><p>Run runCalibration.bat and wait till catchment/00_001 gets filled, then interrupt</p></li>
<li><p>Change to catchments/00_001</p></li>
</ul>
<p><strong>Open the settings-Run_001.init</strong></p>
<ul class="simple">
<li><p>Change load_initial = True to load_initial = False</p></li>
<li><p>save_initial = True</p></li>
<li><p>initSave = $(FILE_PATHS:PathRoot)/CWatM_init/testx</p></li>
<li><p>StepInit = 31/01/1996 (change it to a date 1 month after your StepStart)</p></li>
<li><p>Run runpy00_001.bat</p></li>
</ul>
<p><span class="raw-html">→</span> There should be a file ./CWatM_init/testx_19960131.nc</p>
<ul class="simple">
<li><p>Change to: load_initial = True</p></li>
<li><p>initLoad = $(FILE_PATHS:PathRoot)/CWatM_init/testx_19960131.nc</p></li>
<li><p>Run runpy00_001.bat</p></li>
</ul>
<p><span class="raw-html">→</span> If it work then it used the initial file you generate before (that was just a test)</p>
<p><strong>Now change to:</strong></p>
<ul class="simple">
<li><p>StepStart = 1/1/1961</p></li>
<li><p>StepEnd = 31/12/2013</p></li>
<li><p>load_initial = False</p></li>
<li><p>save_initial = True</p></li>
<li><p>initSave = $(FILE_PATHS:PathRoot)/CWatM_init/station_name</p></li>
<li><p>StepInit = 31/12/2013</p></li>
<li><p>Run runpy00_001.bat</p></li>
</ul>
<p><span class="raw-html">→</span> This should have generated a file ./CWatM_init/station_name_20131231.nc</p>
<p><strong>And again:</strong></p>
<ul class="simple">
<li><p>StepStart = 1/1/1961 (some 20 years or longer)</p></li>
<li><p>StepEnd = 31/12/1995 (a day before your normal running day)</p></li>
<li><p>load_initial = True</p></li>
<li><p>initLoad = $(FILE_PATHS:PathRoot)/CWatM_init/station_name_20131231.nc</p></li>
<li><p>save_initial = True</p></li>
<li><p>initSave = $(FILE_PATHS:PathRoot)/CWatM_init/station_name</p></li>
<li><p>StepInit = 31/12/1995 (a day before your running day)</p></li>
<li><p>Run runpy00_001.bat</p></li>
</ul>
<p><span class="raw-html">→</span> This should have generated a file ./CWatM_init/station_name_19951231.nc</p>
<p><strong>And last part:</strong></p>
<ul class="simple">
<li><p>Change StepStart and StepEnd back to original values</p></li>
<li><p>load_initial = True</p></li>
<li><p>initLoad = $(FILE_PATHS:PathRoot)/CWatM_init/station_name_19951231.nc</p></li>
<li><p>save_initial = False</p></li>
<li><p>Run runpy00_001.bat</p></li>
</ul>
<p><span class="raw-html">→</span> If it works, do the same in the ./template/settings.ini</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>You have now a “warm” start for every calibration run</p>
</div>
</section>
<section id="cutting-out-a-catchment-as-mask-map">
<h2>Cutting out a catchment as mask map<a class="headerlink" href="#cutting-out-a-catchment-as-mask-map" title="Permalink to this heading"></a></h2>
<p>See the .doc file in
P:watmodelCWatMcalibration_tutorialcalibrationtoolscut_catchmentFor a description:</p>
<p><strong>Requirements:</strong> PCRASTER:</p>
<p>We do no need the python version, I think downloading, extracting and setting of the paths in
P:watmodelCWatMcalibration_tutorialcalibrationtoolscut_catchmentcatchconfig_win.ini
Creating the 2 potential evaporation files in advance</p>
<p>Potential evaporation is Calculated with Penman-Monteith in CWatM, but it is not part of the calibration = there is no change in pot. Evaporation. In order to make the calibration computational faster the results of pot evaporation could be stored and used every time.</p>
<p>For the 30min this is done already as global map set, but for the 5min these files become too big. So they have to be produced for each basin separately</p>
<p>Same preparation as for <strong>Creating an initial netcdf file for warm start</strong> see above
There should be a folder catchments00_001 with a working run for 001.</p>
<p><strong>Open the settings-Run_001.init</strong></p>
<p>Change:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>[Option] calc_evaporation = True
[TIME-RELATED_CONSTANTS] SpinUp = None
[EVAPORATION]
OUT_Dir = $(FILE_PATHS:PathOut)
OUT_MAP_Daily = ETRef, EWRef
</pre></div>
</div>
<p><strong>Run runpy00_001.bat</strong>
<span class="raw-html">→</span> There should be a file ETRef.nc and EWRef in the output directory</p>
<p>Rename the files e.g. ETRef.nc to ETRef_yangtze.nc, EWRef.nc to EWRef_yangtze.nc and copy it to PathMeteo (or somewhere else, you have to put the path in)</p>
<p><strong>Open the settings-Run_001.init</strong></p>
<p>Change:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>[Option] calc_evaporation = False
[TIME-RELATED_CONSTANTS] SpinUp = -> to the time it was before
[Meteo]
daily reference evaporation (free water)
E0Maps = $(FILE_PATHS:PathMeteo)/EWRef_yangtze
daily reference evapotranspiration (crop)
ETMaps = $(FILE_PATHS:PathMeteo)/ETRef_yangtze
[EVAPORATION]
OUT_Dir = $(FILE_PATHS:PathOut) !!! outcomment this again - important
OUT_MAP_Daily = ETRef, EWRef
</pre></div>
</div>
<p><strong>Test it:</strong> Run runpy00_001.bat</p>
<p>And change the settings.ini in templates in the same way</p>
</section>
<section id="calibration-of-a-downstream-catchment">
<h2>Calibration of a downstream catchment<a class="headerlink" href="#calibration-of-a-downstream-catchment" title="Permalink to this heading"></a></h2>
<p>Calibration of a downstream catchment (upstream catchment is already calibrated) can be done using:</p>
<ul class="simple">
<li><p>The catchment area of the downstream catchment minus the upstream catchment</p></li>
<li><p>The missing discharge from the upstream catchment is replaced by an inflow file</p></li>
</ul>
<ol class="arabic simple">
<li><p>Cut the mask map, so that the upstream catchment is NOT in the mask map anymore</p></li>
<li><p>Detect the point(s) downstream of the inflow points</p></li>
<li><p>Run the best calibration scenario(s) of the upstream catchments again to produce long timeserie(s) of the outlet(s) point</p></li>
<li><p>Create an inflow file from the long timeseries of outlet(s)</p></li>
<li><p>Create a downstream calibration settings (directories, templates etc.)</p></li>
</ol>
<p><strong>Test the catchment!</strong></p>
<ol class="arabic simple" start="6">
<li><p>Change the settings file of the downstream calibration so that it includes the inflow from upstream</p></li>
</ol>
<p><strong>Test it!</strong>
7. Create initial file for warm start</p>
<section id="cutting-the-mask-map">
<h3>Cutting the mask map<a class="headerlink" href="#cutting-the-mask-map" title="Permalink to this heading"></a></h3>
<p>Assuming you have a mask map of the whole catchment (e.g. Yangtze.map and the station points (here Zhutuo 105.75 28.75 and Yichang 111.25 30.75
1. Creating catchment for Zhutuo: catchment 105.75 28.75 ldd_yangtze.map zhu1.map
2. Creating catchment for Yichang: catchment 111.25 30.75 ldd_yangtze.map yi1.map
3. Creating Yichang without Zhutuo:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">pcrcalc</span> <span class="n">a2</span><span class="o">.</span><span class="n">map</span> <span class="o">=</span> <span class="n">cover</span><span class="p">(</span><span class="n">scalar</span><span class="p">(</span><span class="n">zhu1</span><span class="o">.</span><span class="n">map</span><span class="p">)</span><span class="o">*</span><span class="mi">2</span><span class="p">,</span><span class="n">scalar</span><span class="p">(</span><span class="n">yi1</span><span class="o">.</span><span class="n">map</span><span class="p">))</span>
<span class="n">pcrcalc</span> <span class="n">yichang</span><span class="o">.</span><span class="n">map</span> <span class="o">=</span> <span class="n">boolean</span><span class="p">(</span><span class="k">if</span><span class="p">(</span><span class="n">a2</span><span class="o">.</span><span class="n">map</span> <span class="n">eq</span> <span class="mi">1</span><span class="p">,</span><span class="n">a2</span><span class="o">.</span><span class="n">map</span><span class="p">))</span>
</pre></div>
</div>
<p>Result is a maskmap: Yichang.map</p>
<a class="reference internal image-reference" href="_images/cal_tutor1.jpg"><img alt="_images/cal_tutor1.jpg" src="_images/cal_tutor1.jpg" style="width: 600px;" /></a>
<p>Figure 1: Upstream catchment (blue) and downstream catchment (red)</p>
</section>
<section id="detecting-the-downstream-point">
<h3>Detecting the downstream point<a class="headerlink" href="#detecting-the-downstream-point" title="Permalink to this heading"></a></h3>
<p>The inflow point of the new catchment has to be in the new mask and preferable one grid cell in flow direction below the upstream station e.g. 1 gridcell North East of Zhutuo (see purple circle in fig. 2)</p>
<p>The inflow point has the lon/lat 106.25 29.25</p>
<a class="reference internal image-reference" href="_images/cal_tutor2.jpg"><img alt="_images/cal_tutor2.jpg" src="_images/cal_tutor2.jpg" style="width: 600px;" /></a>
<p>Figure 2: Downstream point</p>
</section>
<section id="run-the-best-calibration-scenario-upstream">
<h3>Run the best calibration scenario upstream<a class="headerlink" href="#run-the-best-calibration-scenario-upstream" title="Permalink to this heading"></a></h3>
<p>In order to get a long inflow timeserie for the inflow point (here: Zhutuo) you need to run the best scenario of the upstream catchment (here: 31_best)</p>
<ul>
<li><p>Change into the folder ../catchments/best</p></li>
<li><p>Change settings file from:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">StepStart</span> <span class="o">=</span> <span class="mi">1</span><span class="o">/</span><span class="mi">1</span><span class="o">/</span><span class="mi">1996</span>
<span class="n">SpinUp</span> <span class="o">=</span> <span class="mi">1</span><span class="o">/</span><span class="mi">1</span><span class="o">/</span><span class="mi">2002</span>
<span class="n">StepEnd</span> <span class="o">=</span> <span class="mi">31</span><span class="o">/</span><span class="mi">12</span><span class="o">/</span><span class="mi">2013</span>
</pre></div>
</div>
</li>
<li><p>To:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">StepStart</span> <span class="o">=</span> <span class="mi">1</span><span class="o">/</span><span class="mi">1</span><span class="o">/</span><span class="mi">1990</span>
<span class="n">SpinUp</span> <span class="o">=</span> <span class="mi">1</span><span class="o">/</span><span class="mi">1</span><span class="o">/</span><span class="mi">1996</span>
<span class="n">StepEnd</span> <span class="o">=</span> <span class="mi">31</span><span class="o">/</span><span class="mi">12</span><span class="o">/</span><span class="mi">2013</span>
</pre></div>
</div>
</li>
</ul>
<p>Results is a time series from 1/1/1990 – 31/12/2013 in: discharge_daily.tss</p>
</section>
<section id="create-an-inflow-file-from-the-long-timeseries-of-outlet-s">
<h3>Create an inflow file from the long timeseries of outlet(s)<a class="headerlink" href="#create-an-inflow-file-from-the-long-timeseries-of-outlet-s" title="Permalink to this heading"></a></h3>
<ul class="simple">
<li><p>Create a folder ../inflow</p></li>
<li><p>Copy the ../catchments/31_best/discharge_daily.tss to ../inflow/zhutuo.tss</p></li>
</ul>
</section>
<section id="create-a-downstream-calibration-settings-directories-templates-etc">
<h3>Create a downstream calibration settings (directories, templates etc.)<a class="headerlink" href="#create-a-downstream-calibration-settings-directories-templates-etc" title="Permalink to this heading"></a></h3>
<p>Create downstream calibration settings as before</p>
<ul class="simple">
<li><p>Copy everything from upstream catchment (e.g. zhutuo) but not catchments</p></li>
<li><p>Create empty catchments folder</p></li>
<li><p>Create a observed discharge file in observed</p></li>
<li><p>Change settings.txt accordingly</p></li>
<li><p>Change settings.ini accordingly</p></li>
</ul>
<p><strong>Test the catchment setting!</strong></p>
<p><strong>But do not create an initial run yet!</strong></p>
</section>
<section id="change-the-settings-file">
<h3>Change the settings file<a class="headerlink" href="#change-the-settings-file" title="Permalink to this heading"></a></h3>
<p>Change the settings file of the downstream calibration so that it includes the inflow from upstream
Change the part of the settings.ini:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>[Option]
inflow = True
[INFLOW]
#-------------------------------------------------------
# if option inflow = true
# the inflow from outside is added as inflowpoints
In_Dir = $(FILE_PATHS:PathRoot)/calibration/calibration_yichang/inflow
# nominal map with locations of (measured)inflow hydrographs [cu m / s]
InflowPoints = 106.25 29.25
InLocal = True
.
# if InflowPoints is a map, this flag is to identify if it is global (False) or local (True)
# observed or simulated input hydrographs as time series [cu m / s]
# Note: that identifiers in time series have to correspond to InflowPoints
# can be several timeseries in one file or different files e.g. main.tss mosel.tss
QInTS = zhutuo.tss
</pre></div>
</div>
<p><strong>Test it!</strong></p>
<p>Generate initial file for warm start
Use initial file for calibration</p>
</section>
</section>
<section id="joining-best-sub-basin-results-to-calibration-maps">
<h2>Joining best sub-basin results to calibration maps<a class="headerlink" href="#joining-best-sub-basin-results-to-calibration-maps" title="Permalink to this heading"></a></h2>
<ol class="arabic simple">
<li><p>You need all runs done for all sub-basins</p></li>
<li><p>A region map</p></li>
</ol>
<p>For each subbasin a unique number e.g. Zambezi basin</p>
<a class="reference internal image-reference" href="_images/cal_tutor3.jpg"><img alt="_images/cal_tutor3.jpg" src="_images/cal_tutor3.jpg" style="width: 600px;" /></a>
<p>Figure 3 Sub-basin map with a unique identifier for each subbasin</p>
<ol class="arabic" start="3">
<li><p>You need a working PCRaster installation</p></li>
<li><p>The settings file settings.txt has to be changed:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">DEFAULT</span><span class="p">]</span>
<span class="n">Root</span> <span class="o">=</span> <span class="n">P</span><span class="p">:</span><span class="o">/</span><span class="n">watmodel</span><span class="o">/</span><span class="n">CWatM</span><span class="o">/</span><span class="n">calibration</span><span class="o">/</span><span class="n">calibration_zambezi</span>
<span class="c1"># root directory where all subbasin are in</span>
<span class="o">.</span>
<span class="p">[</span><span class="n">Catchments</span><span class="p">]</span>
<span class="n">catch</span> <span class="o">=</span> <span class="n">lukulu</span><span class="p">,</span> <span class="n">katima</span><span class="p">,</span> <span class="n">kafue</span><span class="p">,</span> <span class="n">luangwa</span><span class="p">,</span> <span class="n">kwando</span><span class="p">,</span> <span class="n">tete</span>
<span class="c1"># name of the subbasin, has to be the same as the folder name in root</span>
<span class="c1"># the order has to be the same as in the region map</span>
<span class="o">.</span>
<span class="p">[</span><span class="n">region</span><span class="p">]</span>
<span class="n">regionmap</span> <span class="o">=</span> <span class="n">P</span><span class="p">:</span><span class="o">/</span><span class="n">watmodel</span><span class="o">/</span><span class="n">CWatM</span><span class="o">/</span><span class="n">calibration_tutorial</span><span class="o">/</span><span class="n">calibration</span><span class="o">/</span><span class="n">CreateCalibrationMaps</span><span class="o">/</span><span class="n">zambezi_regions</span><span class="o">.</span><span class="n">map</span>
<span class="c1"># region map, the order has to be the same a [Catchment]</span>
<span class="o">.</span>
<span class="p">[</span><span class="n">Path</span><span class="p">]</span>
<span class="n">Templates</span> <span class="o">=</span> <span class="o">%</span><span class="p">(</span><span class="n">Root</span><span class="p">)</span><span class="n">s</span><span class="o">/</span><span class="n">templates</span>
<span class="n">SubCatchmentPath</span> <span class="o">=</span> <span class="o">%</span><span class="p">(</span><span class="n">Root</span><span class="p">)</span><span class="n">s</span><span class="o">/</span><span class="n">catchments</span>
<span class="n">ParamRanges</span> <span class="o">=</span> <span class="o">%</span><span class="p">(</span><span class="n">Root</span><span class="p">)</span><span class="n">s</span><span class="o">/</span><span class="n">Join</span><span class="o">/</span><span class="n">ParamRanges</span><span class="o">.</span><span class="n">csv</span>
<span class="o">.</span>
<span class="n">Result</span> <span class="o">=</span> <span class="n">P</span><span class="p">:</span><span class="o">/</span><span class="n">watmodel</span><span class="o">/</span><span class="n">CWatM</span><span class="o">/</span><span class="n">calibration_tutorial</span><span class="o">/</span><span class="n">calibration</span><span class="o">/</span><span class="n">CreateCalibrationMaps</span><span class="o">/</span><span class="n">results</span>
<span class="c1"># here are the results</span>
<span class="o">.</span>
<span class="n">PCRHOME</span> <span class="o">=</span> <span class="n">C</span><span class="p">:</span>\<span class="n">PCRaster</span>\<span class="nb">bin</span>
<span class="c1"># Where is your PCraster installation?</span>
</pre></div>
</div>
</li>
<li><p>Run python CAL_5_PARAMETER_MAPS.py</p></li>
</ol>
</section>
<section id="calibration-tool-structure">
<h2>Calibration tool structure<a class="headerlink" href="#calibration-tool-structure" title="Permalink to this heading"></a></h2>
</section>
<section id="references">
<h2>References<a class="headerlink" href="#references" title="Permalink to this heading"></a></h2>
<ul class="simple">
<li><p>Beck, H. E., A. I. J. M. van Dijk, A. de Roo, D. G. Miralles, T. R. McVicar, J. Schellekens and L. A. Bruijnzeel (2016). “Global-scale regionalization of hydrologic model parameters.” Water Resources Research 52(5): 3599-3622.</p></li>
<li><p>Deb, K., A. Pratap, S. Agarwal and T. Meyarivan (2002). “A fast and elitist multiobjective genetic algorithm: NSGA-II.” IEEE Transactions on Evolutionary Computation 6(2): 182-197.</p></li>
<li><p>Fortin, F. A., F. M. De Rainville, M. A. Gardner, M. Parizeau and C. Gagńe (2012). “DEAP: Evolutionary algorithms made easy.” Journal of Machine Learning Research 13: 2171-2175.</p></li>
<li><p>Greve, P., L. Gudmundsson, B. Orlowsky and S. I. Seneviratne (2016). “A two-parameter Budyko function to represent conditions under which evapotranspiration exceeds precipitation.” Hydrology and Earth System Sciences 20(6): 2195-2205.</p></li>
<li><p>Gupta, H. V., H. Kling, K. K. Yilmaz and G. F. Martinez (2009). “Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling.” Journal of Hydrology 377(1-2): 80-91.</p></li>
<li><p>Kling, H., M. Fuchs and M. Paulin (2012). “Runoff conditions in the upper Danube basin under an ensemble of climate change scenarios.” Journal of Hydrology 424-425: 264-277.</p></li>
<li><p>Samaniego, L., R. Kumar, S. Thober, O. Rakovec, M. Zink, N. Wanders, S. Eisner, H. Müller Schmied, E. Sutanudjaja, K. Warrach-Sagi and S. Attinger (2017). “Toward seamless hydrologic predictions across spatial scales.” Hydrology and Earth System Sciences 21(9): 4323-4346.</p></li>
</ul>
</section>
</section>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="calibration.html" class="btn btn-neutral float-left" title="11. Calibration tool" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
<a href="resolution.html" class="btn btn-neutral float-right" title="13. Increasing resolution" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>© Copyright 2021, IIASA Water Security.</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>