Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop]: PI13 Documentation Update #127

Merged
merged 53 commits into from
Aug 8, 2024
Merged
Changes from 1 commit
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
acce91b
update intro w/code additions since v1.2.0
gspetro-NOAA Jun 14, 2024
332fe05
Merge branch 'develop' of https://github.com/ufs-community/land-DA_wo…
gspetro-NOAA Jun 14, 2024
56b4483
update tech overview dir structure
gspetro-NOAA Jul 2, 2024
c12eb3b
add info on manual vs auto submission
gspetro-NOAA Jul 2, 2024
f2639fd
several misc changes
gspetro-NOAA Jul 5, 2024
4db3bc4
add plotting info
gspetro-NOAA Jul 8, 2024
bf343fa
update plotting info
gspetro-NOAA Jul 8, 2024
9ea7ce7
add plot images, add extlinks, misc other
gspetro-NOAA Jul 9, 2024
e45555a
add Rocoto chapter, misc edits
gspetro-NOAA Jul 10, 2024
02f3246
rm workflow section in DA chapter
gspetro-NOAA Jul 11, 2024
5c8a82d
add misc
gspetro-NOAA Jul 11, 2024
323e795
minor conf.py updates
gspetro-NOAA Jul 11, 2024
a48cfea
add ConfigWorkflow chapter
gspetro-NOAA Jul 11, 2024
071cad1
explain task parameters & configuration
gspetro-NOAA Jul 12, 2024
ddccaf9
update testing ch
gspetro-NOAA Jul 16, 2024
76f3968
update glossary
gspetro-NOAA Jul 16, 2024
786d6fd
add configworkflow chapter
gspetro-NOAA Jul 16, 2024
69ee11b
update file paths & misc in Model ch
gspetro-NOAA Jul 16, 2024
12ccb8b
misc minor update in Model ch
gspetro-NOAA Jul 16, 2024
778ecd5
Update DA ch
gspetro-NOAA Jul 17, 2024
21ae278
Update letkf_land.yaml info
gspetro-NOAA Jul 17, 2024
03692d2
fix broken intersphinx link
gspetro-NOAA Jul 17, 2024
3df81b1
update jedi doc version
gspetro-NOAA Jul 17, 2024
ed9465a
DA ch updates
gspetro-NOAA Jul 17, 2024
065c15e
misc minor edits
gspetro-NOAA Jul 17, 2024
6dcb318
Merge branch 'ufs-community:develop' into text/us-215
gspetro-NOAA Jul 18, 2024
e783edf
update wflow dir structure diagram
gspetro-NOAA Jul 19, 2024
1363c92
Merge branch 'text/us-215' of github.com:gspetro-NOAA/land-DA_workflo…
gspetro-NOAA Jul 19, 2024
fcdbc36
update wflow dir structure diagram
gspetro-NOAA Jul 19, 2024
9e2e891
add new chapters to introduction
gspetro-NOAA Jul 24, 2024
0fa3832
misc minor Intro edits
gspetro-NOAA Jul 24, 2024
422fbe7
misc tech overview edits
gspetro-NOAA Jul 25, 2024
38e4273
misc tech overview updates
gspetro-NOAA Jul 25, 2024
a959756
rm uw from hierarchical repo structure?
gspetro-NOAA Jul 25, 2024
818bbb6
update table 1.1
gspetro-NOAA Jul 25, 2024
8f7b1e5
update L1 Build/Run ch
gspetro-NOAA Jul 25, 2024
bc19c48
L1 Build/Run ch minor/misc updates
gspetro-NOAA Jul 25, 2024
fa4342b
update container chapter
gspetro-NOAA Jul 25, 2024
0544826
misc minor edits
gspetro-NOAA Jul 25, 2024
f885141
ConfigWorkflow updates
gspetro-NOAA Jul 29, 2024
b3f19c0
minor updates to reference chs
gspetro-NOAA Jul 29, 2024
0ebe51b
minor misc fixes
gspetro-NOAA Jul 29, 2024
fa0b8b0
minor misc updates
gspetro-NOAA Jul 30, 2024
f56d91c
update docs w/changes from PR #129
gspetro-NOAA Aug 5, 2024
a9b0bac
update docs w/changes from PR #129
gspetro-NOAA Aug 5, 2024
5671e7b
update docs w/changes from PR #129
gspetro-NOAA Aug 5, 2024
ac1804e
rm mention of 2019 case
gspetro-NOAA Aug 5, 2024
6c5d384
remove ERA5 model input file section
gspetro-NOAA Aug 8, 2024
f689823
rm vector2tile info
gspetro-NOAA Aug 8, 2024
818be75
add info on grid description files to DA chapter/IODA section
gspetro-NOAA Aug 8, 2024
e98cb6e
rm run without rocoto section
gspetro-NOAA Aug 8, 2024
a492e38
update wording re: landdaroot
gspetro-NOAA Aug 8, 2024
418761d
Merge branch 'ufs-community:develop' into text/us-215
gspetro-NOAA Aug 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update L1 Build/Run ch
gspetro-NOAA committed Jul 25, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 8f7b1e515c41f765a8abb878381f92fbdd821f61
4 changes: 2 additions & 2 deletions doc/source/BackgroundInfo/TechnicalOverview.rst
Original file line number Diff line number Diff line change
@@ -65,8 +65,8 @@ Preconfigured (Level 1) systems for Land DA already have the required external l
* - Platform
- Compiler
- MPI
- ``spack-stack`` Installation
- ``jedi-bundle`` Installation
- *spack-stack* Installation
- *jedi-bundle* Installation
* - Hera
- intel/2021.5.0 /
- impi/2021.5.1
43 changes: 26 additions & 17 deletions doc/source/BuildingRunningTesting/BuildRunLandDA.rst
Original file line number Diff line number Diff line change
@@ -26,14 +26,14 @@ Create a directory for the Land DA experiment (``$LANDDAROOT``):
cd /path/to/landda
export LANDDAROOT=`pwd`
chan-hoo marked this conversation as resolved.
Show resolved Hide resolved

where ``/path/to/landda`` is the path to the directory where the user plans to run Land DA experiments.
where ``/path/to/landda`` is the path to the directory where the user plans to run Land DA experiments. In the experiment configuration file, ``$LANDDAROOT`` is referred to as ``$EXP_BASEDIR``, and refers to the Land DA workflow's parent directory.

.. _GetCode:

Get Code
***********

Clone the Land DA repository. To clone the ``develop`` branch, run:
Clone the Land DA workflow repository. To clone the ``develop`` branch, run:

.. code-block:: console

@@ -97,7 +97,7 @@ To load the workflow environment, run:
module load wflow_<platform>
conda activate land_da

where ``<platform>`` is ``hera`` or ``orion``.
where ``<platform>`` is ``hera`` or ``orion``. This activates the land_da conda environment, and the user typically sees (land_da) in front of the Terminal prompt at this point.

.. _configure-expt:

@@ -120,7 +120,7 @@ where ``<platform>`` is ``hera`` or ``orion``.

Users will need to configure certain elements of their experiment in ``land_analysis.yaml``:

* ``ACCOUNT:`` A valid account name. Hera, Orion, and most NOAA RDHPCS systems require a valid account name; other systems may not
* ``ACCOUNT:`` A valid account name. Hera, Orion, and most NOAA RDHPCS systems require a valid account name; other systems may not (in which case, any value will do).
* ``EXP_BASEDIR:`` The full path to the directory where land-DA_workflow was cloned (i.e., ``$LANDDAROOT``)
* ``FORCING:`` Forcing options; ``gswp3`` or ``era5``
chan-hoo marked this conversation as resolved.
Show resolved Hide resolved
* ``cycledef/spec:`` Cycle specification
@@ -129,14 +129,14 @@ Users will need to configure certain elements of their experiment in ``land_anal

To determine an appropriate ``ACCOUNT`` field for Level 1 systems that use the Slurm job scheduler, run ``saccount_params``. On other systems, running ``groups`` will return a list of projects that the user has permissions for. Not all listed projects/groups have an HPC allocation, but those that do are potentially valid account names.

Users may configure other elements of an experiment in ``land_analysis.yaml`` if desired. The ``land_analysis_*`` files contain reasonable default values for running a Land DA experiment. Users who wish to run a more complex experiment may change the values in these files and the files they reference using information in Sections :numref:`%s <Model>` & :numref:`%s <DASystem>`.
Users may configure other elements of an experiment in ``land_analysis.yaml`` if desired. The ``land_analysis_*.yaml`` files contain reasonable default values for running a Land DA experiment. Users who wish to run a more complex experiment may change the values in these files and the files they reference using information in Sections :numref:`%s <Model>` & :numref:`%s <DASystem>`.

.. _GetData:

Data
------

:numref:`Table %s <Level1Data>` shows the locations of pre-staged data on NOAA :term:`RDHPCS` (i.e., Hera and Orion). These data locations are already included in the ``land_analysis_*`` files but are provided here for informational purposes.
:numref:`Table %s <Level1Data>` shows the locations of pre-staged data on NOAA :term:`RDHPCS` (i.e., Hera and Orion). These data locations are already included in the ``land_analysis_*.yaml`` files but are provided here for informational purposes.

.. _Level1Data:

@@ -150,14 +150,14 @@ Data
| Orion | /work/noaa/epic/UFS_Land-DA_Dev/inputs |
+-----------+--------------------------------------------------+

Users who have difficulty accessing the data on Hera or Orion may download it according to the instructions in :numref:`Section %s <GetDataC>`. Its subdirectories are soft-linked to the ``fix`` directory of the land-DA workflow by the build script ``sorc/app_build.sh``.
Users who have difficulty accessing the data on Hera or Orion may download it according to the instructions in :numref:`Section %s <GetDataC>`. Its subdirectories are soft-linked to the ``fix`` directory of ``land-DA_workflow`` by the build script ``sorc/app_build.sh``.

.. _generate-wflow:

Generate the Rocoto XML File
==============================

Generate the workflow with ``uwtools`` by running:
Generate the workflow XML file with ``uwtools`` by running:

.. code-block:: console

@@ -170,6 +170,8 @@ If the command runs without problems, ``uwtools`` will output a "0 errors found"
[2024-03-01T20:36:03] INFO 0 UW schema-validation errors found
[2024-03-01T20:36:03] INFO 0 Rocoto validation errors found

The generated workflow XML file (``land_analysis.xml``) will be used by the Rocoto workflow manager to determine which tasks (or "jobs") to submit to the batch system and when to submit them (e.g., as soon as task dependencies are satisfied).

Run the Experiment
********************

@@ -221,11 +223,14 @@ To automate task submission, users must be on a system where :term:`cron` is ava
.. code-block:: console

cd parm
conda deactivate # optional
./launch_rocoto_wflow.sh add

To check the status of the experiment, see :numref:`Section %s <VerifySuccess>` on tracking experiment progress.

.. note::

If users run into issues with the launch script, they can run ``conda deactivate`` before running the launch script.

Manual Submission
-------------------

@@ -235,7 +240,7 @@ To run the experiment, issue a ``rocotorun`` command from the ``parm`` directory

rocotorun -w land_analysis.xml -d land_analysis.db

Users will need to issue the ``rocotorun`` command multiple times. The tasks must be run in order, and ``rocotorun`` initiates the next task once its dependencies have completed successfully. Note that the status table printed by ``rocotostat`` only updates after each ``rocotorun`` command. Details on checking experiment status are provided in the :ref:`next section <VerifySuccess>`.
Users will need to issue the ``rocotorun`` command multiple times. The tasks must be run in order, and ``rocotorun`` initiates the next task once its dependencies have completed successfully. Details on checking experiment status are provided in the :ref:`next section <VerifySuccess>`.

.. _VerifySuccess:

@@ -260,7 +265,7 @@ If ``rocotorun`` was successful, the ``rocotostat`` command will print a status
200001030000 post_anal - - - - -
200001030000 plot_stats - - - - -
200001030000 forecast - - - - -
================================================================================================================================
=========================================================================================================
200001040000 prep_obs druby://10.184.3.62:41973 SUBMITTING - 1 0.0
200001040000 pre_anal - - - - -
200001040000 analysis - - - - -
@@ -277,7 +282,7 @@ The experiment has successfully completed when all tasks say SUCCEEDED under STA
Run Without Rocoto
gspetro-NOAA marked this conversation as resolved.
Show resolved Hide resolved
--------------------

Users may choose not to run the workflow with uwtools and Rocoto for a non-cycled run. To run the :term:`J-jobs` scripts in the ``jobs`` directory, navigate to the ``parm`` directory and edit ``run_without_rocoto.sh`` (e.g., using vim or preferred command line editor). Users will likely need to change the ``MACHINE``, ``ACCOUNT``, and ``EXP_BASEDIR`` variables to match their system. Then, run ``run_without_rocoto.sh``:
Users may choose *not* to run the workflow with *uwtools* and Rocoto for a non-cycled run. To run the :term:`J-job <J-jobs>` scripts in the ``jobs`` directory, navigate to the ``parm`` directory and edit ``run_without_rocoto.sh`` (e.g., using vim or preferred command line editor). Users will likely need to change the ``MACHINE``, ``ACCOUNT``, and ``EXP_BASEDIR`` variables to match their system. Then, run the script:

.. code-block:: console

@@ -287,7 +292,7 @@ Users may choose not to run the workflow with uwtools and Rocoto for a non-cycle
Check Experiment Output
=========================

As the experiment progresses, it will generate a number of directories to hold intermediate and output files. The directory structure for those files and directories appears below:
As the experiment progresses, it will generate a number of directories to hold intermediate and output files. The structure of those files and directories appears below:

.. _land-da-dir-structure:

@@ -296,7 +301,7 @@ As the experiment progresses, it will generate a number of directories to hold i
$LANDDAROOT: Base directory
├── land-DA_workflow(<CYCLEDIR>): Home directory of the land DA workflow
└── ptmp (<PTMP>)
└── <envir> (<OPSROOT>)
└── test (<envir> or <OPSROOT>)
└── com (<COMROOT>)
│ ├── landda (<NET>)
│ │ └── vX.Y.Z (<model_ver>)
@@ -319,14 +324,18 @@ Check for the output files for each cycle in the experiment directory:

.. code-block:: console

ls -l $LANDDAROOT/ptmp/test/com/landda/v1.2.1/landda.YYYYMMDD
ls -l $LANDDAROOT/ptmp/test/com/landda/<model_ver>/landda.YYYYMMDD

where ``YYYYMMDD`` is the cycle date. The experiment should generate several restart files.
where ``YYYYMMDD`` is the cycle date, and ``<model_ver>`` is the model version (currently ``v1.2.1`` in the ``develop`` branch). The experiment should generate several restart files.

Plotting Results
-----------------

Additionally, in the ``plot`` subdirectory, users will find images depicting the results of the ``analysis`` task for each cycle as a scatter plot (``hofx_oma_YYYMMDD_scatter.png``) and as a histogram (``hofx_oma_YYYYMMDD_histogram.png``). The scatter plot is named OBS-ANA (i.e., Observation Minus Analysis [OMA]), and it depicts a map of snow depth results. Blue points indicate locations where the observed values are less than the analysis values, and red points indicate locations where the observed values are greater than the analysis values. The title lists the mean and standard deviation of the absolute value of the OMA values. The histogram plots OMA values on the x-axis and frequency density values on the y-axis. The title of the histogram lists the mean and standard deviation of the real value of the OMA values.
Additionally, in the ``plot`` subdirectory, users will find images depicting the results of the ``analysis`` task for each cycle as a scatter plot (``hofx_oma_YYYMMDD_scatter.png``) and as a histogram (``hofx_oma_YYYYMMDD_histogram.png``).

The scatter plot is named OBS-ANA (i.e., Observation Minus Analysis [OMA]), and it depicts a map of snow depth results. Blue points indicate locations where the observed values are less than the analysis values, and red points indicate locations where the observed values are greater than the analysis values. The title lists the mean and standard deviation of the absolute value of the OMA values.

The histogram plots OMA values on the x-axis and frequency density values on the y-axis. The title of the histogram lists the mean and standard deviation of the real value of the OMA values.

.. |logo1| image:: https://raw.githubusercontent.com/wiki/ufs-community/land-DA_workflow/images/LandDAScatterPlot.png
:alt: Map of snow depth in millimeters (observation minus analysis)