From 257e9460110bb459259dd799820f0889f08106a7 Mon Sep 17 00:00:00 2001 From: Vanessa Sochat Date: Tue, 2 Jul 2019 11:23:01 -0400 Subject: [PATCH] adding start to batch examples page Signed-off-by: Vanessa Sochat --- src/docs/getting-started/submitting.md | 8 +- .../user-guide/examples/batch-job-examples.md | 78 +++++++++++++++++++ src/docs/user-guide/running-jobs.md | 2 +- 3 files changed, 83 insertions(+), 5 deletions(-) create mode 100644 src/docs/user-guide/examples/batch-job-examples.md diff --git a/src/docs/getting-started/submitting.md b/src/docs/getting-started/submitting.md index cdf885f4a..63fb4d2de 100644 --- a/src/docs/getting-started/submitting.md +++ b/src/docs/getting-started/submitting.md @@ -151,10 +151,10 @@ Congratulations, you've submitted your first batch job on Sherlock! Actually, quite a lot. Although you now know how to submit a simple batch job, there are many other options and areas to explore in the next sections: -* [Data transfer][url_transfer] -* [Storage][url_storage] -* [Running jobs][url_runningjobs] - + * [Data transfer][url_transfer] + * [Storage][url_storage] + * [Running jobs][url_runningjobs] + * [Batch Job Examples](/docs/user-guide/examples/batch-job-examples/) [comment]: # (link URLs -----------------------------------------------------) diff --git a/src/docs/user-guide/examples/batch-job-examples.md b/src/docs/user-guide/examples/batch-job-examples.md new file mode 100644 index 000000000..e4d4e1fb9 --- /dev/null +++ b/src/docs/user-guide/examples/batch-job-examples.md @@ -0,0 +1,78 @@ +The following are several examples for batch jobs to run on the Sherlock cluster. +If you would like to contribute an example, please [share it here](https://www.github.com/stanford-rc/www.sherlock.stanford.edu/issues). + +> _This post is a living post, meaning that it will be updated with more examples_ + +## Basic +The job below asks for 16GB and can run up to 48 hours. +```bash +#!/bin/bash +#SBATCH --job-name=allthesmallthings +#SBATCH --time=48:00:00 +#SBATCH --ntasks=1 +#SBATCH --cpus-per-task=20 +#SBATCH --mem=16GB +``` + +## Using GPUs +To modify your sbatch script for a GPU, you need to add both `--gres` and `--partition` + +```bash +#!/bin/bash +#SBATCH --job-name=cron +##SBATCH --begin=now+7days +##SBATCH --dependency=singleton +#SBATCH --time=00:02:00 +#SBATCH --mail-type=FAIL +#SBATCH -p gpu +#SBATCH -c 1 +#SBATCH --gres gpu:1 + +# Load modules here +ml load gromacs/2016.3 + +# Run your script +/bin/bash /scratch/users/pancakes/runMe.sh +``` +For more information, see the [gpu](https://www.sherlock.stanford.edu/docs/user-guide/gpu/) and [running jobs](https://www.sherlock.stanford.edu/docs/user-guide/running-jobs/) documentation. + +## Long Running Jobs +Let's say you have a long running job with low memory, and the memory grows over time. Here is an example that would make sure that the job goes for 3 days. The QOS (quality of service) directive tells the job manager that the job will be running longer than a day. +``` +#!/bin/bash +#SBATCH --job-name=normaljob +#SBATCH -p normal +#SBATCH --qos=long +#SBATCH --time=3-00:00:00 +#SBATCH --nodes=1 +#SBATCH --ntasks=1 +#SBATCH --cpus-per-task=32 +#SBATCH --mem=300000 +ml load system devel +srun /bin/bash /scratch/users/minniemouse/myJob.sh +``` + +## Without a Job File +You can submit a job directly (on the command line) without a job file. The same SBATCH directives at the top become the parameter. + +```bash +sbatch --job-name=$job_name -o $job_name.%j.out -e $job_name.%j.err /scratch/users/smiley/scripts/makeSmiles.sh ${file1} ${file2} +``` +## Google Drive Sync + +```bash +#!/bin/bash +#SBATCH --job-name=gdrive +#SBATCH --output=/home/users/julienc/logs/gdrive.out +#SBATCH -p agitler +#SBATCH --time=7-0 ## To be used with --qos=long +#SBATCH --cpus-per-task=1 +#SBATCH --begin=now+48hour +#SBATCH --dependency=singleton + +module load system gdrive +date + +# Example of delete, where the string at the end is the file id on Google Drive +gdrive sync upload --keep-local --delete-extraneous $OAK/Shared/Potatoes/ sdhfshds3u39ur93rioneksfser +``` diff --git a/src/docs/user-guide/running-jobs.md b/src/docs/user-guide/running-jobs.md index fc739b223..7481c93a8 100644 --- a/src/docs/user-guide/running-jobs.md +++ b/src/docs/user-guide/running-jobs.md @@ -152,7 +152,7 @@ account. [comment]: #_ - +For quick examples, see the [batch job examples page](/docs/user-guide/examples/batch-job-examples/). ## Recurring jobs